Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-580

Allow "fail fast" or stop running on first failure

    Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      0

      Description

      We have some slow Selenium builds. Waiting for indication of failure can take a long time. It would be helpful if there was an option to finish the test run on the first failure.

      We could configure this on some of our slower builds so improve the speed of the feedback loop in the cases where there are failures.

        Activity

        Hide
        Deejay added a comment - - edited

        I also require this feature. When using a combination of Spring and Liquibase, if the Spring config contains an error then on each and every test the Spring context will try once more to initialize itself, taking 20+ seconds each time.

        This adds up to a huge amount of time wasted by our Jenkins instance to alert us to the fact that someone made a typo.

        Show
        Deejay added a comment - - edited I also require this feature. When using a combination of Spring and Liquibase, if the Spring config contains an error then on each and every test the Spring context will try once more to initialize itself, taking 20+ seconds each time. This adds up to a huge amount of time wasted by our Jenkins instance to alert us to the fact that someone made a typo.
        Hide
        Tomislav Nakic-Alfirevic added a comment -

        Same problem here: we use surefire + junit to run integration tests (maybe there is a better way, I don't know...) and our test suite takes hours to complete right now, yet most of the time we expect 100% correct execution and would like to know - as soon as possible - which test failed and why.

        I half expect I'm using the wrong tool or using the tool wrong and one or more simple solutions are available. That said, I'm a bit puzzled that this issue remains open for three years: it seems to me enough to add a switch (e.g. -DfastFail=true, default=false) and wrap up test execution as soon as the first test fails, producing the usual test run output. Test methods which haven't been executed might have to be removed from internal data structures so that they do not appear in the reports.

        Show
        Tomislav Nakic-Alfirevic added a comment - Same problem here: we use surefire + junit to run integration tests (maybe there is a better way, I don't know...) and our test suite takes hours to complete right now, yet most of the time we expect 100% correct execution and would like to know - as soon as possible - which test failed and why. I half expect I'm using the wrong tool or using the tool wrong and one or more simple solutions are available. That said, I'm a bit puzzled that this issue remains open for three years: it seems to me enough to add a switch (e.g. -DfastFail=true, default=false) and wrap up test execution as soon as the first test fails, producing the usual test run output. Test methods which haven't been executed might have to be removed from internal data structures so that they do not appear in the reports.
        Hide
        Kristian Rosenvold added a comment -

        Could probably be hooked up to the maven "-ff" option if someone is willing to make the patch

        Show
        Kristian Rosenvold added a comment - Could probably be hooked up to the maven "-ff" option if someone is willing to make the patch
        Hide
        Tomislav Nakic-Alfirevic added a comment -

        It would even be useful if test execution was not interrupted, but the result of each test be output after the test, not after the entire test suite. Something that would work like this:

        $ mvn test | grep FAIL
        SuiteA.test2 FAILED
        SuiteA.test11 FAILED
        SuiteB.test1 FAILED
        [surefire still running]

        I imagine that should be fairly easy to implement, yet it would still provide quick information about any test failures...

        Show
        Tomislav Nakic-Alfirevic added a comment - It would even be useful if test execution was not interrupted, but the result of each test be output after the test, not after the entire test suite. Something that would work like this: $ mvn test | grep FAIL SuiteA.test2 FAILED SuiteA.test11 FAILED SuiteB.test1 FAILED [surefire still running] I imagine that should be fairly easy to implement, yet it would still provide quick information about any test failures...
        Hide
        Gili added a comment -

        Please add TestNG to the list of components. This should be fixed for both frameworks.

        Show
        Gili added a comment - Please add TestNG to the list of components. This should be fixed for both frameworks.
        Andreas Gudian made changes -
        Field Original Value New Value
        Component/s TestNG support [ 12346 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Paul Curren
          • Votes:
            32 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

            • Created:
              Updated: