Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-122

With forkmode once, XML reports are cumulative

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0 (2.2 plugin)
    • Fix Version/s: 2.3
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      1

      Description

      With forkmode set to once, the resulting XML files seem to include the test results of all the tests in the previous suites as well as it's own.

      pseudo example:
      ATest is run, it has 2 test methods. The ATest.xml report says 2 tests run and passed
      BTest is run, it has 2 test methods. The BTest.xml report says 4 tests run and passed.
      CTest is run, it has 3 test methods. The CTest.xml report says 7 tests run and passed.

      When we use cruisecontrol or other reporting tools, it then say 13 tests run and passed instead of 5. What's worse, if a test in ATest fails, it's listed as a failure in all the tests so you get 3 tests failed, not 1. Plus, it's harder to figure out which test suite reallly had the failure.

        Issue Links

          Activity

          Hide
          Gaz Moorst added a comment -

          I get this problem regardless of the forkmode setting.
          It only seems to have been doing this since I started using Maven 2.0.4, with maven-surefire-plugin 2.2 instead of maven 2.0.2 with pluginh 2.1.2.

          Show
          Gaz Moorst added a comment - I get this problem regardless of the forkmode setting. It only seems to have been doing this since I started using Maven 2.0.4, with maven-surefire-plugin 2.2 instead of maven 2.0.2 with pluginh 2.1.2.
          Hide
          Patrice Bertrand added a comment -

          I have the same issue. I use Maven 2.0.4 with the 2.2 plugin. I had to force Maven to use of version 2.1.3 to work around the problem.

          Show
          Patrice Bertrand added a comment - I have the same issue. I use Maven 2.0.4 with the 2.2 plugin. I had to force Maven to use of version 2.1.3 to work around the problem.
          Hide
          Bernard Marshall added a comment -

          I also have the problem using Maven 2.0.4 with the 2.2 plugin. If however I set <forkMode>always</forkMode> the results are not cumulative.

          Show
          Bernard Marshall added a comment - I also have the problem using Maven 2.0.4 with the 2.2 plugin. If however I set <forkMode>always</forkMode> the results are not cumulative.
          Hide
          Mark Chaimungkalanont added a comment -

          I also see this problem. It has definitely only started happening since 2.2. If I add

                <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-surefire-plugin</artifactId>
                  <version>2.1.3</version>
                </plugin>
          

          Things start working again. The linked issue MSUREFIRE-149 has a test project setup that seems to cause this problem consistently.

          My experience re: the forkMode setting is that it happens less frequently with forkMode == always, but it does still seem to happen

          Show
          Mark Chaimungkalanont added a comment - I also see this problem. It has definitely only started happening since 2.2. If I add <plugin> <groupId> org.apache.maven.plugins </groupId> <artifactId> maven-surefire-plugin </artifactId> <version> 2.1.3 </version> </plugin> Things start working again. The linked issue MSUREFIRE-149 has a test project setup that seems to cause this problem consistently. My experience re: the forkMode setting is that it happens less frequently with forkMode == always, but it does still seem to happen
          Hide
          stephane bouchet added a comment -

          Hi,

          Got the same problem. (maven 2.0.4, surefireplugin 2.3-SNAPSHOT)
          this is very annoying to have duplicate entry to my reports.

          Tracking a little bit shows that 2.1.3 plugin use surefire-booter 1.5.3 and 2.2 use surefire-booter 2.0 .

          I think the problem comes from there.

          Stéphane

          Show
          stephane bouchet added a comment - Hi, Got the same problem. (maven 2.0.4, surefireplugin 2.3-SNAPSHOT) this is very annoying to have duplicate entry to my reports. Tracking a little bit shows that 2.1.3 plugin use surefire-booter 1.5.3 and 2.2 use surefire-booter 2.0 . I think the problem comes from there. Stéphane
          Hide
          Eugene Zhuravlev added a comment -

          The fix for this is quite easy: need to override the reset() method for the XMLReporter and clear the list of tests there.

          Show
          Eugene Zhuravlev added a comment - The fix for this is quite easy: need to override the reset() method for the XMLReporter and clear the list of tests there.
          Hide
          Eugene Zhuravlev added a comment -

          Attached is the patch for the XMLReporter that fixes the problem

          Show
          Eugene Zhuravlev added a comment - Attached is the patch for the XMLReporter that fixes the problem
          Hide
          Jason van Zyl added a comment -

          Patch applied. Thanks!

          Show
          Jason van Zyl added a comment - Patch applied. Thanks!
          Hide
          Robert Watkins added a comment -

          For anyone else who stumbles upon this, the workaround above (using the 2.1.3 plugin) has its own set of issues.

          The default values of a few of the configuration arguments for the surefire plugin changed between 2.1.3 and 2.2. The two I've come across were:

          • forkmode (defaults to 'once' in 2.2, but 'none' in 2.1.3)
          • childDelegation (defaults to 'false' in 2.2, but 'true' in 2.1.3)

          The changes in these defaults can cause issues (I got OOM errors and problems with the XML classloaders) that are somewhat hard to diagnose, partially because the 2.2 plugin documentation doesn't indicate that the defaults had changed between versions. (For example, the only reason I noticed the forkmode problem is because I was dumping the running Java processes in order to diagnose the OOM).

          Show
          Robert Watkins added a comment - For anyone else who stumbles upon this, the workaround above (using the 2.1.3 plugin) has its own set of issues. The default values of a few of the configuration arguments for the surefire plugin changed between 2.1.3 and 2.2. The two I've come across were: forkmode (defaults to 'once' in 2.2, but 'none' in 2.1.3) childDelegation (defaults to 'false' in 2.2, but 'true' in 2.1.3) The changes in these defaults can cause issues (I got OOM errors and problems with the XML classloaders) that are somewhat hard to diagnose, partially because the 2.2 plugin documentation doesn't indicate that the defaults had changed between versions. (For example, the only reason I noticed the forkmode problem is because I was dumping the running Java processes in order to diagnose the OOM).

            People

            • Assignee:
              Jason van Zyl
              Reporter:
              Daniel Kulp
            • Votes:
              18 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: