Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-239

surefire plugin needs to be able to fork tests

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      2

      Description

      they can leak memory, so a "fork once for all" option would be good.

      1. pom.xml
        2 kB
        Andy Glick
      2. SurefirePlugin.java
        13 kB
        Andy Glick

        Issue Links

          Activity

          Hide
          Vincent Massol added a comment -

          Note: This is required for the Clover plugin to work properly

          Show
          Vincent Massol added a comment - Note: This is required for the Clover plugin to work properly
          Hide
          Andy Glick added a comment -

          Is the intention to have M2 do the forking, or will the forking behavior be added to surefire, and be passed from the plugin to surefire by parameter? That seems like a better separation of concerns.

          The differences between surefire and the JUnit Ant Task are pretty significant, and surefire's native mode is to run tests in separate threads, using an isolated class loader so it might be a good idea to get the surefire guys input concerning forking the test cases.

          Show
          Andy Glick added a comment - Is the intention to have M2 do the forking, or will the forking behavior be added to surefire, and be passed from the plugin to surefire by parameter? That seems like a better separation of concerns. The differences between surefire and the JUnit Ant Task are pretty significant, and surefire's native mode is to run tests in separate threads, using an isolated class loader so it might be a good idea to get the surefire guys input concerning forking the test cases.
          Hide
          Brett Porter added a comment -

          I agree with doing it in surefire. I didn't think it executed them in separate threads though (just in a separate classloader). I will reply on the surefire list.

          Show
          Brett Porter added a comment - I agree with doing it in surefire. I didn't think it executed them in separate threads though (just in a separate classloader). I will reply on the surefire list.
          Hide
          Andy Glick added a comment -

          I've started work on this. Will add a class ForkMode as an enumerated type in surefire and the plugin will reference it. The surefirebooter class will determine the ForkMode value passed by the plugin

          {NONE, ONCE, EACH_TEST}

          default value will be ONCE and kick off tests according to ForkMode - I intend to add a class SurefireConfiguration to make this similar to plexus.compiler, I'm attempting to create a protocol for parameter passing as we've already got surefire-beanshell, surefire-jython, surefire-web and surefire-xmlrpc, i expect to add surefire-testng and possibly surefire-ivalidator when things settle down a bit

          Show
          Andy Glick added a comment - I've started work on this. Will add a class ForkMode as an enumerated type in surefire and the plugin will reference it. The surefirebooter class will determine the ForkMode value passed by the plugin {NONE, ONCE, EACH_TEST} default value will be ONCE and kick off tests according to ForkMode - I intend to add a class SurefireConfiguration to make this similar to plexus.compiler, I'm attempting to create a protocol for parameter passing as we've already got surefire-beanshell, surefire-jython, surefire-web and surefire-xmlrpc, i expect to add surefire-testng and possibly surefire-ivalidator when things settle down a bit
          Hide
          Brett Porter added a comment -

          options we need to support (from m1):

          • fork mode
          • jvmargs
          • envvars
          • jvm location
          Show
          Brett Porter added a comment - options we need to support (from m1): fork mode jvmargs envvars jvm location
          Hide
          Andy Glick added a comment -

          Brett, what about maven.junit.usefile = false?

          Show
          Andy Glick added a comment - Brett, what about maven.junit.usefile = false?
          Hide
          Brett Porter added a comment -

          that's in another jira task... those were the ones related to forking.

          How is this progressing?

          Show
          Brett Porter added a comment - that's in another jira task... those were the ones related to forking. How is this progressing?
          Hide
          Andy Glick added a comment -

          This is progressing - I have hacked a way (on windows) to fork Surefire tests once and have successfully executed the tests from the surefire subproject using the surefire-booter to fork the Surefire class - the reason for the hack is that there is a problem with plexus-utils - it seems that under windows, which i am using, classpaths with embedded spaces cause runtime.exec to fail

          in addition I'm not driving this with the surefire plugin, but am emulating the surefire plugin. the emulator calls the SurfireBooter class with the same arguments + forkMode aas the plugin

          i have filed PLX-161 which reports the plexus-utils issue

          Show
          Andy Glick added a comment - This is progressing - I have hacked a way (on windows) to fork Surefire tests once and have successfully executed the tests from the surefire subproject using the surefire-booter to fork the Surefire class - the reason for the hack is that there is a problem with plexus-utils - it seems that under windows, which i am using, classpaths with embedded spaces cause runtime.exec to fail in addition I'm not driving this with the surefire plugin, but am emulating the surefire plugin. the emulator calls the SurfireBooter class with the same arguments + forkMode aas the plugin i have filed PLX-161 which reports the plexus-utils issue
          Hide
          Andy Glick added a comment -

          This is the pom and the code for the maven-surefire-plugin

          Show
          Andy Glick added a comment - This is the pom and the code for the maven-surefire-plugin
          Hide
          Andy Glick added a comment -

          Keep realizing that being new to all of these tools leads me to make numerous mistakes. I have produced and checked into svn code that forks surefire tests or runs them in process. The code is in the surefire branch SUREFIRE_FORK_EXEC. I copied the newest Java report classes into the branch. All of the artifacts created from the branch will have version numbers that should not collide with any other versions, neither now or in the future. So it should be easy to checkout the surefire branch and build and deploy its artifacts.

          I've uploaded the Java code and a modified POM for a version of the surefire plugin that is compatible with the surefire changes. To fork the surefire tests you add a <forkMode>once</forkMode> to the surefire plugin's configuration. The plugin's forkMode value defaults to "none".

          Brett suggested, and I intend to implement, a monitor implementation using a listener interface with concrete implmentation classes. That way clients can hook up the surefire component's logging messages if they choose.

          Show
          Andy Glick added a comment - Keep realizing that being new to all of these tools leads me to make numerous mistakes. I have produced and checked into svn code that forks surefire tests or runs them in process. The code is in the surefire branch SUREFIRE_FORK_EXEC. I copied the newest Java report classes into the branch. All of the artifacts created from the branch will have version numbers that should not collide with any other versions, neither now or in the future. So it should be easy to checkout the surefire branch and build and deploy its artifacts. I've uploaded the Java code and a modified POM for a version of the surefire plugin that is compatible with the surefire changes. To fork the surefire tests you add a <forkMode>once</forkMode> to the surefire plugin's configuration. The plugin's forkMode value defaults to "none". Brett suggested, and I intend to implement, a monitor implementation using a listener interface with concrete implmentation classes. That way clients can hook up the surefire component's logging messages if they choose.
          Hide
          Dave Sag added a comment -

          I'd really like to see the surefire plugin support the -enableassertions (aka -ea) flag too. Right now I have to set MAVEN_OPTS=-ea in my profile but this is not portable. As a result unit tests that pass fine on my machine will fail on any other developers machine unless they too have remembered to set the MAVEN_OPTS environment variable correctly.

          Show
          Dave Sag added a comment - I'd really like to see the surefire plugin support the -enableassertions (aka -ea) flag too. Right now I have to set MAVEN_OPTS=-ea in my profile but this is not portable. As a result unit tests that pass fine on my machine will fail on any other developers machine unless they too have remembered to set the MAVEN_OPTS environment variable correctly.
          Hide
          Andy Glick added a comment -

          Dave -

          If you take a look at Brett's remarks of Sep 22, 2005, you'll notice that he mentions jvmargs as a required addition to the supported surefire argument list. In a complete surefire/surefire plugin implementation you would add any "ea" arguments to the jvmargs and you wouldn't have to use MAVEN_OPTS. I actually do sympathize with your desire to see this completed, as it was my desire to see it completed that lead me to attempt to add the forking behavior. There is still some unfinished business WRT forking, but we are going to get to jvmargs, please bear with us.

          Show
          Andy Glick added a comment - Dave - If you take a look at Brett's remarks of Sep 22, 2005, you'll notice that he mentions jvmargs as a required addition to the supported surefire argument list. In a complete surefire/surefire plugin implementation you would add any "ea" arguments to the jvmargs and you wouldn't have to use MAVEN_OPTS. I actually do sympathize with your desire to see this completed, as it was my desire to see it completed that lead me to attempt to add the forking behavior. There is still some unfinished business WRT forking, but we are going to get to jvmargs, please bear with us.
          Hide
          Dave Sag added a comment -

          Hi Andy - thanks for that. Am bearing with you. also note this issue x-references issue MNG-1087.

          Show
          Dave Sag added a comment - Hi Andy - thanks for that. Am bearing with you. also note this issue x-references issue MNG-1087 .
          Hide
          David Jackman added a comment -

          One feature of forking the tests that we're currently using in M1 is the working directory for the test process (maven.junit.dir). It's possible we can accomplish what we need to do by setting environment variables, but for completeness the working directory ought to be included.

          Show
          David Jackman added a comment - One feature of forking the tests that we're currently using in M1 is the working directory for the test process (maven.junit.dir). It's possible we can accomplish what we need to do by setting environment variables, but for completeness the working directory ought to be included.
          Hide
          Mark Proctor added a comment -

          Is that .java and .pom working? If so any chance we could have a surefire snapshot release? We are currently blocked from moving Drools to M2 as maven tests for jython fail if we can't fork.

          Show
          Mark Proctor added a comment - Is that .java and .pom working? If so any chance we could have a surefire snapshot release? We are currently blocked from moving Drools to M2 as maven tests for jython fail if we can't fork.
          Hide
          Joakim Erdfelt added a comment -

          Additional note.
          This is required for the Cobertura plugin to work properly.

          Show
          Joakim Erdfelt added a comment - Additional note. This is required for the Cobertura plugin to work properly.
          Hide
          John Casey added a comment -

          I looked through surefire today, but couldn't find a place in the booter where it forked. Can you close this out if it's already in there, or when it does land there?

          Show
          John Casey added a comment - I looked through surefire today, but couldn't find a place in the booter where it forked. Can you close this out if it's already in there, or when it does land there?
          Hide
          Jason van Zyl added a comment -

          The basic forking is in. There are a couple options that should be added like jvmargs and envars, but i will enter those as separate issues and release what we have.

          Show
          Jason van Zyl added a comment - The basic forking is in. There are a couple options that should be added like jvmargs and envars, but i will enter those as separate issues and release what we have.
          Hide
          Jason van Zyl added a comment -

          Forking is not working on Windows.

          Show
          Jason van Zyl added a comment - Forking is not working on Windows.
          Hide
          Jason van Zyl added a comment -

          Forking is now working in general.

          Show
          Jason van Zyl added a comment - Forking is now working in general.

            People

            • Assignee:
              Jason van Zyl
              Reporter:
              Brett Porter
            • Votes:
              12 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: