Maven Surefire (moved to ASF)
  1. Maven Surefire (moved to ASF)
  2. SUREFIRE-762

Add support for test compile and test runtime separation

    Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.9
    • Fix Version/s: 3.0
    • Component/s: Maven Surefire Plugin
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      1

      Description

      In some cases it is interesting to bind surefire to multiple execution targets but have it operate on different classpaths.

      e.g:

      • Have the same test suite run against multiple JPA providers; EclipseLink and Hibernate
      • Arquillian test suite run against Tomcat and Jetty

      In these cases you would have the TestCompile scope defined in your normal dependency chain scoped as "test", while you can tell surefire to activate another profile during test.

      TestCompile = JPA API
      TestRuntime = Hibernate | EclipseLink

      Example:
      https://gist.github.com/1155271

        Activity

        Hide
        Aslak Knutsen added a comment -

        Git Commit: https://github.com/aslakknutsen/maven-surefire/commit/e914da14c12f96b042bad92f703d57e64f59276c

        Adds the 'additionalProfiles' configuration option. The additional profile ids are used to activated the profile to re calculate the TestClasspath and open up for runtime only dependencies.

        Used in cases where you e.g. only want API on classpath but test against a specific impl. Individual Surefire Executions can be configured to run against different API impls.

        Show
        Aslak Knutsen added a comment - Git Commit: https://github.com/aslakknutsen/maven-surefire/commit/e914da14c12f96b042bad92f703d57e64f59276c Adds the 'additionalProfiles' configuration option. The additional profile ids are used to activated the profile to re calculate the TestClasspath and open up for runtime only dependencies. Used in cases where you e.g. only want API on classpath but test against a specific impl. Individual Surefire Executions can be configured to run against different API impls.
        Hide
        Tibor Digana added a comment -

        @Aslak
        You can use different dependencies in profiles instead of profiles in surefire execution. Are you still interested in this fix, or should i close this?

        Show
        Tibor Digana added a comment - @Aslak You can use different dependencies in profiles instead of profiles in surefire execution. Are you still interested in this fix, or should i close this?
        Hide
        Aslak Knutsen added a comment -

        Yes, this is still relevant.

        Different dependencies in a profile serves a different purpose and require re-execution of the maven process. Allowing Surefire to resolve multiple classpaths in different surefire executions opens for the option to test multiple different API implementations in one Maven execution.

        Currently to test multiple different classpaths resolved in one maven execution you would need to use multiple modules, e.g. core, core-test-junit4, core-test-junit4.2, core-test-junit4.3 etc...When all you really need is to swap the junit version and run all tests multiple times.

        This patch/issue opens up for a test-runtime 'scope' per surefire execution, a scope that is currently missing in Maven.

        Show
        Aslak Knutsen added a comment - Yes, this is still relevant. Different dependencies in a profile serves a different purpose and require re-execution of the maven process. Allowing Surefire to resolve multiple classpaths in different surefire executions opens for the option to test multiple different API implementations in one Maven execution. Currently to test multiple different classpaths resolved in one maven execution you would need to use multiple modules, e.g. core, core-test-junit4, core-test-junit4.2, core-test-junit4.3 etc...When all you really need is to swap the junit version and run all tests multiple times. This patch/issue opens up for a test-runtime 'scope' per surefire execution, a scope that is currently missing in Maven.
        Hide
        Tibor Digana added a comment -

        These params did not help?
        dependenciesToScan
        classpathDependencyExcludes
        If still not satisfied, then I would assign this issue to 3.0.

        Show
        Tibor Digana added a comment - These params did not help? dependenciesToScan classpathDependencyExcludes If still not satisfied, then I would assign this issue to 3.0.
        Hide
        Aslak Knutsen added a comment -

        dependenciesToScan helps some, but different use case in my eyes. The intended use is to share a test suite between multiple implementors. For instance a TCK. As an example, a JPA TCK could provide a test artifact that the Hibernate team use dependenciesToScan in their test setup to include into their build.

        As a user, I'm not interested in the JPA TCK, I have my own tests I want to test against multiple implementations, Hibernate and EclipeLink (that's where this come in).

        Perhaps a good way to look at it would be;

        • I want my code to comply with a given test suite (dependenciesToScan)
        • I want my code to run on multiple implementations (additionalProfiles)

        I could create a common test suite in my project and create multiple test modules with individual dependency sets and import the test suite using dependenciesToScan, but that doesn't solve the multiple modules issue..

        classpathDependencyExcludes works the opposite direction, you would have to include all test-runtime dependencies in test scope and exclude the ones you don't want in the surefire execution. e.g. runtime-a, runtime-b and runtime-c are test scoped, then in execution of runtime-a you need to exclude runtime-b and runtime-c. With only 3 dependencies that's not too hard, but confusing. Now if runtime-a has runtime-a-N dependencies in a group you're in trouble. Not to mention of course that when runtime-a|z are resolved on the same classpath maven will do dependency conflict resolution, meaning depending on just runtime-a might not resolve the same as depending on runtime-a and runtime-b at the same time. So in short, no

        Show
        Aslak Knutsen added a comment - dependenciesToScan helps some, but different use case in my eyes. The intended use is to share a test suite between multiple implementors. For instance a TCK. As an example, a JPA TCK could provide a test artifact that the Hibernate team use dependenciesToScan in their test setup to include into their build. As a user, I'm not interested in the JPA TCK, I have my own tests I want to test against multiple implementations, Hibernate and EclipeLink (that's where this come in). Perhaps a good way to look at it would be; I want my code to comply with a given test suite (dependenciesToScan) I want my code to run on multiple implementations (additionalProfiles) I could create a common test suite in my project and create multiple test modules with individual dependency sets and import the test suite using dependenciesToScan, but that doesn't solve the multiple modules issue.. classpathDependencyExcludes works the opposite direction, you would have to include all test-runtime dependencies in test scope and exclude the ones you don't want in the surefire execution. e.g. runtime-a, runtime-b and runtime-c are test scoped, then in execution of runtime-a you need to exclude runtime-b and runtime-c. With only 3 dependencies that's not too hard, but confusing. Now if runtime-a has runtime-a-N dependencies in a group you're in trouble. Not to mention of course that when runtime-a|z are resolved on the same classpath maven will do dependency conflict resolution, meaning depending on just runtime-a might not resolve the same as depending on runtime-a and runtime-b at the same time. So in short, no
        Hide
        Tibor Digana added a comment -

        Assigned to 3.0. The user should be able to customize the classpath upon the project profile or plugin execution.

        Show
        Tibor Digana added a comment - Assigned to 3.0. The user should be able to customize the classpath upon the project profile or plugin execution.

          People

          • Assignee:
            Tibor Digana
            Reporter:
            Aslak Knutsen
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: