Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-726

Test list preprocessor support for tests to be run

    Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.9
    • Fix Version/s: 3.0
    • Component/s: None
    • Labels:
      None
    • Complexity:
      Intermediate
    • Testcase included:
      yes
    • Patch Submitted:
      Yes
    • Number of attachments :
      4

      Description

      This exposes an interface(named TestListPreprocessor, which has a method named preprocessTestClasses) that can be implemented by a user and injected into surefire plugin configuration to have it used for preprocessing list of tests to be run. Given the original test list, user's preprocessing algorithm can choose to prune or reorder the list and return it back from TestListPreprocessor#preprocessTestClasses, which is then used as the list of tests to be executed.

      The patches attached expose a configuration element named "testPreprocessor" the default value of which is "none". User can choose to set it to a value matching the format "<fully-qualified-class-name>[<group-id>:<artifact-id>:<version>]" and have the class loaded and called with list of tests to be run. The list of tests returned by the call is then considered for execution.

      Patch description:
      The feature is done in 2 patches. Description follows:

      0001-Adds-test-list-preprocessor-support-which-loads-an-o.patch
      This patch actually adds the feature and unit tests. It adds the interface, an abstraction that encapsulates aforementioned configuration, booter serialization/deserialization, directory-scanner changes etc. The changes have effect in both forked and in-process execution mode.

      0002-Integration-test-for-preprocessor.-2-modules-one-pro.patch
      This patch adds an integration test for preprocessor feature. The integration test uses a 2 module setup, where the first module implements the TestListPreprocessor interface which selects only even indexed test classes from the list passed in. The second module(which depends on first) uses the artifact from first and uses the following configuration:
      --------------------
      <configuration>
      <testPreprocessor>test.preprocessor.EvenTestOnlyPreprocessor[org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT]</testPreprocessor>
      </configuration>
      --------------------
      The test then asserts that only 2(0th and 2nd) of the 4 test classes the second module has are executed.

      The patches are created on a git-svn clone using format-patch, please use 'patch -p1 < 0001-Adds...'(while in the trunk directory) to apply the patches on svn repository. If using git-svn, git-am can be directly be invoked.

      The patches are done over http://svn.apache.org/repos/asf/maven/surefire/trunk@1091357 (trunk HEAD) so should apply cleanly on any recent enough revision.

      ===============================================================================

      Context:
      http://stackoverflow.com/questions/5124823/reducing-the-build-time-hudson
      http://code.google.com/p/tlb/issues/detail?id=1
      http://test-load-balancer.github.com

        Issue Links

          Activity

          Hide
          Kristian Rosenvold added a comment - - edited

          Nice strategy for solving this one, we should be thinking about viewing this feature request together with one or more of the linked issues, maybe especially SUREFIRE-530

          Show
          Kristian Rosenvold added a comment - - edited Nice strategy for solving this one, we should be thinking about viewing this feature request together with one or more of the linked issues, maybe especially SUREFIRE-530
          Hide
          Janmejay Singh added a comment -

          It can easily be enhanced to allow injection of testPreprocessor declaration via system-property.

          As in, user has a maven plugin that hooks up in the lifecycle before test and adds a system-property(for instance, surefire-test-preprocessor which has value "test.preprocessor.EvenTestOnlyPreprocessor[org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT]") and the AbstractSurefireMojo#testPreprocessor can first check for configuration and if no config found, can check for the system-property and use first one found.

          This will also make it more flexible, because user can choose the strategy before invoking the build(to choose a different implementation all the user has to do is -Dsurefire-test-preprocessor=test.preprocessor.EvenTestOnlyPreprocessor[org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT]).

          Show
          Janmejay Singh added a comment - It can easily be enhanced to allow injection of testPreprocessor declaration via system-property. As in, user has a maven plugin that hooks up in the lifecycle before test and adds a system-property(for instance, surefire-test-preprocessor which has value "test.preprocessor.EvenTestOnlyPreprocessor [org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT] ") and the AbstractSurefireMojo#testPreprocessor can first check for configuration and if no config found, can check for the system-property and use first one found. This will also make it more flexible, because user can choose the strategy before invoking the build(to choose a different implementation all the user has to do is -Dsurefire-test-preprocessor=test.preprocessor.EvenTestOnlyPreprocessor [org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT] ).
          Hide
          Janmejay Singh added a comment -

          added patch 0003 which applies atop 0002, and exposes the testPreprocessor as a property and adds an integration test for it. Property "maven.test.preprocessor" accepts the testPreprocessor configuration string and uses the same format as explicit configuration.

          Show
          Janmejay Singh added a comment - added patch 0003 which applies atop 0002, and exposes the testPreprocessor as a property and adds an integration test for it. Property "maven.test.preprocessor" accepts the testPreprocessor configuration string and uses the same format as explicit configuration.
          Hide
          Janmejay Singh added a comment -

          Patch 0004 applies atop 0003 and exposes testPreprocessor configuration as property "maven.test.preprocessor" for IntegrationTestMojo

          Show
          Janmejay Singh added a comment - Patch 0004 applies atop 0003 and exposes testPreprocessor configuration as property "maven.test.preprocessor" for IntegrationTestMojo
          Hide
          Parameswaran Raman added a comment -

          Hi,

          I am looking to use TLB with Maven for my project. It would be great if you can push in this patch as quickly as possible as there is a significant improvement (in build time) that we are looking forward to.


          Thanks,
          Params
          parameshr@gmail.com

          Show
          Parameswaran Raman added a comment - Hi, I am looking to use TLB with Maven for my project. It would be great if you can push in this patch as quickly as possible as there is a significant improvement (in build time) that we are looking forward to. – Thanks, Params parameshr@gmail.com
          Hide
          Daniele Dellafiore added a comment -

          Hi. I step into this issue lookig for a way to have a sort of global test setup, without having to deal with suites.
          The purpose is to have a reset of the whole database and reimport of test data at the beginning of the tests, and not pay the reset/reimport on every single test case.
          Maybe this is a nice way to address my problem, what do you think>

          Show
          Daniele Dellafiore added a comment - Hi. I step into this issue lookig for a way to have a sort of global test setup, without having to deal with suites. The purpose is to have a reset of the whole database and reimport of test data at the beginning of the tests, and not pay the reset/reimport on every single test case. Maybe this is a nice way to address my problem, what do you think>
          Hide
          Janmejay Singh added a comment -

          Yes, this can be used as a mechanism to do things before the whole suite executes.

          Show
          Janmejay Singh added a comment - Yes, this can be used as a mechanism to do things before the whole suite executes.
          Hide
          Janmejay Singh added a comment -

          Has the date for 3.0 been decided?

          It so happens that im working with a lot of maven projects these days with very long builds, I can totally use this feature. If 3.0 is still far, is it possible to have this merged in one of the 12.2.x releases?

          Show
          Janmejay Singh added a comment - Has the date for 3.0 been decided? It so happens that im working with a lot of maven projects these days with very long builds, I can totally use this feature. If 3.0 is still far, is it possible to have this merged in one of the 12.2.x releases?

            People

            • Assignee:
              Unassigned
              Reporter:
              Janmejay Singh
            • Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: