Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-745

-Dtest supports multiple test classes but not multiple test methods

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.12.1
    • Component/s: None
    • Labels:
      None
    • Environment:
    • Complexity:
      Intermediate
    • Number of attachments :
      6

      Description

      The -Dtest parameter is very handy for running a specific test class or test method. It also supports running multiple test classes. Unfortunately, it does not permit specifying running multiple test methods. It would be great if this were possible.

      The examples below are from the Apache Commons project.

      WORKS: Run multiple test classes:
      mvn test -Dtest=ImmutablePairTest,StopWatchTest

      WORKS: Run a specific test method:
      mvn test -Dtest=ImmutablePairTest#testBasic

      DOES NOT WORK:
      mvn test -Dtest=StopWatchTest#testStopWatchSimple,StopWatchTest#testStopWatchSimpleGet
      mvn test -Dtest=ImmutablePairTest#testBasic,StopWatchTest#testLang315

      1. multipleMethods.patch
        10 kB
        rainLee
      2. multipleMethods-v2.patch
        5 kB
        rainLee
      3. multipleMethods-v3.patch
        14 kB
        rainLee
      4. multipleMethods-v4.patch
        14 kB
        rainLee
      5. SUREFIRE-745.patch
        18 kB
        rainLee
      6. SUREFIRE-745-v2.patch
        22 kB
        rainLee

        Activity

        Hide
        Tibor Digana added a comment -

        @Jianfeng Sun
        Have you tried with system property -Dtest=... and the same provider ?

        Show
        Tibor Digana added a comment - @Jianfeng Sun Have you tried with system property -Dtest=... and the same provider ?
        Hide
        Tibor Digana added a comment -

        @Jianfeng Sun
        We have fixed such bug https://jira.codehaus.org/browse/SUREFIRE-1028 in 2.18-SNAPSHOT.
        Pls setup repository http://repository.apache.org/content/groups/snapshots for the SNAPSHOT version of maven-surefire.
        Let us know if it works for you.

        Show
        Tibor Digana added a comment - @Jianfeng Sun We have fixed such bug https://jira.codehaus.org/browse/SUREFIRE-1028 in 2.18-SNAPSHOT. Pls setup repository http://repository.apache.org/content/groups/snapshots for the SNAPSHOT version of maven-surefire. Let us know if it works for you.
        Hide
        Jianfeng Sun added a comment - - edited

        @Tibor

        Thanks for the response.

        By using 2.18-SNAPSHOT for both surefire and failsafe,
        1) if I use -Dit.test=class1#method1,class2#method2
        I got this:
        main:
        [INFO] Executed tasks
        [INFO]
        [INFO] — maven-surefire-plugin:2.18-SNAPSHOT:test (default-test) @ selenium —
        [INFO]
        [INFO] — maven-jar-plugin:2.4:jar (default-jar) @ selenium —
        [INFO]
        [INFO] — maven-failsafe-plugin:2.18-SNAPSHOT:integration-test (run-test) @ selenium —
        [INFO] Failsafe report directory: C:\Automation testing tool\workspace\fox.emf.selenium.Oct5\target\surefire-reports\10-15-2014-03-55-PM
        [INFO] Using configured provider org.apache.maven.surefire.junitcore.JUnitCoreProvider
        [INFO] parallel='none', perCoreThreadCount=true, threadCount=0, useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, threadCountMethods=0, parallelOptimized=true

        -------------------------------------------------------
        T E S T S
        -------------------------------------------------------

        -------------------------------------------------------
        T E S T S
        -------------------------------------------------------

        Results :

        Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

        [WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent!
        [INFO]
        [INFO] — properties-maven-plugin:1.0-alpha-2:read-project-properties (default) @ selenium —
        [INFO]
        [INFO] — maven-postman-plugin:0.1.6:send-mail (send_email) @ selenium —

        2) if I use -Dtest=class1#method1,class2#method2
        I got Failed tests:
        class1.method1:21 ? NullPointer
        class1.method2:16 ? NullPointer
        class2.method1:21 ? NullPointer
        class2.method2:17 ? NullPointer

        the java code for NullPointer is "System.getProperty("user.dir")+"/testflow/Spot_TC07_Export Functionality.xls""
        I guess the NullPorinter is from system property user dir.
        I'm sure it will work when we only run one test at a time.

        Show
        Jianfeng Sun added a comment - - edited @Tibor Thanks for the response. By using 2.18-SNAPSHOT for both surefire and failsafe, 1) if I use -Dit.test=class1#method1,class2#method2 I got this: main: [INFO] Executed tasks [INFO] [INFO] — maven-surefire-plugin:2.18-SNAPSHOT:test (default-test) @ selenium — [INFO] [INFO] — maven-jar-plugin:2.4:jar (default-jar) @ selenium — [INFO] [INFO] — maven-failsafe-plugin:2.18-SNAPSHOT:integration-test (run-test) @ selenium — [INFO] Failsafe report directory: C:\Automation testing tool\workspace\fox.emf.selenium.Oct5\target\surefire-reports\10-15-2014-03-55-PM [INFO] Using configured provider org.apache.maven.surefire.junitcore.JUnitCoreProvider [INFO] parallel='none', perCoreThreadCount=true, threadCount=0, useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, threadCountMethods=0, parallelOptimized=true ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent! [INFO] [INFO] — properties-maven-plugin:1.0-alpha-2:read-project-properties (default) @ selenium — [INFO] [INFO] — maven-postman-plugin:0.1.6:send-mail (send_email) @ selenium — 2) if I use -Dtest=class1#method1,class2#method2 I got Failed tests: class1.method1:21 ? NullPointer class1.method2:16 ? NullPointer class2.method1:21 ? NullPointer class2.method2:17 ? NullPointer the java code for NullPointer is "System.getProperty("user.dir")+"/testflow/Spot_TC07_Export Functionality.xls"" I guess the NullPorinter is from system property user dir. I'm sure it will work when we only run one test at a time.
        Hide
        Tibor Digana added a comment -

        @Jianfeng Sun
        Regarding filters (classes & methods) and groups my plan is to build IT tests with surefire-junit47 in 2.18.
        If they fail I will report and fix a bug so we will have chance to get stable surefire plugin.
        Did you combine "parallel=..." with surefire in the above tests ?

        Show
        Tibor Digana added a comment - @Jianfeng Sun Regarding filters (classes & methods) and groups my plan is to build IT tests with surefire-junit47 in 2.18. If they fail I will report and fix a bug so we will have chance to get stable surefire plugin. Did you combine "parallel=..." with surefire in the above tests ?
        Hide
        Jianfeng Sun added a comment - - edited

        @Tibo Digana
        Filters and groups will be great features for surefire. Thank you for the work.
        I didn't use parallel parameter in this test. So I believe by default it should be set as "false".

        And I just tried

        <configuration>
        <parallel>methods</parallel>
        <threadCount>10</threadCount>
        </configuration>

        see the same result.

        Show
        Jianfeng Sun added a comment - - edited @Tibo Digana Filters and groups will be great features for surefire. Thank you for the work. I didn't use parallel parameter in this test. So I believe by default it should be set as "false". And I just tried <configuration> <parallel>methods</parallel> <threadCount>10</threadCount> </configuration> see the same result.

          People

          • Assignee:
            Olivier Lamy
            Reporter:
            reid holmes
          • Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: