Maven Surefire

When you <include> a JUnit TestSuite (with no test methods), no tests are run

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 2.0 (2.2 plugin)
  • Fix Version/s: 2.3
  • Component/s: None
  • Labels:
    None
  • Testcase included:
    yes
  • Number of attachments :
    3

Description

In the attached Maven project, I started with the default Maven getting started archetype and then created an AppTestSuite class, which extends TestSuite and defines the static suite() method, but it has no test methods of its own. Then I added an <includes> configuration that specified <includes><include>**/AppTestSuite.java</include></includes>. But when I run "mvn test", it claims that "There are no tests to run."

I've watched this happening in a debugger. The problem is in surefire-api-2.0.jar in PojoTestSet , on line 249. When attempting to acquire a list of the number of tests to run, instead of invoking the suite method and counting the number of tests that come out, it's attempting to discoverTestMethods on the object (methods whose names begin with "test*"), which naturally won't work for suites.

This is a really serious issue that's going to block testing for us. (We happen to attach a TestSetup decorator to our suite, so we do need to run these tests as a suite rather than as individual test cases.)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Dan Fabulich added a comment -

At a suggestion from groves on IRC, I tried making my AppTestSuite class extend TestCase instead of TestSuite. This forced surefire to count the test cases correctly, and was an effective workaround for my project.

Show
Dan Fabulich added a comment - At a suggestion from groves on IRC, I tried making my AppTestSuite class extend TestCase instead of TestSuite. This forced surefire to count the test cases correctly, and was an effective workaround for my project.
Hide
Permalink
Charlie Groves added a comment -

If JUnitDirectoryTestSuite checked against Test instead of TestCase in its isAssignableFrom test, this wouldn't be an issue. From what I can tell in JUnitTestSet, having the testClasses passed to the constructor consist of anything that implements Test wouldn't be a problem as JUnitTestSet only uses the run and countTestMethods methods.

Show
Charlie Groves added a comment - If JUnitDirectoryTestSuite checked against Test instead of TestCase in its isAssignableFrom test, this wouldn't be an issue. From what I can tell in JUnitTestSet, having the testClasses passed to the constructor consist of anything that implements Test wouldn't be a problem as JUnitTestSet only uses the run and countTestMethods methods.
Hide
Permalink
Mike Perham added a comment -

Charlie, excellent detective work. I made your suggested changes and found this fixes the problem (I replaced my surefire-junit-2.0.jar with the attached snapshot). I do not have commit privileges to the surefire source so this patch will need to be reviewed and committed by someone else.

Show
Mike Perham added a comment - Charlie, excellent detective work. I made your suggested changes and found this fixes the problem (I replaced my surefire-junit-2.0.jar with the attached snapshot). I do not have commit privileges to the surefire source so this patch will need to be reviewed and committed by someone else.
Hide
Permalink
Brett Porter added a comment -

this also should be changed in the TestNG provider

Show
Brett Porter added a comment - this also should be changed in the TestNG provider
Hide
Permalink
Mike Perham added a comment -

Updated surefire-junit and surefire-testng.

Show
Mike Perham added a comment - Updated surefire-junit and surefire-testng.

People

  • Assignee:
    Mike Perham
    Reporter:
    Dan Fabulich
Vote (1)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: