support junit 4.0

It seems like a good first step would be adding a JUnit4Battery to surefire. I'll volunteer to take a first shot at this, basing it on the JUnitBattery.

please do this on the surefire-testng branch (which will be brought back to trunk eventually now that it is not experimental, but isn't yet). It is completely refactored and a much more achievable task there.

RunListener (the JUnit 4.0 equivalent of TestListener) is a class, not an interface, so I can't create a proxy for it. I can't really move forward until they fix it, or unless it's okay for me to have a compile-time dependency on JUnit 4.0. Attached is a first-pass at the code (testing using a modified version of JUnit 4.0 where I changed RunListener to an interface).

I'll take a look. It's not required that you don't use JUnit at all (see the testng provider for example). However, that is used to guarantee classloader separation which is still desirable. IF there is another alternative, that would be fine.

Is there anything I can do to make this happen?

I'm needing this ASAP... I've got about 4000 junit 4 tests to automate.

Is there anything I can do to speed this along or is there beta code out there someplace that I can use in the interim?

I have the same pb than you. I use Junit4 for my unit test and I can't run it in maven and a report.
If you need help, I can do something (testing or more)

I modifed JUnitTestSet to use JUnitCore from junit4 to run
the tests directly. I thought this is a nice way to get all the
annotation stuff working (@Before, timeout=xx) and it works.
At least it runs some exemplary tests.

The problem is now that surefire's reporting classes expect information
which is not provided by junit's RunListner. So no good logs/reports
can be created.

Additionally surefire's POJO approach collides with junit4.

Any idea when this might be done?

I want to know whether it is worth using JUnit 4 yet or not.

Thanks!

Anyone looking for a temporary measure try out http://www.unto.net/wiki/Maven_JUnit4_plugin and see if it works for you.

no timeframe, maybe days/weeks for a first step, more time for something that can be used in production, i am working on it (I am completely new to maven dev)

As a first step, this makes the maven surfire plugin use JUnit4DirectoryTestSuite if junit4 is available. JUnit4DirectoryTestSuite will not support mavens pojo approach anymore as this conflict with junit4 tests.

Looks like I could successfully connect maven reporting with JUnitCore, the problems mentioned above are not existing. Some basic test run now and mvn site creates to surefire html report.

But now I run into MSUREFIREREP-26, I will have a look at that...

With the attached patch I can run some exemplary junit4 tests. I will use it to run tests from a real project later.

The patch has the following features:

• requires Java 1.5 (includes pom change)
• runtime dependency on junit4
• I did not pay attention to classloading issues. Tips what needs to be done/should be done
  are welcome.

I tried to download surefire-junit4.zip. When I tried to run "mvn install" from unziped surefire-junit4 directory. I got the following exceptions:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

GroupId: org.apache.maven.surefire
ArtifactId: surefire-providers
Version: 2.0-SNAPSHOT

Reason: Unable to download the artifact from any repository

org.apache.maven.surefire:surefire-providers:pom:2.0-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2)

[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.reactor.MavenExecutionException: Cannot find parent: org.apache
.maven.surefire:surefire-providers for project: null:surefire-junit4:jar:null
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:365)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:278)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:115)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:256)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)

at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.project.ProjectBuildingException: Cannot find parent
: org.apache.maven.surefire:surefire-providers for project: null:surefire-junit4
:jar:null
at org.apache.maven.project.DefaultMavenProjectBuilder.assembleLineage(D
efaultMavenProjectBuilder.java:1161)
at org.apache.maven.project.DefaultMavenProjectBuilder.buildInternal(Def
aultMavenProjectBuilder.java:674)
at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromSourceFi
leInternal(DefaultMavenProjectBuilder.java:416)
at org.apache.maven.project.DefaultMavenProjectBuilder.build(DefaultMave
nProjectBuilder.java:192)
at org.apache.maven.DefaultMaven.getProject(DefaultMaven.java:515)
at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:447)
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:351)
... 11 more
Caused by: org.apache.maven.project.ProjectBuildingException: POM 'org.apache.ma
ven.surefire:surefire-providers' not found in repository: Unable to download the
artifact from any repository

org.apache.maven.surefire:surefire-providers:pom:2.0-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2)

at org.apache.maven.project.DefaultMavenProjectBuilder.findModelFromRepo
sitory(DefaultMavenProjectBuilder.java:513)
at org.apache.maven.project.DefaultMavenProjectBuilder.assembleLineage(D
efaultMavenProjectBuilder.java:1157)
... 17 more
Caused by: org.apache.maven.artifact.resolver.ArtifactNotFoundException: Unable
to download the artifact from any repository

org.apache.maven.surefire:surefire-providers:pom:2.0-SNAPSHOT

from the specified remote repositories:
central (http://repo1.maven.org/maven2)

at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(De
faultArtifactResolver.java:136)
at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(De
faultArtifactResolver.java:63)
at org.apache.maven.project.DefaultMavenProjectBuilder.findModelFromRepo
sitory(DefaultMavenProjectBuilder.java:467)
... 18 more
Caused by: org.apache.maven.wagon.ResourceDoesNotExistException: Unable to downl
oad the artifact from any repository
at org.apache.maven.artifact.manager.DefaultWagonManager.getArtifact(Def
aultWagonManager.java:260)
at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(De
faultArtifactResolver.java:124)
... 20 more
[INFO] ------------------------------------------------------------------------
[INFO] Total time: < 1 second
[INFO] Finished at: Thu Aug 24 14:27:06 PDT 2006
[INFO] Final Memory: 1M/2M
[INFO] ------------------------------------------------------------------------

Could you please tell me which Maven2 Repository I should add to make it work?

Thanks a lot!

Jian

Hi,

I found that I can use the apache.snapshots repository at:
<repository>
<id>apache.snapshots</id>
<url>http://people.apache.org/repo/m2-snapshot-repository</url>
</repository>

But, I encountered the following up problem:
[INFO] Scanning for projects...
[INFO] -------------------------------------------------------------------------
—
[INFO] Building SureFire JUnit 4.0 Runner
[INFO] task-segment: [compile]
[INFO] -------------------------------------------------------------------------
—
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:compile]
Compiling 3 source files to C:\TrinityPrj\maven2\surefire-junit4\target\classes
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Compilation failure

C:\TrinityPrj\maven2\surefire-junit4\src\main\java\org\apache\maven\surefire\jun
it\RunListenerInvocationHandler.java:[153,29] cannot find symbol
symbol : constructor ReportEntry(java.lang.Object,java.lang.String,java.lang.St
ring,java.lang.Throwable)
location: class org.apache.maven.surefire.report.ReportEntry

C:\TrinityPrj\maven2\surefire-junit4\src\main\java\org\apache\maven\surefire\jun
it\JUnit4TestSet.java:[109,53] invoke(java.lang.Object,java.lang.Object...) in j
ava.lang.reflect.Method cannot be applied to (java.lang.Object)

[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 seconds
[INFO] Finished at: Thu Aug 24 14:56:48 PDT 2006
[INFO] Final Memory: 2M/11M
[INFO] ------------------------------------------------------------------------

Any suggestion to work-around it would be really appreciated!

Thanks,

Jian

I could also not cleanly apply surefire-junit4.zip to the current sources.
From a quick look: current ReportEntry has no constructor of the required form.
You may want to test my patches?

Hi Bernd,

> You may want to test my patches?

Sure. Do you mind telling me the instruction how to apply your patches?

Thanks,

Jian

Jian,

as a first step,

please have a look at
http://maven.apache.org/guides/development/guide-m2-development.html
chapter "Creating and submitting a patch"

There is some information about how to apply a patch

Does that help?

Bernd

As you know, testing is an unavoidable part of any software projects to really harness the quality of software products, since JUnit 4.x is out and its being a de facto standard in testing, I think surefire plugin issues should be solved as soon as possible to really support JUnit 4.x using Maven 2.
Thanks.

As near as I can tell, none of the previous patches work:

• surefire-junit4.zip: Does not compile and after spending a bit looking into it, I don't think it can be made to easily.
• SUREFIRE-31-surefire-trunk.patch & SUREFIRE-31-maven-surefire-plugin.patch: These can be made to compile (if you comment out some stuff) but they're basically a hack (no offense intended) and don't seem to report results correctly.

Both of these patches are based on the JUnit3.x provider, which is the wrong approach, I believe. The code is pretty complicated: involves using proxies (dynamically generated classes, I believe) and a pretty poorly-defined interface. I'm sure there was a reason for writing them this way (perhaps was the standard approach for JUnit3 providers) but it isn't necessary for JUnit4 and should be abandoned.

I have created two new patch files which are working perfectly for me. I created the new JUnit4 provider from scratch, (very) loosely based off of the TestNG provider's approach. To apply these patches, check out the maven-surefire-plugin project from https://svn.apache.org/repos/asf/maven/plugins/trunk/maven-surefire-plugin and apply the SUREFIRE31_karl_maven-surefire-plugin.patch file to it (on Windows: right-click the checked-out folder, go to the TortoiseSVN menu, and select Apply Patch). Then, check out the surefire/surefire-providers/surefire-junit project from https://svn.apache.org/repos/asf/maven/surefire/trunk/surefire-providers/surefire-junit and apply the SUREFIRE31_karl_surefire_surefire-providers_surefire-junit.patch file to it. Then, in the root of each project, run "mvn clean install -Dmaven.test.skip=true" to install the surefire updates to your local Maven repository. After that, you should be good to go.

Oh, you'll need to make sure that no older versions of JUnit (3.x) are included in your POM-- even as transitive dependencies. If one of your other dependencies is bringing JUnit 3 along for the ride, add an exclusion to your POM, like I had to do in the following case:
</dependency>
<dependency>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jpox-maven-plugin</artifactId>
<version>1.0.3-SNAPSHOT</version>
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
<scope>provided</scope>
</dependency>
<dependency>

Hope that helps everyone. Please add a comment if you run into any problems with these patches.

Enjoy!

Oh, I should also mention that you will not be able to run both JUnit3 and JUnit4 tests alongside each other with this patch. Two things would need to be done to make that happen:

• First, split the surefire-junit provider into "surefire-junit3" and "surefire-junit4" providers with separate projects, POMs, and (most-importantly) dependencies.
• Then, further work would need to be done to maven-surefire-plugin. The problem right now is that it can only detect ONE version of JUnit at a time, so if both JUnit3 and JUnit4 are present as dependencies, one will end up "blocking" the other. I'm not familiar enough with Maven's project/dependency API right now to fix that.

If one of the regular committers took care of those two things, I think my patches could be included in the next version. In the meantime, they're a better solution for JUnit4 testing with Maven2 than the separate plugin at http://www.unto.net/wiki/Maven_JUnit4_plugin because the results get included in the normal test phase reports.

This patch is an alternative to the other two I posted. Instead of adding JUnit4 support as a new provider, it patches the existing JUnit3 provider so that it now properly supports the "suite()" method.
If you apply this patch and add a "suite()" method to your JUnit4 test classes per http://junit.sourceforge.net/doc/cookbook/cookbook.htm then the JUnit4 tests will be run and reported correctly by Surefire.

Do the snapshots take into account these patches? I'm debating between junit4 & testNG and it looks like I don't have a choice but to use testNG

Hi,

this is in reply to Bernd's comment from 29/Jul/06 01:49 PM:

> The problem is now that surefire's reporting classes expect information
> which is not provided by junit's RunListner. So no good logs/reports
> can be created.

I'm not sure what information surefire's reporting classes need. However
I made an XMLListener extending RunListener for JUnit 4 which produces (almost) the same output
as the maven surefire XML for JUnit 3. Good reports.

wiedkla

Hi, here a sample output:

klaus@jsesoft:_{/projects/junit4/xmllistener> klaus@jsesoft:}/projects/junit4/xmllistener> mvn package
[INFO] Scanning for projects...
[INFO] ----------------------------------------------------------------------------
[INFO] Building JUnit 4 XML Reports
[INFO] task-segment: [package]
[INFO] ----------------------------------------------------------------------------
[INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
Downloading: http://repo1.maven.org/maven2/apache/ant/1.7.ORC1/ant-1.7.ORC1.pom
[WARNING] Unable to get resource from repository central (http://repo1.maven.org/maven2)
[INFO] [compiler:compile]
Compiling 6 source files to /home/klaus/projects/junit4/xmllistener/target/classes
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
Compiling 1 source file to /home/klaus/projects/junit4/xmllistener/target/test-classes
[INFO] [surefire:test]
[INFO] Surefire report directory: /home/klaus/projects/junit4/xmllistener/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.jsesoft.junit4.reports.XMLListenerTest
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite date="Sat Nov 18 16:57:12 CET 2006" failures="1" id="1" name="org.jsesoft.junit4.reports.ReportSuite" skipped="1" tests="2" time="0.133">
<testcase classname="org.jsesoft.junit4.reports.Report" id="1:4711" name="report" state="passed" time="0.013" />
<testcase classname="org.jsesoft.junit4.reports.Report" id="1:4710" message="check-ignore" name="toBeIgnored" state="ignored" time="0.0010" />
<testcase classname="org.jsesoft.junit4.reports.ReportFail" id="1:4712" name="report" state="failed" time="0.018">
<failure message="failing sample" type="report(org.jsesoft.junit4.reports.ReportFail)">java.lang.AssertionError: failing sample
at org.junit.Assert.fail(Assert.java:69)
at org.junit.Assert.assertTrue(Assert.java:32)
at org.jsesoft.junit4.reports.ReportFail.report(ReportFail.java:14)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)
at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)
at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)
at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:71)
at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
at org.junit.internal.runners.CompositeRunner.run(CompositeRunner.java:29)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
at org.junit.internal.runners.CompositeRunner.run(CompositeRunner.java:29)
at org.junit.runner.JUnitCore.run(JUnitCore.java:121)
at org.junit.runner.JUnitCore.run(JUnitCore.java:100)
at org.junit.runner.JUnitCore.run(JUnitCore.java:91)
at org.jsesoft.junit4.reports.XMLListenerTest.testXMLListener(XMLListenerTest.java:20)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:164)
at junit.framework.TestCase.runBare(TestCase.java:130)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:120)
at junit.framework.TestSuite.runTest(TestSuite.java:228)
at junit.framework.TestSuite.run(TestSuite.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:210)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:135)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:122)
at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:225)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:747)
</failure>
</testcase>
<system-out><![CDATA[OOOOOOOOOOOOOO
]]></system-out>
<system-err><![CDATA[EEEEEEEEEEEEE
]]></system-err>
</testsuite>
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.741 sec

Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [jar:jar]
[INFO] Building jar: /home/klaus/projects/junit4/xmllistener/target/xmllistener-1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12 seconds
[INFO] Finished at: Sat Nov 18 16:57:13 CET 2006
[INFO] Final Memory: 4M/8M
[INFO] ------------------------------------------------------------------------
klaus@jsesoft:~/projects/junit4/xmllistener>

Is anything "official" being worked on for this at the moment? It seems to be blocking many people from migrating to JUnit 4. I too am considering using TestNG now instead.

I'm able to build and install Karl's 2nd method....however, I have no idea how to configure maven to use this. I currently have the snapshot surefire installed in my local repository....how do I tell maven to use this rather than the surefire it's bundled with? THANKS!

<<how do I tell maven to use this rather than the surefire it's bundled with?>>

I can't say for certain but I think Maven will try to use the snapshot version in your local repository automatically. It's been a bit since I had to install this myself, so my memory's a bit foggy. If I had my project compiling right now I'd test that for you... but I don't.

Give it a try and let us know if it works just having it installed. If not, I'll get my stuff compiling again on Monday and give you instructions on how to get Maven to use the surefire snapshot.

Best of luck!

Karl,

You're correct in that it will pick it up appropriately, but I did have to rebuild the maven-surefire-plugin identifying the snapshot release as the dependency in order for it to work.

This works great, thanks for the patch!

I took Karls code and added it to the current surefire trunk. I moved the junit4 code to its own surefire-junit4 provider.
SurefirePlugin.java is patched to use the new provider if a junit dependency with baseVersion.startsWith("4") is found.
Since junit4 is backward compatible, both new-style (@Test) and old-style (extends TestCase) unit tests can be used in the same project.

The old junit3 provider is not touched and it is still the default.

Patch attached...

patch to the surefire trunk
provides surefire-junit4 provider + patch for SurefirePlugin.java

previous patch was missing a pom...

Patches applied. Final form taken from Tom.

Thanks.

Is it possible to run execute testNG, junit4, and junit3 tests together in the same project?