Surefire Plugin fails to handle exception thrown from TestNG @BeforeTest method

The same bug exists with @BeforeSuite

It seems that the TestNGReporter class does not invoke the right methods in Reporter in some cases. For example the onStart(ISuite suite) method is a NO-OP and this triggers the bug described here with @BeforeSuite.

I'm having the same issue, has this ever been resolved?

I still have the exact same issue.

This issue also exists for methods annotated with @BeforeSuite.

Re-Implemented the TestNGReporter method below based on the trunk source code (not submitted to svn). Should now report Before* Errors:

public void onFinish(ITestContext context) {
boolean failureSeen = false;
IResultMap F_rmap = context.getFailedConfigurations();
Iterator F_rset = F_rmap.getAllResults().iterator();
while (F_rset.hasNext())

String rawString = bundle.getString( failureSeen ? "executeException" : "testSetCompletedNormally" );

ReportEntry report = new ReportEntry(source, context.getName(),
groupString(context.getIncludedGroups(), null), rawString);

reportManager.testSetCompleted(report);

reportManager.reset();
}

Re-Implemented the TestNGReporter method below based on the trunk source code (not submitted to svn). Should now report Before* Errors:

public void onFinish(ITestContext context) {
boolean failureSeen = false;
IResultMap F_rmap = context.getFailedConfigurations();
Iterator F_rset = F_rmap.getAllResults().iterator();
while (F_rset.hasNext())

String rawString = bundle.getString( failureSeen ? "executeException" : "testSetCompletedNormally" );

ReportEntry report = new ReportEntry(source, context.getName(),
groupString(context.getIncludedGroups(), null), rawString);

reportManager.testSetCompleted(report);

reportManager.reset();
}

This must have been fixed at some point in 2.4-SNAPSHOT, because it doesn't appear to happen on the latest trunk. Note that I do see SUREFIRE-379 (When an exception occurs in @BeforeMethod, the exception is not recorded) which is bad, but considerably less catastrophic in my opinion.

I think I get this with junit [and surefire] as well.
If a runtime exception is thrown in an @BeforeClass method, maven terminates with "go and see the log generated" message, however the generated log files are blank.
Any help on this?

I am experiencing the same problem:

org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: null
java.lang.NullPointerException
at org.apache.maven.surefire.report.AbstractTextReporter.testFailed(AbstractTextReporter.java:111)
at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:335)
at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:317)
at org.apache.maven.surefire.testng.TestNGReporter.onTestFailure(TestNGReporter.java:102)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1433)
at org.testng.internal.Invoker.runTestListeners(Invoker.java:1417)
at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1152)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:887)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.runWorkers(TestRunner.java:689)
at org.testng.TestRunner.privateRun(TestRunner.java:566)
at org.testng.TestRunner.run(TestRunner.java:466)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:301)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:296)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:276)
at org.testng.SuiteRunner.run(SuiteRunner.java:191)
at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:808)
at org.testng.TestNG.runSuitesLocally(TestNG.java:776)
at org.testng.TestNG.run(TestNG.java:701)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:64)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
at org.apache.maven.surefire.Surefire.run(Surefire.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:285)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:976)

According to "mvn -o help:effective-pom" I am using

<pluginManagement>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
</plugin>
</pluginManagement>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<forkMode>always</forkMode>
<excludeGroups>extended</excludeGroups>
<argLine>-javaagent:$

/salve/salve-agent/2.0-beta1/salve-agent-2.0-beta1.jar</argLine>
<useSystemClassloader>true</useSystemClassloader>
</configuration>
</plugin>
</plugins>

and

<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>5.7</version>
<classifier>jdk15</classifier>
</dependency>

I had a look at the source, line 111 on http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup ...

I also found

http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html

and changed the above configuration to

<useSystemClassloader>false</useSystemClassloader>

This didn't solved the issue for me.

Any ideas?

... I should add that I am using TestNG's @BeforeClass annotation. Looking at the code,

http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup

it seems that the List testResults is not initialized, which can only happen, it seems, if testSetStarting() is not called. Could this be connected to @BeforeClass?