Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-157

Surefire Plugin fails to handle exception thrown from TestNG @BeforeTest method

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0 (2.2 plugin), 2.3
    • Fix Version/s: 2.4
    • Component/s: TestNG support
    • Labels:
      None
    • Environment:
      Windows XP, JDK 1.5, Maven 2.0.4, TestNG 5.1
    • Number of attachments :
      0

      Description

      Create a TestNG test with a method as follows:

      @BeforeTest
      public void beforeTest() {
      throw new RuntimeException("Simulate an exception from a beforeTest method");
      }

      When surefire attempts to run this test, the plugin fails with the following stack trace:

      org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: n
      ull
      java.lang.NullPointerException
      at org.apache.maven.surefire.report.AbstractTextReporter.testFailed(AbstractTextReporter.java:106)
      at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:299)
      at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:281)
      at org.apache.maven.surefire.testng.TestNGReporter.onTestFailure(TestNGReporter.java:97)
      at org.testng.internal.Invoker.runTestListeners(Invoker.java:1164)
      at org.testng.internal.Invoker.runTestListeners(Invoker.java:1149)
      at org.testng.internal.Invoker.handleConfigurationFailure(Invoker.java:191)
      at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:170)
      at org.testng.SuiteRunner.privateRun(SuiteRunner.java:236)
      at org.testng.SuiteRunner.run(SuiteRunner.java:145)
      at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:901)
      at org.testng.TestNG.runSuitesLocally(TestNG.java:863)
      at org.apache.maven.surefire.testng.TestNGExecutor.executeTestNG(TestNGExecutor.java:64)
      at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75)
      at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:225)
      at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:747)

        Activity

        Hide
        Gerhard Langs added a comment -

        Re-Implemented the TestNGReporter method below based on the trunk source code (not submitted to svn). Should now report Before* Errors:

        public void onFinish(ITestContext context) {
        boolean failureSeen = false;
        IResultMap F_rmap = context.getFailedConfigurations();
        Iterator F_rset = F_rmap.getAllResults().iterator();
        while (F_rset.hasNext())

        { ITestResult F_tres = (ITestResult) F_rset.next(); onTestFailure(F_tres); failureSeen = true; }

        String rawString = bundle.getString( failureSeen ? "executeException" : "testSetCompletedNormally" );

        ReportEntry report = new ReportEntry(source, context.getName(),
        groupString(context.getIncludedGroups(), null), rawString);

        reportManager.testSetCompleted(report);

        reportManager.reset();
        }

        Show
        Gerhard Langs added a comment - Re-Implemented the TestNGReporter method below based on the trunk source code (not submitted to svn). Should now report Before* Errors: public void onFinish(ITestContext context) { boolean failureSeen = false; IResultMap F_rmap = context.getFailedConfigurations(); Iterator F_rset = F_rmap.getAllResults().iterator(); while (F_rset.hasNext()) { ITestResult F_tres = (ITestResult) F_rset.next(); onTestFailure(F_tres); failureSeen = true; } String rawString = bundle.getString( failureSeen ? "executeException" : "testSetCompletedNormally" ); ReportEntry report = new ReportEntry(source, context.getName(), groupString(context.getIncludedGroups(), null), rawString); reportManager.testSetCompleted(report); reportManager.reset(); }
        Hide
        Dan Fabulich added a comment -

        This must have been fixed at some point in 2.4-SNAPSHOT, because it doesn't appear to happen on the latest trunk. Note that I do see SUREFIRE-379 (When an exception occurs in @BeforeMethod, the exception is not recorded) which is bad, but considerably less catastrophic in my opinion.

        Show
        Dan Fabulich added a comment - This must have been fixed at some point in 2.4-SNAPSHOT, because it doesn't appear to happen on the latest trunk. Note that I do see SUREFIRE-379 (When an exception occurs in @BeforeMethod, the exception is not recorded) which is bad, but considerably less catastrophic in my opinion.
        Hide
        Roger Pack added a comment -

        I think I get this with junit [and surefire] as well.
        If a runtime exception is thrown in an @BeforeClass method, maven terminates with "go and see the log generated" message, however the generated log files are blank.
        Any help on this?

        Show
        Roger Pack added a comment - I think I get this with junit [and surefire] as well. If a runtime exception is thrown in an @BeforeClass method, maven terminates with "go and see the log generated" message, however the generated log files are blank. Any help on this?
        Hide
        Kaspar Fischer added a comment -

        I am experiencing the same problem:

        org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: null
        java.lang.NullPointerException
        at org.apache.maven.surefire.report.AbstractTextReporter.testFailed(AbstractTextReporter.java:111)
        at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:335)
        at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:317)
        at org.apache.maven.surefire.testng.TestNGReporter.onTestFailure(TestNGReporter.java:102)
        at org.testng.internal.Invoker.runTestListeners(Invoker.java:1433)
        at org.testng.internal.Invoker.runTestListeners(Invoker.java:1417)
        at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1152)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:887)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
        at org.testng.TestRunner.runWorkers(TestRunner.java:689)
        at org.testng.TestRunner.privateRun(TestRunner.java:566)
        at org.testng.TestRunner.run(TestRunner.java:466)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:301)
        at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:296)
        at org.testng.SuiteRunner.privateRun(SuiteRunner.java:276)
        at org.testng.SuiteRunner.run(SuiteRunner.java:191)
        at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:808)
        at org.testng.TestNG.runSuitesLocally(TestNG.java:776)
        at org.testng.TestNG.run(TestNG.java:701)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:64)
        at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
        at org.apache.maven.surefire.Surefire.run(Surefire.java:107)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:285)
        at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:976)

        According to "mvn -o help:effective-pom" I am using

        <pluginManagement>
        <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.4.2</version>
        </plugin>
        </pluginManagement>
        <plugins>
        <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.4.3</version>
        <configuration>
        <forkMode>always</forkMode>
        <excludeGroups>extended</excludeGroups>
        <argLine>-javaagent:$

        {settings.localRepository}

        /salve/salve-agent/2.0-beta1/salve-agent-2.0-beta1.jar</argLine>
        <useSystemClassloader>true</useSystemClassloader>
        </configuration>
        </plugin>
        </plugins>

        and

        <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>5.7</version>
        <classifier>jdk15</classifier>
        </dependency>

        I had a look at the source, line 111 on http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup ...

        I also found

        http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html

        and changed the above configuration to

        <useSystemClassloader>false</useSystemClassloader>

        This didn't solved the issue for me.

        Any ideas?

        Show
        Kaspar Fischer added a comment - I am experiencing the same problem: org.apache.maven.surefire.booter.SurefireExecutionException: null; nested exception is java.lang.NullPointerException: null java.lang.NullPointerException at org.apache.maven.surefire.report.AbstractTextReporter.testFailed(AbstractTextReporter.java:111) at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:335) at org.apache.maven.surefire.report.ReporterManager.testFailed(ReporterManager.java:317) at org.apache.maven.surefire.testng.TestNGReporter.onTestFailure(TestNGReporter.java:102) at org.testng.internal.Invoker.runTestListeners(Invoker.java:1433) at org.testng.internal.Invoker.runTestListeners(Invoker.java:1417) at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1152) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:887) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:689) at org.testng.TestRunner.privateRun(TestRunner.java:566) at org.testng.TestRunner.run(TestRunner.java:466) at org.testng.SuiteRunner.runTest(SuiteRunner.java:301) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:296) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:276) at org.testng.SuiteRunner.run(SuiteRunner.java:191) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:808) at org.testng.TestNG.runSuitesLocally(TestNG.java:776) at org.testng.TestNG.run(TestNG.java:701) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:64) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99) at org.apache.maven.surefire.Surefire.run(Surefire.java:107) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:285) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:976) According to "mvn -o help:effective-pom" I am using <pluginManagement> <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.4.2</version> </plugin> </pluginManagement> <plugins> <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.4.3</version> <configuration> <forkMode>always</forkMode> <excludeGroups>extended</excludeGroups> <argLine>-javaagent:$ {settings.localRepository} /salve/salve-agent/2.0-beta1/salve-agent-2.0-beta1.jar</argLine> <useSystemClassloader>true</useSystemClassloader> </configuration> </plugin> </plugins> and <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>5.7</version> <classifier>jdk15</classifier> </dependency> I had a look at the source, line 111 on http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup ... I also found http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html and changed the above configuration to <useSystemClassloader>false</useSystemClassloader> This didn't solved the issue for me. Any ideas?
        Hide
        Kaspar Fischer added a comment -

        ... I should add that I am using TestNG's @BeforeClass annotation. Looking at the code,

        http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup

        it seems that the List testResults is not initialized, which can only happen, it seems, if testSetStarting() is not called. Could this be connected to @BeforeClass?

        Show
        Kaspar Fischer added a comment - ... I should add that I am using TestNG's @BeforeClass annotation. Looking at the code, http://svn.apache.org/viewvc/maven/surefire/tags/surefire-2.4.3/surefire-api/src/main/java/org/apache/maven/surefire/report/AbstractTextReporter.java?view=markup it seems that the List testResults is not initialized, which can only happen, it seems, if testSetStarting() is not called. Could this be connected to @BeforeClass?

          People

          • Assignee:
            Unassigned
            Reporter:
            Manish Shah
          • Votes:
            4 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: