Maven Surefire

[regression] TestNG suite no longer succeeds, with error "testSetStarting called twice"

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.7.1
  • Fix Version/s: 2.7.2
  • Component/s: TestNG support
  • Labels:
    None
  • Complexity:
    Intermediate
  • Number of attachments :
    0

Description

I'm receiving this error in 2.7.1:
"Error while executing forked tests.; nested exception is java.lang.IllegalStateException: testSetStarting called twice"

This did not occur with 2.5 on the same project. It is using TestNG with a suite XML and Cargo + Selenium

You should be able to reproduce it by checking out http://svn.apache.org/repos/asf/archiva/trunk, changing the surefire version in the root POM, and running:
mvn clean install
cd archiva-modules/archiva-web/archiva-webapp-test
mvn clean install

I'll have to come back to investigate the cause later - the project is quite crusty and there may be some misconfiguration also.

Issue Links

relates to

Bug - A problem which impairs or prevents the functions of the product. SUREFIRE-680 Printout Running TestSuite twice times

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Brett Porter added a comment -

I should point out that this was at a point in time where initializeArchiva(org.apache.archiva.web.test.ArchivaAdminTest) was failing (causing all other tests to be skipped). I'll re-verify whether 2.7.1 works once the test suite is passing correctly.

Show
Brett Porter added a comment - I should point out that this was at a point in time where initializeArchiva(org.apache.archiva.web.test.ArchivaAdminTest) was failing (causing all other tests to be skipped). I'll re-verify whether 2.7.1 works once the test suite is passing correctly.
Hide
Permalink
Brett Porter added a comment - - edited

after repairing the startup problems, the test suite progresses as normal. However, at the end the same error appears, so it seems it's not related to the initializeArchiva method.

I can see if I can come up with a test case soon.

Show
Brett Porter added a comment - - edited after repairing the startup problems, the test suite progresses as normal. However, at the end the same error appears, so it seems it's not related to the initializeArchiva method. I can see if I can come up with a test case soon.
Hide
Permalink
Brett Porter added a comment -

further data - the output is mixed up (seems related to SUREFIRE-639).

In TestSuite.txt:


-------------------------------------------------------------------------------
Test set: TestSuite
-------------------------------------------------------------------------------

Results :

Failed tests: 
  testBuildNumberOfSnapshotArtifact(org.apache.archiva.web.test.BrowseTest)
  testDeleteMetadataProperty(org.apache.archiva.web.test.BrowseTest)
  testMerging(org.apache.archiva.web.test.MergingRepositoriesTest)

Tests run: 123, Failures: 3, Errors: 0, Skipped: 4

ert.java:84)
        at org.testng.Assert.failNotEquals(Assert.java:438)
        at org.testng.Assert.assertTrue(Assert.java:32)
        at org.testng.Assert.assertTrue(Assert.java:42)
        at org.apache.archiva.web.test.BrowseTest.testBuildNumberOfSnapshotArtifact(BrowseTest.java:137)

testDeleteMetadataProperty(org.apache.archiva.web.test.BrowseTest)  Time elapsed: 1.407 sec  <<< FAILURE!
java.lang.AssertionError: 'foo=bar' isn't present. expected:<true> but was:<false>
        at org.testng.Assert.fail(Assert.java:84)
        at org.testng.Assert.failNotEquals(Assert.java:438)
        at org.testng.Assert.assertTrue(Assert.java:32)
        at org.apache.archiva.web.test.parent.AbstractSeleniumTest.assertTextPresent(AbstractSeleniumTest.java:163)
        at org.apache.archiva.web.test.BrowseTest.deleteMetadataProperty(BrowseTest.java:229)
        at org.apache.archiva.web.test.BrowseTest.testDeleteMetadataProperty(BrowseTest.java:170)

testMerging(org.apache.archiva.web.test.MergingRepositoriesTest)  Time elapsed: 3.231 sec  <<< FAILURE!
java.lang.AssertionError: 'No conflicting artifacts' isn't present. expected:<true> but was:<false>
        at org.testng.Assert.fail(Assert.java:84)
        at org.testng.Assert.failNotEquals(Assert.java:438)
        at org.testng.Assert.assertTrue(Assert.java:32)
        at org.apache.archiva.web.test.parent.AbstractSeleniumTest.assertTextPresent(AbstractSeleniumTest.java:163)
        at org.apache.archiva.web.test.MergingRepositoriesTest.testMerging(MergingRepositoriesTest.java:55)

while the output was:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error while executing forked tests.; nested exception is java.lang.IllegalStateException: testSetStarting called twice

Note the partial exception and the overwritten summary in the wrong file.

Show
Brett Porter added a comment - further data - the output is mixed up (seems related to SUREFIRE-639 ). In TestSuite.txt: ------------------------------------------------------------------------------- Test set: TestSuite ------------------------------------------------------------------------------- Results : Failed tests: testBuildNumberOfSnapshotArtifact(org.apache.archiva.web.test.BrowseTest) testDeleteMetadataProperty(org.apache.archiva.web.test.BrowseTest) testMerging(org.apache.archiva.web.test.MergingRepositoriesTest) Tests run: 123, Failures: 3, Errors: 0, Skipped: 4 ert.java:84) at org.testng.Assert.failNotEquals(Assert.java:438) at org.testng.Assert.assertTrue(Assert.java:32) at org.testng.Assert.assertTrue(Assert.java:42) at org.apache.archiva.web.test.BrowseTest.testBuildNumberOfSnapshotArtifact(BrowseTest.java:137) testDeleteMetadataProperty(org.apache.archiva.web.test.BrowseTest) Time elapsed: 1.407 sec <<< FAILURE! java.lang.AssertionError: 'foo=bar' isn't present. expected:< true > but was:< false > at org.testng.Assert.fail(Assert.java:84) at org.testng.Assert.failNotEquals(Assert.java:438) at org.testng.Assert.assertTrue(Assert.java:32) at org.apache.archiva.web.test.parent.AbstractSeleniumTest.assertTextPresent(AbstractSeleniumTest.java:163) at org.apache.archiva.web.test.BrowseTest.deleteMetadataProperty(BrowseTest.java:229) at org.apache.archiva.web.test.BrowseTest.testDeleteMetadataProperty(BrowseTest.java:170) testMerging(org.apache.archiva.web.test.MergingRepositoriesTest) Time elapsed: 3.231 sec <<< FAILURE! java.lang.AssertionError: 'No conflicting artifacts' isn't present. expected:< true > but was:< false > at org.testng.Assert.fail(Assert.java:84) at org.testng.Assert.failNotEquals(Assert.java:438) at org.testng.Assert.assertTrue(Assert.java:32) at org.apache.archiva.web.test.parent.AbstractSeleniumTest.assertTextPresent(AbstractSeleniumTest.java:163) at org.apache.archiva.web.test.MergingRepositoriesTest.testMerging(MergingRepositoriesTest.java:55) while the output was: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running TestSuite [INFO] ------------------------------------------------------------------------ [ERROR] BUILD ERROR [INFO] ------------------------------------------------------------------------ [INFO] Error while executing forked tests.; nested exception is java.lang.IllegalStateException: testSetStarting called twice Note the partial exception and the overwritten summary in the wrong file.
Hide
Permalink
Kristian Rosenvold added a comment -

Fixed with testcase in r1054391. Bug required TestNG AND redirectTestOutputToFile=true AND non-suite setup (>1 test). It would also cause "Test Set Starting" to be printed twice. It is not related to SUREFIRE-639.

I am getting initializeArchiva(org.apache.archiva.web.test.ArchivaAdminTest) failing now, which I suspect is totally different. You may want to re-test your use-case.

Show
Kristian Rosenvold added a comment - Fixed with testcase in r1054391. Bug required TestNG AND redirectTestOutputToFile=true AND non-suite setup (>1 test). It would also cause "Test Set Starting" to be printed twice. It is not related to SUREFIRE-639 . I am getting initializeArchiva(org.apache.archiva.web.test.ArchivaAdminTest) failing now, which I suspect is totally different. You may want to re-test your use-case.

People

  • Assignee:
    Kristian Rosenvold
    Reporter:
    Brett Porter
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: