testSetCompleted called before testSetStarting when using m3 parallel builds

Are you setting the "forkTimeout" parameter in your surefire settings ? We have identified a race condition that could occur if the forked process times out while still outputting to stdout/stderr. Just setting "forkTimeout" is actually enough to trigger
this, it would happen even with "forkMode=never" as long as forkTimeout is set.

Actually the issue would probably happen quite frequently as long as the process outputs to stderr/stdout at the exact time it times out with "forkTimeout".

yes I am. more specifically forkedProcessTimeoutInSeconds.

And can you confirm that either
A) The exception always happens when the tests were supposed to time out
B) Increasing the timeout seems to help (Say doubling its current value)

For now I'm testing with the value off but I can try it by using a lower value than I know a verbose (lots of output) test takes.

Plexus utils 2.0.7 will be appearing in maven central in a few hours. Adding this to surefire should solve this issue;
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
....
<dependencies>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>2.0.7</version>
</dependency>
</dependencies>
</plugin>

Please report back on the effects of this.

I got another one today after setting timeout to 0.
I haven't verified yet that it was running with no timeout settings but it was the first one since setting it.
So it definitely seems to be related as it happened way more frequently before.
I will try it again now with the new plexus-utils but I can't guarantee that I will know the results anytime soon as this is intermittent and I still haven't had time to come up with a test case.

Hey Kristian, Sry I think I misunderstood you earlier.
I added the plexus-utils to surefire 2.7.1 not 2.7.2 and got this error again today.
Did you mean that this also needed 2.7.2 to work correctly?
Unfortunately I can't switch to 2.8-SNAPSHOT just yet as I am locked to using release only versions ATM.

2.7.2 (released) with the addition of plexus-utils 2.0.7 (released) should be identical to 2.8-.SNAPSHOT wrt this error.

I will be releasing 2.8 within the next 48 hours, so I think we can just defer this until 2.8 is released and re-test then.

Sounds good to me

I got this again in 2.8.
I should note, however, that I also turned the timeout back on when I updated to 2.8.
Do you need a stack trace? I can probably get one pretty quickly with the frequency of this bug

[18:16:29]: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.8:test (default-test) on project chameleon-main: Error while executing forked tests.; nested exception is java.lang.IllegalStateException: testSetStarting called twice -> [Help 1]
[18:16:29]: [ERROR] 
[18:16:29]: [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[18:16:29]: [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[18:16:29]: [ERROR] 
[18:16:29]: [ERROR] For more information about the errors and possible solutions, please read the following articles:
[18:16:29]: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[18:16:29]: [ERROR] 
[18:16:29]: [ERROR] After correcting the problems, you can resume the build with the command
[18:16:29]: [ERROR]   mvn <goals> -rf :chameleon-main

The issue you're seeing now is actually a design problem with parallel surefire providers and redirectTestOuputToFile=true, because you're running parallel junit (and parallel m3, which is not relevant). (All the others issues we've been having with 2.7.X and redirectTestOuputToFile= true have been just basic thread safety issues that affect all users of redirectTestOuputToFile).

Until we fixed SUREFIRE-700 there was no way to fix this issue, but now that this is done it should be much simpler. I am working on this for 2.8.1. Until then, the only way to make this reliable is either to turn off parallel junit or redirectTestOuputToFile.

Actually Kristian this is only running parallel m3 (-T 1C) at the moment. redirectTestOutputToFile may be on but definitely not parallel junit.

As of SUREFIRE-732 with the followup in r1100808 this whole logic was simplified to the extent that this problem cannot happen any more. Revised 2.8.2-SNAPSHOT uploaded to r.a.o.