Thank you Alex for opening this issue. The exception stack trace I sent you is actually not the one matching the screenshots in the attachment (made a copy/paste mistake from my test log).
The actual stack trace is:

java.lang.AssertionError: [src/test/resources/screenshots/java6/basics/Basics1SimpleGrid.png] images do not have the same color(s)
at org.fest.assertions.Fail.fail(Fail.java:85)
at org.fest.assertions.GenericAssert.fail(GenericAssert.java:211)
at org.fest.assertions.ImageAssert.isEqualTo(ImageAssert.java:166)
at net.java.dev.designgridlayout.AbstractGuiTest.checkSnapshot(AbstractGuiTest.java:167)
at net.java.dev.designgridlayout.AbstractGuiTest.checkSnapshot(AbstractGuiTest.java:143)
at net.java.dev.designgridlayout.AbstractGuiTest.checkExample(AbstractGuiTest.java:109)
at net.java.dev.designgridlayout.basics.BasicFeaturesTest.checkSimpleGrid(BasicFeaturesTest.java:33)
....

It is essentially the same as the original one except that it was run from another test case and the screenshot reference is correct that time.

After a quick look at the source of ImageAssert, I could not find any obvious hint that would explain this behavior.
It might be interesting to run the TC in debug mode and check how the 2 BufferedImages are different (I suspect their ColorModels might differ but there is no certitude).
By the way it might be useful that the ImageAssert.isEqualTo() creates an exception with a more detailed message, in particular which pixel (x,y) was found different, what was the exepected and the actual colors?

I'll see tonight if I can try to check my tests in debug mode (I never run them from eclipse normally, only from maven...)

My first comments on trying to debug this issue.
I confirm that both BufferedImage have a different ColorModel. A few details about respective properties for both:

ACTUAL SCREENSHOT (XP, from Robot)

• image type TYPE_INT_RGB
• color model DirectColorModel
• raster IntegerInterleavedRaster

EXPECTED SCREENSHOT (VISTA64, from file)

• image type TYPE_CUSTOM
• color model ComponentColorModel
• raster ByteInterleavedRaster

while debugging through ImageAssert.hasEqualColor() I was expecting a failure on the first pixel but no! There was no error in the first pixels line!
I didn't have the patience to step through each of the 115x173 pixels checks!
Thanks to Eclipse conditional breakpoints, I found out that the first different pixel is at (x,y)=(13,15). The Color values returned by getRGB() were:

actual = -1314056 (0xFFEBF2F8)
expected = -1248520 (0xFFECF2F8)

hence, a 1-bit difference in the Red component (EB Vs. EC)!

Now what I don't know is if this difference was due to internal conversions performed by getRGB() or if the 2 PNG are really different!

I need to find some way to investigate that point further...

Bad news! After checking both screenshots files with MS Digital Image, and viewing the pixel color they effectively differ!
This means the problem is probably not in ImageAssert, but either in:

• ScreenshotTaker.takeScreenshotOf()
• or ScreenshotTaker.saveImage()

Hard to tell, I feel a bit lost now, don't really know where to look (and how) to fix that problem...

Anyway, I think this issue should at least enhance the exception message so that we see immediately which pixel differs and the respective colors for expected and actual pixels.

In addition, another enhancement could also be to have some threshold in pixel color comparison, passed as an argument to ImageAssert (default = 0).

for each actual pixel
for each of R, G, B value
if abs(expected - actual) > threshold then fail

that looks more like a workaround, but I would already be pleased with it Hopefully it would solve all my problems of screenshot checks between Vista and XP (I wonder if ImageIO or Robot may be different between both platforms).

One next check I will do is to compare, on Vista, the actual screenshot (BufferedImage taken by Robot) with the saved PNG file. This way I would know if the problem is in Robot or ImageIO...

I'll check that later on tonight and post my conclusions.

Thanks Jean-Francois for looking into this bug in so much detail!

Like you said, it is kind of hard to fix. If the bug is in either takeScreenshotOf or in saveImage, the bug will be actually in the JDK, since FEST simply delegates to methods in the JDK that perform those actions. My guess is that there are actual unexpected differences in JDK implementations for different OSs.

Both your suggestions are excellent! (as usual.) I'll be adding more details to the exception message and a threshold for comparison.

Thanks,
-Alex

Hi Jean-Francois,

I just added the changes you suggested and it works perfectly for the images you sent me. The code in the test org.fest.assertions.FEST107_EqualImagesAreConsideredDifferent looks like this:

BufferedImage actual = reader.read(file("Basics1SimpleGridActual.png"));
    BufferedImage expected = reader.read(file("Basics1SimpleGridExpected.png"));
    assertThat(actual).isEqualTo(expected, threshold(1));

This is a lot better than what we previously had!

Thanks a lot!
-Alex

Thanks Alex for this prompt fix.
Is there any way I can get it (the easiest way for me would be a maven snapshot)?

I think you are right about the fact we won't be able to solve the problems of different screenshots between XP and Vista; maybe I won't waste time to try to spot the origin, Robot or ImageIO, since anyway this would not change the fact we cannot do anything!

Cheers

Jean-Francois

Hi Jean-Francois,

I'll be uploading snapshots to our new, shiny Maven repo tonight (I'll out for the whole day)...Once it's done, I'll send you instructions on how to access the new repo

Thanks!
-Alex

Hi Jean-Francois,

I just uploaded a snapshot release for FEST-Swing (which will use a snapshot release of FEST-Assert.)

The new Maven repo can be found http://svn.codehaus.org/fest/m2/repo/ (both work fine). The groupId has changed from from 'fest' to 'org.easytesting' (to follow the rules of the central Maven repo and eventually upload FEST there.)

The version that contains fixes for the bugs you filed is fest-swing-1.2a1-SNAPSHOT. Please let me know if this is working OK

Thanks,
-Alex

Hello Alex,

thanks for uploading the snapshot.

Unfortunately on the new maven repo, you didn't upload fest-swing-testng jar (all my tests are based on TestNG) and that makes it impossible for me to test the latest fest-swing snapshot (I cannot even try to tune maven artifacts loading to get both jars from the old and new maven repos because the group names have changed so the "old" fest-swing-testng would not find the new fest-swing.

You also mentioned uploading a snapshot fest-swing release elsewhere but I could not find it in any location other than the new maven repo. Is there a new download location for FEST? Is it normal that the docs on codehaus still refer to locations on code.google?

Sorry to bother you a bit more with that but it would be hard for me to revert from TestNG (which I like much) to JUnit 4 (which I still don't appreciate much).

Hope I can test the whole stuff tonight and report my results here.

Cheers,

Jean-Francois

Hello Alex,

now everything is fine with the snapshot in maven repository. All my tests run well now on both XP and VISTA.
Thanks a lot!

Jean-Francois

Thanks a lot Jean-Francois! I'm glad both FEST-106 and FEST-107 are fixed! Thank you so much for the time and effort you spent making sure this is fixed. I truly appreciate it

Best regards,
-Alex

P.S. I'm planning to have a full (non-SNAPSHOT) release by the end of April