jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
  • FEST
  • FEST-216

Make the GUITestRunner take screenshots before the "After" methods are executed and make the directory configurable and permanent

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: Improvement Improvement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: FEST-Swing 1.2a2
  • Fix Version/s: FEST-Swing 1.2rc
  • Component/s: Swing
  • Labels:
    None

Description

We are trying to use the GUITestRunner in order to further track down test failures. We observed however two problems with this approach, that we solved by creating our own test runner. I wonder however, if parts of this should not be integrated into the shipped version, because others may observe the same problems

  1. Running GUI test suites with ant and forkmode true:
    When invoking a bigger test suite with fork mode set to true, each test class is executed in its own jvm. This causes however the GUITestRunner to clean sweap the image folder. Therefore, only the last test in the suite has a chance to show a failure screenshot. We solved this by parametrizing the image directory and delegating the cleanup to the actual ant task.
  1. Take the screenshot before the "After" method is run
    Most of our tests create the GUI component under test in the Before method, perform the actual test in the Test method and dispose the component in the After method. Since the screenshot is however taken after the last After method has been executed, our screenshots usually show an empty desktop

We solved this by creating our own test runner that actually wraps the bare test methods, i. e.

@Override
	protected List<FrameworkMethod> getChildren() {
		List<FrameworkMethod> testMethods = super.getChildren();
		List<FrameworkMethod> wrappedMethods = new ArrayList<FrameworkMethod>();
		for (final FrameworkMethod method : testMethods) {
			wrappedMethods.add(new FrameworkMethod(method.getMethod()) {
				@Override
				public Object invokeExplosively(Object target, Object... params) throws Throwable {
					try {
						return super.invokeExplosively(target, params);
					} catch (Throwable e) {
						takeScreenshot(method);
						throw e;
					}
				}
			});
		}
		return wrappedMethods;
	}

There is no need to override the run method any more.

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Stephen Gade Esven added a comment - 02/Mar/10 7:26 AM

I too find the need to take screen shots after the @Test method but before the @After methods.
But I believe that a cleaner approach is to override the methodBlock to insert Statements between withBefores and withAfters, that take the screen shots on failures.

Show
Stephen Gade Esven added a comment - 02/Mar/10 7:26 AM I too find the need to take screen shots after the @Test method but before the @After methods. But I believe that a cleaner approach is to override the methodBlock to insert Statements between withBefores and withAfters, that take the screen shots on failures.
Hide
Permalink
Bernd Kappler added a comment - 02/Mar/10 8:26 AM

Dear Stephen,

unfortunately I am quite new to junit 4 internal api ... junit 3 was actually a lot easier to understand.
I tried the approach that you described, but had a difficult time to find out, if the actual test method failed, or if its execution was successful. It might be the case that junit 4 allows you to add some convenient notifiers and make the screen shot taking conditional on this...but I simply could not find a good way in the limited time I had.
I completely agree with you that wrapping the test methods looks "hacky".

Bernd

Show
Bernd Kappler added a comment - 02/Mar/10 8:26 AM Dear Stephen, unfortunately I am quite new to junit 4 internal api ... junit 3 was actually a lot easier to understand. I tried the approach that you described, but had a difficult time to find out, if the actual test method failed, or if its execution was successful. It might be the case that junit 4 allows you to add some convenient notifiers and make the screen shot taking conditional on this...but I simply could not find a good way in the limited time I had. I completely agree with you that wrapping the test methods looks "hacky". Bernd
Hide
Permalink
Alex Ruiz added a comment - 11/Mar/10 7:50 AM

Hi Bernd,

I'm going to start working on this issue, I apologize for taking so long (almost a year!)

I have a question though. The GUITestRunner you refer to, is the one for JUnit 4.3.1 or 4.5, we have two and I couldn't figure out if we had both by the time you submitted this bug.

Thanks!
-Alex

Show
Alex Ruiz added a comment - 11/Mar/10 7:50 AM Hi Bernd, I'm going to start working on this issue, I apologize for taking so long (almost a year!) I have a question though. The GUITestRunner you refer to, is the one for JUnit 4.3.1 or 4.5, we have two and I couldn't figure out if we had both by the time you submitted this bug. Thanks! -Alex
Hide
Permalink
Bernd Kappler added a comment - 11/Mar/10 10:15 AM

Hi Alex,

this refers to the GUITestRunner for junit 4.5

Thanks

Bernd

Show
Bernd Kappler added a comment - 11/Mar/10 10:15 AM Hi Alex, this refers to the GUITestRunner for junit 4.5 Thanks Bernd
Hide
Permalink
Alex Ruiz added a comment - 11/Mar/10 9:03 PM

Hi Bernd,

I think I fixed it

I uploaded the latest version (1.2b1-SNAPSHOT) to our Maven repo. Can you please give it a try?

Thanks!
-Alex

Show
Alex Ruiz added a comment - 11/Mar/10 9:03 PM Hi Bernd, I think I fixed it I uploaded the latest version (1.2b1-SNAPSHOT) to our Maven repo. Can you please give it a try? Thanks! -Alex
Hide
Permalink
Brett Vasconcellos added a comment - 04/Jun/10 3:23 PM

Alex, We have this same problem, running Junit 4.7 and Fest 1.2rc. My guess is this can't be fixed without replacing Junit's default test runner with a custom one as Bernd did.

Another workaround is to eliminate your "After" method and do the cleanup (dispose) in the Before method for the previous test, and the AfterStatic for the last test in a class. This works well for us.

Show
Brett Vasconcellos added a comment - 04/Jun/10 3:23 PM Alex, We have this same problem, running Junit 4.7 and Fest 1.2rc. My guess is this can't be fixed without replacing Junit's default test runner with a custom one as Bernd did. Another workaround is to eliminate your "After" method and do the cleanup (dispose) in the Before method for the previous test, and the AfterStatic for the last test in a class. This works well for us.
Hide
Permalink
Alex Ruiz added a comment - 04/Jun/10 3:59 PM

Hi Brett,

Thanks for reporting this issue with JUnit 4.7. I assumed the fix for JUnit 4.5 would be enough. My apologies. Can you please file another bug for this issue using JUnit 4.7?

Many thanks!
-Alex

Show
Alex Ruiz added a comment - 04/Jun/10 3:59 PM Hi Brett, Thanks for reporting this issue with JUnit 4.7. I assumed the fix for JUnit 4.5 would be enough. My apologies. Can you please file another bug for this issue using JUnit 4.7? Many thanks! -Alex
Hide
Permalink
Brett Vasconcellos added a comment - 05/Jun/10 2:19 PM

Looking over this again, i realized our problem is somewhat different. I've filed FEST-361 with the details.

Show
Brett Vasconcellos added a comment - 05/Jun/10 2:19 PM Looking over this again, i realized our problem is somewhat different. I've filed FEST-361 with the details.

People

  • Assignee:
    Alex Ruiz
    Reporter:
    Bernd Kappler
Vote (0)
Watch (1)

Dates

  • Created:
    28/Aug/09 9:58 AM
    Updated:
    05/Jun/10 2:19 PM
    Resolved:
    11/Mar/10 9:03 PM
  • Atlassian JIRA (v5.0.4#731-sha1:3aa7374)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.