Revive the automated Selenium integration tests

This patch contains an alternative to JUnit test. This patch uses TestNG, which allows us to divide into groups and set the dependencies between groups.

This is a simple test, but you can see the operation of the new test. Upload a new version soon with some complex test.

You think that this alternative. If you think a good alternative to JUnit, I will continue with it.

Thanks, Jose! I'll take a look at this and try it out.

Meanwhile, since it contains new work and you intend to do more, could you please submit a contributor license agreement? The form and info about how to submit it are available here: http://www.apache.org/licenses/#clas

New Test for TestNG. They contain tests about project groups, Maven2 project, Login.

It has restructured the file system and have added some of the utilities created by Emmanuel Venisse, They do more easy the creation of the test, although many of them had been modified.

Add more test soon.

I have attached a patch for /src/test/it in continuum-webapp-test module. The classes I modified are the following:

AbstractSeleniumTestCase.java
AbstractContinuumTestCase.java
LoginTest.java
MyAccountTest.java
ProjectGroupTest.java

I commented on the methods that aren't working yet(or still needs to be modified). To successfully run the tests, you need to delete these classes:

AccountSecurityTest.java
AddMavenOneProjectTestCase.java
AddMavenTwoProjectTest.java
AntTest.java
SchedulesPageTest.java

because those aren't updated yet and it might cause selenium to crash. Then execute "mvn clean install -f it-pom.xml"

Will modify the other classes and also will create new classes for features that aren't included since the day these scripts aren't updated. I will add a patch here again soon. Thanks.

Thanks, Jose. I see that your iCLA was recorded (though your name was spelled wrong-- I've asked to get that fixed.)

I'm working with only the second patch (CONTINUUM-2073-TestNG-2.patch) as I assume it replaces the first.

The readme only says to execute 'mvn install -f testng-pom.xml' but I don't see any Cargo plugin config in the pom. Does it assume that Continuum is already running on a particular port? I can add that part in (taken from pom.xml which has the newer config.)

A couple of things before this gets applied. Can you please...

• identify which classes are based on Emmanuel's work?

• add the license header to all of the files and re-submit this? You can see the text to include at http://www.apache.org/legal/src-headers.html , and an example of it in a Java class at http://svn.apache.org/repos/asf/continuum/trunk/continuum-core/src/main/java/org/apache/continuum/repository/DefaultRepositoryService.java

Thanks!

Hi Cora. I'm having trouble with the CONTINUUM-2073-src-test-it.patch file. After I apply it, a file you didn't even touch refuses to compile! Here's what I did:

vn co https://svn.apache.org/repos/asf/continuum/trunk/continuum-webapp-test
cd continuum-webapp-test
patch -p1 < ~/Downloads/CONTINUUM-2073-src-test-it.patch
mvn clean install -f it-pom.xml

and I get:

INFO] Compiling 14 source files to /private/tmp/continuum-webapp-test/target/test-classes
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Compilation failure
could not parse error message: [parsing started /private/tmp/continuum-webapp-test/src/test/it/org/apache/continuum/web/test/AddMavenOneProjectTestCase.java]
[parsing completed 12ms]
...
/private/tmp/continuum-webapp-test/src/test/it/org/apache/continuum/web/test/AddMavenOneProjectTestCase.java:51: unreported exception java.lang.Exception; must be caught or declared to be thrown
        addMavenOneProject( m1PomUrl, "", "", null, validPom );
                          ^

This doesn't happen with a fresh checkout of continuum-webapp-test and the same 'mvn clean install -f it-pom.xml' command.

Can you take I look and let me know if you get the same thing?

Hi Wendy,

When I run the tests(the patch) I removed the other tests that I didn't modified yet and one of them is the class that throws the error you pasted above. Anyway, I will be attaching another patch today to fix the errors of the other tests. Thanks!

Thanks, Cora. If you don't get them all fixed in the next pass, just add a step in the README that says which ones to delete.

Hi Wendy. I attached a new patch ( CONTINUUM-2073-src-test-it-2.patch ). This patch contains the fixes of the other tests that results to errors upon compiling. Same command will be executed as what is in the ReadMe file. This should work now. From my last run of the tests, here are the results:

Tests run: 65, Failures: 32, Errors: 1, Skipped: 0

Some of the test failures are those tests that aren't updated yet. I will be working on it and will attach a patch again here soon. Thanks. -Cora

committed CONTINUUM-2073-src-test-it-2.patch in r751442, thanks Cora! I get slightly different results:

Tests run: 65, Failures: 29, Errors: 2, Skipped: 0

I notice that a 'data' directory is getting created at continuum-webapp-test/data. It should be under 'target' if possible, or else it needs to be added it to the clean plugin's configuration.

Attached a new patch with the Apache header also contains a reference to the classes that have fallen into the work of Emmanuel Venisse. This patch contains new tests about Notifers and BuildDefinitions.

I am going add new test for early Queue, Queue Build, Local Repositories and Purge as soon as I can. I am going postpone Ant, Shell and Maven1 test for final.

Wendy as you say, It is necessary run continuum before run test .

Thanks for correcting the name in Contributor list.

Note: This patch replaces the previous patchs.

Hi Wendy. The 'data' directory that is created at continuum-webapp-test/ is the directory that is set in the General Configuration for Working Directory, Build Output Directory and Release Output Directory(i used the default setting upon running that's why). I will create a script for this to point it to the target directory and will attach a patch again here with additional tests. Thanks!

Jose, I committed the third patch – thanks! It doesn't actually run the tests for me, though. It gets stuck at "Running TestSuite" and then nothing else happens until I hit ctrl-C to stop it.

I added the 'data' directory to the clean plugin config so it will get deleted during 'mvn clean'. (r755054)

If you move the data, that bit can be removed in the same patch.

In these tests I use a more current version of selenium, although this should not be the problem.
I develop the tests at Ubuntu 8.10, Firefox 3.0.2 or Opera, JDK1.6 and Maven 2.0.9.

I'm going to test in Windows and with JDK1.5. I will add new tests (queues, schedules, local repository and purge) on Friday.
Then I will notify the results and the steps that I have followed.

This patch contains new test about Purge, Schedule, Queue, Configuration and LocalRepository. Next time, I will attach a patch with the rest of the test. I run the test is the following environments:

• Ubuntu 8.10, Maven 2.0.9, JDK5, JDK6, Opera, Firefox
• Windows XP, Maven 2.0.9, JDK5, JDK6, Firefox

These are the steps that I have followed:

1. Checkout continuum(755816)
2. Install continuum. mvn install in continuum directory.
3. Run continuum in webapp directory (mvn jetty: run)
4. Configure admin user and default configuration.
5. Adding openqa mirror in setting.xml of Maven:
<mirror>
<id> openqa </ id>
<mirrorOf>! central </ mirrorOf>
<name> OpenQA Mirror </ name>
<url>http://nexus.openqa.org/content/repositories/releases/</ url>
</ mirror>
5. Run continuum test. mvn clean install -f testng-pom.xml in continuum-webapp-test directory

I hope they work successfully.

Jose, I applied the 4th patch, and also updated testng-pom.xml to automatically start and stop the webapp on port 9595.

Do you have any ideas on how to handle the 'create admin user' and 'general configuration' that must be done when the app is first started?
I put some conditional logic in src/test/it/.../AboutTest.java, which isn't pretty, but works.

This set of tests is still not working for me. Here's what I see:

$ mvn clean install -f testng-pom.xml
...
[INFO] Selenium server started
[INFO] [cargo:start

]
[INFO] [surefire:test

]
[INFO] Surefire report directory: /Users/wsmoak/svn/continuum/continuum/continuum-webapp-test/target/surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite

... here, it hangs until I press CTRL-C ...

^C[INFO] Shutting down...
[INFO] 12:01:54,694 INFO [org.mortbay.util.ThreadedServer] Stopping Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=4444]
Destroying 1 processes
Destroying process..
Destroyed 1 processes
imbrium:continuum-webapp-test wsmoak$

I would expect to see the Selenium browser window open, and then for the tests to fail when they hit the 'create admin user' page and can't get past it. But I don't see anything happening.

To register an admin user at the beginning is not difficult, I use annotation @BeforeSuite and it will only be processed to start the suite (test group). I will use 'src/test/it/.../AboutTest.java' to reference.

I'm going to checkout trunk in others PCs to try replicate the problem with this tests. If I can not reproduce the problem I will send a link to a video with the steps I follow to launch the test. Excuse me, I feel much problems with tests.

I'm finishing the basic test, I am pending Maven1, Shell and Ant tests. If I have a little time I hope finish after Holy Week.

Thank you very much for setting cargo plugin in testng-pom.xml.

attached CONTINUUM-2073-src-test-it-3.patch

I did not exclude the tests that were having errors but instead I just commented on the tests that causes errors and repaired some of it. As for this patch:

Results :

Tests run: 54, Failures: 0, Errors: 0, Skipped: 0

I also modified the script to create the /data directory into the target folder so that it will be cleaned when executing clean goal. And also, I have modified the pom.xml to use the snapshot version of selenium(see ReadMe.txt). More tests scripts to add soon. Thanks!

Applied CONTINUUM-2073-src-test-it-3.patch in r763457. Everything passed except for the first test, which timed out. I think the app hadn't started completely yet, so I'll try that again. Thanks, Cora!

Update new version of test with TestNG. Fixed in r766453.

• Restructure source files and generate parent abstract classes.
• Create admin user when initialize DB
• Fixed bugs and modified old tags related i18n.
• Add new test about intallations,buildEnvirotment,buildTemplate,mavenOne,shell,...
• MavenOne and MavenShell project are not actived, pending CONTINUUM-2083

FYI, I switched the build around so that the default pom.xml runs the TestNG tests, and there is an alternate junit-pom.xml.

I think this issue is satisfied, the tests are alive again.

Feel free to use this issue key for minor updates, but it would be good to open a specific issue for major changes or new contributions.

Attached CONTINUUM-2073-TestNG-5.patch.

Modifications:

• modified pom.xml properties tag so that "installs" folder will be created inside target dir so that it will also be deleted when performing "clean" goal
• added postAdminUserCreation() in AboutTest class for /data folder to be created inside target folder also.

Patch fixed in revision 774122.

Thanks, Marecor.

Attached CONTINUUM-2073-TestNG.patch

• modified tests for the fix in CONTINUUM-1848 so that all tests will run successfully. Thanks.

CONTINUUM-2073-TestNG-6 patch applied in r774938. Thanks!

I still get 9 failures locally, but I'll force a build on vmbuild.a.o and see if this clears up the 1 failure there.

... and it still fails on vmbuild:
http://vmbuild.apache.org/continuum/buildResult.action?buildId=191419&projectGroupId=69&projectId=2284&projectName=Continuum+Webapp+Test

Oops, i guess CONTINUUM-2073-TestNG-6.patch does not fix the problem on vmbuild. It has different error:

Results :

Failed tests:
initializeContinuum(org.apache.continuum.web.test.AboutTest)

Tests run: 153, Failures: 1, Errors: 0, Skipped: 152

Wendy, if you won't mind, can you attach the ../continuum-webapp-test/target/surefire-reports/TestSuite.txt file in here. I'd like to see why it failed there. Thanks!

You can view surefire report in:

http://vmbuild.apache.org:8080/continuum/surefireReport.action?projectId=2284&projectGroupId=69&buildId=191185

I can't reply this error. I have checked code and I have not found a reason for this error.

Attached CONTINUUM-2073-TestNG-7.patch

Modifications:

• QueueTest.java - added a method to verify if a build actively running is displayed in the queue page.
• AbstractSeleniumTest.java - added 3 methods (clickAndWait, waitForTextPresent, waitForElementPresent)
• AbstractContinuumTest.java - modified the value of Thread.sleep on waitAddProject method to accomodate a project which builds in a longer time.
• AbstractBuildQueueTest.java
• testng.properties - replaced the properties for testAddMavenTwoProject since the new method I added in the QueueTest.java requires a project which builds for a long time.

Also attached is the "continuum-sandbox-build-queue-test-data-testng.patch" for the sample project used in this patch

Here's the result of the test run locally:

Tests run: 149, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 792.766 sec

Results :

Tests run: 149, Failures: 0, Errors: 0, Skipped: 0

[INFO] [cargo:stop

]
[INFO] [install:install]
[INFO] Installing D:\workspace\continuum2\target\pom-transformed.xml to C:\Docum
ents and Settings\tintin\.m2\repository\org\apache\continuum\continuum-webapp-te
st\1.4.0-SNAPSHOT\continuum-webapp-test-1.4.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 minutes 21 seconds
[INFO] Finished at: Mon May 25 13:18:12 CST 2009
[INFO] Final Memory: 41M/254M
[INFO] ------------------------------------------------------------------------

I noticed another couple of patches here and went ahead and applied patch #7 and continum-build-queue-test-data for the sandbox). Thanks, Tin

After this, I still get failures locally: Tests run: 149, Failures: 12, Errors: 0, Skipped: 6

We've already closed this issue as the tests are definitely alive again. Please open a new issue with a more specific description if you have additional tests or changes to contribute.