Add tests for the Maven2 archetypes

Committed revision 2654.

Reopening as I'm refactoring this.

I've done most of the refactoring work in r2657.
However, some things still need to be fixed:
1. The integration tests are not performed on all profiles (server types). I couldn't get it to work on jonas for example and need to look into that some more.
2. It would be great if I could get the assignment of server port number to be a little bit cleaner, and not use a suystem property as it is now. I would like to filter it into the generated project when creating it from the archetype, but need some time to figure that out.

I had tried to solve the second point by adding an archetype variable, for example cargo.servlet.port with default value 8080; but did not manage to.

I looked into #2, but can't find a cleaner way of doing it. One problem is that filtering by the invoker plugin seems to be limited to POMs, invoker.properties, and files like goals file and profiles file. I'd like properties files specified in invoker.properties to be filter, which I couldn't get to work.
So, we'll leave #2 as it is.

Fair enough... Should we close this issue back again, then?

No, #1 is still an issue for me. For some reason it doesn't work to build with jonas or jboss. The timeout isn't long enough. Haven't domne any research yet, but I'm a little bit surprised as I know it has work in earlier releases when I have tested the archetypes.

Could you try the webapp-single-module for example by enabling step 4 and 5 in src/it/generate-and-build/invoker.properties?

I've taken a look, and there is one big issue: for some reason, the servlet.port argument setting in the maven-invoker-plugin call does not work and the containers are attempted to start on port 8080.

Could you please take a look?

It seems to work for me. I get this in the build.log file, for example:

...
2011-02-23 07:59:14.122:INFO::Started SelectChannelConnector@0.0.0.0:64104
[INFO] [beddedLocalContainer] Jetty 6.x Embedded started on port [64104]
...
[INFO] [beddedLocalContainer] Jetty 6.x Embedded is stopping...
2011-02-23 07:59:16.106:INFO::Stopped SelectChannelConnector@0.0.0.0:64104
...
[WARNING] [talledLocalContainer] INFO: Starting Coyote HTTP/1.1 on http-64104
[INFO] [talledLocalContainer] Tomcat 6.x started on port [64104]
...
[INFO] [talledLocalContainer] Tomcat 6.x is stopping...
[WARNING] [talledLocalContainer] Feb 23, 2011 7:59:52 AM org.apache.coyote.http11.Http11Protocol pause
[WARNING] [talledLocalContainer] INFO: Pausing Coyote HTTP/1.1 on http-64104
[WARNING] [talledLocalContainer] Feb 23, 2011 7:59:53 AM org.apache.catalina.core.StandardService stop
[WARNING] [talledLocalContainer] INFO: Stopping service Catalina
[INFO] [talledLocalContainer] Tomcat 6.x is stopped
...

Or are you talking about the other three containers?

One possible reason for this not working for you is that you haven't build the cargo-maven2-archetypes project as well, as it contains the actual configuration of the plugins involved.

Looking into #1 some more it turns out that the problem is the install dir for the container being java.io.tmp for jonas, jboss and glassfish. It doesn't work on Mac OS. If I specify the installDir to some other folder, the build works for all containers.

We should really look into changing this behavior, as I think that the default settings should work on all platforms incl. Mac OS. I'll file a separate ticket for this (CARGO-895).

Committed revision 2683 for centralizing these comments and making them only once

For performance improvement, we can now point cargo.container.installation.directory to the core/samples/java/installs directory (where we already have downloaded the container)

The pointing of the installation directory OK.

Also, you were right, change of port works correctly. I was looking at the pom.xml and not the logs

One issue: why doesn't the maven-invoker-plugin output the invoked Maven instance's contents to the standard output? It would help us when seeing why things have failed

You can configure it to do so. But very often you configure the invoker plugin to log at debug level and you don't want all of that in the console.
If you want to know why it failed you just look in the build.log file. Like if a test fails with surefire, you check the log for that (it isn't printed to the console).

I don't totally agree with revision 2683. I think we need to address CARGO-895 to fix this, in my opinion, flaw in the design from a Maven perspective.
The good thing working with these archetypes is that we see what doesn't work well. I also see lots of log messages about things we should fix such as using deprecated configuration for the containers, using incorrect values for some container params, etc.

We should also fix CARGO-890.

This also depends on CARGO-517 else people will have to modify things in order to just enter proxy server settings...

Tests fail randomly on the CI, most often with the following stack trace:

[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] null
[INFO] ------------------------------------------------------------------------
[DEBUG] Trace
java.lang.NullPointerException
	at java.util.jar.JarFile.initializeVerifier(JarFile.java:316)
	at java.util.jar.JarFile.getInputStream(JarFile.java:390)
	at sun.misc.URLClassPath$JarLoader$1.getInputStream(URLClassPath.java:620)
	at sun.misc.Resource.cachedInputStream(Resource.java:58)
	at sun.misc.Resource.getByteBuffer(Resource.java:113)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:249)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at org.codehaus.classworlds.RealmClassLoader.loadClassDirect(RealmClassLoader.java:195)
	at org.codehaus.classworlds.DefaultClassRealm.loadClass(DefaultClassRealm.java:255)
	at org.codehaus.plexus.component.factory.java.JavaComponentFactory.newInstance(JavaComponentFactory.java:30)
	at org.codehaus.plexus.DefaultPlexusContainer.createComponentInstance(DefaultPlexusContainer.java:1464)
	at org.codehaus.plexus.component.manager.AbstractComponentManager.createComponentInstance(AbstractComponentManager.java:93)
	at org.codehaus.plexus.component.manager.PerLookupComponentManager.getComponent(PerLookupComponentManager.java:48)
	at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:331)
	at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:440)
	at org.apache.maven.plugin.DefaultPluginManager.getConfiguredMojo(DefaultPluginManager.java:652)
	at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:468)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
	at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
	at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
	at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:375)

See: http://bamboo.ci.codehaus.org/browse/CARGO-DEPLOY-86

Currently I see the following in the webapp-single-module archetype:

<home>${cargo.container.configuration.directory}</home>

This property is not defined in the archetype though. It works in the tests because we set it for the invoker-plugin executing the tests. BUT, I very much dislike our tests having an affect on the archetypes. The archetypes should be showing a best-practice/good example of a project using Cargo. When we tweak the project to fit out testing we don't accomplish this!

I agree, we should not be needing this. Removing the definition...

Now when CARGO-895 has been solved, can we switch on tests on all containers in the archetypes? Or are there still some issues with them on the Bamboo server?

If we want to switch them all, we have to set a large number of random ports for JBoss and GlassFish, and reduce the number of services on JOnAS -else many ports are already in use and we fail. The Java samples tests show this, we allocate tens of ports for these containers...

The whole point of this ticket is that we should test the archetypes. To do that, I think we should test all containers.

In this case, we would have to add all these ports in the archetype and the builder project:

• cargo.rmi.port
• cargo.tomcat.ajp.port
• cargo.glassfish.adminPort
• cargo.jboss.naming.port
• cargo.jboss.classloading.webservice.port
• cargo.jboss.jrmp.port
• cargo.jboss.jrmp.invoker.port
• cargo.jboss.invoker.pool.port
• cargo.jboss.remoting.transport.port
• cargo.jboss.ejb3.remoting.port
• cargo.jboss.transaction.recoveryManager.port
• cargo.jboss.transaction.statusManager.port
• cargo.jboss.remotedeploy.port

and in the JOnAS container set the services list to the same as in the Java samples

Perhaps ignore JBoss? Adding all this clutter to an archetype (which is supposed to be a simple example) is annoying for me.

What's the exact reason we need to do this - is it due to some ports already in use on the CI instance, containers shutdown not releasing ports, we having parallell builds (thus already occupying some ports), etc?

The third option: the CI is used by all Codehaus projects, therefore most ports are already used. And, JBoss requires tens of free ports to run (Tomcat, GlassFish and JOnAS require 2).

So, what do we do? Enable tests for some containers and ignore JBoss, hence adding a few more port definitions?

The basic requirement is that we must not change the archetypes just so that we can run our tests! The archetypes are there for our users, not for us.
However, the idea with this ticket was to check that our archetypes do in fact work. Hmm, tricky...
One idea could be to disable the tests on the CI server (where it will not work running all containers), but having them enabled when we run locally (releasing for example). That would help us detect any issues before doing a release at least. I guess that the embedded jetty profile for the archetypes could run on CI (as it is at the moment), which would give us some basic verification.

Need to do some research to fully answer this.
What we can do is to run the integration tests in a separate profile that CI doesn't trigger. However, that would disable the archetype test on embedded Jetty as well.

OK, I think we have two options:

1. When building on CI we disable the maven-invoker-plugin invocation through the skipInvocation param - no integrations tests at all
2. When building on CI we only run the embedded jetty test (by having a different invokerPropertiesFile)

The first option is easier and cleaner. However, the second option adds a basic level of verification on the CI (which is the purpose of CI).
Today I would suggest the second option. However, if we make the invoker-plugin configuration grow it could possibly cause issues/confusion. But that's a future problem i think.

What we do is that when create a profile (id: bamboo-id or similar), which reconfigures the invoker-plugin (either the skipInvocation or the invokerPropertiesFile param). This profile is only triggered on CI, so when building locally (snapshot build, release build, or whatever) the invoker-plugin is doing a full execution.

The question is how we trigger this extra profile on the Bamboo instance. One way is to specifically specify the profile (for example "mvn clean install -Pbamboo-ci"). One other way is if there's a property (already set) on the CI instance that we can use to trigger the profile. Ali, what do you think?

Well; I still think starting JOnAS, JBoss, GlassFish, Tomcat and Jetty as part of a "normal" build is a bit of an overkill: we'll need more than 10 minutes at each test, so a "simple" CARGO build would take about half an hour. That's pretty long compared to the actual 5-6 minutes.

I would rather prefer a given profile activated on some build plans of the CI that would check archetypes on these containers (preferably, one by one). On "normal" compilation cycles, only checking with one container (embedded Jetty for simplicity) would probably be best.

Remember, the CI is here to do things that most people "would not have the time to do" -like testing on each container one by one.

Yes, but we cannot test this on CI as we can't/shouldn't adapt the archetypes just to fit CI. So, if we want to verify the archetypes we need to do this locally.
But maybe we'll just skip this? Testing on embedded Jetty provides some testing. However, merely a little.

I would tend to think that if the samples pass on all containers + the archetypes pass on any container (for example, Jetty); the archetypes should be OK on the other containers as well.

EXCEPT ... when we break things like ZipUrlInstaller or others, which are tested elsewhere.

Anders, do you know if we could set the configuration properties via Java system properties?

Any ideas on this one?

What I'm wondering if we could, for example, the same way as we define -Dcargo.maven.wait, define things like

-Dcargo.maven.configuration.cargo.jboss.classloading.webservice.port=xxx -Dcargo.maven.configuration.cargo.jboss.transaction.recoveryManager.port=yyy

... then we could set all ports randomly without touching the archetypes.

I'm thinking that we'll just leave it with a test on the embedded jetty container for now.
I really think that we is needed is a better overall test suite, and that could take care of all this. What we do here in the archetype projects should be more of some type of sanity check.

ok?

Fair enough, we should then perhaps close the issue, then

Did some cleaning up in r2856. Closing as fixed.