Starting Jetty in separate JVM

Hi Stevo,

I'm not familiar with the way the code-coverage plugins work. I thought most things worked in-process? At what point would the code whose coverage you want to measure get forked with a normal jar-based project?

Hello Jan,

Cobertura at least, registers a shutdown hook, on which coverage data get stored to a file ( see http://cobertura.svn.sourceforge.net/viewvc/cobertura/trunk/cobertura/src/net/sourceforge/cobertura/coveragedata/ProjectData.java?revision=464&view=markup ). Maven surefire can be configured to fork JVM never, once for all tests, or once for each test (never/once/always), and for simple unit tests this works ok - coverage data get updated at least after last unit test ends. But for functional (e.g. selenium based) tests of an web application, if that web application (with cobertura instrumented code) is deployed using jetty, it runs in same JVM as Maven build. Thus currently, only when build stops (due to either failure or success) does shutdown hook get triggered, and coverage data get written to a file. This is too late for maven coverage report to get generated as part of the build process. On Cobertura FAQ ( http://cobertura.sourceforge.net/faq.html ), at the end of the list there is a Q devoted to this.

This is only one use case where it would be useful to support such a feature in jetty maven plugin. If this is major change, maybe it would be wise to leave the issue open and let users vote and/or add more use cases.

Feature itself maybe can be supported through a simple configuration parameter of run/runExpaneded/runWar mojos, like forkMode for maven surefire plugin, with default being "never" for behavior to remain the same for current maven jetty plugin users, but with option to change the setting so that jetty runs in separate jvm instance.

This is exactly what I'm doing (see [1] ). As you can see from the pom snippet, I'm using recently introduced deploy-war mojo/goal to start jetty and deploy prebuilt application war. Jetty stop doesn't (and shouldn't) stop Maven build JVM, only the server and thus the application. So, as [2] shows, coverage data gets saved only when Maven build stops. What I could do, as a workaround, is to register a listener on application stop to programmatically force cobertura to flush coverage data - but this is not nice as application itself will get dirty with test related code. What I'd prefer is for jetty maven plugin to support via a configuration option to have jetty started in separate JVM from the JVM where Maven build is running, expecting that in that scenario jetty stop will also stop the JVM jetty was running in (still not the one Maven build is running in) and thus trigger shutdown hook for cobertura, allowing coverage report to be generated.

Btw, with recent 6.1.16 release, I had to manually add jetty-util dependency, this was not needed in previous 6.1.15 release. But that's a different issue.

[1] Maven pom.xml snippet

...
<plugin>
<groupId>org.mortbay.jetty</groupId>
<!--
<artifactId>jetty-maven-plugin</artifactId>
<version>7.0.0.pre6</version>
-->
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.16</version>
<configuration>
<scanIntervalSeconds>5</scanIntervalSeconds>
<stopPort>9966</stopPort>
<stopKey>foo</stopKey>
<connectors>
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<port>8080</port>
<maxIdleTime>60000</maxIdleTime>
</connector>
<!--
<connector implementation="org.mortbay.jetty.ssl.SslSelectChannelConnector">
-->
<connector implementation="org.mortbay.jetty.security.SslSocketConnector">
<port>8443</port>
<maxIdleTime>60000</maxIdleTime>
<keystore>$

/src/test/jetty/server.keystore</keystore>
<keyPassword>123456</keyPassword>
</connector>
</connectors>
<webApp>$

/$

-instrumented</contextPath>
</webAppConfig>
<systemProperties>
<systemProperty>
<name>org.apache.commons.logging.Log</name>
<value>org.apache.commons.logging.impl.SimpleLog</value>
</systemProperty>
</systemProperties>
</configuration>
<dependencies>
<!--
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-ssl</artifactId>
<version>7.0.0.pre6</version>
</dependency>
-->
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<version>6.1.16</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>deploy-war</goal>
</goals>
<configuration>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
...

[2] Successful Maven build log snippet with cobertura saving coverage data

...
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 minutes 11 seconds
[INFO] Finished at: Wed Apr 15 16:08:29 CEST 2009
[INFO] Final Memory: 115M/1016M
[INFO] ------------------------------------------------------------------------
2009-04-15 16:08:30.481::INFO: Shutdown hook executing
2009-04-15 16:08:30.481::INFO: Shutdown hook complete
Cobertura: Loaded information on 175 classes.
Cobertura: Saved information on 175 classes.

Just an observation - the fork option would only make sense in the case of daemon == true, otherwise the process would detach and you'd have no way of stopping it. If this issue gets some more votes, I'll push its priority up a bit.

Jan

vote +1 - it would be nice to run jetty i a forked jvm

with such a feature it would be possible to define extra jvm-args for jetty, i.e. starting with java-rebel agent

<configuration>
<daemon>false</daemon>
<fork>true</fork>
<jvmArgs>-Drebel.spring_plugin=true -Drebel.log=true -noverify -javaagent:\tools\javarebel\javarebel.jar</jvmArgs>
</configuration>

Can at least a parameter be added to jetty:stop mojo that would tell the mojo to wait for server to actually stop?

+1 too. I need to start Jetty with Spring Agent so that my AspectJ stuff works...

+1. I need it for testing purposes, where starting the jetty server under the different JVM versions is required.

Jan, I am Java programmer and willing to have a go at enabling this. Would need some instructions and guidance to start though. I was browsing through the code and I see that Jetty6PluginServer wrapper is delegating calls to Jetty's Server class. So I think the question is whether Jetty can be forked when starting as an embedded server.

I've managed to do what I needed with 'maven-cargo-plugin'. I'll paste the relevant part of pom.xml if someone requires something similar - the important tag is '<cargo.java.home>':

<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<id>start-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-container</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<container>
<containerId>$

</containerId>
<type>$

</type>
<home>$

</home>
</container>
<configuration>
<properties>
<cargo.servlet.port>$

</cargo.servlet.port>
<cargo.java.home>$

</cargo.java.home>
</properties>
<deployables>
<deployable>
<groupId>$

</groupId>
<artifactId>$

</artifactId>
<type>war</type>
</deployable>
</deployables>
</configuration>
<wait>false</wait>
</configuration>
</plugin>

Testing template engine code (WebMacro), presumably also applies to Velocity, I get no Cobertura coverage.
I will try the suggestions here, but it would be nice for it to work without configuration.

+1 I need this for code coverage as well..

+1 I need this too! I embedded jetty in another application and stopping jetty exits my application.

To all those watchers of this issue:

Will it meet your needs if the "stop" goal was able to do a System.exit() instead of just stopping jetty?

thanks
Jan

I've implemented a new run-forked mojo that is similar to run-war mojo except that it runs Jetty in a different JVM. Below is a snippet of the plugin configuration that shows how it could be used.

      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>7.5.0-SNAPSHOT</version>
        <executions>
          <execution>
            <id>start-webapp</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>run-forked</goal>
            </goals>
          </execution>
          <execution>
            <id>stop-webapp</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <daemon>true</daemon>
          <stopPort>8181</stopPort>
          <stopKey>stopme</stopKey>
        </configuration>
      </plugin>

As we need to reimplement this forking mojo to remove the dependency on the jetty-runner jar, I've taken the opportunity to revisit it.

What I've implemented is the ability to fork off a new process to run jetty on the unassembled webapp. So, its very similar to the jetty:run mojo, but just in a new jvm.

However, the configuration is somewhat different. For example, you cannot use the <webAppConfig> element to configure the WebAppContext representing the webapp, nor the <connectors>, <loginServices> and <requestLog> to configure the jetty Server. Instead, you use context xml and jetty xml files instead (actually, I've been intending for some time to entirely remove the <connectors>, <loginServices> and <requestLog> elements, as using jettyxml files is a much better way to do it). Here's an example of config suitable for the run-forked goal:

<configure>
  <!-- where the unassembled web.xml is (optional: default value shown) -->
  <webXml>${basedir}/src/main/webapp/WEB-INF/web.xml</webXml>

  <!-- where the root of the unassembled webapp is (optional: default value shown) -->
  <webAppSourceDirectory>${basedir}/src/main/webapp</webAppSourceDirectory>

  <!-- where the tmp directory for the webapp is (optional: default value shown) -->
  <tmpDirectory>${project.build.directory}/tmp</tmpDirectory>

  <!-- where the classes for the webapp are (optional: default value shown) -->
  <classesDirectory>${project.build.outputDirectory}</classesDirectory>

  <!-- where the test classes for the webapp are (optional: default value shown) -->
  <testClassesDirectory>${project.build.testOutputDirectory}</testClassesDirectory>

  <!-- whether or not to put the test classes on the webapp's classpath (default shown) -->
  <useTestClasspath>false</useTestClasspath>

  <!-- context xml file to configure the WebAppContext (optional) -->
  <contextXml>src/main/config/context.xml</contextXml>

  <!-- the path for the deployed webapp (mandatory) -->
  <contextPath>/foo</contextPath>

  <!-- comma sep list of jetty xml files to configure the Server (optional) -->
  <jettyXml>src/main/config/jetty1.xml,src/main/config/jetty2.xml</jettyXml>

  <!-- stop port -->
  <stopPort></stopPort>

  <!-- stop key -->
  <stopKey></stopKey>

  <!-- skip the execution or not (default shown) -->
  <skip>false</skip>
</configure> 

When run with "mvn jetty:run-forked" the above would start up a separate jvm, and start a Server which is configured by the src/main/config/jetty1.xml and src/main/config/jetty2.xml files, and then deploy a WebAppContext to it representing the unassembled webapp that is configured by a combination of the src/main/config/context.xml file, the pom, and generated by the plugin.

Now, my questions for those interested in this type of plugin are:

• what is the lifecycle relationship you want between the plugin and the forked process?
  • should the plugin wait for the child process to finish?
    or
  • should the plugin finish immediately after forking, leaving the child process running? (Won't this leave orphaned processes running?)

If some of you could provide example webapps and poms of how you'd like to run these forked executions that I could test with, that would be very helpful.

I should point out that the work is being done in a branch called "forked-runner" in the codehaus git repo at git@git.codehaus.org/jetty-project.git

Thanks Jan,
I just lifted parts of that code for launching a jetty instance in a separate process. It would be very cool if this was easier to do using the standard Jetty starter classes.

The "forked-runner" branch has been merged into head with commit a043b51.

I still need feedback on how users want the lifecycle of the plugin to relate to the lifecycle of the forked jetty. Currently, whenever one stops so does the other one. Maybe this is not what is required???

thanks
Jan

Thanks for implementing this feature.
Is there any hope of a similar forked option for other goals? In particular, the 'deploy-war' goal. I would like to measure coverage of a war, but my war is pre-assembled.

I'm running Jetty during my integration test phase, and I need the forked jetty to return control back to maven like a regular jetty:run would in a maven lifecycle. I'm using similar configuration as explained in http://maven.apache.org/plugins/maven-failsafe-plugin/usage.html where it binds to the pre-integration-test phase and the post-integration-test phase. It works with just run, now I would like it to work with run-forked, so I can add some extra items to the jvmArgs (such as a javaagent for Jacoco to get code coverage of the Jetty instance)

We have exactly the same use case as Larry Shatzer. The run-forked goal must return when the jetty was started so we can run our integration tests against it. We only need that to specify another javaagent (jacoco) via the jvmArgs parameter.

We have exactly the same use case too as Larry and Sascha!

The combination of maven-failsafe-plugin, maven-jetty-plugin and Sonar/Jacoco seems such a natural fit for integration test coverage that it's a pitty that this use case does not work.

Does somebody know a usable workaround (e.g. manually starting jetty w/o the maven plugin or offline bytecode instrumentation using something else than Jacoco)?

Thanks,

Andreas

My workaround now is using Cargo (http://cargo.codehaus.org/), where I can specify the JVM arguments on the command line. I'm not using their Maven plugin, but have a setup class that takes care of installing/configuring/deploying to Jetty and starting it up.

I've added a new parameter <waitForChild>. This defaults to "true", and has the same behaviour as before, ie the process spawning the exec'ed jetty will wait for it to complete.

If set to "false", the parent completes, leaving the child process running, in which case you need to use mvn jetty:stop to kill it.

I think that satisfies Larry, Sacha and Andreas' use cases, yes?

Jan

Jan, I'm keen to use the new <waitForChild> element that you mention as run-forked blocks the maven JVM when binding it to the pre-integration-test phase and executing mvn clean install. I'm using version 8.1.4.v20120524 of the jetty-maven-plugin but it doesn't seem to acknowledge the <waitForChild> element when added to a <configuration> block. Is there any documentation/ guidance here? Cheers.

Looks like I missed the cut-off for 7.6.4/8.1.4 by a smidgin, so this will be in 7.6.5/8.1.5. If it helpse, jetty snapshots can be obtained from here: https://oss.sonatype.org/content/groups/jetty

Jan

The new <waitForChild> element appears to me to satisfy previous posters' requirements. If this is not the case, please reopen the issue with a detailed description of what is wrong/missing.

thanks
Jan

Thanks, have grabbed it from the Jetty Snapshots repo and am using 8.1.5-SNAPSHOT, Jetty startup logs confirm that 8.1.5-SNAPSHOT is now being used at runtime:

[STDERR] 2012-07-03 11:33:53.486:INFO:omjp.Starter:Started Jetty Server
[STDERR] 2012-07-03 11:33:53.487:INFO:oejs.Server:jetty-8.1.5-SNAPSHOT

Still seems to block though when I execute mvn clean install. Sorry, I'm not trying to turn this JIRA in to a forum-style posting but could I ask you to confirm if I'm using the <waitForChild> element correctly as I'm keen to get this working? Thanks.

<plugin>
	<groupId>org.mortbay.jetty</groupId>
	<artifactId>jetty-maven-plugin</artifactId>
	<version>8.1.5-SNAPSHOT</version>
	<configuration>
		<stopPort>9999</stopPort>
		<stopKey>foo</stopKey>
		<jettyXml>${project.build.testOutputDirectory}/jetty.xml</jettyXml>
		<jettyEnvXml>${project.build.testOutputDirectory}/jetty-webapp.xml</jettyEnvXml>
		<scanIntervalSeconds>10</scanIntervalSeconds>
    <contextPath>/em3/gateway/admin</contextPath>
    <useTestClasspath>false</useTestClasspath>
    <jvmArgs>-Dlog4j.configuration=file:${project.basedir}/src/test/resources/log4j.properties
             -Dem3PropertiesLocation=${project.build.testOutputDirectory}
    </jvmArgs>
    <daemon>true</daemon>
    <!-- Eclipse doesn't give me an auto complete but also doesn't complain about 'waitForChild' here, however Jetty still blocks 'mvn clean install'. -->
    <waitForChild>false</waitForChild>
	</configuration>
	<executions>
		<execution>
			<id>start-jetty</id>
			<phase>pre-integration-test</phase>
			<goals>
				<goal>run-forked</goal>
			</goals>
			<configuration>
                          <daemon>true</daemon>
                          <!-- Eclipse doesn't give me an auto complete but also doesn't complain about 'waitForChild' here, however Jetty still blocks 'mvn clean install'. -->
                          <waitForChild>false</waitForChild>
			</configuration>
		</execution>
		<execution>
			<id>stop-jetty</id>
			<phase>post-integration-test</phase>
			<goals>
				<goal>stop</goal>
			</goals>
		</execution>
	</executions>
</plugin>

Hi Ed,

Looks like the changes that were made to jetty-7 snapshot were not ported over to jetty-8. I've ported and checked in, so the CI build system should kick in soon and do another snapshot build, so stay tuned.

cheers
Jan

Just tried with 8.1.5-20120703.081123-38 and wasn't able to get the waitForChild working. Similar setup to Edd and no Eclipse auto-complete for that attribute in the <configuration>. Going to verify my install works with 7.6.5-SNAPSHOT as I just stumbled onto this ticket today.

<waitForChild> is working on the 7.6.5-20120703.080710-38. During my testing of this feature I found that I am now stuck at two extremes for the jetty startup. One is where the maven lifecycle stops when using the run-forked as part of another phase, pre-integration-test in my case, or the other extreme which is that the maven lifecycle just continues to process the next phase even before Jetty has initialized. When using the "run" goal the maven process wouldn't continue onto the next phase until the init of Jetty was complete. Is there someway to tie into that same state change when dealing with the <waitForChild>, maybe a <waitForInit> or something similar?

Tested this today with SNAPSHOT-7-6-5. I can confirm that waitForChild now has an effect however it's not the effect I was personally hoping it would have. Essentially I wanted it to have the same effect as setting <daemon>true</daemon has when running jetty:run, in that Jetty would start (in a new JVM) and Maven would block until Jetty had started, at which point Maven would continue (in my case running the integration test phase as I have bound run-forked to the pre-integration-test) running the requested goals. What is actually happening is that the forked JVM begins to start but control returns immediately to the parent process which executes my integration tests, these all fail with connection refused errors because Jetty hasn't yet completely started up.

Is there any way to make jetty:run-forked behave more like jetty:run in terms of blocking? I realise that there may be extra difficulties in terms of inter-jvm-communication but this would be great for cases like mine where we need a fresh JVM but want to support running Jetty for integration tests.

I seem to be unable to re-open JIRA but wanted to comment my findings.

Ok, change made so that even if the parent process is going to exit (ie waitForChild is FALSE) then it waits until the forked jetty has started up.

Tested this on 7.6.8-SNAPSHOT, can confirm that run-forked is now starting, waiting and stopping correctly - thanks for this. I have an issue where it hangs in a multi module project build however will raise this as a separate JIRA and will try to put together a sharable example.

r.e. my last comment about multi-module project builds hanging, ignore that it was an issue created within my particular project. My testing of this suggests that it is working perfectly now. Many thanks.