no coverage reported for integration-test

The documentation for cobertura:cobertura flat out says that it runs the "test" phase prior to executing itself, so I assume this currently cannot be configured. There's of course much more complicated cases that people use the integration-test phase for, but for a lot of projects, it'd be enough if you could configure Cobertura to at least include or run integration-test phase instead. Email thread at http://www.mail-archive.com/users@maven.apache.org/msg78743.html.

Someone to work on this ?
Thanks

Here's a patch, with new cobertura:cobertura-integration report mojo, which extends existing cobertura report mojo but instead of test phase executes verify phase to have pre-integration-test, integration-test, and post-integration-test phases to execute as well. To use it one should define in pom reporting section something like this:

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.3-SNAPSHOT</version>
<reportSets>
<reportSet>
<reports>
<report>cobertura-integration</report>
</reports>
</reportSet>
</reportSets>
</plugin>

This has been tested only on a trivial example project, and appears to be working well. I'm absolute beginner when it comes to mojo development, so this patch should be treated in that light, just as an suggestion on how this issue could be resolved.

The second patch adds one more new cobertura:report-only mojo, which executes only validate phase. Idea is that one should be able to use it together with cobertura:clean cobertura:instrument mojos, for fine-grained control over instrumentation process.

I've tested and can confirm that both new Mojo's work well with relatively big multi-module projects.

Can someone else please try too?

Is it possible to get at least report-only mojo included in 2.3? With it, and with existing clean and instrument mojos, one can have more complete control over what and when gets instrumented, run tests and then get a coverage report only without rerunning of tests just for report. cobertura-integration mojo is not needed, but can be useful for simpler use cases, where with less configuration one can get coverage data for tests run in integration-test phase along with coverage data for tests in test phase.

Hi,

How/where do I go to get the 2.3-SNAPSHOT release? I don't see it on ibiblio, but I'd love to get that version to help test this function.

Thanks,
Tom

Snapshot is deployed but I'm not sure that this patch has been applied.

To test the cobertura:report-only mojo patch one can check out latest cobertura-maven-plugin sources from http://svn.codehaus.org/mojo/ under trunk/mojo/cobertura-maven-plugin, apply the patch, install the plugin snapshot in to your local repository, and then use it from your maven project.

Hi Stevo,
I've patched the 2.3 snapshot with your work, and it works on big pom tree (parent/module) configuration.
But how did you solved the "classesDirectory " problem when using : surefire+selenium+cobertura when executing test, deployment, it and site into an unique maven execution ?
Removing the "readOnly" flag from the war plugin slove partially the problem, but i think there is a much simplier solution to run correctly cobertura ....
Can you attach a pom sample and the associated maven command line you use please?

Thanks

Why this patch has not been released to 2.3 version?
It works very well on simple and complexe configuration.

Wish "report-only" mojo was included in 2.3, hopefully it will get included in 2.4, together with this new one, "check-only" mojo (see attached cobertura-maven-plugin_check-only-mojo.patch). Current cobertura:check mojo is heavy, it includes running tests, so if one wants to run tests (e.g. with surefire), auto check coverage (cobertura:check), and get a coverage report in a site (using cobertura:cobertura), tests get run at least 3 times - having tests run twice (once on non instrumented code to check if code is ok, and once on instrumented code to get coverage) takes time but it's understandable, while having them run 3 times is way too much. Point is that cobertura maven plugin should be able to reuse coverage data for both check and for report generation tasks. With report-only and check-only this is feasible - with all these mojos at hand one could:

1) cobertura:clean
2) run the tests on non instrumented code and only if they pass execute...
3) cobertura:instrument
4) run the tests on instrumented code (this would generate coverage data)
5) cobertura:check-only, to check if coverage requirements are met, and only if they are execute...
6) cobertura:report-only, as part of project site generation

CoberturaIntegrationReportMojo.patch can be deleted/ignored, report-only mojo is enough.

Attaching patch with updated check-only and report-only mojos (cobertura-maven-plugin_check-only-and-report-only-mojos_with-IT), now with their integration test. Previous patches can be deleted/ignored.

Updating the issue to reflect that there is now a patch with integration test case

I'm wondering if there is a danger in the new approach of running the individual steps. Let's say I attach the various goals to my default lifecycle. Isn't there a danger that a package step will package up the instrumented classes because the instrument goal does the following:

// Set the instrumented classes to be the new output directory (for other plugins to pick up)
project.getBuild().setOutputDirectory( instrumentedDirectory.getPath() );
System.setProperty( "project.build.outputDirectory", instrumentedDirectory.getPath() );

I tried the new cobertura:cobertura-integration mojo in 2.3-SNAPSHOT downloaded from http://snapshots.repository.codehaus.org. It didn't work on my project. All integration tests were skipped.

Hi, I wanted to be able to run my integration tests with cobertura. I added the http://snapshots.repository.codehaus.org to the repository and did an mvn clean. It then downloaded the jars.
I then ran mvn cobertura:cobertura-integration command but got an error that said the new mojo was not found.

Pasted below is the our put of the mvn clean command that shouws the jars being downloaded.

[INFO] [clean:clean

]
[INFO] Deleting directory C:\Users\abc\workspace\restapi\target
[INFO] [cobertura:clean

]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 seconds
[INFO] Finished at: Thu Apr 22 10:43:14 EDT 2010
[INFO] Final Memory: 9M/247M
[INFO] ------------------------------------------------------------------------
C:\Users\abc\ workspace\restapi>mvn clean
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building restapi
[INFO] task-segment: [clean]
[INFO] ------------------------------------------------------------------------
[INFO] snapshot org.codehaus.mojo:cobertura-maven-plugin:2.3-SNAPSHOT: checking
for updates from org.codehaus.mojo
[INFO] snapshot org.codehaus.mojo:cobertura-maven-plugin:2.3-SNAPSHOT: checking
for updates from nikhil
Downloading: http://snapshots.repository.codehaus.org/org/codehaus/mojo/cobertur
a-maven-plugin/2.3-SNAPSHOT/cobertura-maven-plugin-2.3-20090629.120157-3.pom
5K downloaded (cobertura-maven-plugin-2.3-20090629.120157-3.pom)
Downloading: http://snapshots.repository.codehaus.org/org/codehaus/mojo/mojo/16/
mojo-16.pom
[INFO] Unable to find resource 'org.codehaus.mojo:mojo:pom:16' in repository org
.codehaus.mojo (http://snapshots.repository.codehaus.org)
Downloading: http://repo1.maven.org/maven2/org/codehaus/mojo/mojo/16/mojo-16.pom

Downloading: http://snapshots.repository.codehaus.org/org/codehaus/mojo/cobertur
a-maven-plugin/2.3-SNAPSHOT/cobertura-maven-plugin-2.3-20090629.120157-3.jar
31K downloaded (cobertura-maven-plugin-2.3-20090629.120157-3.jar)
Downloading: http://snapshots.repository.codehaus.org/org/codehaus/mojo/mojo-par
ent/23/mojo-parent-23.pom
[INFO] Unable to find resource 'org.codehaus.mojo:mojo-parent:pom:23' in reposit
ory org.codehaus.mojo (http://snapshots.repository.codehaus.org)
Downloading: http://repo1.maven.org/maven2/org/codehaus/mojo/mojo-parent/23/mojo
-parent-23.pom

[INFO] [clean:clean

]
[INFO] Deleting directory C:\Users\abc\workspace\restapi\target
[INFO] [cobertura:clean

]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 seconds
[INFO] Finished at: Thu Apr 22 10:44:23 EDT 2010
[INFO] Final Memory: 9M/247M
[INFO] ------------------------------------------------------------------------

And here the mvn cobertura:cobertura-integration command being run

C:\Users\abc\workspace\restapi>mvn cobertura:cobertura-integration

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Required goal not found: cobertura:cobertura-integration in org.codehaus.
mojo:cobertura-maven-plugin:2.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: < 1 second
[INFO] Finished at: Thu Apr 22 11:13:27 EDT 2010
[INFO] Final Memory: 3M/247M
[INFO] ------------------------------------------------------------------------
C:\Users\abc\workspace\restapi>

@Nikhil Almeida
Like the logs already tell you: the cobertura-integration goal doesn't exist. Not in cobertura-maven-plugin-2.3 or in the current trunk.
Although there are a lot of votes, this patch hasn't been added to the plugin yet.

Was there any reason why this patch was not applied to the new release done on April 25? Is any developer looking into this? Please, pretty please...

please release it, i need it too!

Can anyone apply the patch and release it - in 2.5?

We are using version 2.4 and it seems to run integration tests but as unit tests (our integration tests are cactus based). So is it confirmed that v 2.4 of maven plugin won't run integration tests?

I applied your patch and now I can run a cobertura report after integration test .. nice one!

I hope your patches get applied in the next release.

At least we can now have a choice if we want to include integration tests into our code coverage or not.

+1 On wishing to see this patch applied - it would tie in nicely with our Hudson reporting

This issue is opened since 01/Mar/08. This is more than 2 years. There is a simple patch that works and it just adds two goals, so the plugin is backward compatible. Why this take so long to do ? May we have a response about when we can have this done or not. Because I'm about to do a fork of this plugin on google code with the possibility to do integration test coverage on war application which is really cool to see functional test coverage. Sorry for the tone, I really like your work
I attach the updated patch which work on latest 2.5-SNAPSHOT.

I think the problem with this issue is that it only covers half of the problem. It will only work for classes executed by the m-failsafe-p, because it can use these instrumented classes. But you might want to do an integration-test on the generated jar/war/.. and that's not possible.
You don't want the (main) artifact to be instrumented, but you might want to have an instrumented version as well (with cobertura classifier) and use that one to do the integration-test.

To cover the first type of integration-tests I'll review the patches. I don't think it would harm to add a report-only goal to the plugin.

I found a small trick that make integration-test works on instrumented jar/war/...

Obviously I use a maven profile that I only trigger during dev and on my ci server.
If my war application does not depend on other artifact, the code is quite simple. I use IT test (which are now understood by husdon ci) and I load my war (multiple times) using jetty during these tests. At the end, the cobertura results are flushed.

<profile>
    <id>it-coverage</id>
    
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <configuration>
                    <formats>
                        <format>xml</format>
                    </formats> 
                    <check>
                        <haltOnFailure>false</haltOnFailure>
                    </check>
                    <instrumentation>
                        <excludes>
                            <exclude>**/*Test.class</exclude>
                        </excludes>
                    </instrumentation>
                </configuration>
                <executions>
                    <execution>
                        <id>cobertura-clean</id>
                        <phase>clean</phase>
                        <goals>
                            <goal>clean</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>cobertura-instrument</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>instrument</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>cobertura-check-only</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>check-only</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    
    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <configuration>
                    <formats>
                        <format>html</format>
                        <format>xml</format>
                    </formats>
                </configuration>
                <reportSets>
                    <reportSet>
                        <reports>
                            <report>report-only</report>
                        </reports>
                    </reportSet>
                </reportSets>
            </plugin>
        </plugins>
    </reporting>
</profile>

But when you have dependencies you want the coverage to go through. so I use a trick. I exclude my dependencies from the war build and instead I unpack the dependencies sources and attach them to the build. Maven compile and instrument these classes and we have the coverage we want

And this is only possible with the patch.

<profile>
    <id>it-coverage</id>
    
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <id>unpack-dependencies-src</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>unpack-dependencies</goal>
                        </goals>
                        <configuration>
                            <classifier>sources</classifier>
                            <includeGroupIds>com.example</includeGroupIds>
                             <outputDirectory>${project.build.directory}/generated-sources/it-dependencies</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>add-it-dep-source</id>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>add-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>${project.build.directory}/generated-sources/it-dependencies</source>
                            </sources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <configuration>
                    <formats>
                        <format>xml</format>
                    </formats> 
                    <check>
                        <haltOnFailure>false</haltOnFailure>
                    </check>
                    <instrumentation>
                        <excludes>
                            <exclude>**/*Test.class</exclude>
                        </excludes>
                    </instrumentation>
                </configuration>
                <executions>
                    <execution>
                        <id>cobertura-clean</id>
                        <phase>clean</phase>
                        <goals>
                            <goal>clean</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>cobertura-instrument</id>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>instrument</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>cobertura-check-only</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>check-only</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            
            <plugin>
                <artifactId>maven-war-plugin</artifactId>
                <configuration>
                    <archiveClasses>false</archiveClasses>
                    <packagingExcludes>WEB-INF/lib/example-*.jar</packagingExcludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
    
    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <configuration>
                    <formats>
                        <format>html</format>
                        <format>xml</format>
                    </formats>
                </configuration>
                <reportSets>
                    <reportSet>
                        <reports>
                            <report>report-only</report>
                        </reports>
                    </reportSet>
                </reportSets>
            </plugin>
        </plugins>
    </reporting>
</profile>

What do you think ?

Maybe MCOBERTURA-114 could help here

After reviewing this issue, I don't think the check-only/report-only is the right solution.
The cobertura-lifecycle has to be extended, and I think it should look something like this:

Before entering the test phase, the java-classes need to be instrumented for test
Before entering the integration-test phase, java-classes need to be instrumented for integration-test.
Now they both have their own .ser file, and during verify all these should be checked and eventually reported.
I think this would result in less configuration and no need to misuse a report-only goal.

I agree. So that way we can have two coverage reports, one for unit tests and the other for integration tests. And so, we can invoke mvn site and got results without running verify goal first, which is required with the patch.

A reasonable alternative for integration testing coverage is here: http://www.sonarsource.org/measure-code-coverage-by-integration-tests-with-sonar/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+Sonar+(Sonar)&utm_content=Google+Reader

It works fine for me.

I use the following workaround:

<profile>
  <!-- Build with IntegrationTestcoverage => instrument classes with cobertura before integrationtests starts. -->
  <id>buildWithIntegrationTestCoverage</id>
  <activation>
    <property>
      <name>buildWithIntegrationTestCoverage</name>
      <value>true</value>
    </property>
  </activation>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.3</version>
        <executions>
          <execution>
            <id>instrument-classes</id>
            <phase>package</phase>
            <goals>
              <goal>instrument</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    
      <!-- Add cobertura as dependency to the jetty-plugin (required for instrumented classes) -->
      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>maven-jetty-plugin</artifactId>
        <dependencies>
          <dependency>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>cobertura-maven-plugin</artifactId>
            <version>2.3</version>
            <type>jar</type>
          </dependency>
        </dependencies>
      </plugin>
    </plugins>
  </build>
</profile>

And then i start the site-generation with:

mvn clean install site -DbuildWithIntegrationTestCoverage=true

Is there any news on this?

There are a couple of patches here, and there's even a fork at http://code.google.com/p/cobertura-it-maven-plugin/wiki/HowToUse to solve this issue.

Can any of it be used to implement this feature?

It seems like Jira is not the right tool to come to a solid/everything covering solution.
So I've started a confluence-page: http://docs.codehaus.org/display/MOJO/Cobertura+Maven+Plugin
This is quite a complex issue and is related to other cobertura-issues.
So let's first describe how the plugin should work.
Any contribution is appreciated.