Cobertura should aggregate results of multi-module builds

You might want to check out the dashboard plugin for this.
http://mojo.codehaus.org/dashboard-maven-plugin/

Does this include the possibility of having the code under test and the test cases in separate modules?
We've got some very large projects where integration tests are located in a separate module, but we still would like to have coverage reports on that.

this is very useful functionality, clover2 provides a great example. will into it

While the dashboard plugin might be useful, from their documentation it looks like this is a summery report. I want to see a combined cobertura report. I have too many modules for opening each one to be effective.

If the dashboard provided links to the cobertura reports this might be workable.

Isn't this issue actually duplicate of MCOBERTURA-33 ?

Our use case is exactly the same as Mike R. Haller's above. I mentioned it in MCOBERTURA-66.

I am using Hudson as a CI integration server for a multi-module project but I can't use Hudson's trending capabilities over COBERTURA's because the parent project is not getting the aggregated reports.

Hello Cobertura team,

I managed to get the plugin work for multi-module coverage, meaning, a test in module B can participate in the code coverage of module A as well as a global coverage result for all modules.
I still have some cleaning to do and problems to solve before contributing but I have one question.

Is it possible to use java 1.5 or do you wish to stay at java 1.4?
Basically, I wrote tests and code using generics and JUnit 4.8. If java 1.4 is mandatory, I would need to change the code.

Cheers

Hello Anthonin

Could you share your updated plugin/patch so that others could try it too?

For those interested, here is the current work for multi-module coverage
http://github.com/bonnefoa/cobertura-maven-plugin

Just so you know, it's spelled "aggregate" not "agregate"; and the manifold spelling mistakes you've made on words derived from it.

Had a look at the source posted by Anthonin Bonnefoy. It basically works for me (specifying <useConsolidated>true</useConsolidated> in the configuration block). However, it fails if you have package-info.java files in your project packages.

I've attached a patch against the trunk of the cobertura plugin which enables the report mojo to generate aggregate reports. there is a new "aggregate" parameter for the report mojo, which, if enabled, generates a roll-up report in every multi-module project in the current build (so you get aggregation at each level in the multi-module project).

I would also like to add a vote for this. I tried to use James' patch, but my intent was with the maven-pdf-plugin, and that did not seem to work out.

using an <aggregate>true</aggregate> flag is consistent with the surefire report mechanism for combining multi-module reports.

I also like the idea of this plugin, and think it works pretty well. I have only two complaints:

1. I agree that 'agregate' should be renamed to 'aggregate' everywhere to be spelled correctly
2. (This is infinitely more important) When I use the plugin to aggregate the reports, compexity becomes 0 for every single thing, when they are not 0 in the un-aggregated files.

I need complexity to be shown for each of the packages, but I am having a hard time finding where complexity would have gotten dropped from the report, in Anthonin's code. Any ideas how to fix this to show complexity again? Thanks.

@ James Ahlborn- I am new here and am not sure where the trunk you attached your code to is located. Would you mind helping me out?

My code, "cobertura_aggregate.diff", is in the "Attachments" at the top of this bug report.

@ James Ahlborn- I used your patch, and it works well, and is more consolidated than the other option (and spelled correctly).

However, I have the same problem here that I have with his other patch. When I run the <aggregate>true</aggregate>, the aggregated report for the parent projects are complete except missing complexities, but the children each have complete reports, with complexities.

What do you think could be happening? Is the .ser file getting cut when it is aggregated? Please let me know if you have any ideas. Feel free to email my personal email with suggestions.

Thanks.

To anyone who would like to aggregate Cobertura's reports in a multi-module Maven project which has a large number of submodules and/or very long pathnames for each, then use James's patch given above but add a few lines of code into CoberturaReportMojo.java. If it is failing because of these long pathnames, the only thing you will notice is that your reports will say all complexities are 0 when they are not in fact 0.

What's happening is that it uses cmd.exe to run its commands, and then tries to include way too many, huge source files, which goes over the cmd.exe 8191 character limit, failing the report aggregation with a general exception.

Here's what you add to CoberturaReportMojo.java to fix it:

In private void executeReport(File curDataFile, File curOutputDirectory, List curCompileSourceRoots), add:

CommandLineArguments cmdLineArgs;
cmdLineArgs = new CommandLineArguments();
cmdLineArgs.setUseCommandsFile(true);
task.setCmdLineArgs(cmdLineArgs);

after you set the task defaults and specifics, but before you check the formatting. This is enough to change to using the command file to list all of the sources which need to be included instead of listing each explicitly on the command line.

You also need to add an import statement at the top:

import org.codehaus.mojo.cobertura.tasks.CommandLineArguments;

Rebuild and install the plugin, and this should fix these problems. Hope it helps!

I have created an integration test for this patch. However, looking at the existing tests, I don't see how the integration tests verify anything except non-crashing. Am I missing something?

Here is a patch revised to include the command-line fix and an integration test. The integration test can be used for a manual verification that the aggregation indeed aggregates.

Benson,

thanks for your work, but I'm sorry to say that it can't be patched. To be precise: in its current state it's not allowed. The problem is the license of Cobertura. It's conflicting with the Codehaus licenses.
In short this means that the plugin can only use the classes under net.sourceforge.cobertura.ant. This is an ever returning issue which slows down the development of this plugin.

Robert,

Help? I didn't change anything about the Cobertura package usage. Did some previous contributor to the patch do so? Or is the problem the command-line fix I picked up that uses some other codehaus stuff?

Honestly, it seems to me in that case that a fork to github would be in the best interests of all concerned. Cobertura is a GPL beast, and if Codehaus has an Apache-like attitude toward GPL, then pushing it elsewhere seems by far the easiest solution.

I'd be happy to set up a little 'organization' at github and grant access to any existing plugin contributors.

--benson

You're right, it's there already in the original patch, just scanned too quick over the code I guess.
The problems are the n.s.c.c.CoverageDataFileHandler and n.s.c.c.ProjectData but IIRC we don't need those classes if we're able to analyse the xml ourselve. So with some extra coding the plugin should be able to get the same result.

You could post your suggestion to the dev-list to ask what the teammembers think about it.

hmm, interesting problem. wasn't aware of the license conflicts when i wrote the patch. i believe that cobertura has a commandline merge program, so you could probably do the work via a commandline program execution instead of in actual code.

Fix committed in rev 13883.

So how do I use this? I have a simple multi-module project at https://github.com/mfriedenhagen/multi-module-sample consisting of a parent-pom project, one lib (core) and two apps. However I do not get a report at all now, even for the subprojects the cobertura report is missing and no xml files are rendered as well (see http://huschteguzzel.de/hudson/job/test-multimodule/). I use maven-3.0.3 and invoke mvn clean install site. Configuration for cobertura in the parent-pom is at line 200

https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-plugin/src/it/mcobertura-65

See this integration test for an example, and in general use the user list for assistance.

1.
I upgraded to sonar 2.5.1 for my project by putting in the following configuration but do not see the results getting merged.

<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<aggregate>true</aggregate>
</configuration>
</plugin>

Is this configuration setting correct?

2.
We use teamcity for continuous integration and have sonar for doing code analysis.
I see the following warning in the log files during sonar build.

[cobertura] WARN [main] net.sourceforge.cobertura.instrument.ClassInstrumenter - No line number information found for class com.test.BumpKey$1. Perhaps you need to compile with debug=true?

In the build configuration, I have already added "-Ddebug=true" in the "Additional Maven command line parameters" section.
Is there any other change required for getting the anonymous inner classes included in code coverage analysis?

I have not got either patch (2.5-SNAPSHOT or 2.5.1) to work in the way that I understood they would - that of collecting and consolidating coverage statistics across modules, so that if you have a test in module A which exercises classes in modules B and C, then the consolidated report would show classes in all three modules A, B, and C.

Currently the reports either only display classes in the same module, or display classes from other modules with 0% coverage. I accept that you can't expect to cover library classes - but these are Maven modules (as well as libraries), which are treated as 'siblings' under the umbrella of a multi-module POM project.

Maybe what is required is to produce a consolidated instrumented SER file (which does seem to happen - "consolidate-cobertura.ser" is created in the target directory of the parent project) and then drive all coverage from that, rather than individual sub-module SER files. It looks as if both patches are just running tests and generating coverage inside modules, and then consolidating.

@Neil - you are correct. your desired behavior is actually a harder problem. it's not a matter of just having a consolidated ser file, it's a matter of running your tests for module A using instrumented classes for A,B, and C. currently, the instrumented classes only live inside the target directory of the sub-module. however, maven, i believe, actually pulls the other sub-module classes from your local repository (e.g. when you are running tests for A, the classes for B and C are coming from your repository). so you essentially need to temporarily deploy the instrumented classes for B and C to your local repository (and presumably overwrite them when you are done). there may be a way to use some serious maven magic to add the instrumented sub-module classes from B's and C's target directory to A's classpath on the fly during test execution, but that's way beyond my maven plugin knowledge.