Mojo's Cobertura Maven Plugin
  1. Mojo's Cobertura Maven Plugin
  2. MCOBERTURA-65

Cobertura should aggregate results of multi-module builds

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.5
    • Labels:
      None
    • Environment:
      Windows
    • Number of attachments :
      2

      Description

      When I run a cobertura report on parent pom project that has child modules I only get reports generated for each module, I also want to see a combined report in the parent.

      1. cobertura_aggregate.diff
        15 kB
        James Ahlborn
      2. MCOBERTURA-65.patch
        25 kB
        Benson Margulies

        Activity

        Hide
        Jim Sellers added a comment -

        You might want to check out the dashboard plugin for this.
        http://mojo.codehaus.org/dashboard-maven-plugin/

        Show
        Jim Sellers added a comment - You might want to check out the dashboard plugin for this. http://mojo.codehaus.org/dashboard-maven-plugin/
        Hide
        Mike R. Haller added a comment -

        Does this include the possibility of having the code under test and the test cases in separate modules?
        We've got some very large projects where integration tests are located in a separate module, but we still would like to have coverage reports on that.

        Show
        Mike R. Haller added a comment - Does this include the possibility of having the code under test and the test cases in separate modules? We've got some very large projects where integration tests are located in a separate module, but we still would like to have coverage reports on that.
        Hide
        John Allen added a comment - - edited

        this is very useful functionality, clover2 provides a great example. will into it

        Show
        John Allen added a comment - - edited this is very useful functionality, clover2 provides a great example. will into it
        Hide
        David Hoffer added a comment -

        While the dashboard plugin might be useful, from their documentation it looks like this is a summery report. I want to see a combined cobertura report. I have too many modules for opening each one to be effective.

        If the dashboard provided links to the cobertura reports this might be workable.

        Show
        David Hoffer added a comment - While the dashboard plugin might be useful, from their documentation it looks like this is a summery report. I want to see a combined cobertura report. I have too many modules for opening each one to be effective. If the dashboard provided links to the cobertura reports this might be workable.
        Hide
        Stevo Slavic added a comment -

        Isn't this issue actually duplicate of MCOBERTURA-33 ?

        Show
        Stevo Slavic added a comment - Isn't this issue actually duplicate of MCOBERTURA-33 ?
        Hide
        Pawel Paprota added a comment -

        Our use case is exactly the same as Mike R. Haller's above. I mentioned it in MCOBERTURA-66.

        Show
        Pawel Paprota added a comment - Our use case is exactly the same as Mike R. Haller's above. I mentioned it in MCOBERTURA-66 .
        Hide
        Bernardo Gomez Palacio added a comment -

        I am using Hudson as a CI integration server for a multi-module project but I can't use Hudson's trending capabilities over COBERTURA's because the parent project is not getting the aggregated reports.

        Show
        Bernardo Gomez Palacio added a comment - I am using Hudson as a CI integration server for a multi-module project but I can't use Hudson's trending capabilities over COBERTURA's because the parent project is not getting the aggregated reports.
        Hide
        Anthonin Bonnefoy added a comment -

        Hello Cobertura team,

        I managed to get the plugin work for multi-module coverage, meaning, a test in module B can participate in the code coverage of module A as well as a global coverage result for all modules.
        I still have some cleaning to do and problems to solve before contributing but I have one question.

        Is it possible to use java 1.5 or do you wish to stay at java 1.4?
        Basically, I wrote tests and code using generics and JUnit 4.8. If java 1.4 is mandatory, I would need to change the code.

        Cheers

        Show
        Anthonin Bonnefoy added a comment - Hello Cobertura team, I managed to get the plugin work for multi-module coverage, meaning, a test in module B can participate in the code coverage of module A as well as a global coverage result for all modules. I still have some cleaning to do and problems to solve before contributing but I have one question. Is it possible to use java 1.5 or do you wish to stay at java 1.4? Basically, I wrote tests and code using generics and JUnit 4.8. If java 1.4 is mandatory, I would need to change the code. Cheers
        Hide
        Asankha Perera added a comment -

        Hello Anthonin

        Could you share your updated plugin/patch so that others could try it too?

        Show
        Asankha Perera added a comment - Hello Anthonin Could you share your updated plugin/patch so that others could try it too?
        Hide
        Anthonin Bonnefoy added a comment -

        For those interested, here is the current work for multi-module coverage
        http://github.com/bonnefoa/cobertura-maven-plugin

        Show
        Anthonin Bonnefoy added a comment - For those interested, here is the current work for multi-module coverage http://github.com/bonnefoa/cobertura-maven-plugin
        Hide
        jieryn added a comment -

        Just so you know, it's spelled "aggregate" not "agregate"; and the manifold spelling mistakes you've made on words derived from it.

        Show
        jieryn added a comment - Just so you know, it's spelled "aggregate" not "agregate"; and the manifold spelling mistakes you've made on words derived from it.
        Hide
        Cullen Kehoe added a comment -

        Had a look at the source posted by Anthonin Bonnefoy. It basically works for me (specifying <useConsolidated>true</useConsolidated> in the configuration block). However, it fails if you have package-info.java files in your project packages.

        Show
        Cullen Kehoe added a comment - Had a look at the source posted by Anthonin Bonnefoy. It basically works for me (specifying <useConsolidated>true</useConsolidated> in the configuration block). However, it fails if you have package-info.java files in your project packages.
        Hide
        James Ahlborn added a comment -

        I've attached a patch against the trunk of the cobertura plugin which enables the report mojo to generate aggregate reports. there is a new "aggregate" parameter for the report mojo, which, if enabled, generates a roll-up report in every multi-module project in the current build (so you get aggregation at each level in the multi-module project).

        Show
        James Ahlborn added a comment - I've attached a patch against the trunk of the cobertura plugin which enables the report mojo to generate aggregate reports. there is a new "aggregate" parameter for the report mojo, which, if enabled, generates a roll-up report in every multi-module project in the current build (so you get aggregation at each level in the multi-module project).
        Hide
        Darren Hartford added a comment - - edited

        I would also like to add a vote for this. I tried to use James' patch, but my intent was with the maven-pdf-plugin, and that did not seem to work out.

        using an <aggregate>true</aggregate> flag is consistent with the surefire report mechanism for combining multi-module reports.

        Show
        Darren Hartford added a comment - - edited I would also like to add a vote for this. I tried to use James' patch, but my intent was with the maven-pdf-plugin, and that did not seem to work out. using an <aggregate>true</aggregate> flag is consistent with the surefire report mechanism for combining multi-module reports.
        Hide
        T G added a comment -

        I also like the idea of this plugin, and think it works pretty well. I have only two complaints:

        1. I agree that 'agregate' should be renamed to 'aggregate' everywhere to be spelled correctly
        2. (This is infinitely more important) When I use the plugin to aggregate the reports, compexity becomes 0 for every single thing, when they are not 0 in the un-aggregated files.

        I need complexity to be shown for each of the packages, but I am having a hard time finding where complexity would have gotten dropped from the report, in Anthonin's code. Any ideas how to fix this to show complexity again? Thanks.

        Show
        T G added a comment - I also like the idea of this plugin, and think it works pretty well. I have only two complaints: 1. I agree that 'agregate' should be renamed to 'aggregate' everywhere to be spelled correctly 2. (This is infinitely more important) When I use the plugin to aggregate the reports, compexity becomes 0 for every single thing, when they are not 0 in the un-aggregated files. I need complexity to be shown for each of the packages, but I am having a hard time finding where complexity would have gotten dropped from the report, in Anthonin's code. Any ideas how to fix this to show complexity again? Thanks.
        Hide
        T G added a comment -

        @ James Ahlborn- I am new here and am not sure where the trunk you attached your code to is located. Would you mind helping me out?

        Show
        T G added a comment - @ James Ahlborn- I am new here and am not sure where the trunk you attached your code to is located. Would you mind helping me out?
        Hide
        James Ahlborn added a comment -

        My code, "cobertura_aggregate.diff", is in the "Attachments" at the top of this bug report.

        Show
        James Ahlborn added a comment - My code, "cobertura_aggregate.diff", is in the "Attachments" at the top of this bug report.
        Hide
        T G added a comment -

        @ James Ahlborn- I used your patch, and it works well, and is more consolidated than the other option (and spelled correctly).

        However, I have the same problem here that I have with his other patch. When I run the <aggregate>true</aggregate>, the aggregated report for the parent projects are complete except missing complexities, but the children each have complete reports, with complexities.

        What do you think could be happening? Is the .ser file getting cut when it is aggregated? Please let me know if you have any ideas. Feel free to email my personal email with suggestions.

        Thanks.

        Show
        T G added a comment - @ James Ahlborn- I used your patch, and it works well, and is more consolidated than the other option (and spelled correctly). However, I have the same problem here that I have with his other patch. When I run the <aggregate>true</aggregate>, the aggregated report for the parent projects are complete except missing complexities, but the children each have complete reports, with complexities. What do you think could be happening? Is the .ser file getting cut when it is aggregated? Please let me know if you have any ideas. Feel free to email my personal email with suggestions. Thanks.
        Hide
        T G added a comment -

        To anyone who would like to aggregate Cobertura's reports in a multi-module Maven project which has a large number of submodules and/or very long pathnames for each, then use James's patch given above but add a few lines of code into CoberturaReportMojo.java. If it is failing because of these long pathnames, the only thing you will notice is that your reports will say all complexities are 0 when they are not in fact 0.

        What's happening is that it uses cmd.exe to run its commands, and then tries to include way too many, huge source files, which goes over the cmd.exe 8191 character limit, failing the report aggregation with a general exception.

        Here's what you add to CoberturaReportMojo.java to fix it:

        In private void executeReport(File curDataFile, File curOutputDirectory, List curCompileSourceRoots), add:

        CommandLineArguments cmdLineArgs;
        cmdLineArgs = new CommandLineArguments();
        cmdLineArgs.setUseCommandsFile(true);
        task.setCmdLineArgs(cmdLineArgs);

        after you set the task defaults and specifics, but before you check the formatting. This is enough to change to using the command file to list all of the sources which need to be included instead of listing each explicitly on the command line.

        You also need to add an import statement at the top:

        import org.codehaus.mojo.cobertura.tasks.CommandLineArguments;

        Rebuild and install the plugin, and this should fix these problems. Hope it helps!

        Show
        T G added a comment - To anyone who would like to aggregate Cobertura's reports in a multi-module Maven project which has a large number of submodules and/or very long pathnames for each, then use James's patch given above but add a few lines of code into CoberturaReportMojo.java. If it is failing because of these long pathnames, the only thing you will notice is that your reports will say all complexities are 0 when they are not in fact 0. What's happening is that it uses cmd.exe to run its commands, and then tries to include way too many, huge source files, which goes over the cmd.exe 8191 character limit, failing the report aggregation with a general exception. Here's what you add to CoberturaReportMojo.java to fix it: In private void executeReport(File curDataFile, File curOutputDirectory, List curCompileSourceRoots), add: CommandLineArguments cmdLineArgs; cmdLineArgs = new CommandLineArguments(); cmdLineArgs.setUseCommandsFile(true); task.setCmdLineArgs(cmdLineArgs); after you set the task defaults and specifics, but before you check the formatting. This is enough to change to using the command file to list all of the sources which need to be included instead of listing each explicitly on the command line. You also need to add an import statement at the top: import org.codehaus.mojo.cobertura.tasks.CommandLineArguments; Rebuild and install the plugin, and this should fix these problems. Hope it helps!
        Hide
        Benson Margulies added a comment -

        I have created an integration test for this patch. However, looking at the existing tests, I don't see how the integration tests verify anything except non-crashing. Am I missing something?

        Show
        Benson Margulies added a comment - I have created an integration test for this patch. However, looking at the existing tests, I don't see how the integration tests verify anything except non-crashing. Am I missing something?
        Hide
        Benson Margulies added a comment -

        Here is a patch revised to include the command-line fix and an integration test. The integration test can be used for a manual verification that the aggregation indeed aggregates.

        Show
        Benson Margulies added a comment - Here is a patch revised to include the command-line fix and an integration test. The integration test can be used for a manual verification that the aggregation indeed aggregates.
        Hide
        Robert Scholte added a comment -

        Benson,

        thanks for your work, but I'm sorry to say that it can't be patched. To be precise: in its current state it's not allowed. The problem is the license of Cobertura. It's conflicting with the Codehaus licenses.
        In short this means that the plugin can only use the classes under net.sourceforge.cobertura.ant. This is an ever returning issue which slows down the development of this plugin.

        Show
        Robert Scholte added a comment - Benson, thanks for your work, but I'm sorry to say that it can't be patched. To be precise: in its current state it's not allowed. The problem is the license of Cobertura . It's conflicting with the Codehaus licenses . In short this means that the plugin can only use the classes under net.sourceforge.cobertura.ant . This is an ever returning issue which slows down the development of this plugin.
        Hide
        Benson Margulies added a comment -

        Robert,

        Help? I didn't change anything about the Cobertura package usage. Did some previous contributor to the patch do so? Or is the problem the command-line fix I picked up that uses some other codehaus stuff?

        Honestly, it seems to me in that case that a fork to github would be in the best interests of all concerned. Cobertura is a GPL beast, and if Codehaus has an Apache-like attitude toward GPL, then pushing it elsewhere seems by far the easiest solution.

        I'd be happy to set up a little 'organization' at github and grant access to any existing plugin contributors.

        --benson

        Show
        Benson Margulies added a comment - Robert, Help? I didn't change anything about the Cobertura package usage. Did some previous contributor to the patch do so? Or is the problem the command-line fix I picked up that uses some other codehaus stuff? Honestly, it seems to me in that case that a fork to github would be in the best interests of all concerned. Cobertura is a GPL beast, and if Codehaus has an Apache-like attitude toward GPL, then pushing it elsewhere seems by far the easiest solution. I'd be happy to set up a little 'organization' at github and grant access to any existing plugin contributors. --benson
        Hide
        Robert Scholte added a comment -

        You're right, it's there already in the original patch, just scanned too quick over the code I guess.
        The problems are the n.s.c.c.CoverageDataFileHandler and n.s.c.c.ProjectData but IIRC we don't need those classes if we're able to analyse the xml ourselve. So with some extra coding the plugin should be able to get the same result.

        You could post your suggestion to the dev-list to ask what the teammembers think about it.

        Show
        Robert Scholte added a comment - You're right, it's there already in the original patch, just scanned too quick over the code I guess. The problems are the n.s.c.c.CoverageDataFileHandler and n.s.c.c.ProjectData but IIRC we don't need those classes if we're able to analyse the xml ourselve. So with some extra coding the plugin should be able to get the same result. You could post your suggestion to the dev-list to ask what the teammembers think about it.
        Hide
        James Ahlborn added a comment -

        hmm, interesting problem. wasn't aware of the license conflicts when i wrote the patch. i believe that cobertura has a commandline merge program, so you could probably do the work via a commandline program execution instead of in actual code.

        Show
        James Ahlborn added a comment - hmm, interesting problem. wasn't aware of the license conflicts when i wrote the patch. i believe that cobertura has a commandline merge program, so you could probably do the work via a commandline program execution instead of in actual code.
        Hide
        Benson Margulies added a comment -

        Fix committed in rev 13883.

        Show
        Benson Margulies added a comment - Fix committed in rev 13883.
        Hide
        Mirko Friedenhagen added a comment - - edited

        So how do I use this? I have a simple multi-module project at https://github.com/mfriedenhagen/multi-module-sample consisting of a parent-pom project, one lib (core) and two apps. However I do not get a report at all now, even for the subprojects the cobertura report is missing and no xml files are rendered as well (see http://huschteguzzel.de/hudson/job/test-multimodule/). I use maven-3.0.3 and invoke mvn clean install site. Configuration for cobertura in the parent-pom is at line 200

        Show
        Mirko Friedenhagen added a comment - - edited So how do I use this? I have a simple multi-module project at https://github.com/mfriedenhagen/multi-module-sample consisting of a parent-pom project, one lib (core) and two apps. However I do not get a report at all now, even for the subprojects the cobertura report is missing and no xml files are rendered as well (see http://huschteguzzel.de/hudson/job/test-multimodule/ ). I use maven-3.0.3 and invoke mvn clean install site . Configuration for cobertura in the parent-pom is at line 200
        Hide
        Benson Margulies added a comment -

        https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-plugin/src/it/mcobertura-65

        See this integration test for an example, and in general use the user list for assistance.

        Show
        Benson Margulies added a comment - https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-plugin/src/it/mcobertura-65 See this integration test for an example, and in general use the user list for assistance.
        Hide
        Miten Morakhia added a comment -

        1.
        I upgraded to sonar 2.5.1 for my project by putting in the following configuration but do not see the results getting merged.

        <reporting>
        <plugins>
        <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.5.1</version>
        <configuration>
        <aggregate>true</aggregate>
        </configuration>
        </plugin>

        Is this configuration setting correct?

        2.
        We use teamcity for continuous integration and have sonar for doing code analysis.
        I see the following warning in the log files during sonar build.

        [cobertura] WARN [main] net.sourceforge.cobertura.instrument.ClassInstrumenter - No line number information found for class com.test.BumpKey$1. Perhaps you need to compile with debug=true?

        In the build configuration, I have already added "-Ddebug=true" in the "Additional Maven command line parameters" section.
        Is there any other change required for getting the anonymous inner classes included in code coverage analysis?

        Show
        Miten Morakhia added a comment - 1. I upgraded to sonar 2.5.1 for my project by putting in the following configuration but do not see the results getting merged. <reporting> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>cobertura-maven-plugin</artifactId> <version>2.5.1</version> <configuration> <aggregate>true</aggregate> </configuration> </plugin> Is this configuration setting correct? 2. We use teamcity for continuous integration and have sonar for doing code analysis. I see the following warning in the log files during sonar build. [cobertura] WARN [main] net.sourceforge.cobertura.instrument.ClassInstrumenter - No line number information found for class com.test.BumpKey$1. Perhaps you need to compile with debug=true? In the build configuration, I have already added "-Ddebug=true" in the "Additional Maven command line parameters" section. Is there any other change required for getting the anonymous inner classes included in code coverage analysis?
        Hide
        Neil Ingle added a comment -

        I have not got either patch (2.5-SNAPSHOT or 2.5.1) to work in the way that I understood they would - that of collecting and consolidating coverage statistics across modules, so that if you have a test in module A which exercises classes in modules B and C, then the consolidated report would show classes in all three modules A, B, and C.

        Currently the reports either only display classes in the same module, or display classes from other modules with 0% coverage. I accept that you can't expect to cover library classes - but these are Maven modules (as well as libraries), which are treated as 'siblings' under the umbrella of a multi-module POM project.

        Maybe what is required is to produce a consolidated instrumented SER file (which does seem to happen - "consolidate-cobertura.ser" is created in the target directory of the parent project) and then drive all coverage from that, rather than individual sub-module SER files. It looks as if both patches are just running tests and generating coverage inside modules, and then consolidating.

        Show
        Neil Ingle added a comment - I have not got either patch (2.5-SNAPSHOT or 2.5.1) to work in the way that I understood they would - that of collecting and consolidating coverage statistics across modules, so that if you have a test in module A which exercises classes in modules B and C, then the consolidated report would show classes in all three modules A, B, and C. Currently the reports either only display classes in the same module, or display classes from other modules with 0% coverage. I accept that you can't expect to cover library classes - but these are Maven modules (as well as libraries), which are treated as 'siblings' under the umbrella of a multi-module POM project. Maybe what is required is to produce a consolidated instrumented SER file (which does seem to happen - "consolidate-cobertura.ser" is created in the target directory of the parent project) and then drive all coverage from that, rather than individual sub-module SER files. It looks as if both patches are just running tests and generating coverage inside modules, and then consolidating.
        Hide
        James Ahlborn added a comment -

        @Neil - you are correct. your desired behavior is actually a harder problem. it's not a matter of just having a consolidated ser file, it's a matter of running your tests for module A using instrumented classes for A,B, and C. currently, the instrumented classes only live inside the target directory of the sub-module. however, maven, i believe, actually pulls the other sub-module classes from your local repository (e.g. when you are running tests for A, the classes for B and C are coming from your repository). so you essentially need to temporarily deploy the instrumented classes for B and C to your local repository (and presumably overwrite them when you are done). there may be a way to use some serious maven magic to add the instrumented sub-module classes from B's and C's target directory to A's classpath on the fly during test execution, but that's way beyond my maven plugin knowledge.

        Show
        James Ahlborn added a comment - @Neil - you are correct. your desired behavior is actually a harder problem. it's not a matter of just having a consolidated ser file, it's a matter of running your tests for module A using instrumented classes for A,B, and C. currently, the instrumented classes only live inside the target directory of the sub-module. however, maven, i believe, actually pulls the other sub-module classes from your local repository (e.g. when you are running tests for A, the classes for B and C are coming from your repository). so you essentially need to temporarily deploy the instrumented classes for B and C to your local repository (and presumably overwrite them when you are done). there may be a way to use some serious maven magic to add the instrumented sub-module classes from B's and C's target directory to A's classpath on the fly during test execution, but that's way beyond my maven plugin knowledge.

          People

          • Assignee:
            Benson Margulies
            Reporter:
            David Hoffer
          • Votes:
            70 Vote for this issue
            Watchers:
            56 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: