Maven Javadoc Plugin
  1. Maven Javadoc Plugin
  2. MJAVADOC-171

Modules in multi-module projects are "built" too often

    Details

    • Type: Bug Bug
    • Status: Reopened Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3
    • Fix Version/s: None
    • Labels:
      None
    • Environment:
      Maven 2.0.8, Linux
    • Testcase included:
      yes
    • Number of attachments :
      6

      Description

      In a multi-module project, all modules are "built" twice for each module. This leads to huge performance problems when many modules are in a project. In the attached sample project, the xmlbeans plugin is executed 27 times for a project with one parent module and two submodules. 18 of these executions can be attributed to the javadoc plugin. With version 2.2, only 3 invocations (once for each project) are caused by the javadoc plugin.

        Issue Links

          Activity

          Hide
          Stefan Seidel added a comment -

          A similar problem appears in the surefire-report plugin. I think the execution of all modules within each module during the site generation could be a generic issue that might need to be solved at maven API level.

          Show
          Stefan Seidel added a comment - A similar problem appears in the surefire-report plugin. I think the execution of all modules within each module during the site generation could be a generic issue that might need to be solved at maven API level.
          Hide
          Vincent Siveton added a comment -

          What is your command line? I can not reproduce it yet.

          Show
          Vincent Siveton added a comment - What is your command line? I can not reproduce it yet.
          Hide
          Stefan Seidel added a comment -
          mvn clean site
          

          Sorry, should've said that before.

          Show
          Stefan Seidel added a comment - mvn clean site Sorry, should've said that before.
          Hide
          Vincent Siveton added a comment -

          Don't know about xmlbeans plugin, but the javadoc plugin generates javadoc for main and test so twice sounds correct.

          Could you give me a log file and point me the lack of execution?

          Show
          Vincent Siveton added a comment - Don't know about xmlbeans plugin, but the javadoc plugin generates javadoc for main and test so twice sounds correct. Could you give me a log file and point me the lack of execution?
          Hide
          Stefan Seidel added a comment -

          It's not about twice. It's about all modules are built twice for each module. I have removed the maven-surefire-report-plugin from the parent pom (to show the problem more distinctively) and when I run clean site the following logs come out with version 2.3 and 2.2 respectively. Notice the vast increase of

          [INFO] Building Unnamed - org.test.forked:...
          [INFO] ------------------------------------------------------
          [INFO] [xmlbeans:xmlbeans {execution: default}]
          [INFO] Nothing to generate.
          

          parts from 2.2 to 2.3.

          Show
          Stefan Seidel added a comment - It's not about twice. It's about all modules are built twice for each module. I have removed the maven-surefire-report-plugin from the parent pom (to show the problem more distinctively) and when I run clean site the following logs come out with version 2.3 and 2.2 respectively. Notice the vast increase of [INFO] Building Unnamed - org.test.forked:... [INFO] ------------------------------------------------------ [INFO] [xmlbeans:xmlbeans {execution: default }] [INFO] Nothing to generate. parts from 2.2 to 2.3.
          Hide
          Dan Fabulich added a comment -

          FYI, I can't get maven-javadoc-plugin to "mvn install" from trunk (I get test failures), but the 2.3 tag builds just fine; when I remove the @aggregator tag from the mojo, it seems to work perfectly. I hypothesize that @aggregator is seriously broken, at least for reporting plugins (or perhaps just plugins that need to @execute forked lifecycles like this one), and at least in this case, it seems to be harmful and unnecessary.

          http://www.nabble.com/%40aggregator-mojo-annotation-td15302246s177.html

          Show
          Dan Fabulich added a comment - FYI, I can't get maven-javadoc-plugin to "mvn install" from trunk (I get test failures), but the 2.3 tag builds just fine; when I remove the @aggregator tag from the mojo, it seems to work perfectly. I hypothesize that @aggregator is seriously broken, at least for reporting plugins (or perhaps just plugins that need to @execute forked lifecycles like this one), and at least in this case, it seems to be harmful and unnecessary. http://www.nabble.com/%40aggregator-mojo-annotation-td15302246s177.html
          Hide
          Vincent Siveton added a comment -

          javadoc 2.4 should resolved it

          Show
          Vincent Siveton added a comment - javadoc 2.4 should resolved it
          Hide
          Stefan Seidel added a comment -

          And added it again in 2.5
          Here are the number of xmlbeans occurences by version:
          2.2: 5
          2.3: 20
          2.4: 0
          2.5: 26
          2.6.1: 26
          2.6: 26
          2.7: 26

          AFAICS, only 2.2 behaved correctly.

          Show
          Stefan Seidel added a comment - And added it again in 2.5 Here are the number of xmlbeans occurences by version: 2.2: 5 2.3: 20 2.4: 0 2.5: 26 2.6.1: 26 2.6: 26 2.7: 26 AFAICS, only 2.2 behaved correctly.
          Hide
          Dan Tran added a comment -

          for my case, i only need to build javadoc for one project, under multiple modules build, it triggers all sibbling project to build javadoc

          Show
          Dan Tran added a comment - for my case, i only need to build javadoc for one project, under multiple modules build, it triggers all sibbling project to build javadoc
          Hide
          Steve Jerman added a comment -

          I think I am running into the same problem with version 2.7 and maven 3. It is trying to fork compilations of sibling projects... Basically it ends up running out of heap space.

          Show
          Steve Jerman added a comment - I think I am running into the same problem with version 2.7 and maven 3. It is trying to fork compilations of sibling projects... Basically it ends up running out of heap space.
          Hide
          Andy Schlaikjer added a comment -

          I think I am running into the same problem with version 2.7 and maven 3. It is trying to fork compilations of sibling projects... Basically it ends up running out of heap space.

          I left a note here with a work-around for this issue.

          Show
          Andy Schlaikjer added a comment - I think I am running into the same problem with version 2.7 and maven 3. It is trying to fork compilations of sibling projects... Basically it ends up running out of heap space. I left a note here with a work-around for this issue.
          Hide
          Herve Boutemy added a comment -

          this should be fixed with MJAVADOC-284
          if you're still having problems, please open another Jira issue with a link to this one

          Show
          Herve Boutemy added a comment - this should be fixed with MJAVADOC-284 if you're still having problems, please open another Jira issue with a link to this one
          Hide
          Stefan Seidel added a comment -

          How many times ...

          Sorry, but saying it's fixed doesn't mean it is. Why can you not just add a unit test?

          Here's an addition to the list:
          2.8: 26

          Sorry to be a nuisance, but what can I do to finally get my site build run properly? Especially since 2.2 or 2.4 do not work with the maven-site-plugin:3.0?

          Show
          Stefan Seidel added a comment - How many times ... Sorry, but saying it's fixed doesn't mean it is. Why can you not just add a unit test? Here's an addition to the list: 2.8: 26 Sorry to be a nuisance, but what can I do to finally get my site build run properly? Especially since 2.2 or 2.4 do not work with the maven-site-plugin:3.0?
          Hide
          Herve Boutemy added a comment -

          sorry to repeat myself too: if you're still having problems, please open another Jira issue with a link to this one

          then, please help us make a repeatable IT
          you're giving numbers I can't calculate myself
          No doubt you're facing something
          please help us learn how to calculate same numbers as you: a grep command to show precisely what you are counting, for example

          but please don't reopen this issue 4 months after we tried to work with you: I don't care if the issue is not really resolved, the new issue is needed just to let us work together at reproducing what you are facing

          Show
          Herve Boutemy added a comment - sorry to repeat myself too: if you're still having problems, please open another Jira issue with a link to this one then, please help us make a repeatable IT you're giving numbers I can't calculate myself No doubt you're facing something please help us learn how to calculate same numbers as you: a grep command to show precisely what you are counting, for example but please don't reopen this issue 4 months after we tried to work with you: I don't care if the issue is not really resolved, the new issue is needed just to let us work together at reproducing what you are facing
          Hide
          Herve Boutemy added a comment -

          I marked the resolution as "Cannot Reproduce" if it helps understanding the effective definitive status of this issue (that effectively seems to not be fixed for you, who are the only guy able to be able to say if it is fixed or not)

          Show
          Herve Boutemy added a comment - I marked the resolution as "Cannot Reproduce" if it helps understanding the effective definitive status of this issue (that effectively seems to not be fixed for you, who are the only guy able to be able to say if it is fixed or not)
          Hide
          Stefan Seidel added a comment -

          Hmm.I have created a test project and attached it to this issue long time ago. I've attached the log files and explained (hopefully clearly) what the difference is and what the problem is. Now what else is there what would be different for a new issue? And what makes you sure that you have fixed this issue?

          The numbers I provided are: extract the "mvnexec.zip", run mvn site, count the number of xmlbeans invocations. I hope this is now reproducible.

          Show
          Stefan Seidel added a comment - Hmm.I have created a test project and attached it to this issue long time ago. I've attached the log files and explained (hopefully clearly) what the difference is and what the problem is. Now what else is there what would be different for a new issue? And what makes you sure that you have fixed this issue? The numbers I provided are: extract the "mvnexec.zip", run mvn site, count the number of xmlbeans invocations. I hope this is now reproducible.
          Hide
          Herve Boutemy added a comment -

          Hmmm

          $ unzip ../mvnexec.zip && cd forked && mvn -v && mvn site
          Archive:  ../mvnexec.zip
             creating: forked/
            inflating: forked/pom.xml          
             creating: forked/sub1/
             creating: forked/sub1/src/
             creating: forked/sub1/src/test/
             creating: forked/sub1/src/test/java/
            inflating: forked/sub1/src/test/java/Simple2Test.java  
            inflating: forked/sub1/pom.xml     
             creating: forked/sub2/
             creating: forked/sub2/src/
             creating: forked/sub2/src/test/
             creating: forked/sub2/src/test/java/
            inflating: forked/sub2/src/test/java/SimpleTest.java  
            inflating: forked/sub2/pom.xml     
          Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200)
          Java version: 1.5.0_22
          Java home: /home/opt/jdk1.5.0_22/jre
          Default locale: fr_FR, platform encoding: UTF-8
          OS name: "linux" version: "2.6.38-10-generic" arch: "amd64" Family: "unix"
          [INFO] Scanning for projects...
          [INFO] Reactor build order: 
          [INFO]   Unnamed - org.test.forked:parent:pom:1
          [INFO]   Unnamed - org.test.forked:sub1:jar:1
          [INFO]   Unnamed - org.test.forked:sub2:jar:1
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Unnamed - org.test.forked:parent:pom:1
          [INFO]    task-segment: [site]
          [INFO] ------------------------------------------------------------------------
          [INFO] ------------------------------------------------------------------------
          [ERROR] BUILD ERROR
          [INFO] ------------------------------------------------------------------------
          [INFO] Error building POM (may not be this project's POM).
          
          
          Project ID: org.apache.maven.plugins:maven-surefire-report-plugin
          
          Reason: Error getting POM for 'org.apache.maven.plugins:maven-surefire-report-plugin' from the repository: Failed to resolve artifact, possibly due to a repository list that is not appropriately equipped for this artifact's metadata.
            org.apache.maven.plugins:maven-surefire-report-plugin:pom:2.8.2-SNAPSHOT
          
          from the specified remote repositories:
            Nexus (http://localhost:8081/nexus/content/groups/public)
          
           for project org.apache.maven.plugins:maven-surefire-report-plugin
          
          
          [INFO] ------------------------------------------------------------------------
          [INFO] For more information, run Maven with the -e switch
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: < 1 second
          [INFO] Finished at: Fri Aug 19 22:36:44 CEST 2011
          [INFO] Final Memory: 15M/150M
          [INFO] ------------------------------------------------------------------------
          Show
          Herve Boutemy added a comment - Hmmm $ unzip ../mvnexec.zip && cd forked && mvn -v && mvn site Archive: ../mvnexec.zip creating: forked/ inflating: forked/pom.xml creating: forked/sub1/ creating: forked/sub1/src/ creating: forked/sub1/src/test/ creating: forked/sub1/src/test/java/ inflating: forked/sub1/src/test/java/Simple2Test.java inflating: forked/sub1/pom.xml creating: forked/sub2/ creating: forked/sub2/src/ creating: forked/sub2/src/test/ creating: forked/sub2/src/test/java/ inflating: forked/sub2/src/test/java/SimpleTest.java inflating: forked/sub2/pom.xml Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200) Java version: 1.5.0_22 Java home: /home/opt/jdk1.5.0_22/jre Default locale: fr_FR, platform encoding: UTF-8 OS name: "linux" version: "2.6.38-10-generic" arch: "amd64" Family: "unix" [INFO] Scanning for projects... [INFO] Reactor build order: [INFO] Unnamed - org.test.forked:parent:pom:1 [INFO] Unnamed - org.test.forked:sub1:jar:1 [INFO] Unnamed - org.test.forked:sub2:jar:1 [INFO] ------------------------------------------------------------------------ [INFO] Building Unnamed - org.test.forked:parent:pom:1 [INFO] task-segment: [site] [INFO] ------------------------------------------------------------------------ [INFO] ------------------------------------------------------------------------ [ERROR] BUILD ERROR [INFO] ------------------------------------------------------------------------ [INFO] Error building POM (may not be this project's POM). Project ID: org.apache.maven.plugins:maven-surefire-report-plugin Reason: Error getting POM for 'org.apache.maven.plugins:maven-surefire-report-plugin' from the repository: Failed to resolve artifact, possibly due to a repository list that is not appropriately equipped for this artifact's metadata. org.apache.maven.plugins:maven-surefire-report-plugin:pom:2.8.2-SNAPSHOT from the specified remote repositories: Nexus (http://localhost:8081/nexus/content/groups/public) for project org.apache.maven.plugins:maven-surefire-report-plugin [INFO] ------------------------------------------------------------------------ [INFO] For more information, run Maven with the -e switch [INFO] ------------------------------------------------------------------------ [INFO] Total time: < 1 second [INFO] Finished at: Fri Aug 19 22:36:44 CEST 2011 [INFO] Final Memory: 15M/150M [INFO] ------------------------------------------------------------------------
          Hide
          Stefan Seidel added a comment -

          This seems to be a problem with your local build environment. The maven-users mailing list is quite helpful in such cases.

          Anyway, here's a cleaner version, for convenience already with the plugin version via system property.

          Run:

          unzip mjavadoc-171.zip
          mvn clean site -Djavadoc.plugin.version=2.8 | grep xmlbeans | grep execution | wc -l
          

          (and replace the value for javadoc.plugin.version with the version to test).

          Exact numbers may differ slightly, but the overall tendency is equivalent.

          Show
          Stefan Seidel added a comment - This seems to be a problem with your local build environment. The maven-users mailing list is quite helpful in such cases. Anyway, here's a cleaner version, for convenience already with the plugin version via system property. Run: unzip mjavadoc-171.zip mvn clean site -Djavadoc.plugin.version=2.8 | grep xmlbeans | grep execution | wc -l (and replace the value for javadoc.plugin.version with the version to test). Exact numbers may differ slightly, but the overall tendency is equivalent.
          Hide
          Stefan Seidel added a comment -

          Oh, and the "Maven 3" part of the pom.xml only is there to show that with Maven 3 and the maven-site-plugin:3.0 the problem is the same (even though only versions 2.7 and 2.8 of maven-javadoc-plugin run sucessfully under Maven 3 + maven-site-plugin 3.0).

          Show
          Stefan Seidel added a comment - Oh, and the "Maven 3" part of the pom.xml only is there to show that with Maven 3 and the maven-site-plugin:3.0 the problem is the same (even though only versions 2.7 and 2.8 of maven-javadoc-plugin run sucessfully under Maven 3 + maven-site-plugin 3.0).
          Hide
          Herve Boutemy added a comment -

          ok, I can reproduce the issue now
          now I'll try to understand why it is doing such executions...

          Show
          Herve Boutemy added a comment - ok, I can reproduce the issue now now I'll try to understand why it is doing such executions...
          Hide
          Herve Boutemy added a comment - - edited

          ok, I reproduced the case with every m-javadoc-p version, and with Maven 2.2.1

          here is the result (and the script to calculate it):

          $ ./mjavadoc-171.sh 
          Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200)
          Java version: 1.5.0_22
          Java home: /home/opt/jdk1.5.0_22/jre
          Default locale: fr_FR, platform encoding: UTF-8
          OS name: "linux" version: "2.6.38-11-generic" arch: "amd64" Family: "unix"
          m-javadoc-p 2.0: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.1: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.2: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.3: 18 xmlbeans, 0 compiler
          m-javadoc-p 2.4: 0 xmlbeans, 0 compiler
          m-javadoc-p 2.5: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.6: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.6.1: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.7: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.8: 24 xmlbeans, 8 compiler

          Here is a few facts I got that can explain some results:

          • test-javadoc goal was added in m-javadoc-p 2.3
          • javadoc-aggregate and test-aggregate were added in m-javadoc-p 2.5, and they "Invoke the execution of the lifecycle phase generate-sources prior to executing itself."

          this does not really explain everything, but since no reportSet has been defined these aggregate reports are run in addition to non-aggregate reports: see MSITE-454 for thought about avoid this by default, because it is not really expected by users

          some numbers are different from yours: can you run the script on your machine and share your results?

          Show
          Herve Boutemy added a comment - - edited ok, I reproduced the case with every m-javadoc-p version, and with Maven 2.2.1 here is the result (and the script to calculate it): $ ./mjavadoc-171.sh Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200) Java version: 1.5.0_22 Java home: /home/opt/jdk1.5.0_22/jre Default locale: fr_FR, platform encoding: UTF-8 OS name: "linux" version: "2.6.38-11-generic" arch: "amd64" Family: "unix" m-javadoc-p 2.0: 3 xmlbeans, 0 compiler m-javadoc-p 2.1: 3 xmlbeans, 0 compiler m-javadoc-p 2.2: 3 xmlbeans, 0 compiler m-javadoc-p 2.3: 18 xmlbeans, 0 compiler m-javadoc-p 2.4: 0 xmlbeans, 0 compiler m-javadoc-p 2.5: 24 xmlbeans, 8 compiler m-javadoc-p 2.6: 24 xmlbeans, 8 compiler m-javadoc-p 2.6.1: 24 xmlbeans, 8 compiler m-javadoc-p 2.7: 24 xmlbeans, 8 compiler m-javadoc-p 2.8: 24 xmlbeans, 8 compiler Here is a few facts I got that can explain some results: test-javadoc goal was added in m-javadoc-p 2.3 javadoc-aggregate and test-aggregate were added in m-javadoc-p 2.5, and they "Invoke the execution of the lifecycle phase generate-sources prior to executing itself." this does not really explain everything, but since no reportSet has been defined these aggregate reports are run in addition to non-aggregate reports: see MSITE-454 for thought about avoid this by default, because it is not really expected by users some numbers are different from yours: can you run the script on your machine and share your results?
          Hide
          Herve Boutemy added a comment -

          IMHO, it's about m-javadoc-p having 4 goals, each having @requiresDependencyResolution compile
          and the way Maven core needs to actually run scopes to do the resolution, and it does it multiple times

          I still don't understand everything, nor have an idea if it's a bug or unexpected consequence of a feature

          but it lead me to a workaround: if you define reportSets with less than the 4 reports, you'll divide the number of executions

          if you configure javadoc report only, without test-javadoc or aggregate versions like this:

          <plugin>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>2.8</version>
            <reportSets>
              <reportSet>
                <reports>
                  <report>javadoc</report>
                </reports>
              </reportSet>
            </reportSets>
          </plugin>

          the multiple executions will disappear (and a lot of m-javadoc-p version will simply blow up, I didn't study why, just came to the conclusion that new versions have less bugs than old ones)

          Show
          Herve Boutemy added a comment - IMHO, it's about m-javadoc-p having 4 goals, each having @requiresDependencyResolution compile and the way Maven core needs to actually run scopes to do the resolution, and it does it multiple times I still don't understand everything, nor have an idea if it's a bug or unexpected consequence of a feature but it lead me to a workaround: if you define reportSets with less than the 4 reports, you'll divide the number of executions if you configure javadoc report only, without test-javadoc or aggregate versions like this: <plugin> <artifactId> maven-javadoc-plugin </artifactId> <version> 2.8 </version> <reportSets> <reportSet> <reports> <report> javadoc </report> </reports> </reportSet> </reportSets> </plugin> the multiple executions will disappear (and a lot of m-javadoc-p version will simply blow up, I didn't study why, just came to the conclusion that new versions have less bugs than old ones)
          Hide
          Andreas Horst added a comment -

          We are having similar issues, which are related both to the way module builds are forked for scope resolutions and the javadoc-plugin. Simply put, some of the modules are forked to often. For instance, we have a project with 3 modules and a simple 'mvn clean site' leads to one of the modules being forked in every build, i.e. the parent and ALL 3 modules; YES, it even forks itself. How is this possible? We only use the 'aggregate' goal of the javadoc-plugin defined in the parent POM and according to the log that's what causing this odd forks. Maybe this is related to MSHARED-266?

          Show
          Andreas Horst added a comment - We are having similar issues, which are related both to the way module builds are forked for scope resolutions and the javadoc-plugin. Simply put, some of the modules are forked to often. For instance, we have a project with 3 modules and a simple 'mvn clean site' leads to one of the modules being forked in every build, i.e. the parent and ALL 3 modules; YES, it even forks itself. How is this possible? We only use the 'aggregate' goal of the javadoc-plugin defined in the parent POM and according to the log that's what causing this odd forks. Maybe this is related to MSHARED-266 ?

            People

            • Assignee:
              Unassigned
              Reporter:
              Stefan Seidel
            • Votes:
              16 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated: