Maven Javadoc Plugin
  1. Maven Javadoc Plugin
  2. MJAVADOC-171

Modules in multi-module projects are "built" too often

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.3
    • Fix Version/s: None
    • Labels:
      None
    • Environment:
      Maven 2.0.8, Linux
    • Testcase included:
      yes
    • Number of attachments :
      7

      Description

      In a multi-module project, all modules are "built" twice for each module. This leads to huge performance problems when many modules are in a project. In the attached sample project, the xmlbeans plugin is executed 27 times for a project with one parent module and two submodules. 18 of these executions can be attributed to the javadoc plugin. With version 2.2, only 3 invocations (once for each project) are caused by the javadoc plugin.

      1. 2.2.log
        7 kB
        Stefan Seidel
      2. 2.3.log
        15 kB
        Stefan Seidel
      3. mjavadoc171.patch
        0.5 kB
        Dan Fabulich

        Issue Links

          Activity

          Hide
          Herve Boutemy added a comment - - edited

          ok, I reproduced the case with every m-javadoc-p version, and with Maven 2.2.1

          here is the result (and the script to calculate it):

          $ ./mjavadoc-171.sh 
          Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200)
          Java version: 1.5.0_22
          Java home: /home/opt/jdk1.5.0_22/jre
          Default locale: fr_FR, platform encoding: UTF-8
          OS name: "linux" version: "2.6.38-11-generic" arch: "amd64" Family: "unix"
          m-javadoc-p 2.0: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.1: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.2: 3 xmlbeans, 0 compiler
          m-javadoc-p 2.3: 18 xmlbeans, 0 compiler
          m-javadoc-p 2.4: 0 xmlbeans, 0 compiler
          m-javadoc-p 2.5: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.6: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.6.1: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.7: 24 xmlbeans, 8 compiler
          m-javadoc-p 2.8: 24 xmlbeans, 8 compiler

          Here is a few facts I got that can explain some results:

          • test-javadoc goal was added in m-javadoc-p 2.3
          • javadoc-aggregate and test-aggregate were added in m-javadoc-p 2.5, and they "Invoke the execution of the lifecycle phase generate-sources prior to executing itself."

          this does not really explain everything, but since no reportSet has been defined these aggregate reports are run in addition to non-aggregate reports: see MSITE-454 for thought about avoid this by default, because it is not really expected by users

          some numbers are different from yours: can you run the script on your machine and share your results?

          Show
          Herve Boutemy added a comment - - edited ok, I reproduced the case with every m-javadoc-p version, and with Maven 2.2.1 here is the result (and the script to calculate it): $ ./mjavadoc-171.sh Apache Maven 2.2.1 (r801777; 2009-08-06 21:16:01+0200) Java version: 1.5.0_22 Java home: /home/opt/jdk1.5.0_22/jre Default locale: fr_FR, platform encoding: UTF-8 OS name: "linux" version: "2.6.38-11-generic" arch: "amd64" Family: "unix" m-javadoc-p 2.0: 3 xmlbeans, 0 compiler m-javadoc-p 2.1: 3 xmlbeans, 0 compiler m-javadoc-p 2.2: 3 xmlbeans, 0 compiler m-javadoc-p 2.3: 18 xmlbeans, 0 compiler m-javadoc-p 2.4: 0 xmlbeans, 0 compiler m-javadoc-p 2.5: 24 xmlbeans, 8 compiler m-javadoc-p 2.6: 24 xmlbeans, 8 compiler m-javadoc-p 2.6.1: 24 xmlbeans, 8 compiler m-javadoc-p 2.7: 24 xmlbeans, 8 compiler m-javadoc-p 2.8: 24 xmlbeans, 8 compiler Here is a few facts I got that can explain some results: test-javadoc goal was added in m-javadoc-p 2.3 javadoc-aggregate and test-aggregate were added in m-javadoc-p 2.5, and they "Invoke the execution of the lifecycle phase generate-sources prior to executing itself." this does not really explain everything, but since no reportSet has been defined these aggregate reports are run in addition to non-aggregate reports: see MSITE-454 for thought about avoid this by default, because it is not really expected by users some numbers are different from yours: can you run the script on your machine and share your results?
          Hide
          Herve Boutemy added a comment -

          IMHO, it's about m-javadoc-p having 4 goals, each having @requiresDependencyResolution compile
          and the way Maven core needs to actually run scopes to do the resolution, and it does it multiple times

          I still don't understand everything, nor have an idea if it's a bug or unexpected consequence of a feature

          but it lead me to a workaround: if you define reportSets with less than the 4 reports, you'll divide the number of executions

          if you configure javadoc report only, without test-javadoc or aggregate versions like this:

          <plugin>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>2.8</version>
            <reportSets>
              <reportSet>
                <reports>
                  <report>javadoc</report>
                </reports>
              </reportSet>
            </reportSets>
          </plugin>

          the multiple executions will disappear (and a lot of m-javadoc-p version will simply blow up, I didn't study why, just came to the conclusion that new versions have less bugs than old ones)

          Show
          Herve Boutemy added a comment - IMHO, it's about m-javadoc-p having 4 goals, each having @requiresDependencyResolution compile and the way Maven core needs to actually run scopes to do the resolution, and it does it multiple times I still don't understand everything, nor have an idea if it's a bug or unexpected consequence of a feature but it lead me to a workaround: if you define reportSets with less than the 4 reports, you'll divide the number of executions if you configure javadoc report only, without test-javadoc or aggregate versions like this: <plugin> <artifactId> maven-javadoc-plugin </artifactId> <version> 2.8 </version> <reportSets> <reportSet> <reports> <report> javadoc </report> </reports> </reportSet> </reportSets> </plugin> the multiple executions will disappear (and a lot of m-javadoc-p version will simply blow up, I didn't study why, just came to the conclusion that new versions have less bugs than old ones)
          Hide
          Andreas Horst added a comment -

          We are having similar issues, which are related both to the way module builds are forked for scope resolutions and the javadoc-plugin. Simply put, some of the modules are forked to often. For instance, we have a project with 3 modules and a simple 'mvn clean site' leads to one of the modules being forked in every build, i.e. the parent and ALL 3 modules; YES, it even forks itself. How is this possible? We only use the 'aggregate' goal of the javadoc-plugin defined in the parent POM and according to the log that's what causing this odd forks. Maybe this is related to MSHARED-266?

          Show
          Andreas Horst added a comment - We are having similar issues, which are related both to the way module builds are forked for scope resolutions and the javadoc-plugin. Simply put, some of the modules are forked to often. For instance, we have a project with 3 modules and a simple 'mvn clean site' leads to one of the modules being forked in every build, i.e. the parent and ALL 3 modules; YES, it even forks itself. How is this possible? We only use the 'aggregate' goal of the javadoc-plugin defined in the parent POM and according to the log that's what causing this odd forks. Maybe this is related to MSHARED-266 ?
          Hide
          Herve Boutemy added a comment -

          more information on forked lifecycles added in MSHARED-333, to help diagnose problems

          Show
          Herve Boutemy added a comment - more information on forked lifecycles added in MSHARED-333 , to help diagnose problems
          Hide
          Herve Boutemy added a comment -

          part of the problem is fixed in MSHARED-266 (will work only for Maven 3), the remaining issue is in Maven Core with MNG-2184

          I attached check test, with following results

          Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-14T18:37:52+01:00)
          Maven home: /home/herve/local/maven2
          Java version: 1.7.0_55, vendor: Oracle Corporation
          Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
          Default locale: fr_FR, platform encoding: UTF-8
          OS name: "linux", version: "3.13.0-24-generic", arch: "amd64", family: "unix"
          m-javadoc-p 2.0: 2 xmlbeans, 0 compiler, 2 javadoc
          m-javadoc-p 2.1: 2 xmlbeans, 0 compiler, 2 javadoc
          m-javadoc-p 2.2: 2 xmlbeans, 0 compiler, 2 javadoc
          m-javadoc-p 2.3: 12 xmlbeans, 0 compiler, 4 javadoc
          m-javadoc-p 2.4: 0 xmlbeans, 0 compiler, 4 javadoc
          m-javadoc-p 2.5: 4 xmlbeans, 2 compiler, 4 javadoc
          m-javadoc-p 2.6: 10 xmlbeans, 4 compiler, 6 javadoc
          m-javadoc-p 2.6.1: 10 xmlbeans, 4 compiler, 6 javadoc
          m-javadoc-p 2.7: 10 xmlbeans, 4 compiler, 6 javadoc
          m-javadoc-p 2.8: 10 xmlbeans, 4 compiler, 6 javadoc
          m-javadoc-p 2.9: 10 xmlbeans, 4 compiler, 6 javadoc
          m-javadoc-p 2.9.1: 10 xmlbeans, 4 compiler, 6 javadoc

          6 javadocs is normal: main and test for the 2 modules and aggregated in parent
          4 compiler: should be 2, but hit by MNG-2184 = Maven doesn't remember that compiler was run during parent aggregated test javadoc which forked generate-test-resources phase (which is after compile)
          10 xmlbeans: should be 3, but same MNG-2184 problem, which costs more since xmlbeans is bound to generate-resources phase

          so what could be done in m-site-p (3.4 will contain the fix by default: MSITE-454) and m-javadoc-p is done: remaining problems are in Maven Core with MNG-2184

          (and I now finally understand what is happenning: that was the hardest part of the fix )

          Show
          Herve Boutemy added a comment - part of the problem is fixed in MSHARED-266 (will work only for Maven 3), the remaining issue is in Maven Core with MNG-2184 I attached check test, with following results Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-14T18:37:52+01:00) Maven home: /home/herve/local/maven2 Java version: 1.7.0_55, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre Default locale: fr_FR, platform encoding: UTF-8 OS name: "linux", version: "3.13.0-24-generic", arch: "amd64", family: "unix" m-javadoc-p 2.0: 2 xmlbeans, 0 compiler, 2 javadoc m-javadoc-p 2.1: 2 xmlbeans, 0 compiler, 2 javadoc m-javadoc-p 2.2: 2 xmlbeans, 0 compiler, 2 javadoc m-javadoc-p 2.3: 12 xmlbeans, 0 compiler, 4 javadoc m-javadoc-p 2.4: 0 xmlbeans, 0 compiler, 4 javadoc m-javadoc-p 2.5: 4 xmlbeans, 2 compiler, 4 javadoc m-javadoc-p 2.6: 10 xmlbeans, 4 compiler, 6 javadoc m-javadoc-p 2.6.1: 10 xmlbeans, 4 compiler, 6 javadoc m-javadoc-p 2.7: 10 xmlbeans, 4 compiler, 6 javadoc m-javadoc-p 2.8: 10 xmlbeans, 4 compiler, 6 javadoc m-javadoc-p 2.9: 10 xmlbeans, 4 compiler, 6 javadoc m-javadoc-p 2.9.1: 10 xmlbeans, 4 compiler, 6 javadoc 6 javadocs is normal: main and test for the 2 modules and aggregated in parent 4 compiler: should be 2, but hit by MNG-2184 = Maven doesn't remember that compiler was run during parent aggregated test javadoc which forked generate-test-resources phase (which is after compile) 10 xmlbeans: should be 3, but same MNG-2184 problem, which costs more since xmlbeans is bound to generate-resources phase so what could be done in m-site-p (3.4 will contain the fix by default: MSITE-454 ) and m-javadoc-p is done: remaining problems are in Maven Core with MNG-2184 (and I now finally understand what is happenning: that was the hardest part of the fix )

            People

            • Assignee:
              Herve Boutemy
              Reporter:
              Stefan Seidel
            • Votes:
              16 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: