Maven 2 & 3
  1. Maven 2 & 3
  2. MNG-4301

Invalid checksums on deploy when using webdav without extension

    Details

    • Complexity:
      Intermediate
    • Number of attachments :
      0

      Description

      With maven 2.2.1, our deployments via webdav are producing invalid checksums, similar to the issue reported in MNG-4235.

      From maven 2.0.8 and previous, the following build extension was required to deploy via webdav:

      <extensions>
      <extension>
      <groupId>org.apache.maven.wagon</groupId>
      <artifactId>wagon-webdav</artifactId>
      <version>1.0-beta-2</version>
      </extension>
      </extensions>

      Starting with maven 2.0.9 (see MNG-3418), webdav was included by default and the aforementioned build extension must be removed from the project. If it was included in the project the deployment would fail as Maven would report multiple versions of wagon-webdav present.

      With maven 2.2.0, our deployment suffered from invalid checksums reported in MNG-4235.

      With maven 2.2.1, we still see the invalid checksums on deployment as reported in MNG-4235. However, I've found that if you add the above build extension into the project, we don't experience this issue (of generating invalid checksums). Is this workaround an intentional change brought about by the fix of MNG-4235?

        Issue Links

          Activity

          Hide
          Dave Cheney added a comment -

          I can confirm that I have seen invalid checksums produced via the webdav wagon using version 2.2.1

          Show
          Dave Cheney added a comment - I can confirm that I have seen invalid checksums produced via the webdav wagon using version 2.2.1
          Hide
          John Casey added a comment -

          The problem may be that preemptive authentication was turned off by default for httpclient in the 1.0-beta-6 release, as was the case for the non-lightweight http wagon. I suspect this is the case in the jackrabbit-webdav as well, since IIRC they both use the same common abstract class. Since preemptive auth is disabled, Maven waits for the server to challenge the request then re-sends the data with the credentials. The problem is that the checksum stream observer is unaware that this challenge has taken place, so it sees the data go by twice without realizing it, thereby skewing the checksums.

          To workaround the problem, you can re-enable preemptive authentication. The information is here:

          http://maven.apache.org/guides/mini/guide-http-settings.html

          But essentially, you'll need to specify something like the following in your settings.xml:

          <settings>
            <servers>
              <server>
                <id>my-server</id>
                <configuration>
                  <httpConfiguration>
                    <put>
                      <params>
                        <param>
                          <name>http.authentication.preemptive</name>
                          <value>%b,true</value>
                        </param>
                      </params>
                    </put>
                  </httpConfiguration>
                </configuration>
              </server>
            </servers>
          </settings>
          
          Show
          John Casey added a comment - The problem may be that preemptive authentication was turned off by default for httpclient in the 1.0-beta-6 release, as was the case for the non-lightweight http wagon. I suspect this is the case in the jackrabbit-webdav as well, since IIRC they both use the same common abstract class. Since preemptive auth is disabled, Maven waits for the server to challenge the request then re-sends the data with the credentials. The problem is that the checksum stream observer is unaware that this challenge has taken place, so it sees the data go by twice without realizing it, thereby skewing the checksums. To workaround the problem, you can re-enable preemptive authentication. The information is here: http://maven.apache.org/guides/mini/guide-http-settings.html But essentially, you'll need to specify something like the following in your settings.xml: <settings> <servers> <server> <id> my-server </id> <configuration> <httpConfiguration> <put> <params> <param> <name> http.authentication.preemptive </name> <value> %b,true </value> </param> </params> </put> </httpConfiguration> </configuration> </server> </servers> </settings>
          Hide
          Stephen Pack added a comment -

          I believe the previous comment is the workaround for maven 2.2.0; for maven 2.2.1, you need to add <wagonProvider>httpclient</wagonProvider> in the configuration. Without that addition, I got the exception "Cannot find setter nor field in org.apache.maven.wagon.providers.http.LightweightHttpWagon for 'httpConfiguration'"

          Show
          Stephen Pack added a comment - I believe the previous comment is the workaround for maven 2.2.0; for maven 2.2.1, you need to add <wagonProvider>httpclient</wagonProvider> in the configuration. Without that addition, I got the exception "Cannot find setter nor field in org.apache.maven.wagon.providers.http.LightweightHttpWagon for 'httpConfiguration'"
          Hide
          deckrider added a comment -

          I tried the following ~/.m2/settings.xml which did not work with maven 2.2.1 (the double md5 issue still exists):


           
          
            <servers>
              <server>
                <id>deployment.webdav</id>
                <username>foo</username>
                <password>bar</password>
                <configuration>
                  <wagonProvider>httpclient</wagonProvider>
                  <httpConfiguration>
                    <put>
                      <params>
                        <param>
                          <name>http.authentication.preemptive</name>
                          <value>%b,true</value>
                        </param>
                      </params>
                    </all>
                  </httpConfiguration>
                </configuration>
              </server>
            </servers>
          
          
          Show
          deckrider added a comment - I tried the following ~/.m2/settings.xml which did not work with maven 2.2.1 (the double md5 issue still exists): <servers> <server> <id>deployment.webdav</id> <username>foo</username> <password>bar</password> <configuration> <wagonProvider>httpclient</wagonProvider> <httpConfiguration> <put> <params> <param> <name>http.authentication.preemptive</name> <value>%b,true</value> </param> </params> </all> </httpConfiguration> </configuration> </server> </servers>
          Hide
          gesha added a comment -

          @deckrider

          thankx for valuable tipps :/ Result is attached>

          + Error stacktraces are turned on.
          Error reading settings.xml: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17)
          Line: 35
          Column: 17
          Error stacktrace:
          org.apache.maven.SettingsConfigurationException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:1
          7)
          Line: 35
          Column: 17
          at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:432)
          at org.apache.maven.cli.MavenCli.main(MavenCli.java:202)
          at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
          at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
          at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
          at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
          Caused by: org.codehaus.plexus.util.xml.pull.XmlPullParserException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n
          </all>... @35:17)
          at hidden.org.codehaus.plexus.util.xml.pull.MXParser.parseEndTag(MXParser.java:1708)
          at hidden.org.codehaus.plexus.util.xml.pull.MXParser.nextImpl(MXParser.java:1143)
          at hidden.org.codehaus.plexus.util.xml.pull.MXParser.next(MXParser.java:1105)
          at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:179)
          at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:86)
          at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseServer(SettingsXpp3Reader.java:1062)
          at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseSettings(SettingsXpp3Reader.java:1158)
          at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.read(SettingsXpp3Reader.java:1579)
          at org.apache.maven.settings.DefaultMavenSettingsBuilder.readSettings(DefaultMavenSettingsBuilder.java:122)
          at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:163)
          at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:154)
          at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:142)
          at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:423)
          ... 10 more

          Show
          gesha added a comment - @deckrider thankx for valuable tipps :/ Result is attached> + Error stacktraces are turned on. Error reading settings.xml: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17) Line: 35 Column: 17 Error stacktrace: org.apache.maven.SettingsConfigurationException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:1 7) Line: 35 Column: 17 at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:432) at org.apache.maven.cli.MavenCli.main(MavenCli.java:202) at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315) at org.codehaus.classworlds.Launcher.launch(Launcher.java:255) at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430) at org.codehaus.classworlds.Launcher.main(Launcher.java:375) Caused by: org.codehaus.plexus.util.xml.pull.XmlPullParserException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.parseEndTag(MXParser.java:1708) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.nextImpl(MXParser.java:1143) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.next(MXParser.java:1105) at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:179) at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:86) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseServer(SettingsXpp3Reader.java:1062) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseSettings(SettingsXpp3Reader.java:1158) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.read(SettingsXpp3Reader.java:1579) at org.apache.maven.settings.DefaultMavenSettingsBuilder.readSettings(DefaultMavenSettingsBuilder.java:122) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:163) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:154) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:142) at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:423) ... 10 more
          Hide
          Stephen Pack added a comment -

          @gesha: replace the </all> in deckrider's snippet with </put>. This workaround works for me using Maven 2.2.1 (note that if you deploy a snapshot that already exists – or anyone on your team doesn't implement the workaround as well – you may get the error, since the artifact being replaced has the issue).

          Show
          Stephen Pack added a comment - @gesha: replace the </all> in deckrider's snippet with </put>. This workaround works for me using Maven 2.2.1 (note that if you deploy a snapshot that already exists – or anyone on your team doesn't implement the workaround as well – you may get the error, since the artifact being replaced has the issue).
          Hide
          Russel Winder added a comment -

          Can someone specify the definitive, official, minimal workaround for this bug?

          Show
          Russel Winder added a comment - Can someone specify the definitive, official, minimal workaround for this bug?
          Hide
          Cornel Masson added a comment -

          We are experiencing the same bug and it's now a show-stopper for our Maven release. We have a large build and are seeing numerous CHECKSUM failures, even on Maven jars from Central.

          We are using:

          • Maven 2.2.1
          • Nexus Open Source Edition, Version: 1.6.0

          We have tried all the suggestions on the Net, including:

          • setting the wagon provider to httpclient
          • setting http.authentication.preemptive to true (as above)
          • setting the specific wagon-webdav version (as above)

          and nothing works. Each time we run the build, a random selection of artifacts fail with CHECKSUM errors. This happens for both internal (self-deployed) and external artifacts.

          Here's an example failure:

          4425K downloaded (groovy-all-1.6.5.jar)
          [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '9c6a38100085d8fbfb3f017ac82e396999e64f92'; remote = 'PK

          If you examine the remote checksum, it seems it references the actual contents of the JAR, not a checksum file(!!).

          We also get 'invalid POM' errors as a result of some failures, so our build actually breaks at the end (no use just ignoring the bad checksums). Example:

          40b downloaded (plexus-utils-1.5.6.pom)
          [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '8394a409e7b98020aac9429157265db0fb8cfac6'; remote = '<?xml' - IGNORING
          [WARNING] POM for 'org.codehaus.plexus:plexus-utils:pom:1.5.6:runtime' is invalid.

          Please help!

          Show
          Cornel Masson added a comment - We are experiencing the same bug and it's now a show-stopper for our Maven release. We have a large build and are seeing numerous CHECKSUM failures, even on Maven jars from Central. We are using: Maven 2.2.1 Nexus Open Source Edition, Version: 1.6.0 We have tried all the suggestions on the Net, including: setting the wagon provider to httpclient setting http.authentication.preemptive to true (as above) setting the specific wagon-webdav version (as above) and nothing works. Each time we run the build, a random selection of artifacts fail with CHECKSUM errors. This happens for both internal (self-deployed) and external artifacts. Here's an example failure: 4425K downloaded (groovy-all-1.6.5.jar) [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '9c6a38100085d8fbfb3f017ac82e396999e64f92'; remote = 'PK If you examine the remote checksum, it seems it references the actual contents of the JAR, not a checksum file(!!). We also get 'invalid POM' errors as a result of some failures, so our build actually breaks at the end (no use just ignoring the bad checksums). Example: 40b downloaded (plexus-utils-1.5.6.pom) [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '8394a409e7b98020aac9429157265db0fb8cfac6'; remote = '<?xml' - IGNORING [WARNING] POM for 'org.codehaus.plexus:plexus-utils:pom:1.5.6:runtime' is invalid. Please help!
          Hide
          Brian Fox added a comment -

          See the trick is that maven is producing these junk checksums upon upload. Even if you fix it, until you repair the checksums in the repo, you will continue to get the checksum errors whenever someone downloads those artifacts.

          The workaround for this bug is to stop producing bad checksums, and then repair the ones that are broken.

          Stop producing bad checksums by:
          Not using Maven 2.2.0
          or not using a wagon with httpclient against an authenticated repo unless you disable preemptive authentication.

          If you're using Nexus, it can repair the checksums simply by doing a "Rebuild Maven Metadata" on the subfolder of the repo where your busted checksums are.

          Show
          Brian Fox added a comment - See the trick is that maven is producing these junk checksums upon upload. Even if you fix it, until you repair the checksums in the repo, you will continue to get the checksum errors whenever someone downloads those artifacts. The workaround for this bug is to stop producing bad checksums, and then repair the ones that are broken. Stop producing bad checksums by: Not using Maven 2.2.0 or not using a wagon with httpclient against an authenticated repo unless you disable preemptive authentication. If you're using Nexus, it can repair the checksums simply by doing a "Rebuild Maven Metadata" on the subfolder of the repo where your busted checksums are.
          Hide
          Cornel Masson added a comment -

          We were seeing these on jars from Maven Central, not just our own ones. Does this mean Central is corrupt?

          Also, if we're using Maven 2.2.1, what is the definitive set of configurations that will make the uploading of corrupt checksums go away?

          Show
          Cornel Masson added a comment - We were seeing these on jars from Maven Central, not just our own ones. Does this mean Central is corrupt? Also, if we're using Maven 2.2.1, what is the definitive set of configurations that will make the uploading of corrupt checksums go away?
          Hide
          Cornel Masson added a comment -

          We found our problem: if you restrict the artifact resolver to a single thread, the problem goes away.

          Bug logged: http://jira.codehaus.org/browse/MNG-4742

          Show
          Cornel Masson added a comment - We found our problem: if you restrict the artifact resolver to a single thread, the problem goes away. Bug logged: http://jira.codehaus.org/browse/MNG-4742
          Hide
          Chris Tanger added a comment -

          For nexus users it also relates to https://issues.sonatype.org/browse/NEXUS-4347

          Show
          Chris Tanger added a comment - For nexus users it also relates to https://issues.sonatype.org/browse/NEXUS-4347
          Hide
          Benson Margulies added a comment -

          Doesn't this net out to a bug in the dav plugin, not maven itself?

          Show
          Benson Margulies added a comment - Doesn't this net out to a bug in the dav plugin, not maven itself?
          Show
          Jason van Zyl added a comment - Please refer to https://cwiki.apache.org/confluence/display/MAVEN/The+Great+JIRA+Cleanup+of+2014
          Hide
          Jason van Zyl added a comment -

          Please refer to https://cwiki.apache.org/confluence/display/MAVEN/The+Great+JIRA+Cleanup+of+2014 if you're wondering why this issue was closed out.

          Show
          Jason van Zyl added a comment - Please refer to https://cwiki.apache.org/confluence/display/MAVEN/The+Great+JIRA+Cleanup+of+2014 if you're wondering why this issue was closed out.

            People

            • Assignee:
              Unassigned
              Reporter:
              Kevin Shekleton
            • Votes:
              23 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: