Maven 2 & 3

Invalid checksums on deploy when using webdav without extension

Details

  • Complexity:
    Intermediate
  • Number of attachments :
    0

Description

With maven 2.2.1, our deployments via webdav are producing invalid checksums, similar to the issue reported in MNG-4235.

From maven 2.0.8 and previous, the following build extension was required to deploy via webdav:

<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-webdav</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>

Starting with maven 2.0.9 (see MNG-3418), webdav was included by default and the aforementioned build extension must be removed from the project. If it was included in the project the deployment would fail as Maven would report multiple versions of wagon-webdav present.

With maven 2.2.0, our deployment suffered from invalid checksums reported in MNG-4235.

With maven 2.2.1, we still see the invalid checksums on deployment as reported in MNG-4235. However, I've found that if you add the above build extension into the project, we don't experience this issue (of generating invalid checksums). Is this workaround an intentional change brought about by the fix of MNG-4235?

Issue Links

This issue is duplicated by:
MNG-4726 Maven generates incorrect MD5 and SHA1 on deply Critical Closed
This issue relates to:
ENUNCIATE-416 invalid maven checksums when deploying RC1 Major Closed
WAGON-304 Wrong checksum when deploy with wagon:httpclient provider Major Closed
This issue is related to:
MNG-4235 [regression] Maven 2.2.0 produces invalid checksums during deployment to secured HTTP repo Blocker Closed
This issue depends upon:
WAGON-277 HttpClient wagon (wagon-http) uploads all files twice when preemptive authentication is disabled Critical Closed
This issue is depended upon by:
MNG-4742 Multi-threaded artifact resolver corrupts artifacts on download Major Open

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Dave Cheney added a comment -

I can confirm that I have seen invalid checksums produced via the webdav wagon using version 2.2.1

Show
Dave Cheney added a comment - I can confirm that I have seen invalid checksums produced via the webdav wagon using version 2.2.1
Hide
Permalink
John Casey added a comment -

The problem may be that preemptive authentication was turned off by default for httpclient in the 1.0-beta-6 release, as was the case for the non-lightweight http wagon. I suspect this is the case in the jackrabbit-webdav as well, since IIRC they both use the same common abstract class. Since preemptive auth is disabled, Maven waits for the server to challenge the request then re-sends the data with the credentials. The problem is that the checksum stream observer is unaware that this challenge has taken place, so it sees the data go by twice without realizing it, thereby skewing the checksums.

To workaround the problem, you can re-enable preemptive authentication. The information is here:

http://maven.apache.org/guides/mini/guide-http-settings.html

But essentially, you'll need to specify something like the following in your settings.xml:

<settings>
  <servers>
    <server>
      <id>my-server</id>
      <configuration>
        <httpConfiguration>
          <put>
            <params>
              <param>
                <name>http.authentication.preemptive</name>
                <value>%b,true</value>
              </param>
            </params>
          </put>
        </httpConfiguration>
      </configuration>
    </server>
  </servers>
</settings>
Show
John Casey added a comment - The problem may be that preemptive authentication was turned off by default for httpclient in the 1.0-beta-6 release, as was the case for the non-lightweight http wagon. I suspect this is the case in the jackrabbit-webdav as well, since IIRC they both use the same common abstract class. Since preemptive auth is disabled, Maven waits for the server to challenge the request then re-sends the data with the credentials. The problem is that the checksum stream observer is unaware that this challenge has taken place, so it sees the data go by twice without realizing it, thereby skewing the checksums. To workaround the problem, you can re-enable preemptive authentication. The information is here: http://maven.apache.org/guides/mini/guide-http-settings.html But essentially, you'll need to specify something like the following in your settings.xml: 
<settings>
  <servers>
    <server>
      <id>my-server</id>
      <configuration>
        <httpConfiguration>
          <put>
            <params>
              <param>
                <name>http.authentication.preemptive</name>
                <value>%b,true</value>
              </param>
            </params>
          </put>
        </httpConfiguration>
      </configuration>
    </server>
  </servers>
</settings>
Hide
Permalink
Stephen Pack added a comment -

I believe the previous comment is the workaround for maven 2.2.0; for maven 2.2.1, you need to add <wagonProvider>httpclient</wagonProvider> in the configuration. Without that addition, I got the exception "Cannot find setter nor field in org.apache.maven.wagon.providers.http.LightweightHttpWagon for 'httpConfiguration'"

Show
Stephen Pack added a comment - I believe the previous comment is the workaround for maven 2.2.0; for maven 2.2.1, you need to add <wagonProvider>httpclient</wagonProvider> in the configuration. Without that addition, I got the exception "Cannot find setter nor field in org.apache.maven.wagon.providers.http.LightweightHttpWagon for 'httpConfiguration'"
Hide
Permalink
deckrider added a comment -

I tried the following ~/.m2/settings.xml which did not work with maven 2.2.1 (the double md5 issue still exists):


 

  <servers>
    <server>
      <id>deployment.webdav</id>
      <username>foo</username>
      <password>bar</password>
      <configuration>
        <wagonProvider>httpclient</wagonProvider>
        <httpConfiguration>
          <put>
            <params>
              <param>
                <name>http.authentication.preemptive</name>
                <value>%b,true</value>
              </param>
            </params>
          </all>
        </httpConfiguration>
      </configuration>
    </server>
  </servers>
Show
deckrider added a comment - I tried the following ~/.m2/settings.xml which did not work with maven 2.2.1 (the double md5 issue still exists): 
 

  <servers>
    <server>
      <id>deployment.webdav</id>
      <username>foo</username>
      <password>bar</password>
      <configuration>
        <wagonProvider>httpclient</wagonProvider>
        <httpConfiguration>
          <put>
            <params>
              <param>
                <name>http.authentication.preemptive</name>
                <value>%b,true</value>
              </param>
            </params>
          </all>
        </httpConfiguration>
      </configuration>
    </server>
  </servers>
Hide
Permalink
gesha added a comment -

@deckrider

thankx for valuable tipps :/ Result is attached>

+ Error stacktraces are turned on.
Error reading settings.xml: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17)
Line: 35
Column: 17
Error stacktrace:
org.apache.maven.SettingsConfigurationException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:1
7)
Line: 35
Column: 17
at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:432)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:202)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.codehaus.plexus.util.xml.pull.XmlPullParserException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n
</all>... @35:17)
at hidden.org.codehaus.plexus.util.xml.pull.MXParser.parseEndTag(MXParser.java:1708)
at hidden.org.codehaus.plexus.util.xml.pull.MXParser.nextImpl(MXParser.java:1143)
at hidden.org.codehaus.plexus.util.xml.pull.MXParser.next(MXParser.java:1105)
at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:179)
at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:86)
at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseServer(SettingsXpp3Reader.java:1062)
at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseSettings(SettingsXpp3Reader.java:1158)
at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.read(SettingsXpp3Reader.java:1579)
at org.apache.maven.settings.DefaultMavenSettingsBuilder.readSettings(DefaultMavenSettingsBuilder.java:122)
at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:163)
at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:154)
at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:142)
at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:423)
... 10 more

Show
gesha added a comment - @deckrider thankx for valuable tipps :/ Result is attached> + Error stacktraces are turned on. Error reading settings.xml: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17) Line: 35 Column: 17 Error stacktrace: org.apache.maven.SettingsConfigurationException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:1 7) Line: 35 Column: 17 at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:432) at org.apache.maven.cli.MavenCli.main(MavenCli.java:202) at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315) at org.codehaus.classworlds.Launcher.launch(Launcher.java:255) at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430) at org.codehaus.classworlds.Launcher.main(Launcher.java:375) Caused by: org.codehaus.plexus.util.xml.pull.XmlPullParserException: end tag name </all> must be the same as start tag <put> from line 28 (position: TEXT seen ...</params>\r\n </all>... @35:17) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.parseEndTag(MXParser.java:1708) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.nextImpl(MXParser.java:1143) at hidden.org.codehaus.plexus.util.xml.pull.MXParser.next(MXParser.java:1105) at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:179) at hidden.org.codehaus.plexus.util.xml.Xpp3DomBuilder.build(Xpp3DomBuilder.java:86) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseServer(SettingsXpp3Reader.java:1062) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.parseSettings(SettingsXpp3Reader.java:1158) at org.apache.maven.settings.io.xpp3.SettingsXpp3Reader.read(SettingsXpp3Reader.java:1579) at org.apache.maven.settings.DefaultMavenSettingsBuilder.readSettings(DefaultMavenSettingsBuilder.java:122) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:163) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:154) at org.apache.maven.settings.DefaultMavenSettingsBuilder.buildSettings(DefaultMavenSettingsBuilder.java:142) at org.apache.maven.cli.MavenCli.buildSettings(MavenCli.java:423) ... 10 more
Hide
Permalink
Stephen Pack added a comment -

@gesha: replace the </all> in deckrider's snippet with </put>. This workaround works for me using Maven 2.2.1 (note that if you deploy a snapshot that already exists – or anyone on your team doesn't implement the workaround as well – you may get the error, since the artifact being replaced has the issue).

Show
Stephen Pack added a comment - @gesha: replace the </all> in deckrider's snippet with </put>. This workaround works for me using Maven 2.2.1 (note that if you deploy a snapshot that already exists – or anyone on your team doesn't implement the workaround as well – you may get the error, since the artifact being replaced has the issue).
Hide
Permalink
Russel Winder added a comment -

Can someone specify the definitive, official, minimal workaround for this bug?

Show
Russel Winder added a comment - Can someone specify the definitive, official, minimal workaround for this bug?
Hide
Permalink
Cornel Masson added a comment -

We are experiencing the same bug and it's now a show-stopper for our Maven release. We have a large build and are seeing numerous CHECKSUM failures, even on Maven jars from Central.

We are using:

  • Maven 2.2.1
  • Nexus Open Source Edition, Version: 1.6.0

We have tried all the suggestions on the Net, including:

  • setting the wagon provider to httpclient
  • setting http.authentication.preemptive to true (as above)
  • setting the specific wagon-webdav version (as above)

and nothing works. Each time we run the build, a random selection of artifacts fail with CHECKSUM errors. This happens for both internal (self-deployed) and external artifacts.

Here's an example failure:

4425K downloaded (groovy-all-1.6.5.jar)
[WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '9c6a38100085d8fbfb3f017ac82e396999e64f92'; remote = 'PK

If you examine the remote checksum, it seems it references the actual contents of the JAR, not a checksum file(!!).

We also get 'invalid POM' errors as a result of some failures, so our build actually breaks at the end (no use just ignoring the bad checksums). Example:

40b downloaded (plexus-utils-1.5.6.pom)
[WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '8394a409e7b98020aac9429157265db0fb8cfac6'; remote = '<?xml' - IGNORING
[WARNING] POM for 'org.codehaus.plexus:plexus-utils:pom:1.5.6:runtime' is invalid.

Please help!

Show
Cornel Masson added a comment - We are experiencing the same bug and it's now a show-stopper for our Maven release. We have a large build and are seeing numerous CHECKSUM failures, even on Maven jars from Central. We are using:
  • Maven 2.2.1
  • Nexus Open Source Edition, Version: 1.6.0
We have tried all the suggestions on the Net, including:
  • setting the wagon provider to httpclient
  • setting http.authentication.preemptive to true (as above)
  • setting the specific wagon-webdav version (as above)
and nothing works. Each time we run the build, a random selection of artifacts fail with CHECKSUM errors. This happens for both internal (self-deployed) and external artifacts. Here's an example failure: 4425K downloaded (groovy-all-1.6.5.jar) [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '9c6a38100085d8fbfb3f017ac82e396999e64f92'; remote = 'PK If you examine the remote checksum, it seems it references the actual contents of the JAR, not a checksum file(!!). We also get 'invalid POM' errors as a result of some failures, so our build actually breaks at the end (no use just ignoring the bad checksums). Example: 40b downloaded (plexus-utils-1.5.6.pom) [WARNING] *** CHECKSUM FAILED - Checksum failed on download: local = '8394a409e7b98020aac9429157265db0fb8cfac6'; remote = '<?xml' - IGNORING [WARNING] POM for 'org.codehaus.plexus:plexus-utils:pom:1.5.6:runtime' is invalid. Please help!
Hide
Permalink
Brian Fox added a comment -

See the trick is that maven is producing these junk checksums upon upload. Even if you fix it, until you repair the checksums in the repo, you will continue to get the checksum errors whenever someone downloads those artifacts.

The workaround for this bug is to stop producing bad checksums, and then repair the ones that are broken.

Stop producing bad checksums by:
Not using Maven 2.2.0
or not using a wagon with httpclient against an authenticated repo unless you disable preemptive authentication.

If you're using Nexus, it can repair the checksums simply by doing a "Rebuild Maven Metadata" on the subfolder of the repo where your busted checksums are.

Show
Brian Fox added a comment - See the trick is that maven is producing these junk checksums upon upload. Even if you fix it, until you repair the checksums in the repo, you will continue to get the checksum errors whenever someone downloads those artifacts. The workaround for this bug is to stop producing bad checksums, and then repair the ones that are broken. Stop producing bad checksums by: Not using Maven 2.2.0 or not using a wagon with httpclient against an authenticated repo unless you disable preemptive authentication. If you're using Nexus, it can repair the checksums simply by doing a "Rebuild Maven Metadata" on the subfolder of the repo where your busted checksums are.
Hide
Permalink
Cornel Masson added a comment -

We were seeing these on jars from Maven Central, not just our own ones. Does this mean Central is corrupt?

Also, if we're using Maven 2.2.1, what is the definitive set of configurations that will make the uploading of corrupt checksums go away?

Show
Cornel Masson added a comment - We were seeing these on jars from Maven Central, not just our own ones. Does this mean Central is corrupt? Also, if we're using Maven 2.2.1, what is the definitive set of configurations that will make the uploading of corrupt checksums go away?
Hide
Permalink
Cornel Masson added a comment -

We found our problem: if you restrict the artifact resolver to a single thread, the problem goes away.

Bug logged: http://jira.codehaus.org/browse/MNG-4742

Show
Cornel Masson added a comment - We found our problem: if you restrict the artifact resolver to a single thread, the problem goes away. Bug logged: http://jira.codehaus.org/browse/MNG-4742
Hide
Permalink
Chris Tanger added a comment -

For nexus users it also relates to https://issues.sonatype.org/browse/NEXUS-4347

Show
Chris Tanger added a comment - For nexus users it also relates to https://issues.sonatype.org/browse/NEXUS-4347
Hide
Permalink
Benson Margulies added a comment -

Doesn't this net out to a bug in the dav plugin, not maven itself?

Show
Benson Margulies added a comment - Doesn't this net out to a bug in the dav plugin, not maven itself?

People

Vote (21)
Watch (25)

Dates

  • Created:
    Updated:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.