Maven 2 & 3

Maven doesn't try to download a dependency when it already exists a version with classifier locally

Details

  • Type: Bug Bug
  • Status: Open Open
  • Priority: Critical Critical
  • Resolution: Unresolved
  • Affects Version/s: 2.0.9, 2.0.10, 2.1.0
  • Fix Version/s: Issues to be reviewed for 3.x
  • Component/s: Dependencies
  • Labels:
    None
  • Environment:
    Java 5, Windows XP
  • Complexity:
    Intermediate
  • Testcase included:
    yes
  • Number of attachments :
    2

Description

When maven installs in the local repository an artifact with a classifier, and not the principal artifact, it won't try in a reactor to download the principal artifact from the remote repository.
I created a testcase to reproduce it. It's quite simple. You unzip it and in the root directory you launch 

mvn site

This build uses clover, thus it installs locally cloverified versions of artifacts.
The build will fail because it doesn't find the SNAPSHOT of the module1 :

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------
1) org.apache.maven.it.test-metadata-local-clover:module1:jar:1.0.0-SNAPSHOT

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=org.apache.maven.it.test-metadata-local-clover -DartifactId=module1 -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/pa
th/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=org.apache.maven.it.test-metadata-local-clover -DartifactId=module1 -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/path
/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.apache.maven.it.test-metadata-local-clover:module2:jar:1.0.0-SNAPSHOT
        2) org.apache.maven.it.test-metadata-local-clover:module1:jar:1.0.0-SNAPSHOT

----------
1 required artifact is missing.

for artifact:
  org.apache.maven.it.test-metadata-local-clover:module2:jar:1.0.0-SNAPSHOT

from the specified remote repositories:
  central (http://maven-proxy.groupe.generali.fr/all),
  remote-repo (file://E:\jtb\workspaces\tests\test-metadata-local-clover/remote-repo)

It's anormal because I set a "local" remote repository in the build where it should try to download it.
You can validate that it is working by launching :

mvn install -f module2/pom.xml

This bug is really annoying for us. It exists for a long long time now. I thought it was due to a problem with metadata sent by archiva, but after migrating to nexus the problem always occurs.
I don't know if the problem is in metadata or in the reactor itself. I think it is the second one. There are many issues opened about the reactor and classifiers.
Is there some who can try if it can reproduce the error with my testcase ?
It should be easy to create a real IT for maven with it if it is necessary.

Attachments

  1. Text File
    MNG-4142.patch
    18/Aug/09 5:03 AM
    2 kB
    David Rousselie

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Arnaud Heritier added a comment -

Something else to add : If I launch after the site build, another build which doesn't produce the main artefact (for example : mvn sonar), it will also fail trying to download dependency. I think that it can be related to maven-metadata and more probably to the maven maven-metadata-local.xml which is generated.
After the mvn site, I have in my local repository in org\apache\maven\it\test-metadata-local-clover\module1\1.0.0-SNAPSHOT:

maven-metadata-central.xml
maven-metadata-local.xml
maven-metadata-remote-repo.xml
maven-metadata-remote-repo.xml.sha1
module1-1.0.0-SNAPSHOT-clover.jar
module1-1.0.0-SNAPSHOT.pom

and in each metadata file :

  • maven-metadata-central.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
</metadata>
  • maven-metadata-local.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <localCopy>true</localCopy>
    </snapshot>
    <lastUpdated>20090423144418</lastUpdated>
  </versioning>
</metadata>
  • maven-metadata-remote-repo.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20090423.140624</timestamp>
      <buildNumber>1</buildNumber>
    </snapshot>
    <lastUpdated>20090423140625</lastUpdated>
  </versioning>
</metadata>

    Do you think that maven-metadata-local.xml is good ? It says that there's a local copy whereas there's no module1-1.0.0-SNAPSHOT.jar in the directory.

Show
Arnaud Heritier added a comment - Something else to add : If I launch after the site build, another build which doesn't produce the main artefact (for example : mvn sonar), it will also fail trying to download dependency. I think that it can be related to maven-metadata and more probably to the maven maven-metadata-local.xml which is generated. After the mvn site, I have in my local repository in org\apache\maven\it\test-metadata-local-clover\module1\1.0.0-SNAPSHOT: 
maven-metadata-central.xml
maven-metadata-local.xml
maven-metadata-remote-repo.xml
maven-metadata-remote-repo.xml.sha1
module1-1.0.0-SNAPSHOT-clover.jar
module1-1.0.0-SNAPSHOT.pom
and in each metadata file :
  • maven-metadata-central.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
</metadata>
  • maven-metadata-local.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <localCopy>true</localCopy>
    </snapshot>
    <lastUpdated>20090423144418</lastUpdated>
  </versioning>
</metadata>
  • maven-metadata-remote-repo.xml 
    <?xml version="1.0" encoding="UTF-8"?>
<metadata xsi:schemaLocation="http://maven.apache.org/METADATA/1.0.0 http://maven.apache.org/xsd/metadata-1.0.0.xsd" xmlns="http://maven.apache.org/METADATA/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <groupId>org.apache.maven.it.test-metadata-local-clover</groupId>
  <artifactId>module1</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20090423.140624</timestamp>
      <buildNumber>1</buildNumber>
    </snapshot>
    <lastUpdated>20090423140625</lastUpdated>
  </versioning>
</metadata>
    Do you think that maven-metadata-local.xml is good ? It says that there's a local copy whereas there's no module1-1.0.0-SNAPSHOT.jar in the directory.
Hide
Permalink
David Rousselie added a comment -

The problem is that when the Maven resolve the artifact version with maven-metadata-local.xml when it exists and it resolves it to 1.0.0-SNAPSHOT whereas only module1-1.0.0-SNAPSHOT-clover.jar (in this exemple). Then when it try to access the file artifact (module1-1.0.0-SNAPSHOT.jar) and fails, it tries to get the same version on remote repositories whereas it should have try to resolve the artifact version with data found in maven-metadata-${remotereponame}.xml before trying to get it from each repository.

I have attached a small patch (base on revision 805295 in branch maven-2.2.x) which might not be the best way to correct this bug : while resolving artifact version, it loads local metadata only if artifact's file exists locally.

Show
David Rousselie added a comment - The problem is that when the Maven resolve the artifact version with maven-metadata-local.xml when it exists and it resolves it to 1.0.0-SNAPSHOT whereas only module1-1.0.0-SNAPSHOT-clover.jar (in this exemple). Then when it try to access the file artifact (module1-1.0.0-SNAPSHOT.jar) and fails, it tries to get the same version on remote repositories whereas it should have try to resolve the artifact version with data found in maven-metadata-${remotereponame}.xml before trying to get it from each repository. I have attached a small patch (base on revision 805295 in branch maven-2.2.x) which might not be the best way to correct this bug : while resolving artifact version, it loads local metadata only if artifact's file exists locally.
Hide
Permalink
David Rousselie added a comment -

A possible correction for bug MNG-4142

Show
David Rousselie added a comment - A possible correction for bug MNG-4142
Hide
Permalink
Peter Liljenberg added a comment -

I've encountered this problem as well when doing distributed builds (using Hudson).
When one build server (#1) builds and deploys an artifact the maven-metadata-local.xml will contain the localCopy attribute:
<versioning>
<snapshot>
<localCopy>true</localCopy>
</snapshot>
<lastUpdated>20091127124227</lastUpdated>
</versioning>

Which means that if another build server builds the same module AFTER server #1 and produce a new artifact in the repository, machine #1 will never download this new SNAPSHOT version when trying to build artifacts depending on the artifact.
This is a huge problem (and a scary one!) for us and I would like to see a fix for this soon

Show
Peter Liljenberg added a comment - I've encountered this problem as well when doing distributed builds (using Hudson). When one build server (#1) builds and deploys an artifact the maven-metadata-local.xml will contain the localCopy attribute: <versioning> <snapshot> <localCopy>true</localCopy> </snapshot> <lastUpdated>20091127124227</lastUpdated> </versioning> Which means that if another build server builds the same module AFTER server #1 and produce a new artifact in the repository, machine #1 will never download this new SNAPSHOT version when trying to build artifacts depending on the artifact. This is a huge problem (and a scary one!) for us and I would like to see a fix for this soon
Hide
Permalink
Brian Laframboise added a comment - - edited

I just ran into this issue myself (with Maven 2.2.1). A build on one of my build machines was failing due to maven not downloading a newer version of a dependency that was available on Nexus because it had previously installed an earlier version of that same dependency into the local repository.

When I removed the <localCopy>true</localCopy> element from the metadata of the non-updating dependency the problem went away - and dutifully came back when I re-added it.

Peter, I agree - this is a scary problem for anybody trying to implement a distributed build system. What I don't understand is why more people are not encountering this issue.

For now I'll attempt a workaround this by adding a cron job, per build agent, to frequently change <localCopy> back to false. If anybody has a better suggestion, I'd love to hear it.

Show
Brian Laframboise added a comment - - edited I just ran into this issue myself (with Maven 2.2.1). A build on one of my build machines was failing due to maven not downloading a newer version of a dependency that was available on Nexus because it had previously installed an earlier version of that same dependency into the local repository. When I removed the <localCopy>true</localCopy> element from the metadata of the non-updating dependency the problem went away - and dutifully came back when I re-added it. Peter, I agree - this is a scary problem for anybody trying to implement a distributed build system. What I don't understand is why more people are not encountering this issue. For now I'll attempt a workaround this by adding a cron job, per build agent, to frequently change <localCopy> back to false. If anybody has a better suggestion, I'd love to hear it.
Hide
Permalink
Martin Todorov added a comment -

Hi,

I would like to ask why the attached patch is not being accepted?
There are currently over three dozen votes for this and it is apparent that people are hitting the problem.

In addition, perhaps the title of the bug is not quite correct. This doesn't just happen for artifacts with classifiers. In our case it happens for POM files where there are several inherited POMs.
We're using Hudson and Nexus had to limit the parents to build on one Hudson node, while all modules which depend on them - on the rest of the nodes.

I also think this is a huge and very obscure problem (esp. with distributed builds) and, provided that there is already a patch, I believe it's about time for it to be reviewed.

Regards,

Martin

Show
Martin Todorov added a comment - Hi, I would like to ask why the attached patch is not being accepted? There are currently over three dozen votes for this and it is apparent that people are hitting the problem. In addition, perhaps the title of the bug is not quite correct. This doesn't just happen for artifacts with classifiers. In our case it happens for POM files where there are several inherited POMs. We're using Hudson and Nexus had to limit the parents to build on one Hudson node, while all modules which depend on them - on the rest of the nodes. I also think this is a huge and very obscure problem (esp. with distributed builds) and, provided that there is already a patch, I believe it's about time for it to be reviewed. Regards, Martin
Hide
Permalink
Arnaud Heritier added a comment -

Were you able to test if the problem exists with maven 3 ?
I agree that this issue is really annoying but for now there is no 2.X version release scheduled.
That's why there was no feedback from dev team.

Show
Arnaud Heritier added a comment - Were you able to test if the problem exists with maven 3 ? I agree that this issue is really annoying but for now there is no 2.X version release scheduled. That's why there was no feedback from dev team.
Hide
Permalink
Martin Todorov added a comment -

Bonjour Arnaud!

As per your request, I have tested this with Maven 3.0 today. I am afraid the problem still persists. Today this actually happened with a completely different project/setup/machine under both Maven 2.2.1 and 3.0.

This is not just annoying, it's actually a show-stopper when it happens to a developer who's not expecting it, as they spend quite some time trying to figure it out.

Please review for the release of Maven 3.1, (if a 2.x version will not be further released).

Regards,

Martin

Show
Martin Todorov added a comment - Bonjour Arnaud! As per your request, I have tested this with Maven 3.0 today. I am afraid the problem still persists. Today this actually happened with a completely different project/setup/machine under both Maven 2.2.1 and 3.0. This is not just annoying, it's actually a show-stopper when it happens to a developer who's not expecting it, as they spend quite some time trying to figure it out. Please review for the release of Maven 3.1, (if a 2.x version will not be further released). Regards, Martin
Hide
Permalink
nodje added a comment -

I've the same problem but with standard SNAPSHOT dependencies, not using classifier, and with 3.0.3.

Show
nodje added a comment - I've the same problem but with standard SNAPSHOT dependencies, not using classifier, and with 3.0.3.
Hide
Permalink
Martin Todorov added a comment - - edited

This bug/feature doesn't really have much to do with classifiers. I believe the original bug report filer just thought it had something to do with the problem.
Perhaps somebody should rename the bug to "Maven doesn't try to download SNAPSHOT artifacts, if you've installed them locally; even if you pass -U".

I believe that by using -U the local metadata should get "unlocked" (which currently doesn't do this) for the locally installed SNAPSHOT artifacts.

Show
Martin Todorov added a comment - - edited This bug/feature doesn't really have much to do with classifiers. I believe the original bug report filer just thought it had something to do with the problem. Perhaps somebody should rename the bug to "Maven doesn't try to download SNAPSHOT artifacts, if you've installed them locally; even if you pass -U". I believe that by using -U the local metadata should get "unlocked" (which currently doesn't do this) for the locally installed SNAPSHOT artifacts.
Hide
Permalink
nodje added a comment -

alright, good then, that's my problem!

Can't wait for the fix to come, hopefully as early as 3.0.4...

Show
nodje added a comment - alright, good then, that's my problem! Can't wait for the fix to come, hopefully as early as 3.0.4...

People

Vote (53)
Watch (27)

Dates

  • Created:
    Updated:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.