History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: MNGECLIPSE-302
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Eugene Kuleshov
Reporter: Aaron Digulla
Votes: 3
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Maven Integration for Eclipse

Plugin tries to download dependencies N times

Created: 14/Mar/07 06:14 AM   Updated: 18/Jul/07 02:13 AM
Component/s: None
Affects Version/s: None
Fix Version/s: 0.0.11

Time Tracking:
Not Specified

File Attachments: 1. HTML File download-bug.zip (61 kb)
2. Text File MNGECLIPSE-302.patch (5 kb)

Issue Links:
Duplicate
 
dependent
 


 Description  « Hide
Since two weeks, I see this in the console:

14.03.07 10:51:31 CET: Reading /groovy-maven-plugin/pom.xml
14.03.07 10:51:32 CET: Downloading Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:20 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:21 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:22 CET: Unable to index org/codehaus/mojo/mojo/13/mojo-13.pom.sha1; Lock obtain timed out: Lock@C:\DOKUME~1\DIGULAA\LOKALE~1\Temp\lucene-0de289983e6540e3d48d6e053dd4ed09-write.lock
14.03.07 10:52:22 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:23 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:24 CET: Unable to index org/codehaus/mojo/mojo/13/mojo-13.pom.sha1; Lock obtain timed out: Lock@C:\DOKUME~1\DIGULAA\LOKALE~1\Temp\lucene-0de289983e6540e3d48d6e053dd4ed09-write.lock
14.03.07 10:52:24 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1
14.03.07 10:52:25 CET: Downloaded Repositorycentral/org/codehaus/mojo/mojo/13/mojo-13.pom.sha1

Am I the only one who sees this?



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Yuri Schimke - 15/Mar/07 11:12 AM
Yeah, we were getting it with a development release of 0.11. We were having other issues with 0.10.

We were also getting issues with an invalid jtds pom in ibiblio, and continually looping and complaining about it. We got past it on two desktops, but couldn't get past this problem on someone elses, so we had to drop back to using mvn eclipse:eclipse, which is a real pain.


Phil Webb - 16/Mar/07 06:28 AM
Are you by any chance using mirrors in your .m2/settings.xml file? If so you might want to track this bug MNGECLIPSE-274.

Aaron Digulla - 17/Mar/07 08:02 AM
I'm using DSMP as a proxy but the files are already cached, to it should deliver them on the first access. I'm a bit puzzled myself. I mean, even if DSMP returns an error or something, the plugin shouldn't try to download the same artifact more then once and, as you can see at the timestamps, it's really pumps! I mean, we're talking 100 download attempts per second.

Aaron Digulla - 17/Mar/07 08:04 AM
I'll try the patch for MNGECLIPSE-274 on Monday.

Eugene Kuleshov - 18/Mar/07 04:00 PM
Please try 0.0.11 dev build from update site at http://m2eclipse.codehaus.org/update/

If it still has this issue, feel free to reopen and provide settings.xml and Eclipse project with Maven's pom.xml sufficient to reproduce. Thanks.


Aaron Digulla - 19/Mar/07 09:19 AM
The bug is in the latest SVN version.

Here is the stack trace when the download loop happens:

Thread [Worker-13] (Suspended (breakpoint at line 58 in TransferListenerAdapter))
TransferListenerAdapter.transferStarted(TransferEvent) line: 58
TransferEventSupport.fireTransferStarted(TransferEvent) line: 103
LightweightHttpWagon(AbstractWagon).fireGetStarted(Resource, File) line: 434
LightweightHttpWagon(AbstractWagon).getTransfer(Resource, File, InputStream, boolean, int) line: 249
LightweightHttpWagon(AbstractWagon).getTransfer(Resource, File, InputStream) line: 239
LightweightHttpWagon(StreamWagon).get(String, File) line: 80
DefaultArtifactManager.getRemoteFile(ArtifactRepository, File, String, String, boolean) line: 350
DefaultArtifactManager.getArtifact(Artifact, ArtifactRepository) line: 263
DefaultArtifactManager.getArtifact(Artifact, List) line: 217
EclipseArtifactResolver(DefaultArtifactResolver).resolve(Artifact, List, ArtifactRepository, boolean) line: 185
EclipseArtifactResolver(DefaultArtifactResolver).resolve(Artifact, List, ArtifactRepository) line: 73
EclipseArtifactResolver.resolve(Artifact, List, ArtifactRepository) line: 57
DefaultMavenProjectBuilder.findModelFromRepository(Artifact, List, ArtifactRepository, boolean) line: 519
DefaultMavenProjectBuilder.buildFromRepository(Artifact, List, ArtifactRepository, boolean) line: 250
MavenMetadataSource.retrieve(Artifact, ArtifactRepository, List) line: 115
DefaultArtifactCollector.recurse(ResolutionNode, Map, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter, List) line: 278
DefaultArtifactCollector.collect(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter, List) line: 70
EclipseArtifactResolver(DefaultArtifactResolver).resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter, List) line: 284
EclipseArtifactResolver.resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter, List) line: 96
EclipseArtifactResolver(DefaultArtifactResolver).resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter) line: 272
EclipseArtifactResolver.resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource, ArtifactFilter) line: 105
EclipseArtifactResolver(DefaultArtifactResolver).resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource) line: 253
EclipseArtifactResolver.resolveTransitively(Set, Artifact, Map, ArtifactRepository, List, ArtifactMetadataSource) line: 114
DefaultMavenProjectBuilder.buildWithDependencies(File, ArtifactRepository, ProfileManager, TransferListener) line: 388
MavenEmbedder.readProjectWithDependencies(MavenExecutionRequest) line: 434
MavenModelManager.readMavenProject(IFile, IProgressMonitor, boolean, boolean) line: 360
BuildPathManager.resolveClasspathEntries(Set, Map, IFile, IFile, boolean, IProgressMonitor) line: 158
BuildPathManager.updateClasspathContainer(IProject, boolean, IProgressMonitor) line: 121
Maven2Builder.build(int, Map, IProgressMonitor) line: 75
BuildManager$2.run() line: 603
SafeRunner.run(ISafeRunnable) line: 37
BuildManager.basicBuild(int, IncrementalProjectBuilder, Map, MultiStatus, IProgressMonitor) line: 167
BuildManager.basicBuild(IProject, int, ICommand[], MultiStatus, IProgressMonitor) line: 201
BuildManager$1.run() line: 230
SafeRunner.run(ISafeRunnable) line: 37
BuildManager.basicBuild(IProject, int, MultiStatus, IProgressMonitor) line: 233
BuildManager.basicBuildLoop(IProject[], IProject[], int, MultiStatus, IProgressMonitor) line: 252
BuildManager.build(int, IProgressMonitor) line: 285
AutoBuildJob.doBuild(IProgressMonitor) line: 154
AutoBuildJob.run(IProgressMonitor) line: 217
Worker.run() line: 58

I've set a breakpoint in TransferListenerAdapter.transferStarted() and this method is called N times with the same repository. N depends on the number of remote repositories in the project (DefaultArtifactManager.getArtifact(Artifact, List) line: 217).

There are a couple of things which puzzle me:

1. I've set a breakpoint in DefaultArtifactManager.java:351 (after wagon.get()!) and I only return here after N download attempts have been made. It seems that wagon.get() already loops through the list of known remote repositories but I couldn't find the place where it does this loop!

It's not the loop from DefaultArtifactManager.getArtifact(Artifact, List) line: 217! This one runs as well. It looks like the two loops run inside of each other (first, getArtifact() will call getRemoteFile(), then getRemoteFile() will invoke transferStarted() N times, then the next repository will be tried).

It's also not the retry-loop in getRemoteFile().

2. The repository in TransferListenerAdapter.transferStarted() is always the same.

Next, DefaultArtifactManager.verifyChecksum() is called which calls wagon.get() again resulting in the same output again even though the files could be downloaded. I think this is because wagon doesn't create the file but a tmp-file. So when you verify the checksum, the artifact doesn't exist yet.

I could reproduce this behavior with the attached project which contains more than one remote repository and which needs an artifact which isn't in the local repo, yet.


Aaron Digulla - 19/Mar/07 09:20 AM
Snapshot of the groovy-maven-plugin

Stefan Seidel - 26/Mar/07 09:06 AM
I have looked into the source and I think the problem is not on the plugin side, this time.

Everytime the plugin does something with the Maven Embedder, it creates a new one and adds a new TransferListenerAdapter. This should not be a problem, because the embedder and thus the execution request and thus the TransferListener are used only for this one action and not further.

But nevertheless, new embedders (or, the "wagon" component used by the embedder) still call the old transferListeners when the new embedder is invoked. This became obvious when I logged TransferListenerAdapter.toString() in its constructor and the event handler methods.


Aaron Digulla - 19/Apr/07 06:30 AM
This is bad because in TransferListenerAdapter.transferCompleted(), the index is updated.

Should I open a new issue against Embedder or do you want to make Jason responsible for this bug here?


Abel Muiño - 12/May/07 12:25 PM
I'm running into the same project (a TransferListener used in a request is never being removed from the wagon).

What's the status of this issue? Has it been reported to the Maven Embedder?


Abel Muiño - 12/May/07 06:22 PM
After examining the code, I've created a new JIRA issue to the Maven project and attached a patch.
You can track it at MNG-2985

Abel Muiño - 27/May/07 03:59 PM
MNG-2985 has been resolved. It should be possible now to build a new release bundling the new maven-embedder with minor changes to existing code.

The new maven embedder should fix this bug (and probably also MNGECLIPSE-320 and MNGECLIPSE-329)

I've got an unnoficial build at http://candy4appfuse.sourceforge.net/site/plugins/org.maven.ide.eclipse_0.0.11.v20070521-2114.jar and can provide a patch if needed (I'm not sure if I need to include the maven-embedder jar in the patch, though).


Stefan Seidel - 28/May/07 01:18 AM
The patch and a link to the embedder would be good, because it is quite an effort to build it from scratch. (Although the JAR is contained in the plugin JAR that you built.)

Abel Muiño - 31/May/07 02:58 PM
Patch for using a newer maven-embeder (not included).

The maven-embedder jar can be found at http://candy4appfuse.sourceforge.net/tmp/. I've included the binary jar and the source code zip (which is bundled with the plugin in previous versions).


Eugene Kuleshov - 03/Jun/07 02:00 AM
I committed new embedder build yesterday. Can anyone please test if this issue still occur with code from trunk?

Abel Muiño - 03/Jun/07 07:50 AM
I see the jar commited to SVN, but the plug-in files haven't been modified yet, so builds from svn head will still be using the old embedder.

For example, check the bundle-classpath entry in MANIFEST.MF from head, which still points to the old jar (lib/maven-embedder-2.1.0.v20070302-2248-dep.jar).

If you want me to create a patch for the committed jar, I can do that.


Eugene Kuleshov - 03/Jun/07 10:29 AM
Oh. Sorry, please wait. I hit into some issue that need to be resolved first.

Eugene Kuleshov - 03/Jun/07 11:08 AM
It is in now

Abel Muiño - 03/Jun/07 04:23 PM - edited
It is working for me from SVN trunk.
3/06/07 23:19:51 CEST: Downloading [central] -> http://repo1.maven.org/maven2/org/hibernate/hibernate-annotations/3.2.1.ga/hibernate-annotations-3.2.1.ga.pom
3/06/07 23:19:51 CEST: Downloaded [central] -> http://repo1.maven.org/maven2/org/hibernate/hibernate-annotations/3.2.1.ga/hibernate-annotations-3.2.1.ga.pom
3/06/07 23:19:53 CEST: Downloading [central] -> http://repo1.maven.org/maven2/org/hibernate/hibernate/3.2.1.ga/hibernate-3.2.1.ga.pom
3/06/07 23:19:53 CEST: Downloaded [central] -> http://repo1.maven.org/maven2/org/hibernate/hibernate/3.2.1.ga/hibernate-3.2.1.ga.pom
3/06/07 23:19:55 CEST: Downloading [appfuse] -> http://static.appfuse.org/repository/org/hibernate/hibernate-annotations/3.2.1.ga/hibernate-annotations-3.2.1.ga.jar
3/06/07 23:20:01 CEST: Downloaded [appfuse] -> http://static.appfuse.org/repository/org/hibernate/hibernate-annotations/3.2.1.ga/hibernate-annotations-3.2.1.ga.jar
3/06/07 23:20:03 CEST: Downloading [appfuse] -> http://static.appfuse.org/repository/org/hibernate/hibernate/3.2.1.ga/hibernate-3.2.1.ga.jar
3/06/07 23:20:31 CEST: Downloaded [appfuse] -> http://static.appfuse.org/repository/org/hibernate/hibernate/3.2.1.ga/hibernate-3.2.1.ga.jar

William Ferguson - 04/Jun/07 08:09 PM
0.0.11.20070603-1200 works fine for me.
Dependencies only get downloaded once - yeeha

Aaron Digulla - 05/Jun/07 09:39 AM
Same here. I think the bug can be closed.

PS: Thanks for truncating my initial comment; I didn't realize that JIRA would include it with every mail.


Derek Clarkson - 18/Jul/07 12:56 AM
I'm sorry to report guys that this is NOT FIXED in the latests trunk. I download and compiled 0.0.11.v2007...etc and I still get the problem. All be it not a serverly. Now it only looks for about 8-10 times each file before moving on so at best I would say it is partially fixed.

However !

This version has introduced a major new bug - dependencies in modules are not resolved at all ! This means that your eclipse project is left in a state where it think there are a lot of missing packages and junits etc will not run at all due to missing jars.


Aaron Digulla - 18/Jul/07 02:13 AM
It is fixed for me when I use the version from SVN (https://svn.codehaus.org/m2eclipse/trunk/org.maven.ide.eclipse).

The dependencies problem is another issue and unrelated to this one. Please open a new issue for that.