dependencyManagement does not work for transitive dependencies

I think we determined this wasn't the behaviour that would be used (depMgmt only applying to the current project and inheritence) - need to review and confirm then close as won't fix, or fix for 2.0.1 if I'm wrong.

Well, with M1 we used entities to ensure global version consistency among different projects. For M2 the dependencyManagement was presented as solution for this. If it is not DM, what can we use else?

I think I agree, as otherwise its just including the dependency falsely to ensure it is managed - just need to check the original discussion.

I think I agree too

Can you provide a link to the original discussion? I just cannot see how this behavior can be tolerated. Since almost every dependency at ibiblio is going to have transitive dependencies with versions in them not fixing this will effectively make transitive dependencies useless as most projects will end up with multiple versions of the same jars in them.

> "most projects will end up with multiple versions of the same jars in them."
That's wrong. What can happen is that different projects will use different versions of the dependencies, but one project always uses one and only one version

This depends. Think of jBoss with its unified classloader and an ear with two integrated wars. If each war has a dependency with a different version, both dependend jars will be in the classpath. It is pure coincidence which one is taken. This issue is therefore our showstopper in M102 to M2 migration.

Meanwhile we are hit by an even worse use case: EJBs! If you add the dependencies to the manifest's classpath, you don't have any control over the transient dependencies anymore. With two EJBs in the same EAR you have quite often the problem, that one of the EJBs is broken, since the two EJBs will refer the artifacts in different versions in their manifest, but the EAR contains naturally only one veriosn of it. This is meanwhile really a blocker for us!

Can someone change the description of this bug to say "transitive" instead of "transient"? That should make it easier to find...

and while you're at it change dependencyManagent into dependencyManagement in the title

This issue messes up the Maven2 based Shale builds too. The bottom line is that it puts the stability of Shale builds totally at the mercy of the creators of downstream dependencies ... there is no current way to override those settings and say "darn it ... I don't care what you say, use this version of the "foo" depency.".

I agree with Jorg and Craig. This would solve a lot of issues we are seeing in building our large system (40-50 modules) with Maven. All too often we need to custom craft each dependency's POM to standardize their versions on the versions we want to use.

My recommended approach to this is to adjust the version conflict resolution algorithm used to calculate "distance" on the dependency graph. Currently, the algorithm treats a transitive dependency inherited from a declared dependency, and a dependency inherited from a parent POM, as being at the same distance. I would advocate that dependencies declared in a parent POM (either explicitly, or with just a version number in a <dependencyManagement> section) should be considered closer, and therefore override version declarations in the transitive dependencies.

+1 for Craig's suggestion. The deterministic and user-controlled resolution of dependencies is a must in developing and testing any complex system, especially those that depend on projects that may be out of a user's control, such as open-source projects.

+1 as well. I need to ensure that I have dependable, reproducible build where I know for sure what dependency versions are being used. Craig's suggestion would solve this.

I would also say that if the dependency resolution algorithm should pick the latest version if all else is the same. I can't tell you the number of times I've had to deal with versions going from e.g. 1.1.2 -> 1.1.1 due to some change. If A requires 1.1.1 and B requires 1.1.2, Maven should default to the latter.

Cocoon now has an issue at http://issues.apache.org/jira/browse/COCOON-1890 which is a hack to try to deal with this issue. This is getting out of hand.

thanks for all the comments. This is certainly a priority to "fix" in 2.1, but won't be able to be done before, since it really changes the behaviour. I think appropriate measures need to be taken to ensure we bump the POM revision and retain the current behaviour for existing projects as this could potentially break builds.

Brett. This can easily be done without breaking compatibility. It wasn't too hard to figure out how to add an "override" child element to the dependencyManagement element. By using "false" as the default value for this compatibility is maintained. The problem I am having is in figuring out how to pass this flag to the artifact resolver since the dependency management object is converted into a simple Map.

Attached is a patch that addresses this issue. Without doing anything maven will behave as it currently does. However, adding <override>true</override> to the dependencyManagement section will cause the dependencyManagement section to be propogated to each child project and the versions in the dependencyManagement entries will override any versions specified in subprojects.

This patch applies to 2.0.x at revision 442440.

uploaded mng1577a.patch. The only change from the original patch was to not have child projects override versions specified in parents.

Anyone tried the patch yet? I'm considering upgrading us to use a patched binary to get this fix.

Pleaes wait until the end of the week. It was requested that I provide unit tests along with the patch. I've had a little trouble with that as Maven without the patches doesn't seem to handle DependencyManagement as documented. I'm going to try to talk to some of the Maven committers at ApacheCon this week to see if I can't get this put to bed.

mng1577b.patch and mng1577trunk.patch were given to Jason at ApacheCon but are being uploaded here for completeness. They contain 2 unit tests. The first, which tests override enabled passes. The second, which attempts to verify that override doesn't change the existing behavior, fails. However, this same unit test fails on an unmodified version of maven, indicating that the current behavior of dependencyManagement doesn't work as documented.

-1 on this patch as is.

If I have a dep in the child POM:

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>compile</scope>
</dependency>

and a dep in depMgmt in the parent POM:

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>

It forces the dep to the 'test' scope despite the fact that the child requires it as compile. The child should always override everything else. WYSIWYG.

I changed this in src/main/java/org/apache/maven/artifact/resolver/DefaultArtifactCollector.java

• if ( artifact.getScope() != null )
  + if ( artifact.getScope() != null && node.getArtifact().getScope() == null )
  {
  node.getArtifact().setScope( artifact.getScope() );

This fixed my problem above but it still does not work properly with transitive dependencies that are marked as optional. They become non-optional if listed in the dependency management section.

A transitive dep has it marked as compile/optional whereas my depMgmt section has it marked as test/non-optional. At runtime this becomes compile/non-optional and gets included in my EAR.

Also when I turn on <override>true</override>, my EAR build goes from 8 seconds to 40 seconds and my -X output goes from 170k to 9MB. I believe this is because it is parsing the same deps over and over, without caching intermediate results.

IOW, this patch needs more work.

Thanks for the feedback. I'll see if I can address these problems. Did you happen to create test cases that I can use?

I've uploaded mng1557c.patch for maven-2.0.x. I updated the patch with these changes:
1. The prior patch was creating a ManagedVersionMap several times for a project. Now it just does it once.
2. If no dependencyManagement section is present then the parent ManagedVersionMap is used instead of creating a clone.
3. If override is enabled then the scope in the dependencyManagement is not used at all (only the version is used). If override is false then it continues to override the scope for compatibility (although I can't imagine a case where this would be desirable).
4. Dependencies in child dependencyManagement sections override their parents. However, managed dependencies specified in the project will override transitive dependencies even if they extend a pom containing the same managed dependency. This is the desired behavior.
5. I tested building myfaces with an unmodifed maven, with override set to false and with override set to true and didn't see any real difference in cpu. The debug file is slightly larger due to a few extra debug log entries.

Please test this again and provide feedback.

Well darn. Don't bother testing mng1577c yet. I created a unit test to test the optional/non-optional issue that was raised and it is failing. FWIW, it does exactly the same thing with an unmodified maven. I'll have to run this under the debugger and figure out where it is coming from as the current patch isn't doing anything with optional/non-optional.

After digging into it it turns out I just didn't understand the results I was seeing. I've added a few tests to test the optional/non-optional behavior, but otherwise the patch is the same. Please test this. I'll also try to create the patch for trunk.

One thing I noticed related to this: if I override a version in dependencies (not in dependencyManagement), the pom of the "incorrect" version is still needed. This might be a problem, if that "incorrent" version of the pom can't be downloaded. It would be good not to require and download it at all. This same thing would then be nice to have also this pluginmanagement fix.

My opinion is that the scope should be also possible to override, because there are so many poms with invalid dependencies. Perhaps we need a second property overrideScope?

Ralph, I'm starting to test d now. First thing I noticed about the d patch is that it has ^Ms in all the new files. I believe this is a no-no. Be careful with your line endings.

Thanks, Mike. I upgraded to IntelliJ 6.0 and didn't check that setting. I'll make sure it is set as virtually everything requires it. I should have to corresponding patch for trunk done by tomorrow. Please let me know how your testing goes.

Uh oh, the depMgmt scope in the parent POM is still overriding the scope given in the leaf POM. This is bad - the leaf should always be able to override its parents settings. The depMgmt settings should only override transitive deps or provide info that is not given in the leaf (like if the scope is not given in the leaf, it should default to the scope in the parent's depMgmt).

And I'm testing against the 2.0.x branch so please target that code first if you are going to get me an updated patch.

Mike,

I suspect the behavior you are reporting is not part of the patch, but is part of how the pom's extend each other. In other words, I believe you will find that you get the same behavior without this patch applied. As part of my debugging I've noticed that every project that inherits from a parent has the dependencyManagement of its parent (plus whatever managed dependencies it introduces, if any). At first I was surprised by this but after thinking about it it made sense to me. However, since the code in my patch is simply taking the dependencyManagement that is provided and then is creating a ManagedVersionMap I've never investigated as to whether the dependencyManagement object was created correctly.

I've noticed other behaviors that are a bit odd. While trying to create unit tests I noticed that if a dependency did not have a version then all dependency versions in that project seemed to be overridden from the dependencyManagement version. If all dependencies had a version then none of the dependencies were overridden. I was only expecting the one dependency to have been affected.

I'll be happy to go on and try to figure out how to fix the behavior you've noted, but I don't think it should keep the patch from being applied as it is since I don't think it is breaking anything that isn't already broken. In fact, I've tried to keep all the broken behavior intact with override set to false in order to maintain backward compatibility.

Ralph, I tried plain 2.0.x HEAD this weekend and it does not fail with the scope issue I noted. So I would assume that the issue is in your patch.

I've uploaded a new patch file for both 2.0.x and trunk. The basic patch is unchanged however the t04 and t05 tests have been modified to verify that the scope is set to the correct value. t04 runs with override=true and so fails against an unpatched trunk (even if the override element is removed). t05 runs with override=false and will produce the same result when run on an unpatched trunk.

In short, these tests indicate that the patch is behaving correctly.

Hi all – Just a quick note to let everyone know that I am working on this issue as well – currently, I'm looking at the e patch for 2.0.x. I have a strong interest in and fair amount of bandwidth for getting this issue resolved, so feel free to contact me or whatever.

Given the long list of comments and the potential hurdles raised in them, is there any direction on the proposed solutions to the problem so far?

Here is an updated version of mng1577e.patch for the 2.0.x branch. It addresses not being able to bootstrap once the patch has been applied. (Some inheritance tests, added by the patch, were not finding their dependency jars.)

Patch to the maven web site to document the new behavior.

Patch applied, thanks Ralph.

This is the 1577 2.0.x patch ported so it will cleanly apply to the trunk @ rev 520633. Unit tests had to be refactored such that artifact names are unique. Verified that transitive deps are resolved correctly using the same small smoke tests and one large build that were tested for the the 2.0.x patch.

Thanks, patch applied on trunk.