Regression: Maven drops dependencies in multi-module build

Attachment with zipped demo project.

Build instructions:

• extract ZIP
• go into extracted root directory
• install parent pom first: mvn -f parent/pom.xml
• install multi-module: mvn install

The build should fail .

Go into module5 and build from there: mvn install

This time it should succeed.

*) Before I've zipped the project I tried to replace any occurrence of "XXX" with "3259" in the 7 pom.xml files, but after that action M208 can suddenly build this! Therefore I've added also an "install.log" to the project root that demonstrates the problem. This means additionally that this build might behave differently on different JDKs. I highly assume that there are still some HashMaps used internally that have influence about the sequence the dependencies are processed, therefore I'd expect Maven to behave differently, e.g. this sequence is different for HashMaps in JDK 6 and the project can be build running M208 with JDK 6! See also first lines of install.log for my environment:

Maven version: 2.0.8-SNAPSHOT
Java version: 1.4.2_13
OS name: "windows xp" version: "5.1" arch: "x86" Family: "windows"

This project depends on an ejb jar that isn't on repo. Any way we can use something else? This will make it impossible to make an IT for this since we can't redistribute the jar.

I installed the jar and I can't reproduce this so far.

I verified this on 2.0.7/8 and 2.1 using java 1.5. It seems ok with 1.6

I noticed that you had exclusions in various places for the artifact that is missing (xstream). Why are those exclusions there? If I remove them, then everything builds fine, but the order of resolution is slightly different.

Improved demo project.

> This project depends on an ejb jar that isn't on repo. Any way we can use something else?
> This will make it impossible to make an IT for this since we can't redistribute the jar.

I replaced javax.ejb:ejb with an old version of jboss:jboss-j2ee and the problem still occurs - so far so good. Also removed the unnecessary parent pom of the root.

> I installed the jar and I can't reproduce this so far.
> I verified this on 2.0.7/8 and 2.1 using java 1.5. It seems ok with 1.6

As I said, fails with JDK 1.4.2 and 1.5.0, works for 1.6.0 - therefore I assume that a HashMap is involved when processing the sequence of the deps, since some internals of the HashMap changed in JDK 1.6 causing this effect - which should not make any difference for any algorithm using a HashMap, since it is an expected behaviour.

> I noticed that you had exclusions in various places for the artifact that is missing (xstream).
> Why are those exclusions there? If I remove them, then everything builds fine, but the
> order of resolution is slightly different.

This is typical for artifacts containing business delegates. Those artifacts are used in web clients and they may not include all dependencies used to implement the business logic on the server. The coincidence here is, that the excluded deps are not used at the client, should not be included in the resulting war and should not even be available as transitive dep. However, the excluded artifact is necessary for running the tests (or compiling, simply add a refernce to the missing class into the test code and the build fails earlier). With M205 Maven still provides this artifact as test dependency, while with M207 and M208 the dependency is suddenly gone.

Module 1 is in the real world basically a test framework on top of jMock and JUnit where we do not really care about all the deps it needs, since it is for test only.

You are also able to use a workaround by declaring the missing dependency explicitly in the POM to be used for test scope, but in our real build it does not really help, since then surefire in M207/M208 is missing the next class from a different artifact that should be available as transitive dep from the test framework.

Point is, that the algorithm calculating the deps and the classpaths in M207/M208 provides different results when run locally or from a parent.

BTW: Thanks for looking at this, it drives us really crazy.

This is related to natural set ordering as assumed. The problem seems to be that the testClasspathElements is missing these artifacts (xstream etc) but they are present when running dependency:resolve or tree. The case that works, sets xstream to compile first then test, the case that doesn't sets it to test then compile. Somewhere in the artifact collector, the scope is getting slammed and then I think the tree is being trimmed for a closer dependency that is excluded. It's a complex example that has dependencyMgt, exclusions and reactor siblings all involved. Could you have found a more complex case

I'm getting close...

Guess, I have to apologize now ...

I am unable to reproduce the error.

Steps:

• Used MNG-3259-2.zip
• Installed parent pom
• Installed multiproject

Maven versions: 2.0.7 and 2.0.8-SNAPSHOT

OS name: "sunos" version: "5.10" arch: "x86" Family: "unix"

Java versions:
Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)
Java HotSpot(TM) Server VM (build 1.5.0_13-b05, mixed mode)
Java HotSpot(TM) Client VM (build 1.4.2_15-b02, mixed mode)

Expected result:

• Multiproject build should fail

Actual result:

• Multiproject build passes.

There is definately an issue. I think though that there are plenty of workarounds and this is a far edge case. I think the best thing is to bump this to 2.0.9 because any fix we make is likely to break something and we should have plenty of time to flush it out.

Oh, please don't. I already reported the problem as regression for 2.0.7, but was not able to find a test case to reproduce the effect:
http://mail-archives.apache.org/mod_mbox/maven-dev/200706.mbox/<F0D7281DAB048B438E8F5EC4ECEFBDDC0173C7E3%40esmail.elsag.de>. I follow all the issues about dependency problems quite close and I really thought that the causing issue has been fixed - unless I tried the new RC.

And, no, we're stuck to 2.0.5, because we don't have a workaround. As said, if we add the missing artifact directly with scope test, the test fails with another missing class. IMHO it is quite scary, that Maven drops dependencies under some circumstances. And staying with M205 starts to be a real pain, since quite some new versions of plugins require now newer versions.

I can also confirm Asgeir's comment, the build does not fail for me on Linux, so it seems to be bound to a Windows installation - for whatever weird reason, but see also:
MNG-2962
MNG-3061

At least the natural order should not make any difference, then we would have a chance of sorting the deps accordingly. Can you simply replace the usage of the involved ordinary HashMaps with LinkedHashMaps?

If it appears only on Windows – could it be related to Windows' weird case-insensitive, but case-preserving file naming?

I would guess that the strings c:\Windows and C:\WINDOWS have different hashcodes (and thus ordering could change), but are considered equal by the file system.

Furthermore, the backslash \ vs forward slash / in file system paths could also cause equivalent paths to be treated different in a HashMap.

Its not as scary as you think. It seems to be related to the fact that you are excluding the same artifacts in some places, but want them in test scope in others. I think this is a rare occurance. You should be able to just add the ones you want as test scope.

I'll try to look some more, but if it's not an easy fix then we have to bump it. The chances of it breaking other behavior is probably greater.

Also, this isn't a windows only issue, i have reproduced it on solaris. For me it only happens on jdk 1.5 or less.

I think we need to bump this to 2.0.9. After looking at it for a while, I have a general idea of the problem, but the fix has a high probability of breaking things further. There are several ways to work around this particular issue and from what I've seen, it occurs only in a very specific case: you are excluding a dependency from being pulled in transitively but then need the same dependency in the test scope.

For us it simply means that M2 is non-usable since 2.0.6.

Add the missing dependencies to your pom and it should be fine right?

As I already said, if I add that particular dep in my real project, Maven complains about the next one missing. We miss a complete tree of transitive runtime deps. And you can recognize that only if you have actually tests that use the missing class indirectly. Without 100% test coverage you may easily end up with a broken ear or web app. Also see the linked issue, we are not the only ones suffering from this problem. If at least there was no natural order of a HashMap involved, then we might be able to "sort" our dependencies in a way where nothing gets lost, but currently it is pure coincidence, if it works.

I tried it. But I can shuffle the sequence of the dependencies how I like, those transitive deps are missing in my real project.

The issue is in the maven-artifact resolver.DefaultArtifactCollector in this vicinity of this line:

                           // Also, we need to ensure that any exclusions it presents are
                            // added to the artifact before we retrive the metadata
                            // for the artifact; otherwise we may end up with unwanted
                            // dependencies.
                            Artifact ma = (Artifact) managedVersions.get( childKey );
                            ArtifactFilter managedExclusionFilter = ma.getDependencyFilter();
                            if ( null != managedExclusionFilter )
                            {
                                System.out.println("Adding Exclusion Filter:"+ma);
                                if ( null != artifact.getDependencyFilter() )
                                {
                                    AndArtifactFilter aaf = new AndArtifactFilter();
                                    aaf.add( artifact.getDependencyFilter() );
                                    aaf.add( managedExclusionFilter );
                                    artifact.setDependencyFilter( aaf );
                                }
                                else
                                {
                                    artifact.setDependencyFilter( managedExclusionFilter );
                                }
                            }
                        }

I believe the scope transformation is taking place before the exclusion is processed, thus the artifact is first dumped out of the compile classpath and then erroneously excluded.

I downloaded MNG-3259-2.zip. I updated maven-2.0.x to the latest code. I ran the build from my ubuntu VM on Windows. It worked. The output from mvn -v there is:
Maven version: 2.0.9-SNAPSHOT
Java version: 1.5.0_14
OS name: "linux" version: "2.6.22-14-generic" arch: "i386" Family: "unix"

I then ran the test in cygwin on Windows. It worked. mvn -v shows
Maven version: 2.0.9-SNAPSHOT
Java version: 1.5.0_09
OS name: "windows xp" version: "5.1" arch: "x86" Family: "windows"

I then went to a "normal" Windows command shell and ran mvn clean and then mvn install. It completed successfully. The output from mvn -v was
Maven version: 2.0.9-SNAPSHOT
Java version: 1.5.0_09
OS name: "windows xp" version: "5.1" arch: "x86" Family: "windows"

Is there anywhere a M209-SNAPSHOT available? I am happy to test it myself. I am really eager to get rid of this issue, since it forces us to uses M205 still.

No. You have to build it your self. To do this you need subversion installed. Then do
1. Create a directory named apache-maven-2.0.9-SNAPSHOT somewhere
2. set M2_HOME to point to the apache-maven-2.0.9-SNAPSHOT directory
3. From a directory where you want the maven source do svn co https://svn.apache.org/repos/asf/maven/components/branches/maven-2.0.x/
4 cd to maven-2.0.x
5 run ./bootstrap.sh
6 add M2_HOME/bin to the beginning of your path
7 do mvn -v and verify that your version is 2.0.9-SNAPSHOT.

Note that I am not claiming that anything was fixed. I just couldn't duplicate the problem on Windows with Java 5.

Hi Ralph, you need 1.4 to duplicate it.

I just tested this on OS X and WinXP, using maven 2.0.7, 2.0.8, and 2.0.9-SNAPSHOT from revId: 630321. All of my tests used JDK 1.4.

In the cases of 2.0.8 and 2.0.9-SNAPSHOT on OS X, it didn't fail.

In the cases of 2.0.7 and 2.0.8 on WinXP, it failed.

In the case of 2.0.9-SNAPSHOT on WinXP, it didn't fail.

I'm not sure, but it looks like it has been fixed inadvertently...not a great thing to say, but there you go. I'd welcome some more eyes on this issue, to see if you can prove me wrong/crazy/something.

I built a distro from that revision, and parked it here:

http://people.apache.org/~jdcasey/maven-drops/2.0.9-SNAPSHOT

Please reopen this issue if you can't replicate my success.

Although I feel a bit uncomfortable, if such a problem suddenly vanished without identification of the originating reason, I have to confirm that this SNAPSHOT is the first version since M205 that builds our complete project tree without any problem:

=========== %< =========
[SNIP]
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
...
[SNIP]289 projects
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44 minutes 3 seconds
[INFO] Finished at: Mon Feb 25 09:59:29 CET 2008
[INFO] Final Memory: 160M/254M
[INFO] ------------------------------------------------------------------------
=========== %< =========

This is really such a great relief! Thanks to all of you looking into this!
BTW John, when did you claim, M209 is gonna released? GO GO GO GO