"provided" scope dependencies must be transitive

Your point may be correct, but your use case is wrong.

if C wants to use Sybase JConnect then it must declare this as a dependency. A could at any time change it's dependencies and "break" this assumption of C's.
It is wrong to use a dependency that you do not declare.

I agree with you in some sense. Sybase JConnect isn't a great example.

However, your "any time change its dependencies" is not at all accurate. A project depends on a released version of, say Apache DBCP. Once it's released, its dependencies are set in stone. The whole point of transitive dependencies is that you benefit from not having to redundantly redeclare you dependencies at each level. When you're upgrading your dependencies, you have time to make a good decision on whether the new dependency tree meets your needs.

Is the logical conclusion of your point that we shouldn't bother with transitive dependencies because we always need the safety of declaring them in each project?

No, you're using "provided" in a wrong manner. A provided means that the artifact is available in the target environment i.e. if you build an enterprise application for a J2EE server you can safely assume that javamail or activation is available, since it is part of the JEE stack. In your case you are introducing a runtime dependency and you should use therefore scope "runtime".

That's why JConnect is a terrible example.

A better example of this sort of dependency is XML Beans. XML Beans will not work based on a "runtime" dependency because your code will require it on the compilation classpath. If all your products depend on XML Beans, you want to be able to declare it as a base dependency and have all your modules inherit that dependency.

If you make provided transitive, then what is the difference between the "provided" and "compile" scopes? Doesn't the compile scope already give the transitive behaviour that you're looking for?

From your original example above, I don't see where the problem is. If project C is using JDBC with a JConnect driver, and JConnect is being provided on the runtime classpath, by the app server or whatever, then I would think your app would run fine.

The XML beans example sounds like it should be compile scope. It is required on the compile classpath, and should be inherited transitively. Why not use the default compile scope for this?

The difference from PROVIDED to the COMPILE dependency would be, that in the PROVIDED scope, the dependent artifact is not packaged in the target artifact (since its provided by the runtime environment), whereas the COMPILE depedency is alwas packaged into the target artifact which is not always what you want. think of an example like a jdbc-driver, which is installed in your tomcat or jboss; you do not want to package this one in your WAR or EAR artifact too, since its PROVIDED....

Sorry I think I'm still missing something. So then what is wrong with PROVIDED the way it currently functions? The JConnect dependency would not be included in the war because it is provided by the container, and project C would be able to use the JConnect driver at runtime because it is provided in the classpath by the app server, right?

The problem is that project C is not able to use JConnect classes directly. It's not on project C's compile classpath.

Why not? Why not indeed. It should be! That's what this bug is aiming to fix.

JConnect is a bad example, because you typically don't access JConnect classes directly, you're more likely to use the JDBC API and hide the implementation details.

Eclipse RCP is a better example (it's a real world example for me). Project A is an Eclipse RCP component. It's used by Project B which is in turn used by Project C. Now, because of the way we package the application, the Eclipse RCP (big) jar files are downloaded separately and are provided for the distribution of my application. The problem is that why should I explicitly declare a dependency from Project C to Eclipse RCP? It should be inherited transitively. The default behaviour should be that I can use the Eclipse RCP API in Project C.

I think distinguishing packaging and compiling would make sense here. I use compile scope as an example:

• for direct dependencies it's clear that they are included in you compile time classpath and also included when packaged to a war for example.
• for transitive dependencies, it's clear they should be packaged but the question is if they should be included in the compile time classpath.

Ok, let's have an example:

1. Current behavior:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: included in A's compile classpath, included in packaging
D: not included in A's compile classpath, not packaged

2. Requested in this issue:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: included in A's compile classpath, included in packaging
D: included in A's compile classpath, not packaged

3. Preferable solution in my opinion:
A -> B (compile) -> C (compile)
A -> B (compile) -> D (provided)

C: not included in A's compile classpath, included in packaging
D: not included in A's compile classpath, not packaged

If I remember correctly there were some talk in the mailing list that maven 2.1 would implement option 3, but I might remember wrong.

edit: this seem to be bit more complicated than that: how about classpath for compiling and running tests etc.
As a quick thought, classpath for compiling tests could be just the direct dependencies and for running tests same as for packaging (but how about transitive test dependencies)?

I think that 'provided' transitive dependencies are important! The case that comes to my mind is OSGi and Eclipse development. You need a certain class/package in order to compile your code; however that same package will be available on the runtime environment as a bundle. You don't want to start adding this jar as an embed library into your bundle! You want to embed only those that are runtime/compile dependencies.

However since 'provided' is NOT transitive you would have to list all the dependencies this bundle you depend on has in your POM! Not exactly a step forward from Maven 1. You would like to pull in these bundle dependencies just for the need of compilation (and actually you would what them in the manifest file too).

So in the end for OSGi and Eclipse transitive provided dependencies would be a really nice thing to have!

What I'd like to do is create a POM-type artifact which is basically just a grouping of dependencies used by other artifacts.

<dependencies>
<!-- jboss/server/default/lib -->
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.3</version>
<scope>provided</scope>
</dependency>
....

Then I would add this artifact as a provided dependency to all artifacts.
The desired result would be an inclusion of all dependencies from this container artifact in my other artifacts, marked as provided.

You may wonder why I'm not just marking them as compile-scoped in the container artifact; here's why:
I have an artifact that creates a ZIP file containing the JBoss AS. This artifact basically just downloads JBoss, extracts it, messes around with its configuration, and packages it as a ZIP file. Additionally, all artifacts from the container artifact above that are not marked as provided (because they are not already provided by JBoss) are added to the jboss/server/default/lib/ directory just before zipping it up. That means I can specify two types of provided dependencies in the container POM above:

• Dependencies provided by the JBoss AS default configuration (<scope>provided</scope> in the container POM).
• Dependencies that I add to my JBoss' server/default/lib directory (<scope>compile</scope> in the container POM).

Every artifact that depends on the container POM and sets that dependency to a <scope>provided</scope> should have all dependencies in the container pom, be they provided or compile, added to the artifact as provided. This is exactly as described by the documentation here:
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
With one single exception: I cannot yet mark provided dependencies in the container POM as transitive.

The table on the link above says "provided" in the intersection of "provided" on the left side and "provided" on the top. Reading the text above the table you will see this implies that:
If a dependency is set to [provided], transitive dependencies of that dependency with the scope [provided] will result in a dependency in the main project with the scope [provided]. If no scope is listed, it means the dependency will be omitted.

I would've thought that the author would've just put a "-" at the intersection of provided with provided seeing as there currently is no such thing as transitive provided dependencies. Or is there?

There is another very important reason why provided dependencies should be transitive. This reason is tests.

If you have some class that needs (lets say) servlet-api and servlet-api is provided, then you cannot use this class in another project, because when you write test, you are not able to instantiate your original class because it imports javax.servlet.* that is not available.

You also cannot extend any created servlet class in any different project, because it will not compile if Serlvet interface is not available on classpath.
This is similar to the note you have just under the table at
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope

I believe that the most typical situation is to scope servlet-api as provided and it shows, that it doesn't work as it should.

Hi,

In the context of a "war" a dependency that is "provided" can mean 1 of two things:

1. It is provided within the application ear
OR
2. It is provided by the environment.

The solution for (1) is easy because we could have "provided" being transitive and have the ear-plugin package the "provided" dependencies.

The solution for (2) is tricky because (as in the servlets example) we need it for compilation and we do NOT want it packaged by the ear-plugin since it is provided by the appserver or jvm path. I considered "runtime" but this would make it unavailable during compilation and with the servlet -api example we need those dependencies during compilation. "optional" for (2) works fine however the reasons why it works are not obvious to the layman.

The solution is to introduce a new state called "environmental" and duplicate functionality around "optional".
Make a "provided" transitive and have all class-packaging plugins (maven-ear-plugin, maven-war-plugin) support the correct packaging of "provided" dependencies.

From my understanding of the maven code base (i've been digging) this is relatively do-able and the semantic changes are not going to be too bad considering the way people work around the problem of creating wars. I have only one problem which is the need to rewrite the manifest within jar/war/etc to support bundled libraries with different jar locations and signed jar/wars but again i believe this can be managed in the context of how people work around skinny jars at the moment.

-Tim

I believe the right behavior is postulated in the Maven Scope Matrix (http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html).

There a combination of provided/provided (for dependency and a dependency in the dependency) leads to a provided scope for the secondary dependency (as it should be).

I'm not entirely sure if a xxx/provided dependency should be transitive. Thinking about OSGi environments, this might lead to version issues.

Anyway, maven (as of 2.2.1) does not even handle the provided/provided case as given in the Maven Scope Matrix.

This 'critical' issue seems to be dead somehow, so I thought I put in my 2 pence and a bugfix.

First a short summary to the problem as it was a bit blurry in all the various comments: I believe (and I'm not alone there) that we need to make provided dependencies transitive. This is how it used to be in the dependency matrix, however it was never implemented that way, hence this issue was created. Instead of solving it by changing the code, the dependency matrix has been amended to indicate that provided dependencies are not transitive. This has happened silently, at least no comment was left in this issue regarding that change of the documentation.

Let me point out why provided dependencies should be transitive with a simple example using the projects A, B and C:
A doesn't depent on anything. B depends on A. And C depends on B.
Now you want to run unit or integration tests on C. It is straightforward to see that C must have a transitive provided dependency on A, because B (which C depends on) needs A to function. Therefore A must be in the classpath, right? With the current implementation, A will not be in the dependency scope of maven at all, because provided dependencies are not regarded as transitive dependencies.

Also the combination compile/optional is not a workaround, as it basically duplicates the provided scope - I however can't see a difference in behaviour between 'compile/optional' and 'provided'.

I'll attach these 'A,B,C' poms so that you can play around yourself.

As a possible resolution I forked mvn in github and provide a bugfix commit:
https://github.com/mpollmeier/maven-3/commit/3339789d6ae2dcbdc0e98d7b47a18f8a2aa2893e

Please let me know your thoughts and pull the change if you like it.

here's my use case:
i have a module with "provided" dependencies on jboss modules. my tests deploy code to an actual jboss server where these "provided" deps exist, but i also have those same provided dependencies on the test classpath. since im booting jboss embedded (using the arquillian project) this causes class-loading issues. i would love for those dependencies to simply not be there during tests.
maybe instead of breaking current behavior (as having "provided" deps on the lass path does make sense in certain scenarios) it would be possible to add a new scope ? "provided-transitive", "provided-test" or whatever that would cause dependencies to be "provided" in tests as well ?

I really like the 3rd solution given by Sei Syvalta (http://jira.codehaus.org/browse/MNG-2205?focusedCommentId=131081&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-131081).

I'd like to vote that one in particular.

I think it would make for more stable builds (or rather updating dependencies), such as:
Dozer (net.sf.dozer:dozer:jar:5.3.2) currently depends on commons-lang:commons-lang:jar:2.5
If in my project I compile-time depend on dozer, I also have commons-lang on the compile classpath.
If I Dozer's next version refactors everything to use Google's Guava instead (com.google.guava:guava:jar:11.0.2), then my code will break.. for something (basically) completely unrelated.

Hi all...
I'm in a OSGi environment and I've the same problem.
I need to compile a project

A -> B (provided) -> C (provided)
B is an OSGi bunlde and A needs to use some C's packages/classes

I agree with Alexandre

IMHO it's important to have a simple option to add a transitive 'provided' dependency into the compile classpath.
Could be a simple solution to add a flag into the pom file (like system scope? ) that allows a provided dependency to be transitive?

i.e.

<dependency>
   <groupId> foo.bar </groupId>
   <artifactId> project </artifactId>
   <version> 1.0.0 </version>
   <scope>provided</scope>
   
   <transitive> true </transitive>

</dependency>

thanks and have a nice day

I have voted for this.
Half-abstract and half-concrete example: A -> B(compile) -> servlet-api(provided), A must have an access to servlet-api classes (or at least have such possibility), because A must sit in web-container, so B has no other choice. In some cases A don't need classes from servlet-api, but this is not a problem, "compile" scope creates more problems in this case, because it pulls unnecessary dependencies along into package.
I think the solution may be removing "optional" tag from dependency definition. Instead the tag "transitivity" can be introduced, which may be flexible and has different values:
1) 'transitive' - always transitive
2) 'optional' - as optional=true in current variant
3) 'detect' - transitive for cases when "The type <NAME> cannot be resolved. It is indirectly referenced from required .class files" compile error appears
4) 'non-transitive' - always non-transitive
5) etc
For backward compatibility 'transitive' can be default for "compile" scope and 'non-transitive' for "provided"

Issue has been created: 06/Apr/06 5:37 AM
6 years old!!! Should be already fixed.

For what it's worth, I no longer agree with this issue that I started 6 years ago! I have a best-practice document which I apply to my projects and have pasted here in this comment. Under this strategy, I always declare compile-time dependencies in any project that has an import; statement referring to the class. I don't rely on transitive dependencies when I'm actually using a dependency for compilation. I think this is what the Maven authors intended, which is why "provided" has stayed like it is for such a long time.

1. <dependencyManagement/> in parent pom
Declare the versions for all dependencies in the project's parent pom using the <dependencyManagement/> section. You're setting the versions that will be used if the submodules decide to declare dependencies. Using dependency management ensures that the version of all of your dependencies is entirely consistent across all your sub-projects. This is especially important if you're creating a single deployment consisting of many of these sub-modules and you are going to deploy one set of dependency jars

Declare few or no dependencies in the <dependencies/> section of the parent pom. It's fair to assume that all your projects will depend on JUnit and a logging framework so you may want to add those. Anything else results in unnecessary dependencies making their way into your sub-projects.
It's not too useful to declare versions at an even higher level (e.g. the pom that's the parent of all your projects) but in some cases it can make life easier.
Failing to add a dependencyManagement section results in a classic problem of scripts being generated in sub-projects which reference jar versions that don't end up being packaged in the final assemby's lib folder.
2. Declare dependencies where you use them - don't rely on transitive dependencies
Transitive dependencies are very useful. If you're using the public API of a library in your code then you shouldn't have to care how its internals work. Your sub-project should declare a dependency on the library that your code is using; Maven will helpfully pull in the library's transitive dependencies. The library is free to change how its internals work and what transitive dependencies it declares for each version.

If you're using a library, you should declare it in the <dependencies/> section of your pom. Don't rely on transitive dependencies. If your Java code (or Spring config; that's code too) uses commons-beanutils then you should declare that dependency. If you don't, your build is brittle; internal changes to other components can break your component's build.

Run:

mvn dependency:analyze
This tool will tell you about artifacts that you've failed to declare dependencies for and will also tell you if you have extra dependencies declared that you don't need. Be careful about dependencies that you don't use in your .java files; the dependency plugin is only scanning those. If you use a class in Spring config then you do really need it, even if the dependency plugin is telling you that you don't. In these cases I find it useful to make sure that the dependency is set to <scope>runtime</scope>. That way, when I get the dependency:analyze report telling me that these are extra dependencies I know that I probably set them to runtime because they're used by spring config. I ignore the runtime dependencies and only weed out the compile scope ones.

3. Use <properties/> sparingly (in the parent pom)
Declaring all dependency versions in properties and then using those properties in the <dependencyManagement/> section is a little like the (bad) practice of unnecessarily declaring all your Java variables right at the top of a method.

When there's a single artifact to configure, favour hardcoding the version into the <version>1.0</version> tag.

Use properties when there are multiple artifacts to control. For example, set:

<spring.version>3.1.1.RELEASE</spring.version>
and then use:

<version>$

</version>
for artifacts like <artifactId>spring-core</artifactId>

Properties are public, not private. When you declare a property in the parent pom, it's available anywhere in the build. Adding a property pollutes the (build) environment.

4. Use the enforcer plugin to guard against old or unwanted dependencies
The enforcer plugin can fail your build based on duplicate classes found (the same classes in more than one jar file on the build classpath) or can allow you to ban old or unwanted dependencies. This is especially useful when the owner of a library has changed the groupId or artifactId of their artifact. This stops the Maven dependency mechanism from picking the latest version; you'll end up with 2 similar artifacts on the classpath. For example, from Spring 2 to Spring 3 they changed an artifactId from spring to spring-core. Without banning the older spring dependency, it can easily transitively creep back onto your classpath. Then when your program runs it's somewhat pot luck as to which jar is first on the classpath and therefore which version of a class gets loaded.

5. Detect whether you're using old plugin or dependency versions
The following goals of the versions plugin can help you keep up to date with the latest dependencies. Ideally, get your continuous integration server to display this information.

mvn versions:display-plugin-updates
mvn versions:display-dependency-updates