Maven
  1. Maven
  2. MNG-2205

"provided" scope dependencies must be transitive

    Details

    • Type: Bug Bug
    • Status: Open Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.x / Backlog
    • Component/s: Dependencies
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      1

      Description

      A provided scope dependency can also be thought of as "compile-only".

      Project A requires Sybase JConnect on the runtime classpath. Project A declares a "provided" dependency on Sybase JConnect.
      Project B depends upon Project A. Project B declares a "compile" dependency on Project A.
      Project C depends upon Project B. Project C declares a "compile" dependency on Project B.

      C
      | - compile dependency
      B
      | - compile dependency
      A
      | - provided dependency
      Sybase JConnect
      

      So, does Project C transitively depend on Sybase JConnect. Yes, of course! The "provided" dependency needs to be transitive.

      Ultimately, when Project C gets deployed, Sybase JConnect needs to be somewhere on the runtime classpath in order for the application to function. It's valid for Project C to assume that Sybase JConnect is available and use JDBC all over the Project C code. Project C is safe to do this because it can happily deduce that Sybase JConnect will be there in the runtime environment because Project A NEEDS IT.

      I've got Use Cases all over my aggregated build which make it absolutely critical and common sense that provided scope dependencies are transitive. For the (very rare) odd case where you don't want to inherit provided dependencies, you can <exclude/> them.

        Issue Links

          Activity

          Hide
          Andrew Williams added a comment -

          Your point may be correct, but your use case is wrong.

          if C wants to use Sybase JConnect then it must declare this as a dependency. A could at any time change it's dependencies and "break" this assumption of C's.
          It is wrong to use a dependency that you do not declare.

          Show
          Andrew Williams added a comment - Your point may be correct, but your use case is wrong. if C wants to use Sybase JConnect then it must declare this as a dependency. A could at any time change it's dependencies and "break" this assumption of C's. It is wrong to use a dependency that you do not declare.
          Hide
          David Boden added a comment -

          I agree with you in some sense. Sybase JConnect isn't a great example.

          However, your "any time change its dependencies" is not at all accurate. A project depends on a released version of, say Apache DBCP. Once it's released, its dependencies are set in stone. The whole point of transitive dependencies is that you benefit from not having to redundantly redeclare you dependencies at each level. When you're upgrading your dependencies, you have time to make a good decision on whether the new dependency tree meets your needs.

          Is the logical conclusion of your point that we shouldn't bother with transitive dependencies because we always need the safety of declaring them in each project?

          Show
          David Boden added a comment - I agree with you in some sense. Sybase JConnect isn't a great example. However, your "any time change its dependencies" is not at all accurate. A project depends on a released version of, say Apache DBCP. Once it's released, its dependencies are set in stone. The whole point of transitive dependencies is that you benefit from not having to redundantly redeclare you dependencies at each level. When you're upgrading your dependencies, you have time to make a good decision on whether the new dependency tree meets your needs. Is the logical conclusion of your point that we shouldn't bother with transitive dependencies because we always need the safety of declaring them in each project?
          Hide
          Jörg Schaible added a comment -

          No, you're using "provided" in a wrong manner. A provided means that the artifact is available in the target environment i.e. if you build an enterprise application for a J2EE server you can safely assume that javamail or activation is available, since it is part of the JEE stack. In your case you are introducing a runtime dependency and you should use therefore scope "runtime".

          Show
          Jörg Schaible added a comment - No, you're using "provided" in a wrong manner. A provided means that the artifact is available in the target environment i.e. if you build an enterprise application for a J2EE server you can safely assume that javamail or activation is available, since it is part of the JEE stack. In your case you are introducing a runtime dependency and you should use therefore scope "runtime".
          Hide
          David Boden added a comment -

          That's why JConnect is a terrible example.

          A better example of this sort of dependency is XML Beans. XML Beans will not work based on a "runtime" dependency because your code will require it on the compilation classpath. If all your products depend on XML Beans, you want to be able to declare it as a base dependency and have all your modules inherit that dependency.

          Show
          David Boden added a comment - That's why JConnect is a terrible example. A better example of this sort of dependency is XML Beans. XML Beans will not work based on a "runtime" dependency because your code will require it on the compilation classpath. If all your products depend on XML Beans, you want to be able to declare it as a base dependency and have all your modules inherit that dependency.
          Hide
          Paul Gier added a comment -

          If you make provided transitive, then what is the difference between the "provided" and "compile" scopes? Doesn't the compile scope already give the transitive behaviour that you're looking for?

          From your original example above, I don't see where the problem is. If project C is using JDBC with a JConnect driver, and JConnect is being provided on the runtime classpath, by the app server or whatever, then I would think your app would run fine.

          The XML beans example sounds like it should be compile scope. It is required on the compile classpath, and should be inherited transitively. Why not use the default compile scope for this?

          Show
          Paul Gier added a comment - If you make provided transitive, then what is the difference between the "provided" and "compile" scopes? Doesn't the compile scope already give the transitive behaviour that you're looking for? From your original example above, I don't see where the problem is. If project C is using JDBC with a JConnect driver, and JConnect is being provided on the runtime classpath, by the app server or whatever, then I would think your app would run fine. The XML beans example sounds like it should be compile scope. It is required on the compile classpath, and should be inherited transitively. Why not use the default compile scope for this?
          Hide
          Ramon Grunder added a comment -

          The difference from PROVIDED to the COMPILE dependency would be, that in the PROVIDED scope, the dependent artifact is not packaged in the target artifact (since its provided by the runtime environment), whereas the COMPILE depedency is alwas packaged into the target artifact which is not always what you want. think of an example like a jdbc-driver, which is installed in your tomcat or jboss; you do not want to package this one in your WAR or EAR artifact too, since its PROVIDED....

          Show
          Ramon Grunder added a comment - The difference from PROVIDED to the COMPILE dependency would be, that in the PROVIDED scope, the dependent artifact is not packaged in the target artifact (since its provided by the runtime environment), whereas the COMPILE depedency is alwas packaged into the target artifact which is not always what you want. think of an example like a jdbc-driver, which is installed in your tomcat or jboss; you do not want to package this one in your WAR or EAR artifact too, since its PROVIDED....
          Hide
          Paul Gier added a comment -

          Sorry I think I'm still missing something. So then what is wrong with PROVIDED the way it currently functions? The JConnect dependency would not be included in the war because it is provided by the container, and project C would be able to use the JConnect driver at runtime because it is provided in the classpath by the app server, right?

          Show
          Paul Gier added a comment - Sorry I think I'm still missing something. So then what is wrong with PROVIDED the way it currently functions? The JConnect dependency would not be included in the war because it is provided by the container, and project C would be able to use the JConnect driver at runtime because it is provided in the classpath by the app server, right?
          Hide
          David Boden added a comment -

          The problem is that project C is not able to use JConnect classes directly. It's not on project C's compile classpath.

          Why not? Why not indeed. It should be! That's what this bug is aiming to fix.

          JConnect is a bad example, because you typically don't access JConnect classes directly, you're more likely to use the JDBC API and hide the implementation details.

          Eclipse RCP is a better example (it's a real world example for me). Project A is an Eclipse RCP component. It's used by Project B which is in turn used by Project C. Now, because of the way we package the application, the Eclipse RCP (big) jar files are downloaded separately and are provided for the distribution of my application. The problem is that why should I explicitly declare a dependency from Project C to Eclipse RCP? It should be inherited transitively. The default behaviour should be that I can use the Eclipse RCP API in Project C.

          Show
          David Boden added a comment - The problem is that project C is not able to use JConnect classes directly. It's not on project C's compile classpath. Why not? Why not indeed. It should be! That's what this bug is aiming to fix. JConnect is a bad example, because you typically don't access JConnect classes directly, you're more likely to use the JDBC API and hide the implementation details. Eclipse RCP is a better example (it's a real world example for me). Project A is an Eclipse RCP component. It's used by Project B which is in turn used by Project C. Now, because of the way we package the application, the Eclipse RCP (big) jar files are downloaded separately and are provided for the distribution of my application. The problem is that why should I explicitly declare a dependency from Project C to Eclipse RCP? It should be inherited transitively. The default behaviour should be that I can use the Eclipse RCP API in Project C.
          Hide
          Sei Syvalta added a comment - - edited

          I think distinguishing packaging and compiling would make sense here. I use compile scope as an example:

          • for direct dependencies it's clear that they are included in you compile time classpath and also included when packaged to a war for example.
          • for transitive dependencies, it's clear they should be packaged but the question is if they should be included in the compile time classpath.

          Ok, let's have an example:

          1. Current behavior:
          A -> B (compile) -> C (compile)
          A -> B (compile) -> D (provided)

          C: included in A's compile classpath, included in packaging
          D: not included in A's compile classpath, not packaged

          2. Requested in this issue:
          A -> B (compile) -> C (compile)
          A -> B (compile) -> D (provided)

          C: included in A's compile classpath, included in packaging
          D: included in A's compile classpath, not packaged

          3. Preferable solution in my opinion:
          A -> B (compile) -> C (compile)
          A -> B (compile) -> D (provided)

          C: not included in A's compile classpath, included in packaging
          D: not included in A's compile classpath, not packaged

          If I remember correctly there were some talk in the mailing list that maven 2.1 would implement option 3, but I might remember wrong.

          edit: this seem to be bit more complicated than that: how about classpath for compiling and running tests etc.
          As a quick thought, classpath for compiling tests could be just the direct dependencies and for running tests same as for packaging (but how about transitive test dependencies)?

          Show
          Sei Syvalta added a comment - - edited I think distinguishing packaging and compiling would make sense here. I use compile scope as an example: for direct dependencies it's clear that they are included in you compile time classpath and also included when packaged to a war for example. for transitive dependencies, it's clear they should be packaged but the question is if they should be included in the compile time classpath. Ok, let's have an example: 1. Current behavior: A -> B (compile) -> C (compile) A -> B (compile) -> D (provided) C: included in A's compile classpath, included in packaging D: not included in A's compile classpath, not packaged 2. Requested in this issue: A -> B (compile) -> C (compile) A -> B (compile) -> D (provided) C: included in A's compile classpath, included in packaging D: included in A's compile classpath, not packaged 3. Preferable solution in my opinion: A -> B (compile) -> C (compile) A -> B (compile) -> D (provided) C: not included in A's compile classpath, included in packaging D: not included in A's compile classpath, not packaged If I remember correctly there were some talk in the mailing list that maven 2.1 would implement option 3, but I might remember wrong. edit: this seem to be bit more complicated than that: how about classpath for compiling and running tests etc. As a quick thought, classpath for compiling tests could be just the direct dependencies and for running tests same as for packaging (but how about transitive test dependencies)?
          Hide
          Alexandre Sauve added a comment -

          I think that 'provided' transitive dependencies are important! The case that comes to my mind is OSGi and Eclipse development. You need a certain class/package in order to compile your code; however that same package will be available on the runtime environment as a bundle. You don't want to start adding this jar as an embed library into your bundle! You want to embed only those that are runtime/compile dependencies.

          However since 'provided' is NOT transitive you would have to list all the dependencies this bundle you depend on has in your POM! Not exactly a step forward from Maven 1. You would like to pull in these bundle dependencies just for the need of compilation (and actually you would what them in the manifest file too).

          So in the end for OSGi and Eclipse transitive provided dependencies would be a really nice thing to have!

          Show
          Alexandre Sauve added a comment - I think that 'provided' transitive dependencies are important! The case that comes to my mind is OSGi and Eclipse development. You need a certain class/package in order to compile your code; however that same package will be available on the runtime environment as a bundle. You don't want to start adding this jar as an embed library into your bundle! You want to embed only those that are runtime/compile dependencies. However since 'provided' is NOT transitive you would have to list all the dependencies this bundle you depend on has in your POM! Not exactly a step forward from Maven 1. You would like to pull in these bundle dependencies just for the need of compilation (and actually you would what them in the manifest file too). So in the end for OSGi and Eclipse transitive provided dependencies would be a really nice thing to have!
          Hide
          Maarten Billemont added a comment -

          What I'd like to do is create a POM-type artifact which is basically just a grouping of dependencies used by other artifacts.

          <dependencies>
          <!-- jboss/server/default/lib -->
          <dependency>
          <groupId>commons-codec</groupId>
          <artifactId>commons-codec</artifactId>
          <version>1.3</version>
          <scope>provided</scope>
          </dependency>
          ....

          Then I would add this artifact as a provided dependency to all artifacts.
          The desired result would be an inclusion of all dependencies from this container artifact in my other artifacts, marked as provided.

          You may wonder why I'm not just marking them as compile-scoped in the container artifact; here's why:
          I have an artifact that creates a ZIP file containing the JBoss AS. This artifact basically just downloads JBoss, extracts it, messes around with its configuration, and packages it as a ZIP file. Additionally, all artifacts from the container artifact above that are not marked as provided (because they are not already provided by JBoss) are added to the jboss/server/default/lib/ directory just before zipping it up. That means I can specify two types of provided dependencies in the container POM above:

          • Dependencies provided by the JBoss AS default configuration (<scope>provided</scope> in the container POM).
          • Dependencies that I add to my JBoss' server/default/lib directory (<scope>compile</scope> in the container POM).

          Every artifact that depends on the container POM and sets that dependency to a <scope>provided</scope> should have all dependencies in the container pom, be they provided or compile, added to the artifact as provided. This is exactly as described by the documentation here:
          http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
          With one single exception: I cannot yet mark provided dependencies in the container POM as transitive.

          The table on the link above says "provided" in the intersection of "provided" on the left side and "provided" on the top. Reading the text above the table you will see this implies that:
          If a dependency is set to [provided], transitive dependencies of that dependency with the scope [provided] will result in a dependency in the main project with the scope [provided]. If no scope is listed, it means the dependency will be omitted.

          I would've thought that the author would've just put a "-" at the intersection of provided with provided seeing as there currently is no such thing as transitive provided dependencies. Or is there?

          Show
          Maarten Billemont added a comment - What I'd like to do is create a POM-type artifact which is basically just a grouping of dependencies used by other artifacts. <dependencies> <!-- jboss/server/default/lib --> <dependency> <groupId>commons-codec</groupId> <artifactId>commons-codec</artifactId> <version>1.3</version> <scope>provided</scope> </dependency> .... Then I would add this artifact as a provided dependency to all artifacts. The desired result would be an inclusion of all dependencies from this container artifact in my other artifacts, marked as provided. You may wonder why I'm not just marking them as compile-scoped in the container artifact; here's why: I have an artifact that creates a ZIP file containing the JBoss AS. This artifact basically just downloads JBoss, extracts it, messes around with its configuration, and packages it as a ZIP file. Additionally, all artifacts from the container artifact above that are not marked as provided (because they are not already provided by JBoss) are added to the jboss/server/default/lib/ directory just before zipping it up. That means I can specify two types of provided dependencies in the container POM above: Dependencies provided by the JBoss AS default configuration (<scope>provided</scope> in the container POM). Dependencies that I add to my JBoss' server/default/lib directory (<scope>compile</scope> in the container POM). Every artifact that depends on the container POM and sets that dependency to a <scope>provided</scope> should have all dependencies in the container pom, be they provided or compile, added to the artifact as provided. This is exactly as described by the documentation here: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope With one single exception: I cannot yet mark provided dependencies in the container POM as transitive. The table on the link above says "provided" in the intersection of "provided" on the left side and "provided" on the top. Reading the text above the table you will see this implies that: If a dependency is set to [provided] , transitive dependencies of that dependency with the scope [provided] will result in a dependency in the main project with the scope [provided] . If no scope is listed, it means the dependency will be omitted. I would've thought that the author would've just put a "-" at the intersection of provided with provided seeing as there currently is no such thing as transitive provided dependencies. Or is there?
          Hide
          Roman Kalukiewicz added a comment -

          There is another very important reason why provided dependencies should be transitive. This reason is tests.

          If you have some class that needs (lets say) servlet-api and servlet-api is provided, then you cannot use this class in another project, because when you write test, you are not able to instantiate your original class because it imports javax.servlet.* that is not available.

          You also cannot extend any created servlet class in any different project, because it will not compile if Serlvet interface is not available on classpath.
          This is similar to the note you have just under the table at
          http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope

          I believe that the most typical situation is to scope servlet-api as provided and it shows, that it doesn't work as it should.

          Show
          Roman Kalukiewicz added a comment - There is another very important reason why provided dependencies should be transitive. This reason is tests. If you have some class that needs (lets say) servlet-api and servlet-api is provided, then you cannot use this class in another project, because when you write test, you are not able to instantiate your original class because it imports javax.servlet.* that is not available. You also cannot extend any created servlet class in any different project, because it will not compile if Serlvet interface is not available on classpath. This is similar to the note you have just under the table at http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope I believe that the most typical situation is to scope servlet-api as provided and it shows, that it doesn't work as it should.
          Hide
          Timothy Twelves added a comment -

          Hi,

          In the context of a "war" a dependency that is "provided" can mean 1 of two things:

          1. It is provided within the application ear
          OR
          2. It is provided by the environment.

          The solution for (1) is easy because we could have "provided" being transitive and have the ear-plugin package the "provided" dependencies.

          The solution for (2) is tricky because (as in the servlets example) we need it for compilation and we do NOT want it packaged by the ear-plugin since it is provided by the appserver or jvm path. I considered "runtime" but this would make it unavailable during compilation and with the servlet -api example we need those dependencies during compilation. "optional" for (2) works fine however the reasons why it works are not obvious to the layman.

          The solution is to introduce a new state called "environmental" and duplicate functionality around "optional".
          Make a "provided" transitive and have all class-packaging plugins (maven-ear-plugin, maven-war-plugin) support the correct packaging of "provided" dependencies.

          From my understanding of the maven code base (i've been digging) this is relatively do-able and the semantic changes are not going to be too bad considering the way people work around the problem of creating wars. I have only one problem which is the need to rewrite the manifest within jar/war/etc to support bundled libraries with different jar locations and signed jar/wars but again i believe this can be managed in the context of how people work around skinny jars at the moment.

          -Tim

          Show
          Timothy Twelves added a comment - Hi, In the context of a "war" a dependency that is "provided" can mean 1 of two things: 1. It is provided within the application ear OR 2. It is provided by the environment. The solution for (1) is easy because we could have "provided" being transitive and have the ear-plugin package the "provided" dependencies. The solution for (2) is tricky because (as in the servlets example) we need it for compilation and we do NOT want it packaged by the ear-plugin since it is provided by the appserver or jvm path. I considered "runtime" but this would make it unavailable during compilation and with the servlet -api example we need those dependencies during compilation. "optional" for (2) works fine however the reasons why it works are not obvious to the layman. The solution is to introduce a new state called "environmental" and duplicate functionality around "optional". Make a "provided" transitive and have all class-packaging plugins (maven-ear-plugin, maven-war-plugin) support the correct packaging of "provided" dependencies. From my understanding of the maven code base (i've been digging) this is relatively do-able and the semantic changes are not going to be too bad considering the way people work around the problem of creating wars. I have only one problem which is the need to rewrite the manifest within jar/war/etc to support bundled libraries with different jar locations and signed jar/wars but again i believe this can be managed in the context of how people work around skinny jars at the moment. -Tim
          Hide
          Timothy Twelves added a comment -

          Just to clarify around "optional" i am referring to a "compile" dependencies that are marked optional. This means that "environmental" dependencies will be there for compilation, tests, execution within maven but excluded from packaging.

          Alternatively you could leave "provided" the way it is and add a new dependency type called "application" that is intended to be provided and packaged by the deliverable application component (war, ear, etc). In the context of a war an dependency marked "application" must be packaged by the ear.

          -Tim

          Show
          Timothy Twelves added a comment - Just to clarify around "optional" i am referring to a "compile" dependencies that are marked optional. This means that "environmental" dependencies will be there for compilation, tests, execution within maven but excluded from packaging. Alternatively you could leave "provided" the way it is and add a new dependency type called "application" that is intended to be provided and packaged by the deliverable application component (war, ear, etc). In the context of a war an dependency marked "application" must be packaged by the ear. -Tim
          Hide
          Denis Kramer added a comment -

          I believe the right behavior is postulated in the Maven Scope Matrix (http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html).

          There a combination of provided/provided (for dependency and a dependency in the dependency) leads to a provided scope for the secondary dependency (as it should be).

          I'm not entirely sure if a xxx/provided dependency should be transitive. Thinking about OSGi environments, this might lead to version issues.

          Anyway, maven (as of 2.2.1) does not even handle the provided/provided case as given in the Maven Scope Matrix.

          Show
          Denis Kramer added a comment - I believe the right behavior is postulated in the Maven Scope Matrix ( http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html ). There a combination of provided/provided (for dependency and a dependency in the dependency) leads to a provided scope for the secondary dependency (as it should be). I'm not entirely sure if a xxx/provided dependency should be transitive. Thinking about OSGi environments, this might lead to version issues. Anyway, maven (as of 2.2.1) does not even handle the provided/provided case as given in the Maven Scope Matrix.
          Hide
          Michael Pollmeier added a comment -

          This 'critical' issue seems to be dead somehow, so I thought I put in my 2 pence and a bugfix.

          First a short summary to the problem as it was a bit blurry in all the various comments: I believe (and I'm not alone there) that we need to make provided dependencies transitive. This is how it used to be in the dependency matrix, however it was never implemented that way, hence this issue was created. Instead of solving it by changing the code, the dependency matrix has been amended to indicate that provided dependencies are not transitive. This has happened silently, at least no comment was left in this issue regarding that change of the documentation.

          Let me point out why provided dependencies should be transitive with a simple example using the projects A, B and C:
          A doesn't depent on anything. B depends on A. And C depends on B.
          Now you want to run unit or integration tests on C. It is straightforward to see that C must have a transitive provided dependency on A, because B (which C depends on) needs A to function. Therefore A must be in the classpath, right? With the current implementation, A will not be in the dependency scope of maven at all, because provided dependencies are not regarded as transitive dependencies.

          Also the combination compile/optional is not a workaround, as it basically duplicates the provided scope - I however can't see a difference in behaviour between 'compile/optional' and 'provided'.

          I'll attach these 'A,B,C' poms so that you can play around yourself.

          As a possible resolution I forked mvn in github and provide a bugfix commit:
          https://github.com/mpollmeier/maven-3/commit/3339789d6ae2dcbdc0e98d7b47a18f8a2aa2893e

          Please let me know your thoughts and pull the change if you like it.

          Show
          Michael Pollmeier added a comment - This 'critical' issue seems to be dead somehow, so I thought I put in my 2 pence and a bugfix. First a short summary to the problem as it was a bit blurry in all the various comments: I believe (and I'm not alone there) that we need to make provided dependencies transitive. This is how it used to be in the dependency matrix, however it was never implemented that way, hence this issue was created. Instead of solving it by changing the code, the dependency matrix has been amended to indicate that provided dependencies are not transitive. This has happened silently, at least no comment was left in this issue regarding that change of the documentation. Let me point out why provided dependencies should be transitive with a simple example using the projects A, B and C: A doesn't depent on anything. B depends on A. And C depends on B. Now you want to run unit or integration tests on C. It is straightforward to see that C must have a transitive provided dependency on A, because B (which C depends on) needs A to function. Therefore A must be in the classpath, right? With the current implementation, A will not be in the dependency scope of maven at all, because provided dependencies are not regarded as transitive dependencies. Also the combination compile/optional is not a workaround, as it basically duplicates the provided scope - I however can't see a difference in behaviour between 'compile/optional' and 'provided'. I'll attach these 'A,B,C' poms so that you can play around yourself. As a possible resolution I forked mvn in github and provide a bugfix commit: https://github.com/mpollmeier/maven-3/commit/3339789d6ae2dcbdc0e98d7b47a18f8a2aa2893e Please let me know your thoughts and pull the change if you like it.
          Hide
          Radai Rosenblatt added a comment -

          here's my use case:
          i have a module with "provided" dependencies on jboss modules. my tests deploy code to an actual jboss server where these "provided" deps exist, but i also have those same provided dependencies on the test classpath. since im booting jboss embedded (using the arquillian project) this causes class-loading issues. i would love for those dependencies to simply not be there during tests.
          maybe instead of breaking current behavior (as having "provided" deps on the lass path does make sense in certain scenarios) it would be possible to add a new scope ? "provided-transitive", "provided-test" or whatever that would cause dependencies to be "provided" in tests as well ?

          Show
          Radai Rosenblatt added a comment - here's my use case: i have a module with "provided" dependencies on jboss modules. my tests deploy code to an actual jboss server where these "provided" deps exist, but i also have those same provided dependencies on the test classpath. since im booting jboss embedded (using the arquillian project) this causes class-loading issues. i would love for those dependencies to simply not be there during tests. maybe instead of breaking current behavior (as having "provided" deps on the lass path does make sense in certain scenarios) it would be possible to add a new scope ? "provided-transitive", "provided-test" or whatever that would cause dependencies to be "provided" in tests as well ?
          Hide
          Dale Wijnand added a comment -

          I really like the 3rd solution given by Sei Syvalta (http://jira.codehaus.org/browse/MNG-2205?focusedCommentId=131081&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-131081).

          I'd like to vote that one in particular.

          I think it would make for more stable builds (or rather updating dependencies), such as:
          Dozer (net.sf.dozer:dozer:jar:5.3.2) currently depends on commons-lang:commons-lang:jar:2.5
          If in my project I compile-time depend on dozer, I also have commons-lang on the compile classpath.
          If I Dozer's next version refactors everything to use Google's Guava instead (com.google.guava:guava:jar:11.0.2), then my code will break.. for something (basically) completely unrelated.

          Show
          Dale Wijnand added a comment - I really like the 3rd solution given by Sei Syvalta ( http://jira.codehaus.org/browse/MNG-2205?focusedCommentId=131081&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-131081 ). I'd like to vote that one in particular. I think it would make for more stable builds (or rather updating dependencies), such as: Dozer (net.sf.dozer:dozer:jar:5.3.2) currently depends on commons-lang:commons-lang:jar:2.5 If in my project I compile-time depend on dozer, I also have commons-lang on the compile classpath. If I Dozer's next version refactors everything to use Google's Guava instead (com.google.guava:guava:jar:11.0.2), then my code will break.. for something (basically) completely unrelated.
          Hide
          Marco Speranza added a comment -

          Hi all...
          I'm in a OSGi environment and I've the same problem.
          I need to compile a project

          A -> B (provided) -> C (provided)
          B is an OSGi bunlde and A needs to use some C's packages/classes

          I agree with Alexandre

          IMHO it's important to have a simple option to add a transitive 'provided' dependency into the compile classpath.
          Could be a simple solution to add a flag into the pom file (like system scope? ) that allows a provided dependency to be transitive?

          i.e.

          <dependency>
             <groupId> foo.bar </groupId>
             <artifactId> project </artifactId>
             <version> 1.0.0 </version>
             <scope>provided</scope>
             
             <transitive> true </transitive>
          
          </dependency>
          

          thanks and have a nice day

          Show
          Marco Speranza added a comment - Hi all... I'm in a OSGi environment and I've the same problem. I need to compile a project A -> B (provided) -> C (provided) B is an OSGi bunlde and A needs to use some C's packages/classes I agree with Alexandre IMHO it's important to have a simple option to add a transitive 'provided' dependency into the compile classpath. Could be a simple solution to add a flag into the pom file (like system scope? ) that allows a provided dependency to be transitive? i.e. <dependency> <groupId> foo.bar </groupId> <artifactId> project </artifactId> <version> 1.0.0 </version> <scope> provided </scope> <transitive> true </transitive> </dependency> thanks and have a nice day
          Hide
          Ivan Bondarenko added a comment -

          I have voted for this.
          Half-abstract and half-concrete example: A -> B(compile) -> servlet-api(provided), A must have an access to servlet-api classes (or at least have such possibility), because A must sit in web-container, so B has no other choice. In some cases A don't need classes from servlet-api, but this is not a problem, "compile" scope creates more problems in this case, because it pulls unnecessary dependencies along into package.
          I think the solution may be removing "optional" tag from dependency definition. Instead the tag "transitivity" can be introduced, which may be flexible and has different values:
          1) 'transitive' - always transitive
          2) 'optional' - as optional=true in current variant
          3) 'detect' - transitive for cases when "The type <NAME> cannot be resolved. It is indirectly referenced from required .class files" compile error appears
          4) 'non-transitive' - always non-transitive
          5) etc
          For backward compatibility 'transitive' can be default for "compile" scope and 'non-transitive' for "provided"

          Show
          Ivan Bondarenko added a comment - I have voted for this. Half-abstract and half-concrete example: A -> B(compile) -> servlet-api(provided), A must have an access to servlet-api classes (or at least have such possibility), because A must sit in web-container, so B has no other choice. In some cases A don't need classes from servlet-api, but this is not a problem, "compile" scope creates more problems in this case, because it pulls unnecessary dependencies along into package. I think the solution may be removing "optional" tag from dependency definition. Instead the tag "transitivity" can be introduced, which may be flexible and has different values: 1) 'transitive' - always transitive 2) 'optional' - as optional=true in current variant 3) 'detect' - transitive for cases when "The type <NAME> cannot be resolved. It is indirectly referenced from required .class files" compile error appears 4) 'non-transitive' - always non-transitive 5) etc For backward compatibility 'transitive' can be default for "compile" scope and 'non-transitive' for "provided"
          Hide
          Sergey added a comment -

          Issue has been created: 06/Apr/06 5:37 AM
          6 years old!!! Should be already fixed.

          Show
          Sergey added a comment - Issue has been created: 06/Apr/06 5:37 AM 6 years old!!! Should be already fixed.
          Hide
          David Boden added a comment -

          For what it's worth, I no longer agree with this issue that I started 6 years ago! I have a best-practice document which I apply to my projects and have pasted here in this comment. Under this strategy, I always declare compile-time dependencies in any project that has an import; statement referring to the class. I don't rely on transitive dependencies when I'm actually using a dependency for compilation. I think this is what the Maven authors intended, which is why "provided" has stayed like it is for such a long time.

          1. <dependencyManagement/> in parent pom
          Declare the versions for all dependencies in the project's parent pom using the <dependencyManagement/> section. You're setting the versions that will be used if the submodules decide to declare dependencies. Using dependency management ensures that the version of all of your dependencies is entirely consistent across all your sub-projects. This is especially important if you're creating a single deployment consisting of many of these sub-modules and you are going to deploy one set of dependency jars

          Declare few or no dependencies in the <dependencies/> section of the parent pom. It's fair to assume that all your projects will depend on JUnit and a logging framework so you may want to add those. Anything else results in unnecessary dependencies making their way into your sub-projects.
          It's not too useful to declare versions at an even higher level (e.g. the pom that's the parent of all your projects) but in some cases it can make life easier.
          Failing to add a dependencyManagement section results in a classic problem of scripts being generated in sub-projects which reference jar versions that don't end up being packaged in the final assemby's lib folder.
          2. Declare dependencies where you use them - don't rely on transitive dependencies
          Transitive dependencies are very useful. If you're using the public API of a library in your code then you shouldn't have to care how its internals work. Your sub-project should declare a dependency on the library that your code is using; Maven will helpfully pull in the library's transitive dependencies. The library is free to change how its internals work and what transitive dependencies it declares for each version.

          If you're using a library, you should declare it in the <dependencies/> section of your pom. Don't rely on transitive dependencies. If your Java code (or Spring config; that's code too) uses commons-beanutils then you should declare that dependency. If you don't, your build is brittle; internal changes to other components can break your component's build.

          Run:

          mvn dependency:analyze
          This tool will tell you about artifacts that you've failed to declare dependencies for and will also tell you if you have extra dependencies declared that you don't need. Be careful about dependencies that you don't use in your .java files; the dependency plugin is only scanning those. If you use a class in Spring config then you do really need it, even if the dependency plugin is telling you that you don't. In these cases I find it useful to make sure that the dependency is set to <scope>runtime</scope>. That way, when I get the dependency:analyze report telling me that these are extra dependencies I know that I probably set them to runtime because they're used by spring config. I ignore the runtime dependencies and only weed out the compile scope ones.

          3. Use <properties/> sparingly (in the parent pom)
          Declaring all dependency versions in properties and then using those properties in the <dependencyManagement/> section is a little like the (bad) practice of unnecessarily declaring all your Java variables right at the top of a method.

          When there's a single artifact to configure, favour hardcoding the version into the <version>1.0</version> tag.

          Use properties when there are multiple artifacts to control. For example, set:

          <spring.version>3.1.1.RELEASE</spring.version>
          and then use:

          <version>$

          {spring.version}

          </version>
          for artifacts like <artifactId>spring-core</artifactId>

          Properties are public, not private. When you declare a property in the parent pom, it's available anywhere in the build. Adding a property pollutes the (build) environment.

          4. Use the enforcer plugin to guard against old or unwanted dependencies
          The enforcer plugin can fail your build based on duplicate classes found (the same classes in more than one jar file on the build classpath) or can allow you to ban old or unwanted dependencies. This is especially useful when the owner of a library has changed the groupId or artifactId of their artifact. This stops the Maven dependency mechanism from picking the latest version; you'll end up with 2 similar artifacts on the classpath. For example, from Spring 2 to Spring 3 they changed an artifactId from spring to spring-core. Without banning the older spring dependency, it can easily transitively creep back onto your classpath. Then when your program runs it's somewhat pot luck as to which jar is first on the classpath and therefore which version of a class gets loaded.

          5. Detect whether you're using old plugin or dependency versions
          The following goals of the versions plugin can help you keep up to date with the latest dependencies. Ideally, get your continuous integration server to display this information.

          mvn versions:display-plugin-updates
          mvn versions:display-dependency-updates

          Show
          David Boden added a comment - For what it's worth, I no longer agree with this issue that I started 6 years ago! I have a best-practice document which I apply to my projects and have pasted here in this comment. Under this strategy, I always declare compile-time dependencies in any project that has an import; statement referring to the class. I don't rely on transitive dependencies when I'm actually using a dependency for compilation. I think this is what the Maven authors intended, which is why "provided" has stayed like it is for such a long time. 1. <dependencyManagement/> in parent pom Declare the versions for all dependencies in the project's parent pom using the <dependencyManagement/> section. You're setting the versions that will be used if the submodules decide to declare dependencies. Using dependency management ensures that the version of all of your dependencies is entirely consistent across all your sub-projects. This is especially important if you're creating a single deployment consisting of many of these sub-modules and you are going to deploy one set of dependency jars Declare few or no dependencies in the <dependencies/> section of the parent pom. It's fair to assume that all your projects will depend on JUnit and a logging framework so you may want to add those. Anything else results in unnecessary dependencies making their way into your sub-projects. It's not too useful to declare versions at an even higher level (e.g. the pom that's the parent of all your projects) but in some cases it can make life easier. Failing to add a dependencyManagement section results in a classic problem of scripts being generated in sub-projects which reference jar versions that don't end up being packaged in the final assemby's lib folder. 2. Declare dependencies where you use them - don't rely on transitive dependencies Transitive dependencies are very useful. If you're using the public API of a library in your code then you shouldn't have to care how its internals work. Your sub-project should declare a dependency on the library that your code is using; Maven will helpfully pull in the library's transitive dependencies. The library is free to change how its internals work and what transitive dependencies it declares for each version. If you're using a library, you should declare it in the <dependencies/> section of your pom. Don't rely on transitive dependencies. If your Java code (or Spring config; that's code too) uses commons-beanutils then you should declare that dependency. If you don't, your build is brittle; internal changes to other components can break your component's build. Run: mvn dependency:analyze This tool will tell you about artifacts that you've failed to declare dependencies for and will also tell you if you have extra dependencies declared that you don't need. Be careful about dependencies that you don't use in your .java files; the dependency plugin is only scanning those. If you use a class in Spring config then you do really need it, even if the dependency plugin is telling you that you don't. In these cases I find it useful to make sure that the dependency is set to <scope>runtime</scope>. That way, when I get the dependency:analyze report telling me that these are extra dependencies I know that I probably set them to runtime because they're used by spring config. I ignore the runtime dependencies and only weed out the compile scope ones. 3. Use <properties/> sparingly (in the parent pom) Declaring all dependency versions in properties and then using those properties in the <dependencyManagement/> section is a little like the (bad) practice of unnecessarily declaring all your Java variables right at the top of a method. When there's a single artifact to configure, favour hardcoding the version into the <version>1.0</version> tag. Use properties when there are multiple artifacts to control. For example, set: <spring.version>3.1.1.RELEASE</spring.version> and then use: <version>$ {spring.version} </version> for artifacts like <artifactId>spring-core</artifactId> Properties are public, not private. When you declare a property in the parent pom, it's available anywhere in the build. Adding a property pollutes the (build) environment. 4. Use the enforcer plugin to guard against old or unwanted dependencies The enforcer plugin can fail your build based on duplicate classes found (the same classes in more than one jar file on the build classpath) or can allow you to ban old or unwanted dependencies. This is especially useful when the owner of a library has changed the groupId or artifactId of their artifact. This stops the Maven dependency mechanism from picking the latest version; you'll end up with 2 similar artifacts on the classpath. For example, from Spring 2 to Spring 3 they changed an artifactId from spring to spring-core. Without banning the older spring dependency, it can easily transitively creep back onto your classpath. Then when your program runs it's somewhat pot luck as to which jar is first on the classpath and therefore which version of a class gets loaded. 5. Detect whether you're using old plugin or dependency versions The following goals of the versions plugin can help you keep up to date with the latest dependencies. Ideally, get your continuous integration server to display this information. mvn versions:display-plugin-updates mvn versions:display-dependency-updates
          Hide
          Tibor Digana added a comment - - edited

          I am using Maven 3.1.1
          This bug is very important and needs to be fixed.
          Especially in Java enterprise architecture the transitive scope of provided is used.
          You have inheritance of modules + parent in reactor:

          webapp module inherits(war) -> services(jar) -> dao(jar)

          So the DAO is dependent on Hibernate. The own Hibernate's dependencies (like. hibernate-jpa-2.0-api, ...) should be excluded, but the same javax artifacts should be included with the scope provided.
          The scope=provided on e.g. JPA API is necessary because we want to have own Hibernate's dependencie in compile time and tests as well. Additionally, the WAR file should NOT have libraries with javax packages in WEB-INF/lib.

          If you compile DAO module only, it succeeds.
          In reality you compile the parent POM in reactor, the build fails in module "services". The class from provided artifact was not inherited by the module "services". The webapp module wouldn't inherit it either.
          So the people make a workaround so that they put the same dependency with provided scope in module services or all modules.

          This is terrible because:
          + the inheritance in Maven is then useless
          + you duplicated the same dependencies with same scope
          + more work when maintaining the modules
          + the POM files are huge

          So the temporary workaround can be used to create parents Services-BOM and DAO-BOM with packaging=pom.
          Inheritance:
          Services-BOM -> DAO-BOM -> reactor's POM
          The DAO will have parent DAO-BOM with provided artifacts, and services will have parent Services-BOM.
          The problem with this impractical workaround is that you cannot use profiles otherwise you are on the beginning again.
          The Services-BOM and DAO-BOM have no noition of profiles in Maven 3.1.1.
          You need the profile bacause the build needs to be customized for different appication servers and anything else.

          Thx

          Show
          Tibor Digana added a comment - - edited I am using Maven 3.1.1 This bug is very important and needs to be fixed. Especially in Java enterprise architecture the transitive scope of provided is used. You have inheritance of modules + parent in reactor: webapp module inherits(war) -> services(jar) -> dao(jar) So the DAO is dependent on Hibernate. The own Hibernate's dependencies (like. hibernate-jpa-2.0-api, ...) should be excluded, but the same javax artifacts should be included with the scope provided. The scope=provided on e.g. JPA API is necessary because we want to have own Hibernate's dependencie in compile time and tests as well. Additionally, the WAR file should NOT have libraries with javax packages in WEB-INF/lib. If you compile DAO module only, it succeeds. In reality you compile the parent POM in reactor, the build fails in module "services". The class from provided artifact was not inherited by the module "services". The webapp module wouldn't inherit it either. So the people make a workaround so that they put the same dependency with provided scope in module services or all modules. This is terrible because: + the inheritance in Maven is then useless + you duplicated the same dependencies with same scope + more work when maintaining the modules + the POM files are huge So the temporary workaround can be used to create parents Services-BOM and DAO-BOM with packaging=pom. Inheritance: Services-BOM -> DAO-BOM -> reactor's POM The DAO will have parent DAO-BOM with provided artifacts, and services will have parent Services-BOM. The problem with this impractical workaround is that you cannot use profiles otherwise you are on the beginning again. The Services-BOM and DAO-BOM have no noition of profiles in Maven 3.1.1. You need the profile bacause the build needs to be customized for different appication servers and anything else. Thx
          Hide
          Tibor Digana added a comment - - edited

          I do not know if the bug is definitely in Maven core or maven-war-plugin 2.2 or 2.4.
          The efective-pom correctly resolves the modules DAO(jar) and Services(jar).
          For sure I will rollback to Maven 2.2.1 because there the profiles in DAO-BOM and Services-BOM work as expected.

          {code reactor's parent(pom) ^ +---------------------+ | ^ | | DAO(jar) -> DAO-BOM(pom) ^ ^ | | Services(jar) -> Services-BOM(pom) ^ | webapp(war)}

          The development profile has dependencies in DAO(jar):
          <groupId>org.hibernate</groupId>
          <artifactId>hibernate-core</artifactId>
          <version>4.2.6.Final</version>

          Th production profile has dependencies in DAO(jar):
          <dependency>
          <groupId>org.hibernate</groupId>
          <artifactId>hibernate-core</artifactId>
          <version>4.2.6.Final</version>
          <exclusions>
          <exclusion>
          <groupId>org.jboss.spec.javax.transaction</groupId>
          <artifactId>jboss-transaction-api_1.1_spec</artifactId>
          </exclusion>
          </exclusions>
          </dependency>
          <dependency>
          <groupId>org.jboss.spec.javax.transaction</groupId>
          <artifactId>jboss-transaction-api_1.1_spec</artifactId>
          <version>1.0.1.Final</version>
          <scope>provided</scope>
          </dependency>

          Show
          Tibor Digana added a comment - - edited I do not know if the bug is definitely in Maven core or maven-war-plugin 2.2 or 2.4. The efective-pom correctly resolves the modules DAO(jar) and Services(jar). For sure I will rollback to Maven 2.2.1 because there the profiles in DAO-BOM and Services-BOM work as expected. {code reactor's parent(pom) ^ +---------------------+ | ^ | | DAO(jar) -> DAO-BOM(pom) ^ ^ | | Services(jar) -> Services-BOM(pom) ^ | webapp(war)} The development profile has dependencies in DAO(jar): <groupId>org.hibernate</groupId> <artifactId>hibernate-core</artifactId> <version>4.2.6.Final</version> Th production profile has dependencies in DAO(jar): <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-core</artifactId> <version>4.2.6.Final</version> <exclusions> <exclusion> <groupId>org.jboss.spec.javax.transaction</groupId> <artifactId>jboss-transaction-api_1.1_spec</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.jboss.spec.javax.transaction</groupId> <artifactId>jboss-transaction-api_1.1_spec</artifactId> <version>1.0.1.Final</version> <scope>provided</scope> </dependency>
          Hide
          Jason van Zyl added a comment -

          It is unlikely we will change the behavior of the provided scope, but it would be possible to create a new 'provided-transitive' if we really wanted this. Changing the definition of existing scopes would be problematic.

          Show
          Jason van Zyl added a comment - It is unlikely we will change the behavior of the provided scope, but it would be possible to create a new 'provided-transitive' if we really wanted this. Changing the definition of existing scopes would be problematic.
          Hide
          Tibor Digana added a comment -

          @Jason
          The problem is that the behavior has changed after Maven 2.2.1.
          New keywords would not be acceptable by the community in my guess - this may take quite long time to implement and finally the Maven users may get lost.
          I prefer simplicity and correct behavior against huge number of keywords.
          The right solution would be to concentrate on fixing hunderds of bugs in the Maven project. First of all to write tests against Maven 2 behavior and then run the same tests on Maven 3, let's see what will happen.

          Show
          Tibor Digana added a comment - @Jason The problem is that the behavior has changed after Maven 2.2.1. New keywords would not be acceptable by the community in my guess - this may take quite long time to implement and finally the Maven users may get lost. I prefer simplicity and correct behavior against huge number of keywords. The right solution would be to concentrate on fixing hunderds of bugs in the Maven project. First of all to write tests against Maven 2 behavior and then run the same tests on Maven 3, let's see what will happen.
          Hide
          Jason van Zyl added a comment -

          I would love it if you concentrated fixing the hundreds of bugs in Maven. Our attempt is in our integrations tests of which there are hundreds which do exactly that: capture behaviour across Maven versions:

          https://github.com/apache/maven-integration-testing

          Feel free to augment. Also make sure that what you're asking for is actually happening in the core and is not a function of a particular plugin which does its own resolution. The WAR plugin for example.

          Look forward to your patches.

          Show
          Jason van Zyl added a comment - I would love it if you concentrated fixing the hundreds of bugs in Maven. Our attempt is in our integrations tests of which there are hundreds which do exactly that: capture behaviour across Maven versions: https://github.com/apache/maven-integration-testing Feel free to augment. Also make sure that what you're asking for is actually happening in the core and is not a function of a particular plugin which does its own resolution. The WAR plugin for example. Look forward to your patches.
          Hide
          Michael Osipov added a comment -

          As a side note: if you introduce model version 5.0.0, we could easily fix the provided scope and leave it in 4.0.0 as-is.

          Show
          Michael Osipov added a comment - As a side note: if you introduce model version 5.0.0, we could easily fix the provided scope and leave it in 4.0.0 as-is.
          Hide
          Jason van Zyl added a comment -

          I still think this would be confusing to have one behaviour with the 4.0.0 version of the POM and another behaviour with the 5.0.0 version of the model. I honestly think this would be confusing and after several years with this behaviour making a new scope name for the new behaviour is the way to go.

          Show
          Jason van Zyl added a comment - I still think this would be confusing to have one behaviour with the 4.0.0 version of the POM and another behaviour with the 5.0.0 version of the model. I honestly think this would be confusing and after several years with this behaviour making a new scope name for the new behaviour is the way to go.
          Hide
          Tibor Digana added a comment -

          I agree with Jason that the model version should not change.
          I agree with other commiters saying that XML schema should not be changed.
          Therefore a new scope should be introduced and we can call it something like "nonrunnable".
          Thus the behavior of scope "provided" should not change in spec.

          Show
          Tibor Digana added a comment - I agree with Jason that the model version should not change. I agree with other commiters saying that XML schema should not be changed. Therefore a new scope should be introduced and we can call it something like "nonrunnable". Thus the behavior of scope "provided" should not change in spec.
          Hide
          Paul Benedict added a comment -

          If this conversation is really about controlling the transitive nature of a dependency, I think we should introduce a <transitive> boolean tag. That won't need a version update since it's a new tag (no one has used it before). Then instead of inventing new scopes, we can just let programmers determine it for themselves based on their own needs.

          Show
          Paul Benedict added a comment - If this conversation is really about controlling the transitive nature of a dependency, I think we should introduce a <transitive> boolean tag. That won't need a version update since it's a new tag (no one has used it before). Then instead of inventing new scopes, we can just let programmers determine it for themselves based on their own needs.
          Hide
          Tibor Digana added a comment -

          @Paul
          We shouldn't introduce a new XML element like <transitive/>.
          The worst and still possible we can do is to introduce new scope.

          Show
          Tibor Digana added a comment - @Paul We shouldn't introduce a new XML element like <transitive/>. The worst and still possible we can do is to introduce new scope.

            People

            • Assignee:
              Unassigned
              Reporter:
              David Boden
            • Votes:
              61 Vote for this issue
              Watchers:
              67 Start watching this issue

              Dates

              • Created:
                Updated: