Issue Details (XML | Word | Printable)

Key: MNG-3989
Type: New Feature New Feature
Status: Closed Closed
Resolution: Won't Fix
Priority: Major Major
Assignee: Unassigned
Reporter: Greg Wilkins
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Maven 2 & 3

Simple handling of external jars

Created: 13/Jan/09 11:27 AM   Updated: 03/Feb/09 04:32 AM   Resolved: 03/Feb/09 03:55 AM
Return to search
Component/s: None
Affects Version/s: 2.0.9
Fix Version/s: None

Time Tracking:
Not Specified

File Attachments: 1. Zip Archive MNG-3989.zip (16 kB)
2. Zip Archive MNG-3989.zip (16 kB)


Complexity: Intermediate


 Description  « Hide

For whatever reason, there will always be jars that don't exist in a maven repository.

There are numerous techniques for these - installing them in your local repo (either manually or with
some bootstrap.sh script or special profile activation). Checking in the jars into a local maven repository that is checked into svn
and then point to it from your settings.xml and/or top level pom (with aid of an env variable).

But all these methods lack a very important features. You can just do: "svn co http:/myproj.com/foo; cd foo; mvn"
If the jars change, you can't just do "svn up; mvn", you have to re-run whatever script/profile installed the repo.
It's all rather a PITA.

What I want, is some way to have a module of a project that contains some non-maven jars that when I
do a "mvn install" in that project, install those jars in my local repository for use by my other modules. If the
jars are not updated, then nothing is done.

With something like this, projects that have external dependencies could describe them to maven and
make them available for use, without manual steps and special scripts.



Anders Kr. Andersen added a comment - 20/Jan/09 11:59 AM

I think it is possible to declare a maven repository in a local file structure in a maven project that is stored in your VCS
And the declare the file structure to be a maven repository

And then reference these as dependencies / or copy them to local repository.

I just haven't seen a description of this.


Greg Wilkins added a comment - 21/Jan/09 04:47 AM

Anders,

you can do that (and I often do), but you need every pom that uses that repository to at least set a property to say where it is on the disk - that is an issue and prevents simply adding new modules or refactoring disk layout. At the best, it establishes a fixed directory relationship between modules (eg ${basedir}/../../project_repo


Brett Porter added a comment - 27/Jan/09 05:47 PM

hi Greg. I think I have a couple of possibilities in mind for this either through current caps or a simple plugin. Just have one question: do you have constraints on how you manage the coordinates and provision of POMs and coordinates for the dependencies?

If it is possible to put down a repo on disk (POMs and all) and have that installed in the local repo from within the project, then there are some reasonable ways to achieve it. Is that what you are looking for?


Greg Wilkins added a comment - 28/Jan/09 05:33 AM

Hi Brett,

yes having a repo on disk (POMs and all) is what I currently do. I define the repository in the top level pom relative to a property. Then each and every pom in that project has to set that property to locate the repository on disk.

It would be much better if there was a way of one module that contained the local repository being able to install it in your local repository. Then other modules could just use it normally without the need for a property or a constrained disk layout.


Brett Porter added a comment - 28/Jan/09 09:02 PM

Greg, yep that's what I meant - a module with a repo that installs to the local repo, rather than others referring to it.

Let me see what I can come up with.


Jason van Zyl added a comment - 29/Jan/09 07:58 PM

In what case can't you make it easier for your users by putting it in a repository? What's the case that made you ask for this?


Greg Wilkins added a comment - 30/Jan/09 10:52 AM

In many enterprises, getting permission to run a repository is difficult (specially when you are trying to move to maven by stealth and converting projects without high level permission as a proof of principal).

More over, I really like the ideal of maven for repeatable builds. I trust that repo.maven.org is going to exist and be well managed into the foreseeable future, but one can't expect enterprises to keep local repositories running and/or consistent, specially in todays climate where said enterprises like to lay-off technical staff using a megaphone!

So a local repository might give me a repeatable build for a week or a month, but 1,2 or 5 years would be a harder to achieve.


Jason van Zyl added a comment - 30/Jan/09 11:14 AM

For a given project starting you should just check in the repository.

If you care about your releases then generally what I'm seeing is that enterprises who understand the importance of using components put their repositories in the same class as their SCM and put it on systems that are just as reliable. They will be there 10 years from now.

I don't really see the advantage of making a tool to take some JARs and putting it in a local repository when in the short term of a pilot project just check the JARs to a file-based repository.


Greg Wilkins added a comment - 30/Jan/09 12:47 PM

Jason,

it's not just for pilot projects. Let's say that I have a huge project with many modules with many dependencies, most of which are available in repo.maven.org

But I have just a few jars that are not available in repo.maven.org. This may be because they just have not been published, or it may be that they are proprietary binary only jars with limited availability, or who knows the reason.

I really do not want to have to setup a local repository just for these few jars.

If there is the ability to have a few jars checked into SCM, that is far simpler and more reliable than creating and securing a private repository. It also means the source can be delivered across organizational boundaries without requiring a repository to be setup.

Now I hate jars checked into SCM, but for better or for worse, it is a common practise.
However, I hate seeing things like "../../repo" in all poms even more!

I've seen maven dropped by a large investment bank because of issues with a local repository.
I've frequently faced opposition to introducing maven simply because svn co; cd; mvn is not
sufficient to build.

I do recognize that a tool such as I'm suggesting could be misued really really badly, but then so
can many things in every project. I do think such a tools would help the adoption of maven by
many organizations and/or projects.

cheers


Anders Kr. Andersen added a comment - 02/Feb/09 06:16 AM

Hi Greg
It seems like it is up to you to come with a solution

I have the same issue with weblogic jar files.

What we need is a well defined way to load artifacts as jar,war, zip etc into a repository. mostly local repository

Consider we made a plugin that could:
a) read from a directory "repository" and load all files it finds there into repository. Where the maven coordinaes will be read from the file structure. (so repository should be structured as a maven 2 repository)
b) construct poms as long as they where not there.

I think this could be great.
The problem I have left is to get mvn clean install executed on this pom, so developers can get started ?


Anders Kr. Andersen added a comment - 02/Feb/09 06:27 AM

And one more thing...

I have problems with developers that sits and uploads files into the repository from here and there, the earth, moon, or from an other planet, without documenting where the things comes from.

So on my last project I required from all developers (sub projects) that they decared a directory "repository" with thse 3'rd party libraries.
And then we had a shell script that uploaded the stuff to repository.

So the point here is to give maven projects a place to document this act.


Brett Porter added a comment - 02/Feb/09 06:57 AM

hey Greg - this structure seems closest to what you are looking for without making modifications. IS it on the right track for what you were looking for?

You can do a bit more by using the dependency plugin or install:install-file (eg, generate POMs), but you still tend to end up with a separate module and declarations can be quite lengthy.

Beyond that it is getting towards writing a custom plugin to copy files directly into the local repository.

In any of these cases, it'll result in a transitive closure that won't be resolvable if the project is deployed into the repository without copying the repository module around (which is probably ok in situations like the weblogic one mentioned).

I'd still be reluctant to build this right into Maven given the number of alternatives available, though.


Greg Wilkins added a comment - 03/Feb/09 03:55 AM

Brett that's BRILLIANT!

It works exactly how I wanted without any changes to maven!

Thanks! I think this should be written up as THE pattern of how to
include extra jars!


Brett Porter added a comment - 03/Feb/09 04:32 AM

cleaned up version