Add a CLI option to ignore missing modules

In this scenario, you should use profiles to express what modules can be safely "turned off"

Our project consists of about 30 modules. To be able to individually combine "active" and "not active" modules, we would need a profile for each of the modules, which would be quite extreme.

Alternately, you could specify the projects to build using "-pl", however even the projects that you do not want to be built need to be present, making the use of sparse checkouts impossible.

My proposal is a way similar to parent projects (i.e. if the parent project is present, that one is used, else the parent is retrieved from repo).

This would allow the following workflow:

• I have a task to work upon, concerning modules A12, A13 and A23.
• I do a sparse checkout of the master project
• I widen the checkout to include A12, A13 and A23 (these steps take about 1/10th of the time it would take to do a complete checkout!)
• I work upon my task, whenever I need a build I just build the whole scaffold project using "mvn -imm install", which builds only parent an my three modules
• I realize my task also concerns A7, so I widen my working copy to include that as well
• nothing changes in respect to compilation etc.., what is there gets built, what is missing is retrieved from repo.

Major benefit of this workflow:

Since my working copy is way smaller EVERY subversion operation is faster by magnitudes.

We do have a multi module projects with more than 40 submodules too.

We can use profiles to make modules optional but that will lead to a long and complex pom.xml.

That would be nice to be able to say "don't fail if modules are not existing".

While I agree that it should be easier to work with a set of modules, allowing Maven to just ignore missing modules is a not a good idea.

How do you check out a sparse tree?

Well in my case this is more in order to provide "sparse" workspaces to developers.

I mean, we have some core modules and many plugin modules.
Some developers are not interested in having the core modules (and in some cases we don't want them to have it, for security reasons).
But they still need to build the whole application regardless they don't have the sources for the core project or for non related plugins.

During the build, missing modules needed (dependency) will be downloaded from the repository.
And missing modules that are not present and not needed won't be packaged.
(An even better solution might be to tell maven that modules which can't be built should be searched in the local/remote repository)

In other words, to have the same behavior for dependency resolutions than for modules resolutions.

Ivan, I don't think this option is really necessary for Maven. Based on your description, your issue seems purely process related. Your developers should be able to check out parts of the tree as necessary but you want to have the rest of tree already deployed to your corporate repository. Regarding building the whole thing, that's really a task for a tool like Hudson.

Paul,
Of course it is related to our process and I understand this is a feature request not a bug.

Regarding the "building the whole thing" I don't think using hudson/jenkins would solve my problem:
My idea was that a developer should be able to generate a version for testing/demo with its modified sources.
I can't see how hudson could be able to pickup sources in its workspace and the missing module sources in the SCM.
And, further more, that means a developer can't build the software if he has not access to hudson. (which is likely to happen often in our case)

Anyway, I will simply stick to the "use profile" solution for now.
Thanks for your feedback

Ivan, have you considered private branches for your developers? It is a great help on large projects because large projects cannot tolerate work that isn't ready for integration. This will then allow Hudson to build "the whole thing" and test/demo.

Yes, this is already what we are doing.

A given developer wants to improve the plugin A which depends on the Core.
He create a branch with that plugin only. (Branching all projects should not be necessary in my opinion)
In its workspace he has only the plugin A and the "main" project (top parent project which reference all the modules and the assembly).

If he wants to compile it, no problem, the Core is resolved as jar-artifacts and retrieved from the maven repository.(local or remote)

Now if he wants someone to test its modifications before requesting integration to the trunk.
He has to generate a testable version.

That's were he needs to trigger the "main" project which has a reference on all modules and does the assembly part.
> Right now it will fail unless he checkout the other modules

I'm not sure of how you think I can "workaround" that with hudson:
> Checkout every modules from Trunk except the ones that are in developer's branch
> Run mvn package on the main project which will trigger ALL the submodules

Let me add another thought:

I do mainly cross-functional work, so there are a many situations where I am forced to quickly correct problems in many different projects that I don't have present in my working copy (multiply by various branches).

So let's look at a typical scenario:

a hotfix is needed for project X, which I do not have on my hd yet. X consists of 100 modules right now (in multiple layers). Our developers usually can work on a partition of that directory tree using all available features. However, my hotfix (or the debug session) requires changes in mutliple partitions.

Right now, I have to checkout all partitions that i need to change code in (which costs a lot of time). Then I either build the whole project - which takes even longer, or I decide only to build that changed modules and their dependants. For that, I can use Mavens -pl and -amd options, but I have to list a changed modules (10, 15?) -> this is the way that hudson/jenkins does when building incrementally.

Now, with my proposal, I would work like this:

I check out only those modules relevant to me (which, BTW, is a brilliant feature of TortoiseSVN 1.7). I make my changes, build the whole thing with the new proposed -imm switch which tells Maven to ignore missing modules, i.e. the reactor is built correctly, it simply does not contain any missing modules. Dependencies to those modules are naturally resolved via the normal mechanisms (local repo, remote repos). The nice thing about this is: once I notice that I need an additional module, I can simply materialize it using subversion, and from that moment, it is simply included.
Actually, this would work a lot like m2e using workspace resolution, since m2e ignores the reactor. Right now I can work perfectly using this setup while using eclipse. As long as I do not try to start an actual Maven build.

Once again, let me raise some important points:

• I do not want to change the default behaviour of Maven
• This mode of work is strongest with Subversion, the speed factor is simply no longer relevant using Mercurial or Git, however I already encountered similar situations using Git where I could have used this feature
• The proposed mechanism would work very similar to Maven's parent resolution (i.e. look in the relative path, if the parent is there, use it, if not, ask repositories)

I stumbled upon an article today that tangentially regards this issue. It is mostly FUD but it also echos some of the recent sentiments here regarding the difficulty of modules/nesting. Just linking to it to help the discussion: http://buzdin.blogspot.com/2012/08/maven-modules-considered-harmful.html