Issue Details (XML | Word | Printable)

Key: MRELEASE-261
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Maria Odea Ching
Reporter: paul.whelan@gmail.com
Votes: 56
Watchers: 49
Operations

If you were logged in you would be able to see more operations.
Maven 2.x Release Plugin

release:prepare shouls support flat directory multimodule projects

Created: 05/Jul/07 09:49 AM   Updated: 24/May/09 10:24 PM
Component/s: prepare
Affects Version/s: None
Fix Version/s: 2.0-beta-10

Time Tracking:
Not Specified

File Attachments: 1. Text File flatProject.main.patch (11 kB)
2. Text File flatProject.test.patch (9 kB)
3. Text File MRELEASE-261-with-its-v3.patch (77 kB)
4. Text File MRELEASE-261-with-its.patch (32 kB)
5. Text File MRELEASE-261.patch (12 kB)
6. Text File PrepareReleaseMojo.patch (1 kB)

Environment: linux / maven2 / svn
Issue Links:
Duplicate
Related
 

Patch Submitted: Yes


 Description  « Hide
What I mean by flat file structure firstly.

parent/pom.xml
module1/pom.xml
module2/pom.xml
.
.
.
module15/pom.xml

the parent references the modules like so

<modules>
<module>../module1</module>
<module>../module2</module>
.
.
.
<module>../module15</module>
</modules>

When i release:prepare only the parent project is tagged the modules projects versions are incremented etc but the modules are not tagged in svn.

I use this structure as i use eclipse as my IDE.

I would love to see a fix for the issue marked as closed here http://jira.codehaus.org/browse/MRELEASE-138. I am currenrly tagging by hand each submodule of the projects but it would be so nice to have the release plugin do this for me.

forgive my english.



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
John Allen added a comment - 18/Jul/07 11:09 AM - edited
Dear all, we have converted release 3 plugin to work just fine with flat based multi-modules. The fix works for all maven project structures as it simply changes an invalid assumption made in the existing code base:-

Currently the 'working directory' of the 'execution project' is assumed to be the top 'ancestor' directory for which all projects contained within the reactor will live under. This is obviously incorrect for flat based projects. The fix is simply a case of determining what the directory in the maven module hierarchy is the common owning directory for all the projects with the hierarchy (which for nested projects happens to be of course the same as the execution project's home, or in other words the working directory').

For instance (please excuse ASCII art)

A typical nested project hierarchy:

D:\TEMP\MYSTUFF <-- HTTP://SVN.ORG/REPOS/MYSTUFF/TRUNK
|+POM.XML
|
|--B
|  |+POM.XML
|  |
|  |--C
|     |+POM.XML
| 
|--D
|  |+POM.XML
|
|--E
   |+POM.XML

The same logical hierarchy but expressed as a flat maven structure:

D:\TEMP\MYSTUFF  <-- HTTP://SVN.ORG/REPOS/MYSTUFF/TRUNK
|--ROOT
|       |+POM.XML
|
|--B
|  |
|  |--B.ROOT
|  |       |+POM.XML
|  |
|  |--C
|     |+POM.XML
|
|--D
|  |+POM.XML
|
|--E
   |+POM.XML

These two are logically equivalent.

The observation is that tagging, branching, checkout, checkin etc are version controlled based operations and the majority of VCS systems provide a file system based model (not bothering with object based VC systems here).

So when we want to tag the stuff in MYSTUFF/TRUNK what we really want to do is create something under the TAGS in SVN (say) that has the trunk contents, thus:

A typical nested project hierarchy:

HTTP://SVN.ORG/REPOS/MYSTUFF/TAGS/MYTAG-1.0/
|+POM.XML
|
|--B
|  |+POM.XML
|  |
|  |--C
|     |+POM.XML
| 
|--D
|  |+POM.XML
|
|--E
   |+POM.XML

And for the flat structure we want:

HTTP://SVN.ORG/REPOS/MYSTUFF/TAGS/MYTAG-1.0/
|--ROOT
|       |+POM.XML
|
|--B
|  |
|  |--B.ROOT
|  |       |+POM.XML
|  |
|  |--C
|     |+POM.XML
|
|--D
|  |+POM.XML
|
|--E
   |+POM.XML

By going through the module hierarchy and finding the common directory ancestor we are able to perform all the correct SVN operations (CO, CI, TAG, ETC).

i.e. For the nested structure the common ancestor directory IS the current directory:

D:\TEMP\MYSTUFF
|+POM.XML
|
|--B
|  |+POM.XML
...

Top common ancestor directory is equal to 'D:\TEMP\MYSTUFF' which happens to be the same as the working directory of the execution project (the top POM.XML).

The same approach (of working out the common ancestor) works the on a flat structure but identifies that D:\TEMP\MYSTUFF is the common directory and NOT the execution projects' working directory which is in fact D:\TEMP\MYSTUFF\ROOT\

D:\TEMP\MYSTUFF
|--ROOT
|       |+POM.XML
|
|--B
|  |
|  |--B.ROOT
|  |       |+POM.XML
...

Cool. So now the only problem is in handling the perform operations where the plugin needs no project, simply checks out everything from the tag URL. Checkout works just fine but there is one critical piece of information missing, namely the path into the 'ROOT' project so that the build can start. This extra bit of information is simply added to the ReleaseConfiguration (that is persisted as release.properties). The perform reads this is an makes that the 'working directory' before it calls maven executor and then kicks off the build.

Note: Maven is full of assumptions regarding the location of modules being 'beneath' the parent project, for instance the default scheme for constructing URLs and SCM URLS has the derived projects taking the parent's URL and tacking its artifactId on the end. For all us flat-project guys you can usually work around these issues through explicit definition of the elements in the child/derived projects.


Barrett Nuzum added a comment - 18/Jul/07 03:14 PM
Hi John.

Thanks for the update.
In "we have converted", does that mean there is an immediate fix, or rather that we need to wait for a future release?

Also, we have scenarios where you may have:

Parent (under cvs:/group/module1)

  • ProjectA (under cvs:/group/module2)
  • ProjectB (under cvs:/anotherGroup/module3)

Will that work in this scenario?


John Allen added a comment - 19/Jul/07 04:17 AM
Well in that case the common directory is '/' which is probably not what you want as all other folders under '/' would be operated on (checked-in, tagged, etc). Our assumption has been that there is some unifying directory that represented the logical base for the modules underneath it.

We have a similar setup in regards to your master corporate pom project, that can, via a profile, include all our other projects. i.e.

/corporate-pom/pom.xml
/project-a/ - lots of submodules etc
/project-b/ - lots of submodules etc
etc

project-a and project-b, etc all have corporate-pom as their parent and, as i've said, corporate-pom can build all the projects (a,b, etc) (via a profile that include <modules><module>../project-a</module> etc </modules>) but one would not want to release the corporate-pom AND all these other projects at the same time. The 'all projects' setting on the corporate-pom is just for building a master site.

In our setup one releases corporate-pom, project-a , project-b etc separately as they represent completing different projects with their own release cycles etc.

Re when will it be avilable? Well we've done the analysis but I'm off for 2 days now so I expect we won't get patches submitted until next week.


paul.whelan@gmail.com added a comment - 19/Jul/07 06:45 AM
Great Stuff Guys

I am not in the office to create a release till the end of the month. I have to say I am having problems with many plugins as a result of using the flat file structure (site plugin for one). This is great news for me as it will save me a lot of time as you can imagine if a project has lots of modules it can become a real pain to do things by hand.

Thanks for your hard work.


John Allen added a comment - 19/Jul/07 09:31 AM
Hi Paul, we have no problems with the site plugin (bar multiple parents being included in the parent menu, see the recently updated MSITE issues) but that is using a site plugin that we've built ourself with a number of patches applied. Specifically though it works for 'flat multimodule' structures.

Henrik Dohlmann added a comment - 05/Nov/07 08:36 AM
Hi John,

Any news on the patch? I have browsed the repository, but it doesn't look like any fixes has been applied regarding this issue.


Barrett Nuzum added a comment - 03/Dec/07 09:42 AM
I got an email from John Allen that signified his team did not make significant progress on modifying the plugin, and has since abandoned the research, using, instead, a plugin to modify eclipse imports.

I'm looking at the solution he found – http://eclipse-tools.sourceforge.net/projecttransfer/usage.html
but I don't think it will work for us.

Paul, have you been able to make any progress?


paul.whelan@gmail.com added a comment - 03/Dec/07 10:47 AM
Nope Sorry I got no where I just put up with the pain.

Paul


John Allen added a comment - 28/Jan/08 05:35 PM - edited
Subclipse plugin for Eclipse can not handle nested projects in Eclipse at all, and from the dialogue on their list, do not intend to. As Subversive provides much better support for nested Subversion structures in Eclipse, and has since become the 'official' (or so I'm informed) Eclipse foundation Subversion plugin, we have moved to using Subversive and find that the Eclipse multi-project import-export plugin works pretty well. Note the impact analysis for the work of changing the release plugin to be more 'directory aware' was pretty good, 3 days would have it cracked I would expect (inc. ITs etc)

David Delbecq added a comment - 18/Mar/08 05:02 AM
Hi all,

making assumption about directory structure and the fact there is a common ancestor is wrong too!

I have this svn structure

and the developper desktop structure based on that svn:

{pre}

/home/user/dev/project/topModule (=http://server/project/topModule/trunk/)
+ pom.xml
+ Module1 (=http://server/project/Module1/trunk/)
+ pom.xml
+ Module2 (=http://server/project/Module2/trunk/)
+ pom.xml

{/pre}

I expect release plugin to read the submodules pom files, and especially the <scm> entries in that poms, that clearly indicate the svn path of each submodule, which should be tagged as this (using 1.1.1 as release version):

currently, only the first tag is generated, not the submodules tags.

The idea is that, during release, if plugin encounter a submodule which as a <scm> which is different than that of parent, it should also tag it.


Adam Leggett added a comment - 05/Apr/08 08:34 AM - edited
Attaching a prospective workaround patch for this issue.

Patch adds a new configuration property 'tagWorkingDirectory' to Prepare mojo. Then in your relative parent pom you can configure like this:

<plugin>
             <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-release-plugin</artifactId>
             <version>2.0-beta-7-PATCHED</version>
             <configuration>
                            <tagWorkingDirectory>${basedir}/..</tagWorkingDirectory>
                            <preparationGoals>
                                        clean verify -f ${artifactId}/pom.xml
                             </preparationGoals>
                             <connectionUrl>scm:svn:svn://localhost/svn/relative-parent/tags/${artifactId}-${release.version}</connectionUrl>
                             <goals>clean deploy</goals>
                             <pomFileName>${artifactId}/pom.xml</pomFileName>
                            <arguments>-f ${artifactId}/pom.xml</arguments>
               </configuration>
</plugin>

The additional config is to ensure the parent pom gets found for the forked lifecycles in release:prepare and release:perform. In addition, due to the release manager now putting the release descriptor (release.properties) in $[basedir}/.. you need to specify the connectionUrl for release:perform. I guess you could write some script to move it or, as above, fill in the url apart from the release version e.g.

#hello-world/hello-world-parent> mvn release:prepare release:perform -Drelease.version=1.7 --batch-mode

Hope this is useful.

Adam


Robin Custers added a comment - 22/May/08 10:49 AM
does the 2.0-beta-8-SNAPSHOT version already includes this fix? If not how can i use the patched version?

I tried apply your workaround to the 2.0-beta-8-SNAPSHOT but got the following error:

[INFO] ----------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO] task-segment: [clean, verify]
[INFO] ----------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Cannot execute mojo: clean. It requires a project with an existing pom.xml, but the build is not using one.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: < 1 second
[INFO] Finished at: Thu May 22 17:04:49 CEST 2008
[INFO] Final Memory: 1M/3M
[INFO] ------------------------------------------------------------------------
[INFO] --------------------------------------------------------------------

Tnx

  • Robin

Neil T Olson added a comment - 25/Jun/08 11:25 AM
Here's what we did for a workaround:

In the example above, I would have created a POM.XML in the D:\TEMP\MYSTUFF directory with one sub module, ROOT\POM.XML. No projects refer to it as a parent. It is only a way to identify which sub-module is the "real" root.

When doing our development in eclipse, we don't have that new POM.XML available. It only shows up when we checkout the whole project. We then run the Manven release:branch and release:prepare mojos from that directory, and everything seems to work fine.

Example:

proj\+pom.xml (sub module: base)
        |
        |--base
        |    |+pom.xml (sub modules: ../client, ../server)
        |
        |--client
        |    |+pom.xml (parent: ../base)
        |
        |--server
             |+pom.xml (parent: ../base)

All maven release stuff is done from the proj directory, but all of the eclipse projects are at their specific project level.

If a fix is done for this, I would love to help out with the testing. We have a relatively complicated configuration (3 levels of hierarchy) that I could test against.
Thanks - Neil


Sébastien Moreno added a comment - 07/Jul/08 12:56 PM

Yes it's work fine,
and it's not a bad concept to have a "build" pom and a "parent" separated.

In our case, the "root" pom is a child of the "parent" in order to benefit the release change version system and retrieving SCM information.

The only bad news is that the generated site mention the "root" module as a real separated module.
But it could be used as a kind of information build summary page.


David J. M. Karlsen added a comment - 02/Dec/08 01:32 PM
Any news on this. Not being able to release a flat structure is a real PITA - and having nearly empty pom.xml's around just to satisfy maven is really bad publicity...

Moritz Rebbert added a comment - 22/Dec/08 07:25 PM - edited
We also using a "flat" hierarchy for some of our projects.

Based on the assumption of a common ancestor directory that has to be tagged, we started to make a patch.
It workes for us in a simple setup with a parent project in the same folder with some direct modules. Also it should not destroy the behavior in hierarchical layouts.

We do not set or change the workingDirectory variable but compute the common ancestor of all projects where we need it. So we technically devided the location where the pom of the root project resides from the directory which will be tagged(or may be branched).

All projects are tagged under a common directory and "release:perform" also do the job for us.

I've you are brave, you could give "flatProject.main.patch" a try.
We also tried to fix the tests in a reasonable way(flatProject.test.patch).

(We also assume that the layout of the working copy matches the layout in a single repository.)

Edit: its based on 2.0-beta-9-SNAPSHOT (Last Changed Rev: 726974)


Abel Muiño added a comment - 22/Apr/09 02:01 AM
I see there are several patches attached to this issue.
What's the current status? Have any of patches been reviewed/tested? Is there something else that non-committers can provide to help in fixing the issue?

Maria Odea Ching added a comment - 29/Apr/09 04:47 AM
I'm currently working on fixing flat multi-module issues with Continuum and I'd probably be touching this issue since Continuum and the maven-release-plugin use the same release manager component. I'll take a look at the attached patches when I get to fixing the releasing part of flat multi-modules.

See comments in CONTINUUM-1569 for details on how we're approaching this in Continuum..


Stan Devitt added a comment - 29/Apr/09 08:53 AM
A workaround that I have used successfully in the past has been:

1. Organize the modules and parent in a flat pattern with <module>../module1</module> etc. in the parent.
2. Add an aggregator pom with one module - the parent. - one level up in the file tree.

Eclipse can ignore the aggregator as it is not referenced by any pom.
The release plugin can release from the aggregator.


Maria Odea Ching added a comment - 13/May/09 04:44 AM
Attaching MRELEASE-261.patch..

The base working directory and the base scm url is determined based on the root project's working dir and scm url, and the relative path of it's modules. Tested the fix with Continuum and the maven-release-plugin.

I'm not very familiar with the source code of the Maven Release component so I would really appreciate it if someone could review the fix before I commit it to trunk. Thanks in advance..


Olivier Lamy added a comment - 13/May/09 05:34 AM
Any chance you create an it test for that ?

Maria Odea Ching added a comment - 13/May/09 05:59 AM - edited
I only have unit tests for getting the base working dir and base scm url. I just saw the it tests in maven-release-plugin now, I'll see if I can write one for a flat multi-module.

Maria Odea Ching added a comment - 14/May/09 03:02 AM
Attaching revised patch with integration tests for a flat multi-module project and a regular multi-module project.

Olivier Lamy added a comment - 14/May/09 03:49 PM
I can't find particular issue (only tested with svn) .
So +1 to commit this.

Maria Odea Ching added a comment - 19/May/09 08:19 AM
Attached patch with a couple of fixes for the tag url written in the pom and additional tests.

Maria Odea Ching added a comment - 20/May/09 09:44 AM
If no one objects to the fix I did, I'll commit it to trunk already..

Maria Odea Ching added a comment - 24/May/09 12:44 AM
Fixed in trunk 778088 (applied MRELEASE-261-with-its-v3.patch).

Maria Odea Ching added a comment - 24/May/09 10:24 PM
Additional fix for determining the common path which was failing in windows was committed to trunk -r778269.