Maven
  1. Maven
  2. MNG-2546

Allow plugin executions in the "super-init" phase before reactor sorting of modules build order

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.4
    • Fix Version/s: 3.0-alpha-6
    • Component/s: Reactor and workspace
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      1

      Description

      As seen here, http://www.nabble.com/How-to-execute-a-plugin-prior-to-the-reactor-sorting--tf2062739.html#a5682349. I also have the need to bind my maven-pde-plugin to a phase before the reactor sorting of project build order happens. My plugin is being developed to build eclipse plugins, features, fragments, update sites and products. Right now I can build plugins and features. However the order has to constantly be managed by the user taking information from the eclipse descriptors and adding it to the pom file. For plugin projects I can bind to a phase before the compile phase and dynamically analyze the eclipse plugin descriptors and add the necessary dependencies/resources to the MavenProject instance and all is well. For feature projects, I also can dynamically analyze the eclipse feature descriptor and add the necessary resources to the MavenProject instance. However, features depend on other plugins, fragments and features. While I can dynamicaly add the plugins, fragments and features to the MavenProject as dependencies they are not taken into context as the reactor has already computed the sorting order.

      What would be perfect is if there was a "super-init" phase that plugins could bind to and be executed in before the normal declared lifecycle happened. Therefore no matter what the lifecycle was, the "super-init" phase would be available. Then plugins could do things like augmenting the super-pom with build #'s/identifiers, dependencies, dynamic projects, etc all before maven gets going. That would solve the problem myself and others have as well as be 100% backwards compatible. This super-init phase (please pick a better name) would e available to reactor and non-reactor builds. A more specific fix would be to allow plugins to ask the reactor to reevaluate the build order.

        Activity

        Hide
        Barrie Treloar added a comment -

        Can you also link to your eclipse pde plugin.

        Some work like this is being done in the eclipse:eclipse plugin and given the small number of people wanting this specialised functionality it would be great if we could work together.

        Show
        Barrie Treloar added a comment - Can you also link to your eclipse pde plugin. Some work like this is being done in the eclipse:eclipse plugin and given the small number of people wanting this specialised functionality it would be great if we could work together.
        Hide
        Binyan added a comment -

        Patch to add support for the super-init phase that executes plugins before the reactor has sorted the projects.

        Show
        Binyan added a comment - Patch to add support for the super-init phase that executes plugins before the reactor has sorted the projects.
        Hide
        Binyan added a comment -

        Here is a link to the maven-pde-plugin, http://www.binyan.com/repos/pde.

        Show
        Binyan added a comment - Here is a link to the maven-pde-plugin, http://www.binyan.com/repos/pde .
        Hide
        Jason van Zyl added a comment -

        Do you have a test for this as I would probably refactor some bits slightly and would want to make sure everything is still good.

        Show
        Jason van Zyl added a comment - Do you have a test for this as I would probably refactor some bits slightly and would want to make sure everything is still good.
        Hide
        Jason van Zyl added a comment -

        Are you going to put the PDE plugin at Eclipse? We definitely can get you hooked up at the Mojo project. Would this plugin also work for OSGi bundles in general?

        Show
        Jason van Zyl added a comment - Are you going to put the PDE plugin at Eclipse? We definitely can get you hooked up at the Mojo project. Would this plugin also work for OSGi bundles in general?
        Hide
        Binyan added a comment -

        I just today asked on the Eclipse PDE list about where I could move this plugin so it could get more exposure and use cases to flush it out more. Personally, I don't have a preference. If it is too come to under the apache license I will need to see about a couple of classes that were provided to me for an EPL project. I don't think they will e an issue with changing the license of those classes, but I'll ask and see.

        And yes, the plugin works for OSGi bundles.

        As for the patch I have an example set of projects I use for testing the maven-pde-plugin. Once I cleanup the code I'll see if I can create a maven test for it. However, feel free to make the small tweaks, I can test it out faster than I can likely learn how to create a maven test.

        Show
        Binyan added a comment - I just today asked on the Eclipse PDE list about where I could move this plugin so it could get more exposure and use cases to flush it out more. Personally, I don't have a preference. If it is too come to under the apache license I will need to see about a couple of classes that were provided to me for an EPL project. I don't think they will e an issue with changing the license of those classes, but I'll ask and see. And yes, the plugin works for OSGi bundles. As for the patch I have an example set of projects I use for testing the maven-pde-plugin. Once I cleanup the code I'll see if I can create a maven test for it. However, feel free to make the small tweaks, I can test it out faster than I can likely learn how to create a maven test.
        Hide
        John Casey added a comment -

        Wanted to link in this conversation, and also add a comment (somewhat excerpted from the mail thread).

        The thread is here:

        http://www.nabble.com/forum/ViewPost.jtp?post=6141419&framed=y

        And my comment is:

        I think we should explore design/development of a reactor-level lifecycle which will run once and only once per maven invocation (per build, not per project). This would allow us to declare some discrete phases in which different categories of reactor-level work takes place, and align plugins to those phases so that a more-or-less declarative ordering at the reactor level can be achieved. In this model, the module builds would take place in a reactor phase somewhere in the middle of the reactor lifecycle, allowing us to bind plugins for a single run before or after the modules are built, or even before or after other reactor-level plugins. It would also replace the current @aggregator syntax for mojos, I think, since you would have more control over that mojo's placement than simply saying, "Run only once; I don't care when."

        The lifecycle/phase pattern works relatively well right now for single-project builds, and for most builds in a given reactor scenario. I think repeating the pattern at the reactor scale would allow us to address reactor-level and project-level plugin binding design changes at the same time, if/when we decide to do that for future releases.

        Please see the email thread for more detail on this proposal.

        Show
        John Casey added a comment - Wanted to link in this conversation, and also add a comment (somewhat excerpted from the mail thread). The thread is here: http://www.nabble.com/forum/ViewPost.jtp?post=6141419&framed=y And my comment is: I think we should explore design/development of a reactor-level lifecycle which will run once and only once per maven invocation (per build, not per project). This would allow us to declare some discrete phases in which different categories of reactor-level work takes place, and align plugins to those phases so that a more-or-less declarative ordering at the reactor level can be achieved. In this model, the module builds would take place in a reactor phase somewhere in the middle of the reactor lifecycle, allowing us to bind plugins for a single run before or after the modules are built, or even before or after other reactor-level plugins. It would also replace the current @aggregator syntax for mojos, I think, since you would have more control over that mojo's placement than simply saying, "Run only once; I don't care when." The lifecycle/phase pattern works relatively well right now for single-project builds, and for most builds in a given reactor scenario. I think repeating the pattern at the reactor scale would allow us to address reactor-level and project-level plugin binding design changes at the same time, if/when we decide to do that for future releases. Please see the email thread for more detail on this proposal.
        Hide
        Binyan added a comment -

        This is really an implementation of the composite pattern IMO and we should flip the architecture. We need to have a default lifecycle the same as it exists today. However the new default lifecycle should have the pre and post reactor phases you wrote about in the email. Then the DefaultMaven class would invoke the LifecycleExecutor implementation, which would execute the specified lifecycle (or new default if one wasn't specified) and the reactor should be bound to a phase in the new default like any other plugin. I think there should be a top level element in the pom to indicate the lifecycle to use too.

        Show
        Binyan added a comment - This is really an implementation of the composite pattern IMO and we should flip the architecture. We need to have a default lifecycle the same as it exists today. However the new default lifecycle should have the pre and post reactor phases you wrote about in the email. Then the DefaultMaven class would invoke the LifecycleExecutor implementation, which would execute the specified lifecycle (or new default if one wasn't specified) and the reactor should be bound to a phase in the new default like any other plugin. I think there should be a top level element in the pom to indicate the lifecycle to use too.
        Hide
        John Casey added a comment -

        I'm not sure I follow; are you saying that we should collapse the reactor phases into the main project build lifecycle as a new version of the existing default, and then selectively execute part of that lifecycle outside the reactor, and other parts within the reactor (i.e. per-project)?

        The reason I like the concept of scoped lifecycles is that any existing project today, when released, may have a need to produce a binary assembly that contains all of the module projects, etc. If we wrap the existing (or any other) lifecycle inside a reactor-level lifecycle, then these people will have the ability to bind the assembly plugin outside of the project-level lifecycle and have it execute at the appropriate time.

        IMO, the semantics of binding to a project-level lifecycle (for war, ear, ejb, etc. packaging) shouldn't change in most cases with this addition.

        However, now that I think a little more about your use cases (eclipse plugin dev), I think I see what you're getting at. You'd like to be able to specify something akin to <packaging>war</packaging> and have the reactor-level lifecycle bindings be provided by the default mapping for the packaging.

        We already have three lifecycles used in Maven, and a given packaging can choose to override any of the three. They are: clean, site, and default. The default one is for "normal" builds, and the other two are pretty self-explanatory, I guess. What if we added another lifecycle called "reactor" or "multimodule" which would allow mapping of things like your pde pre-processor mojo when the packaging == pde-plugin or similar?

        If you want to see how these are constructed, have a look at:

        http://svn.apache.org/repos/asf/maven/components/trunk/maven-core/src/main/resources/META-INF/plexus/components.xml

        and search for 'DefaultLifecycleMapping' to see how the different packaging types are mapped in.

        Would this meet your requirements?

        Show
        John Casey added a comment - I'm not sure I follow; are you saying that we should collapse the reactor phases into the main project build lifecycle as a new version of the existing default, and then selectively execute part of that lifecycle outside the reactor, and other parts within the reactor (i.e. per-project)? The reason I like the concept of scoped lifecycles is that any existing project today, when released, may have a need to produce a binary assembly that contains all of the module projects, etc. If we wrap the existing (or any other) lifecycle inside a reactor-level lifecycle, then these people will have the ability to bind the assembly plugin outside of the project-level lifecycle and have it execute at the appropriate time. IMO, the semantics of binding to a project-level lifecycle (for war, ear, ejb, etc. packaging) shouldn't change in most cases with this addition. However, now that I think a little more about your use cases (eclipse plugin dev), I think I see what you're getting at. You'd like to be able to specify something akin to <packaging>war</packaging> and have the reactor-level lifecycle bindings be provided by the default mapping for the packaging. We already have three lifecycles used in Maven, and a given packaging can choose to override any of the three. They are: clean, site, and default. The default one is for "normal" builds, and the other two are pretty self-explanatory, I guess. What if we added another lifecycle called "reactor" or "multimodule" which would allow mapping of things like your pde pre-processor mojo when the packaging == pde-plugin or similar? If you want to see how these are constructed, have a look at: http://svn.apache.org/repos/asf/maven/components/trunk/maven-core/src/main/resources/META-INF/plexus/components.xml and search for 'DefaultLifecycleMapping' to see how the different packaging types are mapped in. Would this meet your requirements?
        Hide
        Binyan added a comment -

        I'm saying (I think) that maven should have 1 LifecycleExecutor whose job is to execute a single Lifecycle. The lifecycle to execute could be specified on the command line and/or in the pom. I'm mainly thinking about multi-project builds but I don't see big benefits for single projects at my current caffiene level. If the user did NOT specify a lifecycle on the command line or in the pom, then the lifecycle executor would execute a new "default" lifecycle the same as it does today.

        The default lifecycle would have the pre and post reactor phases you previously outlined. The LifecycleExecutor would execute the lifecycle from start to designated phase as it does today. The big switch IMO is that the reactor would be a plugin that is bound to the reactor phases and would be called to execute the reactor across the current default lifecycle (validate ... deploy). This would allow binding before and after the reactor and current projects would continue to work with out any changes.

        I'm thinking we need this

        Lifecycle Name: Phases/Component Lifecycles
        =================================================================
        Pre-Reactor Lifecycle: reactor-init, reactor-start
        Reactor Lifecycle: reactor-build
        Post-Reactor Lifecycle: reactor-stop, reactor-assemble, reactor-verify, reactor-stop
        Default Lifecycle: Pre-Reactor Lifecycle, Reactor Lifecycle, Post-Reactor Lifecycle
        Foo Lifecycle: clean-room-phase, take-out-trash-phase, Default Lifecycle, brush-teeth-phase

        The default lifecycle would be composed of the "Pre-Reactor, Reactor, and Post-Reactor" lifecycles. This is why I thought of the composition pattern because a lifecycle is composed of phases and other lifecycles as components. Now you could bind the assembly plugin to a phase in the pre-reactor lifecycle or you could declare your own lifecycle,like the Foo lifecycle, which declares it's own phases with plugin bindings and a component lifecycle, (i.e. Default lifecycle).

        Was this any clearer?

        Show
        Binyan added a comment - I'm saying (I think) that maven should have 1 LifecycleExecutor whose job is to execute a single Lifecycle. The lifecycle to execute could be specified on the command line and/or in the pom. I'm mainly thinking about multi-project builds but I don't see big benefits for single projects at my current caffiene level. If the user did NOT specify a lifecycle on the command line or in the pom, then the lifecycle executor would execute a new "default" lifecycle the same as it does today. The default lifecycle would have the pre and post reactor phases you previously outlined. The LifecycleExecutor would execute the lifecycle from start to designated phase as it does today. The big switch IMO is that the reactor would be a plugin that is bound to the reactor phases and would be called to execute the reactor across the current default lifecycle (validate ... deploy). This would allow binding before and after the reactor and current projects would continue to work with out any changes. I'm thinking we need this Lifecycle Name: Phases/Component Lifecycles ================================================================= Pre-Reactor Lifecycle: reactor-init, reactor-start Reactor Lifecycle: reactor-build Post-Reactor Lifecycle: reactor-stop, reactor-assemble, reactor-verify, reactor-stop Default Lifecycle: Pre-Reactor Lifecycle, Reactor Lifecycle, Post-Reactor Lifecycle Foo Lifecycle: clean-room-phase, take-out-trash-phase, Default Lifecycle, brush-teeth-phase The default lifecycle would be composed of the "Pre-Reactor, Reactor, and Post-Reactor" lifecycles. This is why I thought of the composition pattern because a lifecycle is composed of phases and other lifecycles as components. Now you could bind the assembly plugin to a phase in the pre-reactor lifecycle or you could declare your own lifecycle,like the Foo lifecycle, which declares it's own phases with plugin bindings and a component lifecycle, (i.e. Default lifecycle). Was this any clearer?
        Hide
        brianfox brianfox added a comment -

        This is exactly what the dependency plugin needs to handle defacto dependencies defined in the config for copy and unpack that may be in the same reactor.

        Show
        brianfox brianfox added a comment - This is exactly what the dependency plugin needs to handle defacto dependencies defined in the config for copy and unpack that may be in the same reactor.
        Hide
        brianfox brianfox added a comment -

        Allowing a plugin to bind to a super-init phase is certainly helpful but I think some way to allow a plugin to bind to a normal phase and participate in the super-init is needed. Take the following example:

        I build a zip file that needs to be unpacked during the compile phase of a sibling project. In the dependent project, I use maven-dependency-plugin:unpack bound to the compile phase. In order to guarantee the reactor order, I would need the plugin to be able to read it's configuration and add a dynamic dependency during the super-init, but then wait to the compile phase of my project to actually unpack.

        I don't see how a single phase would allow this without duplicating the configuration. I see something more like a new "execute" method that gets called during super-init to allow some interaction, or some other way to map methods to a "super" phase. I'm thinking of a case where the plugin author knows that something will always need to be done and doesn't want to require extra pom work to make it work correctly.

        Show
        brianfox brianfox added a comment - Allowing a plugin to bind to a super-init phase is certainly helpful but I think some way to allow a plugin to bind to a normal phase and participate in the super-init is needed. Take the following example: I build a zip file that needs to be unpacked during the compile phase of a sibling project. In the dependent project, I use maven-dependency-plugin:unpack bound to the compile phase. In order to guarantee the reactor order, I would need the plugin to be able to read it's configuration and add a dynamic dependency during the super-init, but then wait to the compile phase of my project to actually unpack. I don't see how a single phase would allow this without duplicating the configuration. I see something more like a new "execute" method that gets called during super-init to allow some interaction, or some other way to map methods to a "super" phase. I'm thinking of a case where the plugin author knows that something will always need to be done and doesn't want to require extra pom work to make it work correctly.
        Hide
        Binyan added a comment -

        Jason, I see you've picked this issue up. If I can shed more light on why this is needed or how it applies in the eclipse plug-in or specifically in the more general building OSGi bundles case then please let me know. I missed seeing Brian F. previous comments but it al most seems like he's talking about a new interface a mojo would implement to get pluggeg into the super-init phase. Similar I imagine to how spring uses its create and destroy interfaces for beans to be hooked into the spring context's lifecycle.

        I imagine that this issue might be addresses/targeted for Maven 2.1, so on a tangential issue will it be possible to dynamically create/augment MavenProject instances and add them to the Reactor's list before it sorts them? I ask because I just last week started taking another crack at solving the Eclipse PDE build issue with Maven. Basically I have written up in a Confluence page everything that the Eclipse PDE build ant scripts do, with the intent of replacing that with several mojos. However, PDE is not about to change and I don't want developers maintaining build info in 2 places, so I want to have a simple pom.xml in every plugin, feature, fragment, etc project that has a mojo bound that will augment the MavenProject instance with data pulled from eclipse's own project files. Utimatley I would want to do this augmentation in the "super-init" phase.

        The layout could be the following:

        MyPluginProject:

        • pom.xml
        • features/
        • com.example.foo/
        • pom.xml
        • ...
        • plugins/
        • com.example.bar/
        • pom.xml
        • ...
        • com.example.baz/
        • pom.xml
        • ...

        Note that the pom files in the individual features and plugins folders might not exist if they can be dynamically created a mojos in the top level pom.

        Show
        Binyan added a comment - Jason, I see you've picked this issue up. If I can shed more light on why this is needed or how it applies in the eclipse plug-in or specifically in the more general building OSGi bundles case then please let me know. I missed seeing Brian F. previous comments but it al most seems like he's talking about a new interface a mojo would implement to get pluggeg into the super-init phase. Similar I imagine to how spring uses its create and destroy interfaces for beans to be hooked into the spring context's lifecycle. I imagine that this issue might be addresses/targeted for Maven 2.1, so on a tangential issue will it be possible to dynamically create/augment MavenProject instances and add them to the Reactor's list before it sorts them? I ask because I just last week started taking another crack at solving the Eclipse PDE build issue with Maven. Basically I have written up in a Confluence page everything that the Eclipse PDE build ant scripts do, with the intent of replacing that with several mojos. However, PDE is not about to change and I don't want developers maintaining build info in 2 places, so I want to have a simple pom.xml in every plugin, feature, fragment, etc project that has a mojo bound that will augment the MavenProject instance with data pulled from eclipse's own project files. Utimatley I would want to do this augmentation in the "super-init" phase. The layout could be the following: MyPluginProject: pom.xml features/ com.example.foo/ pom.xml ... plugins/ com.example.bar/ pom.xml ... com.example.baz/ pom.xml ... Note that the pom files in the individual features and plugins folders might not exist if they can be dynamically created a mojos in the top level pom.
        Hide
        Jason van Zyl added a comment -

        Have you looked at Tom's work?

        http://svn.codehaus.org/m2eclipse/tycho/trunk/

        Show
        Jason van Zyl added a comment - Have you looked at Tom's work? http://svn.codehaus.org/m2eclipse/tycho/trunk/
        Hide
        Binyan added a comment -

        No, but I will now. Thnx.

        Show
        Binyan added a comment - No, but I will now. Thnx.
        Hide
        Jason van Zyl added a comment -

        It's probably the most comprehensive and Tom has been using it in production for quite some time. We have talked about reading manifests directly and it embeds the PDE resolver so it's not just a PDE wrapper. That approach still works and there is the PDE plugin and the PST work:

        http://svn.codehaus.org/m2eclipse/maven-pst/

        Show
        Jason van Zyl added a comment - It's probably the most comprehensive and Tom has been using it in production for quite some time. We have talked about reading manifests directly and it embeds the PDE resolver so it's not just a PDE wrapper. That approach still works and there is the PDE plugin and the PST work: http://svn.codehaus.org/m2eclipse/maven-pst/
        Hide
        Binyan added a comment -

        If I understand the last sentence, then you're saying that we have have 3 separate initiatives going on with Tom's work the PST plug-in (which I'm reading about now) and the PDE plug-in I worked up. If I got that wrong then please correct. I have some work similar to the PST stuff as I created mojos to handle an eclipse target platform and put it into a maven repo too. I started looking at Tom's work before I headed to the office and I'll pick back up after lunch today and after I finish the article on the PST mojos. I'm pretty sure Tom's ahead of me and would welcome the chance to work with him to solve this generic OSGi bundle building problem.

        Show
        Binyan added a comment - If I understand the last sentence, then you're saying that we have have 3 separate initiatives going on with Tom's work the PST plug-in (which I'm reading about now) and the PDE plug-in I worked up. If I got that wrong then please correct. I have some work similar to the PST stuff as I created mojos to handle an eclipse target platform and put it into a maven repo too. I started looking at Tom's work before I headed to the office and I'll pick back up after lunch today and after I finish the article on the PST mojos. I'm pretty sure Tom's ahead of me and would welcome the chance to work with him to solve this generic OSGi bundle building problem.
        Hide
        Brett Porter added a comment -

        this was recently discussed again on Maven dev for a different use case

        Show
        Brett Porter added a comment - this was recently discussed again on Maven dev for a different use case
        Hide
        Jason van Zyl added a comment -

        You can now access the extension points before and after the build is executed in 3.x. We use this exactly for OSGi as you were trying with the PDE plugin.

        Show
        Jason van Zyl added a comment - You can now access the extension points before and after the build is executed in 3.x. We use this exactly for OSGi as you were trying with the PDE plugin.

          People

          • Assignee:
            Jason van Zyl
            Reporter:
            Binyan
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: