Continuum
  1. Continuum
  2. CONTINUUM-782

Add a feature to allow cleaning the m2 local repo once every N days

    Details

    • Type: New Feature New Feature
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3
    • Fix Version/s: 1.2
    • Component/s: Core system
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      6

      Description

      It's really hard with maven to know if your build is working or not. I've been hit several times already by buillds working fine locally and on the CIs but failing for a newcomer trying to build that project. The reason is that remote repos are not immutable and artifacts can disappear or be modified in there.

      It would be nice if continnuum could help notice when this happen. A way to implement this would to be to configure Continuum to clean its local repo every N days.

        Issue Links

          Activity

          Hide
          Jesse McConnell added a comment -

          short term solution would be to use -Dmaven.repo.local=/path/ in a custom scheduled build, could be that if you set it to something relative to the executing project that the repo could be cleaned up with clean...

          perhaps a weekly build schedule with goals clean and install with the added commandline option -Dmaven.repo.local="./target/localRepository" would work...

          Show
          Jesse McConnell added a comment - short term solution would be to use -Dmaven.repo.local=/path/ in a custom scheduled build, could be that if you set it to something relative to the executing project that the repo could be cleaned up with clean... perhaps a weekly build schedule with goals clean and install with the added commandline option -Dmaven.repo.local="./target/localRepository" would work...
          Hide
          Milos Kleint added a comment -

          actually the suggested workaround with -Dmaven.repo.local doesn't work, at least not on the continnum instance at codehaus. My build fails whenever I try to put a relative path.

          What I would like to achieve is a "clean" build for my project.
          1. have a local repository that gets clean everytime
          2. have a local repository that is private to my project (set of projects) and doesn't interfere with other projects running on the same continuum instance
          2a. by project I mean something like mevenide which consists of multiple modules but I want it to build all at once

          I always do a clean build of mevenide prior to commit and commit everything, the major feature of a continuum server for me is to check if my project is buildable with a given set of remote repositories, as cross pollination of my local repository is a common source of problems for me.

          Show
          Milos Kleint added a comment - actually the suggested workaround with -Dmaven.repo.local doesn't work, at least not on the continnum instance at codehaus. My build fails whenever I try to put a relative path. What I would like to achieve is a "clean" build for my project. 1. have a local repository that gets clean everytime 2. have a local repository that is private to my project (set of projects) and doesn't interfere with other projects running on the same continuum instance 2a. by project I mean something like mevenide which consists of multiple modules but I want it to build all at once I always do a clean build of mevenide prior to commit and commit everything, the major feature of a continuum server for me is to check if my project is buildable with a given set of remote repositories, as cross pollination of my local repository is a common source of problems for me.
          Hide
          Maria Catherine Tan added a comment - - edited

          I would like to work on this issue

          Here's what I would like to do:

          1. create a new table for local repositories

          2. create a local repo maintenance page

          • add, edit, delete

          3. scheduled jobs

          • select which repository to clean

          4. build definition

          • select which repository to use ( default to $ {user_home}

            /.m2/repository )

          • enable/disable clean repository before build (for group build definition only)

          Wdyt? If everyone is fine with this then I'll create a sub task for this issue.

          Show
          Maria Catherine Tan added a comment - - edited I would like to work on this issue Here's what I would like to do: 1. create a new table for local repositories 2. create a local repo maintenance page add, edit, delete 3. scheduled jobs select which repository to clean 4. build definition select which repository to use ( default to $ {user_home} /.m2/repository ) enable/disable clean repository before build (for group build definition only) Wdyt? If everyone is fine with this then I'll create a sub task for this issue.
          Hide
          Emmanuel Venisse added a comment -

          Maria, it's good to me.
          Look at CONTINUUM-1693 too to remove some old snapshot. I think Archiva do it already, maybe we'll can reuse the code.

          Show
          Emmanuel Venisse added a comment - Maria, it's good to me. Look at CONTINUUM-1693 too to remove some old snapshot. I think Archiva do it already, maybe we'll can reuse the code.
          Hide
          Wendy Smoak added a comment -

          Can you make sure that a repository does not get cleaned while a project that uses it is building?

          Show
          Wendy Smoak added a comment - Can you make sure that a repository does not get cleaned while a project that uses it is building?
          Hide
          Maria Catherine Tan added a comment -

          I have attached the patch for purging. This patch includes fixes for CONTINUUM-782, CONTINUUM-1693 and CONTINUUM-1759.

          Sorry if I did not put it into chunks. Please review my patch Thanks

          Show
          Maria Catherine Tan added a comment - I have attached the patch for purging. This patch includes fixes for CONTINUUM-782 , CONTINUUM-1693 and CONTINUUM-1759 . Sorry if I did not put it into chunks. Please review my patch Thanks
          Hide
          Maria Catherine Tan added a comment -

          attached purge images

          Show
          Maria Catherine Tan added a comment - attached purge images
          Hide
          Maria Catherine Tan added a comment -

          attached documentation

          Show
          Maria Catherine Tan added a comment - attached documentation
          Hide
          Maria Catherine Tan added a comment -

          attached documentation images

          Show
          Maria Catherine Tan added a comment - attached documentation images
          Hide
          Maria Catherine Tan added a comment -

          I originally plan to create a local repository field in build definition but later moved it to the project group because of CONTINUUM-1465.

          I'm not sure if this one is better or should I put local repository back in the build definition. I've read other jira issues and found out some liked it to have the local repository in the build definition CONTINUUM-265.

          Wdyt?

          Show
          Maria Catherine Tan added a comment - I originally plan to create a local repository field in build definition but later moved it to the project group because of CONTINUUM-1465 . I'm not sure if this one is better or should I put local repository back in the build definition. I've read other jira issues and found out some liked it to have the local repository in the build definition CONTINUUM-265 . Wdyt?
          Hide
          Maria Catherine Tan added a comment -

          Or we could leave the local repository in the project group but still add one per build definition, which will override the local repository in the project group.

          Show
          Maria Catherine Tan added a comment - Or we could leave the local repository in the project group but still add one per build definition, which will override the local repository in the project group.
          Hide
          Maria Catherine Tan added a comment -

          I have attached a newer patch, "CONTINUUM-purge-newer.patch", that contains fixes for:
          1. test failures when purging directories
          2. editing repository when it is in use
          3. error message when unable to remove, edit, purge repository because it is in use

          Please use this patch instead of the old one.

          Show
          Maria Catherine Tan added a comment - I have attached a newer patch, "CONTINUUM-purge-newer.patch", that contains fixes for: 1. test failures when purging directories 2. editing repository when it is in use 3. error message when unable to remove, edit, purge repository because it is in use Please use this patch instead of the old one.
          Hide
          Wendy Smoak added a comment -

          Marica, I have this all applied locally and it looks good, however it's quite a large contribution and I don't see a CLA on file for you on http://people.apache.org/~jim/committers.html . I'm guessing you might be covered by a Corporate CLA, but I don't think I have a way to check that. (And is an iCLA still required even if there is a CCLA?)

          Show
          Wendy Smoak added a comment - Marica, I have this all applied locally and it looks good, however it's quite a large contribution and I don't see a CLA on file for you on http://people.apache.org/~jim/committers.html . I'm guessing you might be covered by a Corporate CLA, but I don't think I have a way to check that. (And is an iCLA still required even if there is a CCLA?)
          Hide
          Maria Catherine Tan added a comment -

          Wendy, I'll file a CLA tomorrow and send through fax. Thanks

          Show
          Maria Catherine Tan added a comment - Wendy, I'll file a CLA tomorrow and send through fax. Thanks
          Hide
          Maria Catherine Tan added a comment -

          Found some more issues with my patch so I have attached continuum-purge-07242008.patch that fixes those issues.

          1. added continuum-configuration.xml in src/test/resources of continuum-purge module ( for revision 678639 )
          2. refactored packages to org.apache.continuum
          3. fixed some purging tests
          4. added creation of default m2 local repository during continuum initialization
          5. set default project group's local repository to #4
          6. fixed repository used when releasing and deploying
          7. set project group's local repository to #4 when adding a new maven 2 project whose project group is set to "defined by pom"
          8. will not be able to edit/remove the default local repository
          9. changed archiva version to 1.1

          Show
          Maria Catherine Tan added a comment - Found some more issues with my patch so I have attached continuum-purge-07242008.patch that fixes those issues. 1. added continuum-configuration.xml in src/test/resources of continuum-purge module ( for revision 678639 ) 2. refactored packages to org.apache.continuum 3. fixed some purging tests 4. added creation of default m2 local repository during continuum initialization 5. set default project group's local repository to #4 6. fixed repository used when releasing and deploying 7. set project group's local repository to #4 when adding a new maven 2 project whose project group is set to "defined by pom" 8. will not be able to edit/remove the default local repository 9. changed archiva version to 1.1
          Hide
          Olivier Lamy added a comment -

          committed in https://svn.apache.org/repos/asf/continuum/branches/CONTINUUM-782/ in rev 679588
          patch changed to handle new field added in a table
          application.xml in webapp to can start new version with a 1.1 db :

            <property>
              <name>org.jpox.autoCreateColumns</name>
              <value>true</value>
            </property>
          
          Show
          Olivier Lamy added a comment - committed in https://svn.apache.org/repos/asf/continuum/branches/CONTINUUM-782/ in rev 679588 patch changed to handle new field added in a table application.xml in webapp to can start new version with a 1.1 db : <property> <name>org.jpox.autoCreateColumns</name> <value> true </value> </property>
          Hide
          Olivier Lamy added a comment -

          fixed in rev 681492.
          THANKS !!

          Show
          Olivier Lamy added a comment - fixed in rev 681492. THANKS !!
          Hide
          Olivier Lamy added a comment -

          reopen issue because images are missing.

          Show
          Olivier Lamy added a comment - reopen issue because images are missing.
          Hide
          Maria Catherine Tan added a comment -

          Hi Olivier The images were already been attached --> purge-images.zip Thanks

          Show
          Maria Catherine Tan added a comment - Hi Olivier The images were already been attached --> purge-images.zip Thanks
          Hide
          Olivier Lamy added a comment -

          missing images added in rev 682541.

          Show
          Olivier Lamy added a comment - missing images added in rev 682541.

            People

            • Assignee:
              Olivier Lamy
              Reporter:
              Vincent Massol
            • Votes:
              5 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: