jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
  • Continuum
  • CONTINUUM-2119

Cleaning up large working directories fails with OutOfMemoryError

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: Bug Bug
  • Status: Reopened Reopened
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: 1.2.3
  • Fix Version/s: 1.4.2 (Beta)
  • Component/s: Core system
  • Labels:
    None
  • Complexity:
    Intermediate
  • Patch Submitted:
    Yes

Description

The action clean-working-directory runs out of heap space while cleaning up huge working directories (30GB / 500.000 files & directories in our case):

2009-03-08 06:00:14,582 [pool-1-thread-1] INFO buildController - Initializing build
2009-03-08 06:00:15,286 [pool-1-thread-1] INFO buildController - Starting build of ivu_plan_nightly_build
2009-03-08 06:00:15,301 [pool-1-thread-1] INFO buildController - Purging exiting working copy
2009-03-08 06:00:15,301 [pool-1-thread-1] INFO buildController - Performing action clean-working-directory
2009-03-08 06:31:39,209 [Thread-3] ERROR taskQueueExecutor#build-project - Error executing task
edu.emory.mathcs.backport.java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:301)
at edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:120)
at org.codehaus.plexus.taskqueue.execution.ThreadedTaskQueueExecutor$ExecutorRunnable.waitForTask(ThreadedTaskQueueExecutor.java:159)
at org.codehaus.plexus.taskqueue.execution.ThreadedTaskQueueExecutor$ExecutorRunnable.run(ThreadedTaskQueueExecutor.java:127)
Caused by: java.lang.OutOfMemoryError: Java heap space

The issue seems to be caused by the implementation in CleanWorkingDirectoryAction: the FileSetManager used to delete the working directory first scans all files & directores to create an object representation of the tree and then deletes the tree by scanning through the objects. As no filter condition is required using org.codehaus.plexus.util.FileUtils to directly delete the tree would avoid these kind of problems.

I'll attach a patch based on 1.2.3.

  • Options
    • Sort By Name
    • Sort By Date
    • Ascending
    • Descending
    • Download All

Attachments

  1. Text File
    CONTINUUM-2119-continuum-core.patch
    10/Mar/09 3:45 AM
    1 kB
    Frank Förstemann

Issue Links

is related to

Bug - A problem which impairs or prevents the functions of the product. MCLEAN-32 Out of Memory Error cleaning target directory with *large* number of sub-directories

  • Critical - Crashes, loss of data, severe memory leak.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Frank Förstemann added a comment - 10/Mar/09 3:45 AM

Patch to use org.codehaus.plexus.util.FileUtils to purge working directory.

Show
Frank Förstemann added a comment - 10/Mar/09 3:45 AM Patch to use org.codehaus.plexus.util.FileUtils to purge working directory.
Hide
Permalink
Emmanuel Venisse added a comment - 22/Apr/09 11:51 PM

Applied in rev.767793. Thanks

Show
Emmanuel Venisse added a comment - 22/Apr/09 11:51 PM Applied in rev.767793. Thanks
Hide
Permalink
Emmanuel Venisse added a comment - 23/Apr/09 3:08 AM

Need to check if symlinks are followed. If they are, the patch must be reverted.

Show
Emmanuel Venisse added a comment - 23/Apr/09 3:08 AM Need to check if symlinks are followed. If they are, the patch must be reverted.
Hide
Permalink
Emmanuel Venisse added a comment - 27/Apr/09 12:09 AM

Need a fix in maven file-management project

Show
Emmanuel Venisse added a comment - 27/Apr/09 12:09 AM Need a fix in maven file-management project

People

  • Assignee:
    Unassigned
    Reporter:
    Frank Förstemann
Vote (0)
Watch (1)

Dates

  • Created:
    10/Mar/09 3:40 AM
    Updated:
    17/Jul/11 8:00 AM
  • Atlassian JIRA (v5.0.4#731-sha1:3aa7374)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.