groovy
  1. groovy
  2. GROOVY-3488

Groovy ant task memory leak (re-opened GROOVY-891)

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Critical Critical
    • Resolution: Won't Fix
    • Affects Version/s: 1.0-JSR-2
    • Fix Version/s: None
    • Component/s: Ant integration
    • Labels:
      None
    • Environment:
    • Testcase included:
      yes
    • Number of attachments :
      0

      Description

      There appears to be a memory leak in the groovy ant task. This becomes apparent when calling from one ant file to a number of others, each with a groovy task.

      I performed the following tests:

      • Increase the complexity and memory used of the groovy script in ant. This had no effect.
      • Increase the number of groovy tasks in a single ant script. This had no effect
      • Increase the total number of ant scripts called. This showed memory growth.

      The following message is typical:

      BUILD FAILED
      D:\projects\config\build\temp\build.xml:13: Following error occured while executing this line
      D:\projects\config\build\temp\dependencies.xml:87: Following error occured while executing this line
      java.lang.OutOfMemoryError

      I have attached a demonstration ant build system similar to our project build where the problem first occurred. For best results set the JVM memory via the ANT_OPTS environment variable to something like 16 Meg ie:

      set ANT_OPTS=-Xmx16M

      and then run ant.

        Issue Links

          Activity

          Hide
          Craig Federighi added a comment -

          Please see test case attached to http://jira.codehaus.org/browse/GROOVY-891.

          This issue still exists under Ant 1.7.1 + Groovy 1.6. The memory leak is severe enough that it causes the AribaWeb (http://aribaweb.org) build to sometimes run out of memory even with Ant configured with a very large heap. (e.g. ANT_OPTS=-Xmx400m -XX:MaxPermSize=300m).

          Show
          Craig Federighi added a comment - Please see test case attached to http://jira.codehaus.org/browse/GROOVY-891 . This issue still exists under Ant 1.7.1 + Groovy 1.6. The memory leak is severe enough that it causes the AribaWeb ( http://aribaweb.org ) build to sometimes run out of memory even with Ant configured with a very large heap. (e.g. ANT_OPTS=-Xmx400m -XX:MaxPermSize=300m).
          Hide
          Daniel Lindner added a comment - - edited

          I had a similar effect when running several Canoo webtests in one Ant script (and therefor one JVM). The webtest starter ant script uses the Groovy ant task to run its scripts written in Groovy.
          The OOME affects the heap, not the permspace.

          A possible fix that solved the issue for me was to release some resources of the GroovyShell that is used in the Groovy ant task.

          In the method execGroovy(), the last two lines construct a new GroovyShell and uses it. Afterwards, the instance is omitted as it's only locally scoped.
          Explicitly freeing up the caches of the shell fixed the OOME for me:

          final GroovyShell groovy = new GroovyShell(classLoader, new Binding(), configuration);
          try {
              parseAndRunScript(groovy, txt, mavenPom, scriptName, null, new AntBuilder(this));
          } finally {
              groovy.resetLoadedClasses();
              groovy.getClassLoader().clearCache();
          }
          

          I hope this helps a bit

          Show
          Daniel Lindner added a comment - - edited I had a similar effect when running several Canoo webtests in one Ant script (and therefor one JVM). The webtest starter ant script uses the Groovy ant task to run its scripts written in Groovy. The OOME affects the heap, not the permspace. A possible fix that solved the issue for me was to release some resources of the GroovyShell that is used in the Groovy ant task. In the method execGroovy(), the last two lines construct a new GroovyShell and uses it. Afterwards, the instance is omitted as it's only locally scoped. Explicitly freeing up the caches of the shell fixed the OOME for me: final GroovyShell groovy = new GroovyShell(classLoader, new Binding(), configuration); try { parseAndRunScript(groovy, txt, mavenPom, scriptName, null , new AntBuilder( this )); } finally { groovy.resetLoadedClasses(); groovy.getClassLoader().clearCache(); } I hope this helps a bit
          Hide
          Daniel Lindner added a comment - - edited

          Well, the line separators didn't work out:

          final GroovyShell groovy = new GroovyShell(classLoader, new Binding(), configuration);
          try {
              parseAndRunScript(groovy, txt, mavenPom, scriptName, null, new AntBuilder(this));
          } finally {
              groovy.resetLoadedClasses();
              groovy.getClassLoader().clearCache();
          }
          
          Show
          Daniel Lindner added a comment - - edited Well, the line separators didn't work out: final GroovyShell groovy = new GroovyShell(classLoader, new Binding(), configuration); try { parseAndRunScript(groovy, txt, mavenPom, scriptName, null , new AntBuilder( this )); } finally { groovy.resetLoadedClasses(); groovy.getClassLoader().clearCache(); }
          Hide
          Daniel Lindner added a comment -

          So be it, then. Sorry

          Show
          Daniel Lindner added a comment - So be it, then. Sorry
          Hide
          Paul King added a comment -

          Daniel, you should try using {code} tags. Click the little help icon while editing/entering.

          Show
          Paul King added a comment - Daniel, you should try using {code} tags. Click the little help icon while editing/entering.
          Hide
          Paul King added a comment - - edited

          Problem is reproducible with Ant 1.8 and Groovy trunk - though it does take until step 38 out of 40 before it gets the OOM error!
          Using groovy.resetLoadedClasses() in the finally block did not seem to fix the problem for the Ant build test case. Perhaps it is worth doing anyway if it fixes similar issues for WebTest. I can't see any downside - though if we can find a fix for both that would be better.

          Show
          Paul King added a comment - - edited Problem is reproducible with Ant 1.8 and Groovy trunk - though it does take until step 38 out of 40 before it gets the OOM error! Using groovy.resetLoadedClasses() in the finally block did not seem to fix the problem for the Ant build test case. Perhaps it is worth doing anyway if it fixes similar issues for WebTest. I can't see any downside - though if we can find a fix for both that would be better.
          Hide
          Davide Gurgone added a comment -

          Hi,
          I don't know if can helps you, but I register a very big memory leak in a code like your, but without use of ant, simply invoking GroovyShell a lot of times.

          Here: http://tinyurl.com/groovy-shell-out-of-memory I opened a thread on mailing list, to describe problem.

          Show
          Davide Gurgone added a comment - Hi, I don't know if can helps you, but I register a very big memory leak in a code like your, but without use of ant, simply invoking GroovyShell a lot of times. Here: http://tinyurl.com/groovy-shell-out-of-memory I opened a thread on mailing list, to describe problem.
          Hide
          blackdrag blackdrag added a comment - - edited

          Davide, that your example fails with a permgen problem is actaully what I expect. Each run of GroovyShell will add a new class and the class is not removed somewhere, that is by design. Instead you are supposed to use a new GroovyShell for each call or at least from time to time. Then the classes could be collected.

          getting the loader fom the shell and call resetLoadedClasses on it is a workaround in case you don't want to create a new instance

          Show
          blackdrag blackdrag added a comment - - edited Davide, that your example fails with a permgen problem is actaully what I expect. Each run of GroovyShell will add a new class and the class is not removed somewhere, that is by design. Instead you are supposed to use a new GroovyShell for each call or at least from time to time. Then the classes could be collected. getting the loader fom the shell and call resetLoadedClasses on it is a workaround in case you don't want to create a new instance
          Hide
          Paul King added a comment -

          The resetLoadedClasses() and clearCache() suggestion have been added to 1_7_X, 1_8_X and trunk. I don't think they fix all cases but certainly seem to help.

          Show
          Paul King added a comment - The resetLoadedClasses() and clearCache() suggestion have been added to 1_7_X, 1_8_X and trunk. I don't think they fix all cases but certainly seem to help.
          Hide
          Pascal Schumacher added a comment -

          Reading the comments I guess this issue should be closed?

          Show
          Pascal Schumacher added a comment - Reading the comments I guess this issue should be closed?
          Hide
          Paul King added a comment -

          Closing this as the discussion is around old versions of Groovy and dependencies. There are other issues addressing memory leakage, e.g. GROOVY-6068, but please reopen a new issue with specific details if you are still having problems with recent versions.

          Show
          Paul King added a comment - Closing this as the discussion is around old versions of Groovy and dependencies. There are other issues addressing memory leakage, e.g. GROOVY-6068 , but please reopen a new issue with specific details if you are still having problems with recent versions.

            People

            • Assignee:
              Paul King
              Reporter:
              Craig Federighi
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: