Details
-
Type:
Bug
-
Status:
Closed
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 1.5.6
-
Fix Version/s: 1.5.7, 1.6-beta-2
-
Component/s: groovy-jdk
-
Labels:None
-
Environment:java 1.5.0_14 and java 1.6.0_04
-
Testcase included:yes
-
Number of attachments :
Description
The GroovySystem is a class that resides in the AppClassLoader. It's final and only contains static attributes and methods.
Inside it, we find a MetaClassRegistry static property, that is initialized with a MetaClassRegistryImpl instance.
The MetaClassRegistryImpl contains a ConcurrentReaderHashMap wich holds references to all Expando classes, and metaclasses.
And that's the problem. Since the ExpandoClasses and MetaClasses are usually created via script by a GroovyClassLoader, this map prevents them be garbage collected, as well as its GroovyClassLoader.
In our application, we use a schema similar to a web server. We create a different class loader for each executed script. This is necessary since users want to modificate scripts and see the results imediatelly. We've even tried to use the GroovyScriptEngine in the past, but it has a very big flaw. its performance is terrible in a network (I even opened another issue for this). Also, we like the benefit of having a different class loader per script, since we may have different execution environments.
As a result, we are suffering the "the dreaded "java.lang.OutOfMemoryError: PermGen space" exception", as described in this article:
http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java
To simulate the problem, run the above java program with -Xmx500m option:
public class Test { public static void main(String[] args) throws Exception { for (int i = 0; i < Integer.MAX_VALUE; i++) { GroovyClassLoader gcl = new GroovyClassLoader(); Leak leak = (Leak)gcl.parseClass(new File("ResourceLeak.groovy")).newInstance(); leak.leakResources(); if (i % 1000 == 0) { System.out.println(i); System.gc(); } } } public static interface Leak { void leakResources(); } }
It uses this script:
class ResourceLeak implements Teste.Leak { void leakResources() { ResourceLeak.metaClass.'static'.test = { //Do nothing } ResourceLeak.test() } }
The problem will occur after 7000 executions. In this sample, we need 500Mb heap, otherwise, the heap space will go out of memory before the permgen space. But in our real situation (with bigger classes, and more classes in the class loader) 200Mb heap is enough, since the permgen space is consumed faster.
This is a non trivial problem in respect to many cases. Currently the MetaClassImpl is designed to keep the default MetaClass as long as it is needed, that means the default can be collected. But as soon as a user sets a custom MetaClass the default is to keep it until the dead come back or the VM is terminated.
Our problem is that we can not know if the MetaClass will be needed or not. The standard case in Groovy is that you execute many scripts that somehow interact with each other. So there is no defined process of script creation, execution and removing resources which I could use to clear the registry. I could now for example say, that the MetaClass should not prevent ResourceLeak from being collected and the MetaClass should be collected along with it. That's true, but for this we would need a reference from the class to the MetaClass. Which could be done for classes created by Groovy, but not for classes from Java. So you could for example create a Java class each time the script is run and change its MetaClass like above and we would get the same problem again. Any solution that tries some kind of tracking or referencing the class as well as the MetaClass failed in more earlier attempts.
But there are solutions to your problem as well. One way would be to use different class loaders loading the Groovy classes with them, so that you can discard all of them together with the registry. Another solution would be to remove each MetaClass you added. To keep track of this I would suggest to exchange the getMetaClass method on Class with a version that allows you to track the added classes. The second version is not threadsafe of course.
ah, yes... why don't you cache the class? There wouldn't be a leak in that case.