TemplateServlet with default configuration leaks classes resulting in a PermGen error under Tomcat

Looks like I jumped to conclusions too early. Switching to GStringTemplateEngine does prevent the PermGen problem, but I am still getting memory problems. I saw increased load on the server where I've deployed groovlets, heap summary gave me >99.999% of Old Generation in use and the load was coming from FullGC cycles happening. I looked at the heap dump with Eclipse's MAT tool, java.beans.introspector was first in the dominator tree, second was the WebAppClassloader, and inside that hashmaps that kept BeanInfo and some Closurex_Metaclass-objects. I think that with GStringTemplateEngine the classes are actually unloaded so PermGen doesn't fill up, but reflection and other metainformation that Groovy requires is left behind from the classes and fill the heap.

Should it be possible to compile and execute the groovy script multiple (endless) times in a loop endlessly, always throwing away the produced class and it's unique classloader? For me this doesn't seem to work as the heap gets full from introspection caches on other objects containing meta information about the groovy scripts. I could use some help here from someone with more groovy knowledge.

BeanInfo of what is kept? of Groovy generated classes? That I can hardly imagine. I guess it is about classes used by the generated classes. Now what happens if a BeanInfo class is loaded? As with any other class, the class stays in memory till the classloader is unloaded. Since you don't run into a permgen problem anymore now, it means the laoder used for generated classes is unloaded, but the beaninfo stuff is loaded on a higher loader, which is not under Groovy control. In short, besids not loading the beaninfo classes there is nothing we can do about them.

I wonder about the Closurex_Metaclass-objects you mention. I guess you mean ClosureMetaClass instead. This class is, unless you set it yourself, not hard referenced, but weak referenced. So if you take a look at memory it is natural to see this class around. But it does not mean they are not unloaded if memory is low.

Have you ensured you are getting OutOfMemoryErrors still?

I attached two screenshots from the heap dump's dominator tree. The java.beans.Instropector's static caches are not being cleared. The caches are under a WeakHashMap, so they should eventually be cleared if no one else is pointing at the items in the caches (although Introspector's javadoc does recommend to flush the caches manually if multiple classloaders are used).

I think the reason why these classes cannot be cleared may be Tomcat's WebappClassLoader. I peeked at Tomcat's source and the HashMap that is visible in the heap dump screenshots is probably WebappClassLoader.resourceEntries. Javadoc describes it as "The cache of ResourceEntry for classes and resources we have loaded, keyed by resource name.".

So the problem I am having may be the result of GSP templates being compiled multiple times and the way Tomcat caches webapplication resources. I first actually tried to reproduce this with Jetty but couldn't and then moved to Tomcat which was the one used on the problematic server.

The server didn't actually throw OOME, but full gc cycles were happening and oldgen was staying at 99.999%. After that I restarted the server and deployed a modified version of Groovy where TemplateServlet.cache is a java.util.concurrent.ConcurrentHashMap instead of a WeakHashMap. Using a normal Map as the cache prevents the templates from being garbage collected so that gets around the problem. If I would now modify those templates repeatedly I would eventually run out of heap, but for me this is not a problem and I can accept such a limitation if I get a stable server in return.

About the BeanInfo stuff, I think everone would prefer groovlets to work without explicit cleaning calls. But I admit that there is no evidence that the data in the Introspector caches is anything that should have been GC'd. And the WebappClassLoader thing, looks like it's a Tomcat bug, and I'm in luck. Looks like it's about to be fixed on the next release of Tomcat 5.5 series and also has already been fixed in 6 series: https://issues.apache.org/bugzilla/show_bug.cgi?id=47987

So I agree, TemplateServlet with GStringTemplateEngine works with the latest tomcat versions. What about the original bug report though, if TemplateServlet is used with SimpleTemplateEngine?

We don't clean the beaninfo caches, because it costs runtime performance and should be done only if it is really needed.

As for the SimpleTemplateEngine, true, that should be fixed

Any more update on this issue

I would like to report the same problem using groovy v1.7.4 (I am using GroovyServlet thought) with tomcat 6 (testing with jetty had also the same result). My PermGen was growing bigger as the loaded classes. Trying to figure out what was going on I tried to list the loaded classes with jconsole and saw a lot of groovlets (the same ones!) being loaded continuously. Falling back to v1.7.3 made the problem disappear.

As I am using a different version of tomcat, jetty and groovy and make use of the groovy.servlet.GroovyServlet I am not quiet sure if it is the same bug. Should I create a new issue?