Details
-
Type:
Bug
-
Status:
Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.0-rc3
-
Fix Version/s: 1.1-beta-1
-
Component/s: jelly/ant integration
-
Labels:None
-
Number of attachments :
Description
The test was done as follows. I took ledge-components project http://objectledge.org/modules/ledge-components, cleaned out the target, then ran statcvs goal. Statcvs generated 160 xdoc documents, 2.2MB of markup total. Then I ran xdocs goal, to tranfrom the xdocs into html, under hprof profiler build into JDK. Settings were: allocation site analysis, heap dump at exit, stacktrace depth 16. The run took about 65 minutes on Athlon 1700XP, 512 phisical RAM. Maximum memory usage was 530MB virtual/240MB resident during the run, it surged some additional 300MB both during heap dump. The generated dump file is about 350MB, 23MB when bzip2 compressed.
I'm going to post an xls file on Jira containg the statistics of objects allocated, and objects alive at the time of VM generation. From what I'm seeing none of the Jelly TagScript objects created gets ever garbage collected. Wading through the heap dump I was able to determine that all of them are connected to the GC root through the main Thread, and a ThreadLocal variable tagHolder declared in jelly's TagScript.java, line 115. Despite calls to tagHolder.set(null), the TagScript object stays connected to the Thread by means of ThreadLocal$ThreadLocalMap$Entry object which is a subclass of java.lang.ref.WeakReference. ThreadLocal implementation in JDK works like that. Now, AFAIK the GC is free to harvest all objects that are not connected to a GC root through hard reference, or a SoftReference. I wasn't able to trace all references in the object graph regarding TagScript objects (there are 21k+ of them!) so I am not sure if there are any other references that connect the TagScripts to a GC root through a permanent refernce, like a static field in some class. I cant tell you it's a dog's work to do that by hand. If I only had a proper memory analysys tool! Alas all the good ones are commercial... On the other hand GC is not required to harvest these objects. Maybe it just didn't bother to harvest because it was not pressed for memory? In my test it had still many megabytes left. Maybe a better test would be running it with too few memory and checking what was left alive, despite the fact GC was desperate to find the space?
At any rate, TagScript.tagHolder is suspicious, and if it's even not the culprit itself (could be if WeakRefernce implementation was broken) it is hiding the real problem (at least in the environment of my test). Note that the variable is not static, which means all TagScript object instances get tied to the Thread. I don't know if the possibility of using each instance concurrently from multiple threads is useful at all, especially that the instances have more state information, not only the tagHolder... Anyway, it's a wrong forum to discuss that. I also heard that Jelly was abandoned by the original author so there is a good chance that noone understands why this tagHolder thingy was introduced and what it does...
Attachments
Activity
This problem has been fixed in jelly 1.0
http://issues.apache.org/jira/browse/JELLY-148
We have continually increased the -Xmx parameter until now where we have reached the limit on our system when we set it at 3500m. Our only alternative is to run the multiproject:site in smaller chunks of subprojects 
Spreadsheet with analysys of objects allocated vs objects alive at the time of VM termination.
Notice that there was no forced GC run before terminating, and GC was not pressed for memory - the test completed successfully.