Causes application to crash due to too many files being open

Environment:

Ubuntu 8.10
$ uname -a
Linux ... 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux

$ java -version
java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode)

Usage:

We are executing a Groovy script at regular intervals using the same instance of GroovyScriptEngine. The number of open files (as reported by the 'lsof' command) keeps increasing every time the script is executed until the number of files the user is allowed to have open is exhaused and a FileNotFoundException (Too many open files) is thrown.

The cause seems to be the URLConnections being opened by the updateCacheEntry method of GroovyScriptEngine.

In the following case, the script being executed is 'execute.script' and this script in turn executes 'request.script'.

lsof -p 7394
...
java 7394 kugan.baladevan 121r REG 8,2 3252 928217 /tmp/jar_cache19005.tmp (deleted)
java 7394 kugan.baladevan 122r REG 8,2 4234 5947467 /.../execute.script
java 7394 kugan.baladevan 123r REG 8,2 348 5947460 /.../request.script
java 7394 kugan.baladevan 124r REG 8,2 4234 5947467 /.../execute.script
java 7394 kugan.baladevan 125r REG 8,2 348 5947460 /.../request.script
java 7394 kugan.baladevan 126u REG 8,2 4234 5947467 /.../execute.script
java 7394 kugan.baladevan 127r REG 8,2 348 5947460 /.../request.script
java 7394 kugan.baladevan 128r REG 8,2 4234 5947467 /.../execute.script
java 7394 kugan.baladevan 129r REG 8,2 348 5947460 /.../request.script
java 7394 kugan.baladevan 130r REG 8,2 4234 5947467 /.../execute.script
java 7394 kugan.baladevan 131r REG 8,2 348 5947460 /.../request.script
...

Getting the InputStream of the URLConnection and closing it seems to fix this issue. The attached svn diff shows the changes.

The patch has been applied in trunk but will likely be rolled back as Jochen has a brand new version of GSE arriving on trunk shortly.
But if you like to live dangerously, grab a copy of the jar of one of the CI build servers now. I will put the patch on the 1_6_X branch, so it should make it into 1.6.4.

Patch applied to 1_6_X branch. Reverted from trunk pending newer version of GSE.

So the thinking was perhaps the new version of GSE that was reworked for 1.7 might make this not a problem or need to have a slightly different solution at least. I think Jochen was thinking that the stream would be closed elsewhere. But that was a while ago - I suspect given feedback that some issue does remain even with new GSE. I just haven't found time to properly investigate yet.

May be either this one or GROOVY-4218 should be closed. They both remain about this issue being present on the new GSE anyway.

I've committed some changes to close streams.
I hope it covers the various cases where we left streams opened.
A snapshot of Groovy 1.7.3 will be deployed automatically in a couple hours or so here:
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.7.3-SNAPSHOT/
(current snapshot dated 03-Jun-2010 04:55, the one with the fix should be older than that)

Kugan, could you please try it in your context and tell us if it improves the situation?
Thanks in advance.

I believe it is there now

Fix versions should be 1.6.9 / 1.7.3?

The initial fix committed by Paul in 1.6.x was there when 1.6.4 was released, if I'm not mistaken, hence why I've set the fix for 1.6.4 and 1.7.3.

Sorry, forgot the history part of it.

Please reopen if there are some further / missing files kept open.