groovy
  1. groovy
  2. GROOVY-3570

Causes application to crash due to too many files being open

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.6.3
    • Fix Version/s: 1.6.4, 1.7.3, 1.8.0
    • Component/s: None
    • Labels:
      None
    • Environment:
      Ubuntu 8.10
    • Patch Submitted:
      Yes
    • Number of attachments :
      1

      Issue Links

        Activity

        Hide
        Kugan Baladevan added a comment -

        Environment:

        Ubuntu 8.10
        $ uname -a
        Linux ... 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux

        $ java -version
        java version "1.6.0_10"
        Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
        Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode)

        Usage:

        We are executing a Groovy script at regular intervals using the same instance of GroovyScriptEngine. The number of open files (as reported by the 'lsof' command) keeps increasing every time the script is executed until the number of files the user is allowed to have open is exhaused and a FileNotFoundException (Too many open files) is thrown.

        The cause seems to be the URLConnections being opened by the updateCacheEntry method of GroovyScriptEngine.

        In the following case, the script being executed is 'execute.script' and this script in turn executes 'request.script'.

        lsof -p 7394
        ...
        java 7394 kugan.baladevan 121r REG 8,2 3252 928217 /tmp/jar_cache19005.tmp (deleted)
        java 7394 kugan.baladevan 122r REG 8,2 4234 5947467 /.../execute.script
        java 7394 kugan.baladevan 123r REG 8,2 348 5947460 /.../request.script
        java 7394 kugan.baladevan 124r REG 8,2 4234 5947467 /.../execute.script
        java 7394 kugan.baladevan 125r REG 8,2 348 5947460 /.../request.script
        java 7394 kugan.baladevan 126u REG 8,2 4234 5947467 /.../execute.script
        java 7394 kugan.baladevan 127r REG 8,2 348 5947460 /.../request.script
        java 7394 kugan.baladevan 128r REG 8,2 4234 5947467 /.../execute.script
        java 7394 kugan.baladevan 129r REG 8,2 348 5947460 /.../request.script
        java 7394 kugan.baladevan 130r REG 8,2 4234 5947467 /.../execute.script
        java 7394 kugan.baladevan 131r REG 8,2 348 5947460 /.../request.script
        ...

        Getting the InputStream of the URLConnection and closing it seems to fix this issue. The attached svn diff shows the changes.

        Show
        Kugan Baladevan added a comment - Environment: Ubuntu 8.10 $ uname -a Linux ... 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux $ java -version java version "1.6.0_10" Java(TM) SE Runtime Environment (build 1.6.0_10-b33) Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode) Usage: We are executing a Groovy script at regular intervals using the same instance of GroovyScriptEngine. The number of open files (as reported by the 'lsof' command) keeps increasing every time the script is executed until the number of files the user is allowed to have open is exhaused and a FileNotFoundException (Too many open files) is thrown. The cause seems to be the URLConnections being opened by the updateCacheEntry method of GroovyScriptEngine. In the following case, the script being executed is 'execute.script' and this script in turn executes 'request.script'. lsof -p 7394 ... java 7394 kugan.baladevan 121r REG 8,2 3252 928217 /tmp/jar_cache19005.tmp (deleted) java 7394 kugan.baladevan 122r REG 8,2 4234 5947467 /.../execute.script java 7394 kugan.baladevan 123r REG 8,2 348 5947460 /.../request.script java 7394 kugan.baladevan 124r REG 8,2 4234 5947467 /.../execute.script java 7394 kugan.baladevan 125r REG 8,2 348 5947460 /.../request.script java 7394 kugan.baladevan 126u REG 8,2 4234 5947467 /.../execute.script java 7394 kugan.baladevan 127r REG 8,2 348 5947460 /.../request.script java 7394 kugan.baladevan 128r REG 8,2 4234 5947467 /.../execute.script java 7394 kugan.baladevan 129r REG 8,2 348 5947460 /.../request.script java 7394 kugan.baladevan 130r REG 8,2 4234 5947467 /.../execute.script java 7394 kugan.baladevan 131r REG 8,2 348 5947460 /.../request.script ... Getting the InputStream of the URLConnection and closing it seems to fix this issue. The attached svn diff shows the changes.
        Hide
        Paul King added a comment -

        The patch has been applied in trunk but will likely be rolled back as Jochen has a brand new version of GSE arriving on trunk shortly.
        But if you like to live dangerously, grab a copy of the jar of one of the CI build servers now. I will put the patch on the 1_6_X branch, so it should make it into 1.6.4.

        Show
        Paul King added a comment - The patch has been applied in trunk but will likely be rolled back as Jochen has a brand new version of GSE arriving on trunk shortly. But if you like to live dangerously, grab a copy of the jar of one of the CI build servers now. I will put the patch on the 1_6_X branch, so it should make it into 1.6.4.
        Hide
        Paul King added a comment -

        Patch applied to 1_6_X branch. Reverted from trunk pending newer version of GSE.

        Show
        Paul King added a comment - Patch applied to 1_6_X branch. Reverted from trunk pending newer version of GSE.
        Hide
        Roshan Dawrani added a comment -

        Paul, any plan to port the fix to 1.7.x + as well? (I see your comment from that time that says "revert patch for time being - awaiting new version of this file")

        Show
        Roshan Dawrani added a comment - Paul, any plan to port the fix to 1.7.x + as well? (I see your comment from that time that says "revert patch for time being - awaiting new version of this file")
        Roshan Dawrani made changes -
        Field Original Value New Value
        Link This issue relates to GROOVY-4218 [ GROOVY-4218 ]
        Hide
        Paul King added a comment -

        So the thinking was perhaps the new version of GSE that was reworked for 1.7 might make this not a problem or need to have a slightly different solution at least. I think Jochen was thinking that the stream would be closed elsewhere. But that was a while ago - I suspect given feedback that some issue does remain even with new GSE. I just haven't found time to properly investigate yet.

        Show
        Paul King added a comment - So the thinking was perhaps the new version of GSE that was reworked for 1.7 might make this not a problem or need to have a slightly different solution at least. I think Jochen was thinking that the stream would be closed elsewhere. But that was a while ago - I suspect given feedback that some issue does remain even with new GSE. I just haven't found time to properly investigate yet.
        Hide
        Roshan Dawrani added a comment -

        May be either this one or GROOVY-4218 should be closed. They both remain about this issue being present on the new GSE anyway.

        Show
        Roshan Dawrani added a comment - May be either this one or GROOVY-4218 should be closed. They both remain about this issue being present on the new GSE anyway.
        Hide
        Guillaume Laforge added a comment -

        I've committed some changes to close streams.
        I hope it covers the various cases where we left streams opened.
        A snapshot of Groovy 1.7.3 will be deployed automatically in a couple hours or so here:
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.7.3-SNAPSHOT/
        (current snapshot dated 03-Jun-2010 04:55, the one with the fix should be older than that)

        Kugan, could you please try it in your context and tell us if it improves the situation?
        Thanks in advance.

        Show
        Guillaume Laforge added a comment - I've committed some changes to close streams. I hope it covers the various cases where we left streams opened. A snapshot of Groovy 1.7.3 will be deployed automatically in a couple hours or so here: http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.7.3-SNAPSHOT/ (current snapshot dated 03-Jun-2010 04:55, the one with the fix should be older than that) Kugan, could you please try it in your context and tell us if it improves the situation? Thanks in advance.
        Hide
        Paul King added a comment -

        I believe it is there now

        Show
        Paul King added a comment - I believe it is there now
        Guillaume Laforge made changes -
        Fix Version/s 1.7.3 [ 16356 ]
        Fix Version/s 1.6.4 [ 15291 ]
        Hide
        Roshan Dawrani added a comment -

        Fix versions should be 1.6.9 / 1.7.3?

        Show
        Roshan Dawrani added a comment - Fix versions should be 1.6.9 / 1.7.3?
        Hide
        Guillaume Laforge added a comment -

        The initial fix committed by Paul in 1.6.x was there when 1.6.4 was released, if I'm not mistaken, hence why I've set the fix for 1.6.4 and 1.7.3.

        Show
        Guillaume Laforge added a comment - The initial fix committed by Paul in 1.6.x was there when 1.6.4 was released, if I'm not mistaken, hence why I've set the fix for 1.6.4 and 1.7.3.
        Hide
        Roshan Dawrani added a comment -

        Sorry, forgot the history part of it.

        Show
        Roshan Dawrani added a comment - Sorry, forgot the history part of it.
        Hide
        Guillaume Laforge added a comment -

        Please reopen if there are some further / missing files kept open.

        Show
        Guillaume Laforge added a comment - Please reopen if there are some further / missing files kept open.
        Guillaume Laforge made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Guillaume Laforge [ guillaume ]
        Fix Version/s 1.8-beta-x [ 15750 ]
        Resolution Fixed [ 1 ]
        Paul King made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Paul King made changes -
        Resolution Fixed [ 1 ]
        Status Closed [ 6 ] Reopened [ 4 ]
        Paul King made changes -
        Fix Version/s 1.8.0 [ 17222 ]
        Fix Version/s 1.8.x [ 15750 ]
        Paul King made changes -
        Status Reopened [ 4 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Guillaume Laforge
            Reporter:
            Kugan Baladevan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: