Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.2, JRuby 1.3, JRuby 1.3.1
    • Fix Version/s: JRuby 1.7.0.RC1
    • Component/s: Compiler
    • Labels:
      None
    • Number of attachments :
      3

      Description

      I'm not sure if this is the same issue as http://jira.codehaus.org/browse/JRUBY-3699, which seems to have a similar error.

      Here's what I know so far:
      The offending file (attached) needs to be compiled AND jarred

      I will work on getting a spec in a few days to reproduce this.

        Activity

        Hide
        Charles Oliver Nutter added a comment -

        This is probably the inspector not properly inspecting some piece of code here that needs a scope. Definitely needs to be fixed. Any progress on that spec?

        Show
        Charles Oliver Nutter added a comment - This is probably the inspector not properly inspecting some piece of code here that needs a scope. Definitely needs to be fixed. Any progress on that spec?
        Hide
        Charles Oliver Nutter added a comment -

        I could not reproduce this, either by running it directly, with -X+C, as a .class, or as a .class inside a jar file.

        Show
        Charles Oliver Nutter added a comment - I could not reproduce this, either by running it directly, with -X+C, as a .class, or as a .class inside a jar file.
        Hide
        Logan Barnett added a comment -

        I've got it narrowed down, but determined that compiling and jarring platform.rb is needed, but there's a few other things needed as well I haven't found out yet.

        This is my test app:

        require 'lib/java/gemini.jar'   #gemini does not work
        require 'lib/java/platform.jar' #platform works
        require 'platform'
        puts Platform.using_osx?
        

        Here's the stack:

        DummyDynamicScope.java:49:in `getBackRef': java.lang.RuntimeException: DummyDynamicScope should never be used for backref storage
                from RubyRegexp.java:1392:in `updateBackRef'
                from RubyRegexp.java:1384:in `updateBackRef'
                from RubyRegexp.java:1374:in `search'
                from RubyRegexp.java:1289:in `op_match'
                from RuntimeHelpers.java:1217:in `match3'
                from platform.rb:20:in `method__4$RUBY$using_osx_p_'
                from platform#using_osx_p_:-1:in `call'
                from platform#using_osx_p_:-1:in `call'
                from WrapperMethod.java:54:in `call'
                from CachingCallSite.java:258:in `cacheAndCall'
                from CachingCallSite.java:77:in `call'
                from main.rb:4:in `__file__'
                from main.rb:-1:in `load'
                from Ruby.java:592:in `runScript'
                from Ruby.java:514:in `runNormally'
                from Ruby.java:360:in `runFromMain'
                from Main.java:268:in `run'
                from Main.java:113:in `run'
                from Main.java:97:in `main'
        

        I'm not sure if that helps or not. I'm going to start removing classes and such from the gemini.jar I've compiled. I'll provide it as an attachment if you're in a hurry to squash this.

        Thanks for the patience (:

        Show
        Logan Barnett added a comment - I've got it narrowed down, but determined that compiling and jarring platform.rb is needed, but there's a few other things needed as well I haven't found out yet. This is my test app: require 'lib/java/gemini.jar' #gemini does not work require 'lib/java/platform.jar' #platform works require 'platform' puts Platform.using_osx? Here's the stack: DummyDynamicScope.java:49:in `getBackRef': java.lang.RuntimeException: DummyDynamicScope should never be used for backref storage from RubyRegexp.java:1392:in `updateBackRef' from RubyRegexp.java:1384:in `updateBackRef' from RubyRegexp.java:1374:in `search' from RubyRegexp.java:1289:in `op_match' from RuntimeHelpers.java:1217:in `match3' from platform.rb:20:in `method__4$RUBY$using_osx_p_' from platform#using_osx_p_:-1:in `call' from platform#using_osx_p_:-1:in `call' from WrapperMethod.java:54:in `call' from CachingCallSite.java:258:in `cacheAndCall' from CachingCallSite.java:77:in `call' from main.rb:4:in `__file__' from main.rb:-1:in `load' from Ruby.java:592:in `runScript' from Ruby.java:514:in `runNormally' from Ruby.java:360:in `runFromMain' from Main.java:268:in `run' from Main.java:113:in `run' from Main.java:97:in `main' I'm not sure if that helps or not. I'm going to start removing classes and such from the gemini.jar I've compiled. I'll provide it as an attachment if you're in a hurry to squash this. Thanks for the patience (:
        Hide
        Logan Barnett added a comment -

        gemini.jar has platform.rb in it. Compiling and jarring platform.rb is not enough to produce the error. If gemini.jar is built without compiled files (just jarred .rb files), the error is also not produced.

        Show
        Logan Barnett added a comment - gemini.jar has platform.rb in it. Compiling and jarring platform.rb is not enough to produce the error. If gemini.jar is built without compiled files (just jarred .rb files), the error is also not produced.
        Hide
        Hiro Asari added a comment -

        Here's another data point that may help with troubleshooting. Right now on trunk, you get a similar error as follows;

        surfboard:jruby[git:master]$ bin/jruby --1.9 test/externals/ruby1.9/ruby/test_time.rb 
        Loaded suite test/externals/ruby1.9/ruby/test_time
        Started
        .EEFDummyDynamicScope.java:49:in `getBackRef': java.lang.RuntimeException: DummyDynamicScope should never be used for backref storage
        	from RubyRational.java:369:in `convertCommon'
        	from RubyRational.java:354:in `convert'
        	from org/jruby/RubyRational$s_method_multi$RUBYINVOKER$convert.gen:-1:in `call'
        	from RubyClass.java:584:in `finvoke'
        	from RuntimeHelpers.java:412:in `invoke'
        	from RubyKernel.java:346:in `new_rational'
        	from org/jruby/RubyKernel$s_method_multi$RUBYINVOKER$new_rational.gen:-1:in `call'
        	from CachingCallSite.java:330:in `cacheAndCall'
        	from CachingCallSite.java:189:in `call'
        	from test_time.rb:139:in `method__16$RUBY$test_at_rational'
        	from test_time#test_at_rational:-1:in `call'
        	from RubyClass.java:429:in `finvoke'
        	from RubyObject.java:1436:in `send'
        	from org/jruby/RubyObject$i_method_multi$RUBYINVOKER$send.gen:-1:in `call'
        	from JavaMethod.java:259:in `call'
        	from CachingCallSite.java:147:in `call'
        	from CallOneArgNode.java:57:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from RescueNode.java:225:in `executeBody'
        	from RescueNode.java:121:in `interpretWithoutJavaExceptions'
        	from RescueNode.java:112:in `interpret'
        	from EnsureNode.java:96:in `interpret'
        	from BeginNode.java:83:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from InterpretedMethod.java:173:in `call'
        	from DefaultMethod.java:169:in `call'
        	from CachingCallSite.java:147:in `call'
        	from CallOneArgNode.java:57:in `interpret'
        	from DAsgnNode.java:110:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from Interpreted19Block.java:186:in `evalBlockBody'
        	from Interpreted19Block.java:138:in `yield'
        	from Block.java:194:in `yield'
        	from RubyArray.java:1580:in `each'
        	from RubyArray.java:1587:in `each19'
        	from org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each19.gen:-1:in `call'
        	from CachingCallSite.java:116:in `callBlock'
        	from CachingCallSite.java:123:in `call'
        	from CallNoArgBlockNode.java:64:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from Interpreted19Block.java:186:in `evalBlockBody'
        	from Interpreted19Block.java:138:in `yield'
        	from Block.java:194:in `yield'
        	from RubyArray.java:1580:in `each'
        	from RubyArray.java:1587:in `each19'
        	from org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each19.gen:-1:in `call'
        	from CachingCallSite.java:300:in `cacheAndCall'
        	from CachingCallSite.java:118:in `callBlock'
        	from CachingCallSite.java:123:in `call'
        	from CallNoArgBlockNode.java:64:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from InterpretedMethod.java:173:in `call'
        	from DefaultMethod.java:169:in `call'
        	from CachingCallSite.java:310:in `cacheAndCall'
        	from CachingCallSite.java:149:in `call'
        	from FCallOneArgNode.java:36:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from InterpretedMethod.java:173:in `call'
        	from DefaultMethod.java:169:in `call'
        	from CachingCallSite.java:310:in `cacheAndCall'
        	from CachingCallSite.java:149:in `call'
        	from CallOneArgNode.java:57:in `interpret'
        	from DAsgnNode.java:110:in `interpret'
        	from NewlineNode.java:104:in `interpret'
        	from BlockNode.java:71:in `interpret'
        	from Interpreted19Block.java:186:in `evalBlockBody'
        	from Interpreted19Block.java:174:in `yield'
        	from Interpreted19Block.java:106:in `call'
        	from Block.java:89:in `call'
        	from RubyProc.java:221:in `call'
        	from RubyProc.java:204:in `call'
        	from Ruby.java:2611:in `tearDown'
        	from Main.java:274:in `run'
        	from Main.java:117:in `run'
        	from Main.java:97:in `main'
        
        Show
        Hiro Asari added a comment - Here's another data point that may help with troubleshooting. Right now on trunk, you get a similar error as follows; surfboard:jruby[git:master]$ bin/jruby --1.9 test/externals/ruby1.9/ruby/test_time.rb Loaded suite test/externals/ruby1.9/ruby/test_time Started .EEFDummyDynamicScope.java:49:in `getBackRef': java.lang.RuntimeException: DummyDynamicScope should never be used for backref storage from RubyRational.java:369:in `convertCommon' from RubyRational.java:354:in `convert' from org/jruby/RubyRational$s_method_multi$RUBYINVOKER$convert.gen:-1:in `call' from RubyClass.java:584:in `finvoke' from RuntimeHelpers.java:412:in `invoke' from RubyKernel.java:346:in `new_rational' from org/jruby/RubyKernel$s_method_multi$RUBYINVOKER$new_rational.gen:-1:in `call' from CachingCallSite.java:330:in `cacheAndCall' from CachingCallSite.java:189:in `call' from test_time.rb:139:in `method__16$RUBY$test_at_rational' from test_time#test_at_rational:-1:in `call' from RubyClass.java:429:in `finvoke' from RubyObject.java:1436:in `send' from org/jruby/RubyObject$i_method_multi$RUBYINVOKER$send.gen:-1:in `call' from JavaMethod.java:259:in `call' from CachingCallSite.java:147:in `call' from CallOneArgNode.java:57:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from RescueNode.java:225:in `executeBody' from RescueNode.java:121:in `interpretWithoutJavaExceptions' from RescueNode.java:112:in `interpret' from EnsureNode.java:96:in `interpret' from BeginNode.java:83:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from InterpretedMethod.java:173:in `call' from DefaultMethod.java:169:in `call' from CachingCallSite.java:147:in `call' from CallOneArgNode.java:57:in `interpret' from DAsgnNode.java:110:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from Interpreted19Block.java:186:in `evalBlockBody' from Interpreted19Block.java:138:in `yield' from Block.java:194:in `yield' from RubyArray.java:1580:in `each' from RubyArray.java:1587:in `each19' from org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each19.gen:-1:in `call' from CachingCallSite.java:116:in `callBlock' from CachingCallSite.java:123:in `call' from CallNoArgBlockNode.java:64:in `interpret' from NewlineNode.java:104:in `interpret' from Interpreted19Block.java:186:in `evalBlockBody' from Interpreted19Block.java:138:in `yield' from Block.java:194:in `yield' from RubyArray.java:1580:in `each' from RubyArray.java:1587:in `each19' from org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each19.gen:-1:in `call' from CachingCallSite.java:300:in `cacheAndCall' from CachingCallSite.java:118:in `callBlock' from CachingCallSite.java:123:in `call' from CallNoArgBlockNode.java:64:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from InterpretedMethod.java:173:in `call' from DefaultMethod.java:169:in `call' from CachingCallSite.java:310:in `cacheAndCall' from CachingCallSite.java:149:in `call' from FCallOneArgNode.java:36:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from InterpretedMethod.java:173:in `call' from DefaultMethod.java:169:in `call' from CachingCallSite.java:310:in `cacheAndCall' from CachingCallSite.java:149:in `call' from CallOneArgNode.java:57:in `interpret' from DAsgnNode.java:110:in `interpret' from NewlineNode.java:104:in `interpret' from BlockNode.java:71:in `interpret' from Interpreted19Block.java:186:in `evalBlockBody' from Interpreted19Block.java:174:in `yield' from Interpreted19Block.java:106:in `call' from Block.java:89:in `call' from RubyProc.java:221:in `call' from RubyProc.java:204:in `call' from Ruby.java:2611:in `tearDown' from Main.java:274:in `run' from Main.java:117:in `run' from Main.java:97:in `main'
        Hide
        Charles Oliver Nutter added a comment -

        1.9 compiler is still in pretty weak shape, waiting for more runs against the interpreter to confirm we've got the logic right. But I will make a mental note of this problem and mark it as 1.9 also.

        Show
        Charles Oliver Nutter added a comment - 1.9 compiler is still in pretty weak shape, waiting for more runs against the interpreter to confirm we've got the logic right. But I will make a mental note of this problem and mark it as 1.9 also.
        Hide
        Charles Oliver Nutter added a comment -

        Still having trouble reproducing this. Logan: Can you try to reconfirm and hunt one of us down on IRC to show us how to produce this?

        Show
        Charles Oliver Nutter added a comment - Still having trouble reproducing this. Logan: Can you try to reconfirm and hunt one of us down on IRC to show us how to produce this?
        Hide
        Logan Barnett added a comment -

        I'll make the time! When are you guys typically on nowadays? The 8 PM nights seem largely over now (:

        Show
        Logan Barnett added a comment - I'll make the time! When are you guys typically on nowadays? The 8 PM nights seem largely over now (:
        Hide
        Thomas E Enebo added a comment -

        1.9 mode does not current use the compiler right now and it appears this is happening for 1.8 code. So this is strictly a JIT issue.

        Show
        Thomas E Enebo added a comment - 1.9 mode does not current use the compiler right now and it appears this is happening for 1.8 code. So this is strictly a JIT issue.
        Hide
        Alex Coles added a comment -

        Attached file should reproduce this issue (at least, as far as I am experiencing it – I am assuming this is the same issue).

        I've had reports of people hitting this problem with DataMapper (and, specifically dm-validations, our validations plugin). I made a reduction from our dm-validations code, and got it down to a few lines: http://gist.github.com/234230

        Show
        Alex Coles added a comment - Attached file should reproduce this issue (at least, as far as I am experiencing it – I am assuming this is the same issue). I've had reports of people hitting this problem with DataMapper (and, specifically dm-validations, our validations plugin). I made a reduction from our dm-validations code, and got it down to a few lines: http://gist.github.com/234230
        Hide
        Alex Coles added a comment -

        And to confirm, if I run with JIT off (-X-C) or in 1.9 mode, then I do not see this issue using the reproduction I attached.

        Show
        Alex Coles added a comment - And to confirm, if I run with JIT off (-X-C) or in 1.9 mode, then I do not see this issue using the reproduction I attached.
        Hide
        Charles Oliver Nutter added a comment -

        I have committed in e6c7ee9 a change to make DummyDynamicScope only warn (in verbose mode) if backref is set/get'ed, rather than raising a Java-level RuntimeException. Since this generally will only affect bodies of code that are using send to call backref-aware methods, the silent swallowing should ideally be ok.

        Those of you that have seen this issue, please give things another go against JRuby master.

        Show
        Charles Oliver Nutter added a comment - I have committed in e6c7ee9 a change to make DummyDynamicScope only warn (in verbose mode) if backref is set/get'ed, rather than raising a Java-level RuntimeException. Since this generally will only affect bodies of code that are using send to call backref-aware methods, the silent swallowing should ideally be ok. Those of you that have seen this issue, please give things another go against JRuby master.
        Hide
        Charles Oliver Nutter added a comment -

        Alex: I didn't realize your reduction actually included a very simple version...I should have read it more carefully.

        Based on your reduction3, I have a trivial reduced case:

        ../jruby-1.4.0/bin/jruby -e "def foo(value, expected); Object; blah = value.send(:=~, expected); end; foo('foo', /foo/)"
        

        The problem here is that the compiler optimization to eliminate local scopes does not do so for "send". Normally, it only detects methods known to set or get $, like the literal = method itself. When you call those methods via send, it does not see them, and therefore does not prepare a scope for them. It does not always fire because often the calling method does have a scope, and the backref ends up getting set there.

        The constant lookup here is what triggers the error. A method that does not appear to set or get $~ but does access a constant gets a DummyDynamicScope just for constant-scoping purposes. This results in the method having a scope, but a scope that refuses to accept $~, so it errors out.

        This is currently hard to fix without also causing a deoptimization for any methods doing .send, which is very heavily used for metaprogrammed code. I do have, in theory, a better way to handle backref logic; but I have not had spare cycles to try to put it in place.

        The temporary fix is not great, because it masks two bugs: the first is the fact that we're not properly preparing for $~ when you use send, and the second is that we're potentially overwriting a previous call's $~ when no constant is present to trigger the error.

        Not sure how to deal with this at the moment. $~ sucks.

        Show
        Charles Oliver Nutter added a comment - Alex: I didn't realize your reduction actually included a very simple version...I should have read it more carefully. Based on your reduction3, I have a trivial reduced case: ../jruby-1.4.0/bin/jruby -e "def foo(value, expected); Object; blah = value.send(:=~, expected); end; foo('foo', /foo/)" The problem here is that the compiler optimization to eliminate local scopes does not do so for "send". Normally, it only detects methods known to set or get $ , like the literal = method itself. When you call those methods via send, it does not see them, and therefore does not prepare a scope for them. It does not always fire because often the calling method does have a scope, and the backref ends up getting set there. The constant lookup here is what triggers the error. A method that does not appear to set or get $~ but does access a constant gets a DummyDynamicScope just for constant-scoping purposes. This results in the method having a scope, but a scope that refuses to accept $~, so it errors out. This is currently hard to fix without also causing a deoptimization for any methods doing .send, which is very heavily used for metaprogrammed code. I do have, in theory, a better way to handle backref logic; but I have not had spare cycles to try to put it in place. The temporary fix is not great, because it masks two bugs: the first is the fact that we're not properly preparing for $~ when you use send, and the second is that we're potentially overwriting a previous call's $~ when no constant is present to trigger the error. Not sure how to deal with this at the moment. $~ sucks.
        Hide
        Charles Oliver Nutter added a comment -

        I am downgrading this to "major" and making the warning only appear in debug mode, since it isn't something the user can fix and won't affect 99% of users. There's a small chance that code which sends a backref or lastline-mutating method could overwrite a previous call's backref or lastline, and that still needs to be remedied. But I'm more and more comfortable saying backref and lastline should not be expected to work across things like evals and sends, since it requires a level of stack manipulation we can't really do.

        The fix for this will likely require a rework of how we handle backref. It needs to be local to the containing method scope (mostly) but mutable by any calls that happen within the body. My current strategy for this would be to have specific named calls (like =~) use one of two ways to set/get backref/lastline:

        • Via a thread-local slot that's used only for the duration of the call. It would be populated with the current backref local variable before the call and used to set that local var after the call completes, avoiding the requirement for a separate heap structure.
        • Via an additional "out" parameter passed on the call stack. In this case, an IRubyObject-implementing carrier object would be prepared for backref-utilizing methods and passed to all calls likely to use backref. Calls that don't use it would simply ignore the extra value.

        At any rate, this is not something easily fixable in 1.5, and the impact to users is extremely low and unlikely to happen.

        Show
        Charles Oliver Nutter added a comment - I am downgrading this to "major" and making the warning only appear in debug mode, since it isn't something the user can fix and won't affect 99% of users. There's a small chance that code which sends a backref or lastline-mutating method could overwrite a previous call's backref or lastline, and that still needs to be remedied. But I'm more and more comfortable saying backref and lastline should not be expected to work across things like evals and sends, since it requires a level of stack manipulation we can't really do. The fix for this will likely require a rework of how we handle backref. It needs to be local to the containing method scope (mostly) but mutable by any calls that happen within the body. My current strategy for this would be to have specific named calls (like =~) use one of two ways to set/get backref/lastline: Via a thread-local slot that's used only for the duration of the call. It would be populated with the current backref local variable before the call and used to set that local var after the call completes, avoiding the requirement for a separate heap structure. Via an additional "out" parameter passed on the call stack. In this case, an IRubyObject-implementing carrier object would be prepared for backref-utilizing methods and passed to all calls likely to use backref. Calls that don't use it would simply ignore the extra value. At any rate, this is not something easily fixable in 1.5, and the impact to users is extremely low and unlikely to happen.
        Hide
        Alex K added a comment -

        I upgraded to 1.9 and now I got this new Warning message
        ":1 warning: DummyDynamicScope should never be used for backref storage"
        a lot. I assume that its somewhere in my code, but how can I now find the reason(s)?

        Show
        Alex K added a comment - I upgraded to 1.9 and now I got this new Warning message ":1 warning: DummyDynamicScope should never be used for backref storage" a lot. I assume that its somewhere in my code, but how can I now find the reason(s)?
        Hide
        Charles Oliver Nutter added a comment -

        Alex: If you are getting this "a lot" then I want to look into it. It indicates that some code is over-optimizing somewhere.

        I'll dig up this warning and see if I can come up with a way to investigate it.

        Show
        Charles Oliver Nutter added a comment - Alex: If you are getting this "a lot" then I want to look into it. It indicates that some code is over-optimizing somewhere. I'll dig up this warning and see if I can come up with a way to investigate it.
        Hide
        Charles Oliver Nutter added a comment -

        I'm going to call this fixed.

        There were recently some key fixes to the compiler (the inspector, really) that ensured scopes would be available for backref methods in more cases. This bug is also very old, and the compiler has been patched numerous times since then for similar issues.

        If this exact issue still exists with a simple reproduction, feel free to reopen.

        Show
        Charles Oliver Nutter added a comment - I'm going to call this fixed. There were recently some key fixes to the compiler (the inspector, really) that ensured scopes would be available for backref methods in more cases. This bug is also very old, and the compiler has been patched numerous times since then for similar issues. If this exact issue still exists with a simple reproduction, feel free to reopen.

          People

          • Assignee:
            Charles Oliver Nutter
            Reporter:
            Logan Barnett
          • Votes:
            5 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: