Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.6.6
    • Fix Version/s: JRuby 1.7.0.RC1
    • Component/s: Parser
    • Labels:
      None
    • Environment:
      Windows 7, Linux, Jruby 1.6.6, Rails 3.0.9
    • Number of attachments :
      1

      Description

      We are seeing an issue with the ERB parsing in Jruby when running in a multi-threaded mode. When too many threads are accessing a line of code which reads and parses a file, typically a YML file, an exception is thrown. This can be reproduced by running the following test in a rails console:

      threads = []
      100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(::Devise.ldap_config || "#

      {Rails.root}

      /config/application.yml")).result)[Rails.env]}}

      Stack trace:
      Exception in thread "RubyThread-1: (irb):2" java.lang.IndexOutOfBoundsException
      at java.nio.ByteBuffer.wrap(ByteBuffer.java:352)
      at org.jruby.RubyEncoding.decode(RubyEncoding.java:198)
      at org.jruby.RubyString.decodeString(RubyString.java:725)
      at org.jruby.RubyString.toString(RubyString.java:701)
      at org.jruby.RubyString.asJavaString(RubyString.java:1228)
      at org.jruby.ext.psych.PsychParser.parse(PsychParser.java:113)
      at org.jruby.ext.psych.PsychParser$i$1$0$parse.call(PsychParser$i$1$0$parse.gen:65535)
      at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
      at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
      at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
      at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
      at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
      at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
      at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
      at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
      at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
      at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
      at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
      at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63)
      at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
      at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
      at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
      at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
      at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
      at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
      at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
      at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
      at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
      at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
      at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
      at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
      at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
      at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
      at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
      at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
      at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
      at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
      at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
      at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
      at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
      at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
      at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:212)
      at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:200)
      at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:131)
      at org.jruby.runtime.Block.call(Block.java:89)
      at org.jruby.RubyProc.call(RubyProc.java:270)
      at org.jruby.RubyProc.call(RubyProc.java:224)
      at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:95)
      at java.lang.Thread.run(Thread.java:662)

        Activity

        Hide
        Charles Oliver Nutter added a comment -

        Could you please:

        • Re-test against a more recent JRuby, like 1.7.0pre1 or master.
        • If it still fails, provide a YAML file and reproduction that does not depend on Rails.

        I suspect this is fixed already, since we've reworked how RubyEncoding.decode works since 1.6.6, but please confirm.

        Show
        Charles Oliver Nutter added a comment - Could you please: Re-test against a more recent JRuby, like 1.7.0pre1 or master. If it still fails, provide a YAML file and reproduction that does not depend on Rails. I suspect this is fixed already, since we've reworked how RubyEncoding.decode works since 1.6.6, but please confirm.
        Hide
        Justin Hart added a comment - - edited

        Hi Charles,

        I retested with the jruby 1.7.0.preview1 and still got the exceptions and stack trace. I opened up a jirb console from windows7 (not a rails console) and ran the following code:

        threads = []
        app_file = "C:/Ruby192/test/application.yml"
        100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(app_file)).result)["development"]}}

        I have attached the YAML file that this snippet is calling.

        Show
        Justin Hart added a comment - - edited Hi Charles, I retested with the jruby 1.7.0.preview1 and still got the exceptions and stack trace. I opened up a jirb console from windows7 (not a rails console) and ran the following code: threads = [] app_file = "C:/Ruby192/test/application.yml" 100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(app_file)).result) ["development"] }} I have attached the YAML file that this snippet is calling.
        Hide
        Charles Oliver Nutter added a comment -

        Thank you! Will investigate.

        Show
        Charles Oliver Nutter added a comment - Thank you! Will investigate.
        Hide
        Charles Oliver Nutter added a comment -

        Appears to be fixed on master. Lots of encoding/decoding fixes there.

        Show
        Charles Oliver Nutter added a comment - Appears to be fixed on master. Lots of encoding/decoding fixes there.
        Hide
        Justin Hart added a comment -

        Sounds good, thanks!

        Show
        Justin Hart added a comment - Sounds good, thanks!
        Hide
        Charles Oliver Nutter added a comment -

        Ok, this actually came back around as a different bug.

        The real issue, which is NOT fixed on master, is that by default ERb uses TOPLEVEL_BINDING for executing the template's code and _erbout as the variable in that binding that holds the eventual String output, meaning that multiple threads can easily step on each other.

        We're going to need to raise this as an MRI issue and probably patch it in some way ourselves.

        Show
        Charles Oliver Nutter added a comment - Ok, this actually came back around as a different bug. The real issue, which is NOT fixed on master, is that by default ERb uses TOPLEVEL_BINDING for executing the template's code and _erbout as the variable in that binding that holds the eventual String output, meaning that multiple threads can easily step on each other. We're going to need to raise this as an MRI issue and probably patch it in some way ourselves.
        Hide
        Justin Hart added a comment -

        Sounds good. Let me know if I can help further!

        Show
        Justin Hart added a comment - Sounds good. Let me know if I can help further!
        Hide
        Charles Oliver Nutter added a comment -

        I have filed this issue with MRI folks, and I think my patch is pretty safe: https://bugs.ruby-lang.org/issues/7046

        I'll talk to Tom about whether we should attempt to include this in RC1.

        The workaround in the meantime is to provide your own, isolated binding to ERB#result.

        Show
        Charles Oliver Nutter added a comment - I have filed this issue with MRI folks, and I think my patch is pretty safe: https://bugs.ruby-lang.org/issues/7046 I'll talk to Tom about whether we should attempt to include this in RC1. The workaround in the meantime is to provide your own, isolated binding to ERB#result.
        Hide
        Charles Oliver Nutter added a comment -
        commit c49e81a3e803abfcbe71609706b71ce4eb6a6024
        Author: Charles Oliver Nutter <headius@headius.com>
        Date:   Fri Sep 21 18:34:26 2012 -0500
        
            Fix JRUBY-6773
            
            This is actually a bug in stdlib. ERB#run and ERB#result both use
            TOPLEVEL_BINDING to evaluate the template, and since by default
            the template is appending to a String in _erbout, concurrent
            template evaluations could step on each other.
            
            My fix has been submitted to MRI in the following bug:
            
            https://bugs.ruby-lang.org/issues/7046
            
            The patch has not been accepted, but in the interest of fixing
            this for JRuby users we are going ahead with it in JRuby 1.7RC1.
        
        :100644 100644 1ed3c60... c08010e... M	lib/ruby/1.8/erb.rb
        :100644 100644 bb47943... 53f6d16... M	lib/ruby/1.9/erb.rb
        
        Show
        Charles Oliver Nutter added a comment - commit c49e81a3e803abfcbe71609706b71ce4eb6a6024 Author: Charles Oliver Nutter <headius@headius.com> Date: Fri Sep 21 18:34:26 2012 -0500 Fix JRUBY-6773 This is actually a bug in stdlib. ERB#run and ERB#result both use TOPLEVEL_BINDING to evaluate the template, and since by default the template is appending to a String in _erbout, concurrent template evaluations could step on each other. My fix has been submitted to MRI in the following bug: https://bugs.ruby-lang.org/issues/7046 The patch has not been accepted, but in the interest of fixing this for JRuby users we are going ahead with it in JRuby 1.7RC1. :100644 100644 1ed3c60... c08010e... M lib/ruby/1.8/erb.rb :100644 100644 bb47943... 53f6d16... M lib/ruby/1.9/erb.rb

          People

          • Assignee:
            Charles Oliver Nutter
            Reporter:
            Justin Hart
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: