JRuby

ERB parsing in JRuby 1.6.6

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.6.6
  • Fix Version/s: JRuby 1.7.0.RC1
  • Component/s: Parser
  • Labels:
    None
  • Environment:
    Windows 7, Linux, Jruby 1.6.6, Rails 3.0.9
  • Number of attachments :
    1

Description

We are seeing an issue with the ERB parsing in Jruby when running in a multi-threaded mode. When too many threads are accessing a line of code which reads and parses a file, typically a YML file, an exception is thrown. This can be reproduced by running the following test in a rails console:

threads = []
100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(::Devise.ldap_config || "#

{Rails.root}

/config/application.yml")).result)[Rails.env]}}

Stack trace:
Exception in thread "RubyThread-1: (irb):2" java.lang.IndexOutOfBoundsException
at java.nio.ByteBuffer.wrap(ByteBuffer.java:352)
at org.jruby.RubyEncoding.decode(RubyEncoding.java:198)
at org.jruby.RubyString.decodeString(RubyString.java:725)
at org.jruby.RubyString.toString(RubyString.java:701)
at org.jruby.RubyString.asJavaString(RubyString.java:1228)
at org.jruby.ext.psych.PsychParser.parse(PsychParser.java:113)
at org.jruby.ext.psych.PsychParser$i$1$0$parse.call(PsychParser$i$1$0$parse.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:75)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:212)
at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:200)
at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:131)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:270)
at org.jruby.RubyProc.call(RubyProc.java:224)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:95)
at java.lang.Thread.run(Thread.java:662)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

Could you please:

  • Re-test against a more recent JRuby, like 1.7.0pre1 or master.
  • If it still fails, provide a YAML file and reproduction that does not depend on Rails.

I suspect this is fixed already, since we've reworked how RubyEncoding.decode works since 1.6.6, but please confirm.

Show
Charles Oliver Nutter added a comment - Could you please: Re-test against a more recent JRuby, like 1.7.0pre1 or master. If it still fails, provide a YAML file and reproduction that does not depend on Rails. I suspect this is fixed already, since we've reworked how RubyEncoding.decode works since 1.6.6, but please confirm.
Hide
Permalink
Justin Hart added a comment - - edited

Hi Charles,

I retested with the jruby 1.7.0.preview1 and still got the exceptions and stack trace. I opened up a jirb console from windows7 (not a rails console) and ran the following code:

threads = []
app_file = "C:/Ruby192/test/application.yml"
100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(app_file)).result)["development"]}}

I have attached the YAML file that this snippet is calling.

Show
Justin Hart added a comment - - edited Hi Charles, I retested with the jruby 1.7.0.preview1 and still got the exceptions and stack trace. I opened up a jirb console from windows7 (not a rails console) and ran the following code: threads = [] app_file = "C:/Ruby192/test/application.yml" 100.times {threads << Thread.new {p YAML.load(ERB.new(File.read(app_file)).result) ["development"] }} I have attached the YAML file that this snippet is calling.
Hide
Permalink
Charles Oliver Nutter added a comment -

Thank you! Will investigate.

Show
Charles Oliver Nutter added a comment - Thank you! Will investigate.
Hide
Permalink
Charles Oliver Nutter added a comment -

Appears to be fixed on master. Lots of encoding/decoding fixes there.

Show
Charles Oliver Nutter added a comment - Appears to be fixed on master. Lots of encoding/decoding fixes there.
Hide
Permalink
Justin Hart added a comment -

Sounds good, thanks!

Show
Justin Hart added a comment - Sounds good, thanks!
Hide
Permalink
Charles Oliver Nutter added a comment -

Ok, this actually came back around as a different bug.

The real issue, which is NOT fixed on master, is that by default ERb uses TOPLEVEL_BINDING for executing the template's code and _erbout as the variable in that binding that holds the eventual String output, meaning that multiple threads can easily step on each other.

We're going to need to raise this as an MRI issue and probably patch it in some way ourselves.

Show
Charles Oliver Nutter added a comment - Ok, this actually came back around as a different bug. The real issue, which is NOT fixed on master, is that by default ERb uses TOPLEVEL_BINDING for executing the template's code and _erbout as the variable in that binding that holds the eventual String output, meaning that multiple threads can easily step on each other. We're going to need to raise this as an MRI issue and probably patch it in some way ourselves.
Hide
Permalink
Justin Hart added a comment -

Sounds good. Let me know if I can help further!

Show
Justin Hart added a comment - Sounds good. Let me know if I can help further!
Hide
Permalink
Charles Oliver Nutter added a comment -

I have filed this issue with MRI folks, and I think my patch is pretty safe: https://bugs.ruby-lang.org/issues/7046

I'll talk to Tom about whether we should attempt to include this in RC1.

The workaround in the meantime is to provide your own, isolated binding to ERB#result.

Show
Charles Oliver Nutter added a comment - I have filed this issue with MRI folks, and I think my patch is pretty safe: https://bugs.ruby-lang.org/issues/7046 I'll talk to Tom about whether we should attempt to include this in RC1. The workaround in the meantime is to provide your own, isolated binding to ERB#result.
Hide
Permalink
Charles Oliver Nutter added a comment - 
commit c49e81a3e803abfcbe71609706b71ce4eb6a6024
Author: Charles Oliver Nutter <headius@headius.com>
Date:   Fri Sep 21 18:34:26 2012 -0500

    Fix JRUBY-6773
    
    This is actually a bug in stdlib. ERB#run and ERB#result both use
    TOPLEVEL_BINDING to evaluate the template, and since by default
    the template is appending to a String in _erbout, concurrent
    template evaluations could step on each other.
    
    My fix has been submitted to MRI in the following bug:
    
    https://bugs.ruby-lang.org/issues/7046
    
    The patch has not been accepted, but in the interest of fixing
    this for JRuby users we are going ahead with it in JRuby 1.7RC1.

:100644 100644 1ed3c60... c08010e... M	lib/ruby/1.8/erb.rb
:100644 100644 bb47943... 53f6d16... M	lib/ruby/1.9/erb.rb
Show
Charles Oliver Nutter added a comment - commit c49e81a3e803abfcbe71609706b71ce4eb6a6024 Author: Charles Oliver Nutter <headius@headius.com> Date: Fri Sep 21 18:34:26 2012 -0500 Fix JRUBY-6773 This is actually a bug in stdlib. ERB#run and ERB#result both use TOPLEVEL_BINDING to evaluate the template, and since by default the template is appending to a String in _erbout, concurrent template evaluations could step on each other. My fix has been submitted to MRI in the following bug: https://bugs.ruby-lang.org/issues/7046 The patch has not been accepted, but in the interest of fixing this for JRuby users we are going ahead with it in JRuby 1.7RC1. :100644 100644 1ed3c60... c08010e... M lib/ruby/1.8/erb.rb :100644 100644 bb47943... 53f6d16... M lib/ruby/1.9/erb.rb

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Justin Hart
Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: