History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: JRUBY-2621
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Charles Oliver Nutter
Reporter: Charles Oliver Nutter
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
JRuby

Something in ruby_parser is too long to compile, but doesn't get broken apart correctly

Created: 08/Jun/08 12:21 PM   Updated: 16/Jul/08 03:00 AM
Component/s: Compiler
Affects Version/s: None
Fix Version/s: JRuby 1.1+

Time Tracking:
Not Specified


 Description  « Hide
This will obviously affect performance of ruby_parser quite a bit:
~/NetBeansProjects/jruby ➔ jruby -J-Djruby.jit.logging.verbose=true -X+C --server -rubygems -rbenchmark -e "gem :ruby_parser; require 'ruby_parser'; 5.times {puts Benchmark.measure {RubyParser.new.parse(File.read('lib/ruby/1.8/rdoc/parsers/parse_rb.rb'))}}"
could not compile: /Users/headius/NetBeansProjects/jruby/lib/ruby/gems/1.8/gems/ruby_parser-1.0.0/lib/ruby_parser.rb because of: "Invalid method Code length 82165 in class file ruby/Users/headius/NetBeansProjects/jruby/lib/ruby/gems/$1_dot_8/gems/ruby_parser_minus_1_dot_0_dot_0/lib/ruby_parser"
java.lang.ClassFormatError: Invalid method Code length 82165 in class file ruby/Users/headius/NetBeansProjects/jruby/lib/ruby/gems/$1_dot_8/gems/ruby_parser_minus_1_dot_0_dot_0/lib/ruby_parser
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:675)
	at org.jruby.util.JRubyClassLoader.defineClass(JRubyClassLoader.java:21)
	at org.jruby.compiler.impl.StandardASMCompiler.loadClass(StandardASMCompiler.java:159)
	at org.jruby.Ruby.tryCompile(Ruby.java:464)
	at org.jruby.Ruby.compileAndLoadFile(Ruby.java:2041)
	at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:56)
	at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:320)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:346)
	at org.jruby.RubyKernel.require(RubyKernel.java:769)


 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Charles Oliver Nutter - 04/Jul/08 08:27 AM
I discovered what the problem was with ruby_parser...it's the main script. It initializes a number of seriously gigantic arrays for the Ruby parser productions. I think there are several of these arrays that would fail to compile individually, much less all of them as the body of a method. And because they're gigantic, they can't easily be split into multiple methods.

This should generally affect only +C compilation since it's in the root of the script. But it's something to look out for in the future. I'll leave this open to try to explore any way possible to compile this script. Toughy though.


Charles Oliver Nutter - 04/Jul/08 11:18 AM
Some experimentation helped me find out that the upper bound of a literal int array is somewhere around 4700 elements. Then it crosses the 65k limit for an individual method.

The giant arrays in question are generated by RACC, so there's little change to get them changed. Instead we should explore the possibility of having arrays over a particular size force splitting by compiling as their own array-populating methods. I'm not going to do this right now, but it's something we'll want to look into in the future for cases like the Ruby parser.


Charles Oliver Nutter - 16/Jul/08 03:00 AM
This is a nontrivial bit of compiler twiddling for very little gain, so I'm punting it to post 1.1.3.