[#JRUBY-2621] Something in ruby_parser is too long to compile, but doesn't get broken apart correctly

JRuby

Something in ruby_parser is too long to compile, but doesn't get broken apart correctly

Created: 08/Jun/08 12:21 PM Updated: 16/Jul/08 03:00 AM

Component/s:

Compiler

Affects Version/s:

None

Fix Version/s:

JRuby 1.1+

Time Tracking:

Not Specified

Description

« Hide

This will obviously affect performance of ruby_parser quite a bit:

~/NetBeansProjects/jruby &#10132; jruby -J-Djruby.jit.logging.verbose=true -X+C --server -rubygems -rbenchmark -e "gem :ruby_parser; require 'ruby_parser'; 5.times {puts Benchmark.measure {RubyParser.new.parse(File.read('lib/ruby/1.8/rdoc/parsers/parse_rb.rb'))}}"
could not compile: /Users/headius/NetBeansProjects/jruby/lib/ruby/gems/1.8/gems/ruby_parser-1.0.0/lib/ruby_parser.rb because of: "Invalid method Code length 82165 in class file ruby/Users/headius/NetBeansProjects/jruby/lib/ruby/gems/$1_dot_8/gems/ruby_parser_minus_1_dot_0_dot_0/lib/ruby_parser"
java.lang.ClassFormatError: Invalid method Code length 82165 in class file ruby/Users/headius/NetBeansProjects/jruby/lib/ruby/gems/$1_dot_8/gems/ruby_parser_minus_1_dot_0_dot_0/lib/ruby_parser
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:675)
	at org.jruby.util.JRubyClassLoader.defineClass(JRubyClassLoader.java:21)
	at org.jruby.compiler.impl.StandardASMCompiler.loadClass(StandardASMCompiler.java:159)
	at org.jruby.Ruby.tryCompile(Ruby.java:464)
	at org.jruby.Ruby.compileAndLoadFile(Ruby.java:2041)
	at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:56)
	at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:320)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:346)
	at org.jruby.RubyKernel.require(RubyKernel.java:769)

All

Comments

Work Log

Change History

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Charles Oliver Nutter - 04/Jul/08 08:27 AM

I discovered what the problem was with ruby_parser...it's the main script. It initializes a number of seriously gigantic arrays for the Ruby parser productions. I think there are several of these arrays that would fail to compile individually, much less all of them as the body of a method. And because they're gigantic, they can't easily be split into multiple methods.

This should generally affect only +C compilation since it's in the root of the script. But it's something to look out for in the future. I'll leave this open to try to explore any way possible to compile this script. Toughy though.

[ Permlink | « Hide ]

Charles Oliver Nutter - 04/Jul/08 11:18 AM

Some experimentation helped me find out that the upper bound of a literal int array is somewhere around 4700 elements. Then it crosses the 65k limit for an individual method.

The giant arrays in question are generated by RACC, so there's little change to get them changed. Instead we should explore the possibility of having arrays over a particular size force splitting by compiling as their own array-populating methods. I'm not going to do this right now, but it's something we'll want to look into in the future for cases like the Ruby parser.

[ Permlink | « Hide ]

Charles Oliver Nutter - 16/Jul/08 03:00 AM

This is a nontrivial bit of compiler twiddling for very little gain, so I'm punting it to post 1.1.3.