[#JRUBY-1852] define_method methods do not JIT and are generally slower unless the code that contains them is compiled before they are defined

JRuby

define_method methods do not JIT and are generally slower unless the code that contains them is compiled before they are defined

Created: 31/Dec/07 03:57 AM Updated: 23/May/08 04:34 AM

Component/s:

Compiler

Affects Version/s:

None

Fix Version/s:

JRuby 1.1+

Time Tracking:

Not Specified

Description

« Hide

~/NetBeansProjects/jruby $ jruby -J-server -e "load 'test/bench/bench_define_method_methods.rb'"
define_method(:foo) {}, 100k * 100 invocations
  5.028000   0.000000   5.028000 (  4.936000)
  4.542000   0.000000   4.542000 (  4.541000)
  4.541000   0.000000   4.541000 (  4.541000)
  4.534000   0.000000   4.534000 (  4.534000)
  4.377000   0.000000   4.377000 (  4.376000)
~/NetBeansProjects/jruby $ jruby -J-server +C -e "load 'test/bench/bench_define_method_methods.rb'"
define_method(:foo) {}, 100k * 100 invocations
  4.273000   0.000000   4.273000 (  4.185000)
  3.750000   0.000000   3.750000 (  3.751000)
  3.403000   0.000000   3.403000 (  3.404000)
  3.453000   0.000000   3.453000 (  3.453000)
  3.358000   0.000000   3.358000 (  3.358000)

Could be a source of Rails bottlenecks.

All

Comments

Work Log

Change History

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Charles Oliver Nutter - 15/Feb/08 12:55 PM

Removing target release from issues that fit any of the following criteria:

No known way to fix them
Java integration enhancements out of scope for 1.1 release
Other out of scope issues for 1.1

[ Permlink | « Hide ]

Charles Oliver Nutter - 12/May/08 02:13 PM

I'd like to make this one happen in 1.1.2 if possible.

[ Permlink | « Hide ]

Charles Oliver Nutter - 23/May/08 04:34 AM

There's really only two ways to fix this: make block bodies be independently jittable, or modify define_method to actually define a normal method using the body of the block, modulo some masgn logic. Neither of these are trivial, and the latter is almost certain to introduce bugs in define_method, so therefore must be approached very cautiously. Given these facts, I'm punting this to post 1.1+ when we will be under less of a time crunch to explore more performance challenges like define_method.

Here is an update on performance; I have added a control run to show the relative difference between a normal method and a define_method method:

Fully interpreted:

$ jruby -X-C -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.207000   0.000000   0.207000 (  0.206000)
define_method(:foo) {1}, 10k * 100 invocations              0.381000   0.000000   0.381000 (  0.380000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.393000   0.000000   0.393000 (  0.392000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.415000   0.000000   0.415000 (  0.415000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.428000   0.000000   0.428000 (  0.427000)

JIT compiled (require used to avoid full compile)

$ jruby -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.141000   0.000000   0.141000 (  0.140000)
define_method(:foo) {1}, 10k * 100 invocations              0.384000   0.000000   0.384000 (  0.385000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.386000   0.000000   0.386000 (  0.386000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.420000   0.000000   0.420000 (  0.420000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.431000   0.000000   0.431000 (  0.431000)

Fully compiled

$ jruby -X+C -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.069000   0.000000   0.069000 (  0.069000)
define_method(:foo) {1}, 10k * 100 invocations              0.264000   0.000000   0.264000 (  0.264000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.300000   0.000000   0.300000 (  0.300000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.303000   0.000000   0.303000 (  0.303000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.274000   0.000000   0.274000 (  0.275000)

The difference between interpreted/jitted and fully compiled for define_method methods appears to be nearly 50% slower in the interpreted/jitted case.

[ Show » ]

Charles Oliver Nutter - 23/May/08 04:34 AM There's really only two ways to fix this: make block bodies be independently jittable, or modify define_method to actually define a normal method using the body of the block, modulo some masgn logic. Neither of these are trivial, and the latter is almost certain to introduce bugs in define_method, so therefore must be approached very cautiously. Given these facts, I'm punting this to post 1.1+ when we will be under less of a time crunch to explore more performance challenges like define_method. Here is an update on performance; I have added a control run to show the relative difference between a normal method and a define_method method: Fully interpreted:

$ jruby -X-C -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.207000   0.000000   0.207000 (  0.206000)
define_method(:foo) {1}, 10k * 100 invocations              0.381000   0.000000   0.381000 (  0.380000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.393000   0.000000   0.393000 (  0.392000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.415000   0.000000   0.415000 (  0.415000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.428000   0.000000   0.428000 (  0.427000)

JIT compiled (require used to avoid full compile)

$ jruby -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.141000   0.000000   0.141000 (  0.140000)
define_method(:foo) {1}, 10k * 100 invocations              0.384000   0.000000   0.384000 (  0.385000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.386000   0.000000   0.386000 (  0.386000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.420000   0.000000   0.420000 (  0.420000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.431000   0.000000   0.431000 (  0.431000)

Fully compiled

$ jruby -X+C -J-server -e "require 'test/bench/bench_define_method_methods.rb'; Benchmark.bmbm {|bm| 5.times { bench_define_method_methods(bm) } }"
control, simple method, 10k * 100 invocations               0.069000   0.000000   0.069000 (  0.069000)
define_method(:foo) {1}, 10k * 100 invocations              0.264000   0.000000   0.264000 (  0.264000)
eval'ed define_method(:baz) {1}, 10k * 100 invocations      0.300000   0.000000   0.300000 (  0.300000)
define_method(:bar) {a = 1}, 10k * 100 invocations          0.303000   0.000000   0.303000 (  0.303000)
b = 1; define_method(:baz) {b = 2}, 10k * 100 invocations   0.274000   0.000000   0.274000 (  0.275000)

The difference between interpreted/jitted and fully compiled for define_method methods appears to be nearly 50% slower in the interpreted/jitted case.