JRuby

Slow parsing of 100k lines file

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.1RC2
  • Fix Version/s: JRuby 1.1.3
  • Component/s: Performance
  • Labels:
    None
  • Environment:
    JRuby 1.1RC2
  • Number of attachments :
    0

Description

The following script is very slow in JRuby compared to MRI:
http://pastie.org/162908

file = "100k.rb" # 100k lines file

# create data file, if it doesn't exist yet
unless (File.exists?(file))
  File.open(file, "w+") { |f|
    f.puts("a = 1")
    100000.times { f.puts("a += 1") }
    f.puts("p a")
  }
end

if defined?(JRUBY_VERSION)
  exec("jruby.bat -X-C #{file}")
else
  exec("ruby #{file}")
end


# MRI is 0.3 sec
# JRuby, depending on the mode, 20 - 45 secs!!!

Activity

Hide
Vladimir Sizikov added a comment -

First, the bufferization to script file reading was added in rev. r6140, and modified "ListNode to use an ArrayList, to avoid the constant arraycopy when adding many nodes" in rev r6146 by headius.

For me, the run time is down to 7 secs from 30-45 secs, so I consider this result to be OK, given this is a very pathological case.

Show
Vladimir Sizikov added a comment - First, the bufferization to script file reading was added in rev. r6140, and modified "ListNode to use an ArrayList, to avoid the constant arraycopy when adding many nodes" in rev r6146 by headius. For me, the run time is down to 7 secs from 30-45 secs, so I consider this result to be OK, given this is a very pathological case.
Hide
Charles Oliver Nutter added a comment -

I committed a fix that modifies ListNode to use an ArrayList rather than an array it grows one element at a time. This actually appeared to improve Rails startup by a couple seconds, and once started there was no appreciable difference in memory consumption. That brings the run time for 100k.rb down to a few seconds. There are other bottlenecks we should look into, however, so we'll leave this open to remind us of an ugly parser edge case with still-too-slow performance.

Show
Charles Oliver Nutter added a comment - I committed a fix that modifies ListNode to use an ArrayList rather than an array it grows one element at a time. This actually appeared to improve Rails startup by a couple seconds, and once started there was no appreciable difference in memory consumption. That brings the run time for 100k.rb down to a few seconds. There are other bottlenecks we should look into, however, so we'll leave this open to remind us of an ugly parser edge case with still-too-slow performance.
Hide
Vladimir Sizikov added a comment -

Reopened the issue per Charlie's comments.

Show
Vladimir Sizikov added a comment - Reopened the issue per Charlie's comments.
Hide
Thomas E Enebo added a comment -

Bumping to 1.1+ for any future perf work

Show
Thomas E Enebo added a comment - Bumping to 1.1+ for any future perf work
Hide
Charles Oliver Nutter added a comment -

I'm going to mark this fixed, since we have put the ArrayList in place and it solved the vast bulk of performance problems. There should be other bugs file for specific perf problems in parsing, or for general parsing perf improvements.

Show
Charles Oliver Nutter added a comment - I'm going to mark this fixed, since we have put the ArrayList in place and it solved the vast bulk of performance problems. There should be other bugs file for specific perf problems in parsing, or for general parsing perf improvements.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: