[#JRUBY-2245] Slow parsing of 100k lines file

JRuby

Slow parsing of 100k lines file

Created: 07/Mar/08 03:14 PM Updated: 08/Jul/08 03:42 PM

Component/s:

Performance

Affects Version/s:

JRuby 1.1RC2

Fix Version/s:

JRuby 1.1.3

Time Tracking:

Not Specified

Environment:

JRuby 1.1RC2

Description

« Hide

The following script is very slow in JRuby compared to MRI:
http://pastie.org/162908

file = "100k.rb" # 100k lines file

# create data file, if it doesn't exist yet
unless (File.exists?(file))
  File.open(file, "w+") { |f|
    f.puts("a = 1")
    100000.times { f.puts("a += 1") }
    f.puts("p a")
  }
end

if defined?(JRUBY_VERSION)
  exec("jruby.bat -X-C #{file}")
else
  exec("ruby #{file}")
end


# MRI is 0.3 sec
# JRuby, depending on the mode, 20 - 45 secs!!!

All

Comments

Work Log

Change History

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Vladimir Sizikov - 08/Mar/08 11:39 AM

First, the bufferization to script file reading was added in rev. r6140, and modified "ListNode to use an ArrayList, to avoid the constant arraycopy when adding many nodes" in rev r6146 by headius.

For me, the run time is down to 7 secs from 30-45 secs, so I consider this result to be OK, given this is a very pathological case.

[ Permlink | « Hide ]

Charles Oliver Nutter - 08/Mar/08 04:33 PM

I committed a fix that modifies ListNode to use an ArrayList rather than an array it grows one element at a time. This actually appeared to improve Rails startup by a couple seconds, and once started there was no appreciable difference in memory consumption. That brings the run time for 100k.rb down to a few seconds. There are other bottlenecks we should look into, however, so we'll leave this open to remind us of an ugly parser edge case with still-too-slow performance.

[ Permlink | « Hide ]

Vladimir Sizikov - 10/Mar/08 11:56 AM

Reopened the issue per Charlie's comments.

[ Permlink | « Hide ]

Thomas E Enebo - 16/Mar/08 11:13 PM

Bumping to 1.1+ for any future perf work

[ Permlink | « Hide ]

Charles Oliver Nutter - 08/Jul/08 03:42 PM

I'm going to mark this fixed, since we have put the ArrayList in place and it solved the vast bulk of performance problems. There should be other bugs file for specific perf problems in parsing, or for general parsing perf improvements.