JRuby (please use github issues at http://bugs.jruby.org)
  1. JRuby (please use github issues at http://bugs.jruby.org)
  2. JRUBY-6285

JRuby 1.7 master on Java7u2 is *slower* running a benchmark than master on Java6

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Not A Bug
    • Affects Version/s: JRuby 1.7.0.pre1
    • Fix Version/s: JRuby 1.7.0.pre1
    • Component/s: Performance
    • Labels:
      None
    • Environment:
      OSX apple-provided java6, openjdk java7, jruby 17 master
    • Number of attachments :
      0

      Description

      1. Run the install_me in this file.

      http://dl.dropbox.com/u/44434337/jruby_perf.tgz

      2. cd rzmq_brokers-0.7.1/test

      3. jruby --server test_consensus 5555

      Here are my results from running master with java6 and java7.

      https://gist.github.com/1491345

      The java6 results are significantly better than the java7 results. This tells me two things. One, that jruby master has some excellent code improvements. Two, somehow java7 is not optimizing those as well as it should.

        Activity

        Hide
        Charles Oliver Nutter added a comment -

        Hmm, I was unable to reproduce your results on my first attempt. Here's what I got: https://gist.github.com/bd3d716fbd30569dbe42

        The first two results are JRuby master on Java 6 and Java 7, respectively. The Java 7 run is faster...not by a lot, but definitely faster.

        The third result is JRuby master on Java 7, but with invokedynamic use disabled. You can see performance drops off measurably, settling in closer to the Java 6 numbers.

        The fourth result is JRuby master on Java 7, using invokedynamic, but forcing all files to compile on load. These numbers are the best of all, beating the Java 6 numbers by a solid 20-25%.

        Try reproducing these numbers again with current JRuby master and 7u2.

        If you are still seeing Java 7 perform worse than Java 6, we'll start to turn on some logging flags to investigate. I suspect it may be a combination of things...native and VM memory overhead (overall CPU use is quite low), lazy compilation confusing the JVM JIT (-X+C helps that), and perhaps other peculiarities in your code or in the libraries that sometimes manifest as slow perf and sometimes do not.

        I will also note it is a very noisy benchmark. Under such circumstances, the usual problem is memory-related.

        Show
        Charles Oliver Nutter added a comment - Hmm, I was unable to reproduce your results on my first attempt. Here's what I got: https://gist.github.com/bd3d716fbd30569dbe42 The first two results are JRuby master on Java 6 and Java 7, respectively. The Java 7 run is faster...not by a lot, but definitely faster. The third result is JRuby master on Java 7, but with invokedynamic use disabled. You can see performance drops off measurably, settling in closer to the Java 6 numbers. The fourth result is JRuby master on Java 7, using invokedynamic, but forcing all files to compile on load. These numbers are the best of all, beating the Java 6 numbers by a solid 20-25%. Try reproducing these numbers again with current JRuby master and 7u2. If you are still seeing Java 7 perform worse than Java 6, we'll start to turn on some logging flags to investigate. I suspect it may be a combination of things...native and VM memory overhead (overall CPU use is quite low), lazy compilation confusing the JVM JIT (-X+C helps that), and perhaps other peculiarities in your code or in the libraries that sometimes manifest as slow perf and sometimes do not. I will also note it is a very noisy benchmark. Under such circumstances, the usual problem is memory-related.
        Hide
        Chuck Remes added a comment -

        This problem was due to the use of the wrong java7 release for OSX. Make sure that the release is "update 2" or u2 for your platform. It contains many perf fixes necessary for invokedynamic code to beat java6.

        Not a bug.

        Show
        Chuck Remes added a comment - This problem was due to the use of the wrong java7 release for OSX. Make sure that the release is "update 2" or u2 for your platform. It contains many perf fixes necessary for invokedynamic code to beat java6. Not a bug.

          People

          • Assignee:
            Unassigned
            Reporter:
            Chuck Remes
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: