Details

    • Type: Task Task
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.7.0.pre1
    • Fix Version/s: JRuby 1.6.7, JRuby 1.7.0.pre1
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      0

      Description

      Failure must be fixed. I believe @ymnk or @nahi have patches in the pipeline for this.

           [java] 4)
           [java] An exception occurred during: after :each
           [java] Zlib::Inflate#<< properly handles incomplete data ERROR
           [java] Zlib::BufError: buffer error
           [java] org/jruby/ext/zlib/RubyZlib.java:429:in `close'
           [java] /Users/headius/projects/jruby/spec/ruby/library/zlib/inflate/append_spec.rb:14:in `(root)'
           [java] org/jruby/RubyKernel.java:1914:in `instance_eval'
           [java] org/jruby/RubyEnumerable.java:1313:in `all?'
           [java] org/jruby/RubyArray.java:1601:in `each'
           [java] /Users/headius/projects/jruby/spec/ruby/library/zlib/inflate/append_spec.rb:4:in `(root)'
           [java] org/jruby/RubyKernel.java:986:in `load'
           [java] /Users/headius/projects/jruby/spec/ruby/library/zlib/inflate/append_spec.rb:56:in `files'
           [java] org/jruby/RubyKernel.java:1914:in `instance_eval'
           [java] org/jruby/RubyArray.java:1601:in `each'
      

        Activity

        Hide
        Hiroshi Nakamura added a comment -

        I added benchmark script at 2ce55b6f on master.

        New JZlib version is 1.3~1.8 times slower (not bad considering Java port)
        but it seems to consume more memories than the original version. The
        slowdown might be from this memory issue.

        % JAVA_HOME=/home/nahi/java/jdk1.6.0_26 jruby164 bench_zlib.rb
        Rehearsal ----------------------------------------------------
        [strip]
        ------------------------------------------ total: 20.949000sec
        
                               user     system      total        real
        Zlib::Deflate      4.120000   0.000000   4.120000 (  4.120000)
        Zlib::Inflate      3.253000   0.000000   3.253000 (  3.253000)
        Zlib::GzipWriter   4.737000   0.000000   4.737000 (  4.737000)
        Zlib::GzipReader   4.640000   0.000000   4.640000 (  4.640000)
        
        % JAVA_HOME=/home/nahi/java/jdk1.6.0_26 jruby bench_zlib.rb
        Rehearsal ----------------------------------------------------
        [strip]
        ------------------------------------------ total: 28.957000sec
        
                               user     system      total        real
        Zlib::Deflate      6.669000   0.000000   6.669000 (  6.670000)
        Zlib::Inflate      5.869000   0.000000   5.869000 (  5.870000)
        Zlib::GzipWriter   7.372000   0.000000   7.372000 (  7.372000)
        Zlib::GzipReader   6.226000   0.000000   6.226000 (  6.226000)
        

        Simple Zlib::Deflate seems to behave the same for 1.6.4(old zlib impl) and 1.7.0.dev(new one). I'll investigate more.

        Show
        Hiroshi Nakamura added a comment - I added benchmark script at 2ce55b6f on master. New JZlib version is 1.3~1.8 times slower (not bad considering Java port) but it seems to consume more memories than the original version. The slowdown might be from this memory issue. % JAVA_HOME=/home/nahi/java/jdk1.6.0_26 jruby164 bench_zlib.rb Rehearsal ---------------------------------------------------- [strip] ------------------------------------------ total: 20.949000sec user system total real Zlib::Deflate 4.120000 0.000000 4.120000 ( 4.120000) Zlib::Inflate 3.253000 0.000000 3.253000 ( 3.253000) Zlib::GzipWriter 4.737000 0.000000 4.737000 ( 4.737000) Zlib::GzipReader 4.640000 0.000000 4.640000 ( 4.640000) % JAVA_HOME=/home/nahi/java/jdk1.6.0_26 jruby bench_zlib.rb Rehearsal ---------------------------------------------------- [strip] ------------------------------------------ total: 28.957000sec user system total real Zlib::Deflate 6.669000 0.000000 6.669000 ( 6.670000) Zlib::Inflate 5.869000 0.000000 5.869000 ( 5.870000) Zlib::GzipWriter 7.372000 0.000000 7.372000 ( 7.372000) Zlib::GzipReader 6.226000 0.000000 6.226000 ( 6.226000) Simple Zlib::Deflate seems to behave the same for 1.6.4(old zlib impl) and 1.7.0.dev(new one). I'll investigate more.
        Hide
        Hiroshi Nakamura added a comment -

        Our original ext/zlib implementation (wrapping java.util.Deflate/Inflate) seems to consume much memories already. I'll try to reduce memory usages of that wrapping part first.

        Anyway, it seems there's no memory leak in JZlib. The problem is the size of byte[] objects allocated and freed.

        Show
        Hiroshi Nakamura added a comment - Our original ext/zlib implementation (wrapping java.util.Deflate/Inflate) seems to consume much memories already. I'll try to reduce memory usages of that wrapping part first. Anyway, it seems there's no memory leak in JZlib. The problem is the size of byte[] objects allocated and freed.
        Hide
        Hiroshi Nakamura added a comment - - edited

        I created the new branch which backports jzlib impl to jruby-1_6 for performance comparison. I could not compare zlib implementations on Java7 + JRuby master because it allocates and frees much memory as I wrote in the previous comment. bench/bench_zlib.rb eventually become >1G RSZ on Linux.

        Here's comparison between JZlib version and original version against jruby-1_6.

        jruby 1.6.5.dev (ruby-1.8.7-p330) (2011-10-11 78a5fa7) (Java HotSpot(TM) 64-Bit Server VM 1.7.0_02-ea) [linux-amd64-java]

        = JZlib version =
        
        ------------------------------------------ total: 24.255000sec
                               user     system      total        real
        Zlib::Deflate      4.910000   0.000000   4.910000 (  4.910000)
        Zlib::Inflate      5.227000   0.000000   5.227000 (  5.227000)
        Zlib::GzipWriter   4.756000   0.000000   4.756000 (  4.756000)
        Zlib::GzipReader   5.476000   0.000000   5.476000 (  5.476000)
        
        
        = original version =
        
        ------------------------------------------ total: 18.511000sec
                               user     system      total        real
        Zlib::Deflate      3.623000   0.000000   3.623000 (  3.623000)
        Zlib::Inflate      3.041000   0.000000   3.041000 (  3.041000)
        Zlib::GzipWriter   4.210000   0.000000   4.210000 (  4.210000)
        Zlib::GzipReader   3.509000   0.000000   3.509000 (  3.510000)
        

        Perf drops are from 10% to 70%. Decompression(Inflate/GzipReader) seems to be a little worth than Compression(Deflate/GzipWriter).

        Show
        Hiroshi Nakamura added a comment - - edited I created the new branch which backports jzlib impl to jruby-1_6 for performance comparison. I could not compare zlib implementations on Java7 + JRuby master because it allocates and frees much memory as I wrote in the previous comment. bench/bench_zlib.rb eventually become >1G RSZ on Linux. Here's comparison between JZlib version and original version against jruby-1_6. jruby 1.6.5.dev (ruby-1.8.7-p330) (2011-10-11 78a5fa7) (Java HotSpot(TM) 64-Bit Server VM 1.7.0_02-ea) [linux-amd64-java] = JZlib version = ------------------------------------------ total: 24.255000sec user system total real Zlib::Deflate 4.910000 0.000000 4.910000 ( 4.910000) Zlib::Inflate 5.227000 0.000000 5.227000 ( 5.227000) Zlib::GzipWriter 4.756000 0.000000 4.756000 ( 4.756000) Zlib::GzipReader 5.476000 0.000000 5.476000 ( 5.476000) = original version = ------------------------------------------ total: 18.511000sec user system total real Zlib::Deflate 3.623000 0.000000 3.623000 ( 3.623000) Zlib::Inflate 3.041000 0.000000 3.041000 ( 3.041000) Zlib::GzipWriter 4.210000 0.000000 4.210000 ( 4.210000) Zlib::GzipReader 3.509000 0.000000 3.509000 ( 3.510000) Perf drops are from 10% to 70%. Decompression(Inflate/GzipReader) seems to be a little worth than Compression(Deflate/GzipWriter).
        Hide
        Hiroshi Nakamura added a comment -

        Changing priority from 'Blocker' to 'Major'.

        I'll do benchmark when JRuby master be a little more stable. For now, it consumes much memory and it's hard to measure stock performance of JZlib.

        Show
        Hiroshi Nakamura added a comment - Changing priority from 'Blocker' to 'Major'. I'll do benchmark when JRuby master be a little more stable. For now, it consumes much memory and it's hard to measure stock performance of JZlib.
        Hide
        Hiroshi Nakamura added a comment -

        Closing this because JZlib imple is already backported to jruby-1_6. Perf should not be a problem.

        We got a severe perf drop report around Encoding on 1.9 mode at jruby-user but it's an Encoding issue, not JZlib's.

        Show
        Hiroshi Nakamura added a comment - Closing this because JZlib imple is already backported to jruby-1_6. Perf should not be a problem. We got a severe perf drop report around Encoding on 1.9 mode at jruby-user but it's an Encoding issue, not JZlib's.

          People

          • Assignee:
            Hiroshi Nakamura
            Reporter:
            Charles Oliver Nutter
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: