Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: JRuby-OSSL 0.7.1
    • Fix Version/s: JRuby-OSSL 0.7.7
    • Component/s: OpenSSL
    • Labels:
    • Environment:
      jruby 1.6.6 (ruby-1.9.2-p312) (2012-01-30 5673572) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_29) [darwin-x86_64-java]
    • Number of attachments :
      1

      Description

      Requests seem to hang over ssl when sending back data read from a file. The file contains a mix of single-byte UTF-8 characters and double-byte UTF-8 characters (all at the front in my example, but not in the real case). Over http it works fine, and the same app works in https and http in MRI.

      With file attached, run rackup and ping http://localhost:3000/ - for me, the request hangs for a bit, and curl responds with: * transfer closed with 2 bytes remaining to read.

      Change :SSLEnable = true to = :SSLEnable = false and hit the http url, and the problem goes away. I've put this test case into Webrick, but was having the same issue with other server.

      Problem is very dependent on length of output - remove a few lines on dots and the issue does not occur.

      1. config.ru
        0.8 kB
        Ben Porterfield

        Activity

        Hide
        Ben Porterfield added a comment - - edited

        Writing to SSL Socket with jruby-ossl in 1.9 doesn't look like it supports multi-byte character encodings at all.

        The function that writes to the buffer expects characters to be single-byte. For ex, this line uses String[] access characters, but decides what chars to pick during the loop based on a value returned in bytes, not characters. syswrite returns actual bytes written - which screws up the buffer if chars are multibyte, since the value of 'remaining' can be off, and at least in my test case become negative.

        It seems to work if I change the buffering to respect actual bytes in the String: https://gist.github.com/1953963

        This doesn't work without String.byteslice, which is new in 1.9.3, so to get around it: https://gist.github.com/1953966

        There's still the fact that this line ignores encoding differences in strings - I'm not sure how that's handled in Ruby, but potentially more issues there.

        Show
        Ben Porterfield added a comment - - edited Writing to SSL Socket with jruby-ossl in 1.9 doesn't look like it supports multi-byte character encodings at all. The function that writes to the buffer expects characters to be single-byte. For ex, this line uses String[] access characters, but decides what chars to pick during the loop based on a value returned in bytes, not characters. syswrite returns actual bytes written - which screws up the buffer if chars are multibyte, since the value of 'remaining' can be off, and at least in my test case become negative. It seems to work if I change the buffering to respect actual bytes in the String: https://gist.github.com/1953963 This doesn't work without String.byteslice, which is new in 1.9.3, so to get around it: https://gist.github.com/1953966 There's still the fact that this line ignores encoding differences in strings - I'm not sure how that's handled in Ruby, but potentially more issues there.
        Hide
        Ben Porterfield added a comment -

        Apparently can't change issue title in jira, but it's worth noting that this bug results in lost data during transfer.

        Show
        Ben Porterfield added a comment - Apparently can't change issue title in jira, but it's worth noting that this bug results in lost data during transfer.
        Hide
        Hiro Asari added a comment -

        Does the problem exist on the latest version of jruby-openssl (0.7.6.1)?

        Show
        Hiro Asari added a comment - Does the problem exist on the latest version of jruby-openssl (0.7.6.1)?
        Hide
        Ben Porterfield added a comment -

        Yes, the problem exists in 0.7.6.1, and also in jruby-ossl master.

        Show
        Ben Porterfield added a comment - Yes, the problem exists in 0.7.6.1, and also in jruby-ossl master.
        Hide
        Ben Porterfield added a comment -

        Specifically, to see the error there needs to be:

        1. A string with multibyte character in it
        2. Enough data to require the "while remain > 0" loop to run more than one time

        When these conditions are met, during the second loop, the remain variable is set to the bytes written in syswrite(), and is then used to determine characters to grab from string. When some characters are multibyte, the positioning is incorrect, since String[] is char-based, not byte-based.

        Show
        Ben Porterfield added a comment - Specifically, to see the error there needs to be: 1. A string with multibyte character in it 2. Enough data to require the "while remain > 0" loop to run more than one time When these conditions are met, during the second loop, the remain variable is set to the bytes written in syswrite(), and is then used to determine characters to grab from string. When some characters are multibyte, the positioning is incorrect, since String[] is char-based, not byte-based.
        Hide
        Ben Porterfield added a comment -

        I've found the fix I was using (using a monkey-patched String.byteslice) was much too slow, so for now I'm getting by via a force_encoding call in do_write: https://gist.github.com/1988286

        Show
        Ben Porterfield added a comment - I've found the fix I was using (using a monkey-patched String.byteslice) was much too slow, so for now I'm getting by via a force_encoding call in do_write: https://gist.github.com/1988286
        Hide
        Patrick Ritchie added a comment -

        Ran into the same thing POSTing images over SSL via restclient.

        The monkey patch on OpenSSL::Buffering#do_write is working for me so far.

        Show
        Patrick Ritchie added a comment - Ran into the same thing POSTing images over SSL via restclient. The monkey patch on OpenSSL::Buffering#do_write is working for me so far.
        Hide
        Hiroshi Nakamura added a comment -

        It sounds this is fixed at upstream. I'll update libs. http://bugs.ruby-lang.org/issues/5233

        Show
        Hiroshi Nakamura added a comment - It sounds this is fixed at upstream. I'll update libs. http://bugs.ruby-lang.org/issues/5233
        Hide
        Hiroshi Nakamura added a comment -
        commit 7a757e0e54599ea17b6cb59c16036fedd5cc3290
        Author: Hiroshi Nakamura <nahi@ruby-lang.org>
        Date:   Mon Apr 30 11:32:10 2012 +0900
        
            JRUBY-6515: sending UTF-8 data over SSL can hang with openssl
            
            Import upstream fix at http://bugs.ruby-lang.org/issues/5233
        

        It will be fixed at 0.7.7.

        Show
        Hiroshi Nakamura added a comment - commit 7a757e0e54599ea17b6cb59c16036fedd5cc3290 Author: Hiroshi Nakamura <nahi@ruby-lang.org> Date: Mon Apr 30 11:32:10 2012 +0900 JRUBY-6515: sending UTF-8 data over SSL can hang with openssl Import upstream fix at http://bugs.ruby-lang.org/issues/5233 It will be fixed at 0.7.7.

          People

          • Assignee:
            Hiroshi Nakamura
            Reporter:
            Ben Porterfield
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: