JRuby

sending UTF-8 data over SSL can hang with openssl

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Critical Critical
  • Resolution: Fixed
  • Affects Version/s: JRuby-OSSL 0.7.1
  • Fix Version/s: JRuby-OSSL 0.7.7
  • Component/s: OpenSSL
  • Labels:
  • Environment:
    jruby 1.6.6 (ruby-1.9.2-p312) (2012-01-30 5673572) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_29) [darwin-x86_64-java]
  • Number of attachments :
    1

Description

Requests seem to hang over ssl when sending back data read from a file. The file contains a mix of single-byte UTF-8 characters and double-byte UTF-8 characters (all at the front in my example, but not in the real case). Over http it works fine, and the same app works in https and http in MRI.

With file attached, run rackup and ping http://localhost:3000/ - for me, the request hangs for a bit, and curl responds with: * transfer closed with 2 bytes remaining to read.

Change :SSLEnable = true to = :SSLEnable = false and hit the http url, and the problem goes away. I've put this test case into Webrick, but was having the same issue with other server.

Problem is very dependent on length of output - remove a few lines on dots and the issue does not occur.

  1. File
    config.ru
    01/Mar/12 12:07 PM
    0.8 kB
    Ben Porterfield

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Ben Porterfield added a comment - - edited

Writing to SSL Socket with jruby-ossl in 1.9 doesn't look like it supports multi-byte character encodings at all.

The function that writes to the buffer expects characters to be single-byte. For ex, this line uses String[] access characters, but decides what chars to pick during the loop based on a value returned in bytes, not characters. syswrite returns actual bytes written - which screws up the buffer if chars are multibyte, since the value of 'remaining' can be off, and at least in my test case become negative.

It seems to work if I change the buffering to respect actual bytes in the String: https://gist.github.com/1953963

This doesn't work without String.byteslice, which is new in 1.9.3, so to get around it: https://gist.github.com/1953966

There's still the fact that this line ignores encoding differences in strings - I'm not sure how that's handled in Ruby, but potentially more issues there.

Show
Ben Porterfield added a comment - - edited Writing to SSL Socket with jruby-ossl in 1.9 doesn't look like it supports multi-byte character encodings at all. The function that writes to the buffer expects characters to be single-byte. For ex, this line uses String[] access characters, but decides what chars to pick during the loop based on a value returned in bytes, not characters. syswrite returns actual bytes written - which screws up the buffer if chars are multibyte, since the value of 'remaining' can be off, and at least in my test case become negative. It seems to work if I change the buffering to respect actual bytes in the String: https://gist.github.com/1953963 This doesn't work without String.byteslice, which is new in 1.9.3, so to get around it: https://gist.github.com/1953966 There's still the fact that this line ignores encoding differences in strings - I'm not sure how that's handled in Ruby, but potentially more issues there.
Hide
Permalink
Ben Porterfield added a comment -

Apparently can't change issue title in jira, but it's worth noting that this bug results in lost data during transfer.

Show
Ben Porterfield added a comment - Apparently can't change issue title in jira, but it's worth noting that this bug results in lost data during transfer.
Hide
Permalink
Hiro Asari added a comment -

Does the problem exist on the latest version of jruby-openssl (0.7.6.1)?

Show
Hiro Asari added a comment - Does the problem exist on the latest version of jruby-openssl (0.7.6.1)?
Hide
Permalink
Ben Porterfield added a comment -

Yes, the problem exists in 0.7.6.1, and also in jruby-ossl master.

Show
Ben Porterfield added a comment - Yes, the problem exists in 0.7.6.1, and also in jruby-ossl master.
Hide
Permalink
Ben Porterfield added a comment -

Specifically, to see the error there needs to be:

1. A string with multibyte character in it
2. Enough data to require the "while remain > 0" loop to run more than one time

When these conditions are met, during the second loop, the remain variable is set to the bytes written in syswrite(), and is then used to determine characters to grab from string. When some characters are multibyte, the positioning is incorrect, since String[] is char-based, not byte-based.

Show
Ben Porterfield added a comment - Specifically, to see the error there needs to be: 1. A string with multibyte character in it 2. Enough data to require the "while remain > 0" loop to run more than one time When these conditions are met, during the second loop, the remain variable is set to the bytes written in syswrite(), and is then used to determine characters to grab from string. When some characters are multibyte, the positioning is incorrect, since String[] is char-based, not byte-based.
Hide
Permalink
Ben Porterfield added a comment -

I've found the fix I was using (using a monkey-patched String.byteslice) was much too slow, so for now I'm getting by via a force_encoding call in do_write: https://gist.github.com/1988286

Show
Ben Porterfield added a comment - I've found the fix I was using (using a monkey-patched String.byteslice) was much too slow, so for now I'm getting by via a force_encoding call in do_write: https://gist.github.com/1988286
Hide
Permalink
Patrick Ritchie added a comment -

Ran into the same thing POSTing images over SSL via restclient.

The monkey patch on OpenSSL::Buffering#do_write is working for me so far.

Show
Patrick Ritchie added a comment - Ran into the same thing POSTing images over SSL via restclient. The monkey patch on OpenSSL::Buffering#do_write is working for me so far.
Hide
Permalink
Hiroshi Nakamura added a comment -

It sounds this is fixed at upstream. I'll update libs. http://bugs.ruby-lang.org/issues/5233

Show
Hiroshi Nakamura added a comment - It sounds this is fixed at upstream. I'll update libs. http://bugs.ruby-lang.org/issues/5233
Hide
Permalink
Hiroshi Nakamura added a comment - 
commit 7a757e0e54599ea17b6cb59c16036fedd5cc3290
Author: Hiroshi Nakamura <nahi@ruby-lang.org>
Date:   Mon Apr 30 11:32:10 2012 +0900

    JRUBY-6515: sending UTF-8 data over SSL can hang with openssl
    
    Import upstream fix at http://bugs.ruby-lang.org/issues/5233

It will be fixed at 0.7.7.

Show
Hiroshi Nakamura added a comment - commit 7a757e0e54599ea17b6cb59c16036fedd5cc3290 Author: Hiroshi Nakamura <nahi@ruby-lang.org> Date: Mon Apr 30 11:32:10 2012 +0900 JRUBY-6515: sending UTF-8 data over SSL can hang with openssl Import upstream fix at http://bugs.ruby-lang.org/issues/5233 It will be fixed at 0.7.7.

People

  • Assignee:
    Hiroshi Nakamura
    Reporter:
    Ben Porterfield
Vote (1)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: