JRuby (please use github issues at http://bugs.jruby.org)
  1. JRuby (please use github issues at http://bugs.jruby.org)
  2. JRUBY-5529

[1.9] undefined method `read_nonblock' for #<OpenSSL::SSL::SSLSocket:0x246adb31> (NoMethodError)

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: JRuby-OSSL 0.7.3
    • Fix Version/s: JRuby 1.7.1
    • Component/s: OpenSSL, Ruby 1.9.2, Ruby 1.9.3
    • Labels:
      None
    • Environment:
    • Number of attachments :
      1

      Description

      /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/bin/jruby --1.9 -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) /Users/vsrikarunyan/workspace/mine/lab/linkedin.rb
      /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/protocol.rb:135:in `rbuf_fill': undefined method `read_nonblock' for #<OpenSSL::SSL::SSLSocket:0x246adb31> (NoMethodError)
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/protocol.rb:116:in `readuntil'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/protocol.rb:126:in `readline'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:2211:in `read_status_line'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:2200:in `read_new'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:1183:in `transport_request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:1169:in `request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:1162:in `request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:627:in `start'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/1.9/net/http.rb:1160:in `request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/gems/1.8/gems/oauth-0.4.4/lib/oauth/consumer.rb:164:in `request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/gems/1.8/gems/oauth-0.4.4/lib/oauth/consumer.rb:197:in `token_request'
      from /app/jruby/JRuby.Framwork/Versions/1.6.0.RC2/lib/ruby/gems/1.8/gems/oauth-0.4.4/lib/oauth/consumer.rb:139:in `get_request_token'
      from /Users/vsrikarunyan/workspace/mine/lab/linkedin.rb:21:in `(root)'
      from org/jruby/RubyKernel.java:1073:in `load19'
      from -e:1:in `(root)'

      Process finished with exit code 1

        Issue Links

          Activity

          Hide
          Hiro Asari added a comment -

          Can you provide us with some simple cases where this is failing?

          Show
          Hiro Asari added a comment - Can you provide us with some simple cases where this is failing?
          Hide
          Karl Baum added a comment -

          I have created a git project with instructions on how to reproduce:

          https://github.com/kbaum/undefined_method_read_nonblock

          Let me know if you need anything else.

          Thanks!

          Show
          Karl Baum added a comment - I have created a git project with instructions on how to reproduce: https://github.com/kbaum/undefined_method_read_nonblock Let me know if you need anything else. Thanks!
          Hide
          Nick Sieger added a comment -

          It's easy to reproduce, just fetch any https website under 1.9 mode. The error message isn't lying. There is no #read_nonblock on SSLSocket.

          The 1.8 version of #rbuf_fill used #sysread, which is defined.

          Show
          Nick Sieger added a comment - It's easy to reproduce, just fetch any https website under 1.9 mode. The error message isn't lying. There is no #read_nonblock on SSLSocket. The 1.8 version of #rbuf_fill used #sysread, which is defined.
          Hide
          Scott Lowe added a comment -

          In addition to fetching https websites, I've also run into this issue when trying to send SSL email, e.g. via Gmail; it's a blocker in my current project which has a stand-alone mail queue.

          Show
          Scott Lowe added a comment - In addition to fetching https websites, I've also run into this issue when trying to send SSL email, e.g. via Gmail; it's a blocker in my current project which has a stand-alone mail queue.
          Hide
          Karl Baum added a comment - - edited

          It was a blocker for me too, so i monkey patched the one method to look like the ruby 1.8 version and it seems to work for me:

          https://gist.github.com/969527

          Show
          Karl Baum added a comment - - edited It was a blocker for me too, so i monkey patched the one method to look like the ruby 1.8 version and it seems to work for me: https://gist.github.com/969527
          Hide
          Andre Meij added a comment -

          Same issue here. Thanks at @Karl Baum, your monkey patch seems to get me running.

          Show
          Andre Meij added a comment - Same issue here. Thanks at @Karl Baum, your monkey patch seems to get me running.
          Hide
          Markus Fischer added a comment -

          Thanks also to @Karl Baum from me, it worked also for me.

          Show
          Markus Fischer added a comment - Thanks also to @Karl Baum from me, it worked also for me.
          Hide
          Charles Oliver Nutter added a comment -

          This will be a bit bigger than just a one-off fix. We could add a temporary alias from new read methods to sysread, since in theory they should function the same other than buffering and blocking, but the IO bits of SSLSocket need a refactoring.

          I checked on the methods we define and the ones defined in 1.9. Here's what we're missing:

              rb_define_method(cSSLSocket, "connect_nonblock",    ossl_ssl_connect_nonblock, 0);
              rb_define_method(cSSLSocket, "accept_nonblock",     ossl_ssl_accept_nonblock, 0);
              rb_define_private_method(cSSLSocket, "sysread_nonblock",    ossl_ssl_read_nonblock, -1);
              rb_define_private_method(cSSLSocket, "syswrite_nonblock",    ossl_ssl_write_nonblock, 1);
              rb_define_method(cSSLSocket, "session_reused?",    ossl_ssl_session_reused, 0);
              rb_define_method(cSSLSocket, "session=",    ossl_ssl_set_session, 1);
          

          So basically we need nonblocking versions of connect, accept, sysread, and syswriter, though the latter two are not public (perhaps used from Ruby code?)

          The odd thing here is that there is no read_nonblock, and if I check SSLSocket's methods under 1.9 it is indeed not defined:

          ~/projects $ ruby1.9.2 -ropenssl -e "p OpenSSL::SSL::SSLSocket.methods.include? 'read_nonblock'"
          false
          

          So the question is why it's attempting to call read_nonblock against SSLSocket. I suspect there may be some coercion missing that wraps SSLSocket in an IO, which then delegates to sysread_nonblock and friends. Hmm.

          Show
          Charles Oliver Nutter added a comment - This will be a bit bigger than just a one-off fix. We could add a temporary alias from new read methods to sysread, since in theory they should function the same other than buffering and blocking, but the IO bits of SSLSocket need a refactoring. I checked on the methods we define and the ones defined in 1.9. Here's what we're missing: rb_define_method(cSSLSocket, "connect_nonblock", ossl_ssl_connect_nonblock, 0); rb_define_method(cSSLSocket, "accept_nonblock", ossl_ssl_accept_nonblock, 0); rb_define_private_method(cSSLSocket, "sysread_nonblock", ossl_ssl_read_nonblock, -1); rb_define_private_method(cSSLSocket, "syswrite_nonblock", ossl_ssl_write_nonblock, 1); rb_define_method(cSSLSocket, "session_reused?", ossl_ssl_session_reused, 0); rb_define_method(cSSLSocket, "session=", ossl_ssl_set_session, 1); So basically we need nonblocking versions of connect, accept, sysread, and syswriter, though the latter two are not public (perhaps used from Ruby code?) The odd thing here is that there is no read_nonblock, and if I check SSLSocket's methods under 1.9 it is indeed not defined: ~/projects $ ruby1.9.2 -ropenssl -e "p OpenSSL::SSL::SSLSocket.methods.include? 'read_nonblock'" false So the question is why it's attempting to call read_nonblock against SSLSocket. I suspect there may be some coercion missing that wraps SSLSocket in an IO, which then delegates to sysread_nonblock and friends. Hmm.
          Hide
          Charles Oliver Nutter added a comment -

          Ok, I think I found it. read_nonblock and many other IOish methods are defined in the .rb files for OpenSSL. The buffering.rb file defines the Buffering module, which has all the methods in question build atop sysread/syswrite/sysread_nonblock/syswrite_nonblock.

          So the simple answer is that we need to implement the missing nonblock methods above and then update the OpenSSL .rb files to 1.9's latest. The methods will exist in 1.8 mode, but there's really no way around that.

          Show
          Charles Oliver Nutter added a comment - Ok, I think I found it. read_nonblock and many other IOish methods are defined in the .rb files for OpenSSL. The buffering.rb file defines the Buffering module, which has all the methods in question build atop sysread/syswrite/sysread_nonblock/syswrite_nonblock. So the simple answer is that we need to implement the missing nonblock methods above and then update the OpenSSL .rb files to 1.9's latest. The methods will exist in 1.8 mode, but there's really no way around that.
          Hide
          Duncan Mak added a comment -

          I'd like to help close this bug.

          I have forked jruby-ossl and implemented sysread_nonblock and syswrite_nonblock. The code is available here: https://github.com/duncanmak/jruby-ossl

          I ran Karl Baum's test case, and on my local machine, here's the output I see:

          duncan@morpheme:~/git/undefined_method_read_nonblock (master)$ JRUBY_OPTS='--1.9' ./test.rb
          Net::POPAuthenticationError: -ERR authentication failed
            check_response_auth at /home/duncan/.rvm/rubies/jruby-1.6.4/lib/ruby/1.9/net/pop.rb:984
          

          Which looks right to me.

          The methods remaining are: connect_nonblock, accept_nonblock, session_reused? and session=.

          connect_nonblock/accept_nonblock:

          I have been going over the C implementation, I think what I need to do is to raise IO::WaitReadable/IO::WaitWritable instead of waitSelect(SelectionKey.OP_READ | SelectionKey.OP_WRITE); – I don't really know what I should do here, though.

          session_reused?:

          From the bit of research that I have done, it looks like SSL sessions are always reusable (unless they're closed?), so is it safe to just return true?

          session=:

          It doesn't look like the SSL implementation in Java allow for this sort of session swapping. Could I just throw UnsupportedOperationException here?

          Any suggestions / code review will be greatly appreciated.

          Thanks!

          Show
          Duncan Mak added a comment - I'd like to help close this bug. I have forked jruby-ossl and implemented sysread_nonblock and syswrite_nonblock. The code is available here: https://github.com/duncanmak/jruby-ossl I ran Karl Baum 's test case, and on my local machine, here's the output I see: duncan@morpheme:~/git/undefined_method_read_nonblock (master)$ JRUBY_OPTS='--1.9' ./test.rb Net::POPAuthenticationError: -ERR authentication failed check_response_auth at /home/duncan/.rvm/rubies/jruby-1.6.4/lib/ruby/1.9/net/pop.rb:984 Which looks right to me. The methods remaining are: connect_nonblock, accept_nonblock, session_reused? and session=. connect_nonblock/accept_nonblock: I have been going over the C implementation, I think what I need to do is to raise IO::WaitReadable/IO::WaitWritable instead of waitSelect(SelectionKey.OP_READ | SelectionKey.OP_WRITE); – I don't really know what I should do here, though. session_reused?: From the bit of research that I have done, it looks like SSL sessions are always reusable (unless they're closed?), so is it safe to just return true? session=: It doesn't look like the SSL implementation in Java allow for this sort of session swapping. Could I just throw UnsupportedOperationException here? Any suggestions / code review will be greatly appreciated. Thanks!
          Hide
          Hiroshi Nakamura added a comment -

          Guys, sorry for keeping this issue long time. Duncan, thanks for looking this issue. I hope I can help you to implement 1.9 support features.

          At first, we should design how we implement -1.8/-1.9 supports in 1 gem. Updating lib/openssl/* with CRuby 1.9 lib would break --1.8 mode... It's best to integrate jruby-ossl into ext/openssl of JRuby (JRUBY-5035) but the branch is on hiatus, just since I don't have enough time...

          > connect_nonblock/accept_nonblock:
          > I have been going over the C implementation, I think what I need to do is to raise
          > IO::WaitReadable/IO::WaitWritable instead of waitSelect(SelectionKey.OP_READ | SelectionKey.OP_WRITE);
          >  I don't really know what I should do here, though.
          

          Basically true, and a little bit extra-handling for before STARTTLS. Can you check if test/openssl/test_pair.rb and test_ssl.rb of CRuby runs fine with your branch?

          And I think I should fix several nonblocking IO of JRuby itself first. (Try test/ruby/test_io.rb of CRuby repo against JRuby --1.9 mode.)

          > session_reused?:
          > From the bit of research that I have done, it looks like SSL sessions are always reusable (unless
          > they're closed?), so is it safe to just return true?
          
          > session=:
          > It doesn't look like the SSL implementation in Java allow for this sort of session swapping. Could
          > I just throw UnsupportedOperationException here?
          

          Yeah, I don't think we can make SSLSession persistent in Java. Session cache is done by JCE in Java, so SSLSession (JRUBY-4371) would be a read only object.

          Any suggestions how we go forward?

          Show
          Hiroshi Nakamura added a comment - Guys, sorry for keeping this issue long time. Duncan, thanks for looking this issue. I hope I can help you to implement 1.9 support features. At first, we should design how we implement - 1.8/ -1.9 supports in 1 gem. Updating lib/openssl/* with CRuby 1.9 lib would break --1.8 mode... It's best to integrate jruby-ossl into ext/openssl of JRuby ( JRUBY-5035 ) but the branch is on hiatus, just since I don't have enough time... > connect_nonblock/accept_nonblock: > I have been going over the C implementation, I think what I need to do is to raise > IO::WaitReadable/IO::WaitWritable instead of waitSelect(SelectionKey.OP_READ | SelectionKey.OP_WRITE); > I don't really know what I should do here, though. Basically true, and a little bit extra-handling for before STARTTLS. Can you check if test/openssl/test_pair.rb and test_ssl.rb of CRuby runs fine with your branch? And I think I should fix several nonblocking IO of JRuby itself first. (Try test/ruby/test_io.rb of CRuby repo against JRuby --1.9 mode.) > session_reused?: > From the bit of research that I have done, it looks like SSL sessions are always reusable (unless > they're closed?), so is it safe to just return true? > session=: > It doesn't look like the SSL implementation in Java allow for this sort of session swapping. Could > I just throw UnsupportedOperationException here? Yeah, I don't think we can make SSLSession persistent in Java. Session cache is done by JCE in Java, so SSLSession ( JRUBY-4371 ) would be a read only object. Any suggestions how we go forward?
          Hide
          Duncan Mak added a comment -

          I tried test_pair.rb and test_ssl.rb:

          duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby-git --1.9 test_ssl.rb -v
          Loaded suite test_ssl
          Started
          OpenSSL::TestSSL#test_client_auth: 5.81 s: E
          OpenSSL::TestSSL#test_client_session: 5.14 s: E
          OpenSSL::TestSSL#test_connect_and_close: 5.14 s: E
          OpenSSL::TestSSL#test_ctx_setup: 0.04 s: .
          OpenSSL::TestSSL#test_not_started_session: 0.42 s: E
          OpenSSL::TestSSL#test_parallel: 5.12 s: E
          OpenSSL::TestSSL#test_post_connection_check: 5.10 s: E
          OpenSSL::TestSSL#test_read_and_write: 5.12 s: E
          OpenSSL::TestSSL#test_server_session: NameError: uninitialized constant OpenSSL::SSL::Session
                  const_missing at org/jruby/RubyModule.java:2569
                        session at /home/duncan/.rvm/gems/jruby-1.6.4/gems/jruby-openssl-0.7.5.dev/lib/openssl/ssl.rb:129
            test_server_session at test_ssl.rb:499
                           call at org/jruby/RubyProc.java:270
                           call at org/jruby/RubyProc.java:258
                    server_loop at test_ssl.rb:80
                           call at org/jruby/RubyProc.java:270
                           call at org/jruby/RubyProc.java:224
          
          duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby-git --1.9 test_pair.rb -v
          Loaded suite test_pair
          Started
          OpenSSL::TestEOF1#test_eof_0: 0.34 s: E
          OpenSSL::TestEOF1#test_eof_0_rw: 0.00 s: .
          OpenSSL::TestEOF1#test_eof_1: 0.10 s: E
          OpenSSL::TestEOF1#test_eof_2: 0.09 s: E
          OpenSSL::TestEOF1#test_eof_3: 0.09 s: E
          OpenSSL::TestEOF2#test_eof_0: 0.11 s: E
          OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: .
          OpenSSL::TestEOF2#test_eof_1: 0.10 s: E
          OpenSSL::TestEOF2#test_eof_2: 0.10 s: E
          OpenSSL::TestEOF2#test_eof_3: 0.10 s: E
          OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E
          OpenSSL::TestPair#test_getc: 0.10 s: E
          OpenSSL::TestPair#test_puts_empty: 0.12 s: E
          OpenSSL::TestPair#test_puts_meta: 0.11 s: E
          OpenSSL::TestPair#test_read_nonblock:
          

          And it hangs on test_read_nonblock - so I guess there's more work to be done.

          Show
          Duncan Mak added a comment - I tried test_pair.rb and test_ssl.rb: duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby-git --1.9 test_ssl.rb -v Loaded suite test_ssl Started OpenSSL::TestSSL#test_client_auth: 5.81 s: E OpenSSL::TestSSL#test_client_session: 5.14 s: E OpenSSL::TestSSL#test_connect_and_close: 5.14 s: E OpenSSL::TestSSL#test_ctx_setup: 0.04 s: . OpenSSL::TestSSL#test_not_started_session: 0.42 s: E OpenSSL::TestSSL#test_parallel: 5.12 s: E OpenSSL::TestSSL#test_post_connection_check: 5.10 s: E OpenSSL::TestSSL#test_read_and_write: 5.12 s: E OpenSSL::TestSSL#test_server_session: NameError: uninitialized constant OpenSSL::SSL::Session const_missing at org/jruby/RubyModule.java:2569 session at /home/duncan/.rvm/gems/jruby-1.6.4/gems/jruby-openssl-0.7.5.dev/lib/openssl/ssl.rb:129 test_server_session at test_ssl.rb:499 call at org/jruby/RubyProc.java:270 call at org/jruby/RubyProc.java:258 server_loop at test_ssl.rb:80 call at org/jruby/RubyProc.java:270 call at org/jruby/RubyProc.java:224 duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby-git --1.9 test_pair.rb -v Loaded suite test_pair Started OpenSSL::TestEOF1#test_eof_0: 0.34 s: E OpenSSL::TestEOF1#test_eof_0_rw: 0.00 s: . OpenSSL::TestEOF1#test_eof_1: 0.10 s: E OpenSSL::TestEOF1#test_eof_2: 0.09 s: E OpenSSL::TestEOF1#test_eof_3: 0.09 s: E OpenSSL::TestEOF2#test_eof_0: 0.11 s: E OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: . OpenSSL::TestEOF2#test_eof_1: 0.10 s: E OpenSSL::TestEOF2#test_eof_2: 0.10 s: E OpenSSL::TestEOF2#test_eof_3: 0.10 s: E OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E OpenSSL::TestPair#test_getc: 0.10 s: E OpenSSL::TestPair#test_puts_empty: 0.12 s: E OpenSSL::TestPair#test_puts_meta: 0.11 s: E OpenSSL::TestPair#test_read_nonblock: And it hangs on test_read_nonblock - so I guess there's more work to be done.
          Hide
          Duncan Mak added a comment -

          I made a few more changes, and now in my branch, I see 3 remaining errors when I run test_pair.rb from CRuby's test/openssl directory:

          duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby --1.9 test_pair.rb -v
          Loaded suite test_pair
          Started
          OpenSSL::TestEOF1#test_eof_0: 0.71 s: .
          OpenSSL::TestEOF1#test_eof_0_rw: 0.00 s: .
          OpenSSL::TestEOF1#test_eof_1: 0.87 s: .
          OpenSSL::TestEOF1#test_eof_2: 0.11 s: .
          OpenSSL::TestEOF1#test_eof_3: 0.11 s: .
          OpenSSL::TestEOF2#test_eof_0: 0.46 s: .
          OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: .
          OpenSSL::TestEOF2#test_eof_1: 0.89 s: .
          OpenSSL::TestEOF2#test_eof_2: 0.11 s: .
          OpenSSL::TestEOF2#test_eof_3: 0.10 s: .
          OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E
          OpenSSL::TestPair#test_getc: 0.09 s: .
          OpenSSL::TestPair#test_puts_empty: 0.10 s: .
          OpenSSL::TestPair#test_puts_meta: 0.11 s: .
          OpenSSL::TestPair#test_read_nonblock: 0.15 s: E
          OpenSSL::TestPair#test_readall: 0.10 s: .
          OpenSSL::TestPair#test_readline: 0.11 s: .
          OpenSSL::TestPair#test_readpartial: 0.11 s: .
          OpenSSL::TestPair#test_write_nonblock: 0.10 s: .
          OpenSSL::TestPair#test_write_nonblock_with_buffered_data: 0.11 s: E
          
          Finished in 4.397000 seconds.
          
            1) Error:
          test_connect_accept_nonblock(OpenSSL::TestPair):
          NoMethodError: undefined method `connect_address' for #<TCPServer:0x111f2041>
              test_pair.rb:200:in `__ensure__'
              org/jruby/RubyBasicObject.java:1685:in `__send__'
          
            2) Error:
          test_read_nonblock(OpenSSL::TestPair):
          OpenSSL::SSL::SSLError: read would raise
              org/jruby/ext/openssl/SSLSocket.java:562:in `sysread_nonblock'
              /home/duncan/.rvm/gems/jruby-1.6.4@ossl-dev/gems/jruby-openssl-0.7.5.dev/lib/openssl/buffering.rb:145:in `read_nonblock'
              test_pair.rb:157:in `test_read_nonblock'
              test_pair.rb:46:in `__ensure__'
              test_pair.rb:45:in `__ensure__'
              test_pair.rb:145:in `test_read_nonblock'
              org/jruby/RubyBasicObject.java:1685:in `__send__'
          
            3) Error:
          test_write_nonblock_with_buffered_data(OpenSSL::TestPair):
          OpenSSL::SSL::SSLError: write would raise
              org/jruby/ext/openssl/SSLSocket.java:600:in `syswrite_nonblock'
              /home/duncan/.rvm/gems/jruby-1.6.4@ossl-dev/gems/jruby-openssl-0.7.5.dev/lib/openssl/buffering.rb:292:in `write_nonblock'
              test_pair.rb:185:in `test_write_nonblock_with_buffered_data'
              test_pair.rb:46:in `__ensure__'
              test_pair.rb:45:in `__ensure__'
              test_pair.rb:183:in `test_write_nonblock_with_buffered_data'
              org/jruby/RubyBasicObject.java:1685:in `__send__'
          
          20 tests, 96 assertions, 0 failures, 3 errors, 0 skips
          
          Show
          Duncan Mak added a comment - I made a few more changes, and now in my branch, I see 3 remaining errors when I run test_pair.rb from CRuby's test/openssl directory: duncan@morpheme:~/Downloads/ruby-1.9.2-p290/test/openssl$ jruby --1.9 test_pair.rb -v Loaded suite test_pair Started OpenSSL::TestEOF1#test_eof_0: 0.71 s: . OpenSSL::TestEOF1#test_eof_0_rw: 0.00 s: . OpenSSL::TestEOF1#test_eof_1: 0.87 s: . OpenSSL::TestEOF1#test_eof_2: 0.11 s: . OpenSSL::TestEOF1#test_eof_3: 0.11 s: . OpenSSL::TestEOF2#test_eof_0: 0.46 s: . OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: . OpenSSL::TestEOF2#test_eof_1: 0.89 s: . OpenSSL::TestEOF2#test_eof_2: 0.11 s: . OpenSSL::TestEOF2#test_eof_3: 0.10 s: . OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E OpenSSL::TestPair#test_getc: 0.09 s: . OpenSSL::TestPair#test_puts_empty: 0.10 s: . OpenSSL::TestPair#test_puts_meta: 0.11 s: . OpenSSL::TestPair#test_read_nonblock: 0.15 s: E OpenSSL::TestPair#test_readall: 0.10 s: . OpenSSL::TestPair#test_readline: 0.11 s: . OpenSSL::TestPair#test_readpartial: 0.11 s: . OpenSSL::TestPair#test_write_nonblock: 0.10 s: . OpenSSL::TestPair#test_write_nonblock_with_buffered_data: 0.11 s: E Finished in 4.397000 seconds. 1) Error: test_connect_accept_nonblock(OpenSSL::TestPair): NoMethodError: undefined method `connect_address' for #<TCPServer:0x111f2041> test_pair.rb:200:in `__ensure__' org/jruby/RubyBasicObject.java:1685:in `__send__' 2) Error: test_read_nonblock(OpenSSL::TestPair): OpenSSL::SSL::SSLError: read would raise org/jruby/ext/openssl/SSLSocket.java:562:in `sysread_nonblock' /home/duncan/.rvm/gems/jruby-1.6.4@ossl-dev/gems/jruby-openssl-0.7.5.dev/lib/openssl/buffering.rb:145:in `read_nonblock' test_pair.rb:157:in `test_read_nonblock' test_pair.rb:46:in `__ensure__' test_pair.rb:45:in `__ensure__' test_pair.rb:145:in `test_read_nonblock' org/jruby/RubyBasicObject.java:1685:in `__send__' 3) Error: test_write_nonblock_with_buffered_data(OpenSSL::TestPair): OpenSSL::SSL::SSLError: write would raise org/jruby/ext/openssl/SSLSocket.java:600:in `syswrite_nonblock' /home/duncan/.rvm/gems/jruby-1.6.4@ossl-dev/gems/jruby-openssl-0.7.5.dev/lib/openssl/buffering.rb:292:in `write_nonblock' test_pair.rb:185:in `test_write_nonblock_with_buffered_data' test_pair.rb:46:in `__ensure__' test_pair.rb:45:in `__ensure__' test_pair.rb:183:in `test_write_nonblock_with_buffered_data' org/jruby/RubyBasicObject.java:1685:in `__send__' 20 tests, 96 assertions, 0 failures, 3 errors, 0 skips
          Hide
          Duncan Mak added a comment -

          To fix the first error, we need to also close JRUBY-5647.

          Show
          Duncan Mak added a comment - To fix the first error, we need to also close JRUBY-5647 .
          Hide
          Duncan Mak added a comment -

          To fix the remaining two errors, we need to make IO.select do what SSLSocket.waitSelect does.

          Show
          Duncan Mak added a comment - To fix the remaining two errors, we need to make IO.select do what SSLSocket.waitSelect does.
          Hide
          Hiroshi Nakamura added a comment - - edited

          If I understand correctly, that means that all internal (not imported) IO must be unblocking as same as CRuby 1.9... But I think you're right. That's the way to go.

          Show
          Hiroshi Nakamura added a comment - - edited If I understand correctly, that means that all internal (not imported) IO must be unblocking as same as CRuby 1.9... But I think you're right. That's the way to go.
          Hide
          Duncan Mak added a comment -

          Turns out the second error has also been filed, JRUBY-5165.

          Show
          Duncan Mak added a comment - Turns out the second error has also been filed, JRUBY-5165 .
          Hide
          Duncan Mak added a comment -

          I started working on JRUBY-5647.

          Show
          Duncan Mak added a comment - I started working on JRUBY-5647 .
          Hide
          Jordan Sissel added a comment -

          Ran into this today. Anything I can help with (Code, etc)?

          Show
          Jordan Sissel added a comment - Ran into this today. Anything I can help with (Code, etc)?
          Hide
          Nick Sieger added a comment -

          More people are running JRuby in 1.9 mode and are running into this issue. It would be nice to get this minimally working, at least.

          Show
          Nick Sieger added a comment - More people are running JRuby in 1.9 mode and are running into this issue. It would be nice to get this minimally working, at least.
          Hide
          Hiroshi Nakamura added a comment -

          Here's the branch Duncan fixes some issue for 1.9.
          https://github.com/duncanmak/jruby-ossl/commits/implement-nonblock-with-dual-lib-support

          The branch also includes my change for 1.8/1.9 double support in one gem: https://github.com/jruby/jruby-ossl/compare/310dd6311d...38d00c7cd7

          To support all of OpenSSL + non-blocking issue, we need to modify IO layer of JRuby itself. But the Duncan's branch would be good to release as an initial try for 1.9 mode.

          Nick, how do you think my 1.8/1.9 support part? If it's really the way to go, I'll merge Duncan's branch to master and push a new JRuby-OSSL gem.

          Show
          Hiroshi Nakamura added a comment - Here's the branch Duncan fixes some issue for 1.9. https://github.com/duncanmak/jruby-ossl/commits/implement-nonblock-with-dual-lib-support The branch also includes my change for 1.8/1.9 double support in one gem: https://github.com/jruby/jruby-ossl/compare/310dd6311d...38d00c7cd7 To support all of OpenSSL + non-blocking issue, we need to modify IO layer of JRuby itself. But the Duncan's branch would be good to release as an initial try for 1.9 mode. Nick, how do you think my 1.8/1.9 support part? If it's really the way to go, I'll merge Duncan's branch to master and push a new JRuby-OSSL gem.
          Hide
          Nick Sieger added a comment -

          NaHi, everything looks great. I had this issue on my agenda for this week already, so if you can make the merges and push a new gem that would be wonderful. Let me know if I can help in any way.

          Show
          Nick Sieger added a comment - NaHi, everything looks great. I had this issue on my agenda for this week already, so if you can make the merges and push a new gem that would be wonderful. Let me know if I can help in any way.
          Hide
          Ilya Grigorik added a comment -

          bump

          Hiroshi, Nick, anything we can help with here? Currently a blocker for a few projects we're looking at with JRuby.

          Show
          Ilya Grigorik added a comment - bump Hiroshi, Nick, anything we can help with here? Currently a blocker for a few projects we're looking at with JRuby.
          Hide
          Hiroshi Nakamura added a comment -

          Hi, sorry that I keep you waiting.

          Would you please try the attached gem built from Duncanmak's development branch? I've not yet tested this on other than Sun/Oracle's 6u29 though...

          Show
          Hiroshi Nakamura added a comment - Hi, sorry that I keep you waiting. Would you please try the attached gem built from Duncanmak's development branch? I've not yet tested this on other than Sun/Oracle's 6u29 though...
          Hide
          David Wood added a comment -

          Yeah, this is bad. Blows up aws-sdk, which uses SSL to speak with Amazon REST APIs.

          The referenced monkey patch appears to work; testing the attached gem will take a little longer.

          Show
          David Wood added a comment - Yeah, this is bad. Blows up aws-sdk, which uses SSL to speak with Amazon REST APIs. The referenced monkey patch appears to work; testing the attached gem will take a little longer.
          Hide
          Ben Macfarlane added a comment -

          Duncan's attached jruby-openssl 0.7.5.dev gem failed for me.
          System: (Mac OSX Lion / JRuby 1.6.5 64bit).
          Test - invoking Stripe's HTTPS-based RESTful payment service.

          JRuby (ruby 1.9.2) The 0.7.5.dev gem leaves JRuby in a hung state with high CPU usage when attempting an SSL request.
          JRuby (ruby 1.9.2) The 0.7.4 gem produces the "undefined method 'read_nonblock'" issue
          JRuby (ruby 1.8.7) - The 0.7.5.dev gem works correctly.
          JRuby (ruby 1.8.7) - The 0.7.4 gem works correctly

          Native Ruby 1.9.2 executes the code successfully.

          Let me know if there's anything I can do to help develop / test a fix on OSX.

          Show
          Ben Macfarlane added a comment - Duncan's attached jruby-openssl 0.7.5.dev gem failed for me. System: (Mac OSX Lion / JRuby 1.6.5 64bit). Test - invoking Stripe's HTTPS-based RESTful payment service. JRuby (ruby 1.9.2) The 0.7.5.dev gem leaves JRuby in a hung state with high CPU usage when attempting an SSL request. JRuby (ruby 1.9.2) The 0.7.4 gem produces the "undefined method 'read_nonblock'" issue JRuby (ruby 1.8.7) - The 0.7.5.dev gem works correctly. JRuby (ruby 1.8.7) - The 0.7.4 gem works correctly Native Ruby 1.9.2 executes the code successfully. Let me know if there's anything I can do to help develop / test a fix on OSX.
          Hide
          Ben Macfarlane added a comment -

          I've pulled the "implement-nonblock-with-dual-lib-support" (0.7.5.dev) branch from github and run the tests using JRuby in 1.9 mode on my Mac (OSX Lion / JRuby 1.6.5 64bit / Java build 1.6.0_26-b03-383-11A511) The tests that cause the process to hang with high CPU are:
          test_ssl.rb :
          test_jruby_4826
          test_integration.rb :
          test_ca_path_name
          test_ssl_verify
          test_pathlen_does_not_appear

          There's a mix of pass/fail/error for the other tests on the platform. I hope to get around to looking more closely at the results in another week or so.

          Show
          Ben Macfarlane added a comment - I've pulled the "implement-nonblock-with-dual-lib-support" (0.7.5.dev) branch from github and run the tests using JRuby in 1.9 mode on my Mac (OSX Lion / JRuby 1.6.5 64bit / Java build 1.6.0_26-b03-383-11A511) The tests that cause the process to hang with high CPU are: test_ssl.rb : test_jruby_4826 test_integration.rb : test_ca_path_name test_ssl_verify test_pathlen_does_not_appear There's a mix of pass/fail/error for the other tests on the platform. I hope to get around to looking more closely at the results in another week or so.
          Hide
          Ilya Grigorik added a comment -

          igrigorik

          { ~/Desktop/undefined_method_read_nonblock } > ruby -v
          jruby 1.7.0.dev (ruby-1.9.3-p0) (2011-12-19 cdb3e75) (OpenJDK 64-Bit Server VM 1.7.0-u2-b21) [darwin-amd64-java]
          igrigorik { ~/Desktop/undefined_method_read_nonblock }

          > gem list jruby-openssl
          jruby-openssl (0.7.5.dev, 0.7.4)

          Trying to run a simple net/https connection, which hangs the process indefinitely: https://gist.github.com/d034dd170b02037c0a9f

          Installed jruby-head is up to date as of yesterday.

          Ben: Any luck from your side for this?

          Show
          Ilya Grigorik added a comment - igrigorik { ~/Desktop/undefined_method_read_nonblock } > ruby -v jruby 1.7.0.dev (ruby-1.9.3-p0) (2011-12-19 cdb3e75) (OpenJDK 64-Bit Server VM 1.7.0-u2-b21) [darwin-amd64-java] igrigorik { ~/Desktop/undefined_method_read_nonblock } > gem list jruby-openssl jruby-openssl (0.7.5.dev, 0.7.4) Trying to run a simple net/https connection, which hangs the process indefinitely: https://gist.github.com/d034dd170b02037c0a9f Installed jruby-head is up to date as of yesterday. Ben: Any luck from your side for this?
          Hide
          Hiroshi Nakamura added a comment -

          I applied an one-off fix to jruby-1_6 for this issue.

          commit 5dbab19d45b89b959532684c71b98e73e254b39f
          Author: Hiroshi Nakamura <nahi@ruby-lang.org>
          Date:   Tue Dec 27 00:36:58 2011 +0900
          
              [1.9] One-off fix for read_nonblock CPU 100% issue
              
              Replace "read_nonblock + IO.select timeout" in 1.9 stdlib with
              "Timeout.timeout + sysread" in 1.8's one.  With current JRuby,
              IO.select([io], nil, nil, timeout) could return promptly even if io is
              not readable. (I'm not sure the condition when it happens at this
              moment.)
              
              Because of this IO.select behavior, rbuf_fill in 1.9/net/protocol.rb
              fails to busy loop of io.read_nonblock and IO.select.  As a workaround,
              this commit replaces rbuf_fill of 1.9 stdlib with 1.8's one which uses
              Timeout + sysread instead.
              
              sysread_nonblock + IO.select is introduced from perf concern.  JRuby has
              a better Timeout logic so this change should not affect users.
              
              I apply this change only for jruby-1_6 branch.  For 1.7, I'll try to
              solve this issue by fixing nonblocking IO at master.
          

          Can someone try again with jruby-1.6.6.dev at http://ci.jruby.org/job/jruby-dist-release/lastSuccessfulBuild/artifact/ ?

          If you're using master, please apply the same patch for a workaround: https://github.com/jruby/jruby/commit/5dbab19d45b89b959532684c71b98e73e254b39f

          Show
          Hiroshi Nakamura added a comment - I applied an one-off fix to jruby-1_6 for this issue. commit 5dbab19d45b89b959532684c71b98e73e254b39f Author: Hiroshi Nakamura <nahi@ruby-lang.org> Date: Tue Dec 27 00:36:58 2011 +0900 [1.9] One-off fix for read_nonblock CPU 100% issue Replace "read_nonblock + IO.select timeout" in 1.9 stdlib with "Timeout.timeout + sysread" in 1.8's one. With current JRuby, IO.select([io], nil, nil, timeout) could return promptly even if io is not readable. (I'm not sure the condition when it happens at this moment.) Because of this IO.select behavior, rbuf_fill in 1.9/net/protocol.rb fails to busy loop of io.read_nonblock and IO.select. As a workaround, this commit replaces rbuf_fill of 1.9 stdlib with 1.8's one which uses Timeout + sysread instead. sysread_nonblock + IO.select is introduced from perf concern. JRuby has a better Timeout logic so this change should not affect users. I apply this change only for jruby-1_6 branch. For 1.7, I'll try to solve this issue by fixing nonblocking IO at master. Can someone try again with jruby-1.6.6.dev at http://ci.jruby.org/job/jruby-dist-release/lastSuccessfulBuild/artifact/ ? If you're using master, please apply the same patch for a workaround: https://github.com/jruby/jruby/commit/5dbab19d45b89b959532684c71b98e73e254b39f
          Hide
          Thomas E Enebo added a comment -

          Indicating this will be a fixed issue for 1.6.6 release.

          Show
          Thomas E Enebo added a comment - Indicating this will be a fixed issue for 1.6.6 release.
          Hide
          bauagonzo added a comment -

          It works with :

          jruby 1.6.6.dev (ruby-1.9.2-p136) (2011-12-28 2fc3a13) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_10) [linux-amd64-java]

          Thanks

          Show
          bauagonzo added a comment - It works with : jruby 1.6.6.dev (ruby-1.9.2-p136) (2011-12-28 2fc3a13) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_10) [linux-amd64-java] Thanks
          Hide
          Alex Khesin added a comment -

          Hiroshi's commit got reverted a few days later by

          https://github.com/jruby/jruby/commit/be3f90ff7b48ad54662a20f8360ff37139ab75fb

          so it is broken again

          Show
          Alex Khesin added a comment - Hiroshi's commit got reverted a few days later by https://github.com/jruby/jruby/commit/be3f90ff7b48ad54662a20f8360ff37139ab75fb so it is broken again
          Hide
          Hiro Asari added a comment -

          I cherry-picked NaHi's commit again to the 1.6 branch: e29ca705a4.

          Show
          Hiro Asari added a comment - I cherry-picked NaHi's commit again to the 1.6 branch: e29ca705a4.
          Hide
          Charles Oliver Nutter added a comment -

          We will put JRuby 1.6.6 out next week, which includes a number of other 1.9 fixes. It would be great if we could get jruby-openssl out too with its 1.9 fixes.

          Nahi: What can I do to help get jossl released?

          Show
          Charles Oliver Nutter added a comment - We will put JRuby 1.6.6 out next week, which includes a number of other 1.9 fixes. It would be great if we could get jruby-openssl out too with its 1.9 fixes. Nahi: What can I do to help get jossl released?
          Hide
          Duncan Mak added a comment -

          Nahi - yeah, let me know also. I'm free this weekend, and would love to put in the finishing touches.

          Show
          Duncan Mak added a comment - Nahi - yeah, let me know also. I'm free this weekend, and would love to put in the finishing touches.
          Hide
          Charles Oliver Nutter added a comment -

          Hiro: Did this change make it into our Ruby stdlib fork? If not, it will get overwritten again.

          Show
          Charles Oliver Nutter added a comment - Hiro: Did this change make it into our Ruby stdlib fork? If not, it will get overwritten again.
          Hide
          Hiroshi Nakamura added a comment -

          Thanks Charles and Duncan. I report the status here.

          I've already merged Duncan's fixes into master branch of jruby-ossl. New release of jruby-ossl gem includes both 1.8 and 1.9 libraries and part of features should work fine on 1.9 mode, too.

          For read_nonblock issue, the fix is needed for jruby's lib/ruby/1.9/net/protocol.rb (5dbab19d45b89b959532684c71b98e73e254b39f for jruby master) for a while. Eventually I need to investigate non-blocking IO of JRuby IO subsystem, but it would need more time. For more details, please see the commit log of about workaround fix. Sorry I can't spent enough time to investigate this issue.

          I didn't merge the fix to our Ruby stdlib fork, since I thought it should be one-off fix for jruby-1_6 (I don't want it affect to master.) Since it would need more time to fix, I'll update our stdlib fork later.

          I was going to release 0.7.5 but I cannot share time for releasing. I'll push it in weekend. Here's History.txt for 0.7.5.

          This release improved 1.9 mode support with help of
          Duncan Mak <duncan@earthaid.net>.  Now jruby-ossl gem includes both 1.8 and 1.9
          libraries and part of features should work fine on 1.9 mode, too.
          
          - JRUBY-6270: Wrong keyUsage check for SSL server
          - JRUBY-6260: OpenSSL::ASN1::Integer#value incompatibility
          - JRUBY-6044: Improve Ecrypted RSA/DSA pem support
          - JRUBY-5972: Allow to load/dump empty PKCS7 data
          - JRUBY-5833: Fix X509Name handling; X509Name RDN can include multiple elements
          - JRUBY-5362: Improved 1.9 support
          - JRUBY-4992: Warn if loaded by non JRuby interpreter
          

          Duncan, 1.9 support is not yet finished. Would you please see TODO-1_9-support.txt in the repo and try to solve some issue? You can run 1.9 tests by; jruby --1.9 -S rake test

          Please bear in mind that you need to apply 5dbab19d45b89b959532684c71b98e73e254b39f to jruby for read_nonblock issue.

          Show
          Hiroshi Nakamura added a comment - Thanks Charles and Duncan. I report the status here. I've already merged Duncan's fixes into master branch of jruby-ossl. New release of jruby-ossl gem includes both 1.8 and 1.9 libraries and part of features should work fine on 1.9 mode, too. For read_nonblock issue, the fix is needed for jruby's lib/ruby/1.9/net/protocol.rb (5dbab19d45b89b959532684c71b98e73e254b39f for jruby master) for a while. Eventually I need to investigate non-blocking IO of JRuby IO subsystem, but it would need more time. For more details, please see the commit log of about workaround fix. Sorry I can't spent enough time to investigate this issue. I didn't merge the fix to our Ruby stdlib fork, since I thought it should be one-off fix for jruby-1_6 (I don't want it affect to master.) Since it would need more time to fix, I'll update our stdlib fork later. I was going to release 0.7.5 but I cannot share time for releasing. I'll push it in weekend. Here's History.txt for 0.7.5. This release improved 1.9 mode support with help of Duncan Mak <duncan@earthaid.net>. Now jruby-ossl gem includes both 1.8 and 1.9 libraries and part of features should work fine on 1.9 mode, too. - JRUBY-6270: Wrong keyUsage check for SSL server - JRUBY-6260: OpenSSL::ASN1::Integer#value incompatibility - JRUBY-6044: Improve Ecrypted RSA/DSA pem support - JRUBY-5972: Allow to load/dump empty PKCS7 data - JRUBY-5833: Fix X509Name handling; X509Name RDN can include multiple elements - JRUBY-5362: Improved 1.9 support - JRUBY-4992: Warn if loaded by non JRuby interpreter Duncan, 1.9 support is not yet finished. Would you please see TODO-1_9-support.txt in the repo and try to solve some issue? You can run 1.9 tests by; jruby --1.9 -S rake test Please bear in mind that you need to apply 5dbab19d45b89b959532684c71b98e73e254b39f to jruby for read_nonblock issue.
          Hide
          Xavier Shay added a comment -

          Commenting so I get notified of updates. FWIW I'm using Karl's monkey-patch for the time being.

          Show
          Xavier Shay added a comment - Commenting so I get notified of updates. FWIW I'm using Karl's monkey-patch for the time being.
          Hide
          Hiro Asari added a comment -

          I re-cherry-picked (is that a word?) NaHi's commit to the master branch: 9db86c6.

          Show
          Hiro Asari added a comment - I re-cherry-picked (is that a word?) NaHi's commit to the master branch: 9db86c6.
          Hide
          Charles Oliver Nutter added a comment -

          FWIW I also pulled the change into our stdlib fork so it won't go away. jruby-ruby_1_9_3@a637894 and jruby-ruby_1_9_2@f844a14.

          Show
          Charles Oliver Nutter added a comment - FWIW I also pulled the change into our stdlib fork so it won't go away. jruby-ruby_1_9_3@a637894 and jruby-ruby_1_9_2@f844a14.
          Hide
          Matt Hauck added a comment -

          I have been looking into using syswrite_nonblock as well, and I don't think that this implementation is correct. It raises an error each time it is called, whose .inspect returns this: #<#<Class:0x2940943>: write would raise>

          It looks like it is trying to raise an IO::WaitWritable each time it is run, but this is not correct either. It looks like this is trying to repeat the following C code:

          if (errno == EWOULDBLOCK || errno == EAGAIN)
          rb_mod_sys_fail(rb_mWaitWritable, "write would block");

          It should only raise a WaitWritable (or WaitReadable for read_nonblock) in the case of EWOULDBLOCK/EAGAIN.

          Show
          Matt Hauck added a comment - I have been looking into using syswrite_nonblock as well, and I don't think that this implementation is correct. It raises an error each time it is called, whose .inspect returns this: #<#<Class:0x2940943>: write would raise> It looks like it is trying to raise an IO::WaitWritable each time it is run, but this is not correct either. It looks like this is trying to repeat the following C code: if (errno == EWOULDBLOCK || errno == EAGAIN) rb_mod_sys_fail(rb_mWaitWritable, "write would block"); It should only raise a WaitWritable (or WaitReadable for read_nonblock) in the case of EWOULDBLOCK/EAGAIN.
          Hide
          Matt Hauck added a comment -

          This patch is more along the lines of my understanding of what read/write_nonblock are supposed to do.

          Show
          Matt Hauck added a comment - This patch is more along the lines of my understanding of what read/write_nonblock are supposed to do.
          Hide
          Matt Hauck added a comment -

          Looking more at CRuby's implementation, I am more convinced the current implementation is wrong. write_would_block is only called if ssl_get_error returns SSL_ERROR_WANT_WRITE and likewise for read_would_block.

          Show
          Matt Hauck added a comment - Looking more at CRuby's implementation, I am more convinced the current implementation is wrong. write_would_block is only called if ssl_get_error returns SSL_ERROR_WANT_WRITE and likewise for read_would_block.
          Hide
          Matt Hauck added a comment -

          Re-implemented read_nonblock and write_nonblock here: https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8

          test_pair now succeeds all read_nonblock/write_nonblock tests.

          jruby --1.9 ~/Ruby/ruby-1.9.2-p290/test/openssl/test_pair.rb -v
          Loaded suite /Users/mhauck/Ruby/ruby-1.9.2-p290/test/openssl/test_pair
          Started
          OpenSSL::TestEOF1#test_eof_0: 0.41 s: .
          OpenSSL::TestEOF1#test_eof_0_rw: 0.01 s: .
          OpenSSL::TestEOF1#test_eof_1: 0.18 s: .
          OpenSSL::TestEOF1#test_eof_2: 0.02 s: .
          OpenSSL::TestEOF1#test_eof_3: 0.02 s: .
          OpenSSL::TestEOF2#test_eof_0: 0.10 s: .
          OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: .
          OpenSSL::TestEOF2#test_eof_1: 0.18 s: .
          OpenSSL::TestEOF2#test_eof_2: 0.02 s: .
          OpenSSL::TestEOF2#test_eof_3: 0.02 s: .
          OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E
          OpenSSL::TestPair#test_getc: 0.02 s: .
          OpenSSL::TestPair#test_puts_empty: 0.02 s: .
          OpenSSL::TestPair#test_puts_meta: 0.02 s: .
          OpenSSL::TestPair#test_read_nonblock: 0.03 s: .
          OpenSSL::TestPair#test_readall: 0.02 s: .
          OpenSSL::TestPair#test_readline: 0.02 s: .
          OpenSSL::TestPair#test_readpartial: 0.02 s: .
          OpenSSL::TestPair#test_write_nonblock: 0.03 s: .
          OpenSSL::TestPair#test_write_nonblock_with_buffered_data: 0.02 s: .
          
          Finished in 1.185000 seconds.
          
            1) Error:
          test_connect_accept_nonblock(OpenSSL::TestPair):
          NoMethodError: undefined method `connect_address' for #<TCPServer:0x399b08cc>
              /Users/mhauck/Ruby/ruby-1.9.2-p290/test/openssl/test_pair.rb:200:in `__ensure__'
              org/jruby/RubyBasicObject.java:1688:in `__send__'
          
          20 tests, 101 assertions, 0 failures, 1 errors, 0 skips
          
          Show
          Matt Hauck added a comment - Re-implemented read_nonblock and write_nonblock here: https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8 test_pair now succeeds all read_nonblock/write_nonblock tests. jruby --1.9 ~/Ruby/ruby-1.9.2-p290/test/openssl/test_pair.rb -v Loaded suite /Users/mhauck/Ruby/ruby-1.9.2-p290/test/openssl/test_pair Started OpenSSL::TestEOF1#test_eof_0: 0.41 s: . OpenSSL::TestEOF1#test_eof_0_rw: 0.01 s: . OpenSSL::TestEOF1#test_eof_1: 0.18 s: . OpenSSL::TestEOF1#test_eof_2: 0.02 s: . OpenSSL::TestEOF1#test_eof_3: 0.02 s: . OpenSSL::TestEOF2#test_eof_0: 0.10 s: . OpenSSL::TestEOF2#test_eof_0_rw: 0.00 s: . OpenSSL::TestEOF2#test_eof_1: 0.18 s: . OpenSSL::TestEOF2#test_eof_2: 0.02 s: . OpenSSL::TestEOF2#test_eof_3: 0.02 s: . OpenSSL::TestPair#test_connect_accept_nonblock: 0.01 s: E OpenSSL::TestPair#test_getc: 0.02 s: . OpenSSL::TestPair#test_puts_empty: 0.02 s: . OpenSSL::TestPair#test_puts_meta: 0.02 s: . OpenSSL::TestPair#test_read_nonblock: 0.03 s: . OpenSSL::TestPair#test_readall: 0.02 s: . OpenSSL::TestPair#test_readline: 0.02 s: . OpenSSL::TestPair#test_readpartial: 0.02 s: . OpenSSL::TestPair#test_write_nonblock: 0.03 s: . OpenSSL::TestPair#test_write_nonblock_with_buffered_data: 0.02 s: . Finished in 1.185000 seconds. 1) Error: test_connect_accept_nonblock(OpenSSL::TestPair): NoMethodError: undefined method `connect_address' for #<TCPServer:0x399b08cc> /Users/mhauck/Ruby/ruby-1.9.2-p290/test/openssl/test_pair.rb:200:in `__ensure__' org/jruby/RubyBasicObject.java:1688:in `__send__' 20 tests, 101 assertions, 0 failures, 1 errors, 0 skips
          Hide
          Hiroshi Nakamura added a comment -

          Thanks Matt, test result improvement sounds nice! I'm going to check your branch and merge next week. Sorry for the delay.

          Show
          Hiroshi Nakamura added a comment - Thanks Matt, test result improvement sounds nice! I'm going to check your branch and merge next week. Sorry for the delay.
          Hide
          Alex Tambellini added a comment -

          I just ran into undefined method write_nonblock when upgrading from jruby-openssl 0.7.4 to 0.7.5 when running jruby in 1.8 mode. I made an issue for it here: https://jira.codehaus.org/browse/JRUBY-6391. Feel free to mark it as a duplicate if you are already aware of it.

          Show
          Alex Tambellini added a comment - I just ran into undefined method write_nonblock when upgrading from jruby-openssl 0.7.4 to 0.7.5 when running jruby in 1.8 mode. I made an issue for it here: https://jira.codehaus.org/browse/JRUBY-6391 . Feel free to mark it as a duplicate if you are already aware of it.
          Hide
          Hiroshi Nakamura added a comment -

          Thanks Matt for updated patch. https://github.com/jruby/jruby-ossl/pull/14 and https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8 is the latest patch, right?

          I confirmed the fix, but having checked the patch, I'm thinking the fix should be done against jruby itself.

          The original waitSelect() in openssl was a copy of RubyThread.select as I wrote in comment. It was a temporary fix for SSLSocket, until we refactor RubyIO for better non-blocking support. Anyway I need to investigate why waitSelect() was broken, because received a bug report that timeout does not work against SSLSocket on jruby 1.6.5 + jruby-openssl 0.7.5.

          Do you think you can apply your patch against jruby and how it works? Or shall I try it?

          Show
          Hiroshi Nakamura added a comment - Thanks Matt for updated patch. https://github.com/jruby/jruby-ossl/pull/14 and https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8 is the latest patch, right? I confirmed the fix, but having checked the patch, I'm thinking the fix should be done against jruby itself. The original waitSelect() in openssl was a copy of RubyThread.select as I wrote in comment. It was a temporary fix for SSLSocket, until we refactor RubyIO for better non-blocking support. Anyway I need to investigate why waitSelect() was broken, because received a bug report that timeout does not work against SSLSocket on jruby 1.6.5 + jruby-openssl 0.7.5. Do you think you can apply your patch against jruby and how it works? Or shall I try it?
          Hide
          Matt Hauck added a comment -

          Hmm, I'm not immediately sure how RubyThread and RubyIO would integrate together. I can can take a look tonight perhaps, but if you know what to do with it, then you can try it.

          Show
          Matt Hauck added a comment - Hmm, I'm not immediately sure how RubyThread and RubyIO would integrate together. I can can take a look tonight perhaps, but if you know what to do with it, then you can try it.
          Hide
          Hiroshi Nakamura added a comment -

          For blocking IO in Ruby, IO.select needs to change Thread state to "sleep" and take care interruption by other Thread (like timeout.)

          For no-blocking IO, we can use selectNow() as you patched, but there are some common logic that should be shared with blocking IO (that's the reason why you change waitSelect(), not just using selectNow() in *_nonblock() methods I guess.)

          I think it's a good chance to merge jruby-ossl's waitSelect() with RubyThread.select() again, on the basis of the fact that we have now a workaround (net/http modification.)

          I'll try it by myself, too.

          Show
          Hiroshi Nakamura added a comment - For blocking IO in Ruby, IO.select needs to change Thread state to "sleep" and take care interruption by other Thread (like timeout.) For no-blocking IO, we can use selectNow() as you patched, but there are some common logic that should be shared with blocking IO (that's the reason why you change waitSelect(), not just using selectNow() in *_nonblock() methods I guess.) I think it's a good chance to merge jruby-ossl's waitSelect() with RubyThread.select() again, on the basis of the fact that we have now a workaround (net/http modification.) I'll try it by myself, too.
          Hide
          Ben Porterfield added a comment -

          Bump! This is a big blocker for us.

          I do not see undefined method, but instead always get error (read would raise), when I'm expected read would block. Karl's patch does not seem to help me with this case either.

          Matt, I did not have success with your patch - resulted in some strange errors (happy to provide detail if needed).

          Anything I can do to help?

          Show
          Ben Porterfield added a comment - Bump! This is a big blocker for us. I do not see undefined method, but instead always get error (read would raise), when I'm expected read would block. Karl's patch does not seem to help me with this case either. Matt, I did not have success with your patch - resulted in some strange errors (happy to provide detail if needed). Anything I can do to help?
          Hide
          Hiroshi Nakamura added a comment -

          Current status: net/protocol and dependents (e.g. net/http) should work now by rbuf_fill patch but SSLSocket#read_nonblock is still not yet implemented properly.

          I wrote "I'll try" but I still cannot promise the date I can try because of my primary job.

          Now JRuby-OSSL is under the process of merging back to JRuby (JRUBY-5035, master branch for 1.7) and it might help to investigate and fix the problem (non-blocking and SSLSocket) at master branch. I keep the "Assignee" of this ticket but feel free to take over. I think I can review fixes proposed.

          I'll write again when I can try to investigate this issue again.

          Show
          Hiroshi Nakamura added a comment - Current status: net/protocol and dependents (e.g. net/http) should work now by rbuf_fill patch but SSLSocket#read_nonblock is still not yet implemented properly. I wrote "I'll try" but I still cannot promise the date I can try because of my primary job. Now JRuby-OSSL is under the process of merging back to JRuby ( JRUBY-5035 , master branch for 1.7) and it might help to investigate and fix the problem (non-blocking and SSLSocket) at master branch. I keep the "Assignee" of this ticket but feel free to take over. I think I can review fixes proposed. I'll write again when I can try to investigate this issue again.
          Hide
          Konstantin Haase added a comment -

          Any ETA for a fix? This is a big blocker for us at Travis CI atm.

          FYI: This issue surfaces when using net-http-persistent.

          Show
          Konstantin Haase added a comment - Any ETA for a fix? This is a big blocker for us at Travis CI atm. FYI: This issue surfaces when using net-http-persistent.
          Hide
          Kevin Menard added a comment -

          Also an issue for excon and consequently fog.

          Show
          Kevin Menard added a comment - Also an issue for excon and consequently fog.
          Hide
          Charles Oliver Nutter added a comment -

          The nonblock methods were added as of 1.7pre2, so I'm going to call this resolved. The provided test case (kbaum repository) appears to work properly now.

          There may still be issues with the implementation of the various nonblocking methods, but since this method was about their lack of existence and inability to use a library that needs them, I think it's fine to resolve. New bugs should be filed as new issues.

          Show
          Charles Oliver Nutter added a comment - The nonblock methods were added as of 1.7pre2, so I'm going to call this resolved. The provided test case (kbaum repository) appears to work properly now. There may still be issues with the implementation of the various nonblocking methods, but since this method was about their lack of existence and inability to use a library that needs them, I think it's fine to resolve. New bugs should be filed as new issues.
          Hide
          Matt Hauck added a comment -

          I am skeptical that this has truly been implemented as it still uses the previous erroneous implementation. I am getting "Write would block" on every write_nonblock and "Read would block" on every read_nonblock call...

          Show
          Matt Hauck added a comment - I am skeptical that this has truly been implemented as it still uses the previous erroneous implementation. I am getting "Write would block" on every write_nonblock and "Read would block" on every read_nonblock call...
          Hide
          Matt Hauck added a comment -

          I would really like to re-open this bug rather than open a new one since it is the same issue. Raising an exception every time a method is called is no better than not defining the method to begin with... especially since it appears that in jruby 1.7 we are not able to replace jruby-openssl with our own forked version, a fork much easier to maintain than a whole jruby fork...

          Show
          Matt Hauck added a comment - I would really like to re-open this bug rather than open a new one since it is the same issue. Raising an exception every time a method is called is no better than not defining the method to begin with... especially since it appears that in jruby 1.7 we are not able to replace jruby-openssl with our own forked version, a fork much easier to maintain than a whole jruby fork...
          Hide
          Charles Oliver Nutter added a comment -

          You're right...I am ashamed that we claimed it was working based on my half-assed attempt.

          We can do this right, but it will take more time to understand the lifecycle of an SSL handshake and how it plays into nonblocking/partial IO. Reopening and marking for JRuby 1.7.1.

          Show
          Charles Oliver Nutter added a comment - You're right...I am ashamed that we claimed it was working based on my half-assed attempt. We can do this right, but it will take more time to understand the lifecycle of an SSL handshake and how it plays into nonblocking/partial IO. Reopening and marking for JRuby 1.7.1.
          Hide
          Charles Oliver Nutter added a comment -

          I have pushed a quick fix for this to the post17 branch that will go into 1.7.1. For a simple script (below) it works properly now. I'm going to proceed to running the openssl tests to work out more issues.

          Commit:

          commit b4af8a56aea8d3b8031bef7fdd364565e543f491
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Oct 24 13:12:45 2012 -0500
          
              Fix read_nonblock to actually attempt the read, rather than raise.
          
          :100644 100644 7c175fc... 47d3ec3... M	src/org/jruby/ext/openssl/SSLSocket.java
          

          Script:

          require 'openssl'
          require 'socket'
          
          Thread.abort_on_exception = true
          
          server = TCPServer.new(10000)
          s = nil
          Thread.new { s = server.accept }
          c = TCPSocket.new('localhost', 10000)
          1 until s
          
          c.puts('hello')
          puts s.gets
          
          # connection is established
          
          ss = OpenSSL::SSL::SSLSocket.new(s)
          cs = OpenSSL::SSL::SSLSocket.new(c)
          
          Thread.new do
            while true
              begin
                puts ss.read_nonblock(1000)
              rescue IO::WaitReadable
                puts "waiting for read"
                sleep 0.5
                retry
              end
            end
          end
          
          loop do
            sleep 1
            cs.write('hello')
          end
          
          Show
          Charles Oliver Nutter added a comment - I have pushed a quick fix for this to the post17 branch that will go into 1.7.1. For a simple script (below) it works properly now. I'm going to proceed to running the openssl tests to work out more issues. Commit: commit b4af8a56aea8d3b8031bef7fdd364565e543f491 Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Oct 24 13:12:45 2012 -0500 Fix read_nonblock to actually attempt the read, rather than raise. :100644 100644 7c175fc... 47d3ec3... M src/org/jruby/ext/openssl/SSLSocket.java Script: require 'openssl' require 'socket' Thread .abort_on_exception = true server = TCPServer. new (10000) s = nil Thread . new { s = server.accept } c = TCPSocket. new ('localhost', 10000) 1 until s c.puts('hello') puts s.gets # connection is established ss = OpenSSL::SSL::SSLSocket. new (s) cs = OpenSSL::SSL::SSLSocket. new (c) Thread . new do while true begin puts ss.read_nonblock(1000) rescue IO::WaitReadable puts "waiting for read" sleep 0.5 retry end end end loop do sleep 1 cs.write('hello') end
          Hide
          Charles Oliver Nutter added a comment -

          Ok, with the above commit and one additional, I'm able to run the read_nonblock test from test_ssl (along with a few others) and I'd really like to say we're good here now.

          commit f3c7847e16f5ce026d96be4259cf4e41034c51bf
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Oct 24 14:51:28 2012 -0500
          
              More fixes for SSLSocket.
              
              * Treat TCPServer#shutdown as unsupported by raising ENOTCONN
              * Unmask several test_ssl tests that pass now.
          
          :100644 100644 de79e18... 9520b70... M	src/org/jruby/ext/socket/RubyTCPServer.java
          :100644 100644 55c1485... 34411ac... M	test/externals/ruby1.9/excludes/OpenSSL/TestSSL.rb
          
          commit b4af8a56aea8d3b8031bef7fdd364565e543f491
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Oct 24 13:12:45 2012 -0500
          
              Fix read_nonblock to actually attempt the read, rather than raise.
          
          :100644 100644 7c175fc... 47d3ec3... M	src/org/jruby/ext/openssl/SSLSocket.java
          
          Show
          Charles Oliver Nutter added a comment - Ok, with the above commit and one additional, I'm able to run the read_nonblock test from test_ssl (along with a few others) and I'd really like to say we're good here now. commit f3c7847e16f5ce026d96be4259cf4e41034c51bf Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Oct 24 14:51:28 2012 -0500 More fixes for SSLSocket. * Treat TCPServer#shutdown as unsupported by raising ENOTCONN * Unmask several test_ssl tests that pass now. :100644 100644 de79e18... 9520b70... M src/org/jruby/ext/socket/RubyTCPServer.java :100644 100644 55c1485... 34411ac... M test/externals/ruby1.9/excludes/OpenSSL/TestSSL.rb commit b4af8a56aea8d3b8031bef7fdd364565e543f491 Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Oct 24 13:12:45 2012 -0500 Fix read_nonblock to actually attempt the read, rather than raise. :100644 100644 7c175fc... 47d3ec3... M src/org/jruby/ext/openssl/SSLSocket.java
          Hide
          Matt Hauck added a comment -

          What about for writing? Not sure if this got lost in the conversation, but cf. my previous suggested implementation: https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8

          Show
          Matt Hauck added a comment - What about for writing? Not sure if this got lost in the conversation, but cf. my previous suggested implementation: https://github.com/matthauck/jruby-ossl/commit/154d6b316867af142d7aedba997b286ecf5506e8
          Hide
          Charles Oliver Nutter added a comment -

          Matt: Your logic looks sound. I probably just missed it in previous conversations. I'll look at getting it (or a form of it) incorporated.

          Show
          Charles Oliver Nutter added a comment - Matt: Your logic looks sound. I probably just missed it in previous conversations. I'll look at getting it (or a form of it) incorporated.
          Hide
          Charles Oliver Nutter added a comment -

          My earlier commits did half of what yours does, and then the following two commits incorporate your additional changes plus some tweaks to allow interrupting read/write threads.

          commit d7deae6aa78797515bcc86223b8e0fa1a1537272
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Oct 24 19:44:03 2012 -0500
          
              Tweaks to previous commit to support interrupting blocked threads.
          
          :100644 100644 da88636... c095582... M	src/org/jruby/ext/openssl/SSLSocket.java
          
          commit 70921f420e6bb1baefbb32c2734eae108fc62e19
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Oct 24 19:24:39 2012 -0500
          
              Incorporate Matt Hauck's SSLSocket nonblock read/write cleanups.
          
          :100644 100644 ec64ed2... da88636... M	src/org/jruby/ext/openssl/SSLSocket.java
          
          Show
          Charles Oliver Nutter added a comment - My earlier commits did half of what yours does, and then the following two commits incorporate your additional changes plus some tweaks to allow interrupting read/write threads. commit d7deae6aa78797515bcc86223b8e0fa1a1537272 Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Oct 24 19:44:03 2012 -0500 Tweaks to previous commit to support interrupting blocked threads. :100644 100644 da88636... c095582... M src/org/jruby/ext/openssl/SSLSocket.java commit 70921f420e6bb1baefbb32c2734eae108fc62e19 Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Oct 24 19:24:39 2012 -0500 Incorporate Matt Hauck's SSLSocket nonblock read/write cleanups. :100644 100644 ec64ed2... da88636... M src/org/jruby/ext/openssl/SSLSocket.java
          Hide
          Matt Hauck added a comment -

          Sweet. Thanks. =)

          Show
          Matt Hauck added a comment - Sweet. Thanks. =)

            People

            • Assignee:
              Charles Oliver Nutter
              Reporter:
              Venkateshwaralu Srikarunyan
            • Votes:
              17 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: