JRuby

SSLSocket.accept forces reverse DNS lookup, not required for most SSL and sometimes causes very poor performance

Details

  • Type: Bug Bug
  • Status: Reopened Reopened
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: JRuby 1.7.0.pre2
  • Fix Version/s: JRuby 1.7.4
  • Component/s: OpenSSL, Ruby 1.9.3
  • Labels:
    None
  • Number of attachments :
    0

Description

Came across issue when a new client was getting ~10s delays before page loads. They were hitting an IP in their network that did not have reverse DNS name associated with it. Turned out that requests were being made by JRuby SSL to do reverse DNS lookup on the IP, and those requests were timing out before continuing with the request.

https://github.com/jruby/jruby/blob/master/src/org/jruby/ext/openssl/SSLSocket.java#L142 - this line will always force a reverse DNS lookup for an SSL connection.

http://docs.oracle.com/javase/1.5.0/docs/api/javax/net/ssl/SSLContext.html#createSSLEngine() - this documentation suggests that passing hostname and port should only be required for certain cipher suites.

Perhaps a flag to force the reverse lookup if you think it's required?

Additionally, there seems to be some potential issues in that SSLSocket is in no way associated with Ruby's Socket or BasicSocket...so, for example, a flag like BasicSocket.do_not_reverse_lookup will not be adhered to when using SSLSocket (See comment https://github.com/jruby/jruby/blob/master/src/org/jruby/ext/openssl/SSLSocket.java#L130, for example).

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Ben Porterfield added a comment -

Removing the calls to getHostName still results in a reverse DNS lookup, so there's probably more than just that one call to deal with. I can run a simple rack app without ssl and there is no reverse DNS, but even with line 142 from above removed properly, there is still a reverse DNS lookup with the same app over ssl.

Show
Ben Porterfield added a comment - Removing the calls to getHostName still results in a reverse DNS lookup, so there's probably more than just that one call to deal with. I can run a simple rack app without ssl and there is no reverse DNS, but even with line 142 from above removed properly, there is still a reverse DNS lookup with the same app over ssl.
Hide
Permalink
Ben Porterfield added a comment -

Also of note - there is no reverse DNS lookup on MRI with SSL.

Show
Ben Porterfield added a comment - Also of note - there is no reverse DNS lookup on MRI with SSL.
Hide
Permalink
Charles Oliver Nutter added a comment -

Found some hints here that it's a JVM problem, and there are some workarounds:

http://stackoverflow.com/questions/3193936/how-to-disable-javas-ssl-reverse-dns-lookup

You are also right about the SSLContext docs indicating host may not be needed for most providers. Perhaps we can try without and then fallback to passing host, and look at the workarounds for the Java issue.

Show
Charles Oliver Nutter added a comment - Found some hints here that it's a JVM problem, and there are some workarounds: http://stackoverflow.com/questions/3193936/how-to-disable-javas-ssl-reverse-dns-lookup You are also right about the SSLContext docs indicating host may not be needed for most providers. Perhaps we can try without and then fallback to passing host, and look at the workarounds for the Java issue.
Hide
Permalink
Ben Porterfield added a comment -

FWIW, this change fixed the problem for my client - no more slow queries! https://github.com/llooker/jruby-ossl/commit/0e81818ca492ab81386469d686dfcd5ddbca7ca7

Client was on Ubuntu 10.04. The problem may still exist on windows (as suggested by your stackoverflow link).

It seems to me that the only problem with this approach is that you'd also need some flag to force the hostname lookup if you are using a cipher suite that requires a hostname (although I'm not sure which ones those are or if JRuby SSL supports them).

Show
Ben Porterfield added a comment - FWIW, this change fixed the problem for my client - no more slow queries! https://github.com/llooker/jruby-ossl/commit/0e81818ca492ab81386469d686dfcd5ddbca7ca7 Client was on Ubuntu 10.04. The problem may still exist on windows (as suggested by your stackoverflow link). It seems to me that the only problem with this approach is that you'd also need some flag to force the hostname lookup if you are using a cipher suite that requires a hostname (although I'm not sure which ones those are or if JRuby SSL supports them).
Hide
Permalink
Charles Oliver Nutter added a comment -

I have code that will attempt to initialize without the host and port, falling back on the host + port version if it fails.

Show
Charles Oliver Nutter added a comment - I have code that will attempt to initialize without the host and port, falling back on the host + port version if it fails.
Hide
Permalink
Charles Oliver Nutter added a comment -

This should work for you. Note this is only on master, but we intend (given enough time) to try to backport to the jruby-openssl gem.

commit d43177d40ee97ab91025db4eb1e4de03c5eb7a1f
Author: Charles Oliver Nutter <headius@headius.com>
Date:   Wed Sep 19 00:33:50 2012 -0500

    Fix JRUBY-6891
    
    SSLSocket.accept forces reverse DNS lookup, not required for most SSL and sometimes causes very poor performance
    
    Try to initialize without host, falling back on the old logic if
    the SSLEngine fails to create.

:100644 100644 f8043ea... ffb24dc... M	src/org/jruby/ext/openssl/SSLContext.java
:100644 100644 7c175fc... 5c5a401... M	src/org/jruby/ext/openssl/SSLSocket.java
Show
Charles Oliver Nutter added a comment - This should work for you. Note this is only on master, but we intend (given enough time) to try to backport to the jruby-openssl gem. commit d43177d40ee97ab91025db4eb1e4de03c5eb7a1f Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Sep 19 00:33:50 2012 -0500 Fix JRUBY-6891 SSLSocket.accept forces reverse DNS lookup, not required for most SSL and sometimes causes very poor performance Try to initialize without host, falling back on the old logic if the SSLEngine fails to create. :100644 100644 f8043ea... ffb24dc... M src/org/jruby/ext/openssl/SSLContext.java :100644 100644 7c175fc... 5c5a401... M src/org/jruby/ext/openssl/SSLSocket.java
Hide
Permalink
Ben Porterfield added a comment -

Great, thanks so much!

Show
Ben Porterfield added a comment - Great, thanks so much!
Hide
Permalink
Patrick Toomey added a comment -

This solution to this bug results in breaking Server Name Indication (SNI). Please see JRUBY-6944 for more details.

Show
Patrick Toomey added a comment - This solution to this bug results in breaking Server Name Indication (SNI). Please see JRUBY-6944 for more details.
Hide
Permalink
Charles Oliver Nutter added a comment -

We had to revert this for 1.7.0 due to the breakage reported in JRUBY-6944. Reopening.

Show
Charles Oliver Nutter added a comment - We had to revert this for 1.7.0 due to the breakage reported in JRUBY-6944 . Reopening.

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Ben Porterfield
Vote (0)
Watch (4)

Dates

  • Created:
    Updated: