Net::POP3 (and possibly other things) cannot verify certificate using JRE's trust anchors

Details

Type: Bug
Status: Reopened
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: JRuby 1.7.0.RC2
Component/s: OpenSSL
Labels:
None
Environment:
Ubuntu Linux, sun jdk 1.6.0_26, rvm 1.6.22, jruby 1.5.6 and 1.6.4

Number of attachments :
0

Description

Using the following ruby script:

#!/usr/bin/env ruby

require 'openssl'
require 'net/pop'

begin

  mail_server = '' # SOME SERVER HERE, e.g. mail.example.com
  mail_user = 'username'
  mail_password = 'password'

  pop = Net::POP3.new(mail_server)
  pop.enable_ssl()
  pop.start(mail_user, mail_password)

  exit 0 if pop.mails.empty?
  i = 0
  pop.each_mail do |m|
    puts "found mail: (snip)"
    m.pop
    i = i + 1
  end
  puts "#{i} mails popped"
  exit 0
end

When run using jruby 1.6.4:

(1:31:29):cmyers@cmyers-ubuntu:3 master] /home/cmyers/jvms/jdk1.6.0_26/bin/java -Djdk.home= -Djruby.home=/home/cmyers/.rvm/rubies/jruby-1.6.4 -Djruby.script=jruby -Djruby.shell=/bin/sh -Djffi.boot.library.path=/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/x86_64-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/ppc-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/i386-Linux -Xmx500m -Xss2048k -Djruby.memory.max=500m -Djruby.stack.max=2048k -Dsun.java.command=org.jruby.Main -Djava.class.path= -Djavax.net.ssl.trustStore=$HOME/.keystore -Xbootclasspath/a:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/jruby.jar org/jruby/Main ./pop-proxy.rb
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
SSL? : true
LoadError: OpenSSL::SSL requires the jruby-openssl gem
    (root) at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/site_ruby/shared/jruby/openssl/autoloads/ssl.rb:8
  do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/site_ruby/shared/jruby/openssl/autoloads/ssl.rb:537
     start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/pop.rb:528
    (root) at ./pop-proxy.rb:20

When run with cruby 1.9.2:

(1:35:40):cmyers@cmyers-ubuntu:3 master] /home/cmyers/.rvm/rubies/ruby-1.9.2-p180/bin/ruby ./pop-proxy.rb 
SSL? : true
found mail: (snip)
found mail: (snip)
2 mails popped

my server has a certificate signed by our internal certificate authority which has been added to my system's certificate store. I also added it to the JDK's keystore using keytool. I also attempted to specify an external keystore by setting -Djavax.net.ssl.trustStore=$HOME/.keystore and adding my certificate authority to that keystore using keytool - again to no avail.

Issue Links

is related to

JRUBY-5010 Unexpected "certificate verify failed" which does not match MRI behavior

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Carl Myers added a comment - 13/Oct/11 4:14 AM

Ack, I pasted the wrong output but I can't edit the issue. The error that complains about the jruby-openssl gem is wrong - the actual error I am seeing is:

(2:11:57):cmyers@cmyers-ubuntu:3 master] /home/cmyers/jvms/jdk1.6.0_26/bin/java -Djdk.home= -Djruby.home=/home/cmyers/.rvm/rubies/jruby-1.6.4 -Djruby.script=jruby -Djruby.shell=/bin/sh -Djffi.boot.library.path=/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/x86_64-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/ppc-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/i386-Linux -Xmx500m -Xss2048k -Djruby.memory.max=500m -Djruby.stack.max=2048k -Dsun.java.command=org.jruby.Main -Djava.class.path= -Djavax.net.ssl.trustStore=$HOME/.keystore -Xbootclasspath/a:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/jruby.jar org/jruby/Main ./pop-proxy.rb
SSL? : true
OpenSSL::SSL::SSLError: certificate verify failed
   connect at org/jruby/ext/openssl/SSLSocket.java:179
  do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/pop.rb:541
     start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/pop.rb:528
    (root) at ./pop-proxy.rb:20

Show

Carl Myers added a comment - 13/Oct/11 4:14 AM Ack, I pasted the wrong output but I can't edit the issue. The error that complains about the jruby-openssl gem is wrong - the actual error I am seeing is: (2:11:57):cmyers@cmyers-ubuntu:3 master] /home/cmyers/jvms/jdk1.6.0_26/bin/java -Djdk.home= -Djruby.home=/home/cmyers/.rvm/rubies/jruby-1.6.4 -Djruby.script=jruby -Djruby.shell=/bin/sh -Djffi.boot.library.path=/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/x86_64-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/ppc-Linux:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/native/i386-Linux -Xmx500m -Xss2048k -Djruby.memory.max=500m -Djruby.stack.max=2048k -Dsun.java.command=org.jruby.Main -Djava.class.path= -Djavax.net.ssl.trustStore=$HOME/.keystore -Xbootclasspath/a:/home/cmyers/.rvm/rubies/jruby-1.6.4/lib/jruby.jar org/jruby/Main ./pop-proxy.rb SSL? : true OpenSSL::SSL::SSLError: certificate verify failed connect at org/jruby/ext/openssl/SSLSocket.java:179 do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/pop.rb:541 start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/pop.rb:528 (root) at ./pop-proxy.rb:20

Hide

Permalink

Carl Myers added a comment - 24/Oct/11 7:01 PM

Actually, this is broken again for certs that should be recognized by the normal listings. I.E. public certs like google's aren't accepted.... Test case code follows:

(16:49:04):cmyers@cmyers-ubuntu:3 master] rvm use jruby-1.6.4
Using /home/cmyers/.rvm/gems/jruby-1.6.4
(16:54:52):cmyers@cmyers-ubuntu:3 master] ./download-google.rb
OpenSSL::SSL::SSLError: certificate verify failed
   connect at org/jruby/ext/openssl/SSLSocket.java:179
   connect at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:586
  do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:553
     start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:542
    (root) at ./download-google.rb:16
./download-google.rb  6.23s user 0.19s system 181% cpu 3.547 total
(16:54:58):cmyers@cmyers-ubuntu:3 master] cat ./download-google.rb
#!/usr/bin/env ruby
#

require 'net/https'
host = 'www.google.com'
path = '/'


https = Net::HTTP.new(host, Net::HTTP.https_default_port)
https.use_ssl = true
https.ssl_timeout = 2
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
#https.ca_file = '/etc/ssl/certs'
#https.verify_depth=2
#https.enable_post_connection_check = true
https.start do |http|
  headers, body = http.get(path)
  if headers.code == "200"
    print body
  else
    puts "#{headers.code} #{headers.message}"
  end
end



(16:59:32):cmyers@cmyers-ubuntu:3 master] rvm use jruby-1.6.4
Using /home/cmyers/.rvm/gems/jruby-1.6.4
(16:59:39):cmyers@cmyers-ubuntu:3 master] ./download-google.rb
OpenSSL::SSL::SSLError: certificate verify failed
   connect at org/jruby/ext/openssl/SSLSocket.java:179
   connect at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:586
  do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:553
     start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:542
    (root) at ./download-google.rb:16
./download-google.rb  6.10s user 0.22s system 190% cpu 3.316 total
(16:59:46):cmyers@cmyers-ubuntu:3 master] rvm use ruby-1.9.2-p180
Using /home/cmyers/.rvm/gems/ruby-1.9.2-p180
(16:59:52):cmyers@cmyers-ubuntu:3 master] ./download-google.rb | head -n2
<!doctype html><html><head><meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"><meta name="description" content="Search the world&#39;s information, including webpages, images, videos and more. Google has many special features to help you find exactly what you&#39;re looking for."><meta name="robots" content="noodp"><title>Google</title><script>window.google={kEI:"RPylToaIOs6cgQeSrp2yDw",getEI:function(a){var b;while(a&&!(a.getAttribute&&(b=a.getAttribute("eid"))))a=a.parentNode;return b||google.kEI},kEXPI:"28936,32034,33076,33104,33287,33372,33406,33491,33502,33612,33616,33620,33646,33668,33811",kCSI:{e:"28936,32034,33076,33104,33287,33372,33406,33491,33502,33612,33616,33620,33646,33668,33811",ei:"RPylToaIOs6cgQeSrp2yDw"},authuser:0,ml:function(){},kHL:"en",time:function(){return(new Date).getTime()},log:function(a,b,c,e){var d=new Image,f=google,h=f.lc,g=f.li,i="";d.onerror=(d.onload=(d.onabort=function(){delete h[g]}));h[g]=d;if(!c&&b.search("&ei=")==-1)i="&ei="+google.getEI(e);var j=c||"/gen_204?atyp=i&ct="+a+"&cad="+b+i+"&zx="+google.time();d.src=j;f.li=g+1},lc:[],li:0,Toolbelt:{},y:{},x:function(a,b){google.y[a.id]=[a,b];return false}};

(16:59:59):cmyers@cmyers-ubuntu:3 master]

Show

Carl Myers added a comment - 24/Oct/11 7:01 PM Actually, this is broken again for certs that should be recognized by the normal listings. I.E. public certs like google's aren't accepted.... Test case code follows: (16:49:04):cmyers@cmyers-ubuntu:3 master] rvm use jruby-1.6.4 Using /home/cmyers/.rvm/gems/jruby-1.6.4 (16:54:52):cmyers@cmyers-ubuntu:3 master] ./download-google.rb OpenSSL::SSL::SSLError: certificate verify failed connect at org/jruby/ext/openssl/SSLSocket.java:179 connect at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:586 do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:553 start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:542 (root) at ./download-google.rb:16 ./download-google.rb 6.23s user 0.19s system 181% cpu 3.547 total (16:54:58):cmyers@cmyers-ubuntu:3 master] cat ./download-google.rb #!/usr/bin/env ruby # require 'net/https' host = 'www.google.com' path = '/' https = Net::HTTP.new(host, Net::HTTP.https_default_port) https.use_ssl = true https.ssl_timeout = 2 https.verify_mode = OpenSSL::SSL::VERIFY_PEER #https.ca_file = '/etc/ssl/certs' #https.verify_depth=2 #https.enable_post_connection_check = true https.start do |http| headers, body = http.get(path) if headers.code == "200" print body else puts "#{headers.code} #{headers.message}" end end (16:59:32):cmyers@cmyers-ubuntu:3 master] rvm use jruby-1.6.4 Using /home/cmyers/.rvm/gems/jruby-1.6.4 (16:59:39):cmyers@cmyers-ubuntu:3 master] ./download-google.rb OpenSSL::SSL::SSLError: certificate verify failed connect at org/jruby/ext/openssl/SSLSocket.java:179 connect at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:586 do_start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:553 start at /home/cmyers/.rvm/rubies/jruby-1.6.4/lib/ruby/1.8/net/http.rb:542 (root) at ./download-google.rb:16 ./download-google.rb 6.10s user 0.22s system 190% cpu 3.316 total (16:59:46):cmyers@cmyers-ubuntu:3 master] rvm use ruby-1.9.2-p180 Using /home/cmyers/.rvm/gems/ruby-1.9.2-p180 (16:59:52):cmyers@cmyers-ubuntu:3 master] ./download-google.rb | head -n2 <!doctype html><html><head><meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"><meta name="description" content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for."><meta name="robots" content="noodp"><title>Google</title><script>window.google={kEI:"RPylToaIOs6cgQeSrp2yDw",getEI:function(a){var b;while(a&&!(a.getAttribute&&(b=a.getAttribute("eid"))))a=a.parentNode;return b||google.kEI},kEXPI:"28936,32034,33076,33104,33287,33372,33406,33491,33502,33612,33616,33620,33646,33668,33811",kCSI:{e:"28936,32034,33076,33104,33287,33372,33406,33491,33502,33612,33616,33620,33646,33668,33811",ei:"RPylToaIOs6cgQeSrp2yDw"},authuser:0,ml:function(){},kHL:"en",time:function(){return(new Date).getTime()},log:function(a,b,c,e){var d=new Image,f=google,h=f.lc,g=f.li,i="";d.onerror=(d.onload=(d.onabort=function(){delete h[g]}));h[g]=d;if(!c&&b.search("&ei=")==-1)i="&ei="+google.getEI(e);var j=c||"/gen_204?atyp=i&ct="+a+"&cad="+b+i+"&zx="+google.time();d.src=j;f.li=g+1},lc:[],li:0,Toolbelt:{},y:{},x:function(a,b){google.y[a.id]=[a,b];return false}}; (16:59:59):cmyers@cmyers-ubuntu:3 master]

Hide

Permalink

Carl Myers added a comment - 24/Oct/11 7:02 PM

the above script is as follows:


#!/usr/bin/env ruby
#

require 'net/https'
host = 'www.google.com'
path = '/'


https = Net::HTTP.new(host, Net::HTTP.https_default_port)
https.use_ssl = true
https.ssl_timeout = 2
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
#https.ca_file = '/etc/ssl/certs'
#https.verify_depth=2
#https.enable_post_connection_check = true
https.start do |http|
  headers, body = http.get(path)
  if headers.code == "200"
    print body
  else
    puts "#{headers.code} #{headers.message}"
  end
end

Show

Carl Myers added a comment - 24/Oct/11 7:02 PM the above script is as follows: #!/usr/bin/env ruby # require 'net/https' host = 'www.google.com' path = '/' https = Net::HTTP. new (host, Net::HTTP.https_default_port) https.use_ssl = true https.ssl_timeout = 2 https.verify_mode = OpenSSL::SSL::VERIFY_PEER #https.ca_file = '/etc/ssl/certs' #https.verify_depth=2 #https.enable_post_connection_check = true https.start do |http| headers, body = http.get(path) if headers.code == "200" print body else puts "#{headers.code} #{headers.message}" end end

Hide

Permalink

Hiroshi Nakamura added a comment - 16/Nov/11 2:33 AM

Sorry for late reply.

Currently, jruby-opensssl gem does not use JDK's certificate store. You need to follow OpenSSL's manner like "https.ca_file = '/etc/ssl/certs'" you're removing now.

Do you have a pem format CA file? Would you please try "https.ca_file = '/path/to/CA_cert.pem'"?

Show

Hiroshi Nakamura added a comment - 16/Nov/11 2:33 AM Sorry for late reply. Currently, jruby-opensssl gem does not use JDK's certificate store. You need to follow OpenSSL's manner like "https.ca_file = '/etc/ssl/certs'" you're removing now. Do you have a pem format CA file? Would you please try "https.ca_file = '/path/to/CA_cert.pem'"?

Hide

Permalink

Eric Anderson added a comment - 31/Jan/12 11:26 AM

Sadly, I agree with the assessment by Hiroshi. Jruby should behave like ruby, but running on the JVM, not like Java but written in Ruby.

Show

Eric Anderson added a comment - 31/Jan/12 11:26 AM Sadly, I agree with the assessment by Hiroshi. Jruby should behave like ruby, but running on the JVM, not like Java but written in Ruby.

Hide

Permalink

Hiroshi Nakamura added a comment - 29/Apr/12 10:59 PM

Changing the summary to fit with the requirement.

The requirement is fine, we might be able to find a good way between OpenSSL and Java...

Show

Hiroshi Nakamura added a comment - 29/Apr/12 10:59 PM Changing the summary to fit with the requirement. The requirement is fine, we might be able to find a good way between OpenSSL and Java...

Hide

Permalink

Matt Hauck added a comment - 21/Jun/12 3:20 PM

Just ran into ~~JRUBY-5010~~ today and was a bit surprised myself. i had hoped that it would by default trust my "lib/security/cacerts" store. I quite agree with Alex's comment on that bug.

As one more argument in support of doing this by default, I am trying to use the "savon" SOAP library to connect over HTTPS to a soap service. In cases like this where you are using a third-party library, you are not the one making the Net::HTTP call, and thus do not have the liberty to specify an application-wide trusted cert store.

Is there at least a workaround for this kind of functionality to work w/ JDK's cert store?

Show

Matt Hauck added a comment - 21/Jun/12 3:20 PM Just ran into JRUBY-5010 today and was a bit surprised myself. i had hoped that it would by default trust my "lib/security/cacerts" store. I quite agree with Alex's comment on that bug. As one more argument in support of doing this by default, I am trying to use the "savon" SOAP library to connect over HTTPS to a soap service. In cases like this where you are using a third-party library, you are not the one making the Net::HTTP call, and thus do not have the liberty to specify an application-wide trusted cert store. Is there at least a workaround for this kind of functionality to work w/ JDK's cert store?

Hide

Permalink

Matt Hauck added a comment - 21/Jun/12 4:58 PM

Interestingly enough this gem is able to do this without specifying any kind of private "ca_file": https://github.com/nahi/httpclient

Show

Matt Hauck added a comment - 21/Jun/12 4:58 PM Interestingly enough this gem is able to do this without specifying any kind of private "ca_file": https://github.com/nahi/httpclient

Hide

Permalink

Matt Hauck added a comment - 21/Jun/12 5:13 PM

Ahh, I discovered how httpclient does it: https://github.com/nahi/httpclient/blob/master/lib/httpclient/cacert.p7s

Given the fact that net/http does not have a way to set a global ca_file for all objects, I think jruby has no other option than to lean on the JDK's trusted store by default.

I'm off now to try to monkey-patch net/http to add a global ca_file...

Show

Matt Hauck added a comment - 21/Jun/12 5:13 PM Ahh, I discovered how httpclient does it: https://github.com/nahi/httpclient/blob/master/lib/httpclient/cacert.p7s Given the fact that net/http does not have a way to set a global ca_file for all objects, I think jruby has no other option than to lean on the JDK's trusted store by default. I'm off now to try to monkey-patch net/http to add a global ca_file...

Hide

Permalink

Matt Hauck added a comment - 21/Jun/12 6:05 PM

Hacking complete: https://gist.github.com/2969144

Show

Matt Hauck added a comment - 21/Jun/12 6:05 PM Hacking complete: https://gist.github.com/2969144

Hide

Permalink

Matt Hauck added a comment - 29/Sep/12 12:38 AM

Hooray, it is resolved! May I ask what the resolution was, since there appeared to be a lack of certainty in some comments above about which direction this was going to go?

Show

Matt Hauck added a comment - 29/Sep/12 12:38 AM Hooray, it is resolved! May I ask what the resolution was, since there appeared to be a lack of certainty in some comments above about which direction this was going to go?

Hide

Permalink

Hiroshi Nakamura added a comment - 23/Dec/12 8:37 AM

Trust related change must be committed with test.

Show

Hiroshi Nakamura added a comment - 23/Dec/12 8:37 AM Trust related change must be committed with test.

Hide

Permalink

Matt Hauck added a comment - 19/Apr/13 12:54 PM

Hmm. Never heard back a response here. What was the actual fix for this issue? It appears that jruby is now using the JRE's truststore by default, but I just wanted to verify that...

Show

Matt Hauck added a comment - 19/Apr/13 12:54 PM Hmm. Never heard back a response here. What was the actual fix for this issue? It appears that jruby is now using the JRE's truststore by default, but I just wanted to verify that...

People

Assignee:

Hiroshi Nakamura

Reporter:

Carl Myers

Vote (0)

Watch (3)

Dates

Created:

13/Oct/11 4:01 AM

Updated:

19/Apr/13 12:54 PM