Details
-
Type:
Wish
-
Status:
Resolved
-
Priority:
Major
-
Resolution: Won't Fix
-
Affects Version/s: 6.1.9, 7.0.0pre0
-
Fix Version/s: 6.1.15.pre0
-
Component/s: Security and SSL
-
Labels:None
-
Environment:Firefox 3.0 beta5 for Windows XP, Jetty 6.1.9 and 7.0.0pre0, jdk 1.6 on Windows XP
-
Number of attachments :3
Description
When using Firefox 3.0 beta5, the first time the browser connects to Jetty, it tries to use the TLS protocol. The handshake fails, and then falls back to SSL, which succeeds.
The fallback is instantaneous when using the SslSocketConnector, however it takes 30 seconds when using the SslSelectChannelConnector. So the initial page takes 30 seconds to load. After that, all is ok until Firefox is restarted.
This delay is determined by the value set for maxIdleTime (i.e. if I change the value to 10000, the delay is reduced to 10 seconds).
This problem only occurs with SslSelectChannelConnector and Firefox 3. The problem doesn't occur with SslSocketConnector or other servers. The problem does not occur when using Firefox2 or IE7 - the page loads instantaneously.
Although reducing maxIdleTime is a functional workaround (are there disadvantages in doing so?), I think this should be solved - especially since SslSelectChannelConnector and 30000 are the default settings in Jetty 7, and NIO is necessary for continuations...
here's a snippet of ssl debug for the initial handshake:
Using SSLEngineImpl.
btpool0-5, READ: TLSv1 Handshake, length = 158 <-[this is where it stops for 30 seconds]
btpool0-5, called closeOutbound()
btpool0-5, closeOutboundInternal()
btpool0-5, SEND TLSv1 ALERT: warning, description = close_notify
btpool0-5, WRITE: TLSv1 Alert, length = 2
btpool0-5, fatal error: 80: problem unwrapping net record
javax.net.ssl.SSLException: Unexpected end of handshake data
btpool0-5, SEND TLSv1 ALERT: fatal, description = internal_error
btpool0-5, Exception sending alert: java.io.IOException: writer side was already closed.
Using SSLEngineImpl.
btpool0-6, READ: SSL v2, contentType = Handshake, translated length = 79
-
-
- ClientHello, SSLv3
.....
- ClientHello, SSLv3
-
-
- Java6SslEngineBug.java
- 18/Mar/09 6:03 PM
- 2 kB
- Greg Wilkins
-
- jetty.txt
- 19/Jan/09 4:02 AM
- 4 kB
- Jan Bartel
-
- tomcat.txt
- 19/Jan/09 4:02 AM
- 2 kB
- Jan Bartel
Activity
I've been poking at this intermittently, and a few more things I've discovered:
- disabling "use TLS" in Firefox 3 removes the delay.
- however, disabling TLS serverside using SSlEngine.setEnabledProtocols(new String[]
{"SSL"}
), or setting the protocol used for the sslcontext doesn't have any effect
- the big differences I can find in the initial TLS handshake sent by Firefox 2 and 3 are sizes, some additional cipher suites in 3, differences in extension struct, including the presence of an "extension type 35, length [0]" in Firefox3 . It is possible that one of these differences is making the connection wait for more input?
I'll continue poking at it, when I have the chance.
Can you check this again and if there is nothing that can be done in jetty, raise a bug on FF
I definitely think this is a bug in either java 6's SSLEngine!
This is some debug out of Ssl end point for java 1.6:
CONSTRUCTED
fill=108
fill=0
inNIOBuffer=108
unwrap=Status = OK HandshakeStatus = NEED_TASK
bytesConsumed = 108 bytesProduced = 0
progress=true
Fill(Buffer)
status=NEED_TASK
run task
Fill(Buffer)
status=NEED_UNWRAP
fill=0
inNIOBuffer=0
break loop
IDLE EXPIRED
You can see that after an initial 108 bytes are received and processed, no bytes are wrapped and sent in return, so it just times out.
Incomparison, java 1.5 does:
CONSTRUCTED
fill=108
fill=0
inNIOBuffer=108
unwrap=Status = OK HandshakeStatus = NEED_TASK
bytesConsumed = 108 bytesProduced = 0
progress=true
Fill(Buffer)
status=NEED_TASK
run task
Fill(Buffer)
status=NEED_WRAP
wrap Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 1084
flushed=1084 of 0
Fill(Buffer)
status=NEED_UNWRAP
fill=0
inNIOBuffer=0
break loop
fill=198
fill=0
inNIOBuffer=198
unwrap=Status = OK HandshakeStatus = NEED_TASK
bytesConsumed = 139 bytesProduced = 0
progress=true
Fill(Buffer)
status=NEED_TASK
run task
Fill(Buffer)
status=NEED_UNWRAP
fill=0
inNIOBuffer=59
unwrap=Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 6 bytesProduced = 0
progress=true
Fill(Buffer)
status=NEED_UNWRAP
fill=0
inNIOBuffer=53
unwrap=Status = OK HandshakeStatus = NEED_WRAP
bytesConsumed = 53 bytesProduced = 0
progress=true
Fill(Buffer)
status=NEED_WRAP
wrap Status = OK HandshakeStatus = NEED_WRAP
bytesConsumed = 0 bytesProduced = 6
flushed=6 of 0
Fill(Buffer)
status=NEED_WRAP
wrap Status = OK HandshakeStatus = FINISHED
You can see a 1084 byte wrap being sent where jdk 1.6 wants and unwrap that never comes. So I think this should be raised as a sun java bug, not a FF bug.
I'll do that.
raised with Sun: internal review ID of 1299662.
Date Created: Tue Jul 22 10:21:11 MDT 2008
Type: bug
Customer Name: Greg Wilkins
Customer Email: gregw@webtide.com
SDN ID: gregwilkins
status: Waiting
Category: java
Subcategory: classes_net
Company: Webtide
release: 6
hardware: x86
OSversion: linux
priority: 4
Synopsis: SslEngine does not work with FF3 using TLS
Description:
FULL PRODUCT VERSION :
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Server VM (build 10.0-b22, mixed mode)
ADDITIONAL OS VERSION INFORMATION :
Linux brick 2.6.24-19-generic #1 SMP Fri Jul 11 23:41:49 UTC 2008 i686 GNU/Linux
A DESCRIPTION OF THE PROBLEM :
The SSL handshake from SslEngine does not work when it has received a TLS handshake from firefox 3
With java 1.5, the sequence of events are:
socket accepted
108 bytes received
unwrap -> Status = OK HandshakeStatus = NEED_TASK bytesConsumed = 108 bytesProduced = 0
run task
status=NEED_WRAP
wrap Status = OK HandshakeStatus = NEED_UNWRAP bytesConsumed = 0 bytesProduced = 1084
flushed=1084
status=NEED_UNWRAP
198 received
... handshake continues successfully!
With java 1.6 the sequence is:
socket accepted
108 bytes received
unwrap -> Status = OK HandshakeStatus = NEED_TASK bytesConsumed = 108 bytesProduced = 0
run task
status=NEED_UNWRAP
0 bytes received.
wait
TIMEOUT!
Firefox does not continue with the handshake because no wrapped data is sent in response to the initial 108 bytes. After this timeout, FF3 falls back from TLS and uses SSL, which succeeds
The effect is that the browser always takes a timeout when connecting the first time (30s by default)
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Download and install either jetty 6.1.11 or jetty 7.0.0pre1 (the usage of SslEngine in jetty 7 is cleaner, but both have the problem). run with:
java -jar start.jar etc/jetty.xml etc/jetty-ssl.xml
point FF3 at https://localhost:8443
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
no delay
ACTUAL -
a long delay before the first page displays
REPRODUCIBILITY :
This bug can be reproduced always.
CUSTOMER SUBMITTED WORKAROUND :
None found other than disabling TLS in FF3
workaround:
Actually, Greg, the SslSelectChannelConnector is defined in etc/jetty-sslengine.xml in v6.1.11 - not etc/jetty-ssl.xml ... as Dan points out in the initial description, using the SslSocketConnector (as defined in etc/jetty-ssl.xml) instead of the SslSelectChannelConnector would be a second workaround.
you might want to fix the report so the Sun guys can reproduce the bug more accurately ![]()
I've changed this to a wish for 7.0.0 rather than a bug on 7.0.0pre4.
I don't think there is anything that we can do on Jetty, but wait for a fixed JVM.
I'm seeing this problem with the latest version of Jetty 6 but running with the IBM 1.5 JVM. The delay is "configurable" through the idle timeout. So I'm not convinced this is a JVM issue.
Some of my colleagues are seeing this with FF2 as well as FF3.
And I'm using the SslSocketConnector:
<New class="org.mortbay.jetty.security.SslSocketConnector">
<Set name="Port">8000</Set>
<Set name="ThreadPool">
<New class="org.mortbay.thread.BoundedThreadPool">
<Set name="MinThreads">1</Set>
<Set name="MaxThreads">5</Set>
</New>
</Set>
<Set name="protocol">SSL_TLS</Set>
<Set name="MaxIdleTime">30000</Set>
<Set name="LowResourceMaxIdleTime">2000</Set>
<Set name="Keystore"><SystemProperty name="ris.root" default="." />/node-config/keystore</Set>
<Set name="Password">OBF:1dvu1hv41m801iuj191wn31x1b1toa1w1c1lk11xmk1y0o1toe1unz1tog1y121xmq1llp1w261tok1x1b</Set>
<Set name="KeyPassword">OBF:1dvu1hv41m801iuj19q11as1bp1w261tok1x1b1wn31bb31apq19qd1is31m4e1hse1dv8</Set>
</New>
Is there any news on this issue? I've encountered it with firefox 3.0*/jetty 6/java 1.6 as well (am using SslSocketConnector as a workaround for now). Thank you!
This posting on the jetty support list from 16th January 2008 sounds as if it is related. I'll attach the jetty and tomcat output files separately:
Matt Doran wrote:
> Hi guys,
>
> We use Jetty as the embedded web server in our product (PaperCut). In fact, we're one of the case studies on the web tide site here <http://www.webtide.com/case_studies/>. ![]()
>
> We've recently upgraded our application to use Java 6 and Jetty 6.1.14 (from Jetty 5). And we've found that establishing an HTTPS connection to the server from Firefox 3 results in a 30 second delay (which doesn't occur using other browsers).
>
> After enabling the JRE's SSL debug logging and googling the errors I found, I came across your issue describing this problem (JETTY-567 <http://jira.codehaus.org/browse/JETTY-567>). So the underlying issue appears to be a bug in the Java SSL implementation. And Greg said that he'd raised the issue with Sun, but I couldn't find the issue in the Sun Bug database.
>
> This is a pretty serious bug, and I'm surprised that there aren't more people affected by this one!
>
> Does anyone have any news on this issue? Or had any feedback from Sun? Does anyone know if this affects other web servers (like Tomcat)?
>
>
>
fyi, I've just tried Tomcat on the same Java 6 VM, and it does not have the delay problem encountered in Jetty.
However analysis of the Java SSL debug logs, shows that it has the same TLSv1 handshake failure, but it doesn't wait the 30 seconds before falling back to the SSLv3 connection. I guess this is a difference in the implementations ... but maybe one of the Jetty developers can make sense of it and improve Jetty's behaviour in this case.
I've attached the SSL logging of a Firefox connection to Jetty 6.1.14 and Tomcat 6.0.18. You can see the TLSv1 handshake failures in both ... but they react in different ways. Jetty waits for more data from the client, and Tomcat fails with the "Unexpected end of handshake data" and moves on immediately.
Regards,
Matt
Files provided by Matt Doran in posting to jetty support on 16/1/2008.
I think that we have seen this same issue (handshake failure with first try when using FF3 and TLS) in an SSL socket server. In our case we are not using Jetty at all – but we are using Sun's JDK6 – jdk1.6.0_11.
I haven't yet verified that this is not an issue w/ JDK5; however, when I was trying to Google the problem, this bug appeared to be the only exact match for the issue were were observing.
This was a serious problem when Firefox 3 was first released, but disappeared with later releases (maybe 3.0.2). Firefox does not delay before switching to SSL anymore, the SSL Handshake exception in the logs is still there.
No, the problem is still there! I just tested it with Firefox 3.0.5, Jetty 6.1.14 on Windows machine and Java 6 u10.
Please discard my previous comment. I was mistaken, problem is still there.
Note:
The fix simply detects the signature of the bug and then close the connection (fail-fast) so that ff3 will delegate to using SSL instead of TLS.
This is a jvm bug on java1.6 where the SSLEngine expects more data from the initial handshake when the client(ff3) already had given it.
The expected state transition:
NOT_HANDHSAKING
unwrap()
NEED_TASK (need to interpret data)
NEED_WRAP (need to respond)
The state transition when the bug happens:
NOT_HANDSHAKING
unwrap()
NEED_TASK (need to interpet data)
NEED_UNWRAP (need more data??? bug)
I'm receiving some test failures on jetty-ssl that might be related.
NOTE: This is an intermittent failure.
testRequestJettyHttps(org.mortbay.jetty.security.SSLEngineTest) Time elapsed: 33.615 sec <<< ERROR!
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:117)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1650)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:739)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.readLine(BufferedReader.java:299)
at java.io.BufferedReader.readLine(BufferedReader.java:362)
at org.mortbay.jetty.security.SSLEngineTest.readResponse(SSLEngineTest.java:220)
at org.mortbay.jetty.security.SSLEngineTest.testRequestJettyHttps(SSLEngineTest.java:173)
Using JDK 1.5.0_15 (Sun) on Linux ia32 (ubuntu hardy), using Maven 2.0.9 (release)
I've tried to get Sun to open a bug on this issue and have created a small reproducible test harness that demonstrates the problem.
I raised a report on the 13/Feb/2009 and received an internal bug number 1455259, but have yet, it's not been escalated to real bug.
Reopening, because we need to chase Sun until the JVM is really fixed.
well sun shows no interest in fixing the bug.
we have a work around now in Jetty, so that the delay is avoided.
If anybody still gets the delay, please reopen.
More information about this. It does not occur with JDK 1.5, only with JDK 1.6 .
To help replicate the problem, here's the instructions:
1. use latest production Sun JDK: JDK 1.6.0 update 6. \
java -version says: "Java(TM) SE Runtime Environment (build 1.6.0_06-b02)"
either Windows or Linux (32 bit)
2. use Jetty 6.1.10 stock installation. Simply edit etc/jetty-ssl.xml and set its content to:
-----------------------------------------------------------------
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
<!-- =============================================================== -->
<!-- Configure SSL for the Jetty Server -->
<!-- this configuration file should be used in combination with -->
<!-- other configuration files. e.g. -->
<!-- java -jar start.jar etc/jetty.xml etc/jetty-ssl.xml -->
<!-- =============================================================== -->
<Configure id="Server" class="org.mortbay.jetty.Server">
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.security.SslSelectChannelConnector">
<Set name="Port">443</Set>
<Set name="maxIdleTime">30000</Set>
<Set name="keystore"><SystemProperty name="jetty.home" default="." />/etc/keystore</Set>
<Set name="password">OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4</Set>
<Set name="keyPassword">OBF:1u2u1wml1z7s1z7a1wnl1u2g</Set>
<Set name="truststore"><SystemProperty name="jetty.home" default="." />/etc/keystore</Set>
<Set name="trustPassword">OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4</Set>
<!-
<Set name="handshakeTimeout">2000</Set>-><!-- Set name="ThreadPool">
<New class="org.mortbay.thread.BoundedThreadPool">
<Set name="minThreads">10</Set>
<Set name="maxThreads">250</Set>
</New>
</Set -->
</New>
</Arg>
</Call>
</Configure>
-------------------------------------------------
this is the standard jetty-ssl.xml , the only difference is the class has changed to SslSelectChannelConnector , and the handshakeTimeout paramerter has been removed (it is not a supported setter in SslSelectChannelConnector, so leaving it in would cause an exception on startup).
3. in Firefox 3, open https://www.server.com (whatever the address of your server is)
4. .... there will be a 30 seconds delay before getting the home page. after the intial connection, all is fine until you restart FireFox
Again, this is a tricky one to pinpoint:
1. it only happens with FireFox 3 (beta5, and also RC1). FireFox 2, IE7, Safari all connect without the delay.
2. it only happens with SslSelectChannelConnector . Using the normal SocketConnector works.
3. it only happens with Java 1.6 . Java 1.5 does not give this problem.
Now, you may think these are quite strict conditions, so the problem would not be very common. However these are the latest versions of FireFox, Java and Jetty, so this problem will be very common, especially when FireFox 3 goes in production in a few days....