Details
Bug
Resolved
Major
Description
We recently had a client bug in one of our service consumers which involved the client sending malformed HTTP requests. In particular, the Content-Length header for PUT requests was off and the client forgot about the socket and never closed the connection.
Jetty's BlockingChannelConnector and SocketConnector both had odd responses to this situation.
SocketConnector goes into a hot loop of throwing SocketException instances and rescuing them:
"qtp1213779113-151" - Thread t@151 java.lang.Thread.State: RUNNABLE at java.lang.Throwable.fillInStackTrace(Native Method) - locked <7a96bbd0> (a java.net.SocketException) at java.lang.Throwable.<init>(Throwable.java:196) at java.lang.Exception.<init>(Exception.java:41) at java.io.IOException.<init>(IOException.java:41) at java.net.SocketException.<init>(SocketException.java:29) at java.net.SocketInputStream.read(SocketInputStream.java:113) at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:388) at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:132) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:209) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:289) at org.eclipse.jetty.http.HttpParser.blockForContent(HttpParser.java:1139) at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:57)
And BlockingChannelConnector spins on reading from the connection:
"qtp2084696615-1065" - Thread t@1065 java.lang.Thread.State: RUNNABLE at org.eclipse.jetty.io.nio.ChannelEndPoint.fill(ChannelEndPoint.java:173) - locked <587fc800> (a java.nio.HeapByteBuffer) at org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.fill(BlockingChannelConnector.java:233) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:289) at org.eclipse.jetty.http.HttpParser.blockForContent(HttpParser.java:1139) at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:57)
They both peg that worker thread indefinitely; it seems like request parsing should respect the timeout settings for the connector. I suspect SelectChannelConnector would also behave badly under these circumstances, but I didn't get the chance to try.