Jetty

100-Continue breakage

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 6.1.2, 6.1.3rc0, 6.1.3rc1, 6.1.3, 6.1.4rc0, 6.1.4rc1, 6.1.4, 6.1.5rc0, 6.1.5rc1, 6.1.5, 6.1.6rc0, 6.1.6rc1, 6.1.6
  • Fix Version/s: 6.1.6rc1
  • Component/s: None
  • Labels:
    None
  • Number of attachments :
    2

Description

Somewhere between revision 1530 and 1789, the servlet behavior for handling 100-Continue changed in such a way as to break the default .NET 1.x SOAP client. Attached is a Win pcap file showing the interaction.

  1. continue.patch
    11/Oct/07 9:14 AM
    7 kB
    Greg Wilkins
  2. ESP calls.pcap
    10/Oct/07 1:26 PM
    2 kB
    Stanford Ng

Activity

Hide
Greg Wilkins added a comment -

I have improved test for RFC section 8.2 in

modules/jetty/src/test/java/org/mortbay/jetty/RFC2616Test.java

and I can't reproduce any problem with 100 continue?

In the trace that you sent me, the 100 continue appears to function OK, but then the connection is reset AFTER the content is sent. So I suspect some other failure has caused the connection to be reset?

Was there any exception generated or error in any log?

Does this request work without the expect-100continues?

Show
Greg Wilkins added a comment - I have improved test for RFC section 8.2 in modules/jetty/src/test/java/org/mortbay/jetty/RFC2616Test.java and I can't reproduce any problem with 100 continue? In the trace that you sent me, the 100 continue appears to function OK, but then the connection is reset AFTER the content is sent. So I suspect some other failure has caused the connection to be reset? Was there any exception generated or error in any log? Does this request work without the expect-100continues?
Hide
Stanford Ng added a comment -

Yeah, this works without the expect-100 continues. Disabling the expect100Continue on the client side via config fixed the problem:

<system.net>
<settings>
<servicePointManager expect100Continue="false" />
</settings>
</system.net>

The strange thing is that this client code works fine with release 6.1.1 and earlier. It might be a .NET issue, but given the circumstances, it seems more likely that it was triggered by a Jetty change.

There's more background on the .NET side here:
http://blogs.msdn.com/joncole/archive/2005/09/08/462659.aspx
http://haacked.com/archive/2004/05/15/http-web-request-expect-100-continue.aspx

Show
Stanford Ng added a comment - Yeah, this works without the expect-100 continues. Disabling the expect100Continue on the client side via config fixed the problem: <system.net> <settings> <servicePointManager expect100Continue="false" /> </settings> </system.net> The strange thing is that this client code works fine with release 6.1.1 and earlier. It might be a .NET issue, but given the circumstances, it seems more likely that it was triggered by a Jetty change. There's more background on the .NET side here: http://blogs.msdn.com/joncole/archive/2005/09/08/462659.aspx http://haacked.com/archive/2004/05/15/http-web-request-expect-100-continue.aspx
Hide
Greg Wilkins added a comment -

I'm not saying this is not a Jetty problem... it is just not an obvious easy to reproduce problem.

Any chance that you can put together a very simple helloword style webapp that will demonstrate this
issue. I don't have a windows dev environment, and have no idea about .NET. But if you can give me
a war file to run and an exe that hits it and reproduces the problem, then I am sure I can track this down
to see why the content is being rejected after the 100 continues.

Show
Greg Wilkins added a comment - I'm not saying this is not a Jetty problem... it is just not an obvious easy to reproduce problem. Any chance that you can put together a very simple helloword style webapp that will demonstrate this issue. I don't have a windows dev environment, and have no idea about .NET. But if you can give me a war file to run and an exe that hits it and reproduces the problem, then I am sure I can track this down to see why the content is being rejected after the 100 continues.
Hide
Greg Wilkins added a comment -

I have reworked some parts of the 100-continue handling. I'm not sure if it will help or not, but it is worth trying.

Show
Greg Wilkins added a comment - I have reworked some parts of the 100-continue handling. I'm not sure if it will help or not, but it is worth trying.
Hide
Stanford Ng added a comment -

Hi Greg,

I appreciate all your help! Oddly enough, I tried to reproduce the problem using just a simple webapp but it didn't exhibit the same behavior. There must be some additional condition necessary to trigger the problem. If I can extract the offending code from our production application, I'll report it back here again.

Thanks again!

Show
Stanford Ng added a comment - Hi Greg, I appreciate all your help! Oddly enough, I tried to reproduce the problem using just a simple webapp but it didn't exhibit the same behavior. There must be some additional condition necessary to trigger the problem. If I can extract the offending code from our production application, I'll report it back here again. Thanks again!
Hide
Stanford Ng added a comment -

Oh the embarassment! We did some digging and it turns out to be a bad (imho) header combination that causes the problem. Turns out the .NET client is using 100-continue but has HTTP Keep alive turned off. This ends up sending a Connection: close w/ the initial request packet.

fwiw, it seems to work fine in Tomcat, but I can see how proper handling is quite ambiguous.

Show
Stanford Ng added a comment - Oh the embarassment! We did some digging and it turns out to be a bad (imho) header combination that causes the problem. Turns out the .NET client is using 100-continue but has HTTP Keep alive turned off. This ends up sending a Connection: close w/ the initial request packet. fwiw, it seems to work fine in Tomcat, but I can see how proper handling is quite ambiguous.
Hide
Stanford Ng added a comment -

Oh, and just to recap from the original bug report, it was also working in the older version of Jetty (revision 1530 and before).

Show
Stanford Ng added a comment - Oh, and just to recap from the original bug report, it was also working in the older version of Jetty (revision 1530 and before).
Hide
Jan Bartel added a comment -

Guess I'll close this issue then

Jan

Show
Jan Bartel added a comment - Guess I'll close this issue then Jan
Hide
P Eger added a comment -

I'm actually leaning on the side of the old jetty behaviour being correct.

Quoth the RFC
http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

  • Upon receiving a request which includes an Expect request-header
    field with the "100-continue" expectation, an origin server MUST
    either respond with 100 (Continue) status and continue to read
    from the input stream, or respond with a final status code. The
    origin server MUST NOT wait for the request body before sending
    the 100 (Continue) response. If it responds with a final status
    code, it MAY close the transport connection or it MAY continue
    to read and discard the rest of the request. It MUST NOT
    perform the requested method if it returns a final status code.

Specifically, "an origin server MUST either respond with 100 (Continue) status and continue to read from the input stream, or respond with a final status code". Since the 100 response is not a final status code, i think the "continue to read from the input stream" implies that it should process the "actual request" before closing.

I would argue that A) the request is not really complete until the body is sent, so the server should not be closing the connection until after that, and B) the current behaviour is quite useless and basically makes HTTP/1.1 keepalive useless for an client that uses 100-continue.

No?

Show
P Eger added a comment - I'm actually leaning on the side of the old jetty behaviour being correct. Quoth the RFC http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html
  • Upon receiving a request which includes an Expect request-header field with the "100-continue" expectation, an origin server MUST either respond with 100 (Continue) status and continue to read from the input stream, or respond with a final status code. The origin server MUST NOT wait for the request body before sending the 100 (Continue) response. If it responds with a final status code, it MAY close the transport connection or it MAY continue to read and discard the rest of the request. It MUST NOT perform the requested method if it returns a final status code.
Specifically, "an origin server MUST either respond with 100 (Continue) status and continue to read from the input stream, or respond with a final status code". Since the 100 response is not a final status code, i think the "continue to read from the input stream" implies that it should process the "actual request" before closing. I would argue that A) the request is not really complete until the body is sent, so the server should not be closing the connection until after that, and B) the current behaviour is quite useless and basically makes HTTP/1.1 keepalive useless for an client that uses 100-continue. No?
Hide
Greg Wilkins added a comment -

I'm missing some information here.....

What is the sequence of events you are seeing when Jetty is behaving badly?

Show
Greg Wilkins added a comment - I'm missing some information here..... What is the sequence of events you are seeing when Jetty is behaving badly?
Hide
Greg Wilkins added a comment -

actually - hold that.. I think I know what it is....

Show
Greg Wilkins added a comment - actually - hold that.. I think I know what it is....
Hide
P Eger added a comment -

.NET Request
-----------------------------
POST /services/Service HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.832)
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://BLAH"
Host: dev-nix:8980
Content-Length: 508
Expect: 100-continue
Connection: Close

Jetty's Response
-----------------------------
HTTP/1.1 100 Continue
<jetty closes tcp connection>

IE Jetty is probably seeing the "Connection: close" header and doing so directly after the 100 continue, as opposed to before where it would process the (SOAP, in this case) request.

Show
P Eger added a comment - .NET Request ----------------------------- POST /services/Service HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.832) Content-Type: text/xml; charset=utf-8 SOAPAction: "http://BLAH" Host: dev-nix:8980 Content-Length: 508 Expect: 100-continue Connection: Close Jetty's Response ----------------------------- HTTP/1.1 100 Continue <jetty closes tcp connection> IE Jetty is probably seeing the "Connection: close" header and doing so directly after the 100 continue, as opposed to before where it would process the (SOAP, in this case) request.
Hide
Greg Wilkins added a comment -

this is a bug ....

Show
Greg Wilkins added a comment - this is a bug ....
Hide
Greg Wilkins added a comment -

... and I have fix it.

HttpGenerator does not close the connection when sending a 100 continues.

Show
Greg Wilkins added a comment - ... and I have fix it. HttpGenerator does not close the connection when sending a 100 continues.
Hide
Stanford Ng added a comment -

Woohoo! Thanks Greg!

Show
Stanford Ng added a comment - Woohoo! Thanks Greg!

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: