Requests fail if header exceeds buffer size

If you think you are regularly going to receive quite large headers, you can always increase the size of the header buffer (which is 4k by default).
Call connector.setHeaderBufferSize(int bytes)

Thanks, Jan - I had already increased the header buffer size to 8K as a work-around.

We have experienced this issue intermittently across all our production servers, and were had not been able to determine what was causing it. It was only by luck that it happened to me on our development server, and I was able to track it down. It only started happening after we upgraded from Jetty 5.

Maybe it is not possible, but I think that even if the request has an abnormally large header, it should still be handled, just like it was under Jetty 5.

If it can't be handled (which would be disappointing - it is not always possible to predict these things), then maybe the client should receive a 413 response, or at least a 500 with indication of what the issue is, rather than just dying...

In Jetty5, troubles begin at 8K boundary because the default buffer size is 8K. Jetty indeed truncates the cookie data. We have reported this correctness problem.
http://sourceforge.net/tracker/index.php?func=detail&aid=1495150&group_id=7322&atid=107322
I completely agree that perhaps 413 is better status code than 500.

Gregw,

Any progress on this one?

Jan

I'm not making progress... so unassigning so that others might grab and resolve.

Not a high priority.... perhaps better documentation in the wiki would help?

How about throwing an HttpException instead of an IOException, so that HttpConnection can appropriately handle the error, and return it to the client? Attached is a diff + testcase for that

One thing I ran into while testing was a NPE when the buffer was full, because poke in NPE doesn't do any validation of the index:

java.lang.IndexOutOfBoundsException: 4096
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:154)
at org.mortbay.io.nio.NIOBuffer.poke(NIOBuffer.java:117)
at org.mortbay.io.AbstractBuffer.put(AbstractBuffer.java:404)
at org.mortbay.jetty.HttpFields$Field.put(HttpFields.java:1383)
at org.mortbay.jetty.HttpGenerator.completeHeader(HttpGenerator.java:502)
at org.mortbay.jetty.HttpConnection.completeResponse(HttpConnection.java:593)
at org.mortbay.jetty.Response.complete(Response.java:1097)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:563)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

(The NPE doesn't show up in the test case, because LocalConnector doesn't use NIOBuffer, and it automatically expands the buffer if the buffer is too small)

Can you review the patch?

thanks
Jan

Good fix athena.

applied

I am getting this error when running apache solr on jetty:

Feb 11, 2009 12:05:45 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/luke params=

status=0 QTime=1
_header=[4488242,3533454,m=4084,g=4096,p=4096,c=4096]=

{}
_buffer=[4488242,3533454,m=4084,g=4096,p=4096,c=4096]=

{}
2009-02-11 12:05:45.592::WARN: handle failed
java.io.IOException: FULL
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:274)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

I am not sure where to change the configuration of
"Call connector.setHeaderBufferSize(int bytes)" as suggested above.

the call to setHeaderBufferSize can be done either in code or in jetty.xml.

For each connector you are using, you need to add a line like:

<Set name='headerBufferSize'>8096</Set>

regards

Thanks Greg

I added this in example/etc/jetty.xml under the extra options section inside of <Configure>

<!-- =========================================================== -->
<!-- extra options -->
<!-- =========================================================== -->
<Set name="stopAtShutdown">true</Set>
<Set name="sendServerVersion">true</Set>
<Set name="headerBufferSize">8096</Set>

and received this error when starting solr:

2009-02-13 11:47:36.689::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2009-02-13 11:47:36.924::WARN: Config error at <Set name="headerBufferSize">8096</Set>
2009-02-13 11:47:36.924::WARN: EXCEPTION
java.lang.NoSuchMethodException: class org.mortbay.jetty.Server.setHeaderBufferSize(class java.lang.String)
at org.mortbay.xml.XmlConfiguration.set(XmlConfiguration.java:411)
at org.mortbay.xml.XmlConfiguration.configure(XmlConfiguration.java:237)
at org.mortbay.xml.XmlConfiguration.configure(XmlConfiguration.java:203)
at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:919)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.mortbay.start.Main.invokeMain(Main.java:183)
at org.mortbay.start.Main.start(Main.java:497)
at org.mortbay.start.Main.main(Main.java:115)
2009-02-13 11:47:36.929::INFO: Shutdown hook executing
2009-02-13 11:47:36.930::INFO: Shutdown hook complete

Audrey,

headerBufferSize is a connector attribute, so it needs to be set on the connector, not on the server.
So look for the connector section of the config file.

Please use the javadoc to learn what attributes can be set on which objects.

regards

It took me awhile to come up with the right jetty.xml to do this and only this. So for the sake of anyone else who might need to google it:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="Server" class="org.mortbay.jetty.Server">
<Call name="addConnector">
<Arg>
<New class="org.mortbay.jetty.nio.SelectChannelConnector">
<Set name="port"><SystemProperty name="jetty.port" default="8080"/></Set>
<Set name='headerBufferSize'>8192</Set>
</New>
</Arg>
</Call>
</Configure>