Jetty

Data visibility issues in org.eclipse.jetty.server.Response?

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 8.1.0
  • Fix Version/s: 8.1.0
  • Component/s: None
  • Labels:
    None
  • Number of attachments :
    0

Description

We're seeing exceptions like the following:

java.lang.IllegalStateException: STREAM
	at org.eclipse.jetty.server.Response.getWriter(Response.java:683) ~[jetty-server.jar:8.1.0.RC2]

For any given request/response we either use getOutputStream (if the response is being compressed) or getWriter (if the response is not being compressed). I have done a pretty careful code walk-through to verify this pattern.

We use Continuation and write the response on "our" thread (and thus call Continuation.complete at the end).

My theory about what's going on is that the state in the (per-connection) Response object has data visibility issues.

  1. A connection (with final Response object R) is used to write a response on "our" thread T1. The output is compressed, so we call getOutputStream, which sets R._outputState to STREAM.
  2. The response completes, so R.recycle is called (not sure what thread this happens on), which sets R._outputState to NONE, but there is no locking or volatile to make sure that write is published properly.
  3. Another request comes in on the same connection and {R}

    is used to write the response on "our" thread T2. The response is not compressed so we call getWriter – this is where the exception is thrown.

  4. The exception is thrown because the write to R._outputState is not visible in T2 so it reads as STREAM.

Am I off in the weeds here?

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Jan Bartel added a comment -

Anders,

Do you have any code snippets or better yet a small test harness? On the face of it, I don't see that the select key to read any new request would be set before the handling of the previous request/response is done, but your analysis does have some intuitive appeal

Also, if you can get some logs when this happens, with full debug logging turned on, it may be helpful.

thanks
Jan

Show
Jan Bartel added a comment - Anders, Do you have any code snippets or better yet a small test harness? On the face of it, I don't see that the select key to read any new request would be set before the handling of the previous request/response is done, but your analysis does have some intuitive appeal Also, if you can get some logs when this happens, with full debug logging turned on, it may be helpful. thanks Jan
Hide
Permalink
Jan Bartel added a comment -

For the code snippet, it would be useful to see how you're assigning a thread to handle the IO in the Continuation.

Also, are these Requests pipelined on the same connection, or are you saying the whole Request/Response cycle completes before the next Request comes in????

I think there's something to investigate here, but it would be useful to have your responses from my previous post and this post to go on.

thanks
Jan

Show
Jan Bartel added a comment - For the code snippet, it would be useful to see how you're assigning a thread to handle the IO in the Continuation. Also, are these Requests pipelined on the same connection, or are you saying the whole Request/Response cycle completes before the next Request comes in???? I think there's something to investigate here, but it would be useful to have your responses from my previous post and this post to go on. thanks Jan
Hide
Permalink
Jan Bartel added a comment -

Hi Anders,

Turns out I don't think you're off in the weeds at all, but probably right on the button with your analysis. We've added a couple of memory boundaries to ensure that your application thread should pick up the correct value of fields within the Response object (amongst others).

I've pushed a snapshot of this to:
https://oss.sonatype.org/content/groups/jetty/org/eclipse/jetty/jetty-distribution/8.1.0-SNAPSHOT/jetty-distribution-8.1.0-20120118.040155-8.zip

Could you download that and give it a try and see if that fixes things for you?

thanks
Jan

Show
Jan Bartel added a comment - Hi Anders, Turns out I don't think you're off in the weeds at all, but probably right on the button with your analysis. We've added a couple of memory boundaries to ensure that your application thread should pick up the correct value of fields within the Response object (amongst others). I've pushed a snapshot of this to: https://oss.sonatype.org/content/groups/jetty/org/eclipse/jetty/jetty-distribution/8.1.0-SNAPSHOT/jetty-distribution-8.1.0-20120118.040155-8.zip Could you download that and give it a try and see if that fixes things for you? thanks Jan
Hide
Permalink
Jan Bartel added a comment -

Anders,

This fix went out in the 8.1.0 release. As I haven't heard from you, I assume this is fixed. If not, please reopen this issue and provide as much detail as you can.

thanks
Jan

Show
Jan Bartel added a comment - Anders, This fix went out in the 8.1.0 release. As I haven't heard from you, I assume this is fixed. If not, please reopen this issue and provide as much detail as you can. thanks Jan

People

  • Assignee:
    Jan Bartel
    Reporter:
    Anders Wallgren
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: