Jetty
  1. Jetty
  2. JETTY-400

CGI servlet hangs if CGI script fills STDERR buffer

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 6.1.3, 6.1.4rc0, 6.1.4rc1, 6.1.4, 6.1.5rc0, 6.1.5rc1, 6.1.5
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      2

      Description

      We've been experiencing occasional hanging behaviour for a while now, but have only now been able to track down the cause.

      It seems that the Jetty CGI servlet only reads from the CGI process' STDOUT handle, and STDERR is simply buffered. If that buffer becomes full, the CGI process will block when it tries to write to STDERR, and Jetty will block (waiting for data to be written to STDOUT). Since the CGI process is blocked, it will never write to (or close) STDOUT and then whole system hangs until something is killed.

      The following perl script can be used to reproduce the problem...

      #!/usr/bin/perl -w

      use strict;

      1. Flood the STDERR buffer
        foreach (1..5000) {
        print STDERR "mds \n";
        }
      1. Write out some data
        use CGI;
        my $cgi = new CGI;
        print $cgi->header();

      print "Hi!\n";

      The specific line where I've seen this blocking behaviour is the following one (in CGI.java)

      (line 369)
      while( (b = is.read()) != -1 && b != (int) '\n' ) {

      with the is.read() call being the culprit...

      Not sure what exactly to suggest as a solution, but clearly the STDERR handle needs to be read (even if the output is ignored).

      See also http://www.faqs.org/faqs/computer-lang/java/programmers/faq/

      6. (Sect. 18) How do I execute a command from Java?

      [*] Use

      Runtime.getRuntime().exec( myCommandString )

      where myCommandString is something like "/full/pathname/command". An
      applet will need to be signed in order to allow this.

      Note, there is a gotcha associated with reading output from commands.
      When the runtime exec's the process, it passes to it 3 streams, for
      stdin, stdout, and stderr; the out and err are buffered but the buffer
      size isn't very big. When your process runs, it reads (if needed) from
      in, and writes to out and err. If it doesn't write more than the
      buffer-size, it can run to completion.

      But if it tries to write more data to one or the other stream than the
      buffer can hold, the write blocks, and your process hangs, waiting for
      you to empty the buffer so it can write some more.

      So after the exec call, get the streams, and read from them in a loop
      until they both hit end-of-stream (don't block on either one, just read
      whatever is available from each, each loop iteration). Then when the
      streams have ended, call the process.waitFor() method to let it finish
      dying.

      1. hang_jetty.cgi.txt
        0.2 kB
        Matt Sheppard
      2. JETTY-400.patch
        2 kB
        Matt Sheppard

        Activity

        Hide
        Matt Sheppard added a comment -

        Attached a copy of the example perl script, since the formatting seems to be a bit over-helpful in jira.

        Show
        Matt Sheppard added a comment - Attached a copy of the example perl script, since the formatting seems to be a bit over-helpful in jira.
        Matt Sheppard made changes -
        Field Original Value New Value
        Attachment hang_jetty.cgi.txt [ 28583 ]
        Hide
        Matt Sheppard added a comment -

        This patch appears to resolve the issue for us, but is clearly not ready to go directly into Jetty.

        I'm also worried that other places where STDOUT is read (IO.copy) may suffer from a similar problem, but I've not been able to create a CGI script to produce any such problem.

        Show
        Matt Sheppard added a comment - This patch appears to resolve the issue for us, but is clearly not ready to go directly into Jetty. I'm also worried that other places where STDOUT is read (IO.copy) may suffer from a similar problem, but I've not been able to create a CGI script to produce any such problem.
        Matt Sheppard made changes -
        Attachment JETTY-400.patch [ 28584 ]
        Hide
        Greg Wilkins added a comment -

        Thanks for the patch, but I resolved this by copying the stderr to system.err
        with the line:

        IO.copyThread(p.getErrorStream(),System.err);

        I think this is the right thing to do?

        Show
        Greg Wilkins added a comment - Thanks for the patch, but I resolved this by copying the stderr to system.err with the line: IO.copyThread(p.getErrorStream(),System.err); I think this is the right thing to do?
        Greg Wilkins made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Matt Sheppard added a comment -

        Sounds reasonable to me.

        I'll do a quick test of svn head and get back to you.

        Show
        Matt Sheppard added a comment - Sounds reasonable to me. I'll do a quick test of svn head and get back to you.
        Hide
        Matt Sheppard added a comment -

        Yep, seems to work well. Thanks for the quick turnaround!

        Show
        Matt Sheppard added a comment - Yep, seems to work well. Thanks for the quick turnaround!

          People

          • Assignee:
            Unassigned
            Reporter:
            Matt Sheppard
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: