An OutOfMemoryError within GzipFilter. It happens randomly on a production system, but I can't reproduce it in my development environment.
This is actually caused by the C heap running out of memory.
The gzip filter creates GZipOutputStream objects, but doesn't call close() as that would close the underlying socket stream. GZipOutputStream calls out to C code which uses the C heap. If GZipOutputStream objects aren't cleaned up by GC fast enough (so finalized can be called), it's possible that the C heap can run out of memory.
This is a known issue in the JVM:
A possible solution:
- Extend GZipOutputStream to free up C memory when finish() is called.
- Modify doGZip() to instantiate the extended class instead of GZipOutputStream
We did something similar to the above, but we did it in a more complicated way to avoid having to modify Jetty source.
This doesn't entirely fix the issue; finish() is never called when an exception is caught in doFilter() and isCommitted() is false. Perhaps wrappedResponse.finish() should be called in the catch, but I'm not sure.