Jetty
  1. Jetty
  2. JETTY-445

IllegalStateException(SelectorManager line 352) cause CPU usage 100% !

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 6.1.6rc0
    • Fix Version/s: None
    • Component/s: NIO
    • Labels:
      None
    • Environment:
      jdk1.6.0_03
      Linux 2.6.21-gentoo-r4
    • Number of attachments :
      0

      Activity

      Hide
      Boyce Lee added a comment -

      Sometimes, cpu usage still get to 100% without logging "java.lang.IllegalStateException
      at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect".

      I change the underlying threadpool implementation to jdk ThreadPoolExecutor, adjust the max poolsize to 20000, It seems the jetty server run stably.

      In addition, I found the doc of org.mortbay.thread.concurrent.ThreadPool has some issue.
      When using an unbounded queue, the value of the maximumPoolSize doesn't have any effect. (See jdk doc)
      So, my new configuration file look like this:
      <New class="org.mortbay.thread.concurrent.ThreadPool">
      <Arg>0</Arg>
      <Set name="corePoolSize">200</Set>
      <Set name="maximumPoolSize">20000</Set>
      </New>

      Show
      Boyce Lee added a comment - Sometimes, cpu usage still get to 100% without logging "java.lang.IllegalStateException at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect". I change the underlying threadpool implementation to jdk ThreadPoolExecutor, adjust the max poolsize to 20000, It seems the jetty server run stably. In addition, I found the doc of org.mortbay.thread.concurrent.ThreadPool has some issue. When using an unbounded queue, the value of the maximumPoolSize doesn't have any effect. (See jdk doc) So, my new configuration file look like this: <New class="org.mortbay.thread.concurrent.ThreadPool"> <Arg>0</Arg> <Set name="corePoolSize">200</Set> <Set name="maximumPoolSize">20000</Set> </New>
      Hide
      Greg Wilkins added a comment -

      20000 is a seriously large number of threads! are you sure you have the RAM for that many stacks?

      Are you still seeing this problem with the patched server?

      Show
      Greg Wilkins added a comment - 20000 is a seriously large number of threads! are you sure you have the RAM for that many stacks? Are you still seeing this problem with the patched server?
      Hide
      Boyce Lee added a comment -

      The problem has been resolved since using the ThreadPoolExecutor.

      I guess that the default threadpool(BoundedThreadPool) has some issues.

      Show
      Boyce Lee added a comment - The problem has been resolved since using the ThreadPoolExecutor. I guess that the default threadpool(BoundedThreadPool) has some issues.
      Hide
      Greg Wilkins added a comment -

      I've added a test harness for BoundedThreadPool and I can't induce any errors in that.

      Show
      Greg Wilkins added a comment - I've added a test harness for BoundedThreadPool and I can't induce any errors in that.
      Hide
      Boyce Lee added a comment -

      I don't know why the WARN appears occasionally on the log file when using BoundedThreadPool. May be synchronization caused?

      After using the ThreadPoolExecutor, there is not any warn.

      For the max pool size(20000) mentioned above, there are no relations with this problem, because the active thread count can't get to the max pool size on current circumstance .

      The WARN IS:
      2007-10-22 11:05:09.672::WARN: dispatch failed!
      2007-10-22 11:05:09.672::WARN: EXCEPTION
      java.lang.IllegalStateException
      at org.mortbay.thread.BoundedThreadPool$PoolThread.dispatch(BoundedThreadPool.java:524)
      at org.mortbay.thread.BoundedThreadPool.dispatch(BoundedThreadPool.java:86)
      at org.mortbay.jetty.nio.SelectChannelConnector$1.dispatch(SelectChannelConnector.java:86)
      at org.mortbay.io.nio.SelectChannelEndPoint.dispatch(SelectChannelEndPoint.java:68)
      at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:476)
      at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:168)
      at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
      at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:517)
      at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

      Show
      Boyce Lee added a comment - I don't know why the WARN appears occasionally on the log file when using BoundedThreadPool. May be synchronization caused? After using the ThreadPoolExecutor, there is not any warn. For the max pool size(20000) mentioned above, there are no relations with this problem, because the active thread count can't get to the max pool size on current circumstance . The WARN IS: 2007-10-22 11:05:09.672::WARN: dispatch failed! 2007-10-22 11:05:09.672::WARN: EXCEPTION java.lang.IllegalStateException at org.mortbay.thread.BoundedThreadPool$PoolThread.dispatch(BoundedThreadPool.java:524) at org.mortbay.thread.BoundedThreadPool.dispatch(BoundedThreadPool.java:86) at org.mortbay.jetty.nio.SelectChannelConnector$1.dispatch(SelectChannelConnector.java:86) at org.mortbay.io.nio.SelectChannelEndPoint.dispatch(SelectChannelEndPoint.java:68) at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:476) at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:168) at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124) at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:517) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

        People

        • Assignee:
          Greg Wilkins
          Reporter:
          Boyce Lee
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved: