JRuby

Using a shared listener socket causes java.nio.channels.IllegalBlockingModeException exceptions to be raised

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Won't Fix
  • Affects Version/s: JRuby 1.6.7
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    JRuby 1.6.7 as well as head on Mac OSX.
  • Testcase included:
    yes
  • Number of attachments :
    0

Description

I recently wrote an example that uses multiple acceptor threads that all listen on the same listener socket. This works fine on MRI 1.9.3, however repeatedly fails in JRuby. The sample app can be found here: https://github.com/ryanlecompte/timeserver

One server is running, simply telnet localhost 3000 and you will see the error.

Server launched: 10 acceptors, 10 workers.

Exception in thread "RubyThread-34: /Users/ryan/fun/timeserver/lib/timeserver/server.rb:37" java.nio.channels.IllegalBlockingModeException
at java.nio.channels.spi.AbstractSelectableChannel.configureBlocking(AbstractSelectableChannel.java:257)
at org.jruby.util.io.SelectBlob.tidyUp(SelectBlob.java:352)
at org.jruby.util.io.SelectBlob.goForIt(SelectBlob.java:94)
at org.jruby.RubyIO.select_static(RubyIO.java:3398)
at org.jruby.RubyIO.select(RubyIO.java:3394)
at org.jruby.RubyIO$INVOKER$s$0$3$select.call(RubyIO$INVOKER$s$0$3$select.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:69)
at org.jruby.ast.CallManyArgsNode.interpret(CallManyArgsNode.java:59)
at org.jruby.ast.IfNode.interpret(IfNode.java:111)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.WhileNode.interpret(WhileNode.java:131)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:209)
at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:197)
at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:128)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:269)
at org.jruby.RubyProc.call(RubyProc.java:223)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:101)
at java.lang.Thread.run(Thread.java:680)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

Ok, finally circled around to this one.

I think your design may be flawed. Your code has N threads all listening for connect events on the same server socket. In MRI, this might work ok because only one thread can be woken up at a time. In JRuby, when the select event fires, potentially all of the threads wake up and try to accept the connection. It's not surprising that you get a cascade of errors as a result.

The error itself is caused because one thread "wins" at selecting and then tries to set the socket back into blocking mode. But other threads are still using the socket in their selectors, and you can't set a socket to blocking when it's still part of a selection event.

I think what you really want to do here is to have a single thread that's doing the select + accept_nonblock, and it parcels out the resulting client socket to a pool of threads waiting to handle requests. That will guarantee you only have one thread that's selecting on the server socket, and you still have N threads handling the actual requests.

I will clean up the error here, but I don't think it's possible to fix JRuby to work the way your code wants it to work.

Show
Charles Oliver Nutter added a comment - Ok, finally circled around to this one. I think your design may be flawed. Your code has N threads all listening for connect events on the same server socket. In MRI, this might work ok because only one thread can be woken up at a time. In JRuby, when the select event fires, potentially all of the threads wake up and try to accept the connection. It's not surprising that you get a cascade of errors as a result. The error itself is caused because one thread "wins" at selecting and then tries to set the socket back into blocking mode. But other threads are still using the socket in their selectors, and you can't set a socket to blocking when it's still part of a selection event. I think what you really want to do here is to have a single thread that's doing the select + accept_nonblock, and it parcels out the resulting client socket to a pool of threads waiting to handle requests. That will guarantee you only have one thread that's selecting on the server socket, and you still have N threads handling the actual requests. I will clean up the error here, but I don't think it's possible to fix JRuby to work the way your code wants it to work.
Hide
Permalink
Charles Oliver Nutter added a comment -

So it's possible there's another explanation: all threads don't wake up, but because some of them are still in selector logic the socket can't be set back to blocking. This would be a problem for any JVM-based framework that's trying to do both blocking and nonblocking operations on a socket while simultaneously trying to do concurrent selection.

We talked a bit on IRC, and here's the salient points:

  • JRuby's IO methods generally mimic MRI's single-threadedness, and so concurrent operations against them can sometimes act oddly
  • NIO channels certainly could support concurrent selects, but not concurrent selects in the presence of both blocking and nonblocking operations
  • nio4r may be a way to do this without using JRuby's MRI-oriented APIs
  • It may also simply be best to have a single acceptor thread farming out client sockets to the workers, rather than several acceptor threads.

I'm going to resolve this as wontfix, since there's not really anything we can do right now with the way we emulate MRI's behavior. I will improve the error such that it fails hard with a useful message when multiple threads fail to do concurrent selects of the same socket.

Show
Charles Oliver Nutter added a comment - So it's possible there's another explanation: all threads don't wake up, but because some of them are still in selector logic the socket can't be set back to blocking. This would be a problem for any JVM-based framework that's trying to do both blocking and nonblocking operations on a socket while simultaneously trying to do concurrent selection. We talked a bit on IRC, and here's the salient points: JRuby's IO methods generally mimic MRI's single-threadedness, and so concurrent operations against them can sometimes act oddly NIO channels certainly could support concurrent selects, but not concurrent selects in the presence of both blocking and nonblocking operations nio4r may be a way to do this without using JRuby's MRI-oriented APIs It may also simply be best to have a single acceptor thread farming out client sockets to the workers, rather than several acceptor threads. I'm going to resolve this as wontfix, since there's not really anything we can do right now with the way we emulate MRI's behavior. I will improve the error such that it fails hard with a useful message when multiple threads fail to do concurrent selects of the same socket.

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Ryan LeCompte
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: