JRuby

DRb request/response problem

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 0.9.0
  • Fix Version/s: JRuby 0.9.1
  • Component/s: Core Classes/Modules
  • Labels:
    None
  • Number of attachments :
    3

Description

I've been trying unsucessfully to get DRb working under JRuby. Here are my test server and client:

test_server.rb
#!/usr/bin/env jruby

require 'drb/drb'

URI="druby://localhost:9876"

class TestServer
 def initialize
   puts "Server started."
 end

 def get_hello
   puts "Hello Server!"
   "Hello Client!"
 end
end

FRONT_OBJECT = TestServer.new
$SAFE = 1
DRb.start_service(URI,FRONT_OBJECT)
DRb.thread.join
test_client.rb
#!/usr/bin/env ruby
require 'drb/drb'

SERVER_URI="druby://localhost:9876"

DRb.start_service

test = DRbObject.new_with_uri(SERVER_URI)
puts test.get_hello

If you notice, the server runs under JRuby, but the client does not. This is by design, as the app I'm trying to build will require that setup. In this configuration, DRb 'half' works. The server runs just fine, and when I run the client, "Hello Server!" prints on the server console. But the client never seems to get a return value, and just hangs. Here's a table of what happens when I change either the server or client to use either ruby or jruby:

Server Client Result
jruby ruby Server prints "Hello Server!", client hangs
jruby jruby Server prints nothing, client hangs
ruby ruby Server prints "Hello Server!", client prints "Hello Client!" and exits
ruby jruby Server prints nothing, client hangs

I've tried this with both the blackdown 1.4.2 JVM and the Sun 1.5 JVM.

Attachments

  1. Text File
    puts_newline_write.patch
    07/Aug/06 10:31 PM
    3 kB
    Blane Dabney
  2. Text File
    puts_newline_write.patch
    07/Aug/06 6:14 PM
    3 kB
    Blane Dabney
  3. Text File
    test_socket_readwrite.patch
    07/Aug/06 8:10 PM
    1 kB
    Charles Oliver Nutter

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Blane Dabney added a comment -

Forgot to mention, this is with jruby 0.9.0

Show
Blane Dabney added a comment - Forgot to mention, this is with jruby 0.9.0
Hide
Permalink
Charles Oliver Nutter added a comment -

Here's a relevant bit of the thread dump when the JRuby-based client hangs:

"main" prio=1 tid=0x0000000040116290 nid=0x4828 runnable [0x00007ffffff08000..0x 00007ffffff0c630]
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
at sun.nio.ch.IOUtil.read(IOUtil.java:206)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:207)

  • locked <0x00002aaaf0396628> (a java.lang.Object)
    at org.jruby.util.IOHandlerNio.fillInBuffer(IOHandlerNio.java:193)
    at org.jruby.util.IOHandlerNio.read(IOHandlerNio.java:261)
    at org.jruby.RubyIO.read(RubyIO.java:953)

There's also a thread waiting to accept, but I assume that's for the response. Since the client's message doesn't seem to get out of the client process, that thread is probably not as interesting.

Show
Charles Oliver Nutter added a comment - Here's a relevant bit of the thread dump when the JRuby-based client hangs: "main" prio=1 tid=0x0000000040116290 nid=0x4828 runnable [0x00007ffffff08000..0x 00007ffffff0c630] at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) at sun.nio.ch.IOUtil.read(IOUtil.java:206) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:207)
  • locked <0x00002aaaf0396628> (a java.lang.Object) at org.jruby.util.IOHandlerNio.fillInBuffer(IOHandlerNio.java:193) at org.jruby.util.IOHandlerNio.read(IOHandlerNio.java:261) at org.jruby.RubyIO.read(RubyIO.java:953)
There's also a thread waiting to accept, but I assume that's for the response. Since the client's message doesn't seem to get out of the client process, that thread is probably not as interesting.
Hide
Permalink
Blane Dabney added a comment -

I've examined the DRb code, and have managed to bastardize a test out of the DRbMessage class. It behaves ALMOST exactly the same as with the DRb test server and client I gave before:

socket_test_server.rb
require 'socket'
require 'thread'

def dump(obj)
  str = Marshal::dump(obj)
  [str.size].pack('N') + str
end

def load(soc)  # :nodoc:
  begin
    sz = soc.read(4)  # sizeof (N)
  rescue
    raise $!.message
  end
  raise 'connection closed' if sz.nil?
  raise 'premature header' if sz.size < 4
  sz = sz.unpack('N')[0]
  begin
    str = soc.read(sz)
  rescue
    raise $!.message
  end
  raise 'connection closed' if str.nil?
  raise 'premature marshal format(can\'t read)' if str.size < sz
  Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end
end

serv = TCPServer.new('localhost',2202)
sock = serv.accept

args = []
num = load(sock)
num.times {|n| args.push load(sock)}
puts args.join('')
sock.write(dump("Hello ")+dump("world!"))
sock.close
socket_test_client.rb
require 'socket'
require 'thread'

def dump(obj)
  str = Marshal::dump(obj)
  [str.size].pack('N') + str
end

def load(soc)  # :nodoc:
  begin
    sz = soc.read(4)  # sizeof (N)
  rescue
    raise $!.message
  end
  raise 'connection closed' if sz.nil?
  raise 'premature header' if sz.size < 4
  sz = sz.unpack('N')[0]
  begin
    str = soc.read(sz)
  rescue
    raise $!.message
  end
  raise 'connection closed' if str.nil?
  raise 'premature marshal format(can\'t read)' if str.size < sz
  Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end
end

socket = TCPSocket.new("localhost",2202)

args = []
args.push(dump(4))
args.push(dump("This\n"))
args.push(dump("is\n"))
args.push(dump("a\n"))
args.push(dump("test"))
socket.write args.join('')
str = load(socket) + load(socket)
puts str
socket.close

The dump and load methods are copied almost verbatim from the DRb code. The changes I made to dump may account for the fact that with this test code, running the server in jruby and the client in ruby DOES work. However, running the server in ruby and the client in jruby produces the exact same problem. Here's the relevant portion of my stack trace:

org.jruby.Main at localhost:59307	
	Thread [main] (Suspended)	
		FileDispatcher.read0(FileDescriptor, long, int) line: not available [native method]	
		SocketDispatcher.read(FileDescriptor, long, int) line: 21	
		IOUtil.readIntoNativeBuffer(FileDescriptor, ByteBuffer, long, NativeDispatcher, Object) line: 233	
		IOUtil.read(FileDescriptor, ByteBuffer, long, NativeDispatcher, Object) line: 206	
		SocketChannelImpl.read(ByteBuffer) line: 207	
		IOHandlerNio.fillInBuffer() line: 193	
		IOHandlerNio.read(int) line: 261	
		RubyBasicSocket(RubyIO).read(IRubyObject[]) line: 954

It might be important to note that if you remove the reply code from the server (the socket.write before the socket.close_ and the receiving code in the client (str = load(socket) + load(socket), and the following puts), it works fine in all cases.

Show
Blane Dabney added a comment - I've examined the DRb code, and have managed to bastardize a test out of the DRbMessage class. It behaves ALMOST exactly the same as with the DRb test server and client I gave before:
socket_test_server.rb
require 'socket'
require 'thread'

def dump(obj)
  str = Marshal::dump(obj)
  [str.size].pack('N') + str
end

def load(soc)  # :nodoc:
  begin
    sz = soc.read(4)  # sizeof (N)
  rescue
    raise $!.message
  end
  raise 'connection closed' if sz.nil?
  raise 'premature header' if sz.size < 4
  sz = sz.unpack('N')[0]
  begin
    str = soc.read(sz)
  rescue
    raise $!.message
  end
  raise 'connection closed' if str.nil?
  raise 'premature marshal format(can\'t read)' if str.size < sz
  Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end
end

serv = TCPServer.new('localhost',2202)
sock = serv.accept

args = []
num = load(sock)
num.times {|n| args.push load(sock)}
puts args.join('')
sock.write(dump("Hello ")+dump("world!"))
sock.close
socket_test_client.rb
require 'socket'
require 'thread'

def dump(obj)
  str = Marshal::dump(obj)
  [str.size].pack('N') + str
end

def load(soc)  # :nodoc:
  begin
    sz = soc.read(4)  # sizeof (N)
  rescue
    raise $!.message
  end
  raise 'connection closed' if sz.nil?
  raise 'premature header' if sz.size < 4
  sz = sz.unpack('N')[0]
  begin
    str = soc.read(sz)
  rescue
    raise $!.message
  end
  raise 'connection closed' if str.nil?
  raise 'premature marshal format(can\'t read)' if str.size < sz
  Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end
end

socket = TCPSocket.new("localhost",2202)

args = []
args.push(dump(4))
args.push(dump("This\n"))
args.push(dump("is\n"))
args.push(dump("a\n"))
args.push(dump("test"))
socket.write args.join('')
str = load(socket) + load(socket)
puts str
socket.close
The dump and load methods are copied almost verbatim from the DRb code. The changes I made to dump may account for the fact that with this test code, running the server in jruby and the client in ruby DOES work. However, running the server in ruby and the client in jruby produces the exact same problem. Here's the relevant portion of my stack trace: 
org.jruby.Main at localhost:59307	
	Thread [main] (Suspended)	
		FileDispatcher.read0(FileDescriptor, long, int) line: not available [native method]	
		SocketDispatcher.read(FileDescriptor, long, int) line: 21	
		IOUtil.readIntoNativeBuffer(FileDescriptor, ByteBuffer, long, NativeDispatcher, Object) line: 233	
		IOUtil.read(FileDescriptor, ByteBuffer, long, NativeDispatcher, Object) line: 206	
		SocketChannelImpl.read(ByteBuffer) line: 207	
		IOHandlerNio.fillInBuffer() line: 193	
		IOHandlerNio.read(int) line: 261	
		RubyBasicSocket(RubyIO).read(IRubyObject[]) line: 954
It might be important to note that if you remove the reply code from the server (the socket.write before the socket.close_ and the receiving code in the client (str = load(socket) + load(socket), and the following puts), it works fine in all cases.
Hide
Permalink
Blane Dabney added a comment -

Threads do not appear to be the issue at all here. changing this:

Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end

to this:

Marshal::load(str)

in both client and server produces the exact same effects (both work in ruby, and when the client is ruby, but not if the client is jruby)

Show
Blane Dabney added a comment - Threads do not appear to be the issue at all here. changing this: 
Thread.exclusive do
    begin
      save = Thread.current[:drb_untaint]
      Thread.current[:drb_untaint] = []
      Marshal::load(str)
    rescue NameError, ArgumentError
      Object.new($!, str)
    ensure
      Thread.current[:drb_untaint].each do |x|
        x.untaint
      end
      Thread.current[:drb_untaint] = save
    end
  end
to this: 
Marshal::load(str)
in both client and server produces the exact same effects (both work in ruby, and when the client is ruby, but not if the client is jruby)
Hide
Permalink
Blane Dabney added a comment -

Ok, I've narrowed this down to something much more basic. I really should have done this test first:

Unable to find source-code formatter for language: working_server.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

serv = TCPServer.new('localhost',2202)
sock = serv.accept

puts sock.gets
sock.puts "world!"
sock.close
Unable to find source-code formatter for language: working_client.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

socket = TCPSocket.new("localhost",2202)

socket.puts "Hello"
puts socket.gets
socket.close

Compare that to this:

Unable to find source-code formatter for language: broken_server.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

serv = TCPServer.new('localhost',2202)
sock = serv.accept

puts sock.read(5)
sock.write "world!"
sock.close
Unable to find source-code formatter for language: broken_client.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

socket = TCPSocket.new("localhost",2202)

socket.write "Hello"
puts socket.read(6)
socket.close

The working client/server works in all cases, the broken ones produce the same effect as before.

Show
Blane Dabney added a comment - Ok, I've narrowed this down to something much more basic. I really should have done this test first:
Unable to find source-code formatter for language: working_server.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

serv = TCPServer.new('localhost',2202)
sock = serv.accept

puts sock.gets
sock.puts "world!"
sock.close
Unable to find source-code formatter for language: working_client.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

socket = TCPSocket.new("localhost",2202)

socket.puts "Hello"
puts socket.gets
socket.close
Compare that to this:
Unable to find source-code formatter for language: broken_server.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

serv = TCPServer.new('localhost',2202)
sock = serv.accept

puts sock.read(5)
sock.write "world!"
sock.close
Unable to find source-code formatter for language: broken_client.rb. Available languages are: javascript, sql, xhtml, actionscript, none, html, xml, java
require 'socket'

socket = TCPSocket.new("localhost",2202)

socket.write "Hello"
puts socket.read(6)
socket.close
The working client/server works in all cases, the broken ones produce the same effect as before.
Hide
Permalink
Blane Dabney added a comment -

I found that the problem to this patch was that the IOHandlerNio.write method was parsing for newlines, to prevent puts from sending the newline as a separate native write when called without a newline in the input string. Removing the newline parsing code from write fixed this bug, and modifying RubyIO.puts to concatenate the newline (if none was given) with the input string before sending it to write prevents write from doing two native writes for puts.

Show
Blane Dabney added a comment - I found that the problem to this patch was that the IOHandlerNio.write method was parsing for newlines, to prevent puts from sending the newline as a separate native write when called without a newline in the input string. Removing the newline parsing code from write fixed this bug, and modifying RubyIO.puts to concatenate the newline (if none was given) with the input string before sending it to write prevents write from doing two native writes for puts.
Hide
Permalink
Charles Oliver Nutter added a comment -

Added a patch for a simple socket read/write test based on the reduced failure case. Not committing yet because there are still issues with the patch.

Show
Charles Oliver Nutter added a comment - Added a patch for a simple socket read/write test based on the reduced failure case. Not committing yet because there are still issues with the patch.
Hide
Permalink
Charles Oliver Nutter added a comment -

The patch looks good for the basic test case given, but appears to break WEBrick. With the patch, Rails's script/server appears to hang trying to deliver request responses. Interrupting the server causes the responses to be delivered.

Show
Charles Oliver Nutter added a comment - The patch looks good for the basic test case given, but appears to break WEBrick. With the patch, Rails's script/server appears to hang trying to deliver request responses. Interrupting the server causes the responses to be delivered.
Hide
Permalink
Blane Dabney added a comment -

I forgot to wrap the flush outside the loop in an !isSync() if block. This fixes a bug I noticed with WEBrick, though I couldn't reproduce headius' bug.

Show
Blane Dabney added a comment - I forgot to wrap the flush outside the loop in an !isSync() if block. This fixes a bug I noticed with WEBrick, though I couldn't reproduce headius' bug.
Hide
Permalink
Charles Oliver Nutter added a comment -

The updated patch does appear to fix my issue with WEBrick, so we may have a good patch here. It fixes a fairly major breakage with read/write in IOHandlerNIO and anything it might break does not appear to show up in any of my tests. I'll let this stew for a day, and then we'll go ahead and commit it.

Show
Charles Oliver Nutter added a comment - The updated patch does appear to fix my issue with WEBrick, so we may have a good patch here. It fixes a fairly major breakage with read/write in IOHandlerNIO and anything it might break does not appear to show up in any of my tests. I'll let this stew for a day, and then we'll go ahead and commit it.
Hide
Permalink
Charles Oliver Nutter added a comment -

Committed updated patch and tests. Everything seems to work well.

Show
Charles Oliver Nutter added a comment - Committed updated patch and tests. Everything seems to work well.
Hide
Permalink
Charles Oliver Nutter added a comment -

Closing 0.9.1 resolved issues.

Show
Charles Oliver Nutter added a comment - Closing 0.9.1 resolved issues.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.