JRuby

fork+exec via ffi fails sometimes (mac os x)

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Won't Fix
  • Affects Version/s: None
  • Fix Version/s: JRuby 1.x+
  • Component/s: Extensions
  • Labels:
    None
  • Environment:
    Mac OS X, Java 6
  • Number of attachments :
    0

Description

{{
$ java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06-153)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_07-b06-57, mixed mode)
$ jruby -v
jruby 1.2.0 (ruby 1.8.6 patchlevel 287) (2009-04-23 rev 6586) [i386-java]
$ curl http://gist.github.com/111495 > vim
$ jirb
irb > load 'vim'
irb > true
irb > load 'vim'
irb > true
irb > load 'vim'
irb > true
irb > load 'vim'
irb > true
irb > load 'vim'
irb > true
[vim doesn't execute], just returns true
}}

doesn't happen after a fix number of loads, after around ~ 10 loads. works ok on linux.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

Hmm, I tried moving some of the arguments out, but it still seemed to do this. This may be a question for Wayne. Wayne, is it possible that we're still doing "too much" work after the fork? Maybe we'd be best providing this as a bare-metal Java-based extension we know doesn't do any additional allocation after the fork? Also, I'm guessing that the JVM forces everything to reach a safe point before doing fork+exec...maybe there's some way we can do the same? I'm checking OpenJDK source now.

Show
Charles Oliver Nutter added a comment - Hmm, I tried moving some of the arguments out, but it still seemed to do this. This may be a question for Wayne. Wayne, is it possible that we're still doing "too much" work after the fork? Maybe we'd be best providing this as a bare-metal Java-based extension we know doesn't do any additional allocation after the fork? Also, I'm guessing that the JVM forces everything to reach a safe point before doing fork+exec...maybe there's some way we can do the same? I'm checking OpenJDK source now.
Hide
Permalink
Charles Oliver Nutter added a comment -

Ok, I found one path to a solution for BSDs that apparently doesn't work on Darwin. On FreeBSD, OpenJDK does this wrapping logic around fork:

require 'ffi'

module Exec
  extend FFI::Library

  attach_function :my_exec, :execl, [:string, :string, :varargs], :int
  attach_function :sys_fork, :__sys_fork, [], :int
  attach_function :thread_kern_sig_defer, :_thread_kern_sig_defer, [], :void
  attach_function :thread_kern_sig_undefer, :_thread_kern_sig_undefer

  def self.fork
    thread_kern_sig_defer
    pid = sys_fork
    if pid != 0
      thread_kern_sig_undefer
    end
    pid
  end
end

vim1 = '/usr/bin/vim'
vim2 = 'vim'
ptr = :pointer
ex = Exec
args = [vim1,vim2,ptr,nil]
if Exec.fork == 0
  p ex.my_exec *args
end

Process.waitall

__sys_fork does not exist on Darwin, so I'm trying another option now.

Show
Charles Oliver Nutter added a comment - Ok, I found one path to a solution for BSDs that apparently doesn't work on Darwin. On FreeBSD, OpenJDK does this wrapping logic around fork: require 'ffi' module Exec extend FFI::Library attach_function :my_exec, :execl, [:string, :string, :varargs], :int attach_function :sys_fork, :__sys_fork, [], :int attach_function :thread_kern_sig_defer, :_thread_kern_sig_defer, [], :void attach_function :thread_kern_sig_undefer, :_thread_kern_sig_undefer def self.fork thread_kern_sig_defer pid = sys_fork if pid != 0 thread_kern_sig_undefer end pid end end vim1 = '/usr/bin/vim' vim2 = 'vim' ptr = :pointer ex = Exec args = [vim1,vim2,ptr,nil] if Exec.fork == 0 p ex.my_exec *args end Process.waitall __sys_fork does not exist on Darwin, so I'm trying another option now.
Hide
Permalink
Charles Oliver Nutter added a comment -

Ok, my other attempt, using vfork, did not appear to work correctly. vfork requires that you do no system calls other than _exit or execve, and there's a lot of overhead between the Ruby FFI call to vfork and the eventual execl call above. So that won't fly unless we're able to move them really close together, which may require doing it in C code.

But the FreeBSD code above makes me wonder if we need to disable signals on Darwin too. I did run into sigaction(2) which can disable signal handling in the child process. It's possible that the reason it fails intermittently is that a signal is received for whatever purpose and the JVM tries to handle it, but can't.

Show
Charles Oliver Nutter added a comment - Ok, my other attempt, using vfork, did not appear to work correctly. vfork requires that you do no system calls other than _exit or execve, and there's a lot of overhead between the Ruby FFI call to vfork and the eventual execl call above. So that won't fly unless we're able to move them really close together, which may require doing it in C code. But the FreeBSD code above makes me wonder if we need to disable signals on Darwin too. I did run into sigaction(2) which can disable signal handling in the child process. It's possible that the reason it fails intermittently is that a signal is received for whatever purpose and the JVM tries to handle it, but can't.
Hide
Permalink
Wayne Meissner added a comment -

This isn't really a jruby or ffi problem - its an artifact of the way the jvm works - you simply cannot fork and reliably execute java code afterwards. The posix_spawn(3) api solves this for the fork+exec case, and is packaged up in Charlie's 'spoon' gem.

Show
Wayne Meissner added a comment - This isn't really a jruby or ffi problem - its an artifact of the way the jvm works - you simply cannot fork and reliably execute java code afterwards. The posix_spawn(3) api solves this for the fork+exec case, and is packaged up in Charlie's 'spoon' gem.

People

  • Assignee:
    Unassigned
    Reporter:
    Jan Berkel
Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: