Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.6.5
    • Fix Version/s: JRuby 1.7.5
    • Component/s: Standard Library
    • Labels:
      None
    • Environment:
      Mac OSX.
    • Number of attachments :
      0

      Description

      It appears Open3.popen3(...) is broken on JRuby in 1.9 mode.
      On 1.8.7, it returns three arguments (stdin, stdout, and stderr).
      On 1.9.X, it returns (stdin, stdout, stderr, and a thread).

      You can use the thread argument to access a Process::Status object,
      and the PID of the spawned command.

      To reproduce:

      stdin, stdout, stderr, thr = Open3.popen3 "ls"
      thr.value # => NoMethodError (called on nil)

      Thanks!

        Issue Links

          Activity

          Hide
          Hiro Asari added a comment -

          I note here that Open3::popen3 has a short circuit that returns IO::open3 for JRuby (https://github.com/jruby/jruby/blob/3b60780b07e8dc5e480987357098756bd281ddf0/lib/ruby/1.9/open3.rb#L74).

          It was not immediately obvious to me how we might reconcile this with IO::open3 alone. We can get the child Java Process easily, but it seemed nontrivial to build a RubyThread from it.

          Show
          Hiro Asari added a comment - I note here that Open3::popen3 has a short circuit that returns IO::open3 for JRuby ( https://github.com/jruby/jruby/blob/3b60780b07e8dc5e480987357098756bd281ddf0/lib/ruby/1.9/open3.rb#L74 ). It was not immediately obvious to me how we might reconcile this with IO::open3 alone. We can get the child Java Process easily, but it seemed nontrivial to build a RubyThread from it.
          Hide
          Robert Gleeson added a comment -

          Oh okay, do you think it is a solvable problem?
          I'd like to add JRuby compatibility to a gem, and
          this is a blocker at the moment.

          Show
          Robert Gleeson added a comment - Oh okay, do you think it is a solvable problem? I'd like to add JRuby compatibility to a gem, and this is a blocker at the moment.
          Hide
          Hiro Asari added a comment -

          So, there is a way to build a RubyThread from a Java Process (RubyProcess.detach()).

          A naive approach goes something like this:

          POpenTuple tuple = popenSpecial(context, args);
          long pid = ShellLauncher.getPidFromProcess(tuple.process);
          IRubyObject thr = RubyProcess.detach(context, runtime.getProcess(), runtime.newFixnum(pid));
          

          but this fails rather miserably.

          $ jruby -S irb
          irb(main):001:0> require 'open3'; i,o,e,t = Open3.popen3("ls")
          => [#<IO:fd 140>, #<IO:fd 139>, #<IO:fd 141>, #<Thread:0x2eb86778 run>]
          irb(main):002:0> t
          => #<Thread:0x2eb86778 dead>
          irb(main):003:0> t.value
          Errno::ECHILD: No child processes - No child processes
          

          I'm probably missing something obvious. I welcome suggestions and improvements.

          Show
          Hiro Asari added a comment - So, there is a way to build a RubyThread from a Java Process ( RubyProcess.detach() ). A naive approach goes something like this: POpenTuple tuple = popenSpecial(context, args); long pid = ShellLauncher.getPidFromProcess(tuple.process); IRubyObject thr = RubyProcess.detach(context, runtime.getProcess(), runtime.newFixnum(pid)); but this fails rather miserably. $ jruby -S irb irb(main):001:0> require 'open3'; i,o,e,t = Open3.popen3("ls") => [#<IO:fd 140>, #<IO:fd 139>, #<IO:fd 141>, #<Thread:0x2eb86778 run>] irb(main):002:0> t => #<Thread:0x2eb86778 dead> irb(main):003:0> t.value Errno::ECHILD: No child processes - No child processes I'm probably missing something obvious. I welcome suggestions and improvements.
          Hide
          Charles Oliver Nutter added a comment -

          Fix in 1.7.1 along with other process-launching APIs.

          Show
          Charles Oliver Nutter added a comment - Fix in 1.7.1 along with other process-launching APIs.
          Hide
          Steven Arnold added a comment -

          I tried this same thing in jruby-1.7.2 and in jruby-head (1.7.3 pre), but the problem still seems to exist.

          [ asymptotic ] > irb
          jruby-1.7.3.dev:001 > require 'open3'
          => true
          jruby-1.7.3.dev:002 > Open3::popen3('ls')
          => #<IO:fd 9>, #<IO:fd 8>, #<IO:fd 10>

          Note that only three objects are returned, not the expected four. The thread object is missing. If this was fixed in 1.7.1, it seems maybe the problem recurred in 1.7.2.

          Show
          Steven Arnold added a comment - I tried this same thing in jruby-1.7.2 and in jruby-head (1.7.3 pre), but the problem still seems to exist. [ asymptotic ] > irb jruby-1.7.3.dev:001 > require 'open3' => true jruby-1.7.3.dev:002 > Open3::popen3('ls') => #<IO:fd 9>, #<IO:fd 8>, #<IO:fd 10> Note that only three objects are returned, not the expected four. The thread object is missing. If this was fixed in 1.7.1, it seems maybe the problem recurred in 1.7.2.
          Hide
          Charles Oliver Nutter added a comment -

          Poking at this today. Need to understand what the thread is for. I presume it's similar to the JVM, waiting on process exit.

          Show
          Charles Oliver Nutter added a comment - Poking at this today. Need to understand what the thread is for. I presume it's similar to the JVM, waiting on process exit.
          Hide
          Charles Oliver Nutter added a comment -

          Yep, that's basically what it's doing. Oddly enough, the open3 logic fits better with JVM process management than most other stuff. I think I can come up with something that will work ok for now.

          Show
          Charles Oliver Nutter added a comment - Yep, that's basically what it's doing. Oddly enough, the open3 logic fits better with JVM process management than most other stuff. I think I can come up with something that will work ok for now.
          Hide
          Charles Oliver Nutter added a comment -

          Squeaky wheel gets the grease.

          commit 7dba21754899867f048559d3712442996ec07a77
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Tue Jun 25 11:48:15 2013 -0500
          
              Implement thread retval for 1.9's Open3.popen3. JRUBY-6409
              
              It turns out that the thread value returned from 1.9's popen3 is
              actually easy to support by simply calling java.lang.Process's
              waitFor method. This is, in fact, much more reliable than a normal
              popen + waitpid due to our racing the JVM to the waitpid call.
              
              There's still issues with the interactivity of the streams (on
              some JVMs/versions) and overall JRuby process management, but this
              patch should allow popen3 to work properly.
              
              Unexcluded some popen3 tests that pass now, too (yay!)
          
          :100644 100644 b7d911f... c094a39... M	src/jruby/kernel19/process.rb
          :100644 100644 194c21d... 57926b9... M	src/org/jruby/RubyIO.java
          :100644 100644 3c3884a... 85e8b5f... M	src/org/jruby/RubyThread.java
          :100644 100644 2f5da5f... b025c4b... M	src/org/jruby/internal/runtime/FutureThread.java
          :100644 100644 22344ce... 2007da5... M	src/org/jruby/internal/runtime/RubyRunnable.java
          :000000 100644 0000000... 1f6f8f1... A	src/org/jruby/internal/runtime/ThreadedRunnable.java
          :100644 100644 2796c03... 7c5b002... M	test/externals/ruby1.9/excludes/TestOpen3.rb
          
          Show
          Charles Oliver Nutter added a comment - Squeaky wheel gets the grease. commit 7dba21754899867f048559d3712442996ec07a77 Author: Charles Oliver Nutter <headius@headius.com> Date: Tue Jun 25 11:48:15 2013 -0500 Implement thread retval for 1.9's Open3.popen3. JRUBY-6409 It turns out that the thread value returned from 1.9's popen3 is actually easy to support by simply calling java.lang.Process's waitFor method. This is, in fact, much more reliable than a normal popen + waitpid due to our racing the JVM to the waitpid call. There's still issues with the interactivity of the streams (on some JVMs/versions) and overall JRuby process management, but this patch should allow popen3 to work properly. Unexcluded some popen3 tests that pass now, too (yay!) :100644 100644 b7d911f... c094a39... M src/jruby/kernel19/process.rb :100644 100644 194c21d... 57926b9... M src/org/jruby/RubyIO.java :100644 100644 3c3884a... 85e8b5f... M src/org/jruby/RubyThread.java :100644 100644 2f5da5f... b025c4b... M src/org/jruby/internal/runtime/FutureThread.java :100644 100644 22344ce... 2007da5... M src/org/jruby/internal/runtime/RubyRunnable.java :000000 100644 0000000... 1f6f8f1... A src/org/jruby/internal/runtime/ThreadedRunnable.java :100644 100644 2796c03... 7c5b002... M test/externals/ruby1.9/excludes/TestOpen3.rb

            People

            • Assignee:
              Charles Oliver Nutter
              Reporter:
              Robert Gleeson
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: