Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.6.5
    • Fix Version/s: JRuby 1.7.5
    • Component/s: Standard Library
    • Labels:
      None
    • Environment:
      Mac OSX.
    • Number of attachments :
      0

      Description

      It appears Open3.popen3(...) is broken on JRuby in 1.9 mode.
      On 1.8.7, it returns three arguments (stdin, stdout, and stderr).
      On 1.9.X, it returns (stdin, stdout, stderr, and a thread).

      You can use the thread argument to access a Process::Status object,
      and the PID of the spawned command.

      To reproduce:

      stdin, stdout, stderr, thr = Open3.popen3 "ls"
      thr.value # => NoMethodError (called on nil)

      Thanks!

        Issue Links

          Activity

          Hide
          Hiro Asari added a comment -

          I note here that Open3::popen3 has a short circuit that returns IO::open3 for JRuby (https://github.com/jruby/jruby/blob/3b60780b07e8dc5e480987357098756bd281ddf0/lib/ruby/1.9/open3.rb#L74).

          It was not immediately obvious to me how we might reconcile this with IO::open3 alone. We can get the child Java Process easily, but it seemed nontrivial to build a RubyThread from it.

          Show
          Hiro Asari added a comment - I note here that Open3::popen3 has a short circuit that returns IO::open3 for JRuby ( https://github.com/jruby/jruby/blob/3b60780b07e8dc5e480987357098756bd281ddf0/lib/ruby/1.9/open3.rb#L74 ). It was not immediately obvious to me how we might reconcile this with IO::open3 alone. We can get the child Java Process easily, but it seemed nontrivial to build a RubyThread from it.
          Hide
          Robert Gleeson added a comment -

          Oh okay, do you think it is a solvable problem?
          I'd like to add JRuby compatibility to a gem, and
          this is a blocker at the moment.

          Show
          Robert Gleeson added a comment - Oh okay, do you think it is a solvable problem? I'd like to add JRuby compatibility to a gem, and this is a blocker at the moment.
          Hiro Asari made changes -
          Field Original Value New Value
          Priority Major [ 3 ] Critical [ 2 ]
          Hiro Asari made changes -
          Link This issue is duplicated by JRUBY-6539 [ JRUBY-6539 ]
          Hide
          Hiro Asari added a comment -

          So, there is a way to build a RubyThread from a Java Process (RubyProcess.detach()).

          A naive approach goes something like this:

          POpenTuple tuple = popenSpecial(context, args);
          long pid = ShellLauncher.getPidFromProcess(tuple.process);
          IRubyObject thr = RubyProcess.detach(context, runtime.getProcess(), runtime.newFixnum(pid));
          

          but this fails rather miserably.

          $ jruby -S irb
          irb(main):001:0> require 'open3'; i,o,e,t = Open3.popen3("ls")
          => [#<IO:fd 140>, #<IO:fd 139>, #<IO:fd 141>, #<Thread:0x2eb86778 run>]
          irb(main):002:0> t
          => #<Thread:0x2eb86778 dead>
          irb(main):003:0> t.value
          Errno::ECHILD: No child processes - No child processes
          

          I'm probably missing something obvious. I welcome suggestions and improvements.

          Show
          Hiro Asari added a comment - So, there is a way to build a RubyThread from a Java Process ( RubyProcess.detach() ). A naive approach goes something like this: POpenTuple tuple = popenSpecial(context, args); long pid = ShellLauncher.getPidFromProcess(tuple.process); IRubyObject thr = RubyProcess.detach(context, runtime.getProcess(), runtime.newFixnum(pid)); but this fails rather miserably. $ jruby -S irb irb(main):001:0> require 'open3'; i,o,e,t = Open3.popen3("ls") => [#<IO:fd 140>, #<IO:fd 139>, #<IO:fd 141>, #<Thread:0x2eb86778 run>] irb(main):002:0> t => #<Thread:0x2eb86778 dead> irb(main):003:0> t.value Errno::ECHILD: No child processes - No child processes I'm probably missing something obvious. I welcome suggestions and improvements.
          Hide
          Charles Oliver Nutter added a comment -

          Fix in 1.7.1 along with other process-launching APIs.

          Show
          Charles Oliver Nutter added a comment - Fix in 1.7.1 along with other process-launching APIs.
          Charles Oliver Nutter made changes -
          Fix Version/s JRuby 1.7.1 [ 18842 ]
          Hiro Asari made changes -
          Link This issue is duplicated by JRUBY-6966 [ JRUBY-6966 ]
          Thomas E Enebo made changes -
          Fix Version/s JRuby 1.7.2 [ 19008 ]
          Fix Version/s JRuby 1.7.1 [ 18842 ]
          Thomas E Enebo made changes -
          Fix Version/s JRuby 1.7.3 [ 19009 ]
          Fix Version/s JRuby 1.7.2 [ 19008 ]
          Hide
          Steven Arnold added a comment -

          I tried this same thing in jruby-1.7.2 and in jruby-head (1.7.3 pre), but the problem still seems to exist.

          [ asymptotic ] > irb
          jruby-1.7.3.dev:001 > require 'open3'
          => true
          jruby-1.7.3.dev:002 > Open3::popen3('ls')
          => #<IO:fd 9>, #<IO:fd 8>, #<IO:fd 10>

          Note that only three objects are returned, not the expected four. The thread object is missing. If this was fixed in 1.7.1, it seems maybe the problem recurred in 1.7.2.

          Show
          Steven Arnold added a comment - I tried this same thing in jruby-1.7.2 and in jruby-head (1.7.3 pre), but the problem still seems to exist. [ asymptotic ] > irb jruby-1.7.3.dev:001 > require 'open3' => true jruby-1.7.3.dev:002 > Open3::popen3('ls') => #<IO:fd 9>, #<IO:fd 8>, #<IO:fd 10> Note that only three objects are returned, not the expected four. The thread object is missing. If this was fixed in 1.7.1, it seems maybe the problem recurred in 1.7.2.
          Thomas E Enebo made changes -
          Fix Version/s JRuby 1.7.4 [ 19106 ]
          Fix Version/s JRuby 1.7.3 [ 19009 ]
          Charles Oliver Nutter made changes -
          Fix Version/s JRuby 1.7.4 [ 19106 ]
          Hide
          Charles Oliver Nutter added a comment -

          Poking at this today. Need to understand what the thread is for. I presume it's similar to the JVM, waiting on process exit.

          Show
          Charles Oliver Nutter added a comment - Poking at this today. Need to understand what the thread is for. I presume it's similar to the JVM, waiting on process exit.
          Hide
          Charles Oliver Nutter added a comment -

          Yep, that's basically what it's doing. Oddly enough, the open3 logic fits better with JVM process management than most other stuff. I think I can come up with something that will work ok for now.

          Show
          Charles Oliver Nutter added a comment - Yep, that's basically what it's doing. Oddly enough, the open3 logic fits better with JVM process management than most other stuff. I think I can come up with something that will work ok for now.
          Hide
          Charles Oliver Nutter added a comment -

          Squeaky wheel gets the grease.

          commit 7dba21754899867f048559d3712442996ec07a77
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Tue Jun 25 11:48:15 2013 -0500
          
              Implement thread retval for 1.9's Open3.popen3. JRUBY-6409
              
              It turns out that the thread value returned from 1.9's popen3 is
              actually easy to support by simply calling java.lang.Process's
              waitFor method. This is, in fact, much more reliable than a normal
              popen + waitpid due to our racing the JVM to the waitpid call.
              
              There's still issues with the interactivity of the streams (on
              some JVMs/versions) and overall JRuby process management, but this
              patch should allow popen3 to work properly.
              
              Unexcluded some popen3 tests that pass now, too (yay!)
          
          :100644 100644 b7d911f... c094a39... M	src/jruby/kernel19/process.rb
          :100644 100644 194c21d... 57926b9... M	src/org/jruby/RubyIO.java
          :100644 100644 3c3884a... 85e8b5f... M	src/org/jruby/RubyThread.java
          :100644 100644 2f5da5f... b025c4b... M	src/org/jruby/internal/runtime/FutureThread.java
          :100644 100644 22344ce... 2007da5... M	src/org/jruby/internal/runtime/RubyRunnable.java
          :000000 100644 0000000... 1f6f8f1... A	src/org/jruby/internal/runtime/ThreadedRunnable.java
          :100644 100644 2796c03... 7c5b002... M	test/externals/ruby1.9/excludes/TestOpen3.rb
          
          Show
          Charles Oliver Nutter added a comment - Squeaky wheel gets the grease. commit 7dba21754899867f048559d3712442996ec07a77 Author: Charles Oliver Nutter <headius@headius.com> Date: Tue Jun 25 11:48:15 2013 -0500 Implement thread retval for 1.9's Open3.popen3. JRUBY-6409 It turns out that the thread value returned from 1.9's popen3 is actually easy to support by simply calling java.lang.Process's waitFor method. This is, in fact, much more reliable than a normal popen + waitpid due to our racing the JVM to the waitpid call. There's still issues with the interactivity of the streams (on some JVMs/versions) and overall JRuby process management, but this patch should allow popen3 to work properly. Unexcluded some popen3 tests that pass now, too (yay!) :100644 100644 b7d911f... c094a39... M src/jruby/kernel19/process.rb :100644 100644 194c21d... 57926b9... M src/org/jruby/RubyIO.java :100644 100644 3c3884a... 85e8b5f... M src/org/jruby/RubyThread.java :100644 100644 2f5da5f... b025c4b... M src/org/jruby/internal/runtime/FutureThread.java :100644 100644 22344ce... 2007da5... M src/org/jruby/internal/runtime/RubyRunnable.java :000000 100644 0000000... 1f6f8f1... A src/org/jruby/internal/runtime/ThreadedRunnable.java :100644 100644 2796c03... 7c5b002... M test/externals/ruby1.9/excludes/TestOpen3.rb
          Charles Oliver Nutter made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Thomas E Enebo [ enebo ] Charles Oliver Nutter [ headius ]
          Fix Version/s JRuby 1.7.5 [ 19233 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Charles Oliver Nutter
              Reporter:
              Robert Gleeson
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: