JRuby (please use github issues at http://bugs.jruby.org)
  1. JRuby (please use github issues at http://bugs.jruby.org)
  2. JRUBY-6162

IO.popen doesn't wait until a process ends but returns immediately

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: JRuby 1.6.4, JRuby 1.6.5
    • Fix Version/s: None
    • Component/s: Miscellaneous
    • Labels:
      None
    • Environment:
      I reproduced this on Ubuntu 11.10 and on an older CentOS 5.6 with JRuby 1.6.3 and JRuby 1.6.5
    • Number of attachments :
      0

      Description

      Hi,

      I hope that I'm wrong, but at the moment this looks like a serious bug to me, but maybe I just didn't do enough RTFM. I found this bug when using ActionMailer with sendmail. The "/usr/sbin/sendmail" process was only sometimes called and sometimes it wasn't (most often it wasn't but around every tenth time it suddenly worked)

      I investigated it deeply and after hours I distilled it down to the following:

      hk@hk:~$ rvm use jruby ruby -e "cmd='/bin/bash -c \"sleep 5\"'; puts Time.now ; system(cmd); puts Time.now; IO.popen(cmd) {} ; puts Time.now "
      Wed Oct 26 16:37:08 +0200 2011
      Wed Oct 26 16:37:13 +0200 2011
      Wed Oct 26 16:37:13 +0200 2011
      hk@hk:~$ rvm use 1.8.7 ruby -e "cmd='/bin/bash -c \"sleep 5\"'; puts Time.now ; system(cmd); puts Time.now; IO.popen(cmd) {} ; puts Time.now "
      Wed Oct 26 16:37:18 +0200 2011
      Wed Oct 26 16:37:23 +0200 2011
      Wed Oct 26 16:37:28 +0200 2011
      hk@hk:~$ 
      

      As one can see, the system call on both Ruby Interpreters waits until the subprocess exits. But with MRI it also stops when using IO.popen, but not with JRuby. Here it returns immediately.

      I found a very ugly workaround for the moment by doing something like the following:

      module ActionMailer
        class Base
          def perform_delivery_sendmail(mail)
            sendmail_args = sendmail_settings[:arguments]
            sendmail_args += " -f \"#{mail['return-path']}\"" if mail['return-path']
      
            IO.popen("#{sendmail_settings[:location]} #{sendmail_args}", "r+") do |f| #r+ geht, w+ geht nicht richtig, bzw. nur manchmal
              f.puts mail.encoded.gsub(/\r/, '')
              f.close_write
              sleep 1 # <---- added this line in order to give sendmail some time to process
            end
          end
        end
      end
      

      Here are some environment infos:

      hk@hk:~$ rvm info jruby
      
      jruby-1.6.5:
      
        system:
          uname:       "Linux hk 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux"
          bash:        "/bin/bash => GNU bash, Version 4.2.10(1)-release (x86_64-pc-linux-gnu)"
          zsh:         " => not installed"
      
        rvm:
          version:      "rvm 1.9.0 by Wayne E. Seguin (wayneeseguin@gmail.com) [https://rvm.beginrescueend.com/]"
      
        ruby:
          interpreter:  "jruby"
          version:      "1.6.5"
          date:         "2011-10-25"
          platform:     "linux-amd64-java"
          patchlevel:   "OpenJDK 64-Bit Server VM 1.6.0_23"
          full_version: "jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (OpenJDK 64-Bit Server VM 1.6.0_23) [linux-amd64-java]"
      
        homes:
          gem:          "/home/hk/.rvm/gems/jruby-1.6.5"
          ruby:         "/home/hk/.rvm/rubies/jruby-1.6.5"
      
        binaries:
          ruby:         "/home/hk/.rvm/rubies/jruby-1.6.5/bin/ruby"
          irb:          "/home/hk/.rvm/rubies/jruby-1.6.5/bin/irb"
          gem:          "/home/hk/.rvm/rubies/jruby-1.6.5/bin/gem"
          rake:         "/home/hk/.rvm/gems/jruby-1.6.5/bin/rake"
      
        environment:
          PATH:         "/home/hk/.rvm/gems/jruby-1.6.5/bin:/home/hk/.rvm/gems/jruby-1.6.5@global/bin:/home/hk/.rvm/rubies/jruby-1.6.5/bin:/home/hk/.rvm/bin:<other stuff>"
          GEM_HOME:     "/home/hk/.rvm/gems/jruby-1.6.5"
          GEM_PATH:     "/home/hk/.rvm/gems/jruby-1.6.5:/home/hk/.rvm/gems/jruby-1.6.5@global"
          MY_RUBY_HOME: "/home/hk/.rvm/rubies/jruby-1.6.5"
          IRBRC:        "/home/hk/.rvm/rubies/jruby-1.6.5/.irbrc"
          RUBYOPT:      ""
          gemset:       ""
      
      
      hk@hk:~$ 
      
      hk@hk:~$ rvm info 1.8.7
      
      ruby-1.8.7-p352:
      
        system:
          uname:       "Linux hk 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux"
          bash:        "/bin/bash => GNU bash, Version 4.2.10(1)-release (x86_64-pc-linux-gnu)"
          zsh:         " => not installed"
      
        rvm:
          version:      "rvm 1.9.0 by Wayne E. Seguin (wayneeseguin@gmail.com) [https://rvm.beginrescueend.com/]"
      
        ruby:
          interpreter:  "ruby"
          version:      "1.8.7"
          date:         "2011-06-30"
          platform:     "x86_64-linux"
          patchlevel:   "2011-06-30 patchlevel 352"
          full_version: "ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]"
      
        homes:
          gem:          "/home/hk/.rvm/gems/ruby-1.8.7-p352"
          ruby:         "/home/hk/.rvm/rubies/ruby-1.8.7-p352"
      
        binaries:
          ruby:         "/home/hk/.rvm/rubies/ruby-1.8.7-p352/bin/ruby"
          irb:          "/home/hk/.rvm/rubies/ruby-1.8.7-p352/bin/irb"
          gem:          "/home/hk/.rvm/rubies/ruby-1.8.7-p352/bin/gem"
          rake:         "/home/hk/.rvm/gems/ruby-1.8.7-p352/bin/rake"
      
        environment:
          PATH:         "/home/hk/.rvm/gems/ruby-1.8.7-p352/bin:/home/hk/.rvm/gems/ruby-1.8.7-p352@global/bin:/home/hk/.rvm/rubies/ruby-1.8.7-p352/bin:<other stuff>"
          GEM_HOME:     "/home/hk/.rvm/gems/ruby-1.8.7-p352"
          GEM_PATH:     "/home/hk/.rvm/gems/ruby-1.8.7-p352:/home/hk/.rvm/gems/ruby-1.8.7-p352@global"
          MY_RUBY_HOME: "/home/hk/.rvm/rubies/ruby-1.8.7-p352"
          IRBRC:        "/home/hk/.rvm/rubies/ruby-1.8.7-p352/.irbrc"
          RUBYOPT:      ""
          gemset:       ""
      
      
      hk@hk:~$ 
      

        Activity

        Hide
        Heiko Seebach added a comment -

        Congrats, Charles
        Thank you very much,
        Heiko

        Show
        Heiko Seebach added a comment - Congrats, Charles Thank you very much, Heiko
        Hide
        Roger Pack added a comment -


        seems to work here at least for trivial cases, thanks much! If waiting on a process is possible in windows now, maybe this bug could also be addressed at some point in the future? http://jira.codehaus.org/browse/JRUBY-4354 Thanks! -r

        Show
        Roger Pack added a comment - seems to work here at least for trivial cases, thanks much! If waiting on a process is possible in windows now, maybe this bug could also be addressed at some point in the future? http://jira.codehaus.org/browse/JRUBY-4354 Thanks! -r
        Hide
        Charles Oliver Nutter added a comment -

        Going to have to reopen this; fixing it to not try to shut down the child process causes some specs to hang...which could lead to hangs for users running RC1. We need to dig a bit deeper and figure out the right magic for popen subprocesses before 1.7 final.

        Reverted 2d497af in 73ce313.

        Show
        Charles Oliver Nutter added a comment - Going to have to reopen this; fixing it to not try to shut down the child process causes some specs to hang...which could lead to hangs for users running RC1. We need to dig a bit deeper and figure out the right magic for popen subprocesses before 1.7 final. Reverted 2d497af in 73ce313.
        Hide
        Charles Oliver Nutter added a comment -

        This issue is still outstanding, and likely can't be fixed until we move away from using Java's process management APIs. We need to be able to do our own waitpid rather than racing the JVM for it, and allow the process and its streams to terminate in the same way as MRI.

        I'm going to move this to github.

        Show
        Charles Oliver Nutter added a comment - This issue is still outstanding, and likely can't be fixed until we move away from using Java's process management APIs. We need to be able to do our own waitpid rather than racing the JVM for it, and allow the process and its streams to terminate in the same way as MRI. I'm going to move this to github.
        Show
        Charles Oliver Nutter added a comment - Moved to https://github.com/jruby/jruby/issues/911 .

          People

          • Assignee:
            Charles Oliver Nutter
            Reporter:
            Heiko Seebach
          • Votes:
            5 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: