Details

    • Type: Bug Bug
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: JRuby 1.1.1
    • Fix Version/s: None
    • Component/s: Core Classes/Modules
    • Labels:
      None
    • Environment:
      MAC OS X Tiger. Java 1.5.0_07. JRuby says: "ruby 1.8.5 (2007-11-22 rev 4842) [ppc-jruby1.1b1]".
    • Number of attachments :
      0

      Description

      A program started with Kernel.system does not get the same STDIN/STDOUT as JRuby had when it started. This can be shown with the following example:

          $ jruby -e 'system "tty"'
          not a tty                       <--- Note this output
          $ ruby -e 'system "tty"'
          /dev/ttyp1
      

      MRI passes on its original STDOUT/STDERR to the tty(1) program. JRuby does not.

      To further demonstrate this problem a simple Ruby script "is-a-tty" like below can be used:

      ---------- is-a-tty ---------------
          #!/usr/local/bin/ruby
      
          def report_tty(name, f)
              puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"]
          end
      
          report_tty "STDIN",  STDIN
          report_tty "STDOUT", STDOUT
          report_tty "STDERR", STDERR
      -------------------------
      

      If I run this with JRuby and MRI I get:

          $ jruby -e 'system "./is-a-tty"'
          STDIN  is NOT a tty
          STDOUT is NOT a tty
          STDERR is NOT a tty
          $ ruby -e 'system "./is-a-tty"'
          STDIN  is a tty
          STDOUT is a tty
          STDERR is a tty
      

        Issue Links

          Activity

          Hide
          Thomas E Enebo added a comment -

          Provided is-a-tty now works as of commit 5668. Main reported issue (calling tty from system) is not fixed however.

          Show
          Thomas E Enebo added a comment - Provided is-a-tty now works as of commit 5668. Main reported issue (calling tty from system) is not fixed however.
          Hide
          Johan Holmberg added a comment -

          I don't understand what the comment "Provided is-a-tty now works ..." means.

          I tried the command "jruby -e 'system "./is-a-tty"'" again, and for me it still gives the same output as in this bug report. I used Subversion revision r5779.

          I wrote "is-a-tty" to show the situation for STDOUT and STDERR (the command tty(1) only reports for STDIN). The program could have been written in for example Perl or C instead. It just happened to be Ruby (note: MRI, not JRuby).

          I believe the program test a real property of the file descriptor in the started process. And the problem is that JRuby wraps the process in pipes that are handled by Java. See the StreamPumper-code in ShellLauncher.java.

          As long as "system" is implemented in this way, I would think that it is impossible to make the problem in this bug-report go away (but I hope I'm wrong

          Show
          Johan Holmberg added a comment - I don't understand what the comment "Provided is-a-tty now works ..." means. I tried the command "jruby -e 'system "./is-a-tty"'" again, and for me it still gives the same output as in this bug report. I used Subversion revision r5779. I wrote "is-a-tty" to show the situation for STDOUT and STDERR (the command tty(1) only reports for STDIN). The program could have been written in for example Perl or C instead. It just happened to be Ruby (note: MRI, not JRuby). I believe the program test a real property of the file descriptor in the started process . And the problem is that JRuby wraps the process in pipes that are handled by Java. See the StreamPumper-code in ShellLauncher.java. As long as "system" is implemented in this way, I would think that it is impossible to make the problem in this bug-report go away (but I hope I'm wrong
          Hide
          Thomas E Enebo added a comment -

          Sorry, I just meant running is_a_tty by itself works and it did not before that commit (sans invoking it via system). Using system to call is_a_tty still does not work (and thats why I left the bug open).

          Show
          Thomas E Enebo added a comment - Sorry, I just meant running is_a_tty by itself works and it did not before that commit (sans invoking it via system). Using system to call is_a_tty still does not work (and thats why I left the bug open).
          Hide
          Charles Oliver Nutter added a comment -

          FYI, this works in MRI because it's handled by the C APIs. Until we migrate to using the same APIs, there's no way we can support this. Still an issue under 1.1.1.

          Show
          Charles Oliver Nutter added a comment - FYI, this works in MRI because it's handled by the C APIs. Until we migrate to using the same APIs, there's no way we can support this. Still an issue under 1.1.1.
          Hide
          Riley Lynch added a comment -

          See also JRUBY-2987.

          Show
          Riley Lynch added a comment - See also JRUBY-2987 .
          Hide
          Hiro Asari added a comment -

          It seems to be fixed in trunk.

          surfboard:jruby asari$ cat ../../sandbox/jruby-1608.rb 
          #!/usr/local/bin/ruby
          
          def report_tty(name, f)
          puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"]
          end
          
          report_tty "STDIN", STDIN
          report_tty "STDOUT", STDOUT
          report_tty "STDERR", STDERR
          surfboard:jruby asari$ jruby -v ../../sandbox/jruby-1608.rb 
          jruby 1.4.0dev (ruby 1.8.6p287) (2009-07-29 6586) (Java HotSpot(TM) Client VM 1.5.0_19) [i386-java]
          STDIN  is a tty
          STDOUT is a tty
          STDERR is a tty
          
          Show
          Hiro Asari added a comment - It seems to be fixed in trunk. surfboard:jruby asari$ cat ../../sandbox/jruby-1608.rb #!/usr/local/bin/ruby def report_tty(name, f) puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"] end report_tty "STDIN", STDIN report_tty "STDOUT", STDOUT report_tty "STDERR", STDERR surfboard:jruby asari$ jruby -v ../../sandbox/jruby-1608.rb jruby 1.4.0dev (ruby 1.8.6p287) (2009-07-29 6586) (Java HotSpot(TM) Client VM 1.5.0_19) [i386-java] STDIN is a tty STDOUT is a tty STDERR is a tty
          Hide
          Johan Holmberg added a comment -

          Sorry Hiro, but you tested something else, not the problem this issue tries to describe. If you read the previous comments on the issue carefully you will see that it is about a process started via "system" in JRuby. It is there that "STDOUT is NOT a tty" occurs.

          I believe Charles Nutters comment from 12/May/08 is correct. I had hoped that using what Charles called the "C API" would be a real possibility. But I don't know enough Java to understand if there are strong arguments against such a solution.

          Show
          Johan Holmberg added a comment - Sorry Hiro, but you tested something else, not the problem this issue tries to describe. If you read the previous comments on the issue carefully you will see that it is about a process started via "system" in JRuby. It is there that "STDOUT is NOT a tty" occurs. I believe Charles Nutters comment from 12/May/08 is correct. I had hoped that using what Charles called the "C API" would be a real possibility. But I don't know enough Java to understand if there are strong arguments against such a solution.
          Hide
          Hiro Asari added a comment -

          Oh, right, Johan, I see my error.

          But I still have this:

          $ cat jruby-1608.rb; jruby -ve 'system "jruby -v ./jruby-1608.rb"'
          #!/usr/local/bin/ruby
          
          def report_tty(name, f)
          puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"]
          end
          
          report_tty "STDIN", STDIN
          report_tty "STDOUT", STDOUT
          report_tty "STDERR", STDERR
          jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java]
          jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java]
          STDIN  is a tty
          STDOUT is a tty
          STDERR is a tty
          
          Show
          Hiro Asari added a comment - Oh, right, Johan, I see my error. But I still have this: $ cat jruby-1608.rb; jruby -ve 'system "jruby -v ./jruby-1608.rb"' #!/usr/local/bin/ruby def report_tty(name, f) puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"] end report_tty "STDIN", STDIN report_tty "STDOUT", STDOUT report_tty "STDERR", STDERR jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java] jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java] STDIN is a tty STDOUT is a tty STDERR is a tty
          Hide
          Johan Holmberg added a comment -

          I think that is because JRuby is cheating, and runs the second "ruby instance" in the same JAVA virtual machine.

          Using your example I get:

          $ jruby -ve 'system "jruby -v ./jruby-1608.rb"'         # JRuby is cheating
          $ jruby -ve 'system "./jruby-1608.rb"'                  # JRuby is still cheating
          $
          $ mv jruby-1608.rb jruby-1608                           # remove file extension
          $ jruby -ve 'system "./jruby-1608"'                     # JRuby finally runs a real new process
          

          I really regret that I didn't write my "is-a-tty" example in Perl or C instead. Then we could have focused on the real issue: the way "system" in JRuby starts processes that are not Ruby scripts (or at least JRuby can't know is a Ruby script).

          The "smartness" of JRuby is interesting and probably useful. But it is not the thing at stake here.

          Show
          Johan Holmberg added a comment - I think that is because JRuby is cheating, and runs the second "ruby instance" in the same JAVA virtual machine. Using your example I get: $ jruby -ve 'system "jruby -v ./jruby-1608.rb"' # JRuby is cheating $ jruby -ve 'system "./jruby-1608.rb"' # JRuby is still cheating $ $ mv jruby-1608.rb jruby-1608 # remove file extension $ jruby -ve 'system "./jruby-1608"' # JRuby finally runs a real new process I really regret that I didn't write my "is-a-tty" example in Perl or C instead. Then we could have focused on the real issue: the way "system" in JRuby starts processes that are not Ruby scripts (or at least JRuby can't know is a Ruby script). The "smartness" of JRuby is interesting and probably useful. But it is not the thing at stake here.
          Hide
          Charles Oliver Nutter added a comment -

          FYI, the only way to solve this would be to fork+exec for subprocesses in the same way that MRI does and pass along the actual tty stdin, etc. OpenJDK, for example, does fork+exec internally, but does not pass the terminal streams along to the subprocess. Until we can do that, subprocesses will never see stdin as a tty.

          Show
          Charles Oliver Nutter added a comment - FYI, the only way to solve this would be to fork+exec for subprocesses in the same way that MRI does and pass along the actual tty stdin, etc. OpenJDK, for example, does fork+exec internally, but does not pass the terminal streams along to the subprocess. Until we can do that, subprocesses will never see stdin as a tty.
          Hide
          Charles Oliver Nutter added a comment -

          This has been fixed in JRuby 1.7 for backquote by using Java 7 process-launching APIs that allow sharing the tty. We plan to expand this to the other process-launching functions. Marking for 1.7.1.

          Show
          Charles Oliver Nutter added a comment - This has been fixed in JRuby 1.7 for backquote by using Java 7 process-launching APIs that allow sharing the tty. We plan to expand this to the other process-launching functions. Marking for 1.7.1.
          Hide
          Jarl Friis added a comment -

          @Charles Oliver Nutter: What commit id has this been fixed in?

          Show
          Jarl Friis added a comment - @Charles Oliver Nutter: What commit id has this been fixed in?
          Hide
          Jarl Friis added a comment -

          We have also ran into this problem over at the childprocess gem even on JRuby-1.7.0-RC2.

          And I consider it a more general problem in java.lang.ProcessBuilder#new because the process returns a BufferedOutputStream when invoking Process#getOutputStream() (But maybe that is out of JRuby scope, I don't know )

          You can see the monkey patch that we have made to work around this problem.

          As you can see in the monkey patch, actually it is possible to use STDIN interactively, you just have to override some methods in eigenclass of the IO object returned from java::io::OutputStream#to_io and ensure that the underlying java stream is flushed because it is a BufferedOutputStream.

          IMHO I consider it wrong that Process#getOutputStream() returns a BufferedOutputStream. I suggest it returns a FileOutputStream so nothing is buffered.

          Show
          Jarl Friis added a comment - We have also ran into this problem over at the childprocess gem even on JRuby-1.7.0-RC2. And I consider it a more general problem in java.lang.ProcessBuilder#new because the process returns a BufferedOutputStream when invoking Process#getOutputStream() (But maybe that is out of JRuby scope, I don't know ) You can see the monkey patch that we have made to work around this problem. As you can see in the monkey patch , actually it is possible to use STDIN interactively, you just have to override some methods in eigenclass of the IO object returned from java::io::OutputStream#to_io and ensure that the underlying java stream is flushed because it is a BufferedOutputStream . IMHO I consider it wrong that Process#getOutputStream() returns a BufferedOutputStream . I suggest it returns a FileOutputStream so nothing is buffered.
          Hide
          Charles Oliver Nutter added a comment -

          Jarl: The backquote fix is in the following commit. We have yet to do similar work for the other process-launching methods, but they could all get the same treatment:

          commit c7c7c7d12f43f56e5e004f8535e42118c04969f0
          Author: Charles Oliver Nutter <headius@headius.com>
          Date:   Wed Jun 13 19:38:13 2012 -0500
          
              Add reimpl of Kernel#` using Java 7 process APIs.
          
          Show
          Charles Oliver Nutter added a comment - Jarl: The backquote fix is in the following commit. We have yet to do similar work for the other process-launching methods, but they could all get the same treatment: commit c7c7c7d12f43f56e5e004f8535e42118c04969f0 Author: Charles Oliver Nutter <headius@headius.com> Date: Wed Jun 13 19:38:13 2012 -0500 Add reimpl of Kernel#` using Java 7 process APIs.
          Hide
          Jarl Friis added a comment -

          Charles: Thanks for the commit info.

          I would have expected that the backquote situation would be a special case of the more general process-launching methods. However the issue that bothers us is the general case. Is there another bugreport that I should watch to follow the status of the more general case?

          Show
          Jarl Friis added a comment - Charles: Thanks for the commit info. I would have expected that the backquote situation would be a special case of the more general process-launching methods. However the issue that bothers us is the general case. Is there another bugreport that I should watch to follow the status of the more general case?
          Hide
          Charles Oliver Nutter added a comment -

          We want to implement the same behavior for all process-launching APIs, but have not had resources to do so. Post 1.7.0 we'll try to tidy this up; there are a number of process-related bugs assigned to 1.7.1 already.

          Show
          Charles Oliver Nutter added a comment - We want to implement the same behavior for all process-launching APIs, but have not had resources to do so. Post 1.7.0 we'll try to tidy this up; there are a number of process-related bugs assigned to 1.7.1 already.
          Hide
          Jarl Friis added a comment -

          The issue I have run into is best described by viewing the workaround that is need to make the STDIN flow into the childprocess. The workaround is the highlighted lines at
          https://github.com/jarib/childprocess/blob/a650ecd070dbc2ba8e6eaabbd91e9d3e034bc704/lib/childprocess/jruby/process.rb#L121-136

          As stated in the comment there: "The stream provided by #to_io is a BufferedeOutputStream, so we have to flush it to make the bytes flow to the child process"

          So is there a "better" bug report already (for 1.7.1) to keep an eye on for that more general process-launching API issue that I have run into?

          Show
          Jarl Friis added a comment - The issue I have run into is best described by viewing the workaround that is need to make the STDIN flow into the childprocess. The workaround is the highlighted lines at https://github.com/jarib/childprocess/blob/a650ecd070dbc2ba8e6eaabbd91e9d3e034bc704/lib/childprocess/jruby/process.rb#L121-136 As stated in the comment there: "The stream provided by #to_io is a BufferedeOutputStream , so we have to flush it to make the bytes flow to the child process" So is there a "better" bug report already (for 1.7.1) to keep an eye on for that more general process-launching API issue that I have run into?
          Hide
          Charles Oliver Nutter added a comment -

          There's not really any one "better" bug. There's several relating to subprocess launching. Ultimately I believe all of system, `, exec, popen, and spawn will end up having Ruby versions like ` does that take advantage of the better process APIs in Java 7.

          If you'd like to start helping to implement that, we'd really appreciate it. The Java 7 code for ` is in src/jruby/kernel/jruby/process_manager.rb. The logic for system, exec, popen, and spawn will be similar, with the addition of processing various argument structures (redirects, etc).

          Show
          Charles Oliver Nutter added a comment - There's not really any one "better" bug. There's several relating to subprocess launching. Ultimately I believe all of system, `, exec, popen, and spawn will end up having Ruby versions like ` does that take advantage of the better process APIs in Java 7. If you'd like to start helping to implement that, we'd really appreciate it. The Java 7 code for ` is in src/jruby/kernel/jruby/process_manager.rb. The logic for system, exec, popen, and spawn will be similar, with the addition of processing various argument structures (redirects, etc).
          Hide
          Jarl Friis added a comment -

          Do you have any start-hacking-tutorial guide or something like that? I have checked out the source from github (so far so good), but what test suite are supposed to pass and under which conditions, and so on... Any short description of directory structure and/or architecture ?

          Show
          Jarl Friis added a comment - Do you have any start-hacking-tutorial guide or something like that? I have checked out the source from github (so far so good), but what test suite are supposed to pass and under which conditions, and so on... Any short description of directory structure and/or architecture ?
          Hide
          Charles Oliver Nutter added a comment -

          I do not, but it is on my list of things to do. The basics:

          • The Ruby-based part of JRuby is in src/jruby, like the process logic I mentioned.
          • The two suites we usually run are "ant test" and "rake test19". The latter is more important for this.
          • Build JRuby with "ant jar", or "ant clean jar" to completely rebuild.

          Hopefully over the weekend I can hack out some articles on how to contribute to JRuby.

          Show
          Charles Oliver Nutter added a comment - I do not, but it is on my list of things to do. The basics: The Ruby-based part of JRuby is in src/jruby, like the process logic I mentioned. The two suites we usually run are "ant test" and "rake test19". The latter is more important for this. Build JRuby with "ant jar", or "ant clean jar" to completely rebuild. Hopefully over the weekend I can hack out some articles on how to contribute to JRuby.

            People

            • Assignee:
              Unassigned
              Reporter:
              Johan Holmberg
            • Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: