STDIN/STDOUT "not a tty" under Kernel.system

Provided is-a-tty now works as of commit 5668. Main reported issue (calling tty from system) is not fixed however.

I don't understand what the comment "Provided is-a-tty now works ..." means.

I tried the command "jruby -e 'system "./is-a-tty"'" again, and for me it still gives the same output as in this bug report. I used Subversion revision r5779.

I wrote "is-a-tty" to show the situation for STDOUT and STDERR (the command tty(1) only reports for STDIN). The program could have been written in for example Perl or C instead. It just happened to be Ruby (note: MRI, not JRuby).

I believe the program test a real property of the file descriptor in the started process. And the problem is that JRuby wraps the process in pipes that are handled by Java. See the StreamPumper-code in ShellLauncher.java.

As long as "system" is implemented in this way, I would think that it is impossible to make the problem in this bug-report go away (but I hope I'm wrong

Sorry, I just meant running is_a_tty by itself works and it did not before that commit (sans invoking it via system). Using system to call is_a_tty still does not work (and thats why I left the bug open).

FYI, this works in MRI because it's handled by the C APIs. Until we migrate to using the same APIs, there's no way we can support this. Still an issue under 1.1.1.

See also JRUBY-2987.

It seems to be fixed in trunk.

surfboard:jruby asari$ cat ../../sandbox/jruby-1608.rb 
#!/usr/local/bin/ruby

def report_tty(name, f)
puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"]
end

report_tty "STDIN", STDIN
report_tty "STDOUT", STDOUT
report_tty "STDERR", STDERR
surfboard:jruby asari$ jruby -v ../../sandbox/jruby-1608.rb 
jruby 1.4.0dev (ruby 1.8.6p287) (2009-07-29 6586) (Java HotSpot(TM) Client VM 1.5.0_19) [i386-java]
STDIN  is a tty
STDOUT is a tty
STDERR is a tty

Sorry Hiro, but you tested something else, not the problem this issue tries to describe. If you read the previous comments on the issue carefully you will see that it is about a process started via "system" in JRuby. It is there that "STDOUT is NOT a tty" occurs.

I believe Charles Nutters comment from 12/May/08 is correct. I had hoped that using what Charles called the "C API" would be a real possibility. But I don't know enough Java to understand if there are strong arguments against such a solution.

Oh, right, Johan, I see my error.

But I still have this:

$ cat jruby-1608.rb; jruby -ve 'system "jruby -v ./jruby-1608.rb"'
#!/usr/local/bin/ruby

def report_tty(name, f)
puts "%-6s is%s a tty" % [name, f.tty? ? "" : " NOT"]
end

report_tty "STDIN", STDIN
report_tty "STDOUT", STDOUT
report_tty "STDERR", STDERR
jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java]
jruby 1.4.0dev (ruby 1.8.7p174) (2009-08-11 6586) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_13) [x86_64-java]
STDIN  is a tty
STDOUT is a tty
STDERR is a tty

FYI, the only way to solve this would be to fork+exec for subprocesses in the same way that MRI does and pass along the actual tty stdin, etc. OpenJDK, for example, does fork+exec internally, but does not pass the terminal streams along to the subprocess. Until we can do that, subprocesses will never see stdin as a tty.

This has been fixed in JRuby 1.7 for backquote by using Java 7 process-launching APIs that allow sharing the tty. We plan to expand this to the other process-launching functions. Marking for 1.7.1.

@Charles Oliver Nutter: What commit id has this been fixed in?

We have also ran into this problem over at the childprocess gem even on JRuby-1.7.0-RC2.

And I consider it a more general problem in java.lang.ProcessBuilder#new because the process returns a BufferedOutputStream when invoking Process#getOutputStream() (But maybe that is out of JRuby scope, I don't know )

You can see the monkey patch that we have made to work around this problem.

As you can see in the monkey patch, actually it is possible to use STDIN interactively, you just have to override some methods in eigenclass of the IO object returned from java::io::OutputStream#to_io and ensure that the underlying java stream is flushed because it is a BufferedOutputStream.

IMHO I consider it wrong that Process#getOutputStream() returns a BufferedOutputStream. I suggest it returns a FileOutputStream so nothing is buffered.

Jarl: The backquote fix is in the following commit. We have yet to do similar work for the other process-launching methods, but they could all get the same treatment:

commit c7c7c7d12f43f56e5e004f8535e42118c04969f0
Author: Charles Oliver Nutter <headius@headius.com>
Date:   Wed Jun 13 19:38:13 2012 -0500

    Add reimpl of Kernel#` using Java 7 process APIs.

Charles: Thanks for the commit info.

I would have expected that the backquote situation would be a special case of the more general process-launching methods. However the issue that bothers us is the general case. Is there another bugreport that I should watch to follow the status of the more general case?

We want to implement the same behavior for all process-launching APIs, but have not had resources to do so. Post 1.7.0 we'll try to tidy this up; there are a number of process-related bugs assigned to 1.7.1 already.

The issue I have run into is best described by viewing the workaround that is need to make the STDIN flow into the childprocess. The workaround is the highlighted lines at
https://github.com/jarib/childprocess/blob/a650ecd070dbc2ba8e6eaabbd91e9d3e034bc704/lib/childprocess/jruby/process.rb#L121-136

As stated in the comment there: "The stream provided by #to_io is a BufferedeOutputStream, so we have to flush it to make the bytes flow to the child process"

So is there a "better" bug report already (for 1.7.1) to keep an eye on for that more general process-launching API issue that I have run into?

There's not really any one "better" bug. There's several relating to subprocess launching. Ultimately I believe all of system, `, exec, popen, and spawn will end up having Ruby versions like ` does that take advantage of the better process APIs in Java 7.

If you'd like to start helping to implement that, we'd really appreciate it. The Java 7 code for ` is in src/jruby/kernel/jruby/process_manager.rb. The logic for system, exec, popen, and spawn will be similar, with the addition of processing various argument structures (redirects, etc).

Do you have any start-hacking-tutorial guide or something like that? I have checked out the source from github (so far so good), but what test suite are supposed to pass and under which conditions, and so on... Any short description of directory structure and/or architecture ?

I do not, but it is on my list of things to do. The basics:

• The Ruby-based part of JRuby is in src/jruby, like the process logic I mentioned.
• The two suites we usually run are "ant test" and "rake test19". The latter is more important for this.
• Build JRuby with "ant jar", or "ant clean jar" to completely rebuild.

Hopefully over the weekend I can hack out some articles on how to contribute to JRuby.