JRuby

IO.sysopen not defined

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.0.0RC2
  • Fix Version/s: JRuby 1.2
  • Component/s: Core Classes/Modules
  • Labels:
    None
  • Environment:
    Solaris 10, Java 1.5, JRuby 2007-05-31 rev 3672
  • Number of attachments :
    2

Description

The IO.sysopen method does not appear to exist in JRuby:

IO.sysopen('some_file') # => JRuby raises NoMethodError

Attachments

  1. Text File
    sysopen.1_1.patch
    09/Dec/08 4:58 PM
    7 kB
    Riley Lynch
  2. Text File
    sysopen.patch
    09/Nov/07 10:42 AM
    11 kB
    Riley Lynch

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

Is there a reason we don't have sysopen defined? Fix for 1.0.2 and 1.1

Show
Charles Oliver Nutter added a comment - Is there a reason we don't have sysopen defined? Fix for 1.0.2 and 1.1
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumped off 1.0.2.

Show
Charles Oliver Nutter added a comment - Bumped off 1.0.2.
Hide
Permalink
Riley Lynch added a comment -

I have got a patch for this about 90% done, but there's a trick – IOHandlers are registered as WeakReferences, and in this case, we want to keep the IOHandler registered even though we're throwing away the instance almost as soon as we instantiate it – since sysopen() only returns the fileno. Any ideas?

Show
Riley Lynch added a comment - I have got a patch for this about 90% done, but there's a trick – IOHandlers are registered as WeakReferences, and in this case, we want to keep the IOHandler registered even though we're throwing away the instance almost as soon as we instantiate it – since sysopen() only returns the fileno. Any ideas?
Hide
Permalink
Riley Lynch added a comment -

Incomplete implementation of sysopen – incompleteness is indicated in test failure. It seems we cannot chmod an open file, or the chmod mode is clobbered when the file is closed. Not sure how to fix this, so attaching patch as is, with the idea that someone may know how to fix it.

Added a second hashtable of IOHandlers to prevent sysopen'd handlers from being GC'd and finalized. (Another option is to store alternately weak and strong references in ioHandlers, but that will incur type checks on each lookup by fileno.)

Moved openInternal() and getModes() from RubyFile to RubyIO.

Show
Riley Lynch added a comment - Incomplete implementation of sysopen – incompleteness is indicated in test failure. It seems we cannot chmod an open file, or the chmod mode is clobbered when the file is closed. Not sure how to fix this, so attaching patch as is, with the idea that someone may know how to fix it. Added a second hashtable of IOHandlers to prevent sysopen'd handlers from being GC'd and finalized. (Another option is to store alternately weak and strong references in ioHandlers, but that will incur type checks on each lookup by fileno.) Moved openInternal() and getModes() from RubyFile to RubyIO.
Hide
Permalink
Riley Lynch added a comment -

The automatic GC/finalization of IOHandlers is also causing problems for JRUBY-1527. See that bug for a question about whether the ioHandlers hashtable should be using WeakReferences or hard ones.

Show
Riley Lynch added a comment - The automatic GC/finalization of IOHandlers is also causing problems for JRUBY-1527. See that bug for a question about whether the ioHandlers hashtable should be using WeakReferences or hard ones.
Hide
Permalink
Charles Oliver Nutter added a comment -

Do you have a suggestion for a better way to handle IO finalization?

Show
Charles Oliver Nutter added a comment - Do you have a suggestion for a better way to handle IO finalization?
Hide
Permalink
Riley Lynch added a comment -

I'm thinking that the IoHandlers table should hold strong references rather than weak ones – that would make JRuby work a little more like MRI in so far as file descriptors remain open unless explicitly closed. Sysopen assumes this to be the case, as does Tempfile (see JRUBY-1527), and there may be other examples.

I'm not sure why WeakReferences were chosen for the IoHandler cache. But if it turns out to be an assumption that IOHandlers need not outlive enclosing Ruby IO instances, we've now got a couple of cases to invalidate that assumption.

Show
Riley Lynch added a comment - I'm thinking that the IoHandlers table should hold strong references rather than weak ones – that would make JRuby work a little more like MRI in so far as file descriptors remain open unless explicitly closed. Sysopen assumes this to be the case, as does Tempfile (see JRUBY-1527), and there may be other examples. I'm not sure why WeakReferences were chosen for the IoHandler cache. But if it turns out to be an assumption that IOHandlers need not outlive enclosing Ruby IO instances, we've now got a couple of cases to invalidate that assumption.
Hide
Permalink
Charles Oliver Nutter added a comment -

Still affects 1.1 branch. Bumping off dead 1.0 branch.

Show
Charles Oliver Nutter added a comment - Still affects 1.1 branch. Bumping off dead 1.0 branch.
Hide
Permalink
Charles Oliver Nutter added a comment -

Riley: Dunno if you're still ou tthere, but I'm going to try to explore this for 1.1.5 (but not tonight). Things may be simpler now with the new IO subsystem to make this work.

Show
Charles Oliver Nutter added a comment - Riley: Dunno if you're still ou tthere, but I'm going to try to explore this for 1.1.5 (but not tonight). Things may be simpler now with the new IO subsystem to make this work.
Hide
Permalink
Riley Lynch added a comment -

I am still out here! Let me know how it goes – I'd be glad to jump back in, if only to get a look at the new IO subsystem.

Show
Riley Lynch added a comment - I am still out here! Let me know how it goes – I'd be glad to jump back in, if only to get a look at the new IO subsystem.
Hide
Permalink
Charles Oliver Nutter added a comment -

Well if you want to have a look, it's a lot easier to cope with than the "IOHandlerStew" we used to have. It should be easier to implement sysopen now. The weak list of open channels is still there, but your technique to create a separate list may work well.

Show
Charles Oliver Nutter added a comment - Well if you want to have a look, it's a lot easier to cope with than the "IOHandlerStew" we used to have. It should be easier to implement sysopen now. The weak list of open channels is still there, but your technique to create a separate list may work well.
Hide
Permalink
Charles Oliver Nutter added a comment -

Unless someone swoops in to implement this in the next couple days, it's won't be in 1.1.5. Punting.

Show
Charles Oliver Nutter added a comment - Unless someone swoops in to implement this in the next couple days, it's won't be in 1.1.5. Punting.
Hide
Permalink
Charles Oliver Nutter added a comment -

Now the cause of multiple rubyspec failures:

     [java] 6)
     [java] IO.sysopen returns the file descriptor for a given path ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:15
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 7)
     [java] IO.sysopen works on directories ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:21
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 8)
     [java] IO.sysopen accepts a mode as second argument FAILED
     [java] Expected to not get Exception 
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:28
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 9)
     [java] IO.sysopen accepts permissions as third argument ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:33
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 10)
     [java] An exception occurred during: after :all ERROR
     [java] Errno::ENOENT: No such file or directory -  No such file or directory - "/private/tmp/rubinius-spec-io-sysopen-65854.txt"
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:10
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
Show
Charles Oliver Nutter added a comment - Now the cause of multiple rubyspec failures: 
     [java] 6)
     [java] IO.sysopen returns the file descriptor for a given path ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:15
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 7)
     [java] IO.sysopen works on directories ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:21
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 8)
     [java] IO.sysopen accepts a mode as second argument FAILED
     [java] Expected to not get Exception 
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:28
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 9)
     [java] IO.sysopen accepts permissions as third argument ERROR
     [java] NoMethodError: undefined method `sysopen' for IO:Class
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:33
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
     [java] 
     [java] 10)
     [java] An exception occurred during: after :all ERROR
     [java] Errno::ENOENT: No such file or directory -  No such file or directory - "/private/tmp/rubinius-spec-io-sysopen-65854.txt"
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:10
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:3
     [java] /Users/headius/projects/jruby/spec/ruby/1.8/core/io/sysopen_spec.rb:54:in `load'
Hide
Permalink
Charles Oliver Nutter added a comment -

We'd love more help Riley, and we promise to commit your patch this time Punting to 1.1+ for someone to help out.

Show
Charles Oliver Nutter added a comment - We'd love more help Riley, and we promise to commit your patch this time Punting to 1.1+ for someone to help out.
Hide
Permalink
Riley Lynch added a comment -

I'm looking into how to port the patch to 1.1.6RC2.

Show
Riley Lynch added a comment - I'm looking into how to port the patch to 1.1.6RC2.
Hide
Permalink
Riley Lynch added a comment -

Fix for 1.1.6_RC2 – same approach as before: Setting up a (non-weak) map to cache IO descriptors opened with sysopen. One potential problem is that the descriptors are never removed from the map – it may be desirable to remove these descriptors from the map when they are closed.

Show
Riley Lynch added a comment - Fix for 1.1.6_RC2 – same approach as before: Setting up a (non-weak) map to cache IO descriptors opened with sysopen. One potential problem is that the descriptors are never removed from the map – it may be desirable to remove these descriptors from the map when they are closed.
Hide
Permalink
Charles Oliver Nutter added a comment -

Riley, I like it. I did trace through the logic for IO#close, and as it snakes through OpenFile it eventually does get to a point where it tries to unregister the descriptor, so I think that angle's covered. In theory, if every sysopen "fd" is later closed properly, there should be no problem. And if there is a problem...well it would be an fd leak under normal Ruby anyway.

Committed with minor modifications in r9212.

Show
Charles Oliver Nutter added a comment - Riley, I like it. I did trace through the logic for IO#close, and as it snakes through OpenFile it eventually does get to a point where it tries to unregister the descriptor, so I think that angle's covered. In theory, if every sysopen "fd" is later closed properly, there should be no problem. And if there is a problem...well it would be an fd leak under normal Ruby anyway. Committed with minor modifications in r9212.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.