JRuby

Specs for all load path, load/require, home dir, url, and other load-logic cases

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: JRuby 1.x+
  • Component/s: Core Classes/Modules
  • Labels:
    None
  • Number of attachments :
    0

Description

There are many different combinations of logic involved in the loading of files. Very few of them have a clear set (or perhaps any set) of codified specifications. This bug can be a placeholder for recording various combinations of behavior, in the hope that a set of specifications will follow (and fixes to JRuby will follow that).

I'll start it off with a failing case in JRuby with load of a file using ./ pathing:

~/projects/jruby ➔ jruby -Ifoo -e "load 'foo/bar.rb'"
foo/bar.rb:1: unhandled exception
	from foo/bar.rb:1:in `load'
	from -e:1

~/projects/jruby ➔ ruby -Ifoo -e "load 'foo/bar.rb'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `load'
	from -e:1

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

From JRUBY-2909:

It seems that largely, Ruby leaves relative paths relative and absolute paths absolute, using those literal paths in logging and traces. It also resolves relative paths dynamically at runtime based on the process CWD:

~/projects/jruby ➔ ruby -Ifoo -e "require 'bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ cd foo
~/projects/jruby/foo ➔ ruby -e "require 'bar'"
./bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/foo ➔ cd ../lib
~/projects/jruby/lib ➔ ruby -I../foo -e "require 'bar'"
../foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ ruby -I../../jruby/foo -e "require 'bar'"
../../jruby/foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ ruby -I/Users/headius/projects/jruby/foo -e "require 'bar'"
/Users/headius/projects/jruby/foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ cd ..
~/projects/jruby ➔ ruby -Ifoo -e "Dir.chdir('lib'); require 'bar'"
-e:1:in `require': no such file to load -- bar (LoadError)
	from -e:1

So our problem, and the reason why we always seem to have full paths in traces, is that we always turn paths into absolute paths, losing that original short path. This also represents a behavioral difference; notice in the last case that when the CWD is changed, relative paths no longer resolve the same.

Show
Charles Oliver Nutter added a comment - From JRUBY-2909: It seems that largely, Ruby leaves relative paths relative and absolute paths absolute, using those literal paths in logging and traces. It also resolves relative paths dynamically at runtime based on the process CWD: 
~/projects/jruby ➔ ruby -Ifoo -e "require 'bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ cd foo
~/projects/jruby/foo ➔ ruby -e "require 'bar'"
./bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/foo ➔ cd ../lib
~/projects/jruby/lib ➔ ruby -I../foo -e "require 'bar'"
../foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ ruby -I../../jruby/foo -e "require 'bar'"
../../jruby/foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ ruby -I/Users/headius/projects/jruby/foo -e "require 'bar'"
/Users/headius/projects/jruby/foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby/lib ➔ cd ..
~/projects/jruby ➔ ruby -Ifoo -e "Dir.chdir('lib'); require 'bar'"
-e:1:in `require': no such file to load -- bar (LoadError)
	from -e:1
So our problem, and the reason why we always seem to have full paths in traces, is that we always turn paths into absolute paths, losing that original short path. This also represents a behavioral difference; notice in the last case that when the CWD is changed, relative paths no longer resolve the same.
Hide
Permalink
Charles Oliver Nutter added a comment -

From JRUBY-2909:

A few more interesting cases:

~/projects/jruby ➔ ruby -Ifoo -e "require 'bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ ruby -e "require 'foo/bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ ruby -e "require './foo/bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1

Note that, while not immediately apparent here, Ruby does appear to attempt to certain paths as-is before trying load paths, and does not appear to ever attach load paths to absolute paths (which makes some sense). So adding a short-circuit that tries the path as-is first seems to also be in the cards.

Show
Charles Oliver Nutter added a comment - From JRUBY-2909: A few more interesting cases: 
~/projects/jruby ➔ ruby -Ifoo -e "require 'bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ ruby -e "require 'foo/bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
~/projects/jruby ➔ ruby -e "require './foo/bar'"
./foo/bar.rb:1: unhandled exception
	from -e:1:in `require'
	from -e:1
Note that, while not immediately apparent here, Ruby does appear to attempt to certain paths as-is before trying load paths, and does not appear to ever attach load paths to absolute paths (which makes some sense). So adding a short-circuit that tries the path as-is first seems to also be in the cards.
Hide
Permalink
Charles Oliver Nutter added a comment -

From JRUBY-2909:

A couple more, showing Ruby does not attempt to search load path for a relative path based on . (CWD):

First with no bar.rb in current dir, but a bar.rb in the 'foo' dir:

~/projects/jruby ➔ ruby -Ifoo -e "require './bar.rb'"
-e:1:in `require': no such file to load -- ./bar.rb (LoadError)
	from -e:1

Now with a bar.rb in current dir:

~/projects/jruby ➔ echo puts \'hello\' > bar.rb
~/projects/jruby ➔ ruby -Ifoo -e "require './bar.rb'"
hello

Obviously we need to put together specs for all the many cases. A few more that need to be specified:

  • home dir using ~ and ~username
  • all URLs, not just the limited support for file: and jar: we have now
  • Windows paths
  • paths with symlinks (to see whether they get expanded or left as-is in traces, etc)

Others?

Show
Charles Oliver Nutter added a comment - From JRUBY-2909: A couple more, showing Ruby does not attempt to search load path for a relative path based on . (CWD): First with no bar.rb in current dir, but a bar.rb in the 'foo' dir: 
~/projects/jruby ➔ ruby -Ifoo -e "require './bar.rb'"
-e:1:in `require': no such file to load -- ./bar.rb (LoadError)
	from -e:1
Now with a bar.rb in current dir: 
~/projects/jruby ➔ echo puts \'hello\' > bar.rb
~/projects/jruby ➔ ruby -Ifoo -e "require './bar.rb'"
hello
Obviously we need to put together specs for all the many cases. A few more that need to be specified:
  • home dir using ~ and ~username
  • all URLs, not just the limited support for file: and jar: we have now
  • Windows paths
  • paths with symlinks (to see whether they get expanded or left as-is in traces, etc)
Others?
Hide
Permalink
Charles Oliver Nutter added a comment -

From JRUBY-2922, mostly fixed now I believe but not spec'ed or tested anywhere:

Note the difference between these two traces:

Ruby:

./test2.rb:3: unhandled exception
	from test1.rb:2:in `require'
	from test1.rb:2

JRuby:

/Users/headius/NetBeansProjects/jruby/./test2.rb:3:in `(unknown)': unhandled exception
	from /Users/headius/NetBeansProjects/jruby/./test2.rb:2:in `require'
	from test1.rb:2:in `(unknown)'

LoadService is currently expanding paths to their full absolute path (and in some cases, to their canonical non-symlinked path) before passing the file off to be parsed and loaded. This is largely why backtraces, trace events, compiled package names, and so on have the full path. Tom Enebo fixed the parser in the last release to not use long paths, and it seems that currently the only remaining issue is in the load process.

Show
Charles Oliver Nutter added a comment - From JRUBY-2922, mostly fixed now I believe but not spec'ed or tested anywhere: Note the difference between these two traces: Ruby: 
./test2.rb:3: unhandled exception
	from test1.rb:2:in `require'
	from test1.rb:2
JRuby: 
/Users/headius/NetBeansProjects/jruby/./test2.rb:3:in `(unknown)': unhandled exception
	from /Users/headius/NetBeansProjects/jruby/./test2.rb:2:in `require'
	from test1.rb:2:in `(unknown)'
LoadService is currently expanding paths to their full absolute path (and in some cases, to their canonical non-symlinked path) before passing the file off to be parsed and loaded. This is largely why backtraces, trace events, compiled package names, and so on have the full path. Tom Enebo fixed the parser in the last release to not use long paths, and it seems that currently the only remaining issue is in the load process.
Hide
Permalink
Charles Oliver Nutter added a comment -

See also JRUBY-3029 for a discussion about .class versus .rb search ordering (fixed along with that bug, but not yet spec'ed/tested well.

Show
Charles Oliver Nutter added a comment - See also JRUBY-3029 for a discussion about .class versus .rb search ordering (fixed along with that bug, but not yet spec'ed/tested well.
Hide
Permalink
Charles Oliver Nutter added a comment -

From JRUBY-3014, still unresolved issues with _FILE_ and jar-based or class-based (precompiled) scripts:

Here's the breakdown:
Compiled and uncompiled Ruby classes from the command line work fine
Compiled and uncompiled Ruby classes in the root of a jar do not properly resolve File.expand_path(File.dirname(_FILE_))
Compiled Ruby classes in a sub-directory of a jar do not properly resolve File.expand_path(File.dirname(_FILE_))
Uncompiled Ruby classes in a sub-directory of a jar DO properly resolve File.expand_path(File.dirname(_FILE_))

#=== file_test.rb ===

puts "__FILE__: #{__FILE__}"
puts "File.dirname(__FILE__): #{File.dirname(__FILE__)}"
puts "File.expand_path(File.dirname(__FILE__)): #{File.expand_path(File.dirname(__FILE__))}"

#=== bootstrap1.rb === (used to test uncompiled Ruby files in a jar)
require 'file_test'

#=== bootstrap2.rb === (used to test uncompiled Ruby files in a jar)
require 'src/file_test'

# GIVES CORRECT OUTPUT
#=== Running the file_test.rb file unmodified using MRI and JRuby ===
ruby file_test.rb 
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

jruby file_test.rb 
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

# GIVES CORRECT OUTPUT
# === Compile the file_test.rb file (jrubyc file_test.rb) ===
java -cp ~/dev/libs/jruby/lib/jruby.jar:. file_test
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

# GIVES INCORRECT OUTPUT
# === Jarring the file_test.rb (jar -cvf file_test.jar file_test.rb bootstrap1.class) and placing jar file in /jar sub-directory ===
# Adding in compiled bootstrap1 class so java has something to launch
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap1 #<-- no output, internally an exception?

# GIVES INCORRECT OUTPUT
# === Jarring the file_test.class (jar -cvf file_test.jar file_test.class) (also placing in /jar sub-directory) ===
# Moved file_test.jar into /jar subdirectory to prevent accidentally pulling in .class file
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar file_test
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test/jar #<-- should have /file_test.jar! on the end

GIVES CORRECT OUTPUT
# === Uncompiled Ruby file in /src directory in jar file ===
# Use bootstrap2 file to require file in 'src/test_file'
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap2
__FILE__: file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src/file_test.rb
File.dirname(__FILE__): file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src
File.expand_path(File.dirname(__FILE__)): file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src #<-- We get the jar file and the sub-dir, yay!

GIVES INCORRECT OUTPUT
# === Compiled Ruby file in /src directory in jar file ===
# As above but with compiled Ruby file, same bootstrap2 file
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap2
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test/jar #<-- missing /file_test.jar!/src
Show
Charles Oliver Nutter added a comment - From JRUBY-3014, still unresolved issues with _FILE_ and jar-based or class-based (precompiled) scripts: Here's the breakdown: Compiled and uncompiled Ruby classes from the command line work fine Compiled and uncompiled Ruby classes in the root of a jar do not properly resolve File.expand_path(File.dirname(_FILE_)) Compiled Ruby classes in a sub-directory of a jar do not properly resolve File.expand_path(File.dirname(_FILE_)) Uncompiled Ruby classes in a sub-directory of a jar DO properly resolve File.expand_path(File.dirname(_FILE_)) 
#=== file_test.rb ===

puts "__FILE__: #{__FILE__}"
puts "File.dirname(__FILE__): #{File.dirname(__FILE__)}"
puts "File.expand_path(File.dirname(__FILE__)): #{File.expand_path(File.dirname(__FILE__))}"

#=== bootstrap1.rb === (used to test uncompiled Ruby files in a jar)
require 'file_test'

#=== bootstrap2.rb === (used to test uncompiled Ruby files in a jar)
require 'src/file_test'

# GIVES CORRECT OUTPUT
#=== Running the file_test.rb file unmodified using MRI and JRuby ===
ruby file_test.rb 
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

jruby file_test.rb 
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

# GIVES CORRECT OUTPUT
# === Compile the file_test.rb file (jrubyc file_test.rb) ===
java -cp ~/dev/libs/jruby/lib/jruby.jar:. file_test
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test

# GIVES INCORRECT OUTPUT
# === Jarring the file_test.rb (jar -cvf file_test.jar file_test.rb bootstrap1.class) and placing jar file in /jar sub-directory ===
# Adding in compiled bootstrap1 class so java has something to launch
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap1 #<-- no output, internally an exception?

# GIVES INCORRECT OUTPUT
# === Jarring the file_test.class (jar -cvf file_test.jar file_test.class) (also placing in /jar sub-directory) ===
# Moved file_test.jar into /jar subdirectory to prevent accidentally pulling in .class file
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar file_test
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test/jar #<-- should have /file_test.jar! on the end

GIVES CORRECT OUTPUT
# === Uncompiled Ruby file in /src directory in jar file ===
# Use bootstrap2 file to require file in 'src/test_file'
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap2
__FILE__: file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src/file_test.rb
File.dirname(__FILE__): file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src
File.expand_path(File.dirname(__FILE__)): file:/Users/david/dev/projects/general/jruby_test/jar/file_test.jar!/src #<-- We get the jar file and the sub-dir, yay!

GIVES INCORRECT OUTPUT
# === Compiled Ruby file in /src directory in jar file ===
# As above but with compiled Ruby file, same bootstrap2 file
java -cp ~/dev/libs/jruby/lib/jruby-complete.jar:file_test.jar bootstrap2
__FILE__: file_test.rb
File.dirname(__FILE__): .
File.expand_path(File.dirname(__FILE__)): /Users/david/dev/projects/general/jruby_test/jar #<-- missing /file_test.jar!/src
Hide
Permalink
Charles Oliver Nutter added a comment -

Still need specs, and there's still outstanding bugs relating to require (like the fact that it always tries cwd even if . isn't in load path).

Show
Charles Oliver Nutter added a comment - Still need specs, and there's still outstanding bugs relating to require (like the fact that it always tries cwd even if . isn't in load path).
Hide
Permalink
Charles Oliver Nutter added a comment -

There are a lot of new specs in RubySpec for load stuff, but we need some of our own for URLs, jar files, all that. It's still fuzzily covered.

Show
Charles Oliver Nutter added a comment - There are a lot of new specs in RubySpec for load stuff, but we need some of our own for URLs, jar files, all that. It's still fuzzily covered.
Hide
Permalink
Hiro Asari added a comment -

Resolving this as (partially) "Fixed".

Show
Hiro Asari added a comment - Resolving this as (partially) "Fixed".

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.