JRuby

rspec with line_number option doesn't find examples with jruby - Maybe more general issue with backtraces?

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Won't Fix
  • Affects Version/s: JRuby 1.1.4
  • Fix Version/s: None
  • Component/s: Application Error
  • Labels:
    None
  • Environment:
    rspec-1.1.3? -> rspec-1.1.11
  • Number of attachments :
    1

Description

With rspec-1.1.11 and jruby-1.4

#linenumber_spec.rb
describe "line number" do
it "should be example one" do
true.should be_true
end
it "should be another example" do
true.should be_true
end
end

> jruby-1.1.4/bin/jruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/
linenumber_spec.rb

Finished in 0.013457 seconds

0 examples, 0 failures

> ruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/linenumber_spec.rb
.

Finished in 0.006864 seconds

1 example, 0 failures

-lenny

Activity

Hide
Lenny Marks added a comment -

Below from rspec mailing list: http://rubyforge.org/pipermail/rspec-users/2008-October/009735.html

I'm not familiar enough with the rspec code to get to the bottom of
this, but I did enough drilling to find its related to the new
backtrace based strategy for
Spec::Runner::SpecParser#spec_name_for(file, line_number). Putting a
print statement inside
Spec::Example::ExampleGroupMethods#registration_backtrace I get the
following which shows that the target spec file isn't even in the
backtrace from jruby.

def registration_backtrace
backtrace = eval("caller", @registration_binding_block)

puts "backtrace: #{backtrace.join("\n")}"

backtrace
end

> ruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/linenumber_spec.rb
backtrace: /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/
example/example_group_methods.rb:27:in `initialize'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/
class.rb:9:in `new'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/
class.rb:9:in `subclass'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/
example_group_methods.rb:66:in `create_nested_example_group'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/
example_group_methods.rb:53:in `describe'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/
example_group_factory.rb:53:in `create_example_group'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/
main.rb:27:in `describe'
/home/jis/compsvc/lenny/tmp/linenumber_spec.rb:1
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
example_group_runner.rb:14:in `load'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
example_group_runner.rb:14:in `load_files'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
example_group_runner.rb:13:in `each'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
example_group_runner.rb:13:in `load_files'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
options.rb:98:in `run_examples'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
command_line.rb:10:in `run'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:4
/Users/Shared/RubyGems/1.8/bin/spec:19:in `load'
/Users/Shared/RubyGems/1.8/bin/spec:19

> jruby-1.1.4/bin/jruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/
linenumber_spec.rb
backtrace: /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/
extensions/class.rb:9:in `eval'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/
example_group_methods.rb:203:in `registration_backtrace'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
spec_parser.rb:35:in `consider_example_groups_for_best_match'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
spec_parser.rb:15:in `spec_name_for'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
spec_parser.rb:14:in `each'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
spec_parser.rb:14:in `spec_name_for'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
options.rb:302:in `set_spec_from_line_number'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
options.rb:114:in `run_examples'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/
command_line.rb:10:in `run'
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:4
/Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:19:in `load'
/Users/Shared/RubyGems/1.8/bin/spec:19

Show
Lenny Marks added a comment - Below from rspec mailing list: http://rubyforge.org/pipermail/rspec-users/2008-October/009735.html I'm not familiar enough with the rspec code to get to the bottom of this, but I did enough drilling to find its related to the new backtrace based strategy for Spec::Runner::SpecParser#spec_name_for(file, line_number). Putting a print statement inside Spec::Example::ExampleGroupMethods#registration_backtrace I get the following which shows that the target spec file isn't even in the backtrace from jruby. def registration_backtrace backtrace = eval("caller", @registration_binding_block) puts "backtrace: #{backtrace.join("\n")}" backtrace end > ruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/linenumber_spec.rb backtrace: /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/ example/example_group_methods.rb:27:in `initialize' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/ class.rb:9:in `new' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/ class.rb:9:in `subclass' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/ example_group_methods.rb:66:in `create_nested_example_group' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/ example_group_methods.rb:53:in `describe' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/ example_group_factory.rb:53:in `create_example_group' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/extensions/ main.rb:27:in `describe' /home/jis/compsvc/lenny/tmp/linenumber_spec.rb:1 /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ example_group_runner.rb:14:in `load' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ example_group_runner.rb:14:in `load_files' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ example_group_runner.rb:13:in `each' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ example_group_runner.rb:13:in `load_files' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ options.rb:98:in `run_examples' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ command_line.rb:10:in `run' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:4 /Users/Shared/RubyGems/1.8/bin/spec:19:in `load' /Users/Shared/RubyGems/1.8/bin/spec:19 > jruby-1.1.4/bin/jruby $GEM_HOME/bin/spec -l 2 $HOME/tmp/ linenumber_spec.rb backtrace: /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/ extensions/class.rb:9:in `eval' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/example/ example_group_methods.rb:203:in `registration_backtrace' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ spec_parser.rb:35:in `consider_example_groups_for_best_match' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ spec_parser.rb:15:in `spec_name_for' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ spec_parser.rb:14:in `each' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ spec_parser.rb:14:in `spec_name_for' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ options.rb:302:in `set_spec_from_line_number' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ options.rb:114:in `run_examples' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/lib/spec/runner/ command_line.rb:10:in `run' /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:4 /Users/Shared/RubyGems/1.8/gems/rspec-1.1.11/bin/spec:19:in `load' /Users/Shared/RubyGems/1.8/bin/spec:19
Hide
Charles Oliver Nutter added a comment -

Ahh, the infamous eval 'caller' thingy. First off, this is a highly suspect bit of code; it's making expectations about how the call stack is structured and what format it will take. Nevertheless, we will try to fix it. I believe some specs were added to the RubySpec project for this, so that may be one place to find a reduced case. We can also just play with various combinations of eval and caller and see what shakes loose.

Unlikely to be fixed in 1.1.5, since we're down to the wire on an internal deadline as it is.

Show
Charles Oliver Nutter added a comment - Ahh, the infamous eval 'caller' thingy. First off, this is a highly suspect bit of code; it's making expectations about how the call stack is structured and what format it will take. Nevertheless, we will try to fix it. I believe some specs were added to the RubySpec project for this, so that may be one place to find a reduced case. We can also just play with various combinations of eval and caller and see what shakes loose. Unlikely to be fixed in 1.1.5, since we're down to the wire on an internal deadline as it is.
Hide
Charles Oliver Nutter added a comment -

Here's an example of the difference between MRI and JRuby. Note that MRI must be keeping the whole stack as a list attached to the binding, which has to be a tremendous waste of resources:

◆ ruby -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `y'
-e:1:in `x'
-e:1
[headius @ cnutter:~/projects/jruby]
◆ jruby -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `eval'
-e:1:in `baz'
-e:1:in `bar'
-e:1:in `foo'
-e:1

Even more damning is the fact that JRuby performs identically to Ruby 1.9.1:

◆ ruby1.9 -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `eval'
-e:1:in `baz'
-e:1:in `bar'
-e:1:in `foo'
-e:1:in `<main>'

It would seem that rspec is depending on a quirky side effect of the 1.8.x implementation of Ruby that can't reasonably be expected to work on other implementations.

Show
Charles Oliver Nutter added a comment - Here's an example of the difference between MRI and JRuby. Note that MRI must be keeping the whole stack as a list attached to the binding, which has to be a tremendous waste of resources:
&#9670; ruby -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `y'
-e:1:in `x'
-e:1
[headius @ cnutter:~/projects/jruby]
&#9670; jruby -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `eval'
-e:1:in `baz'
-e:1:in `bar'
-e:1:in `foo'
-e:1
Even more damning is the fact that JRuby performs identically to Ruby 1.9.1:
&#9670; ruby1.9 -e 'def foo(b); bar(b); end; def bar(b); baz(b); end; def baz(b); eval "caller", b; end; def x; y; end; def y; binding; end; puts foo(x)'
-e:1:in `eval'
-e:1:in `baz'
-e:1:in `bar'
-e:1:in `foo'
-e:1:in `<main>'
It would seem that rspec is depending on a quirky side effect of the 1.8.x implementation of Ruby that can't reasonably be expected to work on other implementations.
Hide
Charles Oliver Nutter added a comment -

Lenny...I'll leave this one open until we come to some consensus with the rspec guys. They're going to have to broach this question with Ruby 1.9.1 anyway, because it behaves the same way we do. There's no trivial fix for this at the moment. It may be worth contacting the rspec guys directly, since they may not be tracking the bugs as closely as we're tracking this one.

Show
Charles Oliver Nutter added a comment - Lenny...I'll leave this one open until we come to some consensus with the rspec guys. They're going to have to broach this question with Ruby 1.9.1 anyway, because it behaves the same way we do. There's no trivial fix for this at the moment. It may be worth contacting the rspec guys directly, since they may not be tracking the bugs as closely as we're tracking this one.
Hide
Wilson Bilkovich added a comment -

In my opinion, the 1.8 output is a bug:
http://redmine.ruby-lang.org/issues/show/146
Please feel free to bump that issue in support.

I have discussed this at length with dchelimsky, and I did an rspec fork on github that is ready for there to be another way to answer this question that RSpec is asking.
Let's design an API for it and add it.
In the above sentence, 'it' means "Asking a Proc on which file/line it was defined"

Show
Wilson Bilkovich added a comment - In my opinion, the 1.8 output is a bug: http://redmine.ruby-lang.org/issues/show/146 Please feel free to bump that issue in support. I have discussed this at length with dchelimsky, and I did an rspec fork on github that is ready for there to be another way to answer this question that RSpec is asking. Let's design an API for it and add it. In the above sentence, 'it' means "Asking a Proc on which file/line it was defined"
Hide
Thomas E Enebo added a comment -

Giving to Charles since he has already spent time on it.. Looks like this may just end up being a resolve?

Show
Thomas E Enebo added a comment - Giving to Charles since he has already spent time on it.. Looks like this may just end up being a resolve?
Hide
Charles Oliver Nutter added a comment -

So I followed up with ruby-core and a number of other folks and I think it's universally agreed that this behavior in 1.8 is unexpected and incorrect. 1.9 doesn't do it, JRuby doesn't do it, and I don't think any new implementation will want to do it. So this is going to be "Won't Fix".

On the light side, I believe rspec guys have patches now for a better way to do this that doesn't depend on caller from a binding doing the 1.8 thing. So please follow up with them about getting a release out that works on JRuby and 1.9.

Show
Charles Oliver Nutter added a comment - So I followed up with ruby-core and a number of other folks and I think it's universally agreed that this behavior in 1.8 is unexpected and incorrect. 1.9 doesn't do it, JRuby doesn't do it, and I don't think any new implementation will want to do it. So this is going to be "Won't Fix". On the light side, I believe rspec guys have patches now for a better way to do this that doesn't depend on caller from a binding doing the 1.8 thing. So please follow up with them about getting a release out that works on JRuby and 1.9.

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: