Using shoulda prevents Rails from running tests in ruby1.9 mode

This bug is reported here because it can be reporoduced only in JRuby. It works fine with MRI 1.9

Confirmed against current master. Looking into it.

A reproduction outside of Rails does not seem to have the same problem:

~/projects/jruby ➔ cat shoulda_context_test.rb 
require 'shoulda'
require 'test/unit'

class RoleTest < Test::Unit::TestCase
  def test_something
    true
  end

  context 'asdf' do
    should 'blah' do
    end
  end
end

~/projects/jruby &#10132; jruby --1.9 shoulda_context_test.rb 
Loaded suite shoulda_context_test
Started
..
Finished in 0.006000 seconds.

2 tests, 0 assertions, 0 failures, 0 errors, 0 skips

Test run options: --seed 49488

This could make it trickier to diagnose.

Ok, this is very weird. If I specify not to launch Ruby "subprocesses" in the same process (our default behavior), it runs fine. The ordering is also odd...if we don't launch in-process, there's two test runs, and the first run shows the output...that's probably the "inner" process. If we launch out of process, I think we're only seeing the results of the second run, which would probably be the "outer" process.

Ok wow, was this ever an involved bug to investigate.

The problem stemmed from how Shoulda built up the name of the test method in response to the "should" call within the "context" block. When the context was not nested, there was no parent context, so parent_name would be nil below:

    def full_name
      parent_name = parent.full_name if am_subcontext?
      return [parent_name, name].join(" ").strip
    end
...
    def create_test_from_should_hash(should)
      test_name = ["test:", full_name, "should", "#{should[:name]}. "].flatten.join(' ').to_sym

When joining an array with non-strings, Ruby 1.9 attempts to coerce those values with to_str, falling back on to_s if the result can't be coerced. It does this by dispatching to to_str and ignoring the resulting error.

We had the same logic in place to ignore the error from the to_str coercion, but we were not clearing the $! "error info" global variable. As a result, MiniTest was refusing to run its at_exit hook that drives all tests:

    ##
    # Registers MiniTest::Unit to run tests at process exit

    def self.autorun
      at_exit {
        next if $! # don't run if there was an exception
        exit_code = MiniTest::Unit.new.run(ARGV)
        exit false if exit_code && exit_code != 0
      } unless @@installed_at_exit
      @@installed_at_exit = true
    end

The reason it worked for me when "really" launching an external process is because my external JRuby instance was running in 1.8 mode. Ruby 1.8 does not handle array element coercion the same way, so the $! variable never had a chance to be incorrectly left set.

The patch is simple; when ignoring the NoMethodError raised by the to_str coercion call, we should also clear $!.

Tricky to find, easy to fix.

JRuby:

commit 3cce50726667579ef1cb3a50953b040108956fb9
Author: Charles Oliver Nutter <headius@headius.com>
Date: Fri Mar 4 01:39:32 2011 -0600

Fix JRUBY-5550: Using shoulda prevents Rails from running tests in ruby1.9 mode

Array#join under 1.9 mode tries to coerce non-strings using to_str. NoMethodError is ignored, but in our case we ignored it and left $! set. This prevented Miniunit from running tests.

RubySpec:

commit 21f836623f6f1a443ee833e9d94d0c2de31f7987
Author: Charles Oliver Nutter <headius@headius.com>
Date: Thu Mar 3 18:53:31 2011 -0600

Add spec for $! result after failed coercion by a core class against a method_missing-implementing class.

Awesome job. Thank you.