Bundler fails to read some gemspecs under JRuby in 1.9 mode

There are some bad YAML files out there. Can you figure out which gem is causing this? Almost every case we've found where YAML broke under our 1.9 parser were actually not valid YAML 1.1. We'd rather not switch back to the more permissive (and more broken) 1.0-ish-sorta "syck" port we run in 1.8 mode.

I wasn't clear. The problem isn't a particular gem – the gemspec the loading fails on is the gemspec for the gem that's using bundler. (The one in the gist actually doesn't have any dependencies.)

The problem arises from the fact that a gemspec can be either YAML or the ruby DSL. In order to handle both cases, bundler first tells rubygems to interpret a gemspec it encounters as YAML. If that fails, bundler attempts to interpret it as ruby.

Bundler catches several exceptions (ArgumentError, SyntaxError, Gem::EndOfYAMLException, Gem::Exception) from Gem::Specification.from_yaml, interprets them as failures and moves on to the ruby DSL mode. When parsing the gemspec in the gist, JRuby's 1.9-mode YAML parser throws a RuntimeError. It bubbles out to the surface and kills bundler.

Summary: the content being parsed is definitely invalid YAML – the problem is that the exception that the parser is throwing on failure is different from what MRI & JRuby in 1.8-mode throw.

Ah-ha, I see. So this is a case where for totally invalid YAML we are blowing up with an internal error, causing Bundler's fallback logic to never failover to the .rb gemspec logic.

I will fix.

Silly me...I was calling Kernel#raise wrong to handle the YAML exception. It should work now.

Can you add a spec for this to RubySpec? Perhaps take some YAML that generates that RuntimeError in JRuby and add it to a spec saying "raises SyntaxError for bad YAML".

commit 01cc1a0930c077547a430d7356c6801e64811f73
Author: Charles Oliver Nutter <headius@headius.com>
Date: Sun Apr 10 19:47:43 2011 -0500

Fix JRUBY-5682: Bundler fails to read some gemspecs under JRuby in 1.9 mode

• When attempting to raise a Psych::SyntaxError, we were actually always raising RuntimeError.

Can you add a spec for this to RubySpec? Perhaps take some YAML that generates that RuntimeError in JRuby and add it to a spec saying "raises SyntaxError for bad YAML".

That's a good idea. I looked into it, though, and on other platforms (e.g., MRI 1.9.2), YAML.load doesn't raise a syntax error for this content:

ruby-1.9.2-p180 :001 > require 'yaml'
 => true 
ruby-1.9.2-p180 :002 > s = "Gem::Specification.new do |s|\n  s.name = 'foo'\n  s.version = '1.0.0'\n# any line starting at column zero at this point provokes the problem\nend\n"
 => "Gem::Specification.new do |s|\n  s.name = 'foo'\n  s.version = '1.0.0'\n# any line starting at column zero at this point provokes the problem\nend\n" 
ruby-1.9.2-p180 :003 > YAML.load s
 => "Gem::Specification.new do |s| s.name = 'foo' s.version = '1.0.0' # any line starting at column zero at this point provokes the problem end"

It turns out the exception that bundler is catching in the original scenario is a Gem::Exception thrown by Gem::Specification.from_yaml when the YAML parses out to a string.