[1.9] Psych::SyntaxError when trying to install will_paginate 3.0.3

Confirm same issue with:

jruby 1.6.6 (ruby-1.9.2-p312) (2012-02-01 5673572) (Java HotSpot(TM) Client VM 1.6.0_29) [darwin-i386-java]

Still, this is an encoding issue; if you pass -U to JRuby, will_paginate will install successfully.

$ ./bin/jruby --1.9 -v; ./bin/jruby --1.9 -S gem list will_paginate; ./bin/jruby --1.9 -U -S gem install will_paginate
jruby 1.6.7.dev (ruby-1.9.2-p312) (2012-02-07 eb67481) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_29) [darwin-x86_64-java]

*** LOCAL GEMS ***


Successfully installed will_paginate-3.0.3
1 gem installed

When Psych gets to read the YAML data from .gemspec, Mislav's last name is mangled to "MarohniÄ" (Capital A with umlaut).

The gem itself contains my name correctly encoded:

wget http://rubygems.org/gems/will_paginate-3.0.3.gem -O- |\
  tar -xO metadata.gz | gunzip | grep Mislav
#=> - Mislav Marohnić

Ironically, dumb Jira mangles my surname here. The above string is "Marohnić".

The difference between previous will_paginate versions (which don't exhibit this problem) is that in them, that character in my name was embedded ruby-encoded rather than literal:

- "Mislav Marohni\xC4\x87"

This leads me to believe this error is because it tries to read the gem's metadata as ASCII rather than UTF-8 so it chokes on non-ascii characters. Previously it didn't choke because the character was in its encoded form.

Interestingly, I can't reproduce the problem on OS X.

Investigating.

I think we may have fixed this with some other core method fix in 1.6.7. I could not reproduce on OS X or Linux on either master or jruby-1_6 branch.

Mislav: can you try 1.6.7 please, either by downloading from http://ci.jruby.org/snapshots/release or by installing from rvm with rvm install jruby-1.6-head --branch jruby-1_6? If you still see the problem, reopen and we'll dig deeper.

Whoops...I forgot to run 1.9 mode on 1.6 branch.

Confirmed here, on OS X.

Looks like this is basically the proper fixes for GzipReader that nahi and I made on master, never ported to 1.6 branch. I figured they might have to come over eventually.

I have the port in process.

Fixed! Please confirm using 1.6.7, if you get a chance (instructions for getting it above).

commit 2a7a82227b4ae90d419fc19890ee09e8544b9fd0
Author: Charles Oliver Nutter <headius@headius.com>
Date:   Wed Feb 15 14:37:14 2012 -0600

    Port GzipReader commits from master for JRUBY-6415.
    
    commit 773a1553da4a8b7a7b736480e04ec4b291ce64f4
    Author: Charles Oliver Nutter <headius@headius.com>
    Date:   Mon Jan 9 02:40:33 2012 -0600
    
        Actually set new bytelists to external encoding in GzipReader.
    
    commit 84e3e3fe2f6013c844fb8fff471ede36aca51f46
    Author: Hiroshi Nakamura <nahi@ruby-lang.org>
    Date:   Wed Feb 1 13:37:00 2012 +0900
    
        Fix GzipReader encoding on 1.9 mode (produced encoding == null String)
    
        It's a fix for "-J-ea pointed out we were creating a bad ByteList with
        no encoding in 1.9 mode..." reported by Tom.
    
        The actual bug is from 1 line, that I forgot to set default
        externalEncoding to runtime.getDefaultExternalEncoding()
        But the problem is from name confusion.  Use proper names, readEncoding
        and writeEncoding for enc, enc2 in Ruby's IO subsystem.