Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.6.5
    • Fix Version/s: JRuby 1.6.7
    • Component/s: None
    • Labels:
      None
    • Environment:
      jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (Java HotSpot(TM) 64-Bit Server VM 1.7.0_01) [Windows 7-amd64-java]
    • Number of attachments :
      0

      Description

      the result of jruby --1.9 -e 'puts Gem.user_home' is garbled, see below

      C:\>chcp
      Active code page: 1252
      
      C:\>echo %USERPROFILE%
      C:\Users\xxx
      
      C:\>ruby --version && ruby -e 'puts Gem.user_home'
      ruby 1.9.2p290 (2011-07-09) [i386-mingw32]
      C:/Users/xxx
      
      C:\>jruby --version && jruby --1.9 -e 'puts Gem.user_home'
      jruby 1.6.5 (ruby-1.8.7-p330) (2011-10-25 9dcd388) (Java HotSpot(TM) 64-Bit Server VM 1.7.0_01) [Win
      dows 7-amd64-java]
      C:/C:₩Users₩xxx
      
      C:\>
      

        Activity

        Hide
        ryenus added a comment -

        It turns out to be a bug in Rubygems, see https://github.com/rubygems/rubygems/pull/229

        Show
        ryenus added a comment - It turns out to be a bug in Rubygems, see https://github.com/rubygems/rubygems/pull/229
        Hide
        ryenus added a comment -

        I tested again on another box with a fresh downloaded jruby 1.6.5 and cannot reproduce the issue.

        Show
        ryenus added a comment - I tested again on another box with a fresh downloaded jruby 1.6.5 and cannot reproduce the issue.
        Hide
        ryenus added a comment -

        Now the reproducing condition is clear, and it's an encoding related issue.

        the issue can be reproduced with jruby -J-Dfile.encoding=cp936 --1.9 -e "puts File.expand_path '~'".

        and chcp doesn't help.

        Show
        ryenus added a comment - Now the reproducing condition is clear, and it's an encoding related issue. the issue can be reproduced with jruby -J-Dfile.encoding=cp936 --1.9 -e "puts File.expand_path '~'". and chcp doesn't help.
        Hide
        Charles Oliver Nutter added a comment -

        If you can reproduce this with some regularity, can you test it against JRuby master (1.6) or JRuby 1.6.6 (snapshots from http://ci.jruby.org/snapshot). There have been many encoding-related fixes recently that aren't in 1.6.5.

        Show
        Charles Oliver Nutter added a comment - If you can reproduce this with some regularity, can you test it against JRuby master (1.6) or JRuby 1.6.6 (snapshots from http://ci.jruby.org/snapshot ). There have been many encoding-related fixes recently that aren't in 1.6.5.
        Hide
        ryenus added a comment - - edited

        I just tested it against master and jruby-1_6, same result:

        $ java -Dfile.encoding=cp936 -jar lib/jruby.jar --1.9 -e "puts RUBY_VERSION, File.expand_path('~')" && git br -v
        
        1.9.3
        
        E:/sandbox/jruby/C:₩Users₩roger
        
        * master      0cd7377 Fix JRUBY-5967
        
        Show
        ryenus added a comment - - edited I just tested it against master and jruby-1_6, same result: $ java -Dfile.encoding=cp936 -jar lib/jruby.jar --1.9 -e "puts RUBY_VERSION, File.expand_path('~')" && git br -v 1.9.3 E:/sandbox/jruby/C:₩Users₩roger * master 0cd7377 Fix JRUBY-5967
        Hide
        ryenus added a comment -

        I think I've located the problem in JRuby (though very likely an external bug).

        In org.jruby.util.OSEnvironment#getAsMapOfRubyStrings, the encoding is retrieved like this: Encoding encoding = runtime.getEncodingService().getLocaleEncoding();

        that line of the code is buggy, when java.nio.charset.Charset.defaultCharset().toString() == "GBK", the above code returns CP949Encoding, I suppose it should return GBKEncoding

        Specifically, the code 5c for '\' in GBK is different for that in CP949, instead in CP949 it's for the 'WON SIGN' (see here: http://www.fileformat.info/info/charset/x-IBM949/decode.htm, seems CP949 is asymmetric).

        I'm just curious, since JVM can correctly report the name of the default charset, why don't we use that to find the correct encoding but use jcoding's code to detect it?

        Show
        ryenus added a comment - I think I've located the problem in JRuby (though very likely an external bug). In org.jruby.util.OSEnvironment#getAsMapOfRubyStrings , the encoding is retrieved like this: Encoding encoding = runtime.getEncodingService().getLocaleEncoding(); that line of the code is buggy, when java.nio.charset.Charset.defaultCharset().toString() == "GBK" , the above code returns CP949Encoding , I suppose it should return GBKEncoding Specifically, the code 5c for '\' in GBK is different for that in CP949, instead in CP949 it's for the 'WON SIGN' (see here: http://www.fileformat.info/info/charset/x-IBM949/decode.htm , seems CP949 is asymmetric). I'm just curious, since JVM can correctly report the name of the default charset, why don't we use that to find the correct encoding but use jcoding's code to detect it?
        Hide
        Hiro Asari added a comment -

        Hmm. This sounds very much like JRUBY-6398. For that bug's fix, I decided to default to ASCII-8BIT if JVM doesn't have a known default Charset. Does that work for you? Can you test one from http://ci.jruby.org/snapshots/release/ ?

        Thank you.

        Show
        Hiro Asari added a comment - Hmm. This sounds very much like JRUBY-6398 . For that bug's fix, I decided to default to ASCII-8BIT if JVM doesn't have a known default Charset. Does that work for you? Can you test one from http://ci.jruby.org/snapshots/release/ ? Thank you.
        Hide
        Hiro Asari added a comment -

        ryenus,

        Sorry, I skipped over some details of your update above. I will look into this a bit further.

        Show
        Hiro Asari added a comment - ryenus, Sorry, I skipped over some details of your update above. I will look into this a bit further.
        Hide
        ryenus added a comment -

        OK, the bug is in jcodings, see https://github.com/jruby/jcodings/pull/2

        Show
        ryenus added a comment - OK, the bug is in jcodings, see https://github.com/jruby/jcodings/pull/2
        Hide
        ryenus added a comment -

        well, that one is probably different, I'm happy to take a look, but definitely later

        Show
        ryenus added a comment - well, that one is probably different, I'm happy to take a look, but definitely later
        Hide
        ryenus added a comment -
        [04:28:36 roger@R /forks/jcodings (fix-gbk)]
        $ cp target/jcodings.jar /sandbox/jruby/build_lib/jcodings.jar
        
        [04:29:08 roger@R /sandbox/jruby (master)]
        $ ant clean jar
        Buildfile: E:\sandbox\jruby\build.xml
        ...
        BUILD SUCCESSFUL
        Total time: 1 minute 58 seconds
        
        [04:31:12 roger@R /sandbox/jruby (master)]
        $ java -jar lib/jruby.jar --1.9 -e "puts File.expand_path '~'"
        C:/Users/roger
        
        [04:31:42 roger@R /sandbox/jruby (master)]
        $ java -Dfile.encoding=gbk -jar lib/jruby.jar --1.9 -e "puts File.expand_path '~'"
        C:/Users/roger
        

        Oh, yeah!

        Show
        ryenus added a comment - [04:28:36 roger@R /forks/jcodings (fix-gbk)] $ cp target/jcodings.jar /sandbox/jruby/build_lib/jcodings.jar [04:29:08 roger@R /sandbox/jruby (master)] $ ant clean jar Buildfile: E:\sandbox\jruby\build.xml ... BUILD SUCCESSFUL Total time: 1 minute 58 seconds [04:31:12 roger@R /sandbox/jruby (master)] $ java -jar lib/jruby.jar --1.9 -e "puts File.expand_path '~'" C:/Users/roger [04:31:42 roger@R /sandbox/jruby (master)] $ java -Dfile.encoding=gbk -jar lib/jruby.jar --1.9 -e "puts File.expand_path '~'" C:/Users/roger Oh, yeah!
        Hide
        Hiro Asari added a comment -

        OK, I merged the pull request. I am not familiar with the release procedure for jcodings, so I'll let Tom and Marcin take care of that. Until the fix is released (I suppose it will be jcodings 1.0.7), the only way to get the fix is to build jruby from source after copying it to build_lib.

        Show
        Hiro Asari added a comment - OK, I merged the pull request. I am not familiar with the release procedure for jcodings, so I'll let Tom and Marcin take care of that. Until the fix is released (I suppose it will be jcodings 1.0.7), the only way to get the fix is to build jruby from source after copying it to build_lib.
        Hide
        ryenus added a comment -

        What about a test, e.g.: https://gist.github.com/1808755

        Show
        ryenus added a comment - What about a test, e.g.: https://gist.github.com/1808755
        Hide
        ryenus added a comment - - edited

        tested with fresh 1.6.7 release, problem fixed. this should have been in 1.6.7 changelog

        Show
        ryenus added a comment - - edited tested with fresh 1.6.7 release, problem fixed. this should have been in 1.6.7 changelog
        Hide
        Charles Oliver Nutter added a comment -

        ryenus: Thank you for following up!

        Show
        Charles Oliver Nutter added a comment - ryenus: Thank you for following up!

          People

          • Assignee:
            Thomas E Enebo
            Reporter:
            ryenus
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: