JRuby

Dir::entries does not support none ascii character

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.6.5
  • Fix Version/s: JRuby 1.7.0.pre1
  • Component/s: Core Classes/Modules
  • Labels:
    None
  • Environment:
    OS:Windows XP SP2
    Jruby version:1.5.6 and 1.6.5
  • Number of attachments :
    0

Description

Assume that we have a directory ,the directory name contains none ascii character(such like "照片",means photo in chinese)。
try Dir.entries(directoryname)

under ruby1.8.6,the method works ok.
under jruby 1.5.6 and jruby 1.6.5,the error is "No such file or directory - No such directory"

Activity

Ascending order - Click to sort in descending order
Hiro Asari made changes -
Field Original Value New Value
Description Assume that we have a directory ,the directory name contains none ascii character(such like "照片",means photo in chinese)。
try Dir.entries(directoryname);
under ruby1.8.6,the method works ok.
under jruby1.5.6andjruby1.6.5,the error is "No such file or directory - No such directory" 		Assume that we have a directory ,the directory name contains none ascii character(such like "照片",means photo in chinese)。
try {{Dir.entries(directoryname)}}

under ruby1.8.6,the method works ok.
under jruby 1.5.6 and jruby 1.6.5,the error is "No such file or directory - No such directory"
Hide
Permalink
ryenus added a comment - - edited

I have reproduced the issue.

For 1.9 mode (currently default for master), top of the stacktrace is:
RubyString.getUnicodeValue() line: 7628
RubyDir.getPath19(ThreadContext, IRubyObject) line: 735
RubyDir.entries19(ThreadContext, IRubyObject, IRubyObject) line: 367

The problem is the path of type RubyString is converted to java.lang.String via method RubyEncoding.decodeUTF8, which assumes the RubyString got passed in is a UTF-8 byte list, it would fail for all not-ASCII-compatible locales, especially CJK-ish Windows.

Checking RubyString class, I found the methods decodeString() and the wrappers asJavaString(), toString() can correctly handle this.

Show
ryenus added a comment - - edited I have reproduced the issue. For 1.9 mode (currently default for master), top of the stacktrace is: RubyString.getUnicodeValue() line: 7628 RubyDir.getPath19(ThreadContext, IRubyObject) line: 735 RubyDir.entries19(ThreadContext, IRubyObject, IRubyObject) line: 367 The problem is the path of type RubyString is converted to java.lang.String via method RubyEncoding.decodeUTF8, which assumes the RubyString got passed in is a UTF-8 byte list, it would fail for all not-ASCII-compatible locales, especially CJK-ish Windows. Checking RubyString class, I found the methods decodeString() and the wrappers asJavaString(), toString() can correctly handle this.
Hide
Permalink
tangguochang added a comment -

Dir.foreach method has the same issue too.

Show
tangguochang added a comment - Dir.foreach method has the same issue too.
Hide
Permalink
ryenus added a comment -

patch for 1.9 mode submitted, see https://github.com/jruby/jruby/pull/159

Show
ryenus added a comment - patch for 1.9 mode submitted, see https://github.com/jruby/jruby/pull/159
Hide
Permalink
Charles Oliver Nutter added a comment -

Resolved on master by ryenus's pull request. Thank you!

commit 766bb8e50525e0379af08731365ecc85228cab55
Author: Charles Oliver Nutter <headius@headius.com>
Date:   Mon May 14 15:23:43 2012 -0500

    Tweaks for #159.

commit c2abc8caed9dfb383d4ab8d54267ba3cf7b81c20
Author: ryenus <ryenus@gmail.com>
Date:   Sun May 6 22:04:30 2012 +0800

    #6606 - respect external encoding instead of hard coding UTF-8

commit 6bb53c8e3207632064e61666caa3d56d72ab0d0b
Author: ryenus <ryenus@gmail.com>
Date:   Sun May 6 22:03:42 2012 +0800

    add test for JRUBY-6606
Show
Charles Oliver Nutter added a comment - Resolved on master by ryenus's pull request. Thank you! commit 766bb8e50525e0379af08731365ecc85228cab55 Author: Charles Oliver Nutter <headius@headius.com> Date: Mon May 14 15:23:43 2012 -0500 Tweaks for #159. commit c2abc8caed9dfb383d4ab8d54267ba3cf7b81c20 Author: ryenus <ryenus@gmail.com> Date: Sun May 6 22:04:30 2012 +0800 #6606 - respect external encoding instead of hard coding UTF-8 commit 6bb53c8e3207632064e61666caa3d56d72ab0d0b Author: ryenus <ryenus@gmail.com> Date: Sun May 6 22:03:42 2012 +0800 add test for JRUBY-6606
Charles Oliver Nutter made changes -
Status Open [ 1 ] Resolved [ 5 ]
Assignee Charles Oliver Nutter [ headius ]
Fix Version/s JRuby 1.7 [ 17049 ]
Resolution Fixed [ 1 ]

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    tangguochang
Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: