JRuby

jirb outputs ???? instead of Japanese characters

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.1RC2
  • Fix Version/s: JRuby 1.6.1
  • Component/s: Miscellaneous
  • Labels:
    None
  • Environment:
    Windows XP, FreeBSD, Ubuntu, Mac OS X
  • Number of attachments :
    6

Description

As reported at http://www.netbeans.org/issues/show_bug.cgi?id=108835, jirb outputs ???? instead of correct Japanese characters. This bug comes from JRuby itself and also appears in 1.0.3.

  1. Text File
    readline.patch
    02/Mar/08 12:12 AM
    2 kB
    Koichiro Ohba
  1. CRuby_Japanese_chars.png
    62 kB
    29/Dec/10 9:18 PM
  2. jirb_swing.jpg
    30 kB
    02/Mar/08 12:32 AM
  3. JRuby_Japanese_chars.png
    69 kB
    29/Dec/10 9:18 PM
  4. screenshot-1.jpg
    25 kB
    07/Mar/08 1:52 AM
  5. Terminal — bash — 80×24.jpg
    65 kB
    25/Dec/10 9:37 PM

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Koichiro Ohba added a comment -

I found a bug around Unicode String processing and made a patch.
I tested it with jirb_swing.
This patch will fix the reported problem on NetBeans if it uses the same JRuby library.

Show
Koichiro Ohba added a comment - I found a bug around Unicode String processing and made a patch. I tested it with jirb_swing. This patch will fix the reported problem on NetBeans if it uses the same JRuby library.
Hide
Permalink
Yoko Harada added a comment -

Thanks, Ohba-san.
As you reported, I got correct Japanese characters on a jirb_swing window after I applied your patch onto JRuby trunk version. However, I still have incorrect, ????, characters when I get jirb run on my shell. Anyway, got much better I think.

Show
Yoko Harada added a comment - Thanks, Ohba-san. As you reported, I got correct Japanese characters on a jirb_swing window after I applied your patch onto JRuby trunk version. However, I still have incorrect, ????, characters when I get jirb run on my shell. Anyway, got much better I think.
Hide
Permalink
Koichiro Ohba added a comment -

I think it's jline-0.9.93.jar which causes '????' problem on jirb.
If that's right, jline has to be fixed.

Show
Koichiro Ohba added a comment - I think it's jline-0.9.93.jar which causes '????' problem on jirb. If that's right, jline has to be fixed.
Hide
Permalink
SUZUKI Hisao added a comment -

There is a workaround: use --noreadline option.
I know it is unsatisfactory, but it works anyway...

Show
SUZUKI Hisao added a comment - There is a workaround: use --noreadline option. I know it is unsatisfactory, but it works anyway...
Hide
Permalink
Damian Steer added a comment -

99% convinced that jline is at fault here. Tried jline trunk which should have fixed this http://sourceforge.net/tracker/index.php?func=detail&aid=1623521&group_id=64033&atid=506056, but I still see the same problem. Confused.

Show
Damian Steer added a comment - 99% convinced that jline is at fault here. Tried jline trunk which should have fixed this http://sourceforge.net/tracker/index.php?func=detail&aid=1623521&group_id=64033&atid=506056 , but I still see the same problem. Confused.
Hide
Permalink
Thomas E Enebo added a comment -

Applied Koichiro's patch in commit 6155. Leaving this open to keep track of shell run jirb not displaying characters properly.

Show
Thomas E Enebo added a comment - Applied Koichiro's patch in commit 6155. Leaving this open to keep track of shell run jirb not displaying characters properly.
Hide
Permalink
Hiro Asari added a comment -

This seems to be fixed in master.

Show
Hiro Asari added a comment - This seems to be fixed in master.
Hide
Permalink
Yoko Harada added a comment - - edited

Short answer would be yes. JRuby's 1.8 mode worked like CRuby. If I dare to say, 1.9 mode works slightly different from CRuby. To show what the difference is, I attached screenshots of both CRuby and JRuby's results (CRuby_Japanese_chars.png and JRuby_Japanese_chars.png). As you can see, JRuby returned a sequence of "\xE7\xB4\x85\xE7\x8E\x89," while CRuby did the exactly the same Japanese characters. But, the returned value from "puts" was correct even in JRuby 1.9 mode, and was the same as CRuby. I'm not sure this difference matters or not. At least, this should be another issue since ?? doesn't show up anymore. So, I think this issue has been fixed.

Show
Yoko Harada added a comment - - edited Short answer would be yes. JRuby's 1.8 mode worked like CRuby. If I dare to say, 1.9 mode works slightly different from CRuby. To show what the difference is, I attached screenshots of both CRuby and JRuby's results (CRuby_Japanese_chars.png and JRuby_Japanese_chars.png). As you can see, JRuby returned a sequence of "\xE7\xB4\x85\xE7\x8E\x89," while CRuby did the exactly the same Japanese characters. But, the returned value from "puts" was correct even in JRuby 1.9 mode, and was the same as CRuby. I'm not sure this difference matters or not. At least, this should be another issue since ?? doesn't show up anymore. So, I think this issue has been fixed.
Hide
Permalink
Charles Oliver Nutter added a comment -

It seems like this might be working now?

~/projects/jruby ➔ jirb
>> x = 'ø'
=> "\303\270"
>> puts x
ø
=> nil
>> quit
Show
Charles Oliver Nutter added a comment - It seems like this might be working now? ~/projects/jruby ➔ jirb >> x = 'ø' => "\303\270" >> puts x ø => nil >> quit
Hide
Permalink
Yoko Harada added a comment -

Yes, 1.8 mode doesn't have any issue.

But, 1.9 mode still has.

 
bin/jruby --1.9 -vS irb --noreadline
jruby 1.6.0.RC1 (ruby 1.9.2 patchlevel 136) (2011-01-27 5f8bfc2) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_22) [darwin-x86_64-java]
irb(main):001:0> x='ø'
=> "\xC3\xB8"
irb(main):002:0> puts x
ø
=> nil

 
/usr/local/bin/ruby -vS irb --noreadline
ruby 1.9.2p136 (2010-12-25) [x86_64-darwin10.5.0]
/usr/local/lib/ruby/1.9.1/irb/ruby-lex.rb:1073: warning: mismatched indentations at 'end' with 'while' at 1071
irb(main):001:0> x='ø'
=> "ø"
irb(main):002:0> puts x
ø
=> nil
Show
Yoko Harada added a comment - Yes, 1.8 mode doesn't have any issue. But, 1.9 mode still has. bin/jruby --1.9 -vS irb --noreadline jruby 1.6.0.RC1 (ruby 1.9.2 patchlevel 136) (2011-01-27 5f8bfc2) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_22) [darwin-x86_64-java] irb(main):001:0> x='ø' => "\xC3\xB8" irb(main):002:0> puts x ø => nil /usr/local/bin/ruby -vS irb --noreadline ruby 1.9.2p136 (2010-12-25) [x86_64-darwin10.5.0] /usr/local/lib/ruby/1.9.1/irb/ruby-lex.rb:1073: warning: mismatched indentations at 'end' with 'while' at 1071 irb(main):001:0> x='ø' => "ø" irb(main):002:0> puts x ø => nil
Hide
Permalink
Charles Oliver Nutter added a comment -

Yoko: That issue seems to be a problem with String#inspect not showing encoded characters, rather than an issue with jirb or jline. The "puts" line in your example works correctly. Perhaps we should file the inspect problem as a separate issue?

Show
Charles Oliver Nutter added a comment - Yoko: That issue seems to be a problem with String#inspect not showing encoded characters, rather than an issue with jirb or jline. The "puts" line in your example works correctly. Perhaps we should file the inspect problem as a separate issue?
Hide
Permalink
Charles Oliver Nutter added a comment -

Calling this resolved in 1.6.1. If there are additional problems, file new bugs. If the original issue is not fixed for someone, open a new bug. Since jirb at console and in Swing works ok for me, I'm going to call this issue resolved and look for additional smaller issues for platform/encoding-specific bugs that remain.

Show
Charles Oliver Nutter added a comment - Calling this resolved in 1.6.1. If there are additional problems, file new bugs. If the original issue is not fixed for someone, open a new bug. Since jirb at console and in Swing works ok for me, I'm going to call this issue resolved and look for additional smaller issues for platform/encoding-specific bugs that remain.
Hide
Permalink
Thomas E Enebo added a comment -

ooops...jirb in console has been fine for a while. The remaining issue was that jirb_swing does not work. I can probably open that as a separate issue since it is not the same cause as this original issue.

Show
Thomas E Enebo added a comment - ooops...jirb in console has been fine for a while. The remaining issue was that jirb_swing does not work. I can probably open that as a separate issue since it is not the same cause as this original issue.
Hide
Permalink
Yoko Harada added a comment -

Charlie: That makes sense. That's why puts worked but a result of assignment was not correct. Did you file inspect issue?

Show
Yoko Harada added a comment - Charlie: That makes sense. That's why puts worked but a result of assignment was not correct. Did you file inspect issue?

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Yoko Harada
Vote (0)
Watch (4)

Dates

  • Created:
    Updated:
    Resolved: