jirb outputs ???? instead of Japanese characters

I found a bug around Unicode String processing and made a patch.
I tested it with jirb_swing.
This patch will fix the reported problem on NetBeans if it uses the same JRuby library.

Thanks, Ohba-san.
As you reported, I got correct Japanese characters on a jirb_swing window after I applied your patch onto JRuby trunk version. However, I still have incorrect, ????, characters when I get jirb run on my shell. Anyway, got much better I think.

I think it's jline-0.9.93.jar which causes '????' problem on jirb.
If that's right, jline has to be fixed.

99% convinced that jline is at fault here. Tried jline trunk which should have fixed this http://sourceforge.net/tracker/index.php?func=detail&aid=1623521&group_id=64033&atid=506056, but I still see the same problem. Confused.

Applied Koichiro's patch in commit 6155. Leaving this open to keep track of shell run jirb not displaying characters properly.

This seems to be fixed in master.

Short answer would be yes. JRuby's 1.8 mode worked like CRuby. If I dare to say, 1.9 mode works slightly different from CRuby. To show what the difference is, I attached screenshots of both CRuby and JRuby's results (CRuby_Japanese_chars.png and JRuby_Japanese_chars.png). As you can see, JRuby returned a sequence of "\xE7\xB4\x85\xE7\x8E\x89," while CRuby did the exactly the same Japanese characters. But, the returned value from "puts" was correct even in JRuby 1.9 mode, and was the same as CRuby. I'm not sure this difference matters or not. At least, this should be another issue since ?? doesn't show up anymore. So, I think this issue has been fixed.

It seems like this might be working now?

~/projects/jruby ➔ jirb
>> x = 'ø'
=> "\303\270"
>> puts x
ø
=> nil
>> quit

Yes, 1.8 mode doesn't have any issue.

But, 1.9 mode still has.

 
bin/jruby --1.9 -vS irb --noreadline
jruby 1.6.0.RC1 (ruby 1.9.2 patchlevel 136) (2011-01-27 5f8bfc2) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_22) [darwin-x86_64-java]
irb(main):001:0> x='ø'
=> "\xC3\xB8"
irb(main):002:0> puts x
ø
=> nil

 
/usr/local/bin/ruby -vS irb --noreadline
ruby 1.9.2p136 (2010-12-25) [x86_64-darwin10.5.0]
/usr/local/lib/ruby/1.9.1/irb/ruby-lex.rb:1073: warning: mismatched indentations at 'end' with 'while' at 1071
irb(main):001:0> x='ø'
=> "ø"
irb(main):002:0> puts x
ø
=> nil

Yoko: That issue seems to be a problem with String#inspect not showing encoded characters, rather than an issue with jirb or jline. The "puts" line in your example works correctly. Perhaps we should file the inspect problem as a separate issue?

Calling this resolved in 1.6.1. If there are additional problems, file new bugs. If the original issue is not fixed for someone, open a new bug. Since jirb at console and in Swing works ok for me, I'm going to call this issue resolved and look for additional smaller issues for platform/encoding-specific bugs that remain.

ooops...jirb in console has been fine for a while. The remaining issue was that jirb_swing does not work. I can probably open that as a separate issue since it is not the same cause as this original issue.

Charlie: That makes sense. That's why puts worked but a result of assignment was not correct. Did you file inspect issue?