JRuby

Unicode encoding problem in CSV.foreach

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.6.5
  • Fix Version/s: JRuby 1.7.0.pre1
  • Component/s: Ruby 1.9.2
  • Labels:
    None
  • Environment:
    Windows, Linux
  • Testcase included:
    yes
  • Number of attachments :
    1

Description

When reading CSV using "CSV.foreach" method there is a problem when CSV contains some unicode characters. As a workaround "reader = CSV.open; reader.each" must be used.
In attachment there is a simple test case containing also outputs from native Ruby 1.9 and JRuby 1.6.5 (running in ruby 1.9 mode)

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Hiro Asari added a comment -

Does the same happen when you force file encoding to be UTF-8?

For example:

set JRUBY_OPTS="--1.9 -J-Dfile.encoding=UTF-8"
Show
Hiro Asari added a comment - Does the same happen when you force file encoding to be UTF-8? For example: set JRUBY_OPTS="--1.9 -J-Dfile.encoding=UTF-8"
Hide
Permalink
Petr Bartos added a comment -

Forcing file encoding to UTF-8 provides the same results at least on my testing environment.

Show
Petr Bartos added a comment - Forcing file encoding to UTF-8 provides the same results at least on my testing environment.
Hide
Permalink
Hiro Asari added a comment -

This is fixed on master. https://gist.github.com/1581606

I don't know what the fix is. I am not inclined to find out, or to back port it to the 1.6 branch.

Show
Hiro Asari added a comment - This is fixed on master. https://gist.github.com/1581606 I don't know what the fix is. I am not inclined to find out, or to back port it to the 1.6 branch.

People

  • Assignee:
    Hiro Asari
    Reporter:
    Petr Bartos
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: