JRuby

IO CRLF compatibility with cruby on Windows

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Fixed
  • Affects Version/s: JRuby 0.9.0, JRuby 0.9.1, JRuby 0.9.2, JRuby 0.9.8, JRuby 0.9.9, JRuby 1.0.0RC1, JRuby 1.0.0RC2
  • Fix Version/s: JRuby 1.5
  • Component/s: Windows
  • Labels:
    None
  • Environment:
    Windows
  • Testcase included:
    yes
  • Number of attachments :
    5

Description

IO puts, each_byte, getc behave differently on windows w.r.t. CRLF handling compared to ruby 1.8.4

Attached test passes under ruby 1.8.4 on windows XP, but fails under JRuby in several places.

Attachments

  1. Text File
    newlineFix-r6068.patch
    28/Feb/08 10:31 PM
    19 kB
    Koichiro Ohba
  2. Text File
    newlinesFix.patch
    11/Aug/06 6:22 AM
    4 kB
    Ola Bini
  3. Text File
    newlinesFix-koichiro.patch
    26/May/07 8:21 AM
    5 kB
    Koichiro Ohba
  4. Text File
    RubyIO.patch
    12/Jan/08 2:14 PM
    1 kB
    Tiziano Merzi
  5. File
    testIOWindows.rb
    05/Aug/06 7:12 PM
    2 kB
    Tim Azzopardi

Issue Links

This issue is related to:
JRUBY-2140 IO.readlines returns native line endings Major Closed
This issue is depended upon by:
JRUBY-4253 JRuby defaults to "\r\n" newlines on Windows, where MRI 1.8 and 1.9 use just "\n" Minor Closed

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Ola Bini added a comment -

A fix for all issues in the test case.

Show
Ola Bini added a comment - A fix for all issues in the test case.
Hide
Permalink
Charles Oliver Nutter added a comment -

This needs to be revisited, and I don't have Windows handy at the moment. If it's easily fixable, we should have it in for 0.9.9.

Show
Charles Oliver Nutter added a comment - This needs to be revisited, and I don't have Windows handy at the moment. If it's easily fixable, we should have it in for 0.9.9.
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumping to 1.0...CRLF is a pain to get working right, and we've had a couple false starts already. I'd prefer we don't monkey with it under pressure.

Show
Charles Oliver Nutter added a comment - Bumping to 1.0...CRLF is a pain to get working right, and we've had a couple false starts already. I'd prefer we don't monkey with it under pressure.
Hide
Permalink
Charles Oliver Nutter added a comment -

In danger of falling off 1.0.

Show
Charles Oliver Nutter added a comment - In danger of falling off 1.0.
Hide
Permalink
Charles Oliver Nutter added a comment -

The fix is reasonable, but it looks like it could add a lot of overhead to sysread:int, which could make our IO performance even worse than it is. Is there another way to accomplish this?

Show
Charles Oliver Nutter added a comment - The fix is reasonable, but it looks like it could add a lot of overhead to sysread:int, which could make our IO performance even worse than it is. Is there another way to accomplish this?
Hide
Permalink
Koichiro Ohba added a comment -

I thought it's very difficult to avoid adding overhead to sysread:int
for Windows, so I changed Ola Bini's patch to minimalize the effect
on other environments. And I updated IOHandlerSeekable.java(rev. 3741)
to sync up with the trunk.

Show
Koichiro Ohba added a comment - I thought it's very difficult to avoid adding overhead to sysread:int for Windows, so I changed Ola Bini's patch to minimalize the effect on other environments. And I updated IOHandlerSeekable.java(rev. 3741) to sync up with the trunk.
Hide
Permalink
Charles Oliver Nutter added a comment -

This is going to need some special-case behavior to, essentially, act completely different on Windows. Since we really are targeting "java" as a platform, this is going to come post 1.0.

Show
Charles Oliver Nutter added a comment - This is going to need some special-case behavior to, essentially, act completely different on Windows. Since we really are targeting "java" as a platform, this is going to come post 1.0.
Hide
Permalink
Charles Oliver Nutter added a comment -

Fix or close for 1.1.

Show
Charles Oliver Nutter added a comment - Fix or close for 1.1.
Hide
Permalink
Charles Oliver Nutter added a comment -

Assigning Windows-related bug to Nick. Nick, looks like the resubmission by Koichiro could be a good compromise to get this working.

Show
Charles Oliver Nutter added a comment - Assigning Windows-related bug to Nick. Nick, looks like the resubmission by Koichiro could be a good compromise to get this working.
Hide
Permalink
Koichiro Ohba added a comment -

Long Time!
I was locked in writing introductory articles about JRuby and Ruby. (BTW, I failed the Ruby Certification!
Have you got that JavaExpert magazine?

Now, I'm going to look into that problem again

Show
Koichiro Ohba added a comment - Long Time! I was locked in writing introductory articles about JRuby and Ruby. (BTW, I failed the Ruby Certification! Have you got that JavaExpert magazine? Now, I'm going to look into that problem again
Hide
Permalink
Thomas E Enebo added a comment -

I would like anything which makes window behave differently a flag. Then Java can still be a platform and behave consistently across host OS's and we can flip a switch to make it behave better on Windows. It will also demarcate this differing behavior explicitly in the source code.

Show
Thomas E Enebo added a comment - I would like anything which makes window behave differently a flag. Then Java can still be a platform and behave consistently across host OS's and we can flip a switch to make it behave better on Windows. It will also demarcate this differing behavior explicitly in the source code.
Hide
Permalink
Tiziano Merzi added a comment -

Hi,

with the patch RubyIO.patch it works in windows

why don't apply the patch?

tiziano

Show
Tiziano Merzi added a comment - Hi, with the patch RubyIO.patch it works in windows why don't apply the patch? tiziano
Hide
Permalink
Charles Oliver Nutter added a comment -

Punting issues from 1.1 RC2 to 1.1 final.

Show
Charles Oliver Nutter added a comment - Punting issues from 1.1 RC2 to 1.1 final.
Hide
Permalink
Koichiro Ohba added a comment -

Sorry to be late. I made a patch for rev6067.
To change the line separator style, set Java property "jruby.line.separator.style=UNIX|WINDOWS|MACOS".
How is it?
This patch also clears up JRUBY-2140.

There are still few problems on Windows test cases.
For example, the following command ends with 2 failure, at line 36 and 68.

jruby -J-Djruby.line.separator.style=windows testIOWindows.rb

The cause of this seems to be another bug, I'll post an issue later.

Show
Koichiro Ohba added a comment - Sorry to be late. I made a patch for rev6067. To change the line separator style, set Java property "jruby.line.separator.style=UNIX|WINDOWS|MACOS". How is it? This patch also clears up JRUBY-2140. There are still few problems on Windows test cases. For example, the following command ends with 2 failure, at line 36 and 68. 
jruby -J-Djruby.line.separator.style=windows testIOWindows.rb
The cause of this seems to be another bug, I'll post an issue later.
Hide
Permalink
Thomas E Enebo added a comment -

Bumping to 1.1+ this should be enabled under new proposed command-line option in JRUBY-2283.

Show
Thomas E Enebo added a comment - Bumping to 1.1+ this should be enabled under new proposed command-line option in JRUBY-2283.
Hide
Permalink
Simon Rozet added a comment -

This issue affects Sinatra with jRuby on Windows. See https://sinatra.lighthouseapp.com/projects/9779/tickets/284-use_in_file_templates-misses-__end__-marker-with-jruby-on-win32 for details

Show
Simon Rozet added a comment - This issue affects Sinatra with jRuby on Windows. See https://sinatra.lighthouseapp.com/projects/9779/tickets/284-use_in_file_templates-misses-__end__-marker-with-jruby-on-win32 for details
Hide
Permalink
Nick Sieger added a comment -

Still an issue. The entire ChannelStream needs to be revamped to deal w/ line separators.The patch here doesn't solve everything. Targeting 1.5 for now.

Show
Nick Sieger added a comment - Still an issue. The entire ChannelStream needs to be revamped to deal w/ line separators.The patch here doesn't solve everything. Targeting 1.5 for now.
Hide
Permalink
Vladimir Sizikov added a comment -

Also, this causes 2 test failures in rubicon's test_io.rb. Note: These test cases are currently excluded, so unexclude first to see the failures.

D:\work\jruby-dev\jruby>jruby test/rubicon/test_io.rb
Loaded suite test/rubicon/test_io
Started
....................F...............F.........
Finished in 0.614 seconds.

  1) Failure:
test_pos=(TestIO)
    [test/rubicon/test_io.rb:721:in `test_pos='
     test/rubicon/test_io.rb:718:in `each'
     test/rubicon/test_io.rb:718:in `test_pos='
     test/rubicon/test_io.rb:708:in `open'
     test/rubicon/test_io.rb:708:in `test_pos=']:
<5> expected but was
<0>.

  2) Failure:
test_seek(TestIO)
    [test/rubicon/test_io.rb:932:in `test_seek'
     test/rubicon/test_io.rb:929:in `each'
     test/rubicon/test_io.rb:929:in `test_seek'
     test/rubicon/test_io.rb:919:in `open'
     test/rubicon/test_io.rb:919:in `test_seek']:
<5> expected but was
<0>.
Show
Vladimir Sizikov added a comment - Also, this causes 2 test failures in rubicon's test_io.rb. Note: These test cases are currently excluded, so unexclude first to see the failures. 
D:\work\jruby-dev\jruby>jruby test/rubicon/test_io.rb
Loaded suite test/rubicon/test_io
Started
....................F...............F.........
Finished in 0.614 seconds.

  1) Failure:
test_pos=(TestIO)
    [test/rubicon/test_io.rb:721:in `test_pos='
     test/rubicon/test_io.rb:718:in `each'
     test/rubicon/test_io.rb:718:in `test_pos='
     test/rubicon/test_io.rb:708:in `open'
     test/rubicon/test_io.rb:708:in `test_pos=']:
<5> expected but was
<0>.

  2) Failure:
test_seek(TestIO)
    [test/rubicon/test_io.rb:932:in `test_seek'
     test/rubicon/test_io.rb:929:in `each'
     test/rubicon/test_io.rb:929:in `test_seek'
     test/rubicon/test_io.rb:919:in `open'
     test/rubicon/test_io.rb:919:in `test_seek']:
<5> expected but was
<0>.
Hide
Permalink
Nick Sieger added a comment -

Fixed in 9d57dbc. Merry Christmas!

Show
Nick Sieger added a comment - Fixed in 9d57dbc. Merry Christmas!

People

Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.