JRuby

Windows line delimiter (\r\n) cause position errors

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Trivial Trivial
  • Resolution: Not A Bug
  • Affects Version/s: None
  • Fix Version/s: JRuby 1.x+
  • Component/s: Windows
  • Labels:
    None
  • Environment:
    The problem is platform independent, it occures when having \r\n as a line delimiter. Those usually appear when creating/editing files under Windows.
  • Number of attachments :
    1

Description

When working with a Ruby code file that contains \r\n as line delimiters, the position of ast-nodes at the end of a line have incorrect position information. Namely the position includes the \r which is in fact one character too much.

The position error origins in the wrappedRead() method of the LexerSource. There the a \r with following \n is jumped and the position increased, which results in a position that exceeds the actual position by one.

Proposed solution is not to skip the \r. Return it as result of wrappedRead() it then should be treated like a white space.

Activity

Ascending order - Click to sort in descending order
Charles Oliver Nutter made changes -
Field Original Value New Value
Fix Version/s JRuby 0.9.9 [ 13280 ]
Charles Oliver Nutter made changes -
Assignee Thomas E Enebo [ enebo ]
Hide
Permalink
Ola Bini added a comment -

Fix seems good, and have been applied. Thanks Thomas.

Show
Ola Bini added a comment - Fix seems good, and have been applied. Thanks Thomas.
Ola Bini made changes -
Assignee Thomas E Enebo [ enebo ] Ola Bini [ olagus ]
Status Open [ 1 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Thomas E Enebo added a comment -

This breaks unix opening windows created files on unix. This was breaking our automated tests and also is not compatible with unix version. We will have to figure out how to do this another way.

Show
Thomas E Enebo added a comment - This breaks unix opening windows created files on unix. This was breaking our automated tests and also is not compatible with unix version. We will have to figure out how to do this another way.
Thomas E Enebo made changes -
Resolution Fixed [ 1 ]
Status Resolved [ 5 ] Reopened [ 4 ]
Hide
Permalink
Thomas E Enebo added a comment -

I should probably give a better explanation as to why this breaks things. If we have a windows formatted file on unix and it has a HEREDOC delimeter; then we try and match 'EOF\n' but we will only find 'EOF\r\n' and never match.

Show
Thomas E Enebo added a comment - I should probably give a better explanation as to why this breaks things. If we have a windows formatted file on unix and it has a HEREDOC delimeter; then we try and match 'EOF\n' but we will only find 'EOF\r\n' and never match.
Ola Bini made changes -
Assignee Ola Bini [ olagus ]
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumping to 1.0...false starts mean we need more analysis, so we'll do it post 099.

Show
Charles Oliver Nutter added a comment - Bumping to 1.0...false starts mean we need more analysis, so we'll do it post 099.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 0.9.9 [ 13280 ]
Fix Version/s JRuby 1.0.0 [ 13070 ]
Hide
Permalink
Mirko Stocker added a comment -

And why not just match on 'EOF\r?\n' ? Or are there more cases like that?

Show
Mirko Stocker added a comment - And why not just match on 'EOF\r?\n' ? Or are there more cases like that?
Hide
Permalink
Thomas E Enebo added a comment -

Here is one:

a=<<'EOF'
heh heh
EOF

One idea I have about this is to finally acknowledge tht IDE consumers and interpreter have different needs. We could save a large chunk of memory by not storing the IDE position info during interp. A Factory and a flag could allow this behavior to be turned on for IDE's and off for interpretation (plus we could then share all position instances for all nodes on the same line for interp). Another continuous integration build could make sure we continue testing IDE position tests.

Show
Thomas E Enebo added a comment - Here is one: a=<<'EOF' heh heh EOF One idea I have about this is to finally acknowledge tht IDE consumers and interpreter have different needs. We could save a large chunk of memory by not storing the IDE position info during interp. A Factory and a flag could allow this behavior to be turned on for IDE's and off for interpretation (plus we could then share all position instances for all nodes on the same line for interp). Another continuous integration build could make sure we continue testing IDE position tests.
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumping to 1.0..seems like we really need an answer for this though.

Show
Charles Oliver Nutter added a comment - Bumping to 1.0..seems like we really need an answer for this though.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.0.0RC1 [ 13070 ]
Fix Version/s JRuby 1.0.0 [ 13457 ]
Hide
Permalink
Charles Oliver Nutter added a comment -

I want this in 1.0, but it's in danger of falling off. If someone wants to revisit these fixes, it will have a better chance of getting in.

Show
Charles Oliver Nutter added a comment - I want this in 1.0, but it's in danger of falling off. If someone wants to revisit these fixes, it will have a better chance of getting in.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.0.0 [ 13502 ]
Fix Version/s JRuby 1.0.0RC2 [ 13457 ]
Hide
Permalink
Charles Oliver Nutter added a comment -

We'll look at adding fixes for things like this, which is primarily an IDE bug, post 1.0.

Show
Charles Oliver Nutter added a comment - We'll look at adding fixes for things like this, which is primarily an IDE bug, post 1.0.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.x [ 13421 ]
Fix Version/s JRuby 1.0.0RC3 [ 13502 ]
Hide
Permalink
Charles Oliver Nutter added a comment -

Fix for 1.1.

Show
Charles Oliver Nutter added a comment - Fix for 1.1.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.1.0 [ 13534 ]
Fix Version/s JRuby 1.x [ 13421 ]
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.1 [ 13820 ]
Fix Version/s JRuby 1.1b1 [ 13534 ]
Hide
Permalink
Charles Oliver Nutter added a comment -

Punting issues from 1.1 RC2 to 1.1 final.

Show
Charles Oliver Nutter added a comment - Punting issues from 1.1 RC2 to 1.1 final.
Charles Oliver Nutter made changes -
Fix Version/s JRuby 1.1RC2 [ 13820 ]
Fix Version/s JRuby 1.1 [ 13990 ]
Hide
Permalink
Thomas E Enebo added a comment -

Bumping

Show
Thomas E Enebo added a comment - Bumping
Thomas E Enebo made changes -
Fix Version/s JRuby 1.1 [ 13990 ]
Fix Version/s JRuby 1.1+ [ 13421 ]
Charles Oliver Nutter made changes -
Component/s Windows [ 13600 ]
Component/s Parser [ 12229 ]
Hide
Permalink
Charles Oliver Nutter added a comment -

Since we forked off our parser for IDE use (JRubyParser) we no longer maintain column/character offset data in JRuby. As a result, this bug is no longer valid, since line terminators can't set anything off.

Show
Charles Oliver Nutter added a comment - Since we forked off our parser for IDE use (JRubyParser) we no longer maintain column/character offset data in JRuby. As a result, this bug is no longer valid, since line terminators can't set anything off.
Charles Oliver Nutter made changes -
Resolution Not A Bug [ 6 ]
Status Reopened [ 4 ] Resolved [ 5 ]
Assignee Charles Oliver Nutter [ headius ]
Charles Oliver Nutter made changes -
Status Resolved [ 5 ] Closed [ 6 ]

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Thomas Corbat
Vote (1)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: