JRuby

Windows line delimiter (\r\n) cause position errors

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Trivial Trivial
  • Resolution: Not A Bug
  • Affects Version/s: None
  • Fix Version/s: JRuby 1.x+
  • Component/s: Windows
  • Labels:
    None
  • Environment:
    The problem is platform independent, it occures when having \r\n as a line delimiter. Those usually appear when creating/editing files under Windows.
  • Number of attachments :
    1

Description

When working with a Ruby code file that contains \r\n as line delimiters, the position of ast-nodes at the end of a line have incorrect position information. Namely the position includes the \r which is in fact one character too much.

The position error origins in the wrappedRead() method of the LexerSource. There the a \r with following \n is jumped and the position increased, which results in a position that exceeds the actual position by one.

Proposed solution is not to skip the \r. Return it as result of wrappedRead() it then should be treated like a white space.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Ola Bini added a comment -

Fix seems good, and have been applied. Thanks Thomas.

Show
Ola Bini added a comment - Fix seems good, and have been applied. Thanks Thomas.
Hide
Permalink
Thomas E Enebo added a comment -

This breaks unix opening windows created files on unix. This was breaking our automated tests and also is not compatible with unix version. We will have to figure out how to do this another way.

Show
Thomas E Enebo added a comment - This breaks unix opening windows created files on unix. This was breaking our automated tests and also is not compatible with unix version. We will have to figure out how to do this another way.
Hide
Permalink
Thomas E Enebo added a comment -

I should probably give a better explanation as to why this breaks things. If we have a windows formatted file on unix and it has a HEREDOC delimeter; then we try and match 'EOF\n' but we will only find 'EOF\r\n' and never match.

Show
Thomas E Enebo added a comment - I should probably give a better explanation as to why this breaks things. If we have a windows formatted file on unix and it has a HEREDOC delimeter; then we try and match 'EOF\n' but we will only find 'EOF\r\n' and never match.
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumping to 1.0...false starts mean we need more analysis, so we'll do it post 099.

Show
Charles Oliver Nutter added a comment - Bumping to 1.0...false starts mean we need more analysis, so we'll do it post 099.
Hide
Permalink
Mirko Stocker added a comment -

And why not just match on 'EOF\r?\n' ? Or are there more cases like that?

Show
Mirko Stocker added a comment - And why not just match on 'EOF\r?\n' ? Or are there more cases like that?
Hide
Permalink
Thomas E Enebo added a comment -

Here is one:

a=<<'EOF'
heh heh
EOF

One idea I have about this is to finally acknowledge tht IDE consumers and interpreter have different needs. We could save a large chunk of memory by not storing the IDE position info during interp. A Factory and a flag could allow this behavior to be turned on for IDE's and off for interpretation (plus we could then share all position instances for all nodes on the same line for interp). Another continuous integration build could make sure we continue testing IDE position tests.

Show
Thomas E Enebo added a comment - Here is one: a=<<'EOF' heh heh EOF One idea I have about this is to finally acknowledge tht IDE consumers and interpreter have different needs. We could save a large chunk of memory by not storing the IDE position info during interp. A Factory and a flag could allow this behavior to be turned on for IDE's and off for interpretation (plus we could then share all position instances for all nodes on the same line for interp). Another continuous integration build could make sure we continue testing IDE position tests.
Hide
Permalink
Charles Oliver Nutter added a comment -

Bumping to 1.0..seems like we really need an answer for this though.

Show
Charles Oliver Nutter added a comment - Bumping to 1.0..seems like we really need an answer for this though.
Hide
Permalink
Charles Oliver Nutter added a comment -

I want this in 1.0, but it's in danger of falling off. If someone wants to revisit these fixes, it will have a better chance of getting in.

Show
Charles Oliver Nutter added a comment - I want this in 1.0, but it's in danger of falling off. If someone wants to revisit these fixes, it will have a better chance of getting in.
Hide
Permalink
Charles Oliver Nutter added a comment -

We'll look at adding fixes for things like this, which is primarily an IDE bug, post 1.0.

Show
Charles Oliver Nutter added a comment - We'll look at adding fixes for things like this, which is primarily an IDE bug, post 1.0.
Hide
Permalink
Charles Oliver Nutter added a comment -

Fix for 1.1.

Show
Charles Oliver Nutter added a comment - Fix for 1.1.
Hide
Permalink
Charles Oliver Nutter added a comment -

Punting issues from 1.1 RC2 to 1.1 final.

Show
Charles Oliver Nutter added a comment - Punting issues from 1.1 RC2 to 1.1 final.
Hide
Permalink
Thomas E Enebo added a comment -

Bumping

Show
Thomas E Enebo added a comment - Bumping
Hide
Permalink
Charles Oliver Nutter added a comment -

Since we forked off our parser for IDE use (JRubyParser) we no longer maintain column/character offset data in JRuby. As a result, this bug is no longer valid, since line terminators can't set anything off.

Show
Charles Oliver Nutter added a comment - Since we forked off our parser for IDE use (JRubyParser) we no longer maintain column/character offset data in JRuby. As a result, this bug is no longer valid, since line terminators can't set anything off.

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Thomas Corbat
Vote (1)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: