Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.6.5
    • Fix Version/s: JRuby 1.7.0.pre1
    • Component/s: Standard Library
    • Labels:
      None
    • Number of attachments :
      1

      Description

      I open this issue mostly for discussion since I suspect that there is not bug, but it represents a real world problem.

      If you use JRuby in 1.8 mode and dump a string containing a TAB character, you get something like this:

      macbeth:~ uwe$ jruby --1.8 -ryaml -S jirb
      >> YAML.dump("36L\tDIESEL")
      => "--- 36L\tDIESEL\n"
      >> YAML.load "--- 36L\tDIESEL\n"
      => "36L\tDIESEL"
      >> 
      

      Notice the TAB character is embedded verbatim in the YAML stream. No quoting, no escaping. YAML.load works fine with this stream.

      Now the same for --1.9 mode:

      macbeth:~ uwe$ jruby --1.9 -ryaml -S jirb
      irb(main):001:0> YAML.dump("36L\tDIESEL")
      => "--- ! \"36L\\tDIESEL\"\n...\n"
      irb(main):002:0> YAML.load "--- ! \"36L\\tDIESEL\"\n...\n"
      => "36L\tDIESEL"
      

      Notice that now the TAB character is both quoted and escaped. YAML.load work fine on this stream as well.

      But what happens if we try to parse the stream generated by JRuby 1.8 with JRuby 1.9?

      macbeth:~ uwe$ jruby --1.9 -ryaml -S jirb
      irb(main):001:0> YAML.load "--- 36L\tDIESEL\n"
      ArgumentError: syntax error in "<reader>", line 1, column 8:
          --- 36L	DIESEL
                 ^
      	from org/jruby/ext/psych/PsychParser.java:273:in `parse'
      	from /Library/Frameworks/JRuby.framework/Versions/1.6.5/lib/ruby/1.9/psych.rb:148:in `parse_stream'
      	from /Library/Frameworks/JRuby.framework/Versions/1.6.5/lib/ruby/1.9/psych.rb:119:in `parse'
      	from /Library/Frameworks/JRuby.framework/Versions/1.6.5/lib/ruby/1.9/psych.rb:106:in `load'
      	from (irb):3:in `evaluate'
      	from org/jruby/RubyKernel.java:1093:in `eval'
      	from org/jruby/RubyKernel.java:1420:in `loop'
      	from org/jruby/RubyKernel.java:1206:in `catch'
      	from org/jruby/RubyKernel.java:1206:in `catch'
      	from /Library/Frameworks/JRuby.framework/Versions/1.6.5/bin/jirb:13:in `(root)'
      

      It blows up. We are seeing this in production, so we need a solution to this problem. On the receiver (consumer) side of the exchange, we use event based parsing with Psych::Parser and Psych::Handler so, as far as I know, reverting to syck is not an option.

      Any help on this is greatly appreciated, and time is of the essence. We are currently just avoiding the problem by disallowing offending input, but we need a permanent solution for this:

      How can we safely generate YAML with JRuby 1.8 and parse it with JRuby 1.9 using event based parsing?

      P.S. On the consumer side, we are actually running JRuby in 1.8 mode, but we pull in Psych explicitly. I mention this only to open up more possibilities for a solution.

        Activity

        Hide
        Uwe Kubosch added a comment -

        The bug itself is major in that a YAML stream with a verbatim tab character in a string scalar causes a syntax error and a runtime exception.

        For our project (fleet control for airport busses and fuel trucks) it is critical since we have no workaround for this.

        The use of the snapshot is important for us, but we can build our own version of JRuby with the snapshot included. However, we would naturally strongly prefer to use the release branch and, in time, an official release.

        The use of the snapshot would be temporary. SnakeYAML has announced that they plan to release version 1.10 in february. I can add the final release to JRuby then.

        Show
        Uwe Kubosch added a comment - The bug itself is major in that a YAML stream with a verbatim tab character in a string scalar causes a syntax error and a runtime exception. For our project (fleet control for airport busses and fuel trucks) it is critical since we have no workaround for this. The use of the snapshot is important for us, but we can build our own version of JRuby with the snapshot included. However, we would naturally strongly prefer to use the release branch and, in time, an official release. The use of the snapshot would be temporary. SnakeYAML has announced that they plan to release version 1.10 in february. I can add the final release to JRuby then.
        Hide
        Thomas E Enebo added a comment -

        I think the main concern is other companies (like RedHat) do maven builds to generate their products. Depending on snapshot builds is really icky for maven folks. I recommend rolling your own build for your case and we will get the real release of snakeYAML when they put it out for our next release.

        Show
        Thomas E Enebo added a comment - I think the main concern is other companies (like RedHat) do maven builds to generate their products. Depending on snapshot builds is really icky for maven folks. I recommend rolling your own build for your case and we will get the real release of snakeYAML when they put it out for our next release.
        Hide
        Uwe Kubosch added a comment -

        OK. I'll add the final SnakeYAML 1.10 jar when it is released.

        When are we planning to release JRuby 1.7.0?

        Show
        Uwe Kubosch added a comment - OK. I'll add the final SnakeYAML 1.10 jar when it is released. When are we planning to release JRuby 1.7.0?
        Hide
        Charles Oliver Nutter added a comment -

        JRuby 1.7 is at least a month away, after we release 1.6.6. Maybe more, depending on how much we want to get in, but we don't want to go too long.

        Show
        Charles Oliver Nutter added a comment - JRuby 1.7 is at least a month away, after we release 1.6.6. Maybe more, depending on how much we want to get in, but we don't want to go too long.
        Hide
        Uwe Kubosch added a comment -

        I just pushed the final version of SnakeYAML 1.10.

        Show
        Uwe Kubosch added a comment - I just pushed the final version of SnakeYAML 1.10.

          People

          • Assignee:
            Thomas E Enebo
            Reporter:
            Uwe Kubosch
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: