JRuby

Regexp + encoding errors in REXML

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: JRuby 1.6.7
  • Component/s: Core Classes/Modules
  • Labels:
    None
  • Number of attachments :
    1

Description

The attached script produces encoding mismatch errors from regexp. It also produces an error when it tries to eventually construct the exception, since the contents of the message are incorrectly encoded. I had to add some logging to rexml's parseexception.rb to get the actual errors to print out:

diff --git a/lib/ruby/1.9/rexml/parseexception.rb b/lib/ruby/1.9/rexml/parseexception.rb
index 0c4d55a..9a2d885 100644
--- a/lib/ruby/1.9/rexml/parseexception.rb
+++ b/lib/ruby/1.9/rexml/parseexception.rb
@@ -21,6 +21,11 @@ module REXML
       end
 
       # Get the stack trace and error message
+      puts err
+      p err.encoding
+      s = super
+      puts s
+      p s.encoding
       err << super
 
       # Add contextual information

It seems we're still having some encoding mismatch problems.

My full output (with extra logging) follows:

UTF-8
#<Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)>
org/jruby/RubyRegexp.java:1504:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/source.rb:210:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:419:in `pull_event'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:231:in `build'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:965:in `parse'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:164:in `xml_in'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:203:in `xml_in'
test.rb:13:in `(root)'
...
#<Encoding:UTF-8>
Exception parsing
#<Encoding:US-ASCII>
#<Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)>
org/jruby/RubyRegexp.java:1504:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/source.rb:210:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:419:in `pull_event'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:231:in `build'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:965:in `parse'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:164:in `xml_in'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:203:in `xml_in'
test.rb:13:in `(root)'
...
#<Encoding:UTF-8>
Exception parsing
#<Encoding:US-ASCII>
#<REXML::ParseException: #<Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)>
org/jruby/RubyRegexp.java:1504:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/source.rb:210:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:419:in `pull_event'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:231:in `build'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:965:in `parse'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:164:in `xml_in'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:203:in `xml_in'
test.rb:13:in `(root)'
...
Exception parsing
Line: 4
Position: 94
Last 80 unconsumed characters:
     <!-- Savi žemÄ&#151;s unitai -->>
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:427:in `pull_event'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:231:in `build'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:965:in `parse'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:164:in `xml_in'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:203:in `xml_in'
test.rb:13:in `(root)'
...
#<Encoding:UTF-8>
#<Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)>
org/jruby/RubyRegexp.java:1504:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/source.rb:210:in `match'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:419:in `pull_event'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:231:in `build'
/Users/headius/projects/jruby/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:965:in `parse'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:164:in `xml_in'
/Users/headius/projects/jruby/lib/ruby/gems/shared/gems/xml-simple-1.1.1/lib/xmlsimple.rb:203:in `xml_in'
test.rb:13:in `(root)'
...
Exception parsing
Line: 4
Position: 94
Last 80 unconsumed characters:
     <!-- Savi žem&#279;s unitai -->
#<Encoding:ASCII-8BIT>
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT
   concat at org/jruby/RubyString.java:2521
     to_s at /Users/headius/projects/jruby/lib/ruby/1.9/rexml/parseexception.rb:29
  message at org/jruby/RubyException.java:266

Note that the last error listed is the one from attempting to append the superclass exception's to_s result to the current error.

  1. File
    jruby-6403.rb
    31/Jan/12 11:26 AM
    0.2 kB
    Charles Oliver Nutter

Activity

Ascending order - Click to sort in descending order
Charles Oliver Nutter made changes -
Field Original Value New Value
Attachment jruby-6403.rb [ 58709 ]
Charles Oliver Nutter made changes -
Status Open [ 1 ] Resolved [ 5 ]
Assignee Charles Oliver Nutter [ headius ]
Fix Version/s JRuby 1.6.7 [ 18315 ]
Resolution Fixed [ 1 ]
Charles Oliver Nutter made changes -
Status Resolved [ 5 ] Closed [ 6 ]

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Charles Oliver Nutter
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: