JRuby

Encoding::UndefinedConversionError when reading from IO stream given by open-uri's Kernel#open()

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: JRuby 1.7.0.pre2
  • Fix Version/s: JRuby 1.7.0.RC1
  • Component/s: Encoding, Standard Library
  • Labels:
    None
  • Environment:
    Mac OS X 10.7.4 Java 1.7.0_05
  • Number of attachments :
    0

Description

jruby-1.7.0.preview2 :001 > require 'open-uri'
 => true 
jruby-1.7.0.preview2 :002 > open('http://www.isc.tamu.edu/~lewing/linux/sit3-shine.7.gif') { |f| f.read }
Encoding::UndefinedConversionError: Input length = 1
	from org/jruby/RubyIO.java:2892:in `read'
	from (irb):2:in `evaluate'
	from /Users/tim/.rvm/rubies/jruby-1.7.0.preview2/lib/ruby/1.9/open-uri.rb:150:in `open_uri'
	from /Users/tim/.rvm/rubies/jruby-1.7.0.preview2/lib/ruby/1.9/open-uri.rb:677:in `open'
	from /Users/tim/.rvm/rubies/jruby-1.7.0.preview2/lib/ruby/1.9/open-uri.rb:33:in `open'
	from (irb):2:in `evaluate'
	from org/jruby/RubyKernel.java:1070:in `eval'
	from org/jruby/RubyKernel.java:1395:in `loop'
	from org/jruby/RubyKernel.java:1178:in `catch'
	from org/jruby/RubyKernel.java:1178:in `catch'
	from /Users/tim/.rvm/rubies/jruby-1.7.0.preview2/bin/irb:17:in `(root)'
jruby-1.7.0.preview2 :003 >

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Tim Olsen added a comment -

This works fine in jruby 1.6.7.2

Show
Tim Olsen added a comment - This works fine in jruby 1.6.7.2
Hide
Permalink
Charles Oliver Nutter added a comment -

Thanks for the clear reproduction! I would guess this is another issue matching encoding with what the server reports; we should not be transcoding in this case since the file is obviously not text.

Show
Charles Oliver Nutter added a comment - Thanks for the clear reproduction! I would guess this is another issue matching encoding with what the server reports; we should not be transcoding in this case since the file is obviously not text.
Hide
Permalink
Charles Oliver Nutter added a comment -

A quick investigation shows that the IO used to read from the remote host has external_encoding = UTF-8 and internal_encoding = ASCII-8BIT. Unsure if we should be using ASCII for both or not.

Show
Charles Oliver Nutter added a comment - A quick investigation shows that the IO used to read from the remote host has external_encoding = UTF-8 and internal_encoding = ASCII-8BIT. Unsure if we should be using ASCII for both or not.
Hide
Permalink
Charles Oliver Nutter added a comment -

On MRI, external = ASCII-8BIT and internal = nil.

Show
Charles Oliver Nutter added a comment - On MRI, external = ASCII-8BIT and internal = nil.
Hide
Permalink
Rubem Azenha added a comment -

I'm having the same issue using multipart file upload in a Rails 3.2 app, using jruby-1.7.0.preview2

Show
Rubem Azenha added a comment - I'm having the same issue using multipart file upload in a Rails 3.2 app, using jruby-1.7.0.preview2
Hide
Permalink
Charles Oliver Nutter added a comment -

I think Tom had some idea what's wrong here. At the very least we're defaulting to improper encodings, which causes binary data to blow up. This will be fixed for RC1.

Show
Charles Oliver Nutter added a comment - I think Tom had some idea what's wrong here. At the very least we're defaulting to improper encodings, which causes binary data to blow up. This will be fixed for RC1.
Hide
Permalink
Tim Olsen added a comment -

I'm also getting this error when trying to drop a mongodb database using Mongoid 3.0.5/moped 1.2.1:

Loading development environment (Rails 3.2.8)
jruby-1.7.0.preview2 :001 > ::Mongoid::Sessions.default.drop
::Mongoid::Sessions.default.drop
Encoding::UndefinedConversionError: Input length = 1
	from org/jruby/RubyIO.java:1400:in `write'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:229:in `write'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:235:in `handle_socket_errors'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:229:in `write'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:146:in `write'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:169:in `with_connection'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:145:in `write'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:517:in `flush'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:123:in `ensure_connected'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:516:in `flush'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:531:in `logging'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:515:in `flush'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:504:in `process'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:70:in `command'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:366:in `refresh'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/cluster.rb:112:in `refresh'
... 4 levels...
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session/context.rb:108:in `with_node'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session/context.rb:50:in `command'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/database.rb:89:in `drop'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session.rb:305:in `with'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/database.rb:88:in `drop'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session.rb:130:in `drop'
	from (irb):1:in `evaluate'
	from org/jruby/RubyKernel.java:1070:in `eval'
	from org/jruby/RubyKernel.java:1395:in `loop'
	from org/jruby/RubyKernel.java:1178:in `catch'
	from org/jruby/RubyKernel.java:1178:in `catch'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start'
	from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands.rb:41:in `(root)'
	from org/jruby/RubyKernel.java:1024:in `require'
	from script/rails:6:in `(root)'jruby-1.7.0.preview2 :002 >
Show
Tim Olsen added a comment - I'm also getting this error when trying to drop a mongodb database using Mongoid 3.0.5/moped 1.2.1: Loading development environment (Rails 3.2.8) jruby-1.7.0.preview2 :001 > ::Mongoid::Sessions.default.drop ::Mongoid::Sessions.default.drop Encoding::UndefinedConversionError: Input length = 1 from org/jruby/RubyIO.java:1400:in `write' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:229:in `write' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:235:in `handle_socket_errors' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:229:in `write' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:146:in `write' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:169:in `with_connection' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/connection.rb:145:in `write' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:517:in `flush' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:123:in `ensure_connected' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:516:in `flush' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:531:in `logging' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:515:in `flush' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:504:in `process' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:70:in `command' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/node.rb:366:in `refresh' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/cluster.rb:112:in `refresh' ... 4 levels... from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session/context.rb:108:in `with_node' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session/context.rb:50:in `command' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/database.rb:89:in `drop' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session.rb:305:in `with' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/database.rb:88:in `drop' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/moped-1.2.1/lib/moped/session.rb:130:in `drop' from (irb):1:in `evaluate' from org/jruby/RubyKernel.java:1070:in `eval' from org/jruby/RubyKernel.java:1395:in `loop' from org/jruby/RubyKernel.java:1178:in `catch' from org/jruby/RubyKernel.java:1178:in `catch' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start' from /Users/tim/.rvm/gems/jruby-1.7.0.preview2/gems/railties-3.2.8/lib/rails/commands.rb:41:in `(root)' from org/jruby/RubyKernel.java:1024:in `require' from script/rails:6:in `(root)'jruby-1.7.0.preview2 :002 >
Hide
Permalink
Charles Oliver Nutter added a comment -

Interestingly, this appears to work ok on master as of today. Did we nail it some time after pre2 release? I don't recall fixing it myself...

system ~/projects/jruby $ jruby -v
jruby 1.7.0.preview2 (1.9.3p203) 2012-08-29 b16f8c3 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_06-b24 [darwin-x86_64]

system ~/projects/jruby $ jruby -ropen-uri -e "open('http://www.isc.tamu.edu/~lewing/linux/sit3-shine.7.gif') { |f| f.read }"
Show
Charles Oliver Nutter added a comment - Interestingly, this appears to work ok on master as of today. Did we nail it some time after pre2 release? I don't recall fixing it myself... system ~/projects/jruby $ jruby -v jruby 1.7.0.preview2 (1.9.3p203) 2012-08-29 b16f8c3 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_06-b24 [darwin-x86_64] system ~/projects/jruby $ jruby -ropen-uri -e "open('http://www.isc.tamu.edu/~lewing/linux/sit3-shine.7.gif') { |f| f.read }"
Hide
Permalink
Tim Olsen added a comment -

open() is now working for me with jruby-head, but ::Mongoid::Sessions.default.drop() still fails

Show
Tim Olsen added a comment - open() is now working for me with jruby-head, but ::Mongoid::Sessions.default.drop() still fails
Hide
Permalink
Tim Olsen added a comment -

I'm using:

$ jruby -v
jruby 1.7.0.preview2 (1.9.3p203) 2012-08-30 b16f8c3 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_05-b06 [darwin-x86_64]

Show
Tim Olsen added a comment - I'm using: $ jruby -v jruby 1.7.0.preview2 (1.9.3p203) 2012-08-30 b16f8c3 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_05-b06 [darwin-x86_64]
Hide
Permalink
Charles Oliver Nutter added a comment -

I would consider that a separate issue. Perhaps you can open a separate bug for it and (hopefully) provide more info + reproduction?

Show
Charles Oliver Nutter added a comment - I would consider that a separate issue. Perhaps you can open a separate bug for it and (hopefully) provide more info + reproduction?
Hide
Permalink
Tim Olsen added a comment -

Sure. I'll create a new issue.

Show
Tim Olsen added a comment - Sure. I'll create a new issue.
Hide
Permalink
Charles Oliver Nutter added a comment -

Calling this fixed, since I and another reporter can no longer reproduce on master.

Show
Charles Oliver Nutter added a comment - Calling this fixed, since I and another reporter can no longer reproduce on master.

People

  • Assignee:
    Charles Oliver Nutter
    Reporter:
    Tim Olsen
Vote (2)
Watch (6)

Dates

  • Created:
    Updated:
    Resolved: