RubyGems tests are not green; fix remaining failures/errors

Not making 1.4...

Failures in Ruby libraries like RubyGems are often easy to track down. For Intro purposes, this could also just be a matter of parceling out the failures that remain today into separate bugs, so they're more approachable.

The 14 failures / 259 errors (on rubygems trunk, jruby master) that I'm seeing are almost all location-related, but there are also some that require OpenSSL.
Installing the jruby-openssl gem leads to a Java exception.

====
A few of these:
====
1) Failure:
test_self_set_paths_nonexistent_home(TestGem) [/Users/colin/oss/rubygems/trunk/test/test_gem.rb:549]:
Expected ["/private/var/folders/E1/E1wP5WJaGmKy3SrhKxZQMk+++TM/Tmp/test_rubygems_17933/other",
"/private/var/folders/E1/E1wP5WJaGmKy3SrhKxZQMk+++TM/Tmp/test_rubygems_17933/gemhome"], not ["/private/var/folders/E1/E1wP5WJaGmKy3SrhKxZQMk+++TM/-Tmp-/test_rubygems_17933/gemhome"].

====
Lots of these (nearly all of the errors are like this, really):
====
2) Error:
test_self_all_load_paths(TestGem):
Errno::ENOENT: No such file or directory - No such file or directory - a-1.gem or /private/var/folders/E1/E1wP5WJaGmKy3SrhKxZQMk+++TM/Tmp/test_rubygems_17933/gemhome/cache/a-1.gem
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/fileutils.rb:505:in `mv'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/fileutils.rb:1395:in `fu_each_src_dest'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/fileutils.rb:1411:in `fu_each_src_dest0'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/fileutils.rb:1393:in `fu_each_src_dest'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/fileutils.rb:494:in `mv'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:278:in `util_build_gem'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:266:in `chdir'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:266:in `util_build_gem'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:363:in `util_make_gems'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:362:in `each'
/Users/colin/oss/rubygems/trunk/test/gemutilities.rb:362:in `util_make_gems'
/Users/colin/oss/rubygems/trunk/test/test_gem.rb:22:in `test_self_all_load_paths'

====
A few of these:
====
4) Failure:
test_self_ensure_gem_directories(TestGem) [/Users/colin/oss/rubygems/trunk/test/test_gem.rb:197]:
Failed assertion, no message given.

=====
A few of these:
=====
17) Error:
test_execute_build(TestGemCommandsCertCommand):
Gem::Exception: SSL is not installed on this system
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/gem_openssl.rb:33:in `ensure_ssl_available'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/security.rb:673:in `build_self_signed_cert'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/commands/cert_command.rb:52:in `initialize'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/command.rb:420:in `call'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/command.rb:420:in `configure_options'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1267:in `call'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1267:in `parse_in_order'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1254:in `catch'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1254:in `parse_in_order'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1248:in `order!'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1339:in `permute!'
file:/Users/colin/oss/jruby/lib/jruby-complete.jar!/META-INF/jruby.home/lib/ruby/1.8/optparse.rb:1360:in `parse!'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/command.rb:327:in `handle_options'
/Users/colin/oss/rubygems/trunk/test/test_gem_commands_cert_command.rb:49:in `test_execute_build'
/Users/colin/oss/rubygems/trunk/test/test_gem_commands_cert_command.rb:48:in `chdir'
/Users/colin/oss/rubygems/trunk/test/test_gem_commands_cert_command.rb:48:in `test_execute_build'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/user_interaction.rb:40:in `use_ui'
/Users/colin/oss/rubygems/trunk/test/../lib/rubygems/user_interaction.rb:63:in `use_ui'
/Users/colin/oss/rubygems/trunk/test/test_gem_commands_cert_command.rb:47:in `test_execute_build'

Patch fixes several path issues, reducing problems to 15 failures, 18 errors when running out of jruby-complete.jar - probably even a few less when running out of installed JRuby.

Very nice...with your patch my test run improves to 12F/18E.

I tried running tests with OpenSSL installed and ran into a jruby-openssl bug (JRUBY-4444), which prevented the suite from running to completion. Getting that fixed would move things forward.

There are also a number of failures related to our lack of native extension support. We may want to get the RubyGems folks to skip those tests on JRuby (and other impls) since they'll never pass for us.

Colin: See what you can do with the remaining failures that are not native ext or OpenSSL-related.

Pushed Colin's fixes in 927a5ba.

Patch 0002 attached makes Marshal.load throw the same exception as MRI (ArgumentError: marshal data too short) when it tries to load a String that's a truncated marshalled object - if that makes sense. New test case added highlighting MRI vs. JRuby difference. JRuby threw an EOFError.

Reduces my test run error count to something like 14-15 failures, 16-17 errors (I've seen it vary a few times, not sure why yet).

Will keep working on tracking down other errors/failures (besides the obvious native problems) as I have time.

Colin: The new patch looks good, but represents a behavioral difference we probably should put in RubySpec rather than in JRuby's tests. It's not hard, but let me know if you need help.

Bug update for now: 0001 patch has been applied, 0002 has a good fix but the test should be moved to RubySpec. Running tests exposes many failures due to missing OpenSSL, but jruby-openssl NPEs.

Patch 0003 removes the test (which is now submitted to RubySpec as issue #190: http://rubyspec.org/projects/rubyspec/issues)

I have pushed your most recent patch in 4d0f765, and thought I'd post an updated list of failures here. Thanks for the additional fixes! We're down to 11 failures, 10 errors.

I've removed the backtraces; clone rubygems and run rake test yourself to get more information. Comments follow each section.

  1) Failure:
test_self_path_duplicate(TestGem) [./test/test_gem.rb:380]:
Expected ["/private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/a",
 "/private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/b",
 "/private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/gemhome"], not ["/private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/gemhome"].

The first 8 failures all look like this first one. Some oddity with temp directories?

  9) Error:
test_execute_build(TestGemCommandsCertCommand):
Errno::ENOENT: No such file or directory - No such file or directory - gem-private_key.pem

I'm not sure about this one. OpenSSL issue?

 10) Error:
test_execute_prerelease(TestGemCommandsUninstallCommand):
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

We will probably need to submit a patch to rubygems to not run the native ext tests under JRuby. Failure 15 is the same thing.

 11) Error:
test_install_domain_both_no_network(TestGemDependencyInstaller):
Gem::RemoteFetcher::FetchError: no data for http://gems.example.com/gems/a-1.gem (http://gems.example.com/gems/a-1.gem)

Failures 11-14 look like this. Is it actually trying to fetch from example.com? Or are we supposed to have a dummy host set up or something?

 16) Failure:
test_class_build_extconf_fail(TestGemExtExtConfBuilder) [./test/test_gem_ext_ext_conf_builder.rb:73]:
Expected /\Aextconf failed:

\/Users\/headius\/projects\/jruby\/bin\/jruby extconf.rb.*
checking for main\(\) in .*?nonexistent/m to match "extconf failed:\n\n/Users/headius/projects/jruby/bin/jruby extconf.rb\nWARNING: JRuby does not support native extensions or the `mkmf' library.\n         Check http://kenai.com/projects/jruby/pages/Home for alternatives.\nextconf.rb:2: undefined method `have_library' for main:Object/Users/headius/projects/rubygems/test/gemutilities.rb:7 warning: SAFE levels are not supported in JRuby
 (NoMethodError)\n".

This one is apparently testing that it can't build a native extension in SAFE mode. We support neither, so another test patch is probably necessary.

 17) Failure:
test_build_extensions_extconf_bad(TestGemInstaller) [./test/test_gem_installer.rb:75]:
Expected /\/Users\/headius\/projects\/jruby\/bin\/jruby: No such file/ to match "/Users/headius/projects/jruby/bin/jruby extconf.rb
 --build_arg1 --build_arg2\nError opening script file: /private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/gemhome/gems/a-2/extconf.rb (No such file or directory)\n".

Looks like another oddity with temp directories?

 18) Error:
test_download_local_read_only(TestGemRemoteFetcher):
IOError: Permission denied

I'm not sure what that one's all about.

 19) Error:
test_explicit_proxy(TestGemRemoteFetcher):
Gem::RemoteFetcher::FetchError: SystemCallError: Unknown error - Connection reset by peer (http://localhost:8980/yaml)

Looks like it may be setting up a local server and trying to connect to it? Perhaps it's not up and running when the test actually tries to connect?

 21) Failure:
test_self_load_specification_syntax_error(TestGemSourceIndex) [./test/test_gem_source_index.rb:178]:
Expected /syntax error/ to match "WARNING:  /private/var/folders/V5/V5W2zlDQHPy94tAnCC5f6U+++TI/-Tmp-/test_rubygems_18593/gemhome/specifications/a-1.gemspec:1: , unexpected end-of-file\n\n1 +\n  ^\nWARNING:  1 +\n".

Some difference in errors?

'21) Failure:' should be fixed by JRUBY-4532.

Anyone seeing the rest?

The test_self_load_specification_syntax_error failure does appear to be gone, but I am still seeing the other var/tmp path failures here...

And "18) Error" should be fixed with JRUBY-4536 and JRUBY-4537.

Thanks for clarification. I'll give more time to check the path and connection problem.

Note that we have many failures in 1.9 mode that look like they may be argument-processing related. Moving this to 1.7RC1 for investigation.

Ok, with recent fixes to block args, etc, the 1.9 mode failures are looking much better. Down to 16 F/E in 1.9 mode and 10 F/E in 1.8 mode.

Would be great to get this green for 1.7 final, though it will definitely require masking out some tests that aren't applicable to JRuby.