JRuby

The paste seemed to get lost, but we understand the issue. This needs to be fixed for 1.0, and basically involves setting the following priorities:

1. Ruby methods crucial to Ruby behavior must not be overwritten (like "class")
2. actual Java method names must exist for Ruby code
3. camel_cased method names come next, only where they don't overwrite
4. shortcut methods for bean properties, etc

And then a set of rules for when a Java method simply can't be expressed on the Ruby side of things (like initialize, for example).

Note that this is the same issue as JRUBY-778, just a different manifestation from the one initially reported there. It boils down to a shortcut method name (inappropriately) being defined for one method, when a real method by that name exists. (In JRUBY-778 it is Container#getInsets being aliased as #insets, even though Container#insets exists; as Jonathan Paisley notes, since Container#getInsets calls Container#insets, this leads to infinite recursion and a SystemStackError, hence the misleading title of JRUBY-778. The issue is the same.)

-Bill

I've got a pretty good start on the fix for this (aka JRUBY-778). If anyone else is working on it, please let me know so we don't duplicate. I should have it done late tonight (gotta go sleep for a few hours now).

Should be faster, too.

-Bill

Late tonight has come and gone No pressure, but this would be a great fix to have in for 0.9.9 (planned for this week). As always, tremendous thanks for your help.

Got bogged down in a couple of issues yesterday. Should be done tonight. (One way or another, I'll submit a patch. There's a quick-and-dirty fix for the immediate problem, but a better fix that resolves other problems and performance issues.)

-Bill

Quick fix attached, resolves the reported issues for this and JRUBY-778.

I'm still working on the 'real' solution, which moves all the setup code now in JavaProxy#setup into JavaClass, and gets everything done in one pass, along with other optimizations; should reduce startup times for Java apps considerably. Should be done tonight or tomorrow. I can open a new issue then if you want to close this one.

-Bill

Update - still working on this, will be done tonight (in the wee hours). Hold the boat!

Update - hit some snags last night. Got them resolved. Testing now. Looks good, everything working. I'm writing some unit tests, will have it ready later tonight (yeah, I know, famous last words, but really).

-Bill

The attached patch should resolve issues JRUBY-814 and JRUBY-778, as well JRUBY-29, JRUBY-73, JRUBY-574, and possibly others. It should also improve performance (especially startup time) for Java/Ruby applications.

I am working on unit tests, but they are not included with this patch. I'll get them in late today/tonight, but I wanted people to have a chance to start testing (at least internally). I have tested thoroughly in my environment (Java 6 / WinXP), but this needs to be tested in other environments, especially Java 5 / Mac OS X, where many of the issues have been reported.

This patch makes a significant (yet compatible!) change in the way Java classes/proxies are exposed in JRuby. In particular, the Java class hierarchy is now represented in Ruby terms, so things like kind_of? and (class) A < B now work as expected. Part of the performance benefit derives from each Java class now only mapping its own methods/fields, rather than those in all its superclasses. Once java.awt.Component has been loaded, for example, its methods are there; JRuby subclasses won't re-implement them (unless, of course, the Java subclass overrides them).

The change should not break any existing applications; JavaProxy and ConcreteJavaProxy still sit atop the inheritance tree, so kind_of?JavaProxy will still work.

One of the changes, to RubyClass, suppresses calls to #inheriited for Java classes installed into the hierarchy. This seemed like a heinous hack to me at first, but I've concluded that it's reasonable, considering we're representing a pre-existing hierarchy. It was necessary to do this to preserve the class A < B syntax for user-created (generated) subclasses.

This change does not yet apply to interfaces in any significant way, but I'm about a day away from an interfaces-as-modules implementation (a little tricky keeping the existing syntax working at the same time). Stay tuned.

You'll notice that Java classes are, by default, mapped into a namespace under the Java module corresponding to the Java package hierarchy (but with all uppercase package names, 'cause the other way won't work in Ruby):

irb(main):001:0> include Java
=> Object
irb(main):002:0> JFrame = javax.swing.JFrame
=> Java::JAVAX::SWING::JFrame
irb(main):003:0> JFrame.ancestors
=> [Java::JAVAX::SWING::JFrame, Java::JAVA::AWT::Frame, Java::JAVA::AWT::Window, Java::JAVA::AWT::Container, Java::JAVA::AWT::Component, Java::JAVA::LANG::Object, ConcreteJavaProxy, JavaProxy, Object, Java, Kernel]

This is really just a nicety, not functional (yet), and may be turned off by setting JRUBY_JAVA_MODULES=false, either from within JRuby or externally:

ENV['JRUBY_JAVA_MODULES'] = 'false'

Module names that were assigned beforehand will remain.

There's probably more I should mention. Ask away....

And please test if you can.

Cheers,

-Bill

When I run this patch I get an error in testPositions.rb:

[junit] Testcase: testPositions.rb(org.jruby.test.ScriptTestSuite$ScriptTest):	FAILED
    [junit] test/testPositions.rb failed, complete failure list follows:
    [junit] EXCEPTION raised Test Source Positions: 0 -- 
    [junit] 	Exception: undefined method `startLine' for #<Java::ORG::JRUBY::LEXER::YACC::SourcePosition:0xd2a7b2 @java_object=:[1,1]:[1,5]>
    [junit] 	./test/testPositions.rb:13:in `method_missing'
    [junit] 	./test/testPositions.rb:13:in `node_string'
    [junit] 	./test/testPositions.rb:17:in `test_pos_ok'
    [junit] 	./test/testPositions.rb:27:in `compare_node'
    [junit] 	./test/testPositions.rb:33:in `call'
    [junit] 	/Users/enebo/Documents/workspace/jruby_trunk/src/builtin/java/collections.rb:29:in `each'
    [junit] 	./test/testPositions.rb:34:in `compare_node'
    [junit] 	./test/testPositions.rb:33:in `compare_node'
    [junit] 	./test/testPositions.rb:40:in `call'
    [junit] 	/Users/enebo/Documents/workspace/jruby_trunk/src/builtin/java/collections.rb:29:in `each'
    [junit] 	./test/testPositions.rb:34:in `compare_node'
    [junit] 	./test/testPositions.rb:40:in `test_tree'
    [junit] 	./test/testPositions.rb:62
    [junit] 	./test/minirunit.rb:99:in `load'
    [junit] 	./test/minirunit.rb:99:in `test_load'
    [junit] 	<script>:3
    [junit] junit.framework.AssertionFailedError: test/testPositions.rb failed, complete failure list follows:
    [junit] EXCEPTION raised Test Source Positions: 0 -- 
    [junit] 	Exception: undefined method `startLine' for #<Java::ORG::JRUBY::LEXER::YACC::SourcePosition:0xd2a7b2 @java_object=:[1,1]:[1,5]>
    [junit] 	./test/testPositions.rb:13:in `method_missing'
    [junit] 	./test/testPositions.rb:13:in `node_string'
    [junit] 	./test/testPositions.rb:17:in `test_pos_ok'
    [junit] 	./test/testPositions.rb:27:in `compare_node'
    [junit] 	./test/testPositions.rb:33:in `call'
    [junit] 	/Users/enebo/Documents/workspace/jruby_trunk/src/builtin/java/collections.rb:29:in `each'
    [junit] 	./test/testPositions.rb:34:in `compare_node'
    [junit] 	./test/testPositions.rb:33:in `compare_node'
    [junit] 	./test/testPositions.rb:40:in `call'
    [junit] 	/Users/enebo/Documents/workspace/jruby_trunk/src/builtin/java/collections.rb:29:in `each'
    [junit] 	./test/testPositions.rb:34:in `compare_node'
    [junit] 	./test/testPositions.rb:40:in `test_tree'
    [junit] 	./test/testPositions.rb:62
    [junit] 	./test/minirunit.rb:99:in `load'
    [junit] 	./test/minirunit.rb:99:in `test_load'
    [junit] 	<script>:3
    [junit] 	at org.jruby.test.ScriptTestSuite$ScriptTest.runTest(ScriptTestSuite.java:136)

SourcePosition is a concrete class of the interface ISourcePosition. getStartLine looks pretty ordinary.

I think I see the problem here: any methods not present on the original Java object are only getting added as the underscore_case versions. See this list of methods from SourcePosition:

=> ["==", "===", "=~", "__id__", "__jcreate!", "__jsend!", "__send__", "adjustStartOffset", "adjust_start_offset", "class", "clone", "com", "display", "dup", "end_line", 
"end_offset", "eql?", "equal?", "equals", "extend", "file", "freeze", "frozen?", "getClass", "getEndLine", "getEndOffset", "getFile", "getStartLine", "getStartOffset", 
"get_class", "get_end_line", "get_end_offset", "get_file", "get_start_line", "get_start_offset", "hash", "hashCode", "hash_code", "id", "include_class", "initialize_copy", 
"inspect", "instance_eval", "instance_of?", "instance_variable_get", "instance_variable_set", "instance_variables", "is_a?", "java", "java_class", "java_kind_of?", 
"java_object", "java_object=", "javax", "kind_of?", "method", "methods", "nil?", "notify", "notifyAll", "notify_all", "object_id", "org", "private_methods", "protected_methods", 
"public_methods", "respond_to?", "send", "singleton_methods", "start_line", "start_offset", "synchronized", "taint", "tainted?", "toString", "to_a", "to_java_object", "to_s", 
"to_string", "type", "union", "untaint", "wait"]

getStartLine is present, no worries there. So this isn't an issue of interface methods not getting presented to Ruby. But see that there's only start_line, and not startLine.

Back when we decided to start giving out the underscore_case versions of methods, we were a little indecisive about whether to do just camelCase or just underscore_case, and ultimately decided that since Java developers would always be looking for camelCase (at least initially) and Ruby developers would be looking for underscore_case (at least initially) we'd simply provide both. The overhead wasn't considerable...just doubling the number of methods provided. Back then, we also considered possibilities like allowing developers to turn off one or the other, turn on one or the other, and so on, but threw those all out because one library might turn them off, one library might turn them on, and then everyone's fighting or erroring out.

I think this discussion applies here. We should continue providing all of the following:

• original Java camelCase versions of methods (obviously)
• underscore_case versions of the original method names
• shortcut versions (getters, setters, etc) in both underscore and camel

Since Bill hasn't circled back around, i'm going to browse through and see if I can't fix this myself. Bill, if you're around, pop into IRC or dig me up on IM or email so we don't duplicate efforts.

I have a local fix for this that's pretty trivial. I've pastied the two altered methods here:

http://pastie.caboo.se/55249

(Developers of the future: yes, it will expire, but by the time it expires it will have been committed)

This basically just adds camelCase aliases to the process, where only ruby_case was being done before. It also short-circuits if a method doesn't match the java property chopper pattern.

With these changes, the patch runs green as can be.

Ya know, I've been thinking about it all day Bill, and I really dislike the all-caps package/module naming. It really seems jarring to me. Compare this:

Java::Util::ArrayList.new

with this:

Java::JAVA::UTIL::ArrayList.new

I think the limitation of a capital first-letter for constants would be best served with as little change to the original name as possible. There are very few cases that wouldn't fit this model well, and they're not big issues:

• Java::Awt::Frame # ideally AWT would be all caps, but it's impossible to programmatically determine that it's an abbreviation.
• Any case where there's both a package name with a leading cap and one witha leading lower. Of course, the all-caps version is even worse here, meaning that any embedded caps that distinguish two different packages (I know, bad idea anyway) would break.

The single-capifying version just seems a lot cleaner to me, and it provides at least as simple a rule: make the first letter a cap, and you're done. I would be willing to prefer :: over . syntax if we went with single-capifying:

mylist = Java::Util::ArrayList.new

instead of either:

mylist = java.util.ArrayList
mylist = Java::java.util.ArrayList

I agree that the ALL::CAPS package representation is jarring and ugly. Wouldn't be my first choice. The initial-capital scheme is much more aesthetically pleasing.

And here's why it won't work:

Java:

import  java.awt.Color;
import  java.awt.color.*;

JRuby:

cls = Java::Java::Awt::Color
pkg  = Java::Java::Awt::Color #=> nope
cls  = Java::Java::Awt::Color::ColorSpace #=> nope

This is an oft-repeated pattern; I only had to look as far as 'a' to find an example in the java.* packages. You'll find only rare examples that conflict with the ALL::CAPS scheme. (With good reason :=) )

As I noted in my previous post, this feature was originally intended as more of a nicety, so the user would see something besides:

irb(main):004:0> JFrame.ancestors
=> [JFrame, #<Class:01x127461b>, #<Class:01xf268de>, #<Class:01x135605a>,...

I still think the [Java::]package.dot.Name syntax is just fine (if a little inefficient) for specifying classes. And we could probably hack the RubyClass #to_s output so the user would see something like:

=> [JFrame,java.awt.Frame,java.awt.Window,java.awt.Container,...

(In fact, I kind of like this; goes back to an earlier notion of having a JavaRubyClass (or RubyJavaClass?) subclass of RubyClass, and getting rid of JavaProxy altogether.)

However, for interfaces, we'll need something different if we don't want to break existing usage:

Now:

class Runner < java.lang.Runnable #ok, a class
...
end

Soon:

class Runner
include Java::JAVA::LANG::Runnable #ok, a module
include Java::JAVA::LANG::Comparable # etc
end

You'll note that Runnable and Comparable are initial-caps even though they're modules; the rule would be that terminal types are initial-caps.

We could probably come up with something else for the transitional period before ClassName < some.interface.Class is deprecated. I've been thinking along the lines of:

class Runner
include Java.interface(java.lang.Runnable)
# -- or --
include Java.module_for(java.lang.Runnable)
end

Anyhow, something to think about (or sleep on; I'm running a sleep deficit and need to catch up!).

Charlie, your patch to my patch looks fine. I'm surprised that didn't show up in my testing. I did get the testPosition errors this morning when I ran ant test, but thought they were related to a breakage in the build that occurred around the time I was preparing to submit my patch.

Anyone get a chance to test on Java 5 / Mac OS X? Here's the acid test (or one, at least):

include Java
class A < javax.swing.JFrame; end
class B < A; end
class C < B
def initialize
super('Hi Mom')
pack
end
end
C.new.visible = true

I'll get some unit tests submitted tomorrow. I'll be getting back to the interface stuff tomorrow, too, so let me know your thoughts.

-Bill

Updated patch, supersedes the previous java_support.patch, and adds::

1. Incorporates Charlie's fixes for camelCase properties
2. Adds a few more reserved Ruby names, and splits reserved names into static/instance.
3. A minor tweak here and there.

Passes all tests now.

-Bill

I was just reading this thread on the mailing list when it struck me how comingling use of dot ('.') with double-colon ('::') as a way of accessing constants and namespaces is hella-confusing. I'm close to thinking that we should just pick one way or the other, and I'm leaning toward dot.

Could we define the packages as anonymous modules with a special "package name" field that is the real name of the package, and then ensure that there are singleton methods on the parent package that will yield the package? Then we wouldn't have to clutter the constant namespace with a bunch of lossy upper-case or capitalized names of packages. And do this while preserving the ancestor hierarchy that Bill's worked hard to provide.

On the other hand, I guess if you follow the rule that lower-case names are accessed with dot (method calls) and upper-case/capitalized names are accessed with double-colon (constant), you won't get into trouble. It's a fine balance between something familiar to Java programmers and something that's more natural Ruby.

Re: '.' versus '::'

I agree with Nick that I like the '.' syntax well. We do periodically get people confused about this.

Re: ALL CAP PACKAGES

Taking Nicks suggestion to heart:

cls = Java::Java::Awt::Color
pkg  = Java::Java::Awt::Color #=> nope
cls  = Java::Java::Awt::Color::ColorSpace #=> nope

Would become:

cls = Java.java.awt.Color
pkg = Java.java.awt.color
cls = Java.awt.color.ColorSpace

I think this solves all problems?

Re: Interface support

I am not sure how this example could work?

class Runner
include Java::JAVA::LANG::Runnable #ok, a module
include Java::JAVA::LANG::Comparable # etc
end

If we pass Runner out to a Java method it will know that Runner is both Runnable and Comparable?

How about this example?

class Runner
include Java::JAVA::LANG::Runnable #ok, a module
include Java::JAVA::LANG::Comparable # etc
end

# Do something with Java and Runner

class Runner
include Java::JAVA::LANG::SomeJavaInterface
end

Thoughts:

I like the dotted syntax better. However...

1. The dotted syntax prevents you from reopening the classes. You can't do, for example:

class Java.java.lang.System; class << self; def [](arg]; getProperty(arg); end; end; end

but you can do:

class Java::Lang::System ...

It also prevents defining class methods directly, for the same reason:

def Java.java.lang.System.foobar # syntax error
def Java::Lang::System.foobar #ok

2. I think the interface example Bill gave is cool, assuming he meant this is how interfaces will be represented on Java types....obviously there's no way for us to reopen a class after it's been created (at the moment you handle the class XX::YY bit) and add interfaces to it after-the-fact. But representing class Thread as having a Runnable module attached maps pretty well in our Java-mimicking Ruby hierarchy.

3. I don't have an answer for the problem of packages and classes named the same. Maybe it doesn't matter as long as people know it's happening? Maybe that's crazy talk? May too much magic?

Java::Awt::Color::RED
Java::Awt::Color::ColorSpace

It might be surprising to Java folks, and it looks like ColorSpace is under the Color class, but it's fairly Rubyish...

I vote we commit this now, and debate the final package naming scheme over the weekend. Tom's going to join me this afternoon to think through some of the issues too.

I've marked the other bugs as fixed since they'll be so when we commit this (and it's nice to have a more realistic count of remaining bugs in the 099 list).

I've been experimenting with Java interfaces today (and Java integration in general), and think what I've come up with works pretty well (well, except for the actual working part; I've played with all the different pieces, but need to pull it all together). Here's where I'm thinking about going (very much open to debate/discussion, of course):

1. Drop the hideous ALL::CAPS packages-as-modules scheme. I don't think we really need it; the dot syntax is fine, especially if we:

2. Make a tiny hack (attached; display_java_class.patch) to the default display of class/module names, resulting in:

irb(main):002:0> JFrame = javax.swing.JFrame
=> JFrame
irb(main):003:0> JFrame.ancestors
=> [JFrame, Java::java.awt.Frame, Java::java.awt.Window, Java::java.awt.Container,
Java::java.awt.Component, Java::java.lang.Object, ConcreteJavaProxy, JavaProxy,
Object, Java, Kernel]
irb(main):004:0>

3. Keep the existing < syntax for "inheriting" interfaces; it's wrong, but I think pretty widely used; it should deprecated, though, and eventually phased out, in favor of:

4. Expose interfaces as modules. I've figured out (and experimented with) juggling the two representations simultaneously. (BTW, JRuby helps out with a BUG, see below, but I'm not counting on it). Thus:

class MyClass
include java.lang.Runnable
include java.util.Observer
def run
puts 'running'
end
def update(observable,arg)
puts "#{observable}; #{arg}"
end
end

This allows n interfaces to be implemented (only one call is made to Java.new_proxy_instance, passing an array of interface classes; basically the same underlying mechanism Ola and I had been promoting with the MultipleInterfaceJavaProxy a while back).

Interfaces, as included modules, will show up in the display of ancestors, and work with kind_of?, A < B, etc. Interfaces present in the Java class hierarchy will be exposed in much the same way as classes now are.

5. The interfaces-as-modules choice then simplifies the matter of extending and implementing simultaneously:

class MyClass < javax.swing.JFrame
include java.awt.event..ActionListener
include javax.swing.event.ListDataListener
... #implementation ...
end

I haven't yet tried this one; the code is there in JavaProxyClass(Factory); I'll find out shortly if it works.

I'm taking a break for a while (gonna catch the end of the Red Sox / Yankees, and a nap), but I'll probably be able to have this done and working sometime in the wee hours tonight/tomorrow.

Let me know what you think.

Oh, and for some reason this is allowed (and works) in JRuby (but not MRI). Like I said, I'm not relying on it:

class A
def boo
'boo!'
end
end
class B
include A
end
B.new.boo
=>"boo!"

-Bill

BTW, you'll need ENV['JRUBY_JAVA_MODULES'] = 'false' to see the effects of the patch.

Bill, can you file this additional patch in a new bug? There are some concerns about the modules-as-interfaces we need to discuss so we won't get this into 099. Specifically, by allowing interfaces to be included as modules, we open ourselves up to this:

class MyClass
  include Runnable
end

x = MyClass.new

class MyClass
  include Serializable
end
# This would have to fail; we already have had to create an instance of MyClass, and by extension we've already created the proxy...so there's no way to add new interfaces to that type.

By allowing interfaces to be added like modules, I believe people will think they can add interfaces at any time. That's the original reason we didn't do it this way.

Anyway, file in a new bug, and we'll talk through it.

Marking as fixed. The additional fixes/work from Bill will need to be post 099, because they're too big and need more discussion/analysis. But the original set of fixes have been committed and working apparently well over the weekend.

FYI, Tom's commit of the original java_support_2 was in 3516.