A clever but weird patch. One problem with it is that it uses to_java, which for many core types (fixnums, strings, etc) will not produce normal Ruby objects. The subsequent inspectHashCode won't work, as a result.
inspectHashCode is also based on java.lang.System.identityHashCode, which is not guaranteed to be unique (it's just the hashcode the object would have if it weren't overridden).
object_id is probably the right way to do this, even though it has quite a bit more overhead for JRuby. I suspect the real problem here is that we don't (and won't) support the "p" format for Array#pack, and so the mask doesn't work right.
I'll try to patch it in another way. Thanks for the interesting shot at it!