Integration of Jikes RVM and JNODE

We raised a licensing problem with the JNode project in September 2006 informing them that we were unhappy that they had taken a number of source code files from the Jikes RVM project and in violation of the CPL changed the copyright information and the license statement from CPL to LGPL. I just spot checked one of the cases of them, and almost 18 months later it is still improperly licensed/attributed.

Here's an example: http://jnode.cvs.sourceforge.net/jnode/jnode/core/src/vmmagic/org/vmmagic/unboxed/Address.java?r1=1.7&r2=1.8

I don't see that we should be trying to change our source structure to accommodate JNode when at least some people involved with JNode have shown fairly blatant disregard for our license and for giving credit where credit is due.

There is no easy way for CPL and GPLv2 source code to live within the same code base. In the past to integrate apache licensed stuff (version 1 and 2) and GPLv2 stuff - what I have had to do is create a third library that contains all the interfaces that are used. Then license that interface library under MIT and then both the libraries can essentially talk through this third library. Of course you need to make sure that it is implementable by another party and that the files are not bundled. All in all it was a royal PITA. Then again it does not seem like that mob really takes license issues seriously and they are not compliant with GPL as is - so they may feel comfortable just ignoring the GPL parts that they want to.

Also - I am not sure that using a namespace such as org.vm is ideal.

While it would be nice to be able to share implementations and code as much as possible (and almost certainly the opt compiler). I suspect it is a bit of an engineering effort to get going.

From speaking to both sides of people no one thinks the licensing is, or they definitely don't want it to be, a big issue. Although the licensing really needs to be respected. I think the name space issue is a bigger one. It's not obvious when you first think about JNODE but their methods for doing interface method invocation, cast checking and instance of checking, are all substantially less optimal than the RVM ways. To make the opt compiler work in JNODE we needed to port these methods (and others) into the RVM. This is silly as JNODE should really use the optimised methods we have in the RVM. On the other hand, isolate support needs to go into some of these core runtime libraries. org.vm is a way to compromise on this, and I imagine that most of the code would be from the RVM and under the CPL - we'd then build on this for JNODE. If people have better ideas on the name space design then please do tell. I'm not clear if org.vm is the best name for the shared runtime services.

Hello Ian,
hello David and Peter,

As a developer of JNode I'd like to comment on that issue too.
First of, about license stuff, I'm not a lawyer and I don't know that much about it

About the license-chance David spoke about: We're really sorry about that, and we're not that type of guys ignoring a license. To explain it: We have an ant-task that updates the header files in our repository to reflect the date, revision,... updates. This task checks on each file, regardless if it's JNode's or another one's. Since the day we got told about that problem we immediately stopped this script but we didn't change the headers back yet :/ On the one hand I think only interfaces are effected (ok, you can still call that "code stealing"), on the other hand we didn't yet identify which files are affected. But this should not be an issue regarding a cooperation and we'll fix that (just need some free time ).

Regarding GPL vs CPL,.. I have no idea. At least for JNode it should be perfectly legal to use CPL code from what I understood about CPL

Ok, second point are the package names. Like Ian said, it makes very much sense to have a common nameing-schema. Both of us would have a much easier life integrating code from the other project. The JNode people would really appreciate that and I also think the initial suggestion is a good one.
We hadn't had time yet to have a deeper look at the code parts, but we'll do that soon. We still have to identify, if that's all what's needed, what to do about classes that are not common between jikesRVM and JNode (e.g. the PureMethod-Annotation you spoke about). And we need to have a deeper look at side effects (e.g. our nano kernel has some knowledge about the java naming schema (e.g. Address class) and probably some reflection stuff).

Anyway, thank you Ian for opening this issue, we (JNode) really like the idea of working together and putting JNode and Jikes on a common code base to be able to exchange code more easily. And like I said, we'll have a deeper look at the needed changes, too and we'll tell you in case there is missing stuff in your list.

As I see it Peter Barth hits the naill right on the head, but I also have problems by using the org.vm as it symbolize it't from the vm organization (the domain is take by Jay Dennis/The Vocal Majority Chorus). As we are two parts, I think we have to find a good namespace for this and maybe not use a org.

Don't know if it matters, but JNode is licensed under LGPLv2 and not GPLv2, but I have to run a check on CPL.

I think worrying about namespace is putting the cart way before the horse.

Mixing CPL and LGPLv2 code has a host of legal problems and as a project we have to take those seriously and full comply with all licenses. So I agree with Peter Donald, I don't see that we will ever have code from both projects closely intermingled. We don't have the ability to change the license of Jikes RVM or dual license the code and I suspect JNode is in a similar position.

At least in the medium term, I also believe that the vast majority of the Jikes RVM community is not going to want additional code added to the "core" Jikes RVM to support JNode intergration if the code doesn't make sense from a Jikes RVM only viewpoint. This really looks to me like a situation where a third project should be started (ala the various GNU classpath + OpenJDK integration projects) that pulls code from both Jikes RVM and JNode and integrates it downstream. Once there was actually some active development going on along this path for a while, then it would be time to see what could be done upstream in Jikes RVM and JNode to make the downstream integration task easier.

ok, plan clarified.

I still think until the downstream project has gathered some steam there isn't a real motivation to change much in the upstream projects. I don't see why we would need to worry about agreeing on a common package prefix until a much later date when we actually see what if anything would be in common that would need to be reflected back into Jikes RVM. Package prefixes are trivially changed via perl if someone downstream is really offended by using org.jikesrvm as the namespace for jikesrvm code. From a Jikes RVM perspective, it's clearly the name space in which the bulk of our code belongs.

I could see at some point that there might be a role for something like org.vmmagic to define a set of basic APIs for the runtime system, but knowing how that should be structured is something that needs to be iterated on a bunch of times with actual code before it really firms up to the point where it would be actionable.

The actual contents of the various packages is harder to change downstream and to the extent the integration project find places where it would be better for the packages to be organized differently we could have specific discussions about that.

So something we've had for quite some time is a version of the Jikes RVM opt compiler that builds under JNODE. For this we had to fix up all of the runtime support to be that of JNODE's (Mohammad Ansari carried out this work). Unfortunately the opt compiler then just sits in JNODE needing explicit intervention to get code built with it. We also removed yield points and on-stack replacement.

I don't see any point reinventing all the features of the RVM's runtime under JNODE and then having to integrate the RVM's opt compiler with the JNODE way of doing these things. It'd be much cleaner imo to take the RVM classes and use them in JNODE. JNODE would need to add isolate support. I could see why with this the JNODE guys wouldn't be happy though, as the core runtime would be sitting in an RVM name space. I think it's not a big deal to have some middle ground name space for vm basic things, but I'm not sure what should be considered basic. Runtime and magic operations seem like obvious candidates, but what about scheduling, the adaptive system and OSR?

ok, then you should be able to see why I'd be unhappy with large hunks of jikesrvm code sitting in JNode without being in the org.jikesrvm package to make it clear where the code is coming from. If a downstream project wants to consume actual implementation (as opposed to API) they should be ok with giving credit where credit is due and letting the namespace, copyright, and license reflect where it comes from.

Yep, no large chunks of the rvm code base or the jnode code base should sit in either name space as it makes attributing ownership... unclear. It would be good to avoid replication of something like the scheduler though. Personally I don't mind which name space my code sits in as long as its being put to best use without being reinvented all the time.

First off, I'm ashamed about the overwritten header containing the wrong license and it is fixed for all files in our trunk. Sorry for that!

Regarding the JNode license itself. We have classpath and openjdk code which is GPLv2+Classpath Exception and JikesRVM is using that too, isn't it? JNode itself uses LGPL which should be even less of a problem as I got told.

Regarding the original issue, the topic name might be to strict. I think from what both of us (JNode and JikesRVM) would benefit from is an easier way to cooperate in the future. That does not mean we should have a single repository or anything like that.
But I think it could be nice for JikesRVM to test the VM, the compiler and other research stuff inside a Java Operating System that gets improved, getting new drivers,... On the other hand JNode is interessted to use a stable part/revision of the compiler or mmtk in our OS to further improve JNode. Though we're not interessted in new testing features, or some experimental or even unstable features. We want to have a stable, good working and safe environment. So merging code is not an option for both of us.

The attempt to integrate Jikes opt compiler into JNode done by Mohammad Ansari showed, that he had very much work to fix the runtime, special classes, exception handling, different pragma classes,... That's why it would be nice if we could at least have a single naming schema for common classes. I do not mind about the names that much, but we wouldn't like to have org.jikes for base classes. Our own VM and compiler should reference core classes from more neutral class names imho

Thanks Peter for sorting out the license! I agree about the compilers referencing a neutrally named set of runtime classes, how do you/JNODE people feel about things like adaptive optimization? This ties into choices about compilers. I know JNODE's compiler performs a little more optimization than our baseline compiler, I also know the GC implementation is conservative (unlike MMTk). I know there's a desire to use MMTk, would there be a desire then to use the Jikes RVM compilers that already generate the necessary reference maps for MMTk to work?

About adaptive optimization, I'm not 100% sure what that means. From what I can tell we want to keep l1a (perhaps also another one) to have a very fast baseline compiler. l1a is an BytecodeVisitor which contains a virtual stack (items may reside in registers, stack or fpu), so the resulting native code is not completly stack based. l1a also does constant propagation to some extend.
For classes in the bootimage and for special annotated classes (CompileHighOptAnnotation or something like that) we'd like to use an optimizing compiler (i.e. jikes). As a future goal, we of course want some kind of profiling for hotspot detection. But I think no one of us has allready ideas on how to do that in detail.

You're right about MMtk, we want to use it some time in the future. BTW, our current memory manager/garbage collector is semi-conservative (if that's the correct term). We know about the memory layout and each object's header contains a ref to the TIB. What we don't know about is the register layout and the stack layout and our yieldpoints (used for scheduling) are not only written at gc-safe-points. I don't know if it's possible to use a subset of MMtk or if we need to implement gcmaps (is that the term for knowing about registers and stack?). But I guess we'd either want to extend our compiler to emit these maps or stay with conservative gcs. So the jikes compiler should be opt-in and not replace our current compiler completly. Keeping l1a and adding profiling/mapgeneration/whatever to our compiler has the simple advantage: We know our compiler very good

So from our point of view, keeping l1a and extend it where needed is the prefered way. Apart from the naming changes I do not know what has to be done additionally in JNode to better match the way jikes works. But in general, do you see any difficulties with that approach?

> would there be a desire then to use the Jikes RVM compilers that already generate the necessary reference maps for
> MMTk to work?

The problem with that would be, that we will not be able to use different compilers (especially our l1a). So I hope this will not be a show stopper and that we can integrate reference maps for l1a too. Btw, I don't know if it's related to gcmaps/reference maps, but our compiler has support for Write Barriers. It is a simple interface that gets called and the implementing class has to insert it's barrier code. E.g. in our default implementation writeXXXStoreBarrier would change the object's color.

Apparently it will take some time until we figure out how to run the collaboration between JNode and JikesRVM.
For me it was really good news while talking to Ian at FOSDEM that actually there is someone on the JikesRVM side really interested in this with some support in the wider JikesRVM community. So shortly after I arrived home today from FOSDEM + business trip it occured to me that for the first time I checked out the JikesRVM sources from SVN and subscribed to the JikesRVM issue tracker. I wonder what the next steps will be...

Probably a few thoughts about the involved projects and the general context of the collaboration...

Mission statements:
JikesRVM: (from the website)
"Jikes RVM (Research Virtual Machine) provides a flexible open testbed to prototype virtual machine technologies and experiment with a large variety of design alternatives."

JNode: (original formulation by the funder)
"JNode is a simple to use & install Java operating system for personal use. It runs on modern devices. Any Java application will run on it, fast & secure! JNode is open source and uses the LGPL license."

Despite these diverging goals the two have in common the choice of Java based implementation and they appear to be a good match in the sense that various results of the JikesRVM research could find immediate application in JNode much easier than in any other JVM not implemented in Java.

JNode has a wider scope as a system since a Java virtual machine is only one component of the operating system, while JikesRVM has a wider scope in its variety of potential Java virtual machine technologies since probably JNode will ever require only a subset of that, those suitable for the case where a Java based Java virtual machine is thightly integrated with a Java based operating system and as such the virtual machine will not ever need to be ported to an other operating system. By mission, JikesRVM targets a small part of the Java community while JNode intends to be suitable for a large variety of users, applictations and real world usecases.

Having in mind these preliminaries we can imagine several levels of collaboration:

1. Sharing only ideas and people, not code.
Tried and tested design patters, techniques, algorithms developed within the realm of JikesRVM could find their way into JNode in JNode speciffic implementations probably by their original authors while taking into account the specific needs of Jnode regarding stability, standards conformance and robustnes in real world usecases.

2. Defining common interfaces for various JVM services and programing to those interfaces in two different codebases. Then varous JikesRVM components could be used as JNode plugins in various JNode configurations and usecases.

3. Creating a JNode specific branch of JikesRVM and gradually replacing the current JNode VM with it.

It might be also possible to selectively apply these alternatives or a combination of them to various subsystems of JNode and JikesRVM.
In my opinion what really matters is to see that a collaboration does really makes sense, each party can see an advantage in working together, sharing ideas, expertise code and finally to work together on licensing and legal issues so that the final result of our work will not create legal issues to any party or to whoever wants to use it.

Hi Levente,

it was a real pleasure meeting up with JNODE developers at FOSDEM and I have the utmost appreciation for everything that JNODE has achieved. I know when Ewout started JNODE the Jikes RVM was an inspiration to him. Unfortunately a lot of the code base in those days was quite crusty, our build system was reliant on bash, we didn't support any Java 5 features, we didn't pack objects. Over time we have worked to slowly close those differences.

My hope is the two projects can work collaboratively on more areas, and in particular have common code whilst respecting distinctions between the projects and licensing. I believe everyone in the Jikes RVM community can see this as being positive, as it allows one implementation to be tested in two different environments. Having spoken to members of the MMTk community, they have been supportive of being able to test GC ideas within the context of JNODE. I'd really encourage everyone interested in this collaboration to add a "+1" comment to this tracker or to vote for it using the JIRA system.

I'm hoping to observe what the Jikes RVM and JNODE communities really want from a collaboration, and I hope to continue to facilitate this through my own contributions. I would be happy to establish a JNODE integration branch for the Jikes RVM and maybe I can work with someone within the JNODE community who can set up the opposite? As can be seen from earlier posts on this topic there is a desire from both the JNODE and Jikes RVM community that attributing development is handled properly, for this reason everything relating to integration needs to be handled in a public manner. I'm also keen that developers on both sides see this as a positive thing and that we aren't bulldozing ahead with this idea and alienating anybody. I'd really love to be in the state where we had a road map for changes to the Jikes RVM and to JNODE, and then I can devote energy to implementing those.

Hi Ian !

I was great to meet you at FOSDEM despite the fact you were speaking too fast for me, which made me understand you at only few percent (I am definitely unused to ear english spoken). Anyway, I know that you are a supporter of JNode (and Jikes RVM of course) and that's great

So, I have voted for this issue and right now add a (+1) comment

I am not sure if I can help for the deepest core of JNode like compiler but, at least, I can help you regarding how to write jnode plugins, compile and run them. I will also try to make people vote for that issue and be involved in its solving.

+1

Who doesn't hate license issues, except lawyers?

hello,
Ian,
Thanks for welcoming this matter in our viewpoint. You know someone is trying to invoke some new killer features in JikesRVM in coming sessions and not only for Java Runtime in near future.

I hope that will open a new door for programmers and developers also in very coming sessions if this works.
So, always [+1] on this matter or topic.
Thanks,
Tanmoy Deb.

Hi Ian, RVM Members and JNode Members,
Sorry for late.
First, I should say thanks to RVM and JNode members, I have learn much from you, especially from Ian. Thank you.

I and my team members are developing a new research Java OS named JUnicorn, which is a single-address-space micro-kernel OS and the process isolated by our new soft-isolated model. JUnicorn is based on RVM, especially RVM's object model and baseline/opt compiler. Right now, JUnicorn can be run on simics successfully, a few services have not implemented completely yet, like file system service which depend on simulation by simics magic instuction. But it will be completed soon.

During the developing JUnicorn, we have meet the same problem about how to make our system and JikesRVM share code. Our solution is that we directly modify RVM compiler to support multiple JTOC and modify the implement of sysCall/Magic etc, most of these codes are redirected to use JUnicorn's services. We use cn.edu.nudt.junicorn name space to write our os code, which like Mem and IO resource managers, scheduler and other services, then we expand RVM's ant build script to build them, and last use our kernel image builder to combine them into a ELF which can booted by grub. During the development, we try to modify RVM's code as less as we can(except MMTk), this can make us keep updating with RVM. We know that it's not easy to build a consummate dynamic compiler for us.

JNode is a great Java OS and develop really fast. I think if we can make a interface between RVM and java OS, JNode and JUnicorn will get benefit from it. It's a not mature idea that we can define the interface which like the interface between MMTk and RVM. I have no clear thought now, but I think should define the interface about exception and magic(need scalable) and sysCall, yield points, GC map, more anoniation directed compiled methods, and JNI methods etc.

LiGen
National University of Defense Technology, ChangSha, China

Thanks LiGen,
JUnicorn sounds very exciting. I think that modularity is a common theme in this and RVM-405. If you could provide information on where we can improve the modularity it would be useful. As this problem is in two of the Google SoC projects this information can help guide those projects if successful students apply. Perhaps information on how JUnicorn approaches isolates would be useful (and a start) ?

The JTOC implementation of JNode and the JikesRVM differ because of JNode's isolate support.