Enable hotswap of groovy classes for debugging

I'm guessing that the reason why hot swap always fails, is because of these two synthetic fields:
public static java.lang.Long __timeStamp;
public static java.lang.Long _timeStamp_239_neverHappen1264308791546;

I wonder what they are used for.

I was planning to look at hotswap during your absence. I also have to figure out how it maps into the other reloading strategy we are looking into.

Once the timestamp is gone, the hideous error that usually occurs is this:

The virtual machine was unable to remove all stack frames running old code from the call stack. The virtual machine is not supplying the debugger with valid data for those frames. Stepping into these obsolete frames may be hazardous to the target virtual machine.

The reason for this error is that the JDT Debug code is attempting to look at the source delta between what the code used to look like and what it looks like now. The parser that is used to look at the code doesn't go into the method bodies. This means for some method foo()

void foo() {
  print 'a'
}

if I change it to

void foo() {
  print 'b'
}

the old and new will appear to be the same due to the method bodies being ignored. Because they are the same the stack frame isn't popped off the debugger and we get into trouble later.

I have a patch that addresses this problem and causes the stack frame to be popped. It is quite ugly because we don't have a patch for jdt.debug.ui - if we patched that things could be much neater. Based on other observations we may need to go down that route anyway to make some features behave (that approach or jdt weaving).

With the patch applied some basic scenarios are working for me just like they would debugging Java code. However, I think due to some groovy caching it isn't quite behaving how you might always expect. Here:

public void run() {
  println 'abc'
  println 'abc'
  println 'abc'
  println 'abc'
  println 'abc'
  println 'abc'
}

If we step through run() and then change one of the (passed) printlns to simply print - the debugger will jump back to the start of the frame but still call the original action. I think it is due to the caching of call destinations. If you change the parameters, that kind of change works fine. so more to be done, but at least we can get past that nasty dialog.

yes, I'm pretty sure it is the call site array created up front and cached. Call sites start off abstract and are optimized as they are encountered. Once optimized there doesn't appear to be a way back other than causing the softreference holding the call site array to be GC'd. Two options:

• drive $createCallSiteArray after a HCR. May prove exceedingly tricky without being able to change jdt.debug.ui directly.

• hope for a groovy option that turns off call site optimizations. Debugging will run more slowly but HCR will behave.

wasnt quite right with the last comment - it isn't enough to drive $createCallSiteArray as the value returned from that needs to be put into the $callSiteArray field. And turning off the optimizations from an AbstractSite to a more precise site isn't enough either. You have to cause reinitialization of the $callSiteArray - this code at the start of any method you want to 'behave' under HCR will fix it!

Field f = Flibble.class.getDeclaredField("\$callSiteArray")
f.setAccessible true
f.set null,null

With that in place, the $callSiteArray will be reinitialized as required - and HCR all works.

Patch to some of the base projects that address the nasty error appearing, it does not address the problem of clearing the callSiteArray.

Patch is relatively ugly because it is being done without changing jdt.debug

here is a neat solution...

I've attached gragent.jar - this is a ClassFileTransformer. It will attempt to clear call site array fields on class redefines. If the debug launch configuration includes the VM argument:
-javaagent:<pathToGragent>/gragent.jar

then with the previous patch applied, HCR works!

Wow! Respect!
Do you mind if we use the same jar in IntelliJ?

There are a combination of things here. As well as the agent, you would also need a groovy compiler that doesn't produce timestamp fields. The patched version we use in groovy-eclipse will have this removed. There is talk on the (groovy) mailing list of making the timestamp field creation optional - but I'm not sure if that has progressed to be a real jira yet.

Hi Peter, just to add: I don't mind if you use the same jar - but it isn't quite in its finished form yet, I just hacked that up and put it here so I didn't lose it in the mess that is my hard drive It will be apache licensed when finished (if that makes any difference). It currently also only clears callsite caching - I haven't yet checked if other caches will also need clearing - the final version will do whatever is needed.

cheers
Andy

Thanks, Andy! I suppose that removing the timestamp fields is actually quite easy via post-compile instrumentation. Or maybe it could be even done with runtime instrumentation, via the same jar? Although I'm not sure that removing two fields and nullifying the third one at the same time is possible

Yes, you are right, the agent could also remove the timestamp fields on the initial define and latter class redefines. This would be more work than nulling the callsites but is definetly achievable. To null the callsites the Class object passed into the transformer is used, whilst the timestamp fields would be removed from the byte[] arrays being passed in on define or redefine.

Here's an enhanced agent source, which removes the timestamp fields at runtime. It should be packaged with ASM, probably jarjar-ed. It can be used independently of an IDE, for example, in Grails.

hotswap improvements waiting on groovy fixes (currently waiting on them pulling out static initializer code they use to initialize method local variables)

Andy, assigning this to you so you can figure out what still needs to be done here and whether or not it's possible to do more work here before 2.0.2 is released.

Will not get to for the 2.1.0 release.

We miss this feature very badly. This is an groovy integration stopper for us.