NoClassDefFoundError exception when using a Grab annotation

Couple of main issues here.

1) The ivy jar isn't part of org.codehaus.groovy right now and so the ivy classes cannot be found to satisfy the Grab. Adding this is a reasonable thing to do.

2) Once downloaded, the dependency isn't visible to the compiler building the code. I have to look into how the Grab wants to adjust the groovyclassloader classpath and build some kind of adaptor that causes the dependency to be visible in our way of doing things. I have an idea how to do this.

I have addressed the missing ivy. That leads onto two problems:

1) The way the Grab implementation stuffs the jar into the groovy classloader. We don't do any type resolution through the groovy classloader in groovy eclipse so that fails. I hacked up a solution but am far from happy with it, involving allowing classloader usage if Grab is in the mix. I would rather ensure the project is modified to include dependencies pulled in via this route - that would address problem (2)

2) There is no groovyclassloader when the code is run after compilation - this means Grab fails at runtime because it can't modify the classloader classpath.

probably moving this to M2 as it needs more thought.

not going to make M1

More complete Grab design.

Basically Grab dependencies are a little transient than hardcoded classpath entries. They aren't in the managed eclipse classpath but may need to be around for a build. This is making me think about having a Grab classpath container, which is created on first discovering a user exploiting @Grab.

This container has an interesting lifecycle:

• it comes into existence once the compiler detects a Grab occurring.
• ideally the fact that it is being used wouldn't be committed into the repo if the user commits the classpath to a repo but I guess that could be tricky. if we can't do that it should support 'delete' - is that usual on a classpath container, can you select it, press delete and simply lose it from the classpath?
• it gets cleared on a full build or when a user chooses to (right click on it > empty container)
• it supports 'promotion' of contents to the proper eclipse classpath (one or many selection, right click > promote to project classpath) - this makes the dependency more permanent after the user has finished playing.
• have to manage versioning so we respect the version grabbed - and indicate a problem if there is a clash with the existing classpath or something grabbed elsewhere (and provide automatic fix)

I'm a little unsure if it is possible to change the project's classpath during a compile.

But, I could imagine some sort of post reconcile listener that detects if @Grab is being used and then makes the classpath changes. The Grab container would then remain in the .classpath (and therefore be committed to any repo) until it is explicitly removed by the user (this can be managed from the project properties or by right clicking -> remove from build path).

I could imagine some grab container operations like: empty, refresh, validate, and maybe some way to search for all uses of @Grab in the project (so the container contents could be managed).

If a post compile/reconcile listener is used to initially create the container, then another build/reconcile would need to be created before errors are resolved.

Not sure what the first step here would be, but I imagine that I could create a stub Grab container that is created when an @Grab is encountered. But, is there a quick way of doing this? I don't want to have to walk the AST to find any possible @Grab annotations. Is it possible that the compiler could add a special error marker to the compilation unit if @Grab is found, but there is not corresponding Grab container?

> I'm a little unsure if it is possible to change the project's classpath during a compile.

Right now we do change the classpath whilst building - but not via eclipse, instead by manipulating the groovy class loader used during the compile - this is what enables AST transforms to be found. A similar thing worked that I prototyped for Grab support, where all the magic happened under the covers, but I'd rather surface the dependencies users are getting into the eclipse project, than hide them. As you say, there may be restrictions on doing it during a compile, although I'm perhaps not saying that 'and after it is dynamically added the project needs to notice immediately', as I'm saying 'the user needs to see this dependency they have introduced'. I still need to get my thoughts straight here.

We don't want to get into a pickle where the code won't compile (even reconcile) on first attempt because we haven't yet added a particular Grab dependency to the project classpath - i'd really like to avoid a scenario where the first compile of something correctly formed actually fails because of using Grab.

> But, I could imagine some sort of post reconcile listener that detects if @Grab is being used and then makes the classpath
> changes. The Grab container would then remain in the .classpath (and therefore be committed to any repo) until it is
> explicitly removed by the user (this can be managed from the project properties or by right clicking -> remove from build
> path).

grab support is ast transform driven, ast transforms are currently banned from execution during reconciling (this may be lifted but needs a good reason to be lifted). However, what you are proposing would be driving the ast transform in a very different way to how it is driven right now, by detecting the annotation ourselves and calling the transform implementation at a different time to do its job and discover the real dependency. This may be feasible but it depends on how easily callable the transform is from a different context, I really would not want to duplicate what Grab and friends do in another place.

> I could imagine some grab container operations like: empty, refresh, validate, and maybe some way to search for all
> uses of @Grab in the project (so the container contents could be managed).

Hmm, after more thinking perhaps I want it all handled automatically. But I still perhaps like the idea of promoting an entry from the container to the 'real' project classpath - the user just needs to be aware that the GrabContainer represents the current state of their code and any changes to a Grab annotation will be automatically reflected in the GrabContainer, so if they don't promote dependencies out of that container, they may lose them when Grabs are removed.

I think I can see a use case where you hack away at a script/class trying things out then when you are happy, you promote the dependency to a 'real one' and remove the @Grab.

> If a post compile/reconcile listener is used to initially create the container, then another build/reconcile would need to be > created before errors are resolved.
>
> Not sure what the first step here would be, but I imagine that I could create a stub Grab container that is created when an
> @Grab is encountered. But, is there a quick way of doing this? I don't want to have to walk the AST to find any possible
> @Grab annotations. Is it possible that the compiler could add a special error marker to the compilation unit if @Grab is
> found, but there is not corresponding Grab container?

I need to think about this and how it relates to what happens for Grab under the covers. I am thinking I really want to avoid the first compile failing even when the code is properly formed, because the Grab is processed too late. hmm.

In reply to comment #6:
> > I'm a little unsure if it is possible to change the project's classpath during
> a compile.
>
> Right now we do change the classpath whilst building - but not via eclipse,
> instead by manipulating the groovy class loader used during the compile - this
> is what enables AST transforms to be found. A similar thing worked that I
> prototyped for Grab support, where all the magic happened under the covers, but
> I'd rather surface the dependencies users are getting into the eclipse project,
> than hide them. As you say, there may be restrictions on doing it during a
> compile, although I'm perhaps not saying that 'and after it is dynamically added
> the project needs to notice immediately', as I'm saying 'the user needs to see
> this dependency they have introduced'. I still need to get my thoughts straight
> here.

I guess there is a bit of wiggle room here. The classpath can be changed from the compiler's point of view immediately, and then after the compile is done, the change gets committed to the .classpath (but of course, this will implicitly trigger a full build, because that's what happens on classpath changes). But, remember, once the Grab container is in place, a change to the contents of the container will not necessary force a build (we could have control over that), but the initial addition of the container will cause a build.

>
> We don't want to get into a pickle where the code won't compile (even reconcile)
> on first attempt because we haven't yet added a particular Grab dependency to
> the project classpath - i'd really like to avoid a scenario where the first
> compile of something correctly formed actually fails because of using Grab.

It seems like a boot strapping problem, or maybe I don't understand how reconcile works well enough. To me, it seems that the time we figure out if a grab annotation is being used is during a parse and isn't that too late to make classpath changes? Maybe not...

I don't think it's too onerous for a user to explicitly say "this project requires grab support". Perhaps a context menu could add it. Or a quickfix could add it. Once we have the container in place, we have a lot more freedom to change its contents underneath without JDT being too concerned.

...

> Hmm, after more thinking perhaps I want it all handled automatically. But I
> still perhaps like the idea of promoting an entry from the container to the
> 'real' project classpath - the user just needs to be aware that the
> GrabContainer represents the current state of their code and any changes to a
> Grab annotation will be automatically reflected in the GrabContainer, so if they
> don't promote dependencies out of that container, they may lose them when Grabs
> are removed.
>
> I think I can see a use case where you hack away at a script/class trying things
> out then when you are happy, you promote the dependency to a 'real one' and
> remove the @Grab.

This could be nicely handled through a QuickAssist...

...

After some time thinking about this I realise:

• I have to make it work when the compiler is used in batch mode, so can't do anything too funky with infrastructure outside of the compiler.

• A call out to the surrounding infrastructure does not need to be made for the code to compile, but it does need to be made for two other reasons:
  1) If the classpath for the project is not updated to include anything grabbed, I imagine code assist will not work that well as it will have no clue where the types came from that were involved in the successful build.
  2) When something is Grabbed, the class appears to get a dependency on some ivy types (in the ivy jar shipped with groovy) - this means to run the compiled result as a Java App, the ivy jar needs to be added to the classpath. We don't need to do this for all groovy code, just those that performed a grab.

I am on the verge of committing the first version that will enable Grab code to build just fine (even reconciling)

> 1) If the classpath for the project is not updated to include anything grabbed,
> I imagine code assist will not work that well as it will have no clue where the
> types came from that were involved in the successful build.

Maybe, but it might be sufficient just to be able to have the correct JDTClassNode for these things. Content assist for Groovy files is based off of class nodes. For Java files, however, this might be more complicated.

> 2) When something is Grabbed, the class appears to get a dependency on some ivy
> types (in the ivy jar shipped with groovy) - this means to run the compiled
> result as a Java App, the ivy jar needs to be added to the classpath. We don't
> need to do this for all groovy code, just those that performed a grab.

Am I correct in saying that scripts that use @Grab will not be run as Java apps, but as groovy scripts? If so, this gives us considerably more leeway in what we stuff on the runtime classpath. What would be sufficient in this case would be that the compiler sets/unsets some kind of property that says "yes, grab is being used" and if the launcher sees this property set, it can do the right thing when launching.

> I am on the verge of committing the first version that will enable Grab code to
> build just fine (even reconciling)
Very nice.

> Am I correct in saying that scripts that use @Grab will not be run as Java apps, but as groovy scripts?
> If so, this gives us considerably more leeway in what we stuff on the runtime classpath. What would be
> sufficient in this case would be that the compiler sets/unsets some kind of property that says "yes,
> grab is being used" and if the launcher sees this property set, it can do the right thing when launching.

I don't believe you are restricted to having to run them as scripts. However, if they are run as scripts,
they just work. They have for a long time.

> Maybe, but it might be sufficient just to be able to have the correct JDTClassNode for these things.
> Content assist for Groovy files is based off of class nodes. For Java files, however, this might be more complicated.

Well I don't think they are JDTClassNodes - since as we've discussed eclipse knows nothing about these types and
so groovy cannot build a JDTClassNode for them. I will get the first stab of support in and we can see what happens.

I just tried it out and although I get a wild variety of errors attempting to run a compiled Grab application (including a class format error at one point!) - it would appear there is a need to use the script launcher so that the classloader can be set properly for the grabbed dependencies. So you are right, these cannot be run as applications. This reduces the work to do - as we should really police their execution as a java application and prevent it!

done what I plan to do - grabbed code should now compile. code assist is a little glitchy

I fixed this glitch:

import groovy.lang.Grab;

@Grab(group="joda-time", module="joda-time", version="1.6")
def printDate() {
       def date = new org.joda.time.DateTime()
       // date.<ctrl+space>
       print date
}

def date2 = new org.joda.time.DateTime()
//date2.<ctrl+space>
printDate()

At the second location 'date2.' you get DateTime related completions. At the first 'date.' you do not. The problem here is that source locations where not being properly propagated and this was confusing the content assist processor.

Let me know if you see any more.

Another isssue here is with code selection and navigation. Since the @Grabbed element is not on the classpath, there is no way to convert it to IType and hence hovers and navigation will not work. I will create a new bug for this.

The good thing is that since the compiler is able to resolve it, things like type inference will work.

nothing further I plan to do here for now

Andy, assigning to you. Should this bug be closed, or should the target of 2.0.2 be moved?

just to note reconciling for grab is currently broken

Andy, assigning to you to see if there's anything you can do before the 2.0.2 release.

Reconcile producing extra errors is fixed with changes for STS-1360.