ClassHelper.ClassHelperCache#classCache is not thread-safe

No wonder, mos of the compiler infrastructure is designed for single thread usage only. I would have suggested to use makeWithoutCaching, but that ends in makeCached too, which uses the map. I think that must be changed then.

As for a workaround... is it impossible to do some kind of warmup?

As a workaround, we'll now do the parsing in a single thread. But that may take quite long. A warmup could help, if I knew exactly what to invoke to ensure everything's warmed up.

If it were IDEA sources, I'd just replace WeakHashMap with ConcurrentWeakHashMap, as we have one

the warmup would have to contain all the ClassHelper#make calls you usually do. The problem cannot be there for the static stuff that is in that class, since the WeakHashMap may not be able to handle concurrent writes, but concurrent reads are no problem

The problem is that I don't do any ClassHelper#make calls. I just say new GroovyShell().parse(text) for different script texts. So it's not clear to me which classes should be cached in the map to account for any possible script text.

I replaced the map with an internal concurrent version from our runtime