LinkageError when obtaining a Cipher instance in a multithreaded environment

I have a very similar problem when using groovy 1.7.0 on Sun JDK 1.6.0_18 when I try to use XStream. It happens only very rarely.

java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "com/thoughtworks/xstream/converters/extended/SubjectConverter"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at org.codehaus.groovy.tools.RootLoader.oldFindClass(RootLoader.java:152)
at org.codehaus.groovy.tools.RootLoader.loadClass(RootLoader.java:124)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at com.thoughtworks.xstream.core.util.CompositeClassLoader.loadClass(CompositeClassLoader.java:69)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at com.thoughtworks.xstream.XStream.dynamicallyRegisterConverter(XStream.java:723)
at com.thoughtworks.xstream.XStream.setupConverters(XStream.java:672)
at com.thoughtworks.xstream.XStream.<init>(XStream.java:445)
at com.thoughtworks.xstream.XStream.<init>(XStream.java:385)
at com.thoughtworks.xstream.XStream.<init>(XStream.java:323)

the most easy solution for this kind of problem is to remove the offending jar delivered by Groovy and located in the lib directory of the distribution. RootLoader does normally not load other jars. Checking the loader configuration would help to verify that RootLoader is not instructed to do so of course.

Thanks Jochen, I gave this a try and it didn't appear to work. Looking at a few other class loaders it appears that their "loadClass" methods are typically synchronized while org.codehaus.groovy.tools.RootLoader's isn't (at least in 1.7.0). I'm wondering if this might be the problem. I've got few threads that might be trying to load a new class nearly simultaneously. Looking at the call stack it might also be a problem with XStream class loader.

if all calls are like the one you have shown, then there cannot be a synchronization problem, since java.lang.ClassLoader.loadClass(ClassLoader.java:248) uses synchronization. We may hvae to rethink the usage of RootLoader a bit, because in java7 ClassLoader#loadClass is by default no longer synchronized. Anyway... I suggested having the jar removed from the lib directory, because that is what is usually laoded by RootLoader and if there is no such class, RootLoader will not load it, thus avoiding to load it. Now the question is, where is that class coming from? It doesn't have to be a synchronization issue in general. It is just that RootLoader is essentially breaking some class loader constraints and that sometimes causes problems. Anyway, there must be a JCE Blowfish provider class in the class path of RootLoader othererwise it couldn't load it. Could you executed:

println this.class.rootLoader.getURLs()

and tell me the output? One of the jars listed there must contain the blowfish provider class and needs to be removed from the classpath

I'm experiencing this problem and it does appear to be thread-related even if it isn't a synchronization issue. I'm running an multi-threaded app. Two threads cause the same class to be loaded. About 30% of the time, one thread fails. About 30% of the time, the other fails. About 30% of the time, everything works (doesn't add up to 100, I know ).

This is the error.

java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "org/mozilla/javascript/optimizer/OptRuntime"

println this.class.rootLoader.getURLs() says that rootLoader is not a property.

OK. This is not a solution to my problem, but I added this at the very beginning of my app and it works reliably now. The class is there and can be loaded, but something in RootLoader doesn't seem to like when multiple threads try to load it at once.

Class.forName("org.mozilla.javascript.optimizer.OptRuntime")

I get similar errors occasionally when kicking off multiple threads each of which does the same thing (with different parameters). In my case each threads is using a unique instance of a class that creates and accessing classes from jars brought in using grapes with @GrabConfig(systemClassLoader=true) in the main script. Groovy Version: 1.7.1 JVM: 1.6.0_20

I've got a similar error using HttpBuilder in a multithreaded environment.
HttpBuilder dependency is resolved using Grapes and the LinkageError appears with LogFactoryImpl probably because HttpBuilder have a transitive dependency on commons-logging that is also available in the lib directory of groovy.
If I remove the commons-logging jar from groovy install directory, the problem doesn't appear anymore.
I don't know if it could help : the script that I've written create a thread pool that performs some http request on my server. The LinkageError appear systematically at the end of the script irrespective of the number of request that I process (10 or 10000 will make the problem appear during the few last requests)

I am also getting this regularly.

The following small script gives this error about 20 - 30% when I am running it with

groovy-1.8.1 and JVM Java HotSpot "1.6.0_21"

List<Thread> threads=[];
int jobs=300;
String url = "This is $

"
Map who=['who':'me'];

for(int i in 0..<jobs) {
threads.add(new Thread(

));
threads[i].start()
}

for(int i in 0..<jobs) {
try

catch (InterruptedException e) {}
}

Exception in thread "Thread-7" org.codehaus.groovy.runtime.InvokerInvocationException: java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "org/codehaus/groovy/runtime/GStringImpl"
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:97)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:883)
at groovy.lang.Closure.call(Closure.java:410)
at groovy.lang.Closure.call(Closure.java:404)
at groovy.lang.Closure.run(Closure.java:488)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "org/codehaus/groovy/runtime/GStringImpl"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at org.codehaus.groovy.tools.RootLoader.oldFindClass(RootLoader.java:152)
at org.codehaus.groovy.tools.RootLoader.loadClass(RootLoader.java:124)
at java.lang.ClassLoader.loadClass(ClassLoader.java:296)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:696)
at groovy.lang.GroovyClassLoader$InnerLoader.loadClass(GroovyClassLoader.java:449)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:793)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at SimpleTemplateScript16.run(SimpleTemplateScript16.groovy:2)
at groovy.text.SimpleTemplateEngine$SimpleTemplate$1.writeTo(SimpleTemplateEngine.java:165)
at groovy.text.SimpleTemplateEngine$SimpleTemplate$1.toString(SimpleTemplateEngine.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:230)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
at linkErrorExample$_run_closure1.doCall(linkErrorExample.groovy:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:883)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
at linkErrorExample$_run_closure1.doCall(linkErrorExample.groovy)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
... 7 more

Perhaps I am missing something but I had a look at the code and the jdk6 implementations of ClassLoader and URLClassLoader and
I believe that Rich is right.

URLClassLoader.findClass (called by oldFindClass) is not syncrhonized and I guess the assumption is that it should be called only from
inside a synchronized context (ClassLoader.loadClass).

If you look at the stacks, it is the RootLoader.oldFindClass -> URLClassLoader.findClass that blows up.

Two threads that would try to load the same class at almost the same time, since RootLoader.loadClass is not syncronized,
could both get null if they reached RootLoader.java line 119
Class c = this.findLoadedClass(name);

at almost the same time (before one of them had managed to run the oldFindClass or loadClass below)

and both get inside - RootLoader.oldFindClass -> URLClassLoader.findClass -> ... -> JVM defineClass1 for the same
class.
I guess this throws the LinkageError.

Am I missing something?

I would add syncrhonized in RootLoader.loadClass but I don't know if this would cause more deadlocks since as far as I understand groovy
uses a non-traditional class loading mechanism.

For convenience:
http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/lang/java/lang/ClassLoader.java.htm
http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/net/java/net/URLClassLoader.java.htm
http://git.codehaus.org/gitweb.cgi?p=groovy-git.git;a=blob;f=src/main/org/codehaus/groovy/tools/RootLoader.java;h=ec391c06dc05adba66384842eb6fbad455eb186a;hb=master

could you try putting a synchronized on loadClass and confirm it solves the issue... I fully expect so, but I would like to have confirmation if possible

Yes I can confirm that making RootLoader.loadClass syncrhonized seems to solve the issue.

Of course its a non-deterministic issue but:

I run the script above without the "synchronized" 50 times and I was getting LinkageError's in 5-10 of the runs.

I made the method synchronized, recompiled and I am not getting any errors (when I did the same 50 runs a couple of times).

I switched back (removed the synchronized) and recompiled and I am getting the errors again in the same rate.

What I don't know is, if this change will make deadlocking during class-loading more common, (I believe I have seen this
happening too, even with groovy 1.8.2)

Anyway I will try to investigate deadlocking further, if I see it again.

There is a relevant change in java 7 and I was wondering if it applies to the groovy class loader.

http://download.oracle.com/javase/7/docs/technotes/guides/lang/cl-mt.html

I made the change as suggested here: a6515c5cd89f064b66d683e03258651c3cad586a