groovy
  1. groovy
  2. GROOVY-3495

LinkageError when obtaining a Cipher instance in a multithreaded environment

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.1
    • Fix Version/s: 1.8.2, 1.9-beta-3, 1.7.11
    • Component/s: None
    • Labels:
      None
    • Environment:
      Using BouncyCastle as security provider
    • Number of attachments :
      0

      Description

      When obtaining a Cipher instance the Groovy rootLoader can throw a LinkageError in relation to the provider implementation classes if multiple threads try get such a Cipher instance.

      No unit test as I can't reliably reproduce the effect...
      An example script to illustrate the issue (may need to run multiple times):

      import javax.crypto.Cipher
      import org.bouncycastle.jce.provider.BouncyCastleProvider
      import java.security.Security

      Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider())

      def getCipherInstance = {Cipher.getInstance("Blowfish/ECB/PKCS5Padding", BouncyCastleProvider.PROVIDER_NAME)}

      new Thread(getCipherInstance).start()
      new Thread(getCipherInstance).start()

      And the exception:

      Exception in thread "Thread-2" org.codehaus.groovy.runtime.InvokerInvocationException: java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "org/bouncycastle/jce/provider/JCEBlockCipher$Blowfish"
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:92)
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:234)
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:893)
      at groovy.lang.Closure.call(Closure.java:279)
      at groovy.lang.Closure.call(Closure.java:274)
      at groovy.lang.Closure.run(Closure.java:355)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.lang.LinkageError: loader (instance of org/codehaus/groovy/tools/RootLoader): attempted duplicate class definition for name: "org/bouncycastle/jce/provider/JCEBlockCipher$Blowfish"
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
      at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
      at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
      at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
      at org.codehaus.groovy.tools.RootLoader.oldFindClass(RootLoader.java:152)
      at org.codehaus.groovy.tools.RootLoader.loadClass(RootLoader.java:124)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
      at java.security.Provider$Service.getImplClass(Provider.java:1262)
      at java.security.Provider$Service.newInstance(Provider.java:1220)
      at javax.crypto.Cipher.getInstance(DashoA13*..)
      at javax.crypto.Cipher.getInstance(DashoA13*..)
      at javax.crypto.Cipher$getInstance$0.call(Unknown Source)
      at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:129)
      at CipherTest$_run_closure1.doCall(CipherTest.groovy:7)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:234)
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:893)
      at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
      at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:151)
      at CipherTest$_run_closure1.doCall(CipherTest.groovy)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
      ... 7 more

        Activity

        Hide
        blackdrag blackdrag added a comment -

        I made the change as suggested here: a6515c5cd89f064b66d683e03258651c3cad586a

        Show
        blackdrag blackdrag added a comment - I made the change as suggested here: a6515c5cd89f064b66d683e03258651c3cad586a
        Hide
        blackdrag blackdrag added a comment -

        your link more or less describes why there are sometimes deadlocks in the interplay between GroovyClassLoader and InnerLoader. But while RootLoader is also not following the usual classloading patterns, it does not interact with a parallel loader, it is hierarchical, only it checks for classes first by its own. I can't be fully sure that there won't be a deadlock problem, but I think it the risk is low. I will make the change tomorrow

        Show
        blackdrag blackdrag added a comment - your link more or less describes why there are sometimes deadlocks in the interplay between GroovyClassLoader and InnerLoader. But while RootLoader is also not following the usual classloading patterns, it does not interact with a parallel loader, it is hierarchical, only it checks for classes first by its own. I can't be fully sure that there won't be a deadlock problem, but I think it the risk is low. I will make the change tomorrow
        Hide
        Dimitris Dimitropoulos added a comment -

        Yes I can confirm that making RootLoader.loadClass syncrhonized seems to solve the issue.

        Of course its a non-deterministic issue but:

        I run the script above without the "synchronized" 50 times and I was getting LinkageError's in 5-10 of the runs.

        I made the method synchronized, recompiled and I am not getting any errors (when I did the same 50 runs a couple of times).

        I switched back (removed the synchronized) and recompiled and I am getting the errors again in the same rate.

        What I don't know is, if this change will make deadlocking during class-loading more common, (I believe I have seen this
        happening too, even with groovy 1.8.2)

        Anyway I will try to investigate deadlocking further, if I see it again.

        There is a relevant change in java 7 and I was wondering if it applies to the groovy class loader.

        http://download.oracle.com/javase/7/docs/technotes/guides/lang/cl-mt.html

        Show
        Dimitris Dimitropoulos added a comment - Yes I can confirm that making RootLoader.loadClass syncrhonized seems to solve the issue. Of course its a non-deterministic issue but: I run the script above without the "synchronized" 50 times and I was getting LinkageError's in 5-10 of the runs. I made the method synchronized, recompiled and I am not getting any errors (when I did the same 50 runs a couple of times). I switched back (removed the synchronized) and recompiled and I am getting the errors again in the same rate. What I don't know is, if this change will make deadlocking during class-loading more common, (I believe I have seen this happening too, even with groovy 1.8.2) Anyway I will try to investigate deadlocking further, if I see it again. There is a relevant change in java 7 and I was wondering if it applies to the groovy class loader. http://download.oracle.com/javase/7/docs/technotes/guides/lang/cl-mt.html
        Hide
        blackdrag blackdrag added a comment -

        could you try putting a synchronized on loadClass and confirm it solves the issue... I fully expect so, but I would like to have confirmation if possible

        Show
        blackdrag blackdrag added a comment - could you try putting a synchronized on loadClass and confirm it solves the issue... I fully expect so, but I would like to have confirmation if possible
        Hide
        Dimitris Dimitropoulos added a comment -

        Perhaps I am missing something but I had a look at the code and the jdk6 implementations of ClassLoader and URLClassLoader and
        I believe that Rich is right.

        URLClassLoader.findClass (called by oldFindClass) is not syncrhonized and I guess the assumption is that it should be called only from
        inside a synchronized context (ClassLoader.loadClass).

        If you look at the stacks, it is the RootLoader.oldFindClass -> URLClassLoader.findClass that blows up.

        Two threads that would try to load the same class at almost the same time, since RootLoader.loadClass is not syncronized,
        could both get null if they reached RootLoader.java line 119
        Class c = this.findLoadedClass(name);

        at almost the same time (before one of them had managed to run the oldFindClass or loadClass below)

        and both get inside - RootLoader.oldFindClass -> URLClassLoader.findClass -> ... -> JVM defineClass1 for the same
        class.
        I guess this throws the LinkageError.

        Am I missing something?

        I would add syncrhonized in RootLoader.loadClass but I don't know if this would cause more deadlocks since as far as I understand groovy
        uses a non-traditional class loading mechanism.

        For convenience:
        http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/lang/java/lang/ClassLoader.java.htm
        http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/net/java/net/URLClassLoader.java.htm
        http://git.codehaus.org/gitweb.cgi?p=groovy-git.git;a=blob;f=src/main/org/codehaus/groovy/tools/RootLoader.java;h=ec391c06dc05adba66384842eb6fbad455eb186a;hb=master

        Show
        Dimitris Dimitropoulos added a comment - Perhaps I am missing something but I had a look at the code and the jdk6 implementations of ClassLoader and URLClassLoader and I believe that Rich is right. URLClassLoader.findClass (called by oldFindClass) is not syncrhonized and I guess the assumption is that it should be called only from inside a synchronized context (ClassLoader.loadClass). If you look at the stacks, it is the RootLoader.oldFindClass -> URLClassLoader.findClass that blows up. Two threads that would try to load the same class at almost the same time, since RootLoader.loadClass is not syncronized, could both get null if they reached RootLoader.java line 119 Class c = this.findLoadedClass(name); at almost the same time (before one of them had managed to run the oldFindClass or loadClass below) and both get inside - RootLoader.oldFindClass -> URLClassLoader.findClass -> ... -> JVM defineClass1 for the same class. I guess this throws the LinkageError. Am I missing something? I would add syncrhonized in RootLoader.loadClass but I don't know if this would cause more deadlocks since as far as I understand groovy uses a non-traditional class loading mechanism. For convenience: http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/lang/java/lang/ClassLoader.java.htm http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/net/java/net/URLClassLoader.java.htm http://git.codehaus.org/gitweb.cgi?p=groovy-git.git;a=blob;f=src/main/org/codehaus/groovy/tools/RootLoader.java;h=ec391c06dc05adba66384842eb6fbad455eb186a;hb=master

          People

          • Assignee:
            blackdrag blackdrag
            Reporter:
            Sebastian Gozin
          • Votes:
            4 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: