Resolve classloading issues (for eg. OSGi) without changing API

Tatu Saloranta added a comment - 24/Aug/10 6:51 PM Ah definitely. Thank you for tackling this; I have been busy fighting all kinds of fires and haven't had time to look into issues. But I am not surprised there are some. Actually there will be one more place where this might be problematic, "mr bean" extension that will construct new classes (under src/mrbean/java). Mostly what is needed is just to figure out parent class loader for custom one used to load constructed classes; not sure whether that is allowed on an OSGi platform not. Anyway this seems small enough to be included both in 1.5.x and trunk (1.6) so I'll try to squeeze it in.

Tatu Saloranta added a comment - 24/Aug/10 11:35 PM I think I will add bit more fluff around suggested patch; partly to reduce side effects, partly because I am paranoid about performance (i.e. want to avoid setting class loaders when not needed), and partly since that allows also supporting use of explicit alternate class loader.

Tatu Saloranta added a comment - 25/Aug/10 12:40 AM Actually I don't think patches for TypeParser and ClassNameIdResolver work; looks like ClassLoader.loadClass() (of context class loader) can not resolve array types, and 3 unit tests fail. Not sure what is the proper fix for this issue yet, nor whether difference is due to Class.forName() working differently or just that types that come from primordial class loader are problematic.

Caspar MacRae added a comment - 25/Aug/10 3:12 AM Argh, i missed the test failures (i'm used to maven failing the build, but ant tricked me by end with "build successful"), i should have paid more attention. OSGi's hierarchy is classloader per bundle, so delegating parent first won't work (the package export directives tell bundles where to find classes and subsequently which classloader to use AFAIK). Could it be using the junit framework classpath/loader that is causing these tests to fail? It definitely works for my OSGi deserialization use case, I'm sorry I can't look at this any further at the moment and apologies for submitting a bad patch.

Tatu Saloranta added a comment - 25/Aug/10 12:36 PM I was wondering the same thing; this could theoretically be related... I should be able to verify that with a simple stand-alone piece of code. I suspect it's not due to that, and have this nagging feeling that I have bitten by something similar earlier (maybe in context of generating method signatures, where primitive and array type naming have some subtleties).

Tatu Saloranta added a comment - 25/Aug/10 9:24 PM Ah, look at that – there's actually 3 argument variant of Class.forName(), which might work well here.... I may be able to make this work after all.

Caspar MacRae added a comment - 26/Aug/10 5:41 AM Hope so - I think the only requirement from the OSGi side of things is to ensure the valueType's ClassLoader is used, as this will pull in the correct ClassLoader (as client bundle using Jackson and entity must import both Jackson and entity bundles, so will therefore have the correct classloader for entity (valueType)).

Tatu Saloranta added a comment - 26/Aug/10 12:02 PM Yeah, Class.forName() itself does work. But one question I have is this: although class loader is set, it's not unset after readValue(). That seems bit risky, that a low-level library changes context (~= default) class loader on the fly, possibly affecting way bigger systems work. I wonder how other libs deal with this. Other approach would be to pipe configured class loader in through the system, but it's rather long control flow from what I dug through. :-/

Tatu Saloranta added a comment - 22/Mar/11 12:26 AM Quick note: implemented JACKSON-527 , which helps with instantiation of handlers (TypeIdResolver, JsonSerializer, JsonDeserializer). However, realized it won't help with value deserialization. For that, change for 1.7 (of using context class loader) might help slightly, although probably not enough. So will leave this open still.

Andrei Pozolotin added a comment - 25/Apr/11 9:37 AM see if this approach might help: http://code.google.com/p/transloader/

Tatu Saloranta added a comment - 20/Jan/12 1:27 PM That could work for some cases, where nothing else helps? Good to know. (also: seems to have a bit of overhead, if making deep copy, so should only be used if must be?)

Tatu Saloranta added a comment - 20/Feb/12 4:24 PM Hmmh. I wish I fully understood the scope, solutions and so on. I don't feel comfortable adding fallbacks without fully understanding them. But if a unit test could be written to point out the issue, that could help. And then we'd need a new feature request: I suspect the way to enable things would be to either add a feature to enable fallback, or perhaps as you suggested, "just do it". One concern is that lookups are scattered in multiple places, with varying access to configuration.

Graeme Walker added a comment - 20/Feb/12 4:37 PM The originally code worked fine.. it wasn't until it was changed to getting it from the thread that it went wrong. A unit test may be possible... but I think Ant is playing a part, as it won't be adding the context class loader to the thread when it creates one for running the Task. The same code works fine in JBoss.. it only fails when running the code from Ant. I would argue the Class.forName should be the default as it will work in most cases and make Casper's issue the exception.

Tatu Saloranta added a comment - 20/Feb/12 6:40 PM Right, I assume it is not a very common case (similarly, failure with original method was not a very common one as far as I understand things). I can not argue strongly either way, since both have always worked for me. It does sound, however, that some sort of abstraction would be useful, to allow customization. Now: where exactly do you get the exception? I can have a look, but it has been a while since I actuallu looked into this issue.

Graeme Walker added a comment - 21/Feb/12 4:01 AM Stacktrace: 2012-02-20 14:21:31,647 ERROR Invalid type id 'com.gtnet.common.client.json.JSONArray' (for id type 'Id.class'): no such class found java.lang.IllegalArgumentException: Invalid type id 'com.gtnet.common.client.json.JSONArray' (for id type 'Id.class'): no such class found at org.codehaus.jackson.map.jsontype.impl.ClassNameIdResolver.typeFromId(ClassNameIdResolver.java:62) at org.codehaus.jackson.map.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:113) at org.codehaus.jackson.map.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) at org.codehaus.jackson.map.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromAny(AsPropertyTypeDeserializer.java:146) at org.codehaus.jackson.map.deser.std.UntypedObjectDeserializer.deserializeWithType(UntypedObjectDeserializer.java:106) at org.codehaus.jackson.map.deser.StdDeserializerProvider$WrappedDeserializer.deserialize(StdDeserializerProvider.java:461) at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863) at com.gtnet.common.client.json.JSONUtil.fromJSON(JSONUtil.java:52) Our JSONUtil class is being run a part of an ANT task where ANT has the JAR containing the JSONArray class and Jackson library in its class path, however somehow, the ANT task does not have the JAR that contains JSONArray in its classpath when Jackson is trying to find the class to create an instance of it. If I change the code to use Class.forName() within ClassNameIdResolver, everything works correctly. As a workaround I am going to add Class.forName to the ClassNameIdResolver as a fallback if the thread classloader can't find it. When the code is running normally withing JBoss, everything works as expected

charlie babitt added a comment - 06/Mar/12 3:54 AM I have exactly the same problem using Jackson on an Android device. There the Class.forName works, the code in the 1.9.3 release doesn' work here. I simply fixed it by uncommenting the line from the code below and removing the other two... //Class<?> cls = Class.forName(id); ClassLoader loader = Thread.currentThread().getContextClassLoader(); Class<?> cls = Class.forName(id, true, loader); Hopefully this issue is resolved soon, so that I can return to an actual release of jackson. Thanks.

Tatu Saloranta added a comment - 06/Mar/12 12:29 PM - edited Problem is, there is no simple fix: using Class.forName() does not work on all platforms. Is it not possible to set contextual class loader on Android? Going back to old way would break someone else's problem; plus, patch releases are not supposed to contain new functionality (a new feature). So at this point we are bit stuck actually. I can add a new flag for 2.0.0 easily, which sounds like a reasonable thing to do.

Tatu Saloranta added a comment - 06/Mar/12 1:02 PM Quick note: created JACKSON-802 to add a new feature for 2.0.0. Will also investigate what else can be done with 1.9 – I am not sure if new abstraction were added; at very minimum it should not be necessary to patch a release, even if some custom code was needed on top of core.

Tatu Saloranta added a comment - 08/Mar/12 11:48 PM Ok: JACKSON-802 is now implemented; so that class loading will now check both: Contextual class loader as primary choice, and Class.forName(...) as fallback I hope this will work on Android; please let me know if not (one of these days folks at Google learn how to build Java platforms I hope :-/ ).

Rudi Wijaya added a comment - 26/Dec/12 3:06 AM - edited We had to use workaround like this, which works but... @SuppressWarnings( "serial" ) public class ValueEMapDeserializer extends StdDeserializer<EMap< String , EList<Value<?>>>> { protected ValueEMapDeserializer() { super (EMap.class); } @Override public EMap< String , EList<Value<?>>> deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException { // TODO: find a better, cleaner workaround :'( final ClassLoader oldCcl = Thread .currentThread().getContextClassLoader(); Thread .currentThread().setContextClassLoader(ValueEMapDeserializer.class.getClassLoader()); final Map< String , List<Value<?>>> map; try { map = jp.readValueAs( new TypeReference<Map< String , List<Value<?>>>>() {}); } finally { Thread .currentThread().setContextClassLoader(oldCcl); } // convert List to EList final Map< String , EList<Value<?>>> transformed = ImmutableMap.copyOf(Maps.transformValues(map, new Function<List<Value<?>>, EList<Value<?>>>() { @Override public EList<Value<?>> apply(List<Value<?>> input) { if (input == null ) return null ; return new BasicEList<Value<?>>(input); } })); return new BasicEMap< String , EList<Value<?>>>(transformed); } } Otherwise we'll get this: 2012-12-26 15:50:47,870 | WARN | tp1348317115-197 | WebApplicationExceptionMapper | pl.WebApplicationExceptionMapper 71 | 214 - org.apache.cxf.cxf-rt-frontend-jaxrs - 2.6.3 | javax.ws.rs.WebApplicationException: com.fasterxml.jackson.databind.JsonMappingException: Invalid type id 'org.soluvas.data.impl.TermValueImpl' ( for id type 'Id.class'): no such class found (through reference chain: id.co.bippo.product.impl.ProductImpl[ "attributes" ]) at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:243) at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:89) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262) at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:236) at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:213) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:154) at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:130) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:221) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:141) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:197) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.ops4j.pax.web.service.internal.HttpServiceStarted$2.invoke(HttpServiceStarted.java:210) at org.ops4j.pax.web.service.internal.$Proxy0.service(Unknown Source) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:447) at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:70) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:559) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1038) at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:117) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:374) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:972) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:74) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:363) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:931) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:992) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.run(BlockingChannelConnector.java:298) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang. Thread .run( Thread .java:722) Caused by: com.fasterxml.jackson.databind.JsonMappingException: Invalid type id 'org.soluvas.data.impl.TermValueImpl' ( for id type 'Id.class'): no such class found (through reference chain: id.co.bippo.product.impl.ProductImpl[ "attributes" ]) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:232) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:197) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1014) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:310) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1169) at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:625) at com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider.readFrom(JacksonJsonProvider.java:443) at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBody(JAXRSUtils.java:1038) at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameter(JAXRSUtils.java:614) at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameters(JAXRSUtils.java:578) at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:238) ... 43 more Caused by: java.lang.IllegalArgumentException: Invalid type id 'org.soluvas.data.impl.TermValueImpl' ( for id type 'Id.class'): no such class found at com.fasterxml.jackson.databind.jsontype.impl.ClassNameIdResolver.typeFromId(ClassNameIdResolver.java:56) at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:151) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:98) at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:104) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:228) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:203) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:23) at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:429) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:310) at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2768) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1492) at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:1317) at org.soluvas.data.ValueEMapDeserializer.deserialize(ValueEMapDeserializer.java:39) at org.soluvas.data.ValueEMapDeserializer.deserialize(ValueEMapDeserializer.java:1) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:375) at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:308) ... 51 more

Tatu Saloranta added a comment - 20/Feb/13 5:35 PM Will close as "wont implement" for 1.x (since such change would be risky for patch releases); if something is still needed for 2.x, should be re-filed at github 'jackson-databind' project.