Jackson JSON Processor

DataBinding : allow private fields processing without having to annotate them

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.4.2
  • Fix Version/s: 1.5
  • Component/s: JSON tree (JsonNode)
  • Labels:
    None
  • Environment:
    Jackson 1.4.2
  • Number of attachments :
    0

Description

Hello Jackson team !

First, thank you for creating this fast Json processing framework. I recently switched from Gson to Jackson on an Android project and the results outperform my expectations (Gson creates tons of temporary objects, and Android GC sucks).

One thing I enjoy with Gson is to be able to serialize/deserialize entities without no fwk specific configuration. I can create pojos with private fields + getters, a private constructor and no setters, and Gson is able to deserialize my entities from Json.

Jackson however requires the fields to either be public, have a specific annotation (@JsonProperty), or have the corresponding setter.

Just to be clear : I'd like to be able to serialize / deserialize such a class :

public class MyBean {
private String name;

public String getName() { return name; }
}

It would be nice if there was an option, disabled as default, to allow private fields with no annotation to be auto detected. Something like (De)SerializationConfig.Feature.AUTO_DETECT_PRIVATE_FIELDS

I only looked at Deserialization for now since it's what I need. I think the main changes would occur in BasicBeanDescription._findPropertyFields (l 762) :

if (!autodetect || !af.isPublic()) { continue; }

=> would become something like

if (!autodetect || (!af.isPublic() && !autodetect_private_fields)) { continue; } }

This feature would be nice for prototyping, and quickly being able to parse Json. It would also help migrating from Gson.

I managed to enable this feature when deserializing Json with Jackson 1.4.2, but the code is dirty... here is what I did anyway :

  • Create a subclass of BasicBeanDescription that overrides _findPropertyFields to remove the isPublic check.
  • Create a subclass of BasicClassIntrospector that overrides forDeserialization method and return my custom BasicBeanDescription instead of a standard BasicBeanDescription.
  • Create an instance of DeserializationConfig injected with the custom BasicClassIntrospector, and inject it in the ObjectMapper.

Here is the ugly code :

ObjectMapper om = new ObjectMapper();

BasicClassIntrospector bci = new BasicClassIntrospector() {
@Override
public BasicBeanDescription forDeserialization(DeserializationConfig cfg, JavaType type,
org.codehaus.jackson.map.ClassIntrospector.MixInResolver r) {
final AnnotationIntrospector ai = cfg.getAnnotationIntrospector();
final AnnotatedClass ac = AnnotatedClass.construct(type.getRawClass(), ai, r);
// everything needed for deserialization, including ignored
// methods
ac.resolveMemberMethods(getDeserializationMethodFilter(cfg), true);
// include all kinds of creator methods:
ac.resolveCreators(true);
// yes, we need info on ignored fields as well
ac.resolveFields(true);
return new BasicBeanDescription(type, ac, ai) {

@Override
public LinkedHashMap<String, AnnotatedField> _findPropertyFields(boolean autodetect,
Collection<String> ignoredProperties, boolean forSerialization) {
Boolean classAD = ai.findFieldAutoDetection(ac);
if (classAD != null) { autodetect = classAD.booleanValue(); }

LinkedHashMap<String, AnnotatedField> results = new LinkedHashMap<String, AnnotatedField>();
for (AnnotatedField af : ac.fields()) {
/*

  • note: some prefiltering has been; no static or
  • transient fields included; nor anything marked as
  • ignorable (@JsonIgnore)
    */

/*

  • So far so good: final check, then; has to either
  • (a) be marked with JsonProperty (or
  • JsonSerialize) OR (b) be public
    */
    String propName = forSerialization ? ai.findSerializablePropertyName(af) : ai
    .findDeserializablePropertyName(af);
    if (propName != null)
    Unknown macro: { if (propName.length() == 0) { propName = af.getName(); } }
    else
    Unknown macro: { // nope, but is a public field? // ****** MY CHANGE }

    }

/*

  • Yup, it is a valid name. But do we have a
  • conflict? Shouldn't usually happen, but it is
  • possible... and for now let's consider it a
  • problem
    */
    AnnotatedField old = results.put(propName, af);
    if (old != null) { String oldDesc = old.getFullName(); String newDesc = af.getFullName(); throw new IllegalArgumentException("Multiple fields representing property \"" + propName + "\": " + oldDesc + " vs " + newDesc); }
    }
    return results;
    }

};
}
};

DeserializationConfig deserConfig = new DeserializationConfig(bci, new JacksonAnnotationIntrospector());

om.setDeserializationConfig(deserConfig);

om.configure(DeserializationConfig.Feature.AUTO_DETECT_FIELDS, true);
om.configure(DeserializationConfig.Feature.CAN_OVERRIDE_ACCESS_MODIFIERS, true);

If you find any cleaner way, please let me know !

Activity

Hide
Tatu Saloranta added a comment -

Thank you for suggestion! (and sample code)
I am glad Jackson was worked well so far, aside from auto-detection. Performance has been an important design criteria, and part of that has been to try to keep processing streamline (including avoiding unnecessary garbage object creation).

This issue (only being able to auto-detect public fields) has been on our radar, but no Jira entry had been added.
My initial thought was to add visibility setting, to allow defining visibility level a field should have to qualify as property. Current default would then be public, but could be changed to private/protected (and maybe package, and 'none').
As an added bonus, maybe there could also be additional property for @JsonAutoDetect, to allow for per-class override (mostly useful when used via annotation mix-ins).

Would this work? I think this could be added for 1.6 (1.5 release is close and feature set is frozen). It really has more to do with how to configure it, like you say, implementation should be easy.
The only other thing is that there are plans to rewrite property-discovery and annotation-introspection pieces, so this could naturally fit within that work.

Show
Tatu Saloranta added a comment - Thank you for suggestion! (and sample code) I am glad Jackson was worked well so far, aside from auto-detection. Performance has been an important design criteria, and part of that has been to try to keep processing streamline (including avoiding unnecessary garbage object creation). This issue (only being able to auto-detect public fields) has been on our radar, but no Jira entry had been added. My initial thought was to add visibility setting, to allow defining visibility level a field should have to qualify as property. Current default would then be public, but could be changed to private/protected (and maybe package, and 'none'). As an added bonus, maybe there could also be additional property for @JsonAutoDetect, to allow for per-class override (mostly useful when used via annotation mix-ins). Would this work? I think this could be added for 1.6 (1.5 release is close and feature set is frozen). It really has more to do with how to configure it, like you say, implementation should be easy. The only other thing is that there are plans to rewrite property-discovery and annotation-introspection pieces, so this could naturally fit within that work.
Hide
Pierre-Yves Ricau added a comment - - edited

I would be great ! Of course, being able to customize it even more is a good idea !

1.6 is OK for me. In fact, in "real" projects, I prefer using annotations with the name attribute set to prevent refactoring mistakes. But such features are good to let users start quickly with Jackson. "I create my pojo, I put Jackson, and voila, it works !".

One last thing : it took me quite some time to understand how to get DataBinding working the right way. The 5min tutorial is good, but then I got lost when trying to customize the behavior and the binding for some specific classes (such as some Enums). Once you understand, it's easy, but Jackson is so customizable that it's hard to know where to look in the JavaDocs.

By the way, thanks again for doing this great job !

Show
Pierre-Yves Ricau added a comment - - edited I would be great ! Of course, being able to customize it even more is a good idea ! 1.6 is OK for me. In fact, in "real" projects, I prefer using annotations with the name attribute set to prevent refactoring mistakes. But such features are good to let users start quickly with Jackson. "I create my pojo, I put Jackson, and voila, it works !". One last thing : it took me quite some time to understand how to get DataBinding working the right way. The 5min tutorial is good, but then I got lost when trying to customize the behavior and the binding for some specific classes (such as some Enums). Once you understand, it's easy, but Jackson is so customizable that it's hard to know where to look in the JavaDocs. By the way, thanks again for doing this great job !
Hide
Tatu Saloranta added a comment -

Great, thank you very much for feedback. I agree that documentation needs more work – I realized early that some kind of tutorial was needed. But there hasn't been enough progress with follow-up advanced/intermediate docs. And especially something that would connect features, it's all split up more according to feature set (esp. via releases) and not by usage cases.

This is also one area where we are hoping to get others to contribute with ideas or documentation, so we appreciate all help (and can give access if you ever need it).

I also agree in that this is one of features that would greatly help straight-forward "no-hassle" object serialization. Jackson has sort of grown into handling pure object serialization case from "pure data binding" background, which will allow it to be used in more cases. That's big reason for gaps that appear (when some part of functionality has been added, and supporting features not yet).

Lastly: I would also be interested in your experiences with Android. I have occasionally heard from users, and resolved some issues. But I have not developed actual apps on Android myself, just played with the emulator. This makes it more challenging to ensure compatibility, but with some feedback it should be doable.

Show
Tatu Saloranta added a comment - Great, thank you very much for feedback. I agree that documentation needs more work – I realized early that some kind of tutorial was needed. But there hasn't been enough progress with follow-up advanced/intermediate docs. And especially something that would connect features, it's all split up more according to feature set (esp. via releases) and not by usage cases. This is also one area where we are hoping to get others to contribute with ideas or documentation, so we appreciate all help (and can give access if you ever need it). I also agree in that this is one of features that would greatly help straight-forward "no-hassle" object serialization. Jackson has sort of grown into handling pure object serialization case from "pure data binding" background, which will allow it to be used in more cases. That's big reason for gaps that appear (when some part of functionality has been added, and supporting features not yet). Lastly: I would also be interested in your experiences with Android. I have occasionally heard from users, and resolved some issues. But I have not developed actual apps on Android myself, just played with the emulator. This makes it more challenging to ensure compatibility, but with some feedback it should be doable.
Hide
Tatu Saloranta added a comment -

Realized that there actually already was an entry for this (JACKSON-213); but since this has more info, closing that one as duplicate.

Show
Tatu Saloranta added a comment - Realized that there actually already was an entry for this (JACKSON-213); but since this has more info, closing that one as duplicate.
Hide
Pierre-Yves Ricau added a comment - - edited

On the Android part, it works great, but there are still some issues, due to the fact that Android doesn't ship with all the standard J2SE classes.

Here is what the logs say when an application tries to use Jackson (core + mapper) :

E/dalvikvm( 763): Could not find method javax.xml.datatype.DatatypeFactory.newInstance, referenced from method org.codehaus.jackson.map.ext.CoreXMLDeserializers.<clinit>
W/dalvikvm( 763): VFY: unable to resolve static method 3437: Ljavax/xml/datatype/DatatypeFactory;.newInstance ()Ljavax/xml/datatype/DatatypeFactory;
W/dalvikvm( 763): VFY: rejecting opcode 0x71 at 0x0000
W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/CoreXMLDeserializers;.<clinit> ()V
W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/CoreXMLDeserializers;
E/dalvikvm( 763): Could not find class 'org.joda.time.DateTime', referenced from method org.codehaus.jackson.map.ext.JodaDeserializers$DateTimeDeserializer.<init>
W/dalvikvm( 763): VFY: unable to resolve const-class 1158 (Lorg/joda/time/DateTime in Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer;
W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x0000
W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer;.<init> ()V
W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer;
E/dalvikvm( 763): Could not find class 'javax.xml.datatype.Duration', referenced from method org.codehaus.jackson.map.ext.CoreXMLSerializers.<clinit>
W/dalvikvm( 763): VFY: unable to resolve const-class 764 (Ljavax/xml/datatype/Duration in Lorg/codehaus/jackson/map/ext/CoreXMLSerializers;
W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x000b
W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/CoreXMLSerializers;.<clinit> ()V
W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/CoreXMLSerializers;
E/dalvikvm( 763): Could not find class 'org.joda.time.DateTime', referenced from method org.codehaus.jackson.map.ext.JodaSerializers.<clinit>
W/dalvikvm( 763): VFY: unable to resolve const-class 1158 (Lorg/joda/time/DateTime in Lorg/codehaus/jackson/map/ext/JodaSerializers;
W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x0009
W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/JodaSerializers;.<clinit> ()V
W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/JodaSerializers;

3 classes are not found :

javax.xml.datatype.DatatypeFactory (in J2SE but not in Android)
javax.xml.datatype.Duration(in J2SE but not in Android)
org.joda.time.DateTime (is JodaTime a Jackson dependency ?)

Please notice that even with those warnings, Jackson seems to work like a charm.

BTW, shouldn't this belong to a separate issue ?

Show
Pierre-Yves Ricau added a comment - - edited On the Android part, it works great, but there are still some issues, due to the fact that Android doesn't ship with all the standard J2SE classes. Here is what the logs say when an application tries to use Jackson (core + mapper) : E/dalvikvm( 763): Could not find method javax.xml.datatype.DatatypeFactory.newInstance, referenced from method org.codehaus.jackson.map.ext.CoreXMLDeserializers.<clinit> W/dalvikvm( 763): VFY: unable to resolve static method 3437: Ljavax/xml/datatype/DatatypeFactory;.newInstance ()Ljavax/xml/datatype/DatatypeFactory; W/dalvikvm( 763): VFY: rejecting opcode 0x71 at 0x0000 W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/CoreXMLDeserializers;.<clinit> ()V W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/CoreXMLDeserializers; E/dalvikvm( 763): Could not find class 'org.joda.time.DateTime', referenced from method org.codehaus.jackson.map.ext.JodaDeserializers$DateTimeDeserializer.<init> W/dalvikvm( 763): VFY: unable to resolve const-class 1158 (Lorg/joda/time/DateTime in Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer; W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x0000 W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer;.<init> ()V W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/JodaDeserializers$DateTimeDeserializer; E/dalvikvm( 763): Could not find class 'javax.xml.datatype.Duration', referenced from method org.codehaus.jackson.map.ext.CoreXMLSerializers.<clinit> W/dalvikvm( 763): VFY: unable to resolve const-class 764 (Ljavax/xml/datatype/Duration in Lorg/codehaus/jackson/map/ext/CoreXMLSerializers; W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x000b W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/CoreXMLSerializers;.<clinit> ()V W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/CoreXMLSerializers; E/dalvikvm( 763): Could not find class 'org.joda.time.DateTime', referenced from method org.codehaus.jackson.map.ext.JodaSerializers.<clinit> W/dalvikvm( 763): VFY: unable to resolve const-class 1158 (Lorg/joda/time/DateTime in Lorg/codehaus/jackson/map/ext/JodaSerializers; W/dalvikvm( 763): VFY: rejecting opcode 0x1c at 0x0009 W/dalvikvm( 763): VFY: rejected Lorg/codehaus/jackson/map/ext/JodaSerializers;.<clinit> ()V W/dalvikvm( 763): Verifier rejected class Lorg/codehaus/jackson/map/ext/JodaSerializers; 3 classes are not found : javax.xml.datatype.DatatypeFactory (in J2SE but not in Android) javax.xml.datatype.Duration(in J2SE but not in Android) org.joda.time.DateTime (is JodaTime a Jackson dependency ?) Please notice that even with those warnings, Jackson seems to work like a charm. BTW, shouldn't this belong to a separate issue ?
Hide
Tatu Saloranta added a comment -

Right: all those dependencies are optional (Joda in general, javax.xml on platforms that do not include full J2SE 1.5; which includes Android, and possibly GAE). They are not mandatory dependencies.
Jackson code does its best to handle these gracefully, but somehow it seems like Android exposes these via logging. Seems like an "undesired feature" to me – trying to load classes that don't exist is a valid operation on JVM, not an exceptional case (as long as resulting exceptions are caught, of course).
With some earlier versions these were not gracefully handled, and Jackson could not be used on Android.

So: it is working as designed from Jackson point of view. But if there are ways to remove noise by rewriting some part of code please let me know.

Show
Tatu Saloranta added a comment - Right: all those dependencies are optional (Joda in general, javax.xml on platforms that do not include full J2SE 1.5; which includes Android, and possibly GAE). They are not mandatory dependencies. Jackson code does its best to handle these gracefully, but somehow it seems like Android exposes these via logging. Seems like an "undesired feature" to me – trying to load classes that don't exist is a valid operation on JVM, not an exceptional case (as long as resulting exceptions are caught, of course). With some earlier versions these were not gracefully handled, and Jackson could not be used on Android. So: it is working as designed from Jackson point of view. But if there are ways to remove noise by rewriting some part of code please let me know.
Hide
Tatu Saloranta added a comment -

For some odd reason I thought this would be easy to implement, and started happily hacking away. I was wrong, but only realized this late enough to have done most of the work anyway.
So: this has now been implemented, will be part of 1.5.0.

There are two ways to change default minimum visibility levels:

(a) Change global baseline settings via ObjectMapper.setVisibilityChecker(); check out builder/fluid pattern that VisibilityChecker uses. This is rather extensible, allowing fully customized rules if anyone wants; default implementation should be fine most common cases.
(b) Use new properties of @JsonAutoDetect (getterVisibility=Visibility.PUBLIC_ONLY, setterVisibility= .... ) to define this for specified class (and its sub-classes).

So, this should allow auto-detecting things using more granular settings.

Show
Tatu Saloranta added a comment - For some odd reason I thought this would be easy to implement, and started happily hacking away. I was wrong, but only realized this late enough to have done most of the work anyway. So: this has now been implemented, will be part of 1.5.0. There are two ways to change default minimum visibility levels: (a) Change global baseline settings via ObjectMapper.setVisibilityChecker(); check out builder/fluid pattern that VisibilityChecker uses. This is rather extensible, allowing fully customized rules if anyone wants; default implementation should be fine most common cases. (b) Use new properties of @JsonAutoDetect (getterVisibility=Visibility.PUBLIC_ONLY, setterVisibility= .... ) to define this for specified class (and its sub-classes). So, this should allow auto-detecting things using more granular settings.
Hide
Pierre-Yves Ricau added a comment -

Thanks a lot for fixing this !

Show
Pierre-Yves Ricau added a comment - Thanks a lot for fixing this !
Hide
Tatu Saloranta added a comment -

Was included in 1.5.0

Show
Tatu Saloranta added a comment - Was included in 1.5.0

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: