jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
Signup
Jackson JSON Processor
  • Jackson JSON Processor
  • JACKSON-536

nested json via url references

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: New Feature New Feature
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Number of attachments :
    0

Description

see original conversation:
http://jackson-users.ning.com/forum/topics/nested-json-via-url-references

use case:

1) there is a need to store multiple related configurations in terracotta server
http://code.google.com/p/terrastore/
which is a clustered json document store

2) for example, this url would return full configurations:
http://server/configurations/conf_01
http://server/configurations/conf_02

3) and this url would return components of full configs:
http://server/components/comp_01
http://server/components/comp_02
http://server/components/comp_03

4) the idea is that full configurations can refer to the components like this:

conf_01.json:
{
// in this case the component definition is embedded in the parent file
compA :

{ a : 1, b : 2}

// and now component refers to external resource:
compB : http://server/components/comp_02
}

conf_02.json:
// both subcomponents are ferfferring to the external components resources

{ compA : http://server/components/comp_02 compB : http://server/components/comp_03 }

5) suggested basic logic: this is what comes to mind:

a) if the target java field type is string - do not try to dereference;
i.e. for:

class Test

{ String field; }

test1.json { field : this-any-string }

and

test2.json { field : http://this-looks-like-url }

will just produce string

b) if the target java field type is NOT string - DO dereference;

class Test { Address address; }
class Address { String field; }

// this is doing normal de-serialization
test1.json
{ address :

{ field : some-data }

}

test2.json

{ address : file://./external-resource.json }

c) too bad string properties can not be url-ed; may be there is way for that?

6) probably has to treat "basedir" or "./" specially, like in http references,
relative to the location of the current document. to avoid absolut urls and enforce
for example that config stays on the same server / in the same folder;

7) may be there is an "official markup" in json for this kind of thing?
or some js libraries like jquery do this in some kind of "standard way"?
I can not imagine I am the first to ask for this

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 8:40 AM - edited

and of course next natural idea which comes to mind is provide json inheritance, like maven pom.xml which has a parent, etc.

come on, if you serializing classes, then json docs also have inherit!

Show
Andrei Pozolotin added a comment - 19/Mar/11 8:40 AM - edited and of course next natural idea which comes to mind is provide json inheritance, like maven pom.xml which has a parent, etc. come on, if you serializing classes, then json docs also have inherit!
Hide
Permalink
Tatu Saloranta added a comment - 19/Mar/11 12:02 PM

Actually, Jackson supports object inheritance with polymorphic type information.

As to automatic resolution; I think this would work best as a property annotation, something like:

@JsonResolve // and maybe allow custom resolver; but default to one that assumes JSON String -> URL
public ConfigObject config;

which would require JSON String value, try to make URL out of it; and use it as relative or absolute (relative resolved via declared source, i.e. JsonParser MUST have been constructed with URL, File or such; may need to add a way to set "base URL", similar to XML.

Trying to auto-detect URLs from values is something that sounds like dangerous thing to do; I am not really sure that would be right thing to do. Of course one could offer a feature to enable such handling, so that if a JSON String is seen, and a POJO is expected, try resolution is a fallback.
This is highly speculative but... well, might be useful. If so, I'd probably want to add bit more generic callback to register. Challenge here is that Strings are valid things to pass to single-string-arg constructor, so would need to figure out what to do with that conflict.

Implementing this is in the edge of things that Jackson might offer; I am not yet 100% if it falls within scope or not. But it might, so why not think of how to do it.

Show
Tatu Saloranta added a comment - 19/Mar/11 12:02 PM Actually, Jackson supports object inheritance with polymorphic type information. As to automatic resolution; I think this would work best as a property annotation, something like: @JsonResolve // and maybe allow custom resolver; but default to one that assumes JSON String -> URL public ConfigObject config; which would require JSON String value, try to make URL out of it; and use it as relative or absolute (relative resolved via declared source, i.e. JsonParser MUST have been constructed with URL, File or such; may need to add a way to set "base URL", similar to XML. Trying to auto-detect URLs from values is something that sounds like dangerous thing to do; I am not really sure that would be right thing to do. Of course one could offer a feature to enable such handling, so that if a JSON String is seen, and a POJO is expected, try resolution is a fallback. This is highly speculative but... well, might be useful. If so, I'd probably want to add bit more generic callback to register. Challenge here is that Strings are valid things to pass to single-string-arg constructor, so would need to figure out what to do with that conflict. Implementing this is in the edge of things that Jackson might offer; I am not yet 100% if it falls within scope or not. But it might, so why not think of how to do it.
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 2:52 PM

re: "Actually, Jackson supports object inheritance with polymorphic type information."
not sure we mean the same thing?

what I mean is that child.json file can refer to parent.json and "override" subset of fields;
if this is the case, can you please link to example? thank you.

Show
Andrei Pozolotin added a comment - 19/Mar/11 2:52 PM re: "Actually, Jackson supports object inheritance with polymorphic type information." not sure we mean the same thing? what I mean is that child.json file can refer to parent.json and "override" subset of fields; if this is the case, can you please link to example? thank you.
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 2:55 PM - edited

re: "@JsonResolve" - this sounds good already; I do not think I could do it myself; I tried to delve into your source - that's too much!

also , I would assume that resolution is optional? that is, if definition is provided - used it
if not - try to go after url, if url lookup/parse fails - then really fail?

Show
Andrei Pozolotin added a comment - 19/Mar/11 2:55 PM - edited re: "@JsonResolve" - this sounds good already; I do not think I could do it myself; I tried to delve into your source - that's too much! also , I would assume that resolution is optional? that is, if definition is provided - used it if not - try to go after url, if url lookup/parse fails - then really fail?
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 2:59 PM

re: "need to add a way to set "base URL", similar to XML." you mean inside json content or as a feature/property to the mapper?

for my use case support limited for FILE & URL are just fine

Show
Andrei Pozolotin added a comment - 19/Mar/11 2:59 PM re: "need to add a way to set "base URL", similar to XML." you mean inside json content or as a feature/property to the mapper? for my use case support limited for FILE & URL are just fine
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 3:01 PM

re: "feature to enable such handling" - feature sounds good, and protects who feel endangered

Show
Andrei Pozolotin added a comment - 19/Mar/11 3:01 PM re: "feature to enable such handling" - feature sounds good, and protects who feel endangered
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 3:01 PM

re: "add bit more generic callback to register" sorry this part I do not follow

Show
Andrei Pozolotin added a comment - 19/Mar/11 3:01 PM re: "add bit more generic callback to register" sorry this part I do not follow
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 3:04 PM

re: "so why not think of how to do it" - well if you do not see how to do it, I am at loss

but why dont you try make something easy enough for you to make, I will test if for my cases,
and I will produce bunch of unit tests for you!

or link me to exmaples so I continue do digest your sources...

Show
Andrei Pozolotin added a comment - 19/Mar/11 3:04 PM re: "so why not think of how to do it" - well if you do not see how to do it, I am at loss but why dont you try make something easy enough for you to make, I will test if for my cases, and I will produce bunch of unit tests for you! or link me to exmaples so I continue do digest your sources...
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 3:08 PM

also, I noticed you use buch of custom tags in your json files like @class@;
I am not sure were you just thinking about or actually using them;

regardless, probably thats another way to differentiate?

Show
Andrei Pozolotin added a comment - 19/Mar/11 3:08 PM also, I noticed you use buch of custom tags in your json files like @class@; I am not sure were you just thinking about or actually using them; regardless, probably thats another way to differentiate?
Hide
Permalink
Andrei Pozolotin added a comment - 19/Mar/11 3:11 PM

finally, why do you think it is dangerous? you are like #1 expert on json now;
why dont you just create new tag/feature/etc in json
and tell the world this is how json will be now

Show
Andrei Pozolotin added a comment - 19/Mar/11 3:11 PM finally, why do you think it is dangerous? you are like #1 expert on json now; why dont you just create new tag/feature/etc in json and tell the world this is how json will be now
Hide
Permalink
Tatu Saloranta added a comment - 21/Mar/11 12:16 PM

Heh. Ok, danger just comes from the fact that automatic resolution may be highly surprising to most users. Also, it could allow actual security compromises, if you can make a JSON document that will then automatically make a request to a URL contained – this has been abused with XML and DTDs already.

As to inventing new tags; yes and no: these are tags that are added via annotations so user specifies them, although there are defaults.

But yes, ability to indicate that a JSON String that looks like a URL reference should be used to include referenced document does sound like a potentially useful feature. Especially since there are no JSON standards to define how to do it.

Show
Tatu Saloranta added a comment - 21/Mar/11 12:16 PM Heh. Ok, danger just comes from the fact that automatic resolution may be highly surprising to most users. Also, it could allow actual security compromises, if you can make a JSON document that will then automatically make a request to a URL contained – this has been abused with XML and DTDs already. As to inventing new tags; yes and no: these are tags that are added via annotations so user specifies them, although there are defaults. But yes, ability to indicate that a JSON String that looks like a URL reference should be used to include referenced document does sound like a potentially useful feature. Especially since there are no JSON standards to define how to do it.
Hide
Permalink
Andrei Pozolotin added a comment - 21/Mar/11 4:35 PM

do you think it is appropriate to ask people on the jackson forums if they want this feature?

Show
Andrei Pozolotin added a comment - 21/Mar/11 4:35 PM do you think it is appropriate to ask people on the jackson forums if they want this feature?
Hide
Permalink
Tatu Saloranta added a comment - 21/Mar/11 6:19 PM

Sure. They can also vote on Jira issues, in case they want to encourage implementation of specific things – considering they are dozens of suggested things, order of implementation can be influenced by users indicating kinds of things they want to see done.

Show
Tatu Saloranta added a comment - 21/Mar/11 6:19 PM Sure. They can also vote on Jira issues, in case they want to encourage implementation of specific things – considering they are dozens of suggested things, order of implementation can be influenced by users indicating kinds of things they want to see done.
Hide
Permalink
Andrei Pozolotin added a comment - 10/Apr/11 8:10 AM

another use case:
http://wiki.ops4j.org/pages/viewpage.action?pageId=12648754

Show
Andrei Pozolotin added a comment - 10/Apr/11 8:10 AM another use case: http://wiki.ops4j.org/pages/viewpage.action?pageId=12648754
Hide
Permalink
Andrei Pozolotin added a comment - 23/Apr/11 12:32 PM

Tatu:
I made like 3 attempts on this - but got lost in jackson jungle every time ;
any chance to get some basic sample from you so I could imporve on?
thank you;
Andrei.

Show
Andrei Pozolotin added a comment - 23/Apr/11 12:32 PM Tatu: I made like 3 attempts on this - but got lost in jackson jungle every time ; any chance to get some basic sample from you so I could imporve on? thank you; Andrei.
Hide
Permalink
Tatu Saloranta added a comment - 24/Apr/11 12:17 AM

I can try something in 2 weeks or so, after I come back from my vacation.

Show
Tatu Saloranta added a comment - 24/Apr/11 12:17 AM I can try something in 2 weeks or so, after I come back from my vacation.
Hide
Permalink
Andrei Pozolotin added a comment - 24/Apr/11 12:24 AM

great!

Show
Andrei Pozolotin added a comment - 24/Apr/11 12:24 AM great!
Hide
Permalink
Andrei Pozolotin added a comment - 02/Aug/11 12:46 PM

few more use cases:

http://jfig.sourceforge.net/

http://code.google.com/p/jconfig/

http://commons.apache.org/configuration/userguide/howto_xml.html

Show
Andrei Pozolotin added a comment - 02/Aug/11 12:46 PM few more use cases: http://jfig.sourceforge.net/ http://code.google.com/p/jconfig/ http://commons.apache.org/configuration/userguide/howto_xml.html
Hide
Permalink
Andrei Pozolotin added a comment - 02/Aug/11 12:50 PM

acutally, JACKSON-538 could be used for nested de-serialization

Show
Andrei Pozolotin added a comment - 02/Aug/11 12:50 PM acutally, JACKSON-538 could be used for nested de-serialization

People

  • Assignee:
    Tatu Saloranta
    Reporter:
    Andrei Pozolotin
Vote (0)
Watch (1)

Dates

  • Created:
    19/Mar/11 8:33 AM
    Updated:
    02/Aug/11 12:50 PM
  • Atlassian JIRA (v5.2.7#850-sha1:b2af0c8)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.