GeoServer
  1. GeoServer
  2. GEOS-1411

Allow for optional callback parameter in GET requests that return json formatted structures

    Details

    • Type: New Feature New Feature
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.3
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      0

      Description

      Any GET request that returns JSON can be made into a flexible cross-domain accessible resource by accepting a "callback" parameter.

      JSON wrapped in parentheses and prepended by some callback can become a JavaScript expression - and added dynamically to an application by appending a script tag.

      Examples:

      A current resource that responds to GET and accepts format=json or something similar

      Request (abbreviated) -
      GET /path/to/resource?format=json

      Response (abbreviated) -
      Content-Type: application/json

      {"key": "value"}

      If this resource accepts a callback parameter, things might look like this

      Request -
      GET /path/to/resource?format=json&callback=someFunc

      Response
      Content-Type: text/javascript

      someFunc(

      {"key": "value"}

      )

      So, the three steps are:
      1) wrap JSON in parentheses
      2) prepend with the callback string (any arbitrary string)
      3) set content type to "text/javascript"

      This allows people to specify function names as callbacks. It also allows something like

      GET /path/to/resource?format=json&callback=var%20someVariable%20%3D%20

      and the response would look like

      var someVariable = (

      {"key": "value"}

      )

      (note that it is fine that the statement doesn't end with a semicolon in that case - it still makes for valid script contents)

        Issue Links

          Activity

          Hide
          Andrea Aime added a comment -

          Hum, it seems something doable. Excuse my ignorance, but why is this needed? Can't you just code them in your javascript directly?
          I guess it's the "cross domain" bit that's tripping me off.
          Any time frame for this one, when do you need it?

          Show
          Andrea Aime added a comment - Hum, it seems something doable. Excuse my ignorance, but why is this needed? Can't you just code them in your javascript directly? I guess it's the "cross domain" bit that's tripping me off. Any time frame for this one, when do you need it?
          Hide
          Tim Schaub added a comment -

          The motivation for this is the "same origin policy." In short, when we make XMLHttpRequest-s client side (the asynchronous or synchronous requests that are the x in ajax), we are restricted to making requests to the same origin as the original document was served from. Same origin means same protocol, same host, and same port.

          So, if I have an application running from http://example.com I can get json, xml, text, or whatever via http from example.com on port 80 - and from nowhere else. Exceptions to this same origin policy are requests for images (img tags can point to other origins), stylesheets (ling tags can be loaded from elsewhere), and scripts (script tags can point to a src from another origin). This last one can be exploited.

          Instead of being restricted to requesting json from a single domain, an application can request json wrapped in parens and with a callback (which makes it javascript instead of just json) from any domain. So with a geoserver running on port 12345 on geoserver.org, I can serve up an application from example.com that fetches json (wrapped in a callback) from that resource. This is very handy. It means that resources don't have to be duplicated. One resource can be accessed from many many places. This makes json (wrapped in a callback) much more appealing than xml (which is always restricted by the same origin policy).

          That said, this is not a priority for anyone right now. However, as long as a server is not serving up static json (i.e. it is doing some work to create that json), it might as well accept a callback parameter and make that resource much more accessible.

          Search for jsonp for more info. Or check out the yahoo json services.

          Show
          Tim Schaub added a comment - The motivation for this is the "same origin policy." In short, when we make XMLHttpRequest-s client side (the asynchronous or synchronous requests that are the x in ajax), we are restricted to making requests to the same origin as the original document was served from. Same origin means same protocol, same host, and same port. So, if I have an application running from http://example.com I can get json, xml, text, or whatever via http from example.com on port 80 - and from nowhere else. Exceptions to this same origin policy are requests for images (img tags can point to other origins), stylesheets (ling tags can be loaded from elsewhere), and scripts (script tags can point to a src from another origin). This last one can be exploited. Instead of being restricted to requesting json from a single domain, an application can request json wrapped in parens and with a callback (which makes it javascript instead of just json) from any domain. So with a geoserver running on port 12345 on geoserver.org, I can serve up an application from example.com that fetches json (wrapped in a callback) from that resource. This is very handy. It means that resources don't have to be duplicated. One resource can be accessed from many many places. This makes json (wrapped in a callback) much more appealing than xml (which is always restricted by the same origin policy). That said, this is not a priority for anyone right now. However, as long as a server is not serving up static json (i.e. it is doing some work to create that json), it might as well accept a callback parameter and make that resource much more accessible. Search for jsonp for more info. Or check out the yahoo json services.
          Hide
          Andrea Aime added a comment -

          Aah, thanks for the explaination and links. Now we need someone to schedule this one for some release...
          Chris?

          Show
          Andrea Aime added a comment - Aah, thanks for the explaination and links. Now we need someone to schedule this one for some release... Chris?
          Hide
          Chris Holmes added a comment -

          Scheduled for my favorite release! But I'll probably up this to 1.6.1 or 1.6.2 when those come.

          Show
          Chris Holmes added a comment - Scheduled for my favorite release! But I'll probably up this to 1.6.1 or 1.6.2 when those come.
          Hide
          Tim Schaub added a comment -

          I think it's probably clear from the above, but I'll add in one more comment anyway.

          People who program simple web applications like WMS. This is because a GetMap request returns an image - and an image request can be made cross-domain (not restricted by the same origin policy). This means if I'm a cheapskate and serve up an application from some shared hosting environment where I have very little control over the server (can't run cgi for example), I can still have full access to the WMS resource at NASA JPL. That's pretty nice - a full global mosaic of public satellite imagery accessible from anywhere.

          On the other hand, WFS typically speaks in XML. This means that it is subject to the same origin policy. So my cheapskate mapping application can't make use of some cool WFS resource hosted elsewhere (because I'd have to set up a proxy to get it from the same origin). However, if simple WFS responses were structured as JSON - and if those WFS resources allowed clients to specify a callback, then they would instantly become usable from anywhere.

          To be overly detailed, here's what the two styles look like:

          // traditional xhr request - restricted to same origin
          function myCallback(request) {
              // handle state change
          }
          var client = new XMLHttpRequest();
          client.onreadystatechange = myCallback;
          client.open("GET", "/geoserver/wfs?request=getFeature&...");
          client.send();
          
          // json-p - not restricted to same origin
          function myCallback(response) {
              // deal with response
          }
          var client = document.createElement("script");
          client.src = "http://some.host/geoserver/wfs?request=getFeature&format=json&callback=myCallback&...";
          document.body.appendChild(client);
          
          Show
          Tim Schaub added a comment - I think it's probably clear from the above, but I'll add in one more comment anyway. People who program simple web applications like WMS. This is because a GetMap request returns an image - and an image request can be made cross-domain (not restricted by the same origin policy). This means if I'm a cheapskate and serve up an application from some shared hosting environment where I have very little control over the server (can't run cgi for example), I can still have full access to the WMS resource at NASA JPL. That's pretty nice - a full global mosaic of public satellite imagery accessible from anywhere. On the other hand, WFS typically speaks in XML. This means that it is subject to the same origin policy. So my cheapskate mapping application can't make use of some cool WFS resource hosted elsewhere (because I'd have to set up a proxy to get it from the same origin). However, if simple WFS responses were structured as JSON - and if those WFS resources allowed clients to specify a callback, then they would instantly become usable from anywhere. To be overly detailed, here's what the two styles look like: // traditional xhr request - restricted to same origin function myCallback(request) { // handle state change } var client = new XMLHttpRequest(); client.onreadystatechange = myCallback; client.open("GET", "/geoserver/wfs?request=getFeature&..."); client.send(); // json-p - not restricted to same origin function myCallback(response) { // deal with response } var client = document.createElement("script"); client.src = "http://some.host/geoserver/wfs?request=getFeature&format=json&callback=myCallback&..."; document.body.appendChild(client);
          Hide
          Andrea Aime added a comment -

          Fixed on 1.7.x and trunk.
          In order to make it work, you have to append
          &format_options=callback:myCallback
          to the request.

          I also updated the documentation here:
          http://geoserver.org/display/GEOSDOC/GeoJSON+Output+Format

          Please try it out (you can contact me and I'll stand up a GeoServer with the patch that you can hit). Also, can you review the docs?

          Show
          Andrea Aime added a comment - Fixed on 1.7.x and trunk. In order to make it work, you have to append &format_options=callback:myCallback to the request. I also updated the documentation here: http://geoserver.org/display/GEOSDOC/GeoJSON+Output+Format Please try it out (you can contact me and I'll stand up a GeoServer with the patch that you can hit). Also, can you review the docs?
          Hide
          Amos Hayes added a comment -

          Wow. This is great. Looking forward to 1.7.3! Thanks Andrea and Tim!

          Show
          Amos Hayes added a comment - Wow. This is great. Looking forward to 1.7.3! Thanks Andrea and Tim!
          Hide
          Andrea Aime added a comment -

          Amos, if you want to try that out, you can already grab a nightly build at:
          http://gridlock.openplans.org/geoserver/1.7.x/

          I was also looking for feedback, if you can try and tell me if that works for you it would be great, as I'm not much of a javascript coder.

          Show
          Andrea Aime added a comment - Amos, if you want to try that out, you can already grab a nightly build at: http://gridlock.openplans.org/geoserver/1.7.x/ I was also looking for feedback, if you can try and tell me if that works for you it would be great, as I'm not much of a javascript coder.
          Hide
          Tim Schaub added a comment -

          This is cool. Thanks for working this in. I was reminded of a security issue that this brings up. I apologize for not thinking this through before. I know it is work to add a config option in the UI - so we should consider how serious this issue is before releasing this.

          Here's a description of what could happen (thanks to crschmidt for clearly laying out the problem):

          Joe works at the CIA. He has access to highly restricted data at http://10.0.0.1/geoserver. While at work, he browses to evil-hacker.com. The people at evil-hacker.com are very clever and are waiting for a GeoServer user with highly restricted data on 10.0.0.1. They have a page on their site that makes a script request:

          <script src="http://10.0.0.1/geoserver/wfs?callback=exploit&..."></script>

          This request goes out to 10.0.0.1 with Joe's credentials. The response is a script that is running on a page from an untrusted domain (evil-hacker.com). The folks at evil-hacker.com have written the "exploit" function to post whatever arguments it is called with back to evil-hacker.com (same origin, no problem). Now evil-hacker.com has whatever Joe got from his restricted server.

          So, this requires two things:
          1) That some hacker sets up a page that guesses the URL of a non-public GeoServer.
          2) That someone with access to that GeoServer browses to the hacker's page.

          This is probably a big enough concern for people with highly restricted data that they would want to turn off that callback option.

          Show
          Tim Schaub added a comment - This is cool. Thanks for working this in. I was reminded of a security issue that this brings up. I apologize for not thinking this through before. I know it is work to add a config option in the UI - so we should consider how serious this issue is before releasing this. Here's a description of what could happen (thanks to crschmidt for clearly laying out the problem): Joe works at the CIA. He has access to highly restricted data at http://10.0.0.1/geoserver . While at work, he browses to evil-hacker.com. The people at evil-hacker.com are very clever and are waiting for a GeoServer user with highly restricted data on 10.0.0.1. They have a page on their site that makes a script request: <script src="http://10.0.0.1/geoserver/wfs?callback=exploit&..."></script> This request goes out to 10.0.0.1 with Joe's credentials. The response is a script that is running on a page from an untrusted domain (evil-hacker.com). The folks at evil-hacker.com have written the "exploit" function to post whatever arguments it is called with back to evil-hacker.com (same origin, no problem). Now evil-hacker.com has whatever Joe got from his restricted server. So, this requires two things: 1) That some hacker sets up a page that guesses the URL of a non-public GeoServer. 2) That someone with access to that GeoServer browses to the hacker's page. This is probably a big enough concern for people with highly restricted data that they would want to turn off that callback option.
          Hide
          Andrea Aime added a comment -

          Ugh... at the moment we a ton of "nice" features that a security conscious user should be able to turn off, and no way to do so. Waiting for the new catalog and new UI to kick in to work on that.
          Let's open a related jira to add a flag to turn off this functionality when we have enough work donein the catalog/new ui to fix this? Or else, I can rollback the patch.

          Show
          Andrea Aime added a comment - Ugh... at the moment we a ton of "nice" features that a security conscious user should be able to turn off, and no way to do so. Waiting for the new catalog and new UI to kick in to work on that. Let's open a related jira to add a flag to turn off this functionality when we have enough work donein the catalog/new ui to fix this? Or else, I can rollback the patch.
          Hide
          Imran Rajjad added a comment -

          the code mentioned on this page is not working, its giving an INVALID LABEL error in FireFox when result is appended to script element

          http://geoserver.org/display/GEOSDOC/GeoJSON+Output+Format

          my url request is : http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON
          error :

          Error: invalid label
          Source File: http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON
          Line: 1, Column: 1
          Source Code:
          Error: invalid label
          Source File: http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON
          Line: 1, Column: 1
          Source Code:
          {"type":"FeatureCollection","features":[{"type":"Feature","id":"states.1","geometry":{"type":"MultiPolygon","coordinates":[[[[37.51099000000001,-88.071564],[37.476273000000006,-88.087883],[37.442852,-88.311707],[37.40930899999999,-88.359177],[37.420292,-8....................

          please note that the function processJSON function never gets called and after debuging it was found that the response has been directly aded to page where the browser cant understand it. I have tried several JSON retreiving libraries but its the same error everywhere.

          the code :

          function loadJSON(url)

          { var headID = document.getElementsByTagName("head")[0]; var newScript = document.createElement("script"); newScript.type = 'text/javascript'; newScript.src = url; headID.appendChild(newScript); }

          function processJSON(jsonData)

          { alert("processJSON"); // document.writeln(jsonData.type); }
          Show
          Imran Rajjad added a comment - the code mentioned on this page is not working, its giving an INVALID LABEL error in FireFox when result is appended to script element http://geoserver.org/display/GEOSDOC/GeoJSON+Output+Format my url request is : http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON error : Error: invalid label Source File: http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON Line: 1, Column: 1 Source Code: Error: invalid label Source File: http://192.172.2.23:8080/geoserver/ows?service=WFS&request=GetFeature&typeName=topp:states&outputFormat=json&format_options=callback:processJSON Line: 1, Column: 1 Source Code: {"type":"FeatureCollection","features":[{"type":"Feature","id":"states.1","geometry":{"type":"MultiPolygon","coordinates":[[[ [37.51099000000001,-88.071564] , [37.476273000000006,-88.087883] , [37.442852,-88.311707] , [37.40930899999999,-88.359177] ,[37.420292,-8.................... please note that the function processJSON function never gets called and after debuging it was found that the response has been directly aded to page where the browser cant understand it. I have tried several JSON retreiving libraries but its the same error everywhere. the code : function loadJSON(url) { var headID = document.getElementsByTagName("head")[0]; var newScript = document.createElement("script"); newScript.type = 'text/javascript'; newScript.src = url; headID.appendChild(newScript); } function processJSON(jsonData) { alert("processJSON"); // document.writeln(jsonData.type); }
          Hide
          Andrea Aime added a comment -

          Imran, your comment is completely out of topic in this bug report, which is dealing with a specific GeoJSon issue. I've already told you to subscribe to the geoserver users mailing list and pointed you to the page with pointers to subscription and the like.

          Show
          Andrea Aime added a comment - Imran, your comment is completely out of topic in this bug report, which is dealing with a specific GeoJSon issue. I've already told you to subscribe to the geoserver users mailing list and pointed you to the page with pointers to subscription and the like.
          Hide
          Imran Rajjad added a comment -

          yes sir/madam i will do that first thing on monday when I get back.. but i was just pointing out that the examples posted here dont work in browsers. please dont get angry i am offcourse not a programing guru but this requests doesnt give any error in geoserver console but for javascript to read this data is a problem,i copied that same code as it is without changing anything, what i want to point out after a week of messing around is that the callback function is not firing, as a result the response received against this request directly apended inside the script element and the browser cant understand it and surpringly its the same error no mater what json library we use. about that parenthesis thing, if that hasnt been fixed can you show a working standalone example

          Show
          Imran Rajjad added a comment - yes sir/madam i will do that first thing on monday when I get back.. but i was just pointing out that the examples posted here dont work in browsers. please dont get angry i am offcourse not a programing guru but this requests doesnt give any error in geoserver console but for javascript to read this data is a problem,i copied that same code as it is without changing anything, what i want to point out after a week of messing around is that the callback function is not firing, as a result the response received against this request directly apended inside the script element and the browser cant understand it and surpringly its the same error no mater what json library we use. about that parenthesis thing, if that hasnt been fixed can you show a working standalone example

            People

            • Assignee:
              Andrea Aime
              Reporter:
              Tim Schaub
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: