Support parsing srsName in OGC HTTP URI format

Ben Caradoc-Davies added a comment - 31/Jul/12 4:59 AM -------- Original Message -------- Subject: Incompatible referencing Citation change required to support OGC HTTP URI srsName parsing? Date: Tue, 31 Jul 2012 17:57:39 +0800 From: Ben Caradoc-Davies To: geotools-devel@lists.sourceforge.net Current OGC policy is for srsName to be encoded as an HTTP URI: http://www.opengis.net/def/crs/EPSG/0/4326 We currently support parsing HTTP srsNames in the GML 2 form: http://www.opengis.net/gml/srs/epsg.xml#4326 This is done by HTTP_AuthorityFactory. The authority for this form, set in Citations.HTTP_OGC, is " http://www.opengis.net ". The problem is that ManyAuthoritiesFactory assumes that the correct factory for an srsName can be found by matching the start of the srsName against the authority for the factory. This assumption prevents me from adding a second factory for " http://www.opengis.net/def/crs/ " because these URIs will also be matched by HTTP_AuthorityFactory. I can either: (1) Change the authority of Citations.HTTP_OGC from " http://www.opengis.net " to " http://www.opengis.net/gml/srs/ " so that I can add a new factory with authority " http://www.opengis.net/def/crs/ ", or (2) Hack HTTP_AuthorityFactory to support the new form. Option (1) appears to be the cleanest because it preserves the one-factory-per-form pattern. However, it will break any third-party code that depends on the authority identifier, making it a backwards-incompatible change. Option (2) would be quite straightforward, but I am worried about hidden magic axis-order behaviour. For historical reasons (we can still see the scars), the GML 2 form is treated as having longitude/latitude axis order by applications such as GeoServer: http://docs.geoserver.org/latest/en/user/services/wfs/basics.html#axis-ordering I do not know where this conversion occurs. Does anyone know of using HTTP_AuthorityFactory to parse the new HTTP URI form http://www.opengis.net/def/crs/EPSG/0/4326 would cause the resulting CRS it to have longitude/latitude axis order? (It is meant to have latitude/longitude axis order.) There is some code in HTTP_AuthorityFactory.defaultAxisOrderHints, but I do not understand what it does nor how it is used. I will be writing unit tests to gather evidence. For more discussion on why we need HTTP URIs, see: https://jira.codehaus.org/browse/GEOT-4160 Kind regards, -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 01/Aug/12 4:47 AM -------- Original Message -------- Subject: Re: [ExternalEmail] [Geotools-devel] Incompatible referencing Citation change required to support OGC HTTP URI srsName parsing? Date: Wed, 01 Aug 2012 17:47:10 +0800 From: Ben Caradoc-Davies To: Justin Deoliveira CC: geotools-devel@lists.sourceforge.net Just as I feared: option (2) causes a GeoServer request with srsName= http://www.opengis.net/def/crs/EPSG/0/4326 to return a response with srsname=" http://www.opengis.net/gml/srs/epsg.xml#4326 " and longitude/latitude order. The underlying cause (I think) is that HTTP_AuthorityFactory is configured with Hints:   FORCE_LONGITUDE_FIRST_AXIS_ORDER = true   FORCE_AXIS_ORDER_HONORING = http Just as described in the javadoc for Hints.FORCE_AXIS_ORDER_HONORING, this causes longitude/latitude order, even for my HTTP URI srsName. It looks like HTTP_AuthorityFactory is locked to one axis order convention. Any objection to me implementing option (1), that is, changing Citations.HTTP_OGC so I can put in another factory for HTTP URI srsName? Kind regards, Ben.

Ben Caradoc-Davies added a comment - 08/Aug/12 2:00 AM Justin, rather than add just a new HTTP_URI_AuthorityFactory that only worked for CRS, I thought it would be more general to refactor the existing URN_AuthorityFactory and URN_Parser so that we support exactly the same functionality for HTTP URIs as we do for URNs. - URN_Type renamed URI_Type and the def types updated - Most of URN_AuthorityFactory pulled up into Abstract_URI_AuthorityFactory - Most of URN_Parser pulled up into URI_Parser - New HTTP_URI_Parser, HTTP_URI_AuthorityFactory - i18n updates, tests in xsd-gml3 - One added GeoServer test class to test srsName in requests (in its own patch).

Ben Caradoc-Davies added a comment - 08/Aug/12 2:01 AM By the way, do you prefer old-school Jira patch review or do you prefer GitHub pull requests?

Ben Caradoc-Davies added a comment - 08/Aug/12 2:11 AM Files changed by the geotools patch: {noformat} modules/extension/xsd/xsd-gml3/src/test/java/org/geotools/gml3/v3_2/GMLParsingTest.java modules/library/metadata/src/main/java/org/geotools/metadata/iso/citation/Citations.java modules/library/metadata/src/main/java/org/geotools/resources/i18n/LoggingKeys.java modules/library/metadata/src/main/java/org/geotools/resources/i18n/Loggings.properties modules/library/metadata/src/main/java/org/geotools/resources/i18n/Loggings_es.properties modules/library/metadata/src/main/java/org/geotools/resources/i18n/Loggings_fr.properties modules/library/referencing/src/main/java/org/geotools/referencing/factory/Abstract_URI_AuthorityFactory.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/AuthorityFactoryAdapter.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/HTTP_AuthorityFactory.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/HTTP_URI_AuthorityFactory.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/HTTP_URI_Parser.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/URI_Parser.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/URI_Type.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/URN_AuthorityFactory.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/URN_Parser.java modules/library/referencing/src/main/java/org/geotools/referencing/factory/URN_Type.java modules/library/referencing/src/main/resources/META-INF/services/org.opengis.referencing.crs.CRSAuthorityFactory modules/library/referencing/src/main/resources/META-INF/services/org.opengis.referencing.cs.CSAuthorityFactory modules/library/referencing/src/main/resources/META-INF/services/org.opengis.referencing.datum.DatumAuthorityFactory modules/library/referencing/src/main/resources/META-INF/services/org.opengis.referencing.operation.CoordinateOperationAuthorityFactory modules/library/referencing/src/test/java/org/geotools/referencing/CrsTest.java modules/library/referencing/src/test/java/org/geotools/referencing/factory/HTTP_AuthorityFactoryTest.java modules/library/referencing/src/test/java/org/geotools/referencing/factory/HTTP_URI_AuthorityFactoryTest.java modules/library/referencing/src/test/java/org/geotools/referencing/factory/HTTP_URI_ParserTest.java modules/library/referencing/src/test/java/org/geotools/referencing/factory/URN_ParserTest.java {noformat}

Ben Caradoc-Davies added a comment - 08/Aug/12 2:37 AM -------- Original Message -------- Subject: Referencing changes to support OGC HTTP URI srsName parsing Date: Wed, 08 Aug 2012 15:36:24 +0800 From: Ben Caradoc-Davies To: Andrea Aime, Justin Deoliveira CC: geotools-devel@lists.sourceforge.net , Jody Garnett For the record, this has turned into a substantial refactoring of URN_AuthorityFactory and its helpers to ensure the new HTTP_URI_AuthorityFactory has the same functionality and that code duplication is avoided: https://jira.codehaus.org/browse/GEOT-4160 This change targets master only, and users should be sheltered by SPI and the factory finders. Users have been warned to not instantiate these classes directly. Please let me know if a proposal is required; I think not. I have assigned this issue to Justin for review as his name was on some of the classes. Thanks in advance, Justin! Kind regards, -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 08/Aug/12 11:10 PM -------- Original Message -------- Subject: Re: Referencing changes to support OGC HTTP URI srsName parsing Date: Thu, 09 Aug 2012 11:53:54 +0800 From: Ben Caradoc-Davies To: Andrea Aime CC: Justin Deoliveira, " geotools-devel@lists.sourceforge.net ", Jody Garnett On 08/08/12 20:08, Andrea Aime wrote: > I only managed to have a quick look at the patch so far, if you don't > mind as the current referencing > maintainer I would like to give it a better look during the weekend > before it get committed. Thanks, if you are the module maintainer I am more than happy for you review the patch. > What I'm wondering after this first look into the patch is the expected > axis order of the new syntax. > The urn form is supposed to be in lat/lon no matter what the referencing > hints are, if the new one is > going to be used for WFS I believe that also will have to respect the > same rules as the URN one. Exactly, that is the goal of the refactoring: common behaviour through code reuse. The new HTTP URI form has the same lat/lon behaviour as the URN form. According to the javadoc, the implementation of the URN form is that lon/lat can be forced by setting Hints.FORCE_AXIS_ORDER_HONORING to include "urn". For HTTP URI this is "http-uri". Both factories ignore this hint set to "http" (as recognised by HTTP_AuthorityFactory); this appears to be tested in URN_AuthorityFactoryTest; HTTP_URI_AuthorityFactoryTest does exactly the same. I say appears because gt-referencing is a jungle and it is often hard to tell what is going on. Kind regards, -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 10/Aug/12 12:14 AM Attached an improved version of the GeoServer patch that increases unit test coverage to include GET request bounding box axis order.

Ben Caradoc-Davies added a comment - 13/Aug/12 9:40 PM -------- Original Message -------- Subject: Re: Referencing changes to support OGC HTTP URI srsName parsing Date: Sun, 12 Aug 2012 23:15:43 +0800 From: Andrea Aime To: Caradoc-Davies, Ben (CESRE, Kensington) CC: Justin Deoliveira , " geotools-devel@lists.sourceforge.net " , Jody Garnett On Thu, Aug 9, 2012 at 5:53 AM, Ben Caradoc-Davies wrote:     Thanks, if you are the module maintainer I am more than happy for     you review the patch. Reviewed the patch, looks good but I still have some doubts about the bits below.         What I'm wondering after this first look into the patch is the         expected         axis order of the new syntax.         The urn form is supposed to be in lat/lon no matter what the         referencing         hints are, if the new one is         going to be used for WFS I believe that also will have to         respect the         same rules as the URN one.     Exactly, that is the goal of the refactoring: common behaviour     through code reuse. The new HTTP URI form has the same lat/lon     behaviour as the URN form.     According to the javadoc, the implementation of the URN form is that     lon/lat can be forced by setting Hints.FORCE_AXIS_ORDER___HONORING     to include "urn". For HTTP URI this is "http-uri". Both factories     ignore this hint set to "http" (as recognised by     HTTP_AuthorityFactory); this appears to be tested in     URN_AuthorityFactoryTest; HTTP_URI_AuthorityFactoryTest does exactly     the same. I say appears because gt-referencing is a jungle and it is     often hard to tell what is going on. GeoServer on startup forces the old http factory to work in lon/lat mode, because that's the common axis order for WFS 1.0, which supported the EPSG:XYWZ and the old http form. The test checks forcing the axis order for http://www.opengis.net/def/crs/CRS/0/84, which afaik is the only way OGC provided to get a WGS84 in lon/lat order, which makes me thing it shouldbe in lon/lat order regardless of the hints you are providing. From the WMS 1.3 spec:     EXAMPLE 1     A <BoundingBox> metadata element for a Layer representing the entire     Earth in the CRS:84 Layer CRS     would be written as     <BoundingBox CRS="CRS:84" minx="-180" miny="-90" maxx="180" maxy="90">.     A BBOX parameter requesting a map of the entire Earth would be     written in this CRS as     BBOX=-180,-90,180,90.     EXAMPLE 2     A <BoundingBox> representing the entire Earth in the EPSG:4326 Layer     CRS would be written as     <BoundingBox CRS="EPSG:4326" minx="-90" miny="-180" maxx="90"     maxy="180">.     A BBOX parameter requesting However, I'm wondering what happens if you set the following:              Hints.putSystemDefault(Hints.FORCE_AXIS_ORDER_HONORING, "http");               Hints.putSystemDefault(Hints.FORCE_LONGITUDE_FIRST_AXIS_ORDER, Boolean.TRUE); and then parse http://www.opengis.net/def/crs/EPSG/0/4326 . The result should be in lat/lon order just like the urn factory would behave Cheers Andrea

Ben Caradoc-Davies added a comment - 13/Aug/12 9:42 PM -------- Original Message -------- Subject: Re: Referencing changes to support OGC HTTP URI srsName parsing Date: Mon, 13 Aug 2012 10:19:49 +0800 From: Ben Caradoc-Davies To: Andrea Aime CC: Justin Deoliveira, " geotools-devel@lists.sourceforge.net ", Jody Garnett Andrea, thanks very much for your prompt review. On 12/08/12 23:15, Andrea Aime wrote: > However, I'm wondering what happens if you set the following: > Hints.putSystemDefault(Hints.FORCE_AXIS_ORDER_HONORING, > "http"); > Hints.putSystemDefault(Hints.FORCE_LONGITUDE_FIRST_AXIS_ORDER, > Boolean.TRUE); > and then parse http://www.opengis.net/def/crs/EPSG/0/4326 . The result > should be in lat/lon order just > like the urn factory would behave This is exactly what happens in the GeoServer test, where the fixture forces lon/lat for "http" ( http://www.opengis.net/gml ). The rest of the test confirms that http://www.opengis.net/def/crs/EPSG/0/4326 remains in lat/lon despite the axis order of http://www.opengis.net/gml/srs/epsg.xml#4326 being lon/lat. This cannot be tested in referencing as the EPSG providers are out of scope. The unit tests instead use CRS for tests. In particular, testWhenForceXY in both HTTP_URI_AuthorityFactoryTest and (and URN_AuthorityFactoryTest on which it was based) set Hints.putSystemDefault(Hints.FORCE_AXIS_ORDER_HONORING, "http"); Hints.putSystemDefault(Hints.FORCE_LONGITUDE_FIRST_AXIS_ORDER, Boolean.TRUE); and then CoordinateReferenceSystem crs = CRS.decode(" http://www.opengis.net/def/crs/CRS/0/84 ",true); assertTrue(CRS.equalsIgnoreMetadata(DefaultGeographicCRS.WGS84, crs)); which I think confirms the expected behaviour. Kind regards, -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 13/Aug/12 9:43 PM -------- Original Message -------- Subject: Re: Referencing changes to support OGC HTTP URI srsName parsing Date: Mon, 13 Aug 2012 14:08:21 +0800 From: Andrea Aime To: Caradoc-Davies, Ben (CESRE, Kensington) CC: Justin Deoliveira , " geotools-devel@lists.sourceforge.net "  Jody Garnett On Mon, Aug 13, 2012 at 4:19 AM, Ben Caradoc-Davies wrote:     This is exactly what happens in the GeoServer test, where the     fixture forces lon/lat for "http" ( http://www.opengis.net/gml ). The     rest of the test confirms that      http://www.opengis.net/def/__crs/EPSG/0/4326     < http://www.opengis.net/def/crs/EPSG/0/4326 > remains in lat/lon     despite the axis order of      http://www.opengis.net/gml/__srs/epsg.xml#4326     < http://www.opengis.net/gml/srs/epsg.xml#4326 > being lon/lat. Good.     This cannot be tested in referencing as the EPSG providers are out     of scope. The unit tests instead use CRS for tests. In particular,     testWhenForceXY in both HTTP_URI_AuthorityFactoryTest and (and     URN_AuthorityFactoryTest on which it was based) set     Hints.putSystemDefault(Hints.__FORCE_AXIS_ORDER_HONORING, "http");     Hints.putSystemDefault(Hints.__FORCE_LONGITUDE_FIRST_AXIS___ORDER,     Boolean.TRUE);     and then     CoordinateReferenceSystem crs =     CRS.decode(" http://www.__opengis.net/def/crs/CRS/0/84     < http://www.opengis.net/def/crs/CRS/0/84 >",__true);     assertTrue(CRS.__equalsIgnoreMetadata(__DefaultGeographicCRS.WGS84,     crs));     which I think confirms the expected behaviour. Not really, as I said, CRS:84 is expected to be in lon/lat regardless of the hints you provide, so the test above is not actually testing the effects of the hints (as there should be none). Cheers Andrea

Ben Caradoc-Davies added a comment - 13/Aug/12 9:44 PM -------- Original Message -------- Subject: Re: Referencing changes to support OGC HTTP URI srsName parsing Date: Mon, 13 Aug 2012 14:18:04 +0800 From: Ben Caradoc-Davies To: Andrea Aime CC: Justin Deoliveira, " geotools-devel@lists.sourceforge.net ", Jody Garnett On 13/08/12 14:08, Andrea Aime wrote: > Not really, as I said, CRS:84 is expected to be in lon/lat regardless of > the hints you provide, > so the test above is not actually testing the effects of the hints (as > there should be none). OK. This test only does what the URN equivalent does. I think the GeoServer test is better. -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 13/Aug/12 9:46 PM -------- Original Message -------- Subject: BBOX axis order discrepancy Date: Fri, 10 Aug 2012 12:20:44 +0800 From: Ben Caradoc-Davies To: geoserver-devel@lists.sourceforge.net While adding even more unit test coverage for srsNames in HTTP URI format, I added some for EPSG codes and HTTP URLs as well, and noticed what looks like a discrepancy between the axis order or a request and the axis order of a response. For WFS 2.0.0 and WFS 1.1.0 GET requests with srsName=EPSG:4326 or srsName= http://www.opengis.net/gml/srs/epsg.xml#4326, the response is returned with srsName= http://www.opengis.net/gml/srs/epsg.xml#4326 and coordinates in longitude/latitude order. However, if the request includes a bbox, it is interpreted as latitude/longitude order, the opposite of the response. Is this intended? (Or just what we are stuck with for backwards compatibility?) It looks like a discrepancy to me. It looks like the documented axis order hack is only applied to encoding, not parsing of a request: http://docs.geoserver.org/latest/en/user/services/wfs/basics.html#axis-ordering (The solution being for everyone to move on to URN and HTTP URI srsNames and try to forget the axis order of evil.) Kind regards, -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 13/Aug/12 9:46 PM -------- Original Message -------- Subject: Re: [Geoserver-devel] BBOX axis order discrepancy Date: Fri, 10 Aug 2012 15:06:57 +0800 From: Ben Caradoc-Davies To: Andrea Aime CC: geoserver-devel@lists.sourceforge.net On 10/08/12 15:04, Andrea Aime wrote: > This is the expected behavior, WFS does not behave like WMS. > In particular, the BBOX is always interpreted in the data native SRS, which > WFS wise is the one declared in the capabilities document, > which is in your case the urn:ogc-x:... form that has lat/lon axis order. Ah, that explains everything. > The bbox also takes a fifth element for the crs, so if you want to express > your bbox in lon/lat you can do so by using > &bbox=lon1,lat1,lon2,lat2,EPSG:4326 Excellent idea! I'll add these to the test. -- Ben Caradoc-Davies Software Engineer CSIRO Earth Science and Resource Engineering Australian Resources Research Centre

Ben Caradoc-Davies added a comment - 13/Aug/12 9:48 PM Attached updated geoserver patch that extends test coverage to include both five-parameter bboxes, as suggested by Andrea.

Ben Caradoc-Davies added a comment - 13/Aug/12 11:07 PM Pushed commit e93bea65f5896602098483438ab3500b7a4ea84b to geotools master. Pushed commit 81424313659c8ae230ff80742d1c57058713512d to geoserver master.

Andrea Aime added a comment - 14/Aug/12 4:25 AM Ben, next time you want to refer to a discussion on the mailing list it may be easier to just provide a link to nabble: http://osgeo-org.1560.n6.nabble.com/Incompatible-referencing-Citation-change-required-to-support-OGC-HTTP-URI-srsName-parsing-td4992078.html

Ben Caradoc-Davies added a comment - 14/Aug/12 7:25 PM I know, I was cursing myself every time I pasted and sanitised an email. I was badly burned the last time nabble decided to reorganise their service, breaking many links on active issues. I am now quite reluctant to use nabble. Cool URLs don't change. I suppose I should use the sourceforge archive links, which seems to be a lot more stable: http://sourceforge.net/mailarchive/forum.php?thread_name=50289C1C.2070308%40csiro.au&forum_name=geotools-devel http://sourceforge.net/mailarchive/forum.php?thread_name=CA%2BnxMTsgY42k%2BSAZAhD551jkhvnsB-QqAqAQMboBPWOzH%2BUZOQ%40mail.gmail.com&forum_name=geoserver-devel

Ben Caradoc-Davies added a comment - 15/Oct/12 10:54 PM Backported to 8.x for inclusion in 8.3. Pushed 96d631435b7c1765dacb608479bb92fc982572f9 to 8.x. Also pushed 7a61aab7b80793be18fc35b776c4d063c575ff2c to GeoServer 2.2.x (the unit test).