GeoTools

urn:ogc:def:crs:EPSG:4326 parsing won't work if forcexy is activated

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Critical Critical
  • Resolution: Fixed
  • Affects Version/s: 2.5-M0
  • Fix Version/s: 2.4.1, 2.5-M1
  • Component/s: referencing
  • Labels:
    None

Description

Parsing urn:ogc:def:crs:EPSG:4326 in GeoServer trunk has become impossible (not sure if yesterday or the day before)
due to a change in behaviour of the referencing subsystem.

Running the following code:

package org.geotools.referencing;

import org.geotools.factory.Hints;

public class ForcedLonFailure {
    public static void main(String[] args) throws Exception {
        // comment out this one and it'll work
        System.setProperty("org.geotools.referencing.forceXY", "true");
        Hints.putSystemDefault(Hints.FORCE_AXIS_ORDER_HONORING, "http");
        System.out.println(CRS.decode("urn:ogc:def:crs:EPSG:4326"));
    }

}

results in the following exception:

Exception in thread "main" org.opengis.referencing.NoSuchAuthorityCodeException:
Authority "URN" is unknown or doesn't match the supplied hints.
Maybe it is defined in an unreachable JAR file?
  at org.geotools.referencing.factory.ManyAuthoritiesFactory.noSuchAuthority(ManyAuthoritiesFactory.java:488)
  at org.geotools.referencing.factory.ManyAuthoritiesFactory.getAuthorityFactory(ManyAuthoritiesFactory.java:466)
  at org.geotools.referencing.factory.ManyAuthoritiesFactory.getCRSAuthorityFactory(ManyAuthoritiesFactory.java:547)
  at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:753)
  at org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:728)
  at org.geotools.referencing.CRS.decode(CRS.java:419)
  at org.geotools.referencing.CRS.decode(CRS.java:353)
  at org.geotools.referencing.ForcedLonFailure.main(ForcedLonFailure.java:12)

This makes the wfs 1.1 and wcs 1.1 protocols unusable (they suggest the usage of the urn syntax).

Issue Links

is depended upon by

Sub-task - The sub-task of the issue GEOT-1685 remove hack introduced for the parser to recognize CRS's from a GML 3.1.1 document when GEOT-1659 is fixed

  • Major - Major loss of function.
  • Open - The issue is open and ready for the assignee to start work on it.
relates to

Bug - A problem which impairs or prevents the functions of the product. GEOT-1388 EPSG codes in URN form do not respect global axis orientation setting

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

New Feature - A new feature of the product, which has yet to be developed. GEOT-859 Add a CRS authority factory for the "urn:ogc:def:...:EPSG" name space

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Andrea Aime added a comment - - edited
Correction, wfs 1.1 is apparently usable... but I have no idea why... the simple test above keeps on failing for me even when using the urn syntax adopted by the wfs 1.1 tests: "urn:x-ogc:def:crs:EPSG:6.11.2:4326". (and I'm running the above tests from the epsg-hsql module, so any geoserver interference should be out of question)
Show
Andrea Aime added a comment - - edited Correction, wfs 1.1 is apparently usable... but I have no idea why... the simple test above keeps on failing for me even when using the urn syntax adopted by the wfs 1.1 tests: "urn:x-ogc:def:crs:EPSG:6.11.2:4326". (and I'm running the above tests from the epsg-hsql module, so any geoserver interference should be out of question)
Hide
Permalink
Andrea Aime added a comment -
Correction, I just discovered wfs tests were working becuase they did not set the forcexy property in time, and the referencing subsystem was already initialized with the lat/lon ordering.
Show
Andrea Aime added a comment - Correction, I just discovered wfs tests were working becuase they did not set the forcexy property in time, and the referencing subsystem was already initialized with the lat/lon ordering.
Hide
Permalink
Martin Desruisseaux added a comment -
Just looked at "svn diff" for every changes in the referencing module since 2007-12-28 22:36:23 and found nothing significant. Trying the test case now...
Show
Martin Desruisseaux added a comment - Just looked at "svn diff" for every changes in the referencing module since 2007-12-28 22:36:23 and found nothing significant. Trying the test case now...
Hide
Permalink
Andrea Aime added a comment -
Martin, you're right... I apologize for the incorrect observation.
In fac the same program fails on 24.x as well... it's really strange, I don't understand exactly what's going on. Yet, the above test is quite simple and it's failing.
That situation is breaking my work on wcs 1.1 because the unit test are initializing the referencing subsystem before geoserver can start (to setup the coverage used by tests) and for some reason they are exibithin the same behaviour as the the above test: if I don't use forcexy I have no exceptions, but everything works lat/lon even for epsg:xxxx, whilst if I force xy orientation I can't use anymore urn authority... I'm sorry I can't provide better info...
Show
Andrea Aime added a comment - Martin, you're right... I apologize for the incorrect observation. In fac the same program fails on 24.x as well... it's really strange, I don't understand exactly what's going on. Yet, the above test is quite simple and it's failing. That situation is breaking my work on wcs 1.1 because the unit test are initializing the referencing subsystem before geoserver can start (to setup the coverage used by tests) and for some reason they are exibithin the same behaviour as the the above test: if I don't use forcexy I have no exceptions, but everything works lat/lon even for epsg:xxxx, whilst if I force xy orientation I can't use anymore urn authority... I'm sorry I can't provide better info...
Hide
Permalink
Martin Desruisseaux added a comment -
I'm slowing collecting some clues. One issue is that "system-wide" hints (either as system properties or as GeoTools.addDefaultHints(...)) are sometime in the way. The problem is that those system-wide hints are automatically added to explicitly-specified hints, i.e.

    ReferencingFactoryFinder.getCRSAuthorityFactory("foo", null);

is equivalent to

    ReferencingFactoryFinder.getCRSAuthorityFactory("foo", GeoTools.getDefaultHints());

but some time we really means no-hint, not even the system-wide ones. The problem is apparent when we search for the list of all available authorities. We use "null" for meaning "every authorities no matter what the hints are", but because default hints are automatically added the search become "any factory having forcexy=true", thus excluding URN.

The best fix is probably to get ride of system-wide hints, because I can invoke "ReferencingFactoryFinder.getFoo(hints)" and means "I want those hints, really those hints" but those hints may be altered in unpredictable way because it depends on the value given to Geotools.addDefauntHints(...) anywhere in the code base, including in places that we don't control (imagine that GeoServer links against a plugin that set a unwanted global hints). This is exactly what happenned in the referencing code that wanted to ask for all authorities and didn't get URN because of the global hints. We can not get ride of global hints at this time because not every GeoTools code accepts hints, but if we could it would probably eliminate a significant risk.

In the maintime I will continue the investigations.
Show
Martin Desruisseaux added a comment - I'm slowing collecting some clues. One issue is that "system-wide" hints (either as system properties or as GeoTools.addDefaultHints(...)) are sometime in the way. The problem is that those system-wide hints are automatically added to explicitly-specified hints, i.e.     ReferencingFactoryFinder.getCRSAuthorityFactory("foo", null); is equivalent to     ReferencingFactoryFinder.getCRSAuthorityFactory("foo", GeoTools.getDefaultHints()); but some time we really means no-hint, not even the system-wide ones. The problem is apparent when we search for the list of all available authorities. We use "null" for meaning "every authorities no matter what the hints are", but because default hints are automatically added the search become "any factory having forcexy=true", thus excluding URN. The best fix is probably to get ride of system-wide hints, because I can invoke "ReferencingFactoryFinder.getFoo(hints)" and means "I want those hints, really those hints" but those hints may be altered in unpredictable way because it depends on the value given to Geotools.addDefauntHints(...) anywhere in the code base, including in places that we don't control (imagine that GeoServer links against a plugin that set a unwanted global hints). This is exactly what happenned in the referencing code that wanted to ask for all authorities and didn't get URN because of the global hints. We can not get ride of global hints at this time because not every GeoTools code accepts hints, but if we could it would probably eliminate a significant risk. In the maintime I will continue the investigations.
Hide
Permalink
Justin Deoliveira added a comment -
Integrated in geotools-trunk #75 (See [http://geo.openplans.org:8080/hudson/job/geotools-trunk/75/])
Show
Justin Deoliveira added a comment - Integrated in geotools-trunk #75 (See [ http://geo.openplans.org:8080/hudson/job/geotools-trunk/75/ ])
Hide
Permalink
Martin Desruisseaux added a comment -
geotools-trunk #75 contains only the fix for URN authority not found when searching for all authorities. I doesn't seem to be suffisient for fixing the issue reported by Andrea.
Show
Martin Desruisseaux added a comment - geotools-trunk #75 contains only the fix for URN authority not found when searching for all authorities. I doesn't seem to be suffisient for fixing the issue reported by Andrea.
Hide
Permalink
Andrea Aime added a comment -
I added Jody to the watchers of this issue because I thing global hints is something he cares about. Jody, any comments?
Show
Andrea Aime added a comment - I added Jody to the watchers of this issue because I thing global hints is something he cares about. Jody, any comments?
Hide
Permalink
Martin Desruisseaux added a comment -
Well, it is not a call for removing them now. But I believe that global hints increase the risk of unexpected behavior. Part (but maybe not all - my investigation is not finished) of the "URN not found" issue was clearly caused by global hints interfering with a code that really wanted "no hints", not "default hints". It would be nice to progressively reduce our dependency to global hints until we can really remove them (if peoples agree). It may take a year or so...
Show
Martin Desruisseaux added a comment - Well, it is not a call for removing them now. But I believe that global hints increase the risk of unexpected behavior. Part (but maybe not all - my investigation is not finished) of the "URN not found" issue was clearly caused by global hints interfering with a code that really wanted "no hints", not "default hints". It would be nice to progressively reduce our dependency to global hints until we can really remove them (if peoples agree). It may take a year or so...
Hide
Permalink
Martin Desruisseaux added a comment -
I just had a though... I remember that a while ago, Jody suggested that FactoryFinder methods should not be static. The issue reported here sound like an argument in favor of Jody's proposal. It would not be an immediate fix for the issue reported here, but may help to improve the sanity:

* Taking advantage that GeoTools 2.4 is not yet released, replace most ReferencingFactoryFinder static methods
   by ordinary ones. Note that it is a significant API break. The deprecated FactoryFinder class would stay static.

* Provides a ReferencingFactoryFinder.DEFAULT constant.

* Provides some mechanic allowing to specify different default hints to different ReferencingFactoryFinder
   instances. Maybe a ReferencingFactoryFinder(Hints) constructor?

* GeoServer could uses its own instance of ReferencingFactoryFinder with exactly the hints it wants,
   and be sure that those hints are not changed by some other users invoking GeoTools.addDefaultHints.


Show
Martin Desruisseaux added a comment - I just had a though... I remember that a while ago, Jody suggested that FactoryFinder methods should not be static. The issue reported here sound like an argument in favor of Jody's proposal. It would not be an immediate fix for the issue reported here, but may help to improve the sanity: * Taking advantage that GeoTools 2.4 is not yet released, replace most ReferencingFactoryFinder static methods    by ordinary ones. Note that it is a significant API break. The deprecated FactoryFinder class would stay static. * Provides a ReferencingFactoryFinder.DEFAULT constant. * Provides some mechanic allowing to specify different default hints to different ReferencingFactoryFinder    instances. Maybe a ReferencingFactoryFinder(Hints) constructor? * GeoServer could uses its own instance of ReferencingFactoryFinder with exactly the hints it wants,    and be sure that those hints are not changed by some other users invoking GeoTools.addDefaultHints.
Hide
Permalink
Andrea Aime added a comment -
That looks like a major change to me? All of the GeoServer code interfaces with the referencing susbsystem using CRS utility method, as does most of the vector datastores and coverage readers.
Would it be possible to provide the CRS object with the referencing factory finder we need?
Show
Andrea Aime added a comment - That looks like a major change to me? All of the GeoServer code interfaces with the referencing susbsystem using CRS utility method, as does most of the vector datastores and coverage readers. Would it be possible to provide the CRS object with the referencing factory finder we need?
Hide
Permalink
Martin Desruisseaux added a comment -
I'm thinking about a smaller change. The CRS utility methods would stay unchanged. Only ReferencingFactoryFinder methods would become non-static (we may apply the same to other FactoryFinder). The FactoryFinder constructor would stay private - we will not allows custom instance at this time (it will be easy to revisit that later if we want).

ReferencingFactoryFinder would contains two constants:

public static final ReferencingFactoryFinder DEFAULT = ....
public static final ReferencingFactoryFinder STRICT = ....

DEFAULT would behave as current static methods. STRICT would apply the Hints exactly as-is, without merge with default hints.

Users would only need to replace calls like:

    ReferencingFactoryFinder.getCRSAuthorityFactory("EPSG", hints);

by

    ReferencingFactoryFinder.DEFAULT.getCRSAuthorityFactory("EPSG", hints);

i.e. they only need to insert DEFAULT.

Referencing module would uses STRICT for its internal use most of the time. This is correct since when a factory asks for an other factory, its Hints have already been merged with default hints if the users created the first factory with DEFAULT.
Show
Martin Desruisseaux added a comment - I'm thinking about a smaller change. The CRS utility methods would stay unchanged. Only ReferencingFactoryFinder methods would become non-static (we may apply the same to other FactoryFinder). The FactoryFinder constructor would stay private - we will not allows custom instance at this time (it will be easy to revisit that later if we want). ReferencingFactoryFinder would contains two constants: public static final ReferencingFactoryFinder DEFAULT = .... public static final ReferencingFactoryFinder STRICT = .... DEFAULT would behave as current static methods. STRICT would apply the Hints exactly as-is, without merge with default hints. Users would only need to replace calls like:     ReferencingFactoryFinder.getCRSAuthorityFactory("EPSG", hints); by     ReferencingFactoryFinder.DEFAULT.getCRSAuthorityFactory("EPSG", hints); i.e. they only need to insert DEFAULT. Referencing module would uses STRICT for its internal use most of the time. This is correct since when a factory asks for an other factory, its Hints have already been merged with default hints if the users created the first factory with DEFAULT.
Hide
Permalink
Martin Desruisseaux added a comment -
Proposal page there: http://docs.codehaus.org/display/GEOTOOLS/ReferencingFactoryFinder+incompatible+change
Show
Martin Desruisseaux added a comment - Proposal page there: http://docs.codehaus.org/display/GEOTOOLS/ReferencingFactoryFinder+incompatible+change
Hide
Permalink
Jean-Pierre Fiset added a comment -
I have experienced a problem which I thought was related to this problem. An exception similar to the one in the description was reported while running geoserver-1.6.0-RC3. However, the issue had to do with running multiple instances of geoserver within one instance of tomcat.

I am reporting this here in case other users come to this bug report the same way I did.

The solution is to run a single instance of geoserver within Tomcat.

Andreas explained that there can be side effects between various instances of geoserver in Tomcat because system environment variables are used to communicate information between geoserver and geotools. Since Tomcat does not firewall the system environment variables between various running web applications, there can be race conditions between two instances of geoserver. This is the theory that was advanced, and so far it appears to hold.
Show
Jean-Pierre Fiset added a comment - I have experienced a problem which I thought was related to this problem. An exception similar to the one in the description was reported while running geoserver-1.6.0-RC3. However, the issue had to do with running multiple instances of geoserver within one instance of tomcat. I am reporting this here in case other users come to this bug report the same way I did. The solution is to run a single instance of geoserver within Tomcat. Andreas explained that there can be side effects between various instances of geoserver in Tomcat because system environment variables are used to communicate information between geoserver and geotools. Since Tomcat does not firewall the system environment variables between various running web applications, there can be race conditions between two instances of geoserver. This is the theory that was advanced, and so far it appears to hold.
Hide
Permalink
Sébastien Geindre added a comment -
I have also experiment this problem.

I deployed the geoserver 1.6-RC3 war file.
I made several undeploy, deploy due to geoserver_data_dir related path which was wrong

and when i requested via wfs 1.1, i got this exception :
25 janv. 16:38:21 WARN [geoserver.ows] -
org.geoserver.wfs.WFSException: We have had issues trying to flip axis of GEOGCS["WGS 84",
  DATUM["World Geodetic System 1984",
    SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]],
    AUTHORITY["EPSG","6326"]],
  PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]],
  UNIT["degree", 0.017453292519943295],
  AXIS["Geodetic longitude", EAST],
  AXIS["Geodetic latitude", NORTH],
  AUTHORITY["EPSG","4326"]]
        at org.geoserver.wfs.WFSReprojectionUtil.getDeclaredCrs(WFSReprojectionUtil.java:46)
        at org.geoserver.wfs.GetFeature.toDataQuery (GetFeature.java:390)
        at org.geoserver.wfs.GetFeature.run(GetFeature.java:250)
        at org.geoserver.wfs.DefaultWebFeatureService.getFeature(DefaultWebFeatureService.java:108)
...
Caused by: org.opengis.referencing.NoSuchAuthorityCodeException : L'autorité "URN" n'est pas connue ou ne correspond pas aux indices spécifiés. Le fichier JAR qui la définie n'est peut-être pas accessible?
        at org.geotools.referencing.factory.ManyAuthoritiesFactory.noSuchAuthority(ManyAuthoritiesFactory.java:544)
        at org.geotools.referencing.factory.ManyAuthoritiesFactory.getAuthorityFactory(ManyAuthoritiesFactory.java :522)
...


The solution for me was to stop and restart tomcat....
hope it helps...
Show
Sébastien Geindre added a comment - I have also experiment this problem. I deployed the geoserver 1.6-RC3 war file. I made several undeploy, deploy due to geoserver_data_dir related path which was wrong and when i requested via wfs 1.1, i got this exception : 25 janv. 16:38:21 WARN [geoserver.ows] - org.geoserver.wfs.WFSException: We have had issues trying to flip axis of GEOGCS["WGS 84",   DATUM["World Geodetic System 1984",     SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]],     AUTHORITY["EPSG","6326"]],   PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]],   UNIT["degree", 0.017453292519943295],   AXIS["Geodetic longitude", EAST],   AXIS["Geodetic latitude", NORTH],   AUTHORITY["EPSG","4326"]]         at org.geoserver.wfs.WFSReprojectionUtil.getDeclaredCrs(WFSReprojectionUtil.java:46)         at org.geoserver.wfs.GetFeature.toDataQuery (GetFeature.java:390)         at org.geoserver.wfs.GetFeature.run(GetFeature.java:250)         at org.geoserver.wfs.DefaultWebFeatureService.getFeature(DefaultWebFeatureService.java:108) ... Caused by: org.opengis.referencing.NoSuchAuthorityCodeException : L'autorité "URN" n'est pas connue ou ne correspond pas aux indices spécifiés. Le fichier JAR qui la définie n'est peut-être pas accessible?         at org.geotools.referencing.factory.ManyAuthoritiesFactory.noSuchAuthority(ManyAuthoritiesFactory.java:544)         at org.geotools.referencing.factory.ManyAuthoritiesFactory.getAuthorityFactory(ManyAuthoritiesFactory.java :522) ... The solution for me was to stop and restart tomcat.... hope it helps...
Hide
Permalink
Martin Desruisseaux added a comment -
Commited a fix attempt on trunk as of revision 28974. Waiting for confirmation that it works before to merge with the 2.4 branch.

At the opposite of what I said before, I realized that it was not necessary to modify ReferencingFactoryFinder API. I tried a more transparent approach, introducing the package-privated StrictHints class, so users don't need to care about it. However I still not about whatever we should (on a conceptual point of view) turn static methods into non-static ones. The problem with non-static methods is that it give choice to users - sometime a good thing but sometime a source of confusion. I have meet cases where I was unsure about whatever I should use STRICT or DEFAULT factory finder. Keeping the API as it stand today avoid both this confusion and compatibility break.
Show
Martin Desruisseaux added a comment - Commited a fix attempt on trunk as of revision 28974. Waiting for confirmation that it works before to merge with the 2.4 branch. At the opposite of what I said before, I realized that it was not necessary to modify ReferencingFactoryFinder API. I tried a more transparent approach, introducing the package-privated StrictHints class, so users don't need to care about it. However I still not about whatever we should (on a conceptual point of view) turn static methods into non-static ones. The problem with non-static methods is that it give choice to users - sometime a good thing but sometime a source of confusion. I have meet cases where I was unsure about whatever I should use STRICT or DEFAULT factory finder. Keeping the API as it stand today avoid both this confusion and compatibility break.
Hide
Permalink
Justin Deoliveira added a comment -
Integrated in geotools-trunk #182 (See [http://gridlock.openplans.org:8080/hudson/job/geotools-trunk/182/])
Show
Justin Deoliveira added a comment - Integrated in geotools-trunk #182 (See [ http://gridlock.openplans.org:8080/hudson/job/geotools-trunk/182/ ])
Hide
Permalink
Andrea Aime added a comment -
Hum, running the sample program I've submitted at the beginning of this issue I'm not getting an exception anymore, but the urn factory is returning lon/lat ordered crs? Wasn't it supposed to return lat/lon no matter what?
Show
Andrea Aime added a comment - Hum, running the sample program I've submitted at the beginning of this issue I'm not getting an exception anymore, but the urn factory is returning lon/lat ordered crs? Wasn't it supposed to return lat/lon no matter what?
Hide
Permalink
Martin Desruisseaux added a comment -
Yes it is a bug. I didn't realized this undesirable side-effect.
Show
Martin Desruisseaux added a comment - Yes it is a bug. I didn't realized this undesirable side-effect.
Hide
Permalink
Martin Desruisseaux added a comment -
Hopefully fixed on trunk as of revision 28977. However we are still lacking good test coverage, so any feedback from GeoServer would be appreciated.
Show
Martin Desruisseaux added a comment - Hopefully fixed on trunk as of revision 28977. However we are still lacking good test coverage, so any feedback from GeoServer would be appreciated.
Hide
Permalink
Justin Deoliveira added a comment -
Integrated in geotools-trunk #183 (See [http://gridlock.openplans.org:8080/hudson/job/geotools-trunk/183/])
Show
Justin Deoliveira added a comment - Integrated in geotools-trunk #183 (See [ http://gridlock.openplans.org:8080/hudson/job/geotools-trunk/183/ ])
Hide
Permalink
Martin Desruisseaux added a comment -
Merged on the 2.4 branch as of revision 29058. Flag this issue as "resolved", waiting for confirmation that it works for GeoServer. Please close this issue if it produces the expected results with GeoServer.
Show
Martin Desruisseaux added a comment - Merged on the 2.4 branch as of revision 29058. Flag this issue as "resolved", waiting for confirmation that it works for GeoServer. Please close this issue if it produces the expected results with GeoServer.
Hide
Permalink
Andrea Aime added a comment -
I can confirm this one is fixed, but unfortunately another referencing issue is blocking the geoserver release now... don't know if it's a side effect of this fix or not thought
Show
Andrea Aime added a comment - I can confirm this one is fixed, but unfortunately another referencing issue is blocking the geoserver release now... don't know if it's a side effect of this fix or not thought
Hide
Permalink
Andrea Aime added a comment -
Mass closing all issues that have been in "resolved" state for 2 months or more without any feedback or update
Show
Andrea Aime added a comment - Mass closing all issues that have been in "resolved" state for 2 months or more without any feedback or update

People

  • Assignee:
    Martin Desruisseaux
    Reporter:
    Andrea Aime
Vote (1)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: