GeoServer

CQL filter could not parce - Cyrillic data from attribute table of SHP file

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.1-beta1
  • Fix Version/s: 2.1-beta2
  • Component/s: None
  • Labels:
    None
  • Environment:
    Windows XP, GeoServer 2.1-beta1
  • Number of attachments :
    4

Description

I installed GeoServer 2.1-beta1.
I publish one shp file with attribute table that contain Cyrillic and Latin letter.

I have problem with CQL filter. I describe situation below:

In attachment/correct.JPG/ - when I filer data from field with Latin letters or numerical -> Filter work correctly.

! BUT /see attachment - error.JPG/ when I try to filter data from filed name that is in Cyrillic or data in column are in Cyrillic, I receive error "Could not parse CQL filter list. Lexical error at line 1, column 1. Encountered : "\u041d"(1053), after Parsing: Номернаи=0.10.

How can I solve this problem? In web page attributes are displated correct. Problem is with CQL filter. I publish shp file with: charset windows-1251

Thanks

  1. File
    data.rar
    15/Oct/10 1:53 AM
    13 kB
    Stefan
  1. correct.JPG
    38 kB
    14/Oct/10 7:45 AM
  2. error.JPG
    39 kB
    14/Oct/10 7:45 AM
  3. error2.JPG
    33 kB
    14/Oct/10 2:14 PM

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Andrea Aime added a comment -

A critical bug fix is one affecting a large amount of users. Downgrading to major

Show
Andrea Aime added a comment - A critical bug fix is one affecting a large amount of users. Downgrading to major
Hide
Permalink
Andrea Aime added a comment -

Hi Mauricio, I'm cc'ing you on this issue because it involves CQL. Can you tell without deep investigation if this cyrillic parsing issue is built into CQL or if, alternatively, has anything to do with URL encoding or generally speaking http transport?

Show
Andrea Aime added a comment - Hi Mauricio, I'm cc'ing you on this issue because it involves CQL. Can you tell without deep investigation if this cyrillic parsing issue is built into CQL or if, alternatively, has anything to do with URL encoding or generally speaking http transport?
Hide
Permalink
Andrea Aime added a comment -

What I see from the filter is that your attribute name is in cyrillic, not the data itself. Afaik the CQL specification allows only for attribute names in ASCII chars.
Mauricio, is ECQL attribute name escaping able to deal with other charset?

Show
Andrea Aime added a comment - What I see from the filter is that your attribute name is in cyrillic, not the data itself. Afaik the CQL specification allows only for attribute names in ASCII chars. Mauricio, is ECQL attribute name escaping able to deal with other charset?
Hide
Permalink
Mauricio Pazos added a comment -

Hi Andrea: the implementation is available in trunk for CQL and ECQL. Have a look at
org.geotools.filter.text.cql2.CQLAttributeNameTest
method (test case):
localCharactersetInProperty()

Unfortunately, this solution is not complete. I found some problem using Korean words.
I have thought some solutions, but I need a couple of free days in order to hack with javacc or develop an alternative.

Show
Mauricio Pazos added a comment - Hi Andrea: the implementation is available in trunk for CQL and ECQL. Have a look at org.geotools.filter.text.cql2.CQLAttributeNameTest method (test case): localCharactersetInProperty() Unfortunately, this solution is not complete. I found some problem using Korean words. I have thought some solutions, but I need a couple of free days in order to hack with javacc or develop an alternative.
Hide
Permalink
Stefan added a comment -

Hi,
not only attribute name. I have column whit Latin name of attribute name and data are in Cyrillic.
For example IDENT = ДОБРИЧ.

Again the same error:
Could not parse CQL filter list. Lexical error at line 1, column 1. Encountered : "\u041d"(1053), after Parsing: IDENT = ДОБРИЧ.

Thanks

Show
Stefan added a comment - Hi, not only attribute name. I have column whit Latin name of attribute name and data are in Cyrillic. For example IDENT = ДОБРИЧ. Again the same error: Could not parse CQL filter list. Lexical error at line 1, column 1. Encountered : "\u041d"(1053), after Parsing: IDENT = ДОБРИЧ. Thanks
Hide
Permalink
Stefan added a comment -

This is error in data , not in name of column/attribute name - it is in Latin "fid"/.

Show
Stefan added a comment - This is error in data , not in name of column/attribute name - it is in Latin "fid"/.
Hide
Permalink
Stefan added a comment - - edited

HI,
I upload example SHP file, that contain cyrillic data in attribute table.:

data.rar

Show
Stefan added a comment - - edited HI, I upload example SHP file, that contain cyrillic data in attribute table.: data.rar
Hide
Permalink
Mauricio Pazos added a comment -

Understood, the literal string expression requires a similar improvement.

Show
Mauricio Pazos added a comment - Understood, the literal string expression requires a similar improvement.
Hide
Permalink
Stefan added a comment -

The situation is similar with old version of Geoserver.
I tried all versions but I don't have solution for problem with cyrillic data.
Example file: "data.rar"
Is it possible somebody to give solution or idea how to fix it?

Show
Stefan added a comment - The situation is similar with old version of Geoserver. I tried all versions but I don't have solution for problem with cyrillic data. Example file: "data.rar" Is it possible somebody to give solution or idea how to fix it?
Hide
Permalink
Mauricio Pazos added a comment -

My Options are:

  • If it is not critical you can try to modify the lexical process and send me the patch.
  • Wait for my free time.
  • If this bug is critical for your project, Axios could offer a professional service, in order to reduce your risk and cost.
Show
Mauricio Pazos added a comment - My Options are: If it is not critical you can try to modify the lexical process and send me the patch. Wait for my free time. If this bug is critical for your project, Axios could offer a professional service, in order to reduce your risk and cost.
Hide
Permalink
Stefan added a comment -

Ok,thanks.

I will wait for your solution.

Show
Stefan added a comment - Ok,thanks. I will wait for your solution.
Hide
Permalink
Mauricio Pazos added a comment -

The following issue was added in GeoTools project:
http://jira.codehaus.org/browse/GEOT-3296

Show
Mauricio Pazos added a comment - The following issue was added in GeoTools project: http://jira.codehaus.org/browse/GEOT-3296
Hide
Permalink
Mauricio Pazos added a comment -

Hello Andrea: GEOT-3296 was fixed, I think you can resolve this issue.

Show
Mauricio Pazos added a comment - Hello Andrea: GEOT-3296 was fixed, I think you can resolve this issue.
Hide
Permalink
Andrea Aime added a comment -

Fixed on GeoTools

Show
Andrea Aime added a comment - Fixed on GeoTools
Hide
Permalink
Andrea Aime added a comment -

Mass closing all issues that have been in the resolved state for the last month without anyone commenting or reopening them

Show
Andrea Aime added a comment - Mass closing all issues that have been in the resolved state for the last month without anyone commenting or reopening them

People

  • Assignee:
    Andrea Aime
    Reporter:
    Stefan
Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: