GeoTools
  1. GeoTools
  2. GEOT-2623

Fail to read GBK attribute in dbf

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.6-M1
    • Fix Version/s: 2.6-RC1
    • Component/s: shapefile plugin
    • Labels:
      None
    • Environment:
      win32 jre1.5

      Description

      current version doesn't take chinsese character into account.

      1. DbaseFileReader.java
        19 kB
        limonolith
      2. DbfTest.java
        5 kB
        limonolith
      3. GEOT-2623_testedWithGeopublisher.patch
        22 kB
        Stefan Alfons Tzeggai
      4. utf8supportAndTests.tar.gz
        6 kB
        Stefan Alfons Tzeggai
      1. modified.jpg
        73 kB
      2. qgis.jpg
        248 kB
      3. udig.jpg
        77 kB

        Activity

        Hide
        limonolith added a comment -
        We can create a shapefile programaticlly and read the attributes again to validate the function.
        In the DbfTest, I add a point with attributes(Name,Adress,Weight).When I read the attribute from again the name and address changed.
        Show
        limonolith added a comment - We can create a shapefile programaticlly and read the attributes again to validate the function. In the DbfTest, I add a point with attributes(Name,Adress,Weight).When I read the attribute from again the name and address changed.
        Hide
        Michael Steigemann added a comment -
        Two shapefile sets with switched column order. They can be used to find the filtering error, if there are special characters like ß in a column.
        Show
        Michael Steigemann added a comment - Two shapefile sets with switched column order. They can be used to find the filtering error, if there are special characters like ß in a column.
        Hide
        Stefan Alfons Tzeggai added a comment -
        I had some UTF8 problems. It turned out that the DbaseFileReader.java attached by limonolith just fixed all my problems.

        My attached utf8supportAndTests.tar.gz contains

        1. a small and royalty-free shapefile with an UTF8 encoded DBF

        2. a patch that
        ** adds some JUnit-Tests that access the utf8 attributes
        ** replaces the existing DbaseFileReader with the one given by limonolith
        ** adds the optimizations from http://jira.codehaus.org/browse/GEOT-2683 to the new DbfFileReader.

        "mvn test" just passed... so i would be happy to see this comitted next week.


        Show
        Stefan Alfons Tzeggai added a comment - I had some UTF8 problems. It turned out that the DbaseFileReader.java attached by limonolith just fixed all my problems. My attached utf8supportAndTests.tar.gz contains 1. a small and royalty-free shapefile with an UTF8 encoded DBF 2. a patch that ** adds some JUnit-Tests that access the utf8 attributes ** replaces the existing DbaseFileReader with the one given by limonolith ** adds the optimizations from http://jira.codehaus.org/browse/GEOT-2683 to the new DbfFileReader. "mvn test" just passed... so i would be happy to see this comitted next week.
        Hide
        Stefan Alfons Tzeggai added a comment -
        After Andrea reviewed the patch and made it compatible with Java 1.5, i tested the patch again and did some minimal changes:
        * removed try catch UnsupportedCharset() around the utf8 test, because utf8 is a standard charset of every JVM
        * oneBytePerChar = encoder.maxBytesPerChar() == 1.0f removed the second double check: "&& encoder.averageBytesPerChar() == 1.0f; "
        * added final modifies where possible for performance and removed unused imports

        Please don't forget to commit the binary resource files contained in the earlier .tar.gz for the JUnit tests.
        Show
        Stefan Alfons Tzeggai added a comment - After Andrea reviewed the patch and made it compatible with Java 1.5, i tested the patch again and did some minimal changes: * removed try catch UnsupportedCharset() around the utf8 test, because utf8 is a standard charset of every JVM * oneBytePerChar = encoder.maxBytesPerChar() == 1.0f removed the second double check: "&& encoder.averageBytesPerChar() == 1.0f; " * added final modifies where possible for performance and removed unused imports Please don't forget to commit the binary resource files contained in the earlier .tar.gz for the JUnit tests.
        Hide
        Andrea Aime added a comment -
        Applied on trunk.
        Thanks for all the help everybody.
        Show
        Andrea Aime added a comment - Applied on trunk. Thanks for all the help everybody.

          People

          • Assignee:
            Andrea Aime
            Reporter:
            limonolith
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: