UTF-8 Support

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.0-beta-2
  • Fix Version/s: 2.0
  • Labels:
    None
  • Environment:
    Linux, Java 1.6, Maven 2.0.6

Description

I am using czech language in sources. When I run mvn javancss:report I got this:

[INFO] [javancss:report]
TokenMgrError in /home/fers/prj/exekutori/pes/shared/tools/tools/src/main/java/cz/softeu/tools/hibernate/ServletPrinter.java
Lexical error at line 120, column 34. Encountered: "\r" (13), after : "\"Po"

Caused by: java.lang.Exception: javancss.TokenMgrError: Lexical error at line 120, column 34. Encountered: "\r" (13), after : "\"Po"
... 23 more
Caused by: javancss.TokenMgrError: Lexical error at line 120, column 34. Encountered: "\r" (13), after : "\"Po"
at javancss.JavaParserTokenManager.getNextToken(JavaParserTokenManager.java:1781)
at javancss.JavaParser.jj_ntk(JavaParser.java:11397)
at javancss.JavaParser.VariableInitializer(JavaParser.java:1585)
at javancss.JavaParser.VariableDeclarator(JavaParser.java:1546)
at javancss.JavaParser.LocalVariableDeclaration(JavaParser.java:3771)
at javancss.JavaParser.BlockStatement(JavaParser.java:3691)
at javancss.JavaParser.Block(JavaParser.java:3684)
at javancss.JavaParser.MethodDeclaration(JavaParser.java:1846)
at javancss.JavaParser.ClassBodyDeclaration(JavaParser.java:975)
at javancss.JavaParser.ClassBody(JavaParser.java:846)
at javancss.JavaParser.UnmodifiedClassDeclaration(JavaParser.java:775)
at javancss.JavaParser.ClassDeclaration(JavaParser.java:706)
at javancss.JavaParser.TypeDeclaration(JavaParser.java:567)
at javancss.JavaParser.CompilationUnit(JavaParser.java:300)
at javancss.Javancss._measureSource(Javancss.java:190)
at javancss.Javancss._measureSource(Javancss.java:155)
at javancss.Javancss._measureFiles(Javancss.java:282)
at javancss.Javancss._measureRoot(Javancss.java:308)
at javancss.Javancss.<init>(Javancss.java:604)
at org.codehaus.mojo.javancss.NcssExecuter.execute(NcssExecuter.java:90)
... 22 more

The file contains:
String text = "Počet záznamů: <input type=\"text\" name=\"max\" size=\"30\" value=\"" + m + "\" />";

(non ascii characters with UTF-8)

Issue Links

depends upon

New Feature - A new feature of the product, which has yet to be developed. JAVANCSS-6 add an option to define encoding used to read source files

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.
is related to

Bug - A problem which impairs or prevents the functions of the product. MJNCSS-10 Plugin crashes when Java elements have special characters

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

New Feature - A new feature of the product, which has yet to be developed. MJNCSS-31 add an "encoding" parameter for source files and use ${project.build.sourceEncoding} as default value

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Jean-Laurent de Morlhon added a comment -

I believe this issue is related to MJNCSS-10.

Could you try to fix the problem by adding a forceEncoding parameter to the proper value (utf-8) to your javancss maven plugin configuration ?

For a little more doc : http://mojo.codehaus.org/javancss-maven-plugin/report-mojo.html#forceEncoding

Tells us if it fixes your issue.

Show
Jean-Laurent de Morlhon added a comment - I believe this issue is related to MJNCSS-10. Could you try to fix the problem by adding a forceEncoding parameter to the proper value (utf-8) to your javancss maven plugin configuration ? For a little more doc : http://mojo.codehaus.org/javancss-maven-plugin/report-mojo.html#forceEncoding Tells us if it fixes your issue.
Hide
Permalink
Rémy Sanlaville added a comment -

I also have the same problem.
I try to use the forceEncoding parameter to the utf-8 value and it still does not work:
> Lexical error at line 54, column 30. Encountered: "\u00a7" (167), after : ""

My config;
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>javancss-maven-plugin</artifactId>
<version>2.0-beta-2</version>
<configuration>
<forceEncoding>utf-8</forceEncoding>
</configuration>
</plugin>

Show
Rémy Sanlaville added a comment - I also have the same problem. I try to use the forceEncoding parameter to the utf-8 value and it still does not work: > Lexical error at line 54, column 30. Encountered: "\u00a7" (167), after : "" My config; <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>javancss-maven-plugin</artifactId> <version>2.0-beta-2</version> <configuration> <forceEncoding>utf-8</forceEncoding> </configuration> </plugin>
Hide
Permalink
takanori added a comment - - edited

I am using javancss-maven-plugin 2.0-beta-2.

I crate the patch for it.
--->NcssReportMojo_diff.txt

Even if JavaNCSS throw an exception,
it will be forced to output the report.

I crate a new option 'failOnError'.

When the source encoding is UTF-8,
it is fail to output the report.
I think it is the JavaNCSS problem.

But, even in such cases,
JavaNCSS will output the report(xml).

So, even if JavaNCSS throw an exception,
I want to get the report.

Please marge this patch.

Show
takanori added a comment - - edited I am using javancss-maven-plugin 2.0-beta-2. I crate the patch for it. --->NcssReportMojo_diff.txt Even if JavaNCSS throw an exception, it will be forced to output the report. I crate a new option 'failOnError'. When the source encoding is UTF-8, it is fail to output the report. I think it is the JavaNCSS problem. But, even in such cases, JavaNCSS will output the report(xml). So, even if JavaNCSS throw an exception, I want to get the report. Please marge this patch.
Hide
Permalink
Herve Boutemy added a comment -

I just checked javancss sources:

  • there is no encoding support to read java source files: platform encoding is used, and no configuration is provided
  • XML output file is written without encoding specification in the XML stream, then it should normally be UTF-8, but it the file is written with platform encoding, which is a real bug

In this situation, javancss-maven-plugin does what is possible: it uses platform encoding whenever possible, to be consistent with javancss.
But if your sources encoding isn't consistent with your platform encoding, there is nothing javancss-maven-plugin can do until javancss encoding support is made configurable.

I'll contact javancss author to propose help for encoding support.

Show
Herve Boutemy added a comment - I just checked javancss sources:
  • there is no encoding support to read java source files: platform encoding is used, and no configuration is provided
  • XML output file is written without encoding specification in the XML stream, then it should normally be UTF-8, but it the file is written with platform encoding, which is a real bug
In this situation, javancss-maven-plugin does what is possible: it uses platform encoding whenever possible, to be consistent with javancss. But if your sources encoding isn't consistent with your platform encoding, there is nothing javancss-maven-plugin can do until javancss encoding support is made configurable. I'll contact javancss author to propose help for encoding support.
Hide
Permalink
Gregory Levilain added a comment -

Hello, my sources are encoded in UTF-8, and I also get an error from javancss-maven-plugin 2.0-beta-2, while parsing a String containing special characters. Isn't it possible to deal with this kind of error as Warnings instead of errors, so the build will succeed anyway ?
Here is the error log:

[INFO] [javancss:report]
ParseException in D:_pd_leroymerlin\vdc\modules\metier\src\main\java\com\adeo\vdc\acteur\manager\impl\ActeurManagerImpl.java
Last useful checkpoint: "com.adeo.vdc.acteur.manager.impl.ActeurManagerImpl.cleanString(String)"
Encountered "\u00c3" at line 259, column 76.
Was expecting one of:
"instanceof" ...
";" ...
"," ...
">" ...
"<" ...
"?" ...
"==" ...
"<=" ...
">=" ...
"!=" ...
"||" ...
"&&" ...
"++" ...
"--" ...
"+" ...
"-" ...
"*" ...
"/" ...
"&" ...
"|" ...
"^" ...
"%" ...
"<<" ...
">>" ...
">>>" ...
"." ...
"[" ...
"(" ...
"=" ...
"*=" ...
"/=" ...
"%=" ...
"+=" ...
"-=" ...
"<<=" ...
">>=" ...
">>>=" ...
"&=" ...
"^=" ...
"|=" ...

Show
Gregory Levilain added a comment - Hello, my sources are encoded in UTF-8, and I also get an error from javancss-maven-plugin 2.0-beta-2, while parsing a String containing special characters. Isn't it possible to deal with this kind of error as Warnings instead of errors, so the build will succeed anyway ? Here is the error log: [INFO] [javancss:report] ParseException in D:_pd_leroymerlin\vdc\modules\metier\src\main\java\com\adeo\vdc\acteur\manager\impl\ActeurManagerImpl.java Last useful checkpoint: "com.adeo.vdc.acteur.manager.impl.ActeurManagerImpl.cleanString(String)" Encountered "\u00c3" at line 259, column 76. Was expecting one of: "instanceof" ... ";" ... "," ... ">" ... "<" ... "?" ... "==" ... "<=" ... ">=" ... "!=" ... "||" ... "&&" ... "++" ... "--" ... "+" ... "-" ... "*" ... "/" ... "&" ... "|" ... "^" ... "%" ... "<<" ... ">>" ... ">>>" ... "." ... "[" ... "(" ... "=" ... "*=" ... "/=" ... "%=" ... "+=" ... "-=" ... "<<=" ... ">>=" ... ">>>=" ... "&=" ... "^=" ... "|=" ...
Hide
Permalink
Jean-Laurent de Morlhon added a comment -

Best way to deal with it, right now, if you have already played with the forceEncoding option (which is a workaround : http://mojo.codehaus.org/javancss-maven-plugin/report-mojo.html#forceEncoding) is to exclude the sources from being parsed if they are a few of them which causes problem (see http://mojo.codehaus.org/javancss-maven-plugin/examples/include.html)

Show
Jean-Laurent de Morlhon added a comment - Best way to deal with it, right now, if you have already played with the forceEncoding option (which is a workaround : http://mojo.codehaus.org/javancss-maven-plugin/report-mojo.html#forceEncoding) is to exclude the sources from being parsed if they are a few of them which causes problem (see http://mojo.codehaus.org/javancss-maven-plugin/examples/include.html)
Hide
Permalink
Herve Boutemy added a comment -

XML encoding fixed in r9684 (by upgrading JavaNCSS dependency to 30.51 to benefit from JAVANCSS-5).
encoding support for source files will be treated in MJNCSS-31

Show
Herve Boutemy added a comment - XML encoding fixed in r9684 (by upgrading JavaNCSS dependency to 30.51 to benefit from JAVANCSS-5). encoding support for source files will be treated in MJNCSS-31

People

  • Assignee:
    Herve Boutemy
    Reporter:
    Petr Ferschmann
Vote (6)
Watch (4)

Dates

  • Created:
    Updated:
    Resolved: