Maven 2 & 3

the encoding parameter in xml declaration of POM is ignored

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: 2.0.8
  • Component/s: POM::Encoding
  • Labels:
    None
  • Complexity:
    Intermediate
  • Patch Submitted:
    Yes
  • Number of attachments :
    8

Description

DefaultMavenProjectBuilder reads POM in system default character encoding, and the encoding parameter in xml declartion is ignored.
to fix this problem, We should

  • fix modello-plugin-xpp3 to use the xml parser which is able to handle the encoding parameter properly
  • regenerate maven-model using fixed modello-plugin-xpp3
  • fix DefaultMavenProjectBuilder to use regenerated maven-model properly.

Attachments

  1. File
    DefaultMavenProjectBuilder.diff
    29/Apr/06 1:41 AM
    3 kB
    Naoki Nose
  2. File
    MNG-2254_artifact.diff
    21/Aug/07 1:52 AM
    2 kB
    Herve Boutemy
  3. File
    MNG-2254_components.diff
    25/Aug/07 9:20 AM
    12 kB
    Herve Boutemy
  4. File
    MNG-2254_components-new.diff
    12/Oct/07 2:26 PM
    13 kB
    Herve Boutemy
  5. File
    MNG-2254.diff
    15/Jul/07 3:46 AM
    3 kB
    Herve Boutemy
  6. File
    MNG-2254-2.diff
    15/Jul/07 8:51 AM
    10 kB
    Herve Boutemy
  7. File
    mng-2254-PomEncoding.tgz
    11/Oct/07 4:05 PM
    2 kB
    Herve Boutemy
  8. File
    modello-plugin-xpp3.diff
    29/Apr/06 1:40 AM
    8 kB
    Naoki Nose

Issue Links

This issue is duplicated by:
MNG-2025 POM is still not read using the right encoding Critical Closed
MNG-2618 pom.xml with amperstands and variables Major Closed
MNG-2932 Encoding chaos Major Closed
MNG-2818 maven 2 requires a UTF-8 locale to build properly Major Closed
MNG-2057 The Maven 2.0.2 XML parser fails to parse a UTF-8 POM that begins with the optional byte-order mark. Major Closed
This issue relates to:
CONTINUUM-2507 Error releasing project with POM encoded in UTF-8 with BOM Minor Open
MPLUGINTESTING-5 Respect file encoding of POMs Major Closed
DOXIA-60 Use a external XML Pull parser instead of plexus one Critical Open
MNG-2255 Use a external XML Pull parser instead of plexus one Critical Closed
MREPOSITORY-10 add XML encoding support for POM reading/writing Major Closed
MDEPLOY-66 add XML encoding support for POM reading/writing Major Closed
MINSTALL-44 add XML encoding support for POM reading/writing Major Closed
This issue is related to:
MNG-2362 Deployed POM is not valid XML Critical Closed
MRELEASE-87 Poms are written with wrong encodings Critical Closed
MNG-2216 Add default encodings section to POM Major Open
MNG-2215 Change the default input/output encoding to UTF-8 instead of ISO-8859-1 Major Closed
MNG-2187 Improve error message when the pom is encoded in the wrong charset Minor Closed
This issue depends upon:
PLXUTILS-11 MXParser can't handle the encoding declaration in XML declaration Major Closed
This issue is depended upon by:
MANTTASKS-14 Ant Tasks do not work on the ZOS Critical Open
MANTTASKS-79 add XML encoding support for pom.xml and settings.xml Major Closed

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Naoki Nose added a comment -

Attached files are the patch to fix this issue.
These files depend on the patch of MXParser which I attached to the following issue.

http://jira.codehaus.org/browse/MNG-2148

using a external standard xml pull parser may be better solution, as discussed in the following issue.

http://jira.codehaus.org/browse/DOXIA-60

Show
Naoki Nose added a comment - Attached files are the patch to fix this issue. These files depend on the patch of MXParser which I attached to the following issue. http://jira.codehaus.org/browse/MNG-2148 using a external standard xml pull parser may be better solution, as discussed in the following issue. http://jira.codehaus.org/browse/DOXIA-60
Hide
Permalink
Carlos Sanchez added a comment -

it'd be better to swith to a external xml parser, see MNG-2255

Show
Carlos Sanchez added a comment - it'd be better to swith to a external xml parser, see MNG-2255
Hide
Permalink
Stefan Hübner added a comment -

As I just stated in MNG-2255 I think, that "...the parser used to read in maven's model isn't the problem really. It's more likely the way the POM is handed to the parser."

See MNG-2255 for full discussion of the issue.

Show
Stefan Hübner added a comment - As I just stated in MNG-2255 I think, that "...the parser used to read in maven's model isn't the problem really. It's more likely the way the POM is handed to the parser." See MNG-2255 for full discussion of the issue.
Hide
Permalink
Herve Boutemy added a comment -

for problems about encodings, there are ways to write a Reader that automagically detects the encoding of a XML stream:
see https://rome.dev.java.net/apidocs/0_5/com/sun/syndication/io/XmlReader.html

this permits to use a reader as a source for an XML parser without loosing any information: the magic is done by the reader, not by the parser.

Such a solution would be an easy fix: the actual MavenXpp3Reader signature could be sufficient (even if using an InputStream would be more classical)

Show
Herve Boutemy added a comment - for problems about encodings, there are ways to write a Reader that automagically detects the encoding of a XML stream: see https://rome.dev.java.net/apidocs/0_5/com/sun/syndication/io/XmlReader.html this permits to use a reader as a source for an XML parser without loosing any information: the magic is done by the reader, not by the parser. Such a solution would be an easy fix: the actual MavenXpp3Reader signature could be sufficient (even if using an InputStream would be more classical)
Hide
Permalink
Jason van Zyl added a comment -

Herve, if you want take a crack at this I would certainly incorporate this into Modello.

Show
Jason van Zyl added a comment - Herve, if you want take a crack at this I would certainly incorporate this into Modello.
Hide
Permalink
Herve Boutemy added a comment -

Here is a patch to use XmlReader from plexus-utils 1.4.3-SNAPSHOT from MPLEXUS-11 branch.
Modello change is not absolutely necessary: adding read(InputStream) to read(Reader) is only an enhencement to make people prefer the first one, but using the second one with an XmlReader instance is strictly the same

Show
Herve Boutemy added a comment - Here is a patch to use XmlReader from plexus-utils 1.4.3-SNAPSHOT from MPLEXUS-11 branch. Modello change is not absolutely necessary: adding read(InputStream) to read(Reader) is only an enhencement to make people prefer the first one, but using the second one with an XmlReader instance is strictly the same
Hide
Permalink
Herve Boutemy added a comment -

Here is a new patch fixing encoding detection for pom.xml, settings.xml and profiles.xml

Show
Herve Boutemy added a comment - Here is a new patch fixing encoding detection for pom.xml, settings.xml and profiles.xml
Hide
Permalink
Herve Boutemy added a comment -

see http://docs.codehaus.org/display/MAVENUSER/XML+encoding for the roadmap to XML encoding support in Maven2

Show
Herve Boutemy added a comment - see http://docs.codehaus.org/display/MAVENUSER/XML+encoding for the roadmap to XML encoding support in Maven2
Hide
Permalink
Jason van Zyl added a comment -

I will take a look, I have walked through everything else this week: plexus and modello, so now I can push in the changes into Maven's core.

Show
Jason van Zyl added a comment - I will take a look, I have walked through everything else this week: plexus and modello, so now I can push in the changes into Maven's core.
Hide
Permalink
Herve Boutemy added a comment -

splitted the patch into 2 pieces since maven-artifact has been moved to artifact

Show
Herve Boutemy added a comment - splitted the patch into 2 pieces since maven-artifact has been moved to artifact
Hide
Permalink
Andrew Williams added a comment -

Can you please clarify which patches apply and which do not

Show
Andrew Williams added a comment - Can you please clarify which patches apply and which do not
Hide
Permalink
Herve Boutemy added a comment - - edited

the 2 latest only are to be applied:

  • {{MNG-2254_artifact.diff}} for maven/artifact/trunk and maven/components/branches/maven-2.0.x/maven-artifact
  • {{MNG-2254_components.diff}} for maven/components/trunk and maven/components/branches/maven-2.0.x
Show
Herve Boutemy added a comment - - edited the 2 latest only are to be applied:
  • {{MNG-2254_artifact.diff}} for maven/artifact/trunk and maven/components/branches/maven-2.0.x/maven-artifact
  • {{MNG-2254_components.diff}} for maven/components/trunk and maven/components/branches/maven-2.0.x
Hide
Permalink
Brian Fox added a comment -

Can someone make an IT for this? See the instructions here: http://docs.codehaus.org/display/MAVEN/Creating+a+Maven+Integration+Test

Show
Brian Fox added a comment - Can someone make an IT for this? See the instructions here: http://docs.codehaus.org/display/MAVEN/Creating+a+Maven+Integration+Test
Hide
Permalink
Herve Boutemy added a comment -

Hi Brian,

Here is an IT to check that pom.xml encoding is properly detected: the sample pom.xml is in utf-16, then if encoding is not detected, the build simply fails (no complicated test to check that an accentuated character would have been correctly read)

I checked that it fails without the patch, not that it works with the patch applied...

Show
Herve Boutemy added a comment - Hi Brian, Here is an IT to check that pom.xml encoding is properly detected: the sample pom.xml is in utf-16, then if encoding is not detected, the build simply fails (no complicated test to check that an accentuated character would have been correctly read) I checked that it fails without the patch, not that it works with the patch applied...
Hide
Permalink
Brian Fox added a comment -

Thanks Herve, the test looks good. I wasn't able to get the patches applied cleanly. Can you update them and then I'll apply and test? (i'm going to add the test to the core suite)

Show
Brian Fox added a comment - Thanks Herve, the test looks good. I wasn't able to get the patches applied cleanly. Can you update them and then I'll apply and test? (i'm going to add the test to the core suite)
Hide
Permalink
Herve Boutemy added a comment -

ok: here is a new patch

Show
Herve Boutemy added a comment - ok: here is a new patch
Hide
Permalink
Herve Boutemy added a comment -

work done:

  • r585265 for trunk
  • r585268 for 2.0.x branch
Show
Herve Boutemy added a comment - work done:
  • r585265 for trunk
  • r585268 for 2.0.x branch

People

Vote (9)
Watch (4)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.