Maven JAR Plugin
  1. Maven JAR Plugin
  2. MJAR-50

"Invalid Header" in jar's Manifest (Specification-Title attribute) when tab char in pom Description

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0
    • Fix Version/s: 2.2
    • Component/s: None
    • Labels:
      None
    • Number of attachments :
      0

      Description

      I typed a <description> element into my pom; the description spanned 2 lines, and my text editor put a tab char at the beginning of the 2nd line.
      The project built correctly, but another project depended on the resulting jar.
      The compilation of the depending project failed with "invalid header" on the jar I'd just compiled.

      When I looked at the jar's manifest, I saw the Specification-Title attribute had the tab character still in it from the <description> element.
      Replacing the tab with spaces in the pom solved the problem.
      However, this was valid xml that caused a compilation problem, so I think it's a minor bug with the jar production code.
      Note that the full description is trimmed before going into the manifest, so any

      Easy to work around, but it would be nice if whitespace in the <description> element were consolidated into spaces before copying to the manifest.
      Doesn't need any fancy layout stuff - just any multiple instances of tabs, spaces, lf or cr should be replaced by a single space.
      Try (I haven't tried to compile this!):

          public String consolidateWhitespace(String input){
              StringTokenizer st = new StringTokenizer(input);
              StringBuffer rv = new StringBuffer();
              while (st.hasMoreTokens()){
                  rv.append(st.nextToken() + (st.hasMoreTokens()?" ":""));
              }
              return rv.toString();
          }
      

        Issue Links

          Activity

          Hide
          Mike Laurie added a comment -

          Sorry - in the middle of that description, I meant to say:
          "Note that the full description is trimmed before going into the manifest (I think), so any leading or trailing tabs don't seem recreate the problem."

          Show
          Mike Laurie added a comment - Sorry - in the middle of that description, I meant to say: "Note that the full description is trimmed before going into the manifest (I think), so any leading or trailing tabs don't seem recreate the problem."
          Hide
          Wendy Smoak added a comment -

          Possible duplicate of MJAR-4 and MJAR-34

          Show
          Wendy Smoak added a comment - Possible duplicate of MJAR-4 and MJAR-34
          Hide
          Brett Porter added a comment -

          since MJAR-4 and MJAR-34 are fixed, can anyone confirm this one is also fixed?

          Show
          Brett Porter added a comment - since MJAR-4 and MJAR-34 are fixed, can anyone confirm this one is also fixed?
          Hide
          Dennis Lundberg added a comment -

          I'll test it.

          Show
          Dennis Lundberg added a comment - I'll test it.
          Hide
          Dennis Lundberg added a comment -

          This is with a two-line <description> in the pom, second line starting with a tab character.

          Using no configuration options I don't see the "Specification-Title" attribute in the manifest at all.

          Using this configuration:
          <configuration>
          <archive>
          <manifestEntries>
          <Specification-Title>$

          {pom.description}</Specification-Title>
          </manifestEntries>
          </archive>
          </configuration>
          gives me this error message:
          "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section"
          which is strange considering that it wasn't there in the first example.

          Using this configuration:
          <configuration>
          <archive>
          <manifestEntries>
          <custom>${pom.description}

          </custom>
          </manifestEntries>
          </archive>
          </configuration>
          gives me the same error as above.

          So I've had no luck in reproducing the problem reported in this issue, but might have discovered another one...

          Show
          Dennis Lundberg added a comment - This is with a two-line <description> in the pom, second line starting with a tab character. Using no configuration options I don't see the "Specification-Title" attribute in the manifest at all. Using this configuration: <configuration> <archive> <manifestEntries> <Specification-Title>$ {pom.description}</Specification-Title> </manifestEntries> </archive> </configuration> gives me this error message: "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section" which is strange considering that it wasn't there in the first example. Using this configuration: <configuration> <archive> <manifestEntries> <custom>${pom.description} </custom> </manifestEntries> </archive> </configuration> gives me the same error as above. So I've had no luck in reproducing the problem reported in this issue, but might have discovered another one...
          Hide
          Mike Laurie added a comment -

          The problem has indeed been masked because of other fixes.
          3 things left to consider:

          • I think newlines in manifest values hasn't actually been fixed in maven-archiver
          • maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0 so jar-plugin pom needs updating if archiver-2.1 is to be used.
          • The Embedded error: The attribute ... may not occur more than once in the same section seems to be a new problem (regression) in archiver-2.1 - I've reproduced it, and it stops me from testing the newlines issue. It's not restricted to the Specification-Title attribute.

          Discussion:
          The issue at hand has been masked by changes in MJAR-39.
          The fix in MJAR-39 ensures that the Specification-Title manifest attribute (if included) now contains project name (not project description)
          However, I found (like you, Dennis) that the Specification-Title was not in the jar's manifest at all.
          This is in turn an effect of MJAR-38 whereby the Specification-... elements are not included by default.

          I've just tested this in maven-archiver-2.1, which was released on 27th June, and descriptions containing newline-tab combinations no longer cause an invalid header error.

          However , the pom for maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0:

          maven-jar-plugin-2.0.pom
          ...
          <dependency>
              <groupId>org.apache.maven</groupId>
              <artifactId>maven-archiver</artifactId>
              <version>2.0</version>
          </dependency>
          ...

          This means that by default it's still using the old archiver.
          When I updated my local repository's pom for maven-jar-plugin-2.0 to require version 2.1, I was able to test the new version.
          Can the pom be changed, or a new version of it released? I'm not sure how these things are done.

          I discovered that the deviation from the jar spec is actually the inclusion of linefeeds in the continuation lines which I don't think has actually been fixed.
          For reference, here's part of the jar spec from Sun, indicating why the problem happened:

            newline:      CR LF | LF | CR (not followed by LF) 
            value:        SPACE *otherchar newline *continuation 
            continuation: SPACE *otherchar newline 
            otherchar:    any UTF-8 character except NUL, CR and LF
          

          In an attribute value, a newline followed by a tab is not valid; a newline followed by a space to indicate a continuation would be valid.

          If there are further similar problems with jar manifests, then it may be the plexus jar class that's at fault.
          (org.codehaus.plexus.archiver.jar.Manifest)
          I don't think it should allow a manifest attrubite to contain \r or \n characters.

          To ensure Maven doesn't add \r or \n chars to manifest attributes, they could be stripped out in MavenArchiver:

          org.apache.maven.archiver.MavenArchiver line 107
              private void addManifestAttribute( Manifest manifest, String key, String value )
                  throws ManifestException
              {
                  // Use the empty string to suppress a Manifest entry
                  if ( value != null && !"".equals( value ) )
                  {
          /*+*/
                      //Don't allow newlines in manifest attribute values - plexus jar doesn't 
                      // currently strip them out, and they can result in an invalid manifest.
                      value=value.replace('\r',' ');
                      value=value.replace('\n',' ');
          /*+*/
                      Manifest.Attribute attr = new Manifest.Attribute( key, value );
                      manifest.addConfiguredAttribute( attr );
                  }
              }
          

          I don't know why you get "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section"
          I got the same error with the following configuration when I was trying to test:

          pom build/plugins section
            <plugin>
              <groupId>org.apache.maven.plugins</groupId> 
              <artifactId>maven-jar-plugin</artifactId>
              <configuration>
                <archive>
                  <manifestEntries>
                    <Dodgy-Entry>
                      Here's a manifest entry with a newline
          			followed by a tab (actually several tabs)
                    </Dodgy-Entry>
                  </manifestEntries>
                </archive>
              </configuration>
            </plugin>
          

          This doesn't occur with archiver-2.0; in 2.0 the dodgy entry is successfully written to the manifest with the incorrect manifest attribute format:

          snippet from resulting manifest
          Dodgy-Entry: Here's a manifest entry with a newline
          								followed b
           y a tab (actually several tabs)
          
          Show
          Mike Laurie added a comment - The problem has indeed been masked because of other fixes. 3 things left to consider: I think newlines in manifest values hasn't actually been fixed in maven-archiver maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0 so jar-plugin pom needs updating if archiver-2.1 is to be used. The Embedded error: The attribute ... may not occur more than once in the same section seems to be a new problem (regression) in archiver-2.1 - I've reproduced it, and it stops me from testing the newlines issue. It's not restricted to the Specification-Title attribute. Discussion: The issue at hand has been masked by changes in MJAR-39 . The fix in MJAR-39 ensures that the Specification-Title manifest attribute (if included) now contains project name (not project description) However, I found (like you, Dennis) that the Specification-Title was not in the jar's manifest at all. This is in turn an effect of MJAR-38 whereby the Specification-... elements are not included by default. I've just tested this in maven-archiver-2.1, which was released on 27th June, and descriptions containing newline-tab combinations no longer cause an invalid header error. However , the pom for maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0: maven-jar-plugin-2.0.pom ... <dependency> <groupId> org.apache.maven </groupId> <artifactId> maven-archiver </artifactId> <version> 2.0 </version> </dependency> ... This means that by default it's still using the old archiver. When I updated my local repository's pom for maven-jar-plugin-2.0 to require version 2.1, I was able to test the new version. Can the pom be changed, or a new version of it released? I'm not sure how these things are done. I discovered that the deviation from the jar spec is actually the inclusion of linefeeds in the continuation lines which I don't think has actually been fixed. For reference, here's part of the jar spec from Sun , indicating why the problem happened: newline: CR LF | LF | CR (not followed by LF) value: SPACE *otherchar newline *continuation continuation: SPACE *otherchar newline otherchar: any UTF-8 character except NUL, CR and LF In an attribute value, a newline followed by a tab is not valid; a newline followed by a space to indicate a continuation would be valid. If there are further similar problems with jar manifests, then it may be the plexus jar class that's at fault. (org.codehaus.plexus.archiver.jar.Manifest) I don't think it should allow a manifest attrubite to contain \r or \n characters. To ensure Maven doesn't add \r or \n chars to manifest attributes, they could be stripped out in MavenArchiver: org.apache.maven.archiver.MavenArchiver line 107 private void addManifestAttribute( Manifest manifest, String key, String value ) throws ManifestException { // Use the empty string to suppress a Manifest entry if ( value != null && !"".equals( value ) ) { /*+*/ //Don't allow newlines in manifest attribute values - plexus jar doesn't // currently strip them out, and they can result in an invalid manifest. value=value.replace('\r',' '); value=value.replace('\n',' '); /*+*/ Manifest.Attribute attr = new Manifest.Attribute( key, value ); manifest.addConfiguredAttribute( attr ); } } I don't know why you get "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section" I got the same error with the following configuration when I was trying to test: pom build/plugins section <plugin> <groupId> org.apache.maven.plugins </groupId> <artifactId> maven-jar-plugin </artifactId> <configuration> <archive> <manifestEntries> <Dodgy-Entry> Here's a manifest entry with a newline followed by a tab (actually several tabs) </Dodgy-Entry> </manifestEntries> </archive> </configuration> </plugin> This doesn't occur with archiver-2.0; in 2.0 the dodgy entry is successfully written to the manifest with the incorrect manifest attribute format: snippet from resulting manifest Dodgy-Entry: Here's a manifest entry with a newline followed b y a tab (actually several tabs)
          Hide
          Mike Perham added a comment -

          Mike, jar-pugin 2.1 will depend on archiver 2.1. It already does in SVN but has not been released yet. The "more than once in the same section" error needs to be tracked down and squashed before we release jar-plugin 2.1 IMO. If you have any more data on the problem, please open a separate JIRA MJAR issue and post everything you have there.

          Show
          Mike Perham added a comment - Mike, jar-pugin 2.1 will depend on archiver 2.1. It already does in SVN but has not been released yet. The "more than once in the same section" error needs to be tracked down and squashed before we release jar-plugin 2.1 IMO. If you have any more data on the problem, please open a separate JIRA MJAR issue and post everything you have there.
          Hide
          Mike Perham added a comment -

          Ok, I spent an hour looking at this.

          maven-archiver 2.1 is hosed AFAICT. You cannot override manifest entries or merge Class-Path values with that version. When I upgrade jar-plugin to use maven-archiver 2.2-SNAPSHOT, everything works as expected. This is the configuration I played with.

            <build>
          	<plugins>
          	  <plugin>
          		<artifactId>maven-jar-plugin</artifactId>
          		<configuration>
          			<archive>
          				<index>true</index>
          				<manifest>
          					<addDefaultSpecificationEntries>false</addDefaultSpecificationEntries>
          					<addDefaultImplementationEntries>false</addDefaultImplementationEntries>
          					<mainClass>com.acme.Main</mainClass>
          					<addClasspath>true</addClasspath>
          				</manifest>
          				<manifestEntries>
          					<Class-Path>foo/bar</Class-Path>
          					<Specification-Title>Foo</Specification-Title>
          				</manifestEntries>
          			</archive>
          		</configuration>
          	  </plugin>
          	</plugins>
            </build>
          

          It looks like we will need to release maven-archiver again.

          Show
          Mike Perham added a comment - Ok, I spent an hour looking at this. maven-archiver 2.1 is hosed AFAICT. You cannot override manifest entries or merge Class-Path values with that version. When I upgrade jar-plugin to use maven-archiver 2.2-SNAPSHOT, everything works as expected. This is the configuration I played with. <build> <plugins> <plugin> <artifactId> maven-jar-plugin </artifactId> <configuration> <archive> <index> true </index> <manifest> <addDefaultSpecificationEntries> false </addDefaultSpecificationEntries> <addDefaultImplementationEntries> false </addDefaultImplementationEntries> <mainClass> com.acme.Main </mainClass> <addClasspath> true </addClasspath> </manifest> <manifestEntries> <Class-Path> foo/bar </Class-Path> <Specification-Title> Foo </Specification-Title> </manifestEntries> </archive> </configuration> </plugin> </plugins> </build> It looks like we will need to release maven-archiver again.
          Hide
          Mike Perham added a comment -

          BTW with 2.2-SNAPSHOT, it still creates illegal manifest entries with something like:

          <manifestEntries>
          <Specification-Title>$

          {pom.description}

          </Specification-Title>
          </manifestEntries>

          It's just less likely now that we are not populating any fields based on description anymore. So the problem is not fixed, just far less likely to occur.

          Show
          Mike Perham added a comment - BTW with 2.2-SNAPSHOT, it still creates illegal manifest entries with something like: <manifestEntries> <Specification-Title>$ {pom.description} </Specification-Title> </manifestEntries> It's just less likely now that we are not populating any fields based on description anymore. So the problem is not fixed, just far less likely to occur.
          Hide
          Mike Laurie added a comment -

          Wow - thanks Mike; you've been busy!
          It's good to know the "more than once in the same section" error is already fixed in archiver 2.2.

          The only thing causing this MJAR issue now seems to be the possibility of including \r and \n characters in manifest header fields. While we're at it, I guess that NUL characters also ought to be quashed since they're the only other invalid attribute chars according to the sun spec. I think these characters should be replaced with spaces as a matter of course.

          Last question: Does the archiver-plugin take responsibility for this, or should plexus change their jar.Manifest class to replace them? I think plexus should be changed anyway, since it currently allows invalid jar manifests to be created. However, is it worth changing the archiver-plugin to fix the problem now, since it's an easy fix and there needs to be a release soon anyway?

          Show
          Mike Laurie added a comment - Wow - thanks Mike; you've been busy! It's good to know the "more than once in the same section" error is already fixed in archiver 2.2. The only thing causing this MJAR issue now seems to be the possibility of including \r and \n characters in manifest header fields. While we're at it, I guess that NUL characters also ought to be quashed since they're the only other invalid attribute chars according to the sun spec. I think these characters should be replaced with spaces as a matter of course. Last question: Does the archiver-plugin take responsibility for this, or should plexus change their jar.Manifest class to replace them? I think plexus should be changed anyway, since it currently allows invalid jar manifests to be created. However, is it worth changing the archiver-plugin to fix the problem now, since it's an easy fix and there needs to be a release soon anyway?
          Hide
          Mike Laurie added a comment -

          Plexus are aware of this issue and it will get fixed there (PLX-185).
          I guess this issue doesn't need any more attention, so despite the fact that it's not actually fixed and released yet, I'm confident it's being addressed elsewhere.
          I'd be happy to close it.

          Show
          Mike Laurie added a comment - Plexus are aware of this issue and it will get fixed there ( PLX-185 ). I guess this issue doesn't need any more attention, so despite the fact that it's not actually fixed and released yet, I'm confident it's being addressed elsewhere. I'd be happy to close it.
          Hide
          Olivier Lamy added a comment -

          If I correctly read sun specs.

            newline:      CR LF | LF | CR (not followed by LF) 
            value:        SPACE *otherchar newline *continuation 
            continuation: SPACE *otherchar newline 
            otherchar:    any UTF-8 character except NUL, CR and LF
          

          I don't why tab can't be considered as legal value ?

          Show
          Olivier Lamy added a comment - If I correctly read sun specs. newline: CR LF | LF | CR (not followed by LF) value: SPACE *otherchar newline *continuation continuation: SPACE *otherchar newline otherchar: any UTF-8 character except NUL, CR and LF I don't why tab can't be considered as legal value ?
          Hide
          Mike Laurie added a comment -

          A tab can be a legal value in a manifest, but when the XML's formatting was copied directly into the manifest, we ended up with a newline followed directly by a tab, which is not valid for a manifest.

          My suggestion to condense all whitespace was made purely because a manifest is bound to be laid out differently from an XML description (which is likely to have tab-based indents), so why not condense tabs while we're necessarily going about removing newlines? Not strictly necessary though - you're right, as long as continuation lines always start with a space (which they weren't)

          Show
          Mike Laurie added a comment - A tab can be a legal value in a manifest, but when the XML's formatting was copied directly into the manifest, we ended up with a newline followed directly by a tab, which is not valid for a manifest. My suggestion to condense all whitespace was made purely because a manifest is bound to be laid out differently from an XML description (which is likely to have tab-based indents), so why not condense tabs while we're necessarily going about removing newlines? Not strictly necessary though - you're right, as long as continuation lines always start with a space (which they weren't)
          Hide
          Olivier Lamy added a comment -

          fix with PLX-185.

          Show
          Olivier Lamy added a comment - fix with PLX-185 .

            People

            • Assignee:
              Olivier Lamy
              Reporter:
              Mike Laurie
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: