jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
  • Maven 2.x JAR Plugin
  • MJAR-50

"Invalid Header" in jar's Manifest (Specification-Title attribute) when tab char in pom Description

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Trivial Trivial
  • Resolution: Fixed
  • Affects Version/s: 2.0
  • Fix Version/s: 2.2
  • Component/s: None
  • Labels:
    None

Description

I typed a <description> element into my pom; the description spanned 2 lines, and my text editor put a tab char at the beginning of the 2nd line.
The project built correctly, but another project depended on the resulting jar.
The compilation of the depending project failed with "invalid header" on the jar I'd just compiled.

When I looked at the jar's manifest, I saw the Specification-Title attribute had the tab character still in it from the <description> element.
Replacing the tab with spaces in the pom solved the problem.
However, this was valid xml that caused a compilation problem, so I think it's a minor bug with the jar production code.
Note that the full description is trimmed before going into the manifest, so any

Easy to work around, but it would be nice if whitespace in the <description> element were consolidated into spaces before copying to the manifest.
Doesn't need any fancy layout stuff - just any multiple instances of tabs, spaces, lf or cr should be replaced by a single space.
Try (I haven't tried to compile this!):

public String consolidateWhitespace(String input){
        StringTokenizer st = new StringTokenizer(input);
        StringBuffer rv = new StringBuffer();
        while (st.hasMoreTokens()){
            rv.append(st.nextToken() + (st.hasMoreTokens()?" ":""));
        }
        return rv.toString();
    }

Issue Links

depends upon

Bug - A problem which impairs or prevents the functions of the product. PLX-185 Archiver does not handle manifest entries with linefeeds correctly

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Mike Laurie added a comment - 07/Jul/06 10:37 AM

Sorry - in the middle of that description, I meant to say:
"Note that the full description is trimmed before going into the manifest (I think), so any leading or trailing tabs don't seem recreate the problem."

Show
Mike Laurie added a comment - 07/Jul/06 10:37 AM Sorry - in the middle of that description, I meant to say: "Note that the full description is trimmed before going into the manifest (I think), so any leading or trailing tabs don't seem recreate the problem."
Hide
Permalink
Wendy Smoak added a comment - 07/Jul/06 10:52 AM

Possible duplicate of MJAR-4 and MJAR-34

Show
Wendy Smoak added a comment - 07/Jul/06 10:52 AM Possible duplicate of MJAR-4 and MJAR-34
Hide
Permalink
Brett Porter added a comment - 19/Jul/06 7:31 AM

since MJAR-4 and MJAR-34 are fixed, can anyone confirm this one is also fixed?

Show
Brett Porter added a comment - 19/Jul/06 7:31 AM since MJAR-4 and MJAR-34 are fixed, can anyone confirm this one is also fixed?
Hide
Permalink
Dennis Lundberg added a comment - 19/Jul/06 2:44 PM

I'll test it.

Show
Dennis Lundberg added a comment - 19/Jul/06 2:44 PM I'll test it.
Hide
Permalink
Dennis Lundberg added a comment - 19/Jul/06 3:49 PM

This is with a two-line <description> in the pom, second line starting with a tab character.

Using no configuration options I don't see the "Specification-Title" attribute in the manifest at all.

Using this configuration:
<configuration>
<archive>
<manifestEntries>
<Specification-Title>${pom.description}</Specification-Title>
</manifestEntries>
</archive>
</configuration>
gives me this error message:
"Embedded error: The attribute "Specification-Title" may not occur more than once in the same section"
which is strange considering that it wasn't there in the first example.

Using this configuration:
<configuration>
<archive>
<manifestEntries>
<custom>${pom.description}</custom>
</manifestEntries>
</archive>
</configuration>
gives me the same error as above.

So I've had no luck in reproducing the problem reported in this issue, but might have discovered another one...

Show
Dennis Lundberg added a comment - 19/Jul/06 3:49 PM This is with a two-line <description> in the pom, second line starting with a tab character. Using no configuration options I don't see the "Specification-Title" attribute in the manifest at all. Using this configuration: <configuration> <archive> <manifestEntries> <Specification-Title>${pom.description}</Specification-Title> </manifestEntries> </archive> </configuration> gives me this error message: "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section" which is strange considering that it wasn't there in the first example. Using this configuration: <configuration> <archive> <manifestEntries> <custom>${pom.description}</custom> </manifestEntries> </archive> </configuration> gives me the same error as above. So I've had no luck in reproducing the problem reported in this issue, but might have discovered another one...
Hide
Permalink
Mike Laurie added a comment - 20/Jul/06 8:27 AM

The problem has indeed been masked because of other fixes.
3 things left to consider:

  • I think newlines in manifest values hasn't actually been fixed in maven-archiver
  • maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0 so jar-plugin pom needs updating if archiver-2.1 is to be used.
  • The Embedded error: The attribute ... may not occur more than once in the same section seems to be a new problem (regression) in archiver-2.1 - I've reproduced it, and it stops me from testing the newlines issue. It's not restricted to the Specification-Title attribute.

Discussion:
The issue at hand has been masked by changes in MJAR-39.
The fix in MJAR-39 ensures that the Specification-Title manifest attribute (if included) now contains project name (not project description)
However, I found (like you, Dennis) that the Specification-Title was not in the jar's manifest at all.
This is in turn an effect of MJAR-38 whereby the Specification-... elements are not included by default.

I've just tested this in maven-archiver-2.1, which was released on 27th June, and descriptions containing newline-tab combinations no longer cause an invalid header error.

However , the pom for maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0:

maven-jar-plugin-2.0.pom
...
<dependency>
    <groupId>org.apache.maven</groupId>
    <artifactId>maven-archiver</artifactId>
    <version>2.0</version>
</dependency>
...

This means that by default it's still using the old archiver.
When I updated my local repository's pom for maven-jar-plugin-2.0 to require version 2.1, I was able to test the new version.
Can the pom be changed, or a new version of it released? I'm not sure how these things are done.

I discovered that the deviation from the jar spec is actually the inclusion of linefeeds in the continuation lines which I don't think has actually been fixed.
For reference, here's part of the jar spec from Sun, indicating why the problem happened:

  newline:      CR LF | LF | CR (not followed by LF) 
  value:        SPACE *otherchar newline *continuation 
  continuation: SPACE *otherchar newline 
  otherchar:    any UTF-8 character except NUL, CR and LF

In an attribute value, a newline followed by a tab is not valid; a newline followed by a space to indicate a continuation would be valid.

If there are further similar problems with jar manifests, then it may be the plexus jar class that's at fault.
(org.codehaus.plexus.archiver.jar.Manifest)
I don't think it should allow a manifest attrubite to contain \r or \n characters.

To ensure Maven doesn't add \r or \n chars to manifest attributes, they could be stripped out in MavenArchiver:

org.apache.maven.archiver.MavenArchiver line 107
private void addManifestAttribute( Manifest manifest, String key, String value )
        throws ManifestException
    {
        // Use the empty string to suppress a Manifest entry
        if ( value != null && !"".equals( value ) )
        {
/*+*/
            //Don't allow newlines in manifest attribute values - plexus jar doesn't 
            // currently strip them out, and they can result in an invalid manifest.
            value=value.replace('\r',' ');
            value=value.replace('\n',' ');
/*+*/
            Manifest.Attribute attr = new Manifest.Attribute( key, value );
            manifest.addConfiguredAttribute( attr );
        }
    }

I don't know why you get "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section"
I got the same error with the following configuration when I was trying to test:

pom build/plugins section
<plugin>
    <groupId>org.apache.maven.plugins</groupId> 
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
      <archive>
        <manifestEntries>
          <Dodgy-Entry>
            Here's a manifest entry with a newline
			followed by a tab (actually several tabs)
          </Dodgy-Entry>
        </manifestEntries>
      </archive>
    </configuration>
  </plugin>

This doesn't occur with archiver-2.0; in 2.0 the dodgy entry is successfully written to the manifest with the incorrect manifest attribute format:

snippet from resulting manifest
Dodgy-Entry: Here's a manifest entry with a newline
								followed b
 y a tab (actually several tabs)
Show
Mike Laurie added a comment - 20/Jul/06 8:27 AM The problem has indeed been masked because of other fixes. 3 things left to consider:
  • I think newlines in manifest values hasn't actually been fixed in maven-archiver
  • maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0 so jar-plugin pom needs updating if archiver-2.1 is to be used.
  • The Embedded error: The attribute ... may not occur more than once in the same section seems to be a new problem (regression) in archiver-2.1 - I've reproduced it, and it stops me from testing the newlines issue. It's not restricted to the Specification-Title attribute.
Discussion: The issue at hand has been masked by changes in MJAR-39. The fix in MJAR-39 ensures that the Specification-Title manifest attribute (if included) now contains project name (not project description) However, I found (like you, Dennis) that the Specification-Title was not in the jar's manifest at all. This is in turn an effect of MJAR-38 whereby the Specification-... elements are not included by default. I've just tested this in maven-archiver-2.1, which was released on 27th June, and descriptions containing newline-tab combinations no longer cause an invalid header error. However , the pom for maven-jar-plugin-2.0 has an explicit dependency on maven-archiver-2.0:
maven-jar-plugin-2.0.pom
...
<dependency>
    <groupId>org.apache.maven</groupId>
    <artifactId>maven-archiver</artifactId>
    <version>2.0</version>
</dependency>
...
This means that by default it's still using the old archiver. When I updated my local repository's pom for maven-jar-plugin-2.0 to require version 2.1, I was able to test the new version. Can the pom be changed, or a new version of it released? I'm not sure how these things are done. I discovered that the deviation from the jar spec is actually the inclusion of linefeeds in the continuation lines which I don't think has actually been fixed. For reference, here's part of the jar spec from Sun, indicating why the problem happened:
  newline:      CR LF | LF | CR (not followed by LF) 
  value:        SPACE *otherchar newline *continuation 
  continuation: SPACE *otherchar newline 
  otherchar:    any UTF-8 character except NUL, CR and LF
In an attribute value, a newline followed by a tab is not valid; a newline followed by a space to indicate a continuation would be valid. If there are further similar problems with jar manifests, then it may be the plexus jar class that's at fault. (org.codehaus.plexus.archiver.jar.Manifest) I don't think it should allow a manifest attrubite to contain \r or \n characters. To ensure Maven doesn't add \r or \n chars to manifest attributes, they could be stripped out in MavenArchiver:
org.apache.maven.archiver.MavenArchiver line 107
private void addManifestAttribute( Manifest manifest, String key, String value )
        throws ManifestException
    {
        // Use the empty string to suppress a Manifest entry
        if ( value != null && !"".equals( value ) )
        {
/*+*/
            //Don't allow newlines in manifest attribute values - plexus jar doesn't 
            // currently strip them out, and they can result in an invalid manifest.
            value=value.replace('\r',' ');
            value=value.replace('\n',' ');
/*+*/
            Manifest.Attribute attr = new Manifest.Attribute( key, value );
            manifest.addConfiguredAttribute( attr );
        }
    }
I don't know why you get "Embedded error: The attribute "Specification-Title" may not occur more than once in the same section" I got the same error with the following configuration when I was trying to test:
pom build/plugins section
<plugin>
    <groupId>org.apache.maven.plugins</groupId> 
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
      <archive>
        <manifestEntries>
          <Dodgy-Entry>
            Here's a manifest entry with a newline
			followed by a tab (actually several tabs)
          </Dodgy-Entry>
        </manifestEntries>
      </archive>
    </configuration>
  </plugin>
This doesn't occur with archiver-2.0; in 2.0 the dodgy entry is successfully written to the manifest with the incorrect manifest attribute format:
snippet from resulting manifest
Dodgy-Entry: Here's a manifest entry with a newline
								followed b
 y a tab (actually several tabs)
Hide
Permalink
Mike Perham added a comment - 20/Jul/06 3:06 PM

Mike, jar-pugin 2.1 will depend on archiver 2.1. It already does in SVN but has not been released yet. The "more than once in the same section" error needs to be tracked down and squashed before we release jar-plugin 2.1 IMO. If you have any more data on the problem, please open a separate JIRA MJAR issue and post everything you have there.

Show
Mike Perham added a comment - 20/Jul/06 3:06 PM Mike, jar-pugin 2.1 will depend on archiver 2.1. It already does in SVN but has not been released yet. The "more than once in the same section" error needs to be tracked down and squashed before we release jar-plugin 2.1 IMO. If you have any more data on the problem, please open a separate JIRA MJAR issue and post everything you have there.
Hide
Permalink
Mike Perham added a comment - 20/Jul/06 4:32 PM

Ok, I spent an hour looking at this.

maven-archiver 2.1 is hosed AFAICT. You cannot override manifest entries or merge Class-Path values with that version. When I upgrade jar-plugin to use maven-archiver 2.2-SNAPSHOT, everything works as expected. This is the configuration I played with.

<build>
	<plugins>
	  <plugin>
		<artifactId>maven-jar-plugin</artifactId>
		<configuration>
			<archive>
				<index>true</index>
				<manifest>
					<addDefaultSpecificationEntries>false</addDefaultSpecificationEntries>
					<addDefaultImplementationEntries>false</addDefaultImplementationEntries>
					<mainClass>com.acme.Main</mainClass>
					<addClasspath>true</addClasspath>
				</manifest>
				<manifestEntries>
					<Class-Path>foo/bar</Class-Path>
					<Specification-Title>Foo</Specification-Title>
				</manifestEntries>
			</archive>
		</configuration>
	  </plugin>
	</plugins>
  </build>

It looks like we will need to release maven-archiver again.

Show
Mike Perham added a comment - 20/Jul/06 4:32 PM Ok, I spent an hour looking at this. maven-archiver 2.1 is hosed AFAICT. You cannot override manifest entries or merge Class-Path values with that version. When I upgrade jar-plugin to use maven-archiver 2.2-SNAPSHOT, everything works as expected. This is the configuration I played with.
<build>
	<plugins>
	  <plugin>
		<artifactId>maven-jar-plugin</artifactId>
		<configuration>
			<archive>
				<index>true</index>
				<manifest>
					<addDefaultSpecificationEntries>false</addDefaultSpecificationEntries>
					<addDefaultImplementationEntries>false</addDefaultImplementationEntries>
					<mainClass>com.acme.Main</mainClass>
					<addClasspath>true</addClasspath>
				</manifest>
				<manifestEntries>
					<Class-Path>foo/bar</Class-Path>
					<Specification-Title>Foo</Specification-Title>
				</manifestEntries>
			</archive>
		</configuration>
	  </plugin>
	</plugins>
  </build>
It looks like we will need to release maven-archiver again.
Hide
Permalink
Mike Perham added a comment - 20/Jul/06 4:36 PM

BTW with 2.2-SNAPSHOT, it still creates illegal manifest entries with something like:

<manifestEntries>
<Specification-Title>${pom.description}</Specification-Title>
</manifestEntries>

It's just less likely now that we are not populating any fields based on description anymore. So the problem is not fixed, just far less likely to occur.

Show
Mike Perham added a comment - 20/Jul/06 4:36 PM BTW with 2.2-SNAPSHOT, it still creates illegal manifest entries with something like: <manifestEntries> <Specification-Title>${pom.description}</Specification-Title> </manifestEntries> It's just less likely now that we are not populating any fields based on description anymore. So the problem is not fixed, just far less likely to occur.
Hide
Permalink
Mike Laurie added a comment - 21/Jul/06 3:34 AM

Wow - thanks Mike; you've been busy!
It's good to know the "more than once in the same section" error is already fixed in archiver 2.2.

The only thing causing this MJAR issue now seems to be the possibility of including \r and \n characters in manifest header fields. While we're at it, I guess that NUL characters also ought to be quashed since they're the only other invalid attribute chars according to the sun spec. I think these characters should be replaced with spaces as a matter of course.

Last question: Does the archiver-plugin take responsibility for this, or should plexus change their jar.Manifest class to replace them? I think plexus should be changed anyway, since it currently allows invalid jar manifests to be created. However, is it worth changing the archiver-plugin to fix the problem now, since it's an easy fix and there needs to be a release soon anyway?

Show
Mike Laurie added a comment - 21/Jul/06 3:34 AM Wow - thanks Mike; you've been busy! It's good to know the "more than once in the same section" error is already fixed in archiver 2.2. The only thing causing this MJAR issue now seems to be the possibility of including \r and \n characters in manifest header fields. While we're at it, I guess that NUL characters also ought to be quashed since they're the only other invalid attribute chars according to the sun spec. I think these characters should be replaced with spaces as a matter of course. Last question: Does the archiver-plugin take responsibility for this, or should plexus change their jar.Manifest class to replace them? I think plexus should be changed anyway, since it currently allows invalid jar manifests to be created. However, is it worth changing the archiver-plugin to fix the problem now, since it's an easy fix and there needs to be a release soon anyway?
Hide
Permalink
Mike Laurie added a comment - 21/Jul/06 4:08 AM

Plexus are aware of this issue and it will get fixed there (PLX-185).
I guess this issue doesn't need any more attention, so despite the fact that it's not actually fixed and released yet, I'm confident it's being addressed elsewhere.
I'd be happy to close it.

Show
Mike Laurie added a comment - 21/Jul/06 4:08 AM Plexus are aware of this issue and it will get fixed there (PLX-185). I guess this issue doesn't need any more attention, so despite the fact that it's not actually fixed and released yet, I'm confident it's being addressed elsewhere. I'd be happy to close it.
Hide
Permalink
Olivier Lamy added a comment - 14/Dec/07 6:02 PM

If I correctly read sun specs.

  newline:      CR LF | LF | CR (not followed by LF) 
  value:        SPACE *otherchar newline *continuation 
  continuation: SPACE *otherchar newline 
  otherchar:    any UTF-8 character except NUL, CR and LF

I don't why tab can't be considered as legal value ?

Show
Olivier Lamy added a comment - 14/Dec/07 6:02 PM If I correctly read sun specs.
  newline:      CR LF | LF | CR (not followed by LF) 
  value:        SPACE *otherchar newline *continuation 
  continuation: SPACE *otherchar newline 
  otherchar:    any UTF-8 character except NUL, CR and LF
I don't why tab can't be considered as legal value ?
Hide
Permalink
Mike Laurie added a comment - 14/Dec/07 8:22 PM

A tab can be a legal value in a manifest, but when the XML's formatting was copied directly into the manifest, we ended up with a newline followed directly by a tab, which is not valid for a manifest.

My suggestion to condense all whitespace was made purely because a manifest is bound to be laid out differently from an XML description (which is likely to have tab-based indents), so why not condense tabs while we're necessarily going about removing newlines? Not strictly necessary though - you're right, as long as continuation lines always start with a space (which they weren't)

Show
Mike Laurie added a comment - 14/Dec/07 8:22 PM A tab can be a legal value in a manifest, but when the XML's formatting was copied directly into the manifest, we ended up with a newline followed directly by a tab, which is not valid for a manifest. My suggestion to condense all whitespace was made purely because a manifest is bound to be laid out differently from an XML description (which is likely to have tab-based indents), so why not condense tabs while we're necessarily going about removing newlines? Not strictly necessary though - you're right, as long as continuation lines always start with a space (which they weren't)
Hide
Permalink
Olivier Lamy added a comment - 01/Jan/08 5:46 PM

fix with PLX-185.

Show
Olivier Lamy added a comment - 01/Jan/08 5:46 PM fix with PLX-185.

People

  • Assignee:
    Olivier Lamy
    Reporter:
    Mike Laurie
Vote (1)
Watch (2)

Dates

  • Created:
    07/Jul/06 10:35 AM
    Updated:
    01/Jan/08 5:46 PM
    Resolved:
    01/Jan/08 5:46 PM
  • Atlassian JIRA (v5.0.4#731-sha1:3aa7374)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.