regression: duplicate files added to the assembly

2.2-beta-2 is badly broken due to this. Does it really take so much time to fix the regression? Pleease?

This is reproduced when one tries e.g. to configure permissions on the files in the directory using multiple include/exclude patterns. Each FileSet is instead copied and duplicated in zip. E.g. real config that broke: http://svn.galaxy.muleforge.org/browse/galaxy/trunk/netboot/mule1/distribution/assembly.xml?r=487

This jacks up my world as my jar signer freaks out when it sees the duplicate files.

I'm having this problem too when making a WAR file.

Any news on a possible fix?? It's been 3 weeks now without an update.

This is happening also with tar.gz format.
The resulting tar file contains many copies of several Jar artifacts, multiplying its size by ~4.
When extracting, only one copy is left per artifact.

I have a project with several submodules, each of them creates a ZIP during build.

When I build submodules one by one, they produce correct ZIPs.
But when i build the entire project with all submodules, their ZIPs contain duplicated entries.

Similar situation while assembling multimodule project to ZIP using
<moduleSets>
<moduleSet>
<includes>
...
in assembly.xml

I found this bug while trying to merge 2 <fileset> directories expecting that the 2nd would overwrite the first set of files where they overlap (in using one fileset as "default" files and the 2nd fileset as overrides).

The result was as described in the bug - 2 identical files added to the resulting zip file. Upon extraction of the zip, I was prompted for an option to override one of the files. I also verified that the case on both files was lower case, so its NOT an issue with the same filename with different case; ie creating the archive on unix and extracting on Windows.

This bug has been open for a long time. It would be appreciated if this was fixed ASAP. Its a huge blocker. Thanks!

example assembly.xml file:

<assembly xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/assembly-1.1.0-SNAPSHOT.xsd">

<id>bin</id>

<formats>
<format>zip</format>
</formats>

<fileSets>
<fileSet>
<!--must use absolute reference with $

/src/main/config/defaults/</directory>
<outputDirectory>$

to achieve filtering -->
<directory>$

/target/work/</outputDirectory>
<filtered>true</filtered>
</fileSet>
</fileSets>

</assembly>

Until this is fixed, here is an example work around, if you are interested (the modules parent is a default jboss-assembly module - the child module inherits the src and config of the default jboss-assembly):

pom.xml:

<plugins>

<!-- generate resources. merge default and current configurations. -->
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>generate-resources</phase>
<configuration>
<tasks>
<copy todir="$

/generated-resources/"
preservelastmodified="true" verbose="true" overwrite="true">
<fileset dir="$

/src/main/config/"/>
</copy>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>

<-- maven assembly plugin -->
</plugins>

You'll need to specify the following in your plugin's configuration section:

<archiverConfig>
  <duplicateBehavior>skip</duplicateBehavior>
</archiverConfig>

the default duplicates setting for the zip archiver is "add", meaning allow duplicates. This logic has been preserved, and using the archiverConfig element of the plugin configuration you an override it. BTW, the duplicates-handling logic from the zip archiver has been generalized out to the AbstractArchiver, so all archivers should handle it.

With following configuration placed in parent's POM:

<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-3-SNAPSHOT</version>
<configuration>
<archiverConfig>
<duplicateBehaviour>skip</duplicateBehaviour>
</archiverConfig>
<descriptors>
<descriptor>src/main/assembly/assembly.xml</descriptor>
</descriptors>
<outputDirectory>target</outputDirectory>
</configuration>
</plugin>

used in multimodule project (parent, then few modules as child), this is still adding duplicates into the final ZIP archive. <duplicateBehaviour>fail</duplicateBehaviour> does not fail as well.

I will appreciate any help.

With duplicateBehavior instead of duplicateBehaviour (one letter changed) it still does not work.

Shouldn't the default behaviour be the 'correct' behaviour? What percentage of people actually want duplicate files in their artefacts?

Why should we try to preserve incorrect behaviour? This sort of thing cripples products - just look at the www.

Please point it out if there's something I'm missing.

I heartily agree with Antony here - please reopen this issue!

I agree as well.

I just to give one more voice on the topic, I also strongly agree that the "normal" behavior is not to duplicate files.

Shouldn't the default behaviour be the 'correct' behaviour?

+1, following Maven's spirit of "convention over configuration" where the convention of course should be sensible, i.e. cover the major use case.

I agree that the default behaviour should be to not allow duplicates. I also think that the default should be to overwrite, not to skip. So that it works just like copying files to a directory.

I strongly agree. Every time we have to put workarounds in our poms we're just taking another step back to ant build.xmls. I honestly can't think of any case where it would be useful to have two of the same file in an archive. My vote would be to overwrite and warn.

+1 on defaulting to overwrite, at least if the assembly is building a jar or a war...

Hi everyone, glad to see you all agree!

Have opened a new issue to cover this, as this one has been officially closed and doesn't seem to be getting much attention:
http://jira.codehaus.org/browse/MASSEMBLY-361

Please go and vote for it!

investigating

default behavior is set to skip. Because of the way archive resources are processed, implementing overwrite is going to be a lot more involved. From the perspective of the assembly plugin, overwrite vs. skip doesn't make much difference, since users cannot control which type of assembly section is processed first, and any of them could overwrite entries from any other. Having duplicate files that are different will lead to non-deterministic results as long as you're not talking about any two specific sections, or entries within a single section, so there is no difference in the general case between skip and overwrite.

In terms of two fileSets, however, it makes sense to allow overwrites. Since this is pretty complex to implement in plexus-archiver, I'll create a new issue for it and assign it to 2.2 (final).

verified as fixed in src/it/projects/bugs/massembly-285

Let me make sure I'm clear on this. If my assembly.xml file has a "dependencySets" node (containing a single "dependencySet") followed by a "fileSets" node (containing a single "fileSet"), neither the order of the nodes nor the intrinsic meaning of the nodes affects the order in which they are processed? The processing order is non-deterministic?

Is this fixed yet? I've still having the duplicate files issue with the default version of the plugin. With 2.2-beta-3 and 2.2-beta-4, there are no duplicates, but NO module dependencies are included, even using <includeDependencies>true</includeDependencies>.

Also, 'overwrite' is not a valid duplicate-file behavior:

[INFO] Failed to create assembly: Error creating assembly archive test-zip:
Failed to configure archiver: org.codehaus.plexus.archiver.zip.ZipArchiver

Invalid duplicate-file behavior: 'overwrite'. Please specify one of: [skip, add,
preserve, fail]

There's still a missing behavior which is "replace/update/overwrite" like zip "-u" option.
I need this and I guess it's the most wanted behavior when archiving files already present in a zip, we often want to update/replace the file already archived, not to skip the new one (or preserve the old one).

Please reopen issue as the fix doesn't provide same behavior as what was produced by version 2.1 (overwrite old files with new ones).

For testing, I use the following assembly and expect files in the zip being those from resources_new directory:
<assembly>
<id>resources-test</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>src/main/resources_old</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
<fileSet>
<directory>src/main/resources_new</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
</assembly>

Why has this been flagged as fixed? If two filesets contain files with identical names, I still get those awkward duplicates (2.2 beta5 and final):

	<fileSet>
			<!-- copy indexer and its dependencies-->
			<directory>${project.build.directory}/deps/indexer</directory>
			<outputDirectory>glassfishv3/glassfish/domains/domain1/autodeploy/jobservice/WEB-INF/lib</outputDirectory>
		</fileSet>
		<fileSet>
			<!-- deploy jobservice to glassfish from unpacked WAR -->
			<directory>${project.build.directory}/deps/jobservice</directory>
			<outputDirectory>glassfishv3/glassfish/domains/domain1/autodeploy/jobservice</outputDirectory>
		</fileSet>

Please reopen this issue. I've the same problems as Daniel

thanks

This is still an issue in maven-assembly-plugin-2.2.2.

The maven-shade-plugin (mentioned in jar-with-dependencies pre-defined descriptor) can create standalone (uber) jars and does not duplicate files.

<!-- Shade can create a standalone jar -->
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>1.5</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals><goal>shade</goal></goals>
      <configuration>
        <transformers>
          <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
            <mainClass>${mainClass}</mainClass>
          </transformer>
        </transformers>
        <shadedArtifactAttached>true</shadedArtifactAttached>
        <shadedClassifierName>standalone</shadedClassifierName>
      </configuration>
    </execution>
  </executions>
</plugin>

Either version 2.2.2 of the Assembly plugin has a regression or the issue has not been fixed. I am currently creating zip files from the assembly that contains duplicate jar files.

Some background : I am using the assembly to create a zip file of an application. Why Zip ? because the application requires multiple configuration files and spans larger than just Java so I cannot use a uber Jar (I would prefer that but technical limitations out of my control force me otherwise).

This package is built correctly.

To make life simpler I made it possible to extend this package with a project that can add extra classes (plugins and such) and configuration from a standard layout in the project that gets picked up by the assembly and merged with the previous vanilla package. the end result is a fully assembled zip file with all the customized parts in the right place.

Now, behaviour changed across version wheras previously I would overwrite the original package with the content of the new one, now I have to start from the overrides and complete withe the original files. Easily fixed through changing the order I declared the filesets in the assembly. When using filesets strictly nothing gets added twice, though the replace was changed to skip which is a bit counter intuitive.

BUT....

Since I extend some parts of the original system, I will share some dependencies with this one. These dependencies are already present in the original package and get added through a file set (I get the package and unzip it in a temporary folder where I integrate it`s content inthe assembly through a fileset).

I also add the dependencies of the new customized project, and these get added twice.

I think there is a regression here when filesets and dependencysets interact where if files are present in both they get added twice in the zip file. Now I would re-open this current task but it seems I cannot so I will open a sub task instead.