Maven 2.x Dependency Plugin

Figuring out duplicate class definitions using the Analyze goal

Details

  • Type: Improvement Improvement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: 2.1
  • Fix Version/s: None
  • Component/s: analyze
  • Labels:
    None
  • Testcase included:
    yes
  • Patch Submitted:
    Yes
  • Number of attachments :
    2

Description

Hi,

I've pretty frequently run into issues where changes to the library structure of some product (that is, changing the way that classes are grouped into libraries) leads to the same classes being defined in more than one place. This can lead to system-dependent problems, because different versions of the same class are being loaded by different systems.

I was going to create a new goal for the dependency plugin to check for duplicate classes, but when I looked a bit closer at the analyze goal, it already had all the information needed to do that check as well, so I came up with some changes that add this functionality.

The intended usage is something like:

mvn dependency:analyze -DcheckDuplicateClasses

I get the feeling that I might want to add the ability to exclude certain packages (that I might be comfortable are safe to have duplicates of), so I added this option too:

mvn dependency:analyze -DcheckDuplicateClasses -DexcludePrefixes="org., net.sf.cglib, javax.xml, junit."

The output looks something like:

[WARNING] Duplicate class definitions found:
[WARNING] com.shopzilla.common.data.ObjectFactory defined in:
[WARNING] com.shopzilla.site.url.c14n:model:jar:1.4:compile
[WARNING] com.shopzilla.common.data:data-model-schema:jar:1.11:compile
[WARNING] com.shopzilla.site.category.CategoryProvider defined in:
[WARNING] com.shopzilla.site2.sasClient:sas-client-core:jar:5.47:compile
[WARNING] com.shopzilla.site2.service:common-web:jar:5.50:compile

A couple of notes:

  • I was unable to get configuration (setting checkDuplicateClasses, etc.) using the pom to work, but I think that might be due to lack of understanding on my part.
  • I don't fully understand the effect of calling compileProject() during unit tests, but I think it may be sufficient to call it only once for the duplicateClasses project, during setUp(). That would speed up the unit tests.
  • I haven't added duplicate class definition checking to the AnalyzeReportMojo, because I wanted to get some feedback on whether this addition was felt to be valuable before spending any time on that.
  • A lot of the unit test dummy code in the attached diff files needs cleaning up, but again I wanted to wait with that until hearing whether this might be useful to others.
  • I made an API change in the ProjectDependencyAnalyzer interface, which might be an issue if there are other implementations than the default one. That change was only needed to support the 'exclude package' feature, which might not be super-important.

Cheers,
Petter

  1. File
    dependency-analyzer.diff
    27/Jul/10 2:23 AM
    42 kB
    Petter Måhlén
  2. File
    dependency-plugin.diff
    27/Jul/10 2:23 AM
    6 kB
    Petter Måhlén

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Petter Måhlén added a comment -

By the way, to make it easier for other people who are interested in using this feature, I have created a couple of repositories with the code. The repositories are:

http://github.com/pettermahlen/maven-dependency-analyzer-fork
http://github.com/pettermahlen/maven-dependency-plugin-fork

For instructions on how to start using it, see http://pettermahlen.com/2010/08/05/finding-duplicate-class-definitions-using-maven/.

Show
Petter Måhlén added a comment - By the way, to make it easier for other people who are interested in using this feature, I have created a couple of repositories with the code. The repositories are: http://github.com/pettermahlen/maven-dependency-analyzer-fork http://github.com/pettermahlen/maven-dependency-plugin-fork For instructions on how to start using it, see http://pettermahlen.com/2010/08/05/finding-duplicate-class-definitions-using-maven/ .
Hide
Permalink
Brian Fox added a comment -

Your timing couldn't have been better. We were just discussing the need for this the other day to help diagnose potential conflicts when switching from Maven 2.x to Maven 3.x. I'll take a look and get this merged in.

Show
Brian Fox added a comment - Your timing couldn't have been better. We were just discussing the need for this the other day to help diagnose potential conflicts when switching from Maven 2.x to Maven 3.x. I'll take a look and get this merged in.
Hide
Permalink
Raymond Feng added a comment -

Coincidentally, I wrote a plugin for the Apache Tuscany project (tuscany.apache.org) that do the similar job. You can find the code at:

https://svn.apache.org/repos/asf/tuscany/maven-plugins/trunk/maven-dependency-plugin/

It comes with a few additional features over this proposal:

  • It can scan jars from a directory
  • It groups duplicate classes under the list of artifacts
  • It goes one step further to check the size and CRC of the class file and marks the different cases in the report (potentially we can use BCEL to further compare the two classes):
  • X: the duplicate classes have different size (most likely different versions)
  • ?: the duplicate classes have the same size but different CRC (probably the same version but were built separately)
  • : the duplicate classes have the same size and CRC (most likely the same version).

I think the maven-dependency-plugin should be the place to go. I would like to work with you guys to get my code merged into this proposal.

Show
Raymond Feng added a comment - Coincidentally, I wrote a plugin for the Apache Tuscany project (tuscany.apache.org) that do the similar job. You can find the code at: https://svn.apache.org/repos/asf/tuscany/maven-plugins/trunk/maven-dependency-plugin/ It comes with a few additional features over this proposal: It can scan jars from a directory It groups duplicate classes under the list of artifacts It goes one step further to check the size and CRC of the class file and marks the different cases in the report (potentially we can use BCEL to further compare the two classes): X: the duplicate classes have different size (most likely different versions) ?: the duplicate classes have the same size but different CRC (probably the same version but were built separately) : the duplicate classes have the same size and CRC (most likely the same version). I think the maven-dependency-plugin should be the place to go. I would like to work with you guys to get my code merged into this proposal.
Hide
Permalink
Petter Måhlén added a comment -

I think Raymond's proposal is superior to mine, so that should probably be the focus for the merge.

Show
Petter Måhlén added a comment - I think Raymond's proposal is superior to mine, so that should probably be the focus for the merge.
Hide
Permalink
Brian Fox added a comment -

Raymond: Any chance you can write some unit/it tests for this new code? Currently it has none, and if you look at the mdep plugin, it is currently very thoroughly tested.

Show
Brian Fox added a comment - Raymond: Any chance you can write some unit/it tests for this new code? Currently it has none, and if you look at the mdep plugin, it is currently very thoroughly tested.
Hide
Permalink
Stephen Connolly added a comment -

See

https://svn.codehaus.org/mojo/trunk/sandbox/extra-enforcer-rules/src/main/java/org/codehaus/mojo/enforcer/rule/BanDuplicateClassesRule.java@13945

Show
Stephen Connolly added a comment - See https://svn.codehaus.org/mojo/trunk/sandbox/extra-enforcer-rules/src/main/java/org/codehaus/mojo/enforcer/rule/BanDuplicateClassesRule.java@13945
Hide
Permalink
Shaun added a comment -

I too could use this functionality and was considering writing it myself. I've been playing with the aforementioned 'BanDuplicateClassesRule' in the enforcer plugin, however, it does not seem to scan transititve dependencies

There is also this:

http://el4j.sourceforge.net/plugins/maven-duplicatefinder-plugin/index.html

Which doesn't seem to do much of anything either - I hope this get's rolled into one of these standard plugins soon.

Show
Shaun added a comment - I too could use this functionality and was considering writing it myself. I've been playing with the aforementioned 'BanDuplicateClassesRule' in the enforcer plugin, however, it does not seem to scan transititve dependencies There is also this: http://el4j.sourceforge.net/plugins/maven-duplicatefinder-plugin/index.html Which doesn't seem to do much of anything either - I hope this get's rolled into one of these standard plugins soon.
Hide
Permalink
Shaun added a comment -

I've submitted a patch to the BanDuplicateClassesRule on: MENFORCER-120

The output looks something like this:

  Found in: 
    org.springframework:spring-context:jar:2.5.6:compile
    org.springframework:org.springframework.context:jar:2.5.6.A:compile
  Duplicate classes:
    org/springframework/validation/ObjectError.class
    org/springframework/scheduling/SchedulingAwareRunnable.class
Show
Shaun added a comment - I've submitted a patch to the BanDuplicateClassesRule on: MENFORCER-120 The output looks something like this: Found in: org.springframework:spring-context:jar:2.5.6:compile org.springframework:org.springframework.context:jar:2.5.6.A:compile Duplicate classes: org/springframework/validation/ObjectError.class org/springframework/scheduling/SchedulingAwareRunnable.class

People

  • Assignee:
    Unassigned
    Reporter:
    Petter Måhlén
Vote (11)
Watch (9)

Dates

  • Created:
    Updated: