Maven 2 & 3
  1. Maven 2 & 3
  2. MNG-2344

Support for functional/integration tests is *way* too limited

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 2.0.4
    • Fix Version/s: None
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      0

      Description

      I am a recent convert to Maven2 ... after listening to advice from many people I respect (including Wendy Smoak and Brett Porter) we are in the midst of migrating the build environment for the Shale Framework (part of the http://struts.apache.org community) over to Maven2. For the most part, this has been all goodness ... the dependency management, and the hierarchical dependency resolution, make for a dramatic simplification in the overall environment.

      That being said, there is one major area where I'm really disappointed ... and it's almost enough to tip my personal scale back towards hand crafted Ant builds. It relates to support for testing of web applications. My plan is to leverage the Maven2 "archetype" capability to make it very easy to build "starter" web applications that are already set up to use Shale. To encourage best practices, I want to set up "out of the box" testing for the application to be developed.

      As we all know, this kind of testing can be divided into at least two major categories ... unit tests of the individual server side classes, and functiona/integration tests that want to deploy the application first, then interact with it (using a framework such as HttpUnit or HtmlUnit) to verify that the rendered results match expectations. In my previous Ant-based build world, I had two different targets ("test" and "systest") that could exercise the two different kinds of tests. Much to my surprise, it seems to be difficult-to-impossible to set up a similar sceanrio (two kinds of tests) in the same project when you're using Maven2.

      I could live with this (although I'd still grumble a lot) if it was just an issue of the sample apps that we ship with Shale. I can teach committers to run integration tests from a separate module before doing SVN commits. But that is not what I want to present to potential developers that will build webapps based on a Shale-provided archtetype. And it shouldn't be what you want for Maven2 users in general either ... support for both unit tests and integration tests seems like a mission critical feature that is not adequately supported at present.

        Issue Links

          Activity

          Hide
          Craig McClanahan added a comment -

          MNG-591 does not come close to covering what actually is needed here. The following are some comments I made on the Struts mailing list, in response to Brett's encouragement to provide more feedback:
          ---------------------------------------------------------------------------------------------------------------------------
          ---------------------------------------------------------------------------------------------------------------------------
          Maven2 needs to support integration testing as a first class notion in the architecture of what you envision a "project" to be. It's not just webapps .. you've got the same sort of issue with EJBs, or web services, or anything that gets deployed in a container. Unit tests just don't give you the confidence you need that the application will actually work. I've seen too many cases where all the unit tests on a webapp all pass with flying colors, but it throws an HTTP 500 on the welcome page because of a stupid coding error in the JSP page that wasnt' tested with the unit tests.

          What's needed is a complete additional test environment, with its own lifecycle, and its own classpath (i.e. dependencies tagged to this scope so you only load things like HttpUnit or HtmlUnit here). If integration tests exist, they should be part of the default "mvn install" processing, just like unit tests are, unless it is explicitly disabled. Don't pretend that there is only one kind of "test"!!!

          Otherwise, you guys are not being serious about trying to encouraging best practices in build environments .
          ---------------------------------------------------------------------------------------------------------------------------

          Show
          Craig McClanahan added a comment - MNG-591 does not come close to covering what actually is needed here. The following are some comments I made on the Struts mailing list, in response to Brett's encouragement to provide more feedback: --------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------- Maven2 needs to support integration testing as a first class notion in the architecture of what you envision a "project" to be. It's not just webapps .. you've got the same sort of issue with EJBs, or web services, or anything that gets deployed in a container. Unit tests just don't give you the confidence you need that the application will actually work. I've seen too many cases where all the unit tests on a webapp all pass with flying colors, but it throws an HTTP 500 on the welcome page because of a stupid coding error in the JSP page that wasnt' tested with the unit tests. What's needed is a complete additional test environment, with its own lifecycle, and its own classpath (i.e. dependencies tagged to this scope so you only load things like HttpUnit or HtmlUnit here). If integration tests exist, they should be part of the default "mvn install" processing, just like unit tests are, unless it is explicitly disabled. Don't pretend that there is only one kind of "test"!!! Otherwise, you guys are not being serious about trying to encouraging best practices in build environments . ---------------------------------------------------------------------------------------------------------------------------
          Hide
          Brett Porter added a comment -

          I was just trying to aggregate all the information under one work unit. But I'll leave this open.

          Show
          Brett Porter added a comment - I was just trying to aggregate all the information under one work unit. But I'll leave this open.
          Hide
          Brett Porter added a comment -

          found the right issue this time

          Show
          Brett Porter added a comment - found the right issue this time
          Hide
          David Svanberg added a comment -

          Even when offline set to true in settings.xml:

          [INFO] artifact org.apache.maven.plugins:maven-scm-plugin: checking for updates from central

          [WARNING] repository metadata for: 'artifact org.apache.maven.plugins:maven-scm-plugin' could not be retrieved from repository: central due to an error: Error transferring file
          [INFO] Repository 'central' will be blacklisted

          Show
          David Svanberg added a comment - Even when offline set to true in settings.xml: [INFO] artifact org.apache.maven.plugins:maven-scm-plugin: checking for updates from central [WARNING] repository metadata for: 'artifact org.apache.maven.plugins:maven-scm-plugin' could not be retrieved from repository: central due to an error: Error transferring file [INFO] Repository 'central' will be blacklisted
          Hide
          David Svanberg added a comment -

          Sorry this was a comment for #2433 ... Silly me.

          Show
          David Svanberg added a comment - Sorry this was a comment for #2433 ... Silly me.

            People

            • Assignee:
              Brett Porter
              Reporter:
              Craig McClanahan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: