JBehave
  1. JBehave
  2. JBEHAVE-394

Hudson Plugin to import JBehave reports

    Details

    • Type: New Feature New Feature
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2
    • Component/s: Hudson Support
    • Labels:
      None
    • Environment:
      Hudsdon
    • Number of attachments :
      8

      Description

      As a developer I want to see a Hudson build test result showing failed scenarios so that I can see what broke my build

      As a developer I want scenarios containing pending steps and no failed steps to appear in build test result not as a fail or pass so that I can see what is missing from a story

      As a Manager I want to see individual scenarios in the build graph so that I can monitor progress of stories

      As a Manager I want to see Hudson build trend graph containing failed, pending and successful scenarios so that I can monitor the stability of my build

      1. org.jbehave.examples.trader.stories.claims_with_null_calendar.xml
        1 kB
        Seth Carter
      2. pom.xml
        3 kB
        Alan Parkinson
      3. TEST--778446379.xml
        0.4 kB
        Seth Carter
      1. eclipse-jbehave-runner.png
        110 kB
      2. hudson-jbehave-runner.png
        75 kB

        Activity

        Hide
        Alan Parkinson added a comment -

        I'm also seeing corrupted XML files being read by the Hudson plugin before any processing occurs. When reading a file xUnit will validate the XML file and then print "[xUnit] [ERROR] - The plugin hasn't been performed correctly: Conversion error Error to convert - A file not found" in the hudson build console if the file is not valid XML.

        To rule out filters causing problem I will add some unit tests but I don't think this will be a problem because the XSL is only looking for elements that decide a test success, fail or skip.

        I can now confirm Hudson "Maven2/3" jobs that generate surefire reports are the cause of missing jbehave reports in the test UI. The other hudson jobs type run ok) and you can run the examples (without XML corruptions) using maven within a "freestyle" hudson job and the tests will appear in the UI correctly.

        There are related two causes for this problem:
        1) Hudson can only process one test report in the UI. Mauro, you might haven seen two reports in the UI showing the same results and one having the links to failured test giving a 404.
        2) xUnit aggregates test results from other tools to avoid duplicating test reports. It doesn't do this with the Maven2/3 job as it overrides Hudsons testing model and adds very Maven specific code. I have been through the xUnit and Hudson code and I haven't found away of injecting or aggregating the test results in a "Maven2/3" job to avoid issue 1.

        The only way I can see of supporting the "Maven2/3" job within the Hudson plugin is to write some Hudson Maven specific code which could be complicated. My suggestion is to document the limitation of job type in the Hudson plugin and develop a solution for a future release.

        Show
        Alan Parkinson added a comment - I'm also seeing corrupted XML files being read by the Hudson plugin before any processing occurs. When reading a file xUnit will validate the XML file and then print " [xUnit] [ERROR] - The plugin hasn't been performed correctly: Conversion error Error to convert - A file not found" in the hudson build console if the file is not valid XML. To rule out filters causing problem I will add some unit tests but I don't think this will be a problem because the XSL is only looking for elements that decide a test success, fail or skip. I can now confirm Hudson "Maven2/3" jobs that generate surefire reports are the cause of missing jbehave reports in the test UI. The other hudson jobs type run ok) and you can run the examples (without XML corruptions) using maven within a "freestyle" hudson job and the tests will appear in the UI correctly. There are related two causes for this problem: 1) Hudson can only process one test report in the UI. Mauro, you might haven seen two reports in the UI showing the same results and one having the links to failured test giving a 404. 2) xUnit aggregates test results from other tools to avoid duplicating test reports. It doesn't do this with the Maven2/3 job as it overrides Hudsons testing model and adds very Maven specific code. I have been through the xUnit and Hudson code and I haven't found away of injecting or aggregating the test results in a "Maven2/3" job to avoid issue 1. The only way I can see of supporting the "Maven2/3" job within the Hudson plugin is to write some Hudson Maven specific code which could be complicated. My suggestion is to document the limitation of job type in the Hudson plugin and develop a solution for a future release.
        Hide
        Paul Hammant added a comment -

        Folks. Refer to JBEHAVE-420 (nothing to do with Hudson)

        I may have fixed the broken XML issue. I am not sure if JBEHAVE-420 relates to JBEHAVE-394 or not.

        Show
        Paul Hammant added a comment - Folks. Refer to JBEHAVE-420 (nothing to do with Hudson) I may have fixed the broken XML issue. I am not sure if JBEHAVE-420 relates to JBEHAVE-394 or not.
        Show
        Alan Parkinson added a comment - Proposed documentation change: https://github.com/aparkinson/jbehave-core/commit/d5bc77a06743ba91ed598efe7bfce19b677ba79b
        Hide
        Paul Hammant added a comment -

        I can confirm that the fix for JBEHAVE-420 now allows for the XML to be consumed into Hudson results. At least it does for me in respect of my stories/scenarios.

        Show
        Paul Hammant added a comment - I can confirm that the fix for JBEHAVE-420 now allows for the XML to be consumed into Hudson results. At least it does for me in respect of my stories/scenarios.
        Hide
        Alan Parkinson added a comment -

        I've added some extra testing for filters and some small XSL changes
        https://github.com/aparkinson/jbehave-core/commit/fc6b0fa9fae5d43ca2e0dd88b50a3086e99fa858

        Show
        Alan Parkinson added a comment - I've added some extra testing for filters and some small XSL changes https://github.com/aparkinson/jbehave-core/commit/fc6b0fa9fae5d43ca2e0dd88b50a3086e99fa858

          People

          • Assignee:
            Mauro Talevi
            Reporter:
            Alan Parkinson
          • Votes:
            2 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: