Maven Surefire

after running test, when tests fail, it's hard to the find the failure reason

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.4.3
  • Fix Version/s: 2.13
  • Component/s: Maven Surefire Plugin
  • Labels:
    None
  • Complexity:
    Intermediate
  • Number of attachments :
    0

Description

Maven & surefire has been here for a while but still I'm missing this feature. When my tests fail, I'd like to know exact failure reason.

1. Surefire summary does not show the stack trace / line number / assertion. Browsing results files in target dir is frustrating & time consuming.
2. It is impossible to use surefire-report plugin in fully automated way. If I plug report-only goal to phase test, this goal is not executed when phase test fails

Current workaround in my project is that every maven submodule has a handy batch file with 'mvn surefire-report:report-only'.

One of the ways of implementing it is adding stacktrace/assertion information in test summary.

What do you guys think about this idea?

Issue Links

relates to

New Feature - A new feature of the product, which has yet to be developed. SUREFIRE-726 Test list preprocessor support for tests to be run

  • Major - Major loss of function.
  • Open - The issue is open and ready for the assignee to start work on it.

Improvement - An improvement or enhancement to an existing feature or task. SUREFIRE-631 turn on useFile=false from the command line?

  • Major - Major loss of function.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Benjamin Bentmann added a comment -

1. Surefire summary does not show the stack trace / line number / assertion. Browsing results files in target dir is frustrating & time consuming.

And you already tried setting the plugin's parameter useFile to false?

Show
Benjamin Bentmann added a comment - 1. Surefire summary does not show the stack trace / line number / assertion. Browsing results files in target dir is frustrating & time consuming. And you already tried setting the plugin's parameter useFile to false?
Hide
Permalink
Szczepan Faber added a comment -

No I haven't - it didn't occur me at all that this option can be any helpful. Thanks for the hint!

Anyway, I will see how useful it is and get back.

Show
Szczepan Faber added a comment - No I haven't - it didn't occur me at all that this option can be any helpful. Thanks for the hint! Anyway, I will see how useful it is and get back.
james strachan made changes -
Field Original Value New Value
Link This issue relates to SUREFIRE-631 [ SUREFIRE-631 ]
Hide
Permalink
james strachan added a comment -

I think this issue is mostly resolved now - though it would be nice to be able to turn off useFile from the command line like this related issue

Show
james strachan added a comment - I think this issue is mostly resolved now - though it would be nice to be able to turn off useFile from the command line like this related issue
Kristian Rosenvold made changes -
Component/s Maven Surefire Plugin [ 12561 ]
Hide
Permalink
Alexander Klimetschek added a comment -

For working with the file reports, have a look at SUREFIRE-701 - it provides a showSuccess=false option that simply omits successful reports. Together with disableXmlReport=true to get rid of the xml files in target/surefire-reports, very useful when you develop & only look for the failed test reports.

Show
Alexander Klimetschek added a comment - For working with the file reports, have a look at SUREFIRE-701 - it provides a showSuccess=false option that simply omits successful reports. Together with disableXmlReport=true to get rid of the xml files in target/surefire-reports, very useful when you develop & only look for the failed test reports.
Kristian Rosenvold made changes -
Link This issue relates to SUREFIRE-726 [ SUREFIRE-726 ]
Hide
Permalink
Kristian Rosenvold added a comment -

Fixed with greatly improved summary display

Show
Kristian Rosenvold added a comment - Fixed with greatly improved summary display
Kristian Rosenvold made changes -
Fix Version/s 2.13 [ 18833 ]
Resolution Fixed [ 1 ]
Status Open [ 1 ] Closed [ 6 ]
Assignee Kristian Rosenvold [ krosenvold ]

People

  • Assignee:
    Kristian Rosenvold
    Reporter:
    Szczepan Faber
Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: