JBehave

Using GivenScenarios causes failed tests to be forgotten

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 2.5.3
  • Fix Version/s: 2.5.4, 3.0
  • Component/s: Core
  • Labels:
    None
  • Testcase included:
    yes
  • Number of attachments :
    1

Description

We have multiple related scenarios within the same JBehave script. This seems to be supported functionality but a major issue came to light earlier: using GivenScenarios at the start of a later scenario will cause any failed scenarios thus far to be forgotten. For example:

""
Story: Test showing that using a given scenario skews the overall result

Scenario: Fail

GivenScenarios: com/example/sandpit/jbehave/noop
Given I do nothing
Then I fail

Scenario: Succeed

GivenScenarios: com/example/sandpit/jbehave/noop
Given I do nothing
Then I pass
""

The JUnit result for running a scenario with the above script is a pass, even though the "I fail" step threw an exception. If you remove the GivenScenarios line from the Succeed case then the JUnit result is a fail.

Attached is an Eclipse project that demonstrates this. This is a major issue for us: we run the JBehave tests from our build farm in an automated process. This bug would give us the green light for the whole build despite tests failing.

Activity

Hide
Mauro Talevi added a comment -

Added failure_followed_by_given_scenarios.scenario in trader example reproducing issue, based on example project provided.

The problem stems from the fact that the failed state is per story and running given scenarios resets the state.

Show
Mauro Talevi added a comment - Added failure_followed_by_given_scenarios.scenario in trader example reproducing issue, based on example project provided. The problem stems from the fact that the failed state is per story and running given scenarios resets the state.
Hide
Mauro Talevi added a comment -

Error state reset only for non-embedded stories.

Show
Mauro Talevi added a comment - Error state reset only for non-embedded stories.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: