jMock

Fail fast when run concurrently

Details

  • Type: Improvement Improvement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: 2.6.0
  • Component/s: JMock 2.x.x Library
  • Labels:
    None
  • Environment:
    any
  • Number of attachments :
    0

Description

Steve F suggested I file this enhancement when we saw this behaviour at youDevise.

When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).

This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)

Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

Issue Links

This issue relates to:
JMOCK-213 Make jMock thread-safe Major Resolved

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Douglas Squirrel added a comment -

I just noticed JMock-213 which says JMock should be threadsafe. Obviously this would be preferable - I assumed thread safety was not possible (it sounds hard!)

Show
Douglas Squirrel added a comment - I just noticed JMock-213 which says JMock should be threadsafe. Obviously this would be preferable - I assumed thread safety was not possible (it sounds hard!)
Hide
Permalink
Joerg Schaible added a comment -

Actually the expectation should not take a special thread id into account. When you define exactly(N) calls, it does not matter if all of those are done from the same thread or not.

Show
Joerg Schaible added a comment - Actually the expectation should not take a special thread id into account. When you define exactly(N) calls, it does not matter if all of those are done from the same thread or not.
Hide
Permalink
Steve Freeman added a comment -

@Joerg. Actually it does, you can get misleading messages if one thread is still making calls while the test thread is trying to display the result.

Show
Steve Freeman added a comment - @Joerg. Actually it does, you can get misleading messages if one thread is still making calls while the test thread is trying to display the result.
Hide
Permalink
Joerg Schaible added a comment -

@Steve. This was addressed originally in JMOCK-59. All threads had been terminated before validation of the result happened ...

Show
Joerg Schaible added a comment - @Steve. This was addressed originally in JMOCK-59. All threads had been terminated before validation of the result happened ...
Hide
Permalink
Nat Pryce added a comment - - edited

The problem with that approach is that threads calling a mock object might have been started by a third-party library (a messaging API, for example), so the test has no control over their ThreadGroup.

JMock cannot automagically make the test synchronise with the threads started by the system under test. The Mockery can be thread-safe but at some point the programmer has to take responsibility for synchronisation between the test and their code.

Show
Nat Pryce added a comment - - edited The problem with that approach is that threads calling a mock object might have been started by a third-party library (a messaging API, for example), so the test has no control over their ThreadGroup. JMock cannot automagically make the test synchronise with the threads started by the system under test. The Mockery can be thread-safe but at some point the programmer has to take responsibility for synchronisation between the test and their code.
Hide
Permalink
Sebastian Sickelmann added a comment -

There is a way to "control" threads created and started by third-party libraries. It is a litle bit tricky but it works. I use it in my project mockinject.

In mockinject i use ThreadLocal variables to store individual state, so every unittest can run it its own thread. To adress the problem of rescueing the ThreadLocal to newly created threads (also from third-party libraries), I catch all Thread-creation-steps and inject own threads that does the rescueing process. It is a little bit tricky and slows down the execution-speed a bit but works realy nice (have some problems with swing-event-thread but that is another story)

Show
Sebastian Sickelmann added a comment - There is a way to "control" threads created and started by third-party libraries. It is a litle bit tricky but it works. I use it in my project mockinject. In mockinject i use ThreadLocal variables to store individual state, so every unittest can run it its own thread. To adress the problem of rescueing the ThreadLocal to newly created threads (also from third-party libraries), I catch all Thread-creation-steps and inject own threads that does the rescueing process. It is a little bit tricky and slows down the execution-speed a bit but works realy nice (have some problems with swing-event-thread but that is another story)
Hide
Permalink
Nat Pryce added a comment -

I'm uncomfortable with changing how third-party code behaves and then testing against it. I want to test that my code works with the third-party code as is, not after I've mucked about with how it creates threads.

I think this is a sidetrack. It's no good waiting for third-party threads to terminate. A third-party library might start long-lived daemon threads that outlast any single use of that API by a test. They might exist to keep a connection alive, for example. In which case, waiting for the threads to terminate would make the test hang.

See JMOCK-215 for how we're solving the issue of a test waiting for threads to finish. Feedback welcome.

Show
Nat Pryce added a comment - I'm uncomfortable with changing how third-party code behaves and then testing against it. I want to test that my code works with the third-party code as is, not after I've mucked about with how it creates threads. I think this is a sidetrack. It's no good waiting for third-party threads to terminate. A third-party library might start long-lived daemon threads that outlast any single use of that API by a test. They might exist to keep a connection alive, for example. In which case, waiting for the threads to terminate would make the test hang. See JMOCK-215 for how we're solving the issue of a test waiting for threads to finish. Feedback welcome.
Hide
Permalink
Nat Pryce added a comment -

I've implemented this. However, getting a clear failure is difficult because any error is going to be thrown onto the background thread, not onto the test thread. What I've done is throw ConcurrentModificationException on the background thread and print a warning to stderr in case the thread swallows the exception. The test should fail with an expectation failure, but if it doesn't (e.g. if the test only contains 'allowing' clauses) at least the error log will contain obvious warnings.

Show
Nat Pryce added a comment - I've implemented this. However, getting a clear failure is difficult because any error is going to be thrown onto the background thread, not onto the test thread. What I've done is throw ConcurrentModificationException on the background thread and print a warning to stderr in case the thread swallows the exception. The test should fail with an expectation failure, but if it doesn't (e.g. if the test only contains 'allowing' clauses) at least the error log will contain obvious warnings.
Hide
Permalink
Nat Pryce added a comment -

Resolved for when the user does not change the Mockery's imposteriser. I'm not entirely happy with the implementation but it covers 90% of cases I think.

Show
Nat Pryce added a comment - Resolved for when the user does not change the Mockery's imposteriser. I'm not entirely happy with the implementation but it covers 90% of cases I think.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.