How do I instrument test runners in SUnit?

Question

I would like to instrument SUnit tests in Pharo. What is the proper way to change how tests are run?

Example:

I want to introduce a timeout to tests, each test I run should be aborted after a given delay.

Problem:

SUnit does not feature a dedicated TestRunner in the model that would allow me to introduce changes easily. I can create a new subclass of TestResult and use the API there (runCase:, addError: ...) to get the enough control. However it feels strange to change the result class to modify the behavior on how tests are run.

I am used to SMark where I have a dedicated runner to modify these things.

Monthly

Answer 1

(DISCLAIMER: I do not know what version of SUnit Pharo uses, so the concrete API may differ. Still, I hope this helps a bit.)

Subclassing TestResult is the way to go even though it may sound bit weird.

The general idea for customizing a new 'SUnit' instance is:

• for run-specific customization, do it in TestResult (namely #runCase: aTestCase debugged: aBoolean)
• for testcase-specific customization, do it in TestCase (namely #performTest)

I personally want to keep SUnit as simple as possible and here I see no reason for introducing a new class. Why? Look to recent changes in JUnit. :-) I do agree that SUnit deserves more documentation.

For examples how to implement timeout, skipping, etc., have a look at TestReport in Smalltalk/X.

Gur

I'll dig into the Smalltalk/X version, Pharo's SUnit didn't change much I think.I did this approach and it works nicely if your tests that locally require special care. However if I want to do something global it doesn't work. – Monthly

So yes, you do subclass TestResult to get custom behavior for me this still sounds wrong :), it's a result, it should not have anything to do with running :/. But Smalltalk/X has a much better protocol on Result addError:detail: and Co in Pharo are simply addError: so you miss half the information. I think I'll add that for pharo. – Monthly