How do you repeat a test in PHPUnit?

Question

I know about the "--repeat" option, but I'd rather define the repeating within the test and per test. In my unit tests there are tests I don't want to repeat, and there are tests I want to repeat more than others.

I was thinking:

protected function tearDown() {
  if (test has not been ran 3 times) {
      $this->runTest(); // Re-run the test
  }
}

This doesn't seem to work, nor does $this->run(). I've looked at the PHPUnit source code but I'm not sure. I'm guessing it's checking the test status and if it's been ran it denies running it again.

Bibliofilm

why would you need that in a unit-test? Oo – Sadirah

It's for an abstract class that extends PHPUnit_Framework_TestCase. The purpose is to define X repeats per test instead of global X repeats denoted by --repeat. – Bibliofilm

I'd also like to find a solution for this. @Sadirah - my case is that I'm testing a service over socket connection (thrift:php). It may happen that tests fail only because of network overload, which in fact doesn't mean . In case of Timeout exception I'd like to rerun the test, so that it passes or any other error/exception is thrown. – Substantialize

@tkoomzaaskz cant you just mock the remote end? or increase the timeout? – Sadirah

@Sadirah I cannot increase the timeout, since I'd need to modify settings of the external service, which I don't want to do. Could you explain what do you mean by mocking the remote end? If you meant that I shall use a mock object instead of a real service, this won't do, because we're testing the service and service-php communication. So the service has to be a real instance. – Substantialize

@tkoomzaaskz yes, that's what I meant and what is suggested in the PHPUnit Manual, too: "When your application interacts with a web service you want to test it without actually interacting with the web service.". It's not a unit-test when you are testing against the real service. – Sadirah

@Gordon: what solution would you recommend then (when I want to provide advanced tests for a real webservice)? – Substantialize

@tkoomzaaskz You could use Selenium to do an End-To-End Test, but in general I'd recommend not to test it at all. – Sadirah

Answer 1

PHPUnit has a repeat option within the command line runner. This works as follows for a test that repeats 3 times:

phpunit --repeat 3 myTest.php

Bartlet

I think this is only valid before version 10 – Mighell

@MarshallDavis Yes, it has been removed from the version 10. Here's a quick around: github.com/sebastianbergmann/phpunit/issues/… – Pazpaza

Answer 2

This is a bit of a roundabout way of doing this but it's the cleanest I could come up with:

/**
 * @dataProvider numberOfTests
 */
public function test()
{
    // Do your test
}

public function numberOfTests() 
{
    for ($i = 0; $i < 100; $i++) {
       yield [];
    }
}

The benefit of this is setUp and tearDown methods will be run for each invocation of the loop.

Harris

This is a good solution as setup teardown methods are run for each invocation of the testcase – Janina

Answer 3

There's far more to running a test than setUp, run, and tearDown. For one thing, each test method is run against a new instance of the test case. Don't forget about @dataProvider and the other annotations, code coverage, etc. You really don't want to do this.

For the few cases you absolutely need it, code the loop in the test method itself.

Foliate

Answer 4

As @reusen mentioned, we had the command

phpunit --repeat 3 myTest.php

But this feature has been removed and not planned to be re-implemented. At least, not at the moment.

Currently there's this workaround

while php vendor/bin/phpunit tests/ExampleTest.php --stop-on-failure; do echo; done

Disrelish

Answer 5

If you use terminal fish, you can run

for i in seq(seq 10)                                                                                                                                          
    sail artisan test --filter MyControllerTest
end

Coker

Answer 6

I think you need to take a step back and create a test that runs your tests!

You need a loop that goes something along the lines of:

$myTest = \my\test\class();
foreach($iterations){
    $myTest->setup();
    $myTest->doTestyStuff();
    $myTest->tearDown();
}

the code you posted won't work because each test needs the setup and teardown to be run each time the test runs.

Uraeus

I had it in tearDown because of the fact tearDown is ran after each test. That's how I was thinking of creating a loop. I would break the loop by checking a static iterator or something. What you're suggesting above I think I can work from, but I'm trying to run each test X number of times and not every test needs to be ran more than once or the same number of times. – Bibliofilm

Answer 7

Can this not be achieved with a do-while loop?

protected function tearDown() {
    $i = 0;
    do {
        $this->runTest(); // Re-run the test
        $i++;
    } while($i < 3);
}

Anastaciaanastas

The repeating isn't my problem, I can figure that out easily enough. It's that calling $test->runTest() does nothing. Also since tearDown is called after every test, your code assuming $test->runTest() worked would put it in an indefinite loop. – Bibliofilm