Rerun failed unit test - Xcode XCTest

Question

When I run my XCTests, I'd like to automatically rerun, once, any integration (unit/ui) test that fails. Is this possible?

This would be done in the same test, without having to hit 'run' again on my tests, or any part of my tests. I'm running all my tests from the command line.

I am doing UI tests that use the network to make calls to the server. If there is a significant server problem, the test will/should fail and report an error. However, if it is only a temporary problem with the request, it would be nice to rerun the test automatically and see if it passes. Also, with the current state of Xcode UI testing there are some occasional problems where it will crash for an obscure reason, and it would be nice to rerun the test automatically to see if it passes the second time.

It would be especially nice if it could output what happened, i.e. "The test failed the first time, because of failure getting refreshed snapshot, but passed the second time"

Wesle

The issue is that you shouldn't be relying on network calls for your tests. You can stub out those network requests with NSURLProtocol subclasses or use something like OHTTPStubs, otherwise you'll need to write a script that runs your tests, looks for errors and attempts to rerun those tests. – Amiens

This isn't possible if the OP is writing an integration test with real users and data. – Millerite

Answer 1

You can now do this in Xcode without any scripts.

1. Press ⌘ + 6 or select the test assistant view.
2. Filter for failed tests
3. Select tests, right click and choose Run x test methods > Run in all classes
4. (optional) If you want to run the same tests again press ⌃ + ⌥ + ⌘ + G for a quick way to run them right away.

---

---

Punishable

Answer 2

Task list to accomplish this without fastlane and with granular test reruns

• create a script to rerun the test and capture the list of failed tests, then rerun the tests using the -only-testing:TEST-IDENTIFIER flag of xcodebuild.
• parse the resultBundle or .xcresult to find the failed tests
• write code to erase the device or simulator executing the tests
• reboot the device if it is a real one to clear any possible dialogs
• use XCAttachment to collect things you care about, like app logs
• use xchtmlreport to generate a nice webpage from the results bundle with your snapshots and attachments.

Millerite

Answer 3

Fastlane can make it possible here is the awesome blog post regarding same

Stabilizing the CI By Re-runing Flaky iOS XCUI Tests

Showoff

cons with this approach: dependency on fastlane, inability to rerun a single failed test without rerunning the entire suite, example does not handle real devices. – Millerite