Resetting iOS simulator between tests

Question

I have a set of automated test cases set up in instruments using tuneup.js to test an app. I decided to use tuneup.js as it allowed me to separate my tests into individual test cases and run the whole set from one individual script, this works fine if all the tests run ok, however if one fails, all the tests fail as the simulator is left in an unknown state (I have written my tests so they all start and end on the same login screen) Is there a way to reset the simulator, or restart the app between test cases?

Soil

Have you by any chance found a solution to this? – Commissar

No unfortunately not, still looking but not really holding much hope of finding a solution - have asked Apple the question directly so will update if I get an answer – Soil

Did you get the answer from Apple? – Adelaidaadelaide

no, unfortunately not -- I never got round to resolving before I handed it over to someone else, however I will ask and see if theres been any update since then – Soil

Answer 1

Try to launch tests from the command line. UI Automation allows to execute only one test in one run. After the test will be completed (does not matter if it was failed or passed) - application will be kicked by the system (UIAutomation). At least it works with real devices.

Your command line launch script will work in the following manner: 1. Reads configuration file (can be any file txt or xml) with path to your tests. At this point you will have an array with path to your tests and total tests count. 2. Then using simple 'for' loop (from 1 to 'testcount') it will launch UIAutomation with required parameters. One of the parameters will be the path to your test script that was read from the configuration file.

You can also put the path to the 'configuration file' as a parameter for your command line launch script. This will allow you to run any test set simply calling the launch script with required configuration file.

Frowzy

Answer 2

I wrote a script that will reset the contents & settings of all versions and devices for the iOS Simulator. It grabs the device names and version numbers from the menu, so it will include any new devices or iOS versions that Apple releases simulators for.

It's easy to run manually or use in a build-script. I would suggest adding it as a Pre-Action Run Script before the build.

https://github.com/michaelpatzer/ResetAllSimulators

Pinkard

Answer 3

Having failed tests leave your app in an unknown state is one of the main problems with using Apple's instruments tool as-is. We solved this in a framework called Illuminator (on GitHub, and inspired by tuneup.js) in two ways.

First, we wrote an automation bridge -- a channel for RPC with the app being tested, which allows us to reset our app before each test.

In cases where that's not sufficient, the Illuminator test runner has an option to re-run each failed test in its own pristine launch of the simulator (e.g. with --retest 1x,solo).

Mercola