UIAutomation test invocation from the command line with Xcode 6

Question

Running UIAutomation tests from the command line seems to break often with new Xcode releases (judging by past posts). Having never used the command line scripting for this, I found this post from 2012: Automation Instrument from the Command Line.

The Problem: My command returns without error, without output results and without anything logged to the system console. The simulator does not even launch!

Checking some of the paths for updates (notably, the Automation trace instrument path), I came up with this command. Note the path in the first parameter (it's different than in past Xcode releases):

instruments -t "/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate"         
"/Users/sohail/Library/Developer/CoreSimulator/Devices/7232A640-A9D2-4626-A2AD-37AFFF706718/data/Containers/Bundle/Application/D07FEC4B-76AD-4844-8362-08E771B81053/MyAppName.app"
-e UIASCRIPT  "/Users/sohail/source/MyAppName/MyAppNameAutomationTests/TestRunner.js" 
-e UIARESULTSPATH "Users/sohail/source/MyAppName/MyAppNameAutomationTests/TestResults"

This might be easier to read by selecting "raw" from this gist.

Of course:

• I validated that the path specified to my .app actually exists; it came into being after a successful build and run.
• I validated that in my specified output folder ("TestResults") that in fact, nothing got logged.
• I validated that the aforementioned TestRunner.js file I specified, could be found at the path specified, and successfully runs in the Automation Instrument interactively with the Instruments app.
• I've checked Apple's Xcode6/iOS8 pre-release documentation (login required; see section title, "Executing an Automation Instrument Script from the Command Line"), and nothing jumps out at me as wrong, since I'm just targeting the simulator.

My suspicion

• I'm missing some flag or switch somewhere.

Thoughts anyone?

Montiel

Answer 1

I saw the exact same issue, after explicitly provided -w $DEVICE parameter, my tests finally got started

instruments -t /Applications/Xcode6.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate 
-w "iPhone 5s (8.0 Simulator)" /path/to/my/TestApp.app -e UIASCRIPT /tmp/script.js -e UIARESULTSPATH /tmp

Libyan

That did it! Past documentation said it was optional, but now we know that the -w flag is mandatory. – Montiel

FYI - I decided to go "all in" and put together a bash script to run this command, and do it intelligently by dynamically finding the relevant paths of one's simulators and apps, which often change: github.com/idStar/ui-automation-runner – Montiel

Have you had any issues using this with a physical device? – Collocate

What is actually is the template? What does it do? -t /Applications/Xcode6.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.xrplugin/Contents/Resources/Automation.tracetemplate – Blessed

@Blessed instruments has many templates, automation is only one of them, see: developer.apple.com/library/mac/documentation/DeveloperTools/… and developer.apple.com/library/mac/documentation/AnalysisTools/… – Libyan