Running UIAutomation scripts from Xcode
Asked Answered
Y

6

20

Did anyone succeed in setting up automated UIAutomation tests in Xcode?

I'm trying to set up a target in my Xcode project that should run all the UIAutomation scripts I prepared. Currently, the only Build Phase of this target is this Run Script block:

TEMPLATE="/Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate"
MY_APP="/Users/Me/Library/Application Support/iPhone Simulator/6.0/Applications/564ED15A-A435-422B-82C4-5AE7DBBC27DD/MyApp.app"
RESULTS="/Users/Me/Projects/MyApp/Tests/UI/Traces/Automation.trace"
SCRIPT="/Users/Me/Projects/MyApp/Tests/UI/SomeTest.js"
instruments -t $TEMPLATE $MY_APP -e UIASCRIPT $SCRIPT -e UIARESULTSPATH $RESULTS

When I build this target it succeeds after a few seconds, but the script didn't actually run. In the build log I get these errors:

instruments[7222:707] Failed to load Mobile Device Locator plugin
instruments[7222:707] Failed to load Simulator Local Device Locator plugin
instruments[7222:707] Automation Instrument ran into an exception while trying to run the script.  UIATargetHasGoneAWOLException
+0000 Fail: An error occurred while trying to run the script.
Instruments Trace Complete (Duration : 1.077379s; Output : /Users/Me/Projects/MyApp/Tests/UI/Traces/Automation.trace)

I am pretty sure, that my javascript and my run script are both correct, because if I run the exact same instruments command in bash it works as expected. Could this be a bug in Xcode?

Yazzie answered 17/12, 2012 at 22:25 Comment(4)
Have you tried to run instruments via xcrun - maybe that helps to get its environment setup properly? e.g. xcrun instruments -t [...]Kapor
Ow, for some more inspiration, have a look at a little project I started with a colleague: jenkins-automation.Kapor
I get the same results. Very strange, since you seem to do the same thing. But you run this script outside of Xcode, right? My problem is, that it's not working if I try to run it as a target. If I just run the script on terminal it's fine.Yazzie
Yes, that script is run via Jenkins, outside of Xcode.Kapor
Y
4

I finally found a solution for this problem. It seems like Xcode is running the Run Scripts with limited rights. I'm not entirely sure, what causes the instruments command to fail, but using su to change to your user will fix it.

su $USER -l -c <instruments command>

Obviously, this will ask you for your password, but you can't enter it when running as a script. I didn't find a way to specify the password for su, however if you run it as root, you don't have to specify one. Luckily sudo can accept a password via the pipe:

echo <password> | sudo -S su $USER -l -c <instruments command>

If you don't want to hardcode your password (always a bad idea), you could use some AppleScript to ask for the password.

I posted the resulting script below. Copy that to a *.sh file in your project and run that script from a Run Script.

#!/bin/bash

# This script should run all (currently only one) tests, independently from
# where it is called from (terminal, or Xcode Run Script).

# REQUIREMENTS: This script has to be located in the same folder as all the
# UIAutomation tests. Additionally, a *.tracetemplate file has to be present
# in the same folder. This can be created with Instruments (Save as template...)

# The following variables have to be configured:
EXECUTABLE="TestApp.app"

# Optional. If not set, you will be prompted for the password.
#PASSWORD="password"

# Find the test folder (this script has to be located in the same folder).
ROOT="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

# Prepare all the required args for instruments.
TEMPLATE=`find $ROOT -name '*.tracetemplate'`
EXECUTABLE=`find ~/Library/Application\ Support/iPhone\ Simulator | grep "${EXECUTABLE}$"`
SCRIPTS=`find $ROOT -name '*.js'`

# Prepare traces folder
TRACES="${ROOT}/Traces/`date +%Y-%m-%d_%H-%M-%S`"
mkdir -p "$TRACES"

# Get the name of the user we should use to run Instruments.
# Currently this is done, by getting the owner of the folder containing this script.
USERNAME=`ls -l "${ROOT}/.." | grep \`basename "$ROOT"\` | awk '{print $3}'`

# Bring simulator window to front. Depending on the localization, the name is different.
osascript -e 'try
    tell application "iOS Simulator" to activate
on error
    tell application "iOS-Simulator" to activate
end try'

# Prepare an Apple Script that promts for the password.
PASS_SCRIPT="tell application \"System Events\"
activate
display dialog \"Password for user $USER:\" default answer \"\" with hidden answer
text returned of the result
end tell"

# If the password is not set directly in this script, show the password prompt window.
if [ -z "$PASSWORD" ]; then
    PASSWORD=`osascript -e "$PASS_SCRIPT"`
fi

# Run all the tests.
for SCRIPT in $SCRIPTS; do
    echo -e "\nRunning test script $SCRIPT"
    COMMAND="instruments -t \"$TEMPLATE\" \"$EXECUTABLE\" -e UIASCRIPT \"$SCRIPT\""
    COMMAND="echo '$PASSWORD' | sudo -S su $USER -l -c '$COMMAND'"
    echo "$COMMAND"
    eval $COMMAND > results.log

    SCRIPTNAME=`basename "$SCRIPT"`
    TRACENAME=`echo "$SCRIPTNAME" | sed 's_\.js$_.trace_g'`
    mv *.trace "${TRACES}/${TRACENAME}"

    if [ `grep " Fail: " results.log | wc -l` -gt 0 ]; then
        echo "Test ${SCRIPTNAME} failed. See trace for details."
        open "${TRACES}/${TRACENAME}"
        exit 1
        break
    fi

done

rm results.log
Yazzie answered 4/1, 2013 at 17:5 Comment(2)
Thanks for your answer. I found this script work well in my case. But there is a problem with output file. I can not find any *.trace file. Any help?Desist
I dont understand this line mv *.trace "${TRACES}/${TRACENAME}". I dont see any trace file in my computer. Please helpDesist
E
1

It seems as though this really might be an Xcode problem; at any rate, at least one person has filed a Radar report on it. Someone in this other thread claims you can work around this exception by disconnecting any iDevices that are currently connected to the computer, but I suspect that does not apply when you're trying to run the script as an Xcode target.

I would suggest filing a Radar report as well; you may get further details on the issue from Apple, or at least convince them that many people are having the problem and they ought to figure out what's going on.

Sorry for a not-terribly-helpful answer (should have been a comment, but comments and links/formatting do not mix very well). Please update this question with anything you find out on the issue.

Electrocardiograph answered 27/12, 2012 at 8:10 Comment(2)
Sorry, I'm going to accept my new solution instead. Thanks for you help, anyways.Yazzie
Don't apologize - thanks for taking the time to come back and update the thread! This workaround will end up being useful for many of us as well.Electrocardiograph
M
1

Note: this is not a direct answer to the question, but it is an alternative solution to the underlying problem.

While searching for in-depth information about UIAutomation, I stumbled across a framework by Square called KIF (Keep it functional). It is a integration testing framework that allows for many of the same features as UIAutomation, but the great thing about is is that you can just write your integration tests in Objective-C.

It is very easy to setup (via CocoaPods), they have good examples too, and the best thing is that it's a breeze to set up with your CI system like Jenkins.

Have a look at: http://github.com/square/KIF

Messinger answered 2/2, 2013 at 9:45 Comment(1)
Yes I know, I did come across questions like these when KIF actually solved my underlying problem. I added a note for clarification. Cheers.Messinger
B
1

Late to the game but I have a solution that works for Xcode 5.1. Don't know if that's what broke the above solution or not. With the old solution I was still getting:

Failed to load Mobile Device Locator plugin, etc.

However, this works for the release version of Xcode 5.1.

echo <password> | sudo -S -u username xcrun instruments

Notice I removed the unneeded su command and added the xcrun command. The xcrun was the magic that was needed.

Here is my complete command:

echo <password> | sudo -S -u username xcrun instruments\ 
  -w "iPhone Retina (3.5-inch) - Simulator - iOS 7.1"\
  -D "${PROJECT_DIR}/TestResults/Traces/Traces.trace"\
  -t "${DEVELOPER_DIR}/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate"\
  "${BUILT_PRODUCTS_DIR}/MyApp.app"\
  -e UIARESULTSPATH "${PROJECT_DIR}/TestResults"\
  -e UIASCRIPT "${PROJECT_DIR}/UITests/main.js"

By the way if you type:

instruments -s devices

you will get a list of all the supported devices you can use for the -w option.

Edit: To make this work for different people checking out the project replace the following:

echo <password> | sudo -S -u username xcrun instruments

with

sudo -u ${USER} xcrun instruments

Since you are just doing an sudo to the same user no password is required.

Burrton answered 19/3, 2014 at 18:42 Comment(0)
E
0

Take a look at this tutorial that explains how to have Automated UI testing with Jenkins. It also uses Jasmine in the tutorial though. http://shaune.com.au/automated-ui-testing-for-ios-apps-uiautomation-jasmine-jenkins/ hope this helps. It has an example project file so you can download that as a template. Hope this helps.

Enlil answered 25/12, 2012 at 7:18 Comment(0)
F
0

In XCode - if you load up organizer (XCode->Window->Organizer)

Then select your machine under devices -> 'Enable Developer Mode'

This should remove the need for prompts with instruments.

Fagaly answered 24/5, 2014 at 20:41 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.