how can I improve iPhone UI Automation?

Question

I was googling a lot in order to find a solution for my problems with UI Automation. I found a post that nice summarizes the issues:

There's no way to run tests from the command line.(...)

There's no way to set up or reset state. (...)

Part of the previous problem is that UI Automation has no concept of discrete tests. (...)

There's no way to programmatically retrieve the results of the test run. (...)

source: https://content.pivotal.io/blog/iphone-ui-automation-tests-a-decent-start

Problem no. 3 can be solved with jasmine (https://github.com/pivotal/jasmine-iphone)

How about other problems? Have there been any improvements introduced since that post (July 20, 2010)?

And one more problem: is it true that the only existing method for selecting a particular UI element is adding an accessibility label in the application source code?

Patinated

Answer 1

While UI Automation has improved since that post was made, the improvements that I've seen have all been related to reliability rather than new functionality.

He brings up good points about some of the issues with using UI Automation for more serious testing. If you read the comments later on, there's a significant amount of discussion about ways to address these issues.

The topic of running tests from the command line is discussed in this question, where a potential solution is hinted at in the Apple Developer Forums. I've not tried this myself.

You can export the results of a test after it is run, which you could parse offline.

Finally, in regards to your last question, you can address UI elements without assigning them an accessibility label. Many common UIKit controls are accessible by default, so you can already target them by name. Otherwise, you can pick out views from their location in the display hierarchy, like in the following example:

var tableView = mainWindow.tableViews()[0];

As always, if there's something missing from the UI Automation tool that is important to you, file an enhancement request so that it might find its way into the next version of the SDK.

Gerbold

Thanks a lot for detailed answer. I will read pages you refers to later, but after quick look I think they could help me. Regarding "serious testing" - this is my case, since I'm going to integrate this process with my CI server. – Patinated

Sorry, but integrating Instruments in CI will be pain in the ass. Currently I am using Sikuli (sikuli.org) to hack around issues that you are pointing, but still I wouldn't put that in CI. For CI I would use this approach cocoawithlove.com/2009/12/… – Hydromedusa

yoosiba: You really prefer using a generic approach like Sikuli? How would you compare the time efforts when writing standard gui test cases for UIAutomation and Sikuli? – Maestricht

[Shameless plug] If you're looking for something that is designed to be run as part of CI, check out my open-source Frank project - vimeo.com/21860134 – Vadnee

Answer 2

Have you tried IMAT? https://code.intuit.com/sf/sfmain/do/viewProject/projects.ginsu . It uses the native javascript sdk that Apple provides and can be triggered via command line or Instruments.

In response to each of your questions:

There's no way to run tests from the command line.(...)

Apple now provides this. With IMAT, you can kick off tests via command line or via Instruments. Before Apple provided the command line interface, we were using AppleScript to bring up Instruments and then kick off the tests - nasty.

There's no way to set up or reset state. (...)

Check out this state diagram: https://code.intuit.com/sf/wiki/do/viewPage/projects.ginsu/wiki/RecoveringFromTestFailures

Part of the previous problem is that UI Automation has no concept of discrete tests. (...)

Agreed. Both IMAT and tuneup.js (https://github.com/alexvollmer/tuneup_js#readme) allow for this.

There's no way to programmatically retrieve the results of the test run. (...)

Reading the trailing plist file is not trivial. IMAT provides a jUnit like report after a test run by reading the plist file and this is picked up by my CI Tool (Teamcity, Jenkins, CruiseControl)

Pedrick

There's a link to this still available on my githhub page. github.com/rdpfeffer/IMAT – Dionedionis

btw, it would be great if you could accept one of the answers here ;-) @Pedrick seems to have answered all of your concerns. – Dionedionis

Answer 3

Try to check the element hierarchy, the table can be placed over a UIScrollView.

var tableV = mainWindowTarget.scrollViews()[0].tableViews()[0].scrollToElementWithName("Name of element inside the cell");

the above script will work even the element is in 12th cell(but the name should be exactly the same as mentioned inside the cell)

Underpass

Answer 4

Check out http://lemonjar.com/blog/?p=69 It talks about how to run UIA from the command line

Southsouthwest

Welcome to Stack Overflow! Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. – Scraper

agree with Kev. but it's good news that new sdk finally has cmd support. thanks for good news! – Patinated