Background
I am looking for a way to implement a scheme similar to what Frank library uses to implement "Automated acceptance tests for native iOS apps", but I want this scheme rely on native iOS/MacOSX technologies. Sorry for the following TLDR but it deserves verbose explanation.
1. Here is a short overview of how Frank works:
it has Client and Server parts.
Server part is embedded into an application that we want to run acceptance tests against. Frank tutorials show us how to create a duplicate target of an app's main target and embed Frank HTTP server to it.
Client part - is a mainly a Cucumber that runs plain text scenarios: each scenario contains directives that should be run against an app (fill text field, touch button, ensure that a specific element exists on a page etc...). Also each scenario launches its own instance of app by this means providing a fresh state every time we enter a new scenario.
Client (Cucumber and Ruby-to-Objective-C bridge) communicates with Server (HTTP server embedded into an app) via HTTP protocol. It uses special conventions so client can tell server what an app should do so the specific scenario could be performed.
2. Recently I've found the following article written by the author of Frank Pete Hodgson:
http://blog.thepete.net/blog/2012/11/18/writing-ios-acceptance-tests-using-kiwi/
In which he suggests more simple way of writing acceptance tests for the developers who don't like to rely on external tools like Cucumber and Ruby. Let me quote the author himself:
Before I start, let me be clear that I personally wouldn’t use this approach to writing acceptance tests. I much prefer using a higher-level language like ruby to write these kinds of tests. The test code is way less work and way more expressive, assuming you’re comfortable in ruby. And that’s why I wanted to try this experiment. I’ve spoken to quite a few iOS developers over time who are not comfortable writing tests in ruby. They are more comfortable in Objective-C than anything else, and would like to write their tests in the same language they use for their production code. Fair enough.
This blog post inspired me to quickly roll my own very raw and primitive tool that does exactly what Pete described in his blog post: NativeAutomation.
Indeed, like it was described by Pete, it is possible to run acceptance tests by using just Kiwi/PublicAutomation setup placed in a simple OCTests target. I really liked it because:
- It is just pure C/Objective-C. It was very easy to build initial bunch of C helpers, that look like Capybara helpers:
tapButtonWithTitle, fillTextFieldWithPlaceholder, hasLabelWithTitle and so on...
It does not require external tools and languages: no need in Cucumber/Ruby or anything else. NativeAutomation itself uses just PublicAutomation that Frank also uses. PublicAutomation is needed to simulate user interactions on an app's screen: touches, fills, gestures...
It is very handy to run these tests right from Xcode by just running a Cocoa Unit Tests bundle. (Though command-line builds are easy as well).
Problem
The problem Kiwi/PublicAutomation approach has is that the whole testing suite is embedded into the app's bundle. It means that after each scenario is run, it is not possible to reset application to force it to be in a fresh state before next scenario begins execution. The only way to workaround this problem is to write Kiwi's beforeEach hook with a method that performs a soft-reset of an application like:
+ (void)resetApplication {
[Session invalidateSession];
[LocationManager resetInstance];
[((NavigationController *)[UIApplication sharedApplication].delegate.window.rootViewController) popToRootViewControllerAnimated:NO];
[OHHTTPStubs removeAllStubs];
cleanDatabase();
shouldEventuallyBeTrue(^BOOL{
return FirstScreen.isCurrentScreen;
});
but in case of application which involve networking, asynchronous jobs, core data, file operations it becomes hard to perform a real teardown of a stuff left by previous scenario.
Question
The problem described above made me thinking about if it is possible to implement a more complex approach similar to Frank's approach, with a second app that works apart from main app's bundle and does not rely on external tools like Cucumber(Ruby).
Here is how I see the way it could be done.
Besides a main app (MainApp) there is a second iOS (or Mac OS X) app (AcceptanceTestsRunnerApp) that contains the whole acceptance tests suite and runs this suite against a main app bundle:
It does fire up new simulator instance before it enters each new scenario and executes current scenario against current simulator's app's instance.
The question is:
I am not well awared about Mac OSX or iOS technologies that would allow me to do that: I don't know if it is even possible to setup a Mac OS X / iOS application (AcceptanceTestsRunnerApp) that could take a control over a main app (MainApp) and run acceptance tests scenarios against it.
I will be thankful for any thoughts/suggestions that people who feel more comfortable with having native Objective-C tool for writing acceptance tests for iOS applications might have.
UPDATED LATER
...I did read some documentation about XPC services but the irony about it is that scheme I am looking for should be quite opposite to the scheme XPC documentation suggests:
Ideally I want my AcceptanceTestsRunnerApp dominate over MainApp: run it and control it (user interactions, assertions about view hierarchy) via some object proxying to MainApp's application delegate while XPC services setup would assume XPC service (AcceptanceTestsRunnerApp) to be subordinate to an app (MainApp) and will require XPC service to live inside app's bundle which I want to avoid by all means.
...My current reading is Distributed Objects Programming Topics. It seems to me that I will find my answer there. If no one provides me with guides or directions I will post an answer about my own researches and thoughts.
...This is the next step in my quest: Distributed Objects on iOS.