Xcode 8 UI Testing Taking Very Long

Question

I'm writing UI tests in XCode 8 for a very complicated app. I'm discovering several issues, one of them being that each test I run takes several minutes.

The flow for the particular test I'm running isn't even that complicated.

I get many "Wait for app to idle" messages.

Does anyone know why the tests take so long to run, or do you have any suggestions on how I can speed them up? The WWDC demo was basically instant, but that was a very simple app.

Here's an example of what I see, and how long it takes. (The test is also still running!)

t =   504.16s         Wait for app to idle
t =   625.29s         App animations complete notification not received, will attempt to continue.
t =   625.30s         Synthesize event
t =   625.67s         Wait for app to idle

Thanks!

Update: When I slightly interact with the app, while keeping the elements I need visible, like by pulling on a tableview, the tests continue.

Five

What is it that your app is doing during the long gaps? – Cowberry

@matt, I'm not really sure? It looks like all of the views are loaded. There is a device sync running, do you think that could be it? – Five

Answer 1

Thanks @Oletha - I found the issue, which may be specific to this app:

After deleting large amounts of code at a time to see where the issue was, we found that we were calling beginRefreshing and endRefreshing on a subclass of UIRefreshControl, on consecutive lines.

Something about that stalled the UI, enough to hold the app in "Wait for app to idle" for minutes.

We replaced those two lines with a custom method, and now tests are running normally.

Five

I also face this problem. can you teach me how to write the custom code? – Carbuncle

You're also experiencing long lags with beginRefreshing and endRefreshing? – Five

Could one of you elaborate on this? I'm running into a very similar situation – Brocatel

From what I understand about it, with XCTest, whenever there is any animation anywhere in the app, even the network activity indicator at the top of the screen, the framework will wait for that to end before continuing. The app I'm working on deals with a device sync, so something about the way the code was written was causing piggy-backed network requests, leaving the app idle for a long time, since there was a background process running. I was using XCTest for a while; Google's EarlGrey (github.com/google/EarlGrey) is way faster and better to use for me. Check it out! – Five

Answer 2

The framework will wait for the app to become static, so if any views are animating or any activity indicators are shown, possibly including the network activity indicator in the status bar, the framework will hang until the app is still. This is to prevent race conditions with animations and components loading while the UI tests run.

If you think it's OK to override some of this behaviour to speed up your UI tests, you can try experimenting with removing activity indicators.

Simone

This seems like a really good lead - When I interact with the app slightly, like by pulling down a tableview, the test continues. Also, the network activity indicator will run every thirty seconds, but the test is still stalled even when it's not spinning. How would you go about overriding isAnimating properties? – Five

I'm not the best person to tell you about animations in views, but from what I understand, you might want to look at how you're handling animation completion blocks in your application. See if they're completing properly? I think I was mistaken about the isAnimating property as it's only for UIImageViews. Going to remove that bit from my answer. – Simone

Answer 3

I encountered the same issue. The solution was to move the call to endRefreshing from the handler to the end of a network method triggered in the handler. In other words, if you refresh a table view, make sure to endRefreshing after all data has been loaded.

Resistor

Answer 4

In my case was because I had some animations (UIView.animate...) so I had to UIView.setAnimationsEnabled(false) when running the app from the UI tests (you can send launchArguments or something).

Chinchin

Answer 5

Set launchArguments just before calling app.launch():

app.launchArguments = ["enable-testing"]
app.launch()

Then in the AppDelegate, check launch arguments:

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
    #if DEBUG
        if CommandLine.arguments.contains("enable-testing") {
            UIView.setAnimationsEnabled(false)
        }
    #endif
    // ...
}

Thats all!

Scaife

Answer 6

I had the same issue - on an actual device I would continually see "Wait for app to idle"

2 simple solutions:

1) If you're running on a device, make sure it's kept still (i.e not moving). Running tests while holding the device it would always be waiting for the app to idle. Placed flat on a desk, tests ran fine.

2) Alternatively, run your tests on the simulator if possible

Carat