UI test fails when it types text into a text view when run by an Xcode bot

Question

I have the following XCTest UI test that types text into a text view.

let textView = app.textViews.elementBoundByIndex(0)
textView.tap()
textView.typeText("Hello world")

When run as an Xcode bot it shows the following error for the typeText call.

Assertion: UI Testing Failure - failed: Timed out waiting for key event to complete

Interestingly, when I run it manually from the Xcode on the same computer the test passes. This test also passed in Xcode bot before upgrade to Xcode 7.1 / iOS 9.1. What can be the source of the problem?

Here is an isolated demo with the UI test: https://github.com/exchangegroup/UITestTextViewDemo

iOS 9.1 Simulator, OS X 10.11.1 (15B42), Xcode 7.1 (7B91b), OS X Server 5.0.15 (15S4033)

Reported to Apple.

Amphibolous

Apple screwed something up with XC7.1/iOS 9.1 update for Bots. Our green test suite turned to red in an instant. Failing with either this error, "Timeout waiting for screenshot" or "Timeout starting the sim". I really hope that XC7.2 will fix this. – Godmother

I'm seeing the same issue. I don't think I have a "fix", but I've found that if I 'wait' a little longer, sometimes the test will pass. My particular issue is with a WKWebView that has an internal "content editable" field, but the principle here is the same. After a 'tap' on the editor, I issue a run-loop wait: NSRunLoop.currentRunLoop().runUntilDate(NSDate(timeIntervalSinceNow: 3)) – Excursive

something new? i have the same problem in xcode 7.3.1 – Yaupon

Answer 1

I found a solution for my case and I hope it helps you as well.

In my setUp() and tearDown() (seems redundant I know) I put XCUIApplication().terminate(). This is ensuring that the app is terminated before running the next test and it seems to be doing the job.

override func setUp() {
    XCUIApplication().terminate()
    super.setUp()
    continueAfterFailure = false
    XCUIApplication().launch()
}
override func tearDown() {
    super.tearDown()
    XCUIApplication().terminate()
}

I filed a bug with Apple but for the time being this is getting me around the error that you were seeing. Hope that helps!

Thankful

Answer 2

I believe the underlying issue is that "Connect Hardware Keyboard" is turned on by default. Even if you turn it off for the main user, the _xcsbuildd user still uses the default. I was able to solve the problem by adding a pre test action to the scheme with the following script:

if [ `defaults read com.apple.iphonesimulator ConnectHardwareKeyboard` -eq 1 ]
 then
  defaults write com.apple.iphonesimulator ConnectHardwareKeyboard -bool false
  killall "Simulator"
fi

Karankaras

The script is really useful, I always have to remember to disable hardware keyboard while running tests on my dev machine. I never experienced this problem on the Xcode Server, but it is useful to have just to make sure it works as expected. – Bloxberg

Answer 3

I found out another solution, since the one proposed by Konnor wasn't working for me: I was having a lot of problems using UITest on Xcode Bot, and the common symptom was that testing on simulator took a lot more time than on the local machine, so I thought that the _xcsbuildd user used to run the tests was in some way slowed down.

My solution is simple: just promote _xcsbuildd user to be a "normal" user. This has another advantage: if you want you can login with that user and see the tests run in the simulator during integration, so it is easier to debug!

Here is how I did it:

sudo dscl . -create /Users/_xcsbuildd UserShell /bin/bash
sudo dscl . -create /Users/_xcsbuildd FirstName Xcode
sudo dscl . -create /Users/_xcsbuildd LastName Server
sudo dscl . -create /Users/_xcsbuildd FullName "Xcode Server"
sudo dscl . -create /Users/_xcsbuildd PrimaryGroupID 20

Then change the password with:

sudo dscl . -passwd /Users/_xcsbuildd

I was not able to make the user show up in the fast login window, anyway you should see a "other user" where you can insert the username "_xcsbuildd" and the password you chose

Nonfeasance

Answer 4

I ran into this issue as well. I have written a UI test to capture screenshots with Snapshot (fastlane) and it runs fine on an iPad Air & Pro simulator. However, on an iPad Retina or iPad 2 simulator I get this error message, either running from command line or directly from Xcode.

A solution in my case was to add some sleep() functions between the typeText() statements and the error went away.

Edit: this only fixes running manually from Xcode, through command line it still causes a test failure. I also noticed that it works on all simulators for 64-bit devices, but not on earlier devices.

Edit 2: I found a way to work around this issue by using the iOS 9.0 simulator instead of iOS 9.3, as mentioned in this answer: https://mcmap.net/q/663131/-ui-testing-failure-neither-element-nor-any-descendant-has-keyboard-focus-on-textview. This seems a good workaround until it gets fixed in a new version of Xcode.

Vasos

Answer 5

Other than connect hardware keyboard, none of these things really worked for me. What did work was turning off the Mac's screen saver and display sleep in system settings. My suspicion is the simulator performs oddly when the Mac is locked or doesn't have a monitor to draw to.

With those two settings off, I still get a few random UI test failures. Connecting hardware monitors or screen sharing so the simulator is drawn on screen somewhere seems to resolve those though.

Mouton

Answer 6

I am using xcode 8.2.1 and running tests on ios 9.3 versions. One simple hack is to add a sleep for 2-5 secs after tapping on the text box and before typing on it. Though, this is not a permanent solution.

ANOTHER RELIABLE SOLUTION

Unselect all keyboard preferences in settings before running the tests.

"KeyboardAllowPaddle": false,
"KeyboardAssistant": false,
"KeyboardAutocapitalization": false,
"KeyboardAutocorrection": false,
"KeyboardCapsLock": false,
"KeyboardCheckSpelling": false,
"KeyboardPeriodShortcut": false,
"KeyboardPrediction": false,
"KeyboardShowPredictionBar": false

Gradey