Is it possible to use Xcode UI Testing on an app using SpriteKit?

Question

I want to use UI testing for a game using SKSpriteKit.

As my first tries did not work I wonder if it possible to use Xcode UI Testing with SpriteKit.

Iridectomy

I wonder if it would be possible to UI test the SpriteKit nodes by making UIAccessibilityElement objects that relate to each node. – Firry

@ChrisLivdahl Yes, that is an interesting idea. On Session 406 of WWDC2015, UI Testing in Xcode stated: "Layers, sprites and other graphic objects" are no accessible by default. So sprites are not visible to UIAccessibility by default. What I have to figure is how to add sprites visible to UIAccessibility. – Iridectomy

@ChrisLivdahl I followed your idea and succeeded in accessing SKSpriteNode on UI tests. I'll give the answer in the incoming days. – Iridectomy

Answer 1

The main idea is to create the accessibility material for elements that you want to UI test. That's mean:

1. List all accessible elements contained in the scene

2. Configure settings for each of these elements, especially framedata.

Step by Step

This answer is for Swift 3 and is mainly based on Accessibility (Voice Over) with Sprite Kit

Let's say I want to make the SpriteKit button named tapMe accessible.

List of accessible elements.

Add an array of UIAccessibilityElementto the Scene.

 var accessibleElements: [UIAccessibilityElement] = []

Scene's cycle life

I need to update two methods: didMove(to:)and willMove(from:).

override func didMove(to view: SKView) {
    isAccessibilityElement       = false
    tapMe.isAccessibilityElement = true
}

As scene is the accessibility controller, documentation stated it must return False to isAccessibilityElement.

And:

override func willMove(from view: SKView) {
    accessibleElements.removeAll()
}

Override UIAccessibilityContainer methods

3 methods are involved: accessibilityElementCount(), accessibilityElement(at index:) and index(ofAccessibilityElement. Please allow me to introduce an initAccessibility() method I'll describe later.

override func accessibilityElementCount() -> Int {
    initAccessibility()
    return accessibleElements.count
}

override func accessibilityElement(at index: Int) -> Any? {

    initAccessibility()
    if (index < accessibleElements.count) {
        return accessibleElements[index]
    } else {
        return nil
    }
}

override func index(ofAccessibilityElement element: Any) -> Int {
    initAccessibility()
    return accessibleElements.index(of: element as! UIAccessibilityElement)!
}

Initialize accessibility for the Scene

func initAccessibility() {

    if accessibleElements.count == 0 {

        // 1.
        let elementForTapMe   = UIAccessibilityElement(accessibilityContainer: self.view!)

        // 2.
        var frameForTapMe = tapMe.frame

        // From Scene to View
        frameForTapMe.origin = (view?.convert(frameForTapMe.origin, from: self))!

        // Don't forget origins are different for SpriteKit and UIKit:
        // - SpriteKit is bottom/left
        // - UIKit is top/left
        //              y
        //  ┌────┐       ▲
        //  │    │       │   x
        //  ◉────┘       └──▶
        //
        //                   x
        //  ◉────┐       ┌──▶
        //  │    │       │
        //  └────┘     y ▼
        //
        // Thus before the following conversion, origin value indicate the bottom/left edge of the frame.
        // We then need to move it to top/left by retrieving the height of the frame.
        //


        frameForTapMe.origin.y = frameForTapMe.origin.y - frameForTapMe.size.height


        // 3.
        elementForTapMe.accessibilityLabel   = "tap Me"
        elementForTapMe.accessibilityFrame   = frameForTapMe
        elementForTapMe.accessibilityTraits  = UIAccessibilityTraitButton

        // 4.
        accessibleElements.append(elementForTapMe)

    }
}

1. Create UIAccessibilityElement for tapMe
2. Compute frame data on device's coordinates. Don't forget that frame's origin is the top/left corner for UIKit
3. Set data for UIAccessibilityElement
4. Add this UIAccessibilityElement to list of all accessible elements in scene.

Now tapMe is accessible from UI testing perspective.

References

• Session 406, UI Testing in Xcode, WWDC 2015

• eyes off eyes on — Voiceover accessibility in SpriteKit

• How do I support VoiceOver in a SpriteKit game? | Apple Developer Forums

• swift - Accessibility (Voice Over) with Sprite Kit - Stack Overflow

Iridectomy

This is awesome! Very nice write-up as well! – Firry

@ChrisLivdahl Thank you! – Iridectomy

I was struggling to get this to work until I realized that the accessibility frame of my scene was incorrect. Even though you set the scene's isAccessibilityElement to false, you still need to make sure that the accessibilityFrame of your scene matches the bounds of the view. – Kenwee

Answer 2

According to Apple developer forum discussion, Integrate UITest with SpriteKit is not currently possible:

This is likely not currently possible, but it probably could be

Update 2017-02-19

According to comment by @ChrisLivdahl this may be achieved by using UIAccessibility — Session 406, UI Testing in Xcode, WWDC 2015.

The idea is to make the element needed UI Testable.

Iridectomy

For completeness can you add a link to this forum discussion? - Never mind - found it! Will make the edit myself... – Beauvais

Answer 3

Update early 2021

A much shorter solution tested with SpriteKit, SwiftUI App and Swift 5.4

The earlier approaches didn't seem to work anymore when using XCUI Tests. The basis of my app is a SwiftUI app that has a SKScene as its main view. In order for it to work it was at the end quite simple actually and required much less steps for me to work.

1. Deactivate accessibility of the scene by adding only one line to the didMove() method

override func didMove(to view: SKView) {
    isAccessibilityElement = false
}

As mentioned in Dominique Vial's answer

2. Conforming your nodes to UIAccessibilityIdentification protocol

class YourNode: SKSpriteNode, UIAccessibilityIdentification {
    var accessibilityIdentifier: String?
//...
}

Mentioned here

Any node that needs to be accessible by the UI Test needs to conform to this protocol. Update Extension, instead of subclassing every node I'm using now the extension below.

3. Assign the accessibilityIdentifier and activate accessibility of your nodes objects.

let yourNode = YourNode()
yourNode.isAccessibilityElement = true
yourNode.accessibilityIdentifier = "nodeID"

4. That's it! Run your tests!

func testingNodes() throws {
   app = XCUIApplication()
   app.launch()
   let node = app.otherElements["nodeID"]
   XCTAssert(node.waitForExistence(timeout: 1))
}

5. Optional: set accessibilityTraits

yourNode.accessibilityTraits = [.button, .updatesFrequently]


let nodeAsButton = app.buttons["nodeID"]

You can set specific trait like .button to tell the accessibility API what your node is. This is useful to differentiate your nodes better during testing but also if you're planning to implement actual accessibility features for the user than this should be set correctly for it to work.

Update 1 - Using Extensions:

Instead of subclassing the nodes I'm using now the following extension on SKNode. I now only set the accessibilityLabel an no longer the accessibilityIdentifier

 extension SKNode: UIAccessibilityIdentification {
    public var accessibilityIdentifier: String? {
        get {
            super.accessibilityLabel
        }
        set(accessibilityIdentifier) {
            super.accessibilityLabel = accessibilityIdentifier
        }
    }
}

Update 2 - Making all descendants of SKScene accessible

To allow for nodes and all of their descendants to be accessible I've ended up using the following extension on SKNode. This adds each node to the scene's accessibilityElements.

extension SKNode: UIAccessibilityIdentification {
    public var accessibilityIdentifier: String? {
        get {
            super.accessibilityLabel
        }
        set(accessibilityIdentifier) {
            super.accessibilityLabel = accessibilityIdentifier
        }
    }

    func makeUITestAccessible(label: String, traits: UIAccessibilityTraits) {

        accessibilityLabel = label
        isAccessibilityElement = true
        accessibilityTraits = traits

        if let scene = scene {
            if scene.accessibilityElements == nil {
                scene.accessibilityElements = [self]
            } else {
                scene.accessibilityElements?.append(self)
            }
        }
    }
}

Epner

Answer 4

I've implemented an example project based on your (@Domsware's) awesome answer, and I've confirmed this trick works well for both Xcode UI Testing Framework and KIF.

Hope this example helps for anyone who is interested in this topic :)

Dover