XCUIElement - Obtain Image value

Question

I have an variable that's of type .Image and class XCUIElement. Something like this:

var image = app.descendantsMatchingType(.Image).elementAtIndex(0)

Is there a way I can extract the actual image so I can compare it to another image?

I've tried caling the value method, but it returns a string. Casting it to a UIImage always fails.

Zeus

Answer 1

I have had a conversation about this with the Apple Developer Tools evangelist recently. There is currently no way of accessing the actual image from an image view, button, etc. Similarly, there is no way to access other properties of views that might be of interest, like "isHidden" or "attributedText", etc. I was told that the engineers on the UI Testing team are interested in the use cases that people are wanting access to these properties for, so it would be very helpful -- both for them and for the other people who want this feature -- if you would file a bug report / feature request asking for it at https://bugreport.apple.com

As a tip regarding the "value" property on an XCUIElement, at least for now this appears to map to the "accessibilityValue" property of whatever view the XCUIElement is referencing. So if you set that accessibilityValue of a view you are interested in to contain some information you are interested in verifying, then this can possibly help in testing. Two things to be aware of though:

1) Even though the "value" property of an XCUIElement is of type "id", the type of the accessibilityValue property is "NSString". I don't know what would happen if you try to force some non-string value (like an image) into accessibilityValue and then try to retrieve it from the "value" property of XCUIElement, but I suspect it wouldn't work well. Partially because:

2) The accessibilityValue property of a view is actually used by Apple's VoiceOver feature for the vision impaired. When the value is set, it will be read out loud when the user taps on that element (which is why it's supposed to be a string).

I also covered the issue with not being able to access properties of view via XCUIElement in more detail here: http://www.danielhall.io/exploring-the-new-ui-testing-features-of-xcode-7

Sharecropper

This pretty much summarises the limits of the UI testing framework as of now ? Have you used it in the real world to actualy provide any value ? – Ozonize

@Ozonize Yeah, the limits / unsupported cases seem to outnumber the supported cases for me currently. I think the fundamental approach using accessibility to expose information "the way it would appear to an end user" is good in many ways, but makes things like getting access to a UIImage very difficult. As for me, I've used it to write some tests that have value, but ultimately, I have to stick with KIF or Calabash because there are too many tests that can't be written with Xcode UITesting, and I'm not sure that it's worth it to have 2 different UITesting frameworks installed and maintained. – Sharecropper

Yeah exactly. I tried to do a very simple test to check if a UIButton has selected == YES ...and soon I realised that I am unable to do it. Then I wanted to make a test to see if my ViewController has displayed the relevant view (since I can switch between a few) - again not possible. Hope Apple sort this out as the framework as you've mentioned in your article seems promising. May I ask you how exactly are you using Calabash for UI tests ? Thanks – Ozonize

Not using is currently, but last project I worked on had Calabash tests to validate 1) Navigating to every screen 2) The presence / absence of various UI elements based on whether user was logged in 3) Color and other visual changes to UI elements that happen based on the presence of certain data or as a result of user actions. – Sharecropper

To clarify: NSObject property accessibilityValue is mapped to XCUIElement property value, as NSString. Misleadingly, the more logical XCUIElement property accessibilityValue is always nil. Also, for UIImageView to be accessible, the isAccessibilityElement property needs to be set to YES. For UIImage (non- UIView) to be accessible, it needs to be reported through the UIAccessibilityContainer protocol. – Funnel

Answer 2

I know it may be not exactly what you're looking for, but I managed to write a test that checks if the visual representation of a UIImage on the screen has changed.

I'm using a screenshot() method of XCUIElement which returns an instance of XCUIScreenshot:

let myImage = XCUIApplication().images["myAccessibilityIdentifier"]
let screenshotBefore = myImage.screenshot()
//...
//do some actions that change the image being displayed
//...
let screenshotAfter = myImage.screenshot()

//Validating that the image changed as intended
XCTAssertNotEqual(screenshotBefore.pngRepresentation, screenshotAfter.pngRepresentation)

The screenshots will be the size of the image as rendered on the screen which may be different to the original image of course.

It's important to compare the PNG representations using the pngRepresentation property, but not the XCUIScreenshot objects because the two objects will always be different internally.

This technique can't test that the image displayed on the screen is exactly what is needed but at least can detect changes in the image.

Hylozoism