How can my XCode UI test detect that the screen has changed?

Question

I'm trying to create a tree walker using XCode 7's XCTest UI testing functionality that systematically explores a deterministic tree of table views to a specified depth. It almost works, except that I cannot reliably detect whether or not the tap on any given element has actually triggered a transition to a new screen. I have a hacky method that works most of the time, which is to detect that either the nav bar label has changed OR that the number of menu elements has changed.

The first test has false negatives, because successive screens can have identical nav bar labels (the codebase I'm walking is not mine). The second test has false positives, because sometimes clicking on a table element does not transition to another table/screen but instead adds extra elements to the current screen.

After doing some reading, it seemed that using accessibility labels might be the way to go. So I set a UID for the nav bar's accessibility label in the application code (in viewDidAppear) and then test for it in the UI test code. It feels like this ought to work, but I only ever get back a value of nil in the test code.

I will freely confess that I am a noob when it comes to UI testing and am mainly cutting/pasting/adapting other people's code without having a clear understanding of what I'm doing. So I'm probably making some kind of naive blunder, in the accessibility label code itself and possibly at the conceptual level for detecting that the screen has changed; maybe there's something much simpler/more idiomatic I could be doing.

In the application:

- (void)viewDidAppear: (BOOL)animated {

    [super viewDidAppear: animated];

    //...

    UINavigationBar* navBar = self.navigationController.navigationBar;
    if( navBar )
    {
        static NSInteger s_UID = 0;
        navBar.topItem.accessibilityLabel = [NSString stringWithFormat:@"UID-%ld", s_UID++];
    }
}

In the XCTest UI test:

- (NSString*) navBarAccessibilityLabel: (XCUIApplication*) app
{
    NSString* result = NULL;

    XCUIElementQuery *navBars = app.navigationBars;
    XCUIElement* firstElem = [navBars.staticTexts elementBoundByIndex:0];
    if( firstElem )
    {
        result = (NSString*)firstElem.accessibilityLabel; // This is always nil
    }

    return result;
}

Note that firstElem IS found, and that I am able to extract e.g. firstElem.label from it very happily.

Any help appreciated.

Flanch

did u get any solution? @Flanch ? – Sacring

Answer 1

You can test if elements are accessible:

Left click inside a UI test and start recording, the simulator will start and you can click on the element inside the simulator, if the element is accessible Xcode will write test code for you inside the testExample method.

Or run your app in the Simulator and open the Accessibility Inspector, and hover over the element as details will appear in the Inspector.

Apart from that, I think in your case you could verify this by adding a navigation item title for each screen i.e. self.navigationItem.title = @"myScreen";

And then verify the screen with an assertion i.e.

// let app = XCUIApplication() // Swift

UIApplication *app = [UIApplication alloc] init]; // Objective-C

XCTAssertEqual(app.navigationBars.element.identifier, "myScreen") // Don't forget to import class XCTest

Broadbent

This is definitely useful. Thanks. However, so far as I can tell, this alters the actual visible title onscreen. I was hoping that some kind of invisible tag of some kind could be added that would not mutate the observable values at all. (Well, I realize that ultimately all accessibility data is supposed to be observable in SOME form, even if it's audio.) – Flanch

Answer 2

Assuming you have given your view controllers proper identifiers (so you can distinguish among them), you can add this to the view controllers involved:

#ifdef UITESTING
- (void) viewDidAppear: animated
{
  [super viewDidAppear: animated];
  // do your checking here
}
#endif

In your project's Test target, under the build settings there is a section Proprocessor. There you can set Preprocessor macros. Just add 'UITESTING=1' to the Debug and/or Release setting. The effect is that in your test build UITESTING will be defined, so all code under #ifdef UITESTING will be included by the preprocessor.

Trey

To clarify, is this code that will go in the UI test program, or in the main application code? – Flanch

This would be an addition to your application code in order to support testing. I'm guessing some proper #ifdef could keep it out of your production code. – Trey

Okay...I feel like I'm missing something. What kind of logic is intended to go into "// do your checking here"? Why would I be checking for something in the application code? I had assumed that the application code would need to apply some kind of identifier label to something, and that the UI test code would leverage that later to detect a screen change. I can't immediately see the usefulness of what you are proposing in solving this problem (or in offering a solution to the more general problem of detecting a screen change). Keeping this logic out of production code is not a major concern. – Flanch

You formulated your problem as 'I cannot reliably detect whether or not the tap on any given element has actually triggered a transition to a new screen'. So in short my answer is: use the viewDidAppear callback to get notified anytime a new screen has been opened. That is the exact answer to the question. – Trey

And how is this information communicated across to the XCTest UI testing process? (Just to check, are you familiar with using XCTest?) – Flanch

And my problem was not formulated as you suggest, but rather as 'My XCode UI test cannot reliably detect whether or not the tap on any given element has actually triggered a transition to a new screen'. An XCode UI test runs in its own process, with access to a subset of the main app's state. – Flanch

Answer 3

I would use snapshot testing using Facebook's Snapshot test library.

View-based testing means verifying that what the user sees is what you want the user to see. Doing this means being able to ensure that different versions of your views, or different states of your views, continue to look the same. View-based testing can be used to provide a high-level test covering a lot of use cases surrounding an object.

You can find a great article with more detail here.

Shy