Accessibility test automation on Android

Question

I recently started looking into automating accessibility testing on Android. There isn't much information out there on the web. Has anyone explored this or are currently doing this? If so, can you share your ideas/approach?

Seems like Android's uiautomator relies on Accessibility features working but it doesn't support testing Accessibility. If it relies on Accessibility features, does that mean that basic validation like accessible label exists, etc. can be done by just executing UI tests using uiautomator?

This is a new area for me so any information could be helpful.

Massicot

You can use the linting tool for validation of content descriptions, see developer.android.com/tools/help/lint.html. Most accessibility problems, however, are interaction issues and will only be found during interactive testing. – Wowser

Thanks for pointing me to the linting tool but it's for finding issues in the source code. It'll definitely be useful but I am mainly looking for some form of automated testing at UI level where I don't have access to source. – Massicot

You can check the answer that I posted to your other question, I tried to cover in more detail: https://mcmap.net/q/1915622/-test-accessibility-talkback-for-application – Ceramics

Answer 1

Here's a great introduction to Accessibility testing in Android. It basically boils down to:

• Manually test your app for visual issues with Accessibility Scanner
• Turn on TalkBack and manually test your app to find hearing impaired issues
• To find font scaling and layout issues, use Large Text
• Definitely lint check, but make sure that 'Image without contentDescription' is set to Severity = Error
• Any/all accessibility issues you find or that are reoccurring, write an Espresso test to fail when that accessibility issue is violated in the future
• For automation, you'll also need to consider how to perform visual validation of certain screen artifacts and audio analysis if hearing impaired functionality is a requirement.

Also, I recommend watching this presentation from GTAC 2015 on accessibility testing for some great context on the topic.

For automated tests that check for accessibility, I'd very much recommend starting with issues that can be identified in elements that are shared across screens (menus, layouts, themes, custom controls). While they won't catch the one-off errors that will occasionally pop up, they'll address issues that happen everywhere in your app, a 'prioritize by volume' approach if you will.

Also, if your team uses Android Studio then you definitely want to push for the ability to write Espresso tests that reside with the code. QA are part of the development process, period. Getting access to a subfolder where your tests reside shouldn't be a problem unless there some legal bologna to deal with. For instance, split out the 'androidTest' folder as a submodule where you have pull/push rights as a tester, but only read rights to the rest of the app so you can compile and run yourself. If you're writing Appium tests, it may be harder to ask your dev team to run them as part of their own BVT/smoke test process during builds, but not unheard of.

As for visual analysis and audio injection/confirmation, these are advanced capabilities that you'll probably need to use some service or commercial tool for.

Best of luck!

Cuneiform

Answer 2

I agree with Paul's answer in its entirety and he links to some extremely helpful resources (so please have a look at them!), but if all you're looking for is basic accessibility test coverage as you suggest (e.g. checking for accessible labels on all your components), your use case might be a good one for something like Continuum for Mobile, specifically the Android variant. You can then do more manual passes once you've found the more basic violations that can be detected using automated tools; as of right now, manual testing is always necessary for total compliance with accessibility standards, but something like this will get you closer.

Blower