Is there Anyone who Successfully Implement UI Test Automation Regime using Microsoft UI Automation?

Question

I am looking for an Automated UI test framework/ software tool. In the past I have been using TestComplete, and although it's a good piece of software, but the concept of GUI test automation was deemed to be sufficiently difficult that I wrote a few posts to complain about it.

One of the problems with third party test automation tool is that you have to learn new language in order to be productive on it, not to mention that the tooling support is poor. I am now planning to look into Microsoft UI Automation that comes with .Net 3.0 and the White Framework. But before I do that, I want to know what's the outcome there.

Anyone has any experience to share on this? Have you create a sustainable and successful test suite using UI automation on your application?

Edit: This seems like a very hard question. I would setup bounty for this if I don't receive any answers within these few days.

Dearly

This is something I definitely would like to take a look into, but I'll be focused on testing and code reviewing lower layers of our application for the time being. Anyway, I took a look on the links you provided and then I did some research and ended up finding Microsoft UI Automation team's blog. I'm sharing it here just in case you haven't come across it yet: blogs.msdn.com/winuiautomation – Moonstruck

Answer 1

Cold Response... Better I answer a dummy answer, just in case there is no good answer to this.

Dearly

@Greg, that was because I offered a bounty for this question, but when the period ended, none of the answer was satisfactory but I was still made to chose an accept as an accepted answer. Which was the point of writing and accepting a dummy answer. – Dearly

There is no reason to accept any answer if there is no answer that is worth accepting it. Leave the question open! – Absence

Answer 2

This is an extremely late response (you probably already shipped), but I tested a WPF application with it.

I had a lot of flakiness while I was using the beta1 version of VS2010, and encountered some bugs. Using the release version, I had fewer problems.

It was challenging because I was learning WPF, VS UI automation, and WPF/Windows accessibility all at the same time. The fact that it is hard to create bullet proof automated UI tests added to this difficulty.

After the VS2010 RTM, I still had intermittent issues with a native/custom page that we wrapped. I think this was partially due to a parenting issue, and partially due to a threading/process issue.

The biggest headache I encountered was when I was trying to test against some control we used from the WPF toolkit, in particular the DataGrid control. It had some known UI automation bugs, and I had a devil of a time working around them.

I also encountered some challenges where the UI automation framework was caching data behind the scenes. I would have to re-instantiate controls in order to get updated data on them (particularly visibility, when a control was poorly parented).

If I had a pure WPF or pure Win32 app, that didn't inherit UI from other applications, wrap things in WebBrowser controls, use custom controls, etc, then I'd probably use it again. If you haven't made a decision yet, and your app uses any of those, then I'd see if I could use something else. Maybe a simpler set of scripts to do more limited integration testing, and try to cover the rest with unit tests, using mock objects.

I haven't used it within the last 6 months, though, so your mileage might vary.

Ivo

Answer 3

We use the White test framework here with great results.

The framework uses win32 messaging to find the controls and interact with them. It's fairly slow on large forms with a lot of controls however, that's the only drawback I've encountered. We automate the test running using buildbot and nunit-console too.

Bypath

Answer 4

I wrote an ironruby-based gem called Bewildr that wraps the MS UI Automation framework. It has been successfully used to automate MS WPF apps at various companies including the BBC. Here's a step-by-step guide to using it: http://www.natontesting.com/2011/08/27/step-by-step-example-of-bdding-a-wpf-app-with-cucumber-rspec-ironruby-and-bewildr/

Marque

Answer 5

I have been using MS UI Automation framework using C#.net and i found this extremely simplem and helpful.No issues were observed as such but it lacks report generating support so you need to write your own logic for that based on your specific test case.

Overall satisfaction : 8 / 10.

Lei

Answer 6

I have no experience with Microsoft UI Automation, but I'm using AutoIt (http://www.autoitscript.com/autoit3/) to do a little bit of GUI automation. I'm using it to test my Visual Studio add-in. Not really an answer to your question, but it could be interesting for you.

Regards,

Sebastiaan

Swainson

Answer 7

Ive used a lightweight version of UI automated testing for the .net framework using a similar approach to this:

http://msdn.microsoft.com/en-us/magazine/cc163864.aspx

I did encounter issues using tabs and deciding which panels were going to be used, but that eventually led to a redesign of the form (so it pretty much found issues in design too!)

Ambur