Running UI automation tests on build server

Question

We use UI Automation and Nunit to create tests UI tests for WPF application. We've created tests that work fine when you run them from a local machine. Those tests never run successfully on our build server (using TeamCity). Build always hang after opening application window. But if I am logged in (remote desktop), on our build server all UI Automation tests also run successfully. So I am guessing that it probably has something to do with running active windows session. Any ideas how to convince our build server to create active windows session or any other solutions for making those tests run on build server?

Maudiemaudlin

Answer 1

You don't have many options. I will list the two I know, the most preferred option first:

• Set up a virtual machine on your build server. Your builds execute in the virtual machine. You can lock the host (aka your buildserver) keeping things secure.
• Keep someone logged on all the time. This offcourse creates a security problem. You can alleviate this problem a little by removing the mouse, keyboard and the screen and only access the buildserver through RDP or something similar.

---

Edit

Take a look at this TestComplete FAQ item: Can TestComplete execute scripts when the computer is locked?

Ibnsaud

We execute all builds in the virtual machine on our build server. And all tests runs under local account (not system). It didn't solve the issue. Keeping someone logged in all the time is not an option. As I said we have 3 virtual machines, where this build can run. So each time build chooses the fastest available vm. Local user on vm is normally logged in, but we can't make sure it isn't locked. And we would really like to automate this process as much as it possible, so manual log in is the least attractive option. – Maudiemaudlin

@andreja, I have update the reply with a link to Testcomplete's Q&A. I doubt there is any other way around this. You have to keep someone logged on. I don't see where this poses a security problem when your VM is "unlocked" and the host is "locked" (but I'm not a system administrator). – Ibnsaud

@andreja: If you would like to automate the login, you can set a few registry key values, and your build user will be logged in after the build VM booted. If you would like, I can look up the according reg keys. – Paean

After running it for 2 weeks we find out that only keep someone logged in works, so thank you. – Maudiemaudlin

Answer 2

OK, I'm just guessing here.

Try and run the TeamCity service with a local build server user instead of the system account. Maybe you have to login with that account once, before starting a new build.

Paean

We already tried that. When I said if I am logged in (remote desktop), on our build server all UI Automation tests also run successfully, I have been logged in as that user and all tests are running under that user. Problem is, the second when this user is locked test failed. They also failed if I run them on local machine and lock the computer while running. – Maudiemaudlin

Answer 3

It definatley sounds like you need to run your tests with an interactive session as opposed to a service. Adding the "Allow Service to interact with desktop" might help, but this is not supported in Vista any more apparently.

If you can run your builds interactivley as a command line, not a serivice that should work too.

We used to run our UIAutomation tests using the visual studo 2008 load agent to distribute them, running as a command line tool on VM's with no problem.

I also agree that you probably should't be running UI tests on a build server a part of your daily build.

Mona

Thank you but as you mention services are no longer allowed to interact with the user's desktop in Vista and Windows Server 2008. We use Windows server 2008. Running as console user also locks the desktop, and builds fail. – Maudiemaudlin

Answer 4

Build always hang after opening application window.

Tests that instantiate the UI? That's not going to work, e.g. if you get a modal dialog the build will hang. This is the reason the MVP pattern was invented, to isolate the active presentation code from a concrete view.

Are you using a mock view in your automated tests?

Yt

I don't belive in is not going to work :) We are not using mock view. – Maudiemaudlin

Although designing your UI code to be testable solves a multitude of problems, there are still certain scenarios where you want to test things that simply require an interactive environment for the test to be meaningful. (Not necessarily unit tests, but you want to automate your integration tests too, right?) – Rainout

Sometimes you just want to test that clicking a button opens the appropriate dialog. (i.e. you WANT to test the UI.) – Dextran