How to automate the test running process using TestComplete?

Question

We are trying to integrate tests in our daily builds using TestComplete, so far we have a machine dedicated for testing and our build script copies to this machine everything TestComplete needs for its tests (Application, Database, Test script project and source files, etc).

Basically we can open the TestComplete project manually and run the tests.

Now we want to automate that process, so how do you do it? Or how do you think would be the simplest and best way to make this automation?

Keeping it short, we want to automate the process of opening TestComplete after each build, run all the tests and send an email with the test results.

Anyone can share some experience about this?

Thanks.

Solis

Answer 1

Answering my own question:

The solution was writing a little C# application which sits on the system tray and monitors a folder.

When a new folder (containing the tests source code) are added to the monitored folder TestComplete is called using the command line, then the application catch its ExitCode and send an email with the Log file generated attached to it.

Depending on the ExitCode I know what happened in the tests, the possible ExitCodes are:

0 - The last test did not produce errors or warnings.
1 - The last test results include warnings but no errors.
2 - The last test results include errors.
3 - The test cannot be run because of an error

More information about the ExitCodes can be found on TestComplete's Help file.

Solis

Answer 2

Well, although I have not used TestComplete I have used a competing package called QA Wizard Pro. Since you are asking this question I am assuming that it isn't something that is natively supported by TestComplete. QA Wizard is the same way and they expect it to be run manually instead of automatically, though there are test run files that can be run. For QA Wizard I created a batch file that was run nightly from the task scheduler. The account to run the software must be able to interact with the desktop and a user must be logged in with a display. I used a free piece of software called AutoHotKey to automate the running of the tests and then some Cygwin tools to parse the results and trigger an email through Blat with the results. It isn't a perfect solution but it does work.

Delilahdelimit

Answer 3

You should also look at using TestExecute. This is a (much cheaper) program from Automated QA that will execute TestComplete scripts.

This will save you from having to have a full TestComplete license for your build/test server.

Euphrosyne

Answer 4

if you have TestExecute, try this. works everytime....

C:\PROGRA~1\AUTOMA~1\TESTEX~1\Bin\TestExecute.exe "path\Project.pjs" /r /e

Catharinecatharsis

Answer 5

Set wshShell = CreateObject("WScript.Shell")
wshShell.Run("""C:\Program Files\Automated QA\TestComplete 6\Bin\TestComplete.exe"" ""C:\Documents and Settings\My Documents\TestComplete 6 Projects\abc\abc.pjs(your script path)"" /r /p:(Project Name) /u:(Unit Name) /rt:(Method to be executed) /e /SilentMode")

Copy above lines in Notepad and save it as .vbs file.
Make a .bat file and put it on your integrated server.
Browse the path of above mentioned .vbs file through bat file your TestComplete exe.

For bat file you can write directly these lines in Notepad as

C:\WINDOWS\system32\cmd.exe
WScript.Echo ""
Set wshShell = CreateObject("WScript.Shell")
wshShell.Run("""C:\Program Files\Automated QA\TestComplete 6\Bin\TestComplete.exe"" ""C:\Documents and Settings\My Documents\TestComplete 6 Projects\abc\abc.pjs"" /r /p:prj1 /u:Unit1 /rt:Test1 /e") 

Save this txt file with .bat extension. Afterwards generate a task through your CI server.

Landsknecht

Answer 6

For people still looking for this: SmartBear released a plug-in of TestComplete for Jenkins. So, it can now be used without the need of hacking things in.

Information about the plug-in: https://plugins.jenkins.io/TestComplete

Press release: https://smartbear.com/news/news-releases/smartbear-simplifies-continuous-delivery/

Aerometeorograph

Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. – Eugine

Answer 7

There are different methods to do this activity. The Best and Most powerful method is using Cruise control.NET for Continuous integration of testing/Development Cycle.

Second Method is create a batch file to run Test complete script using command line parameter. Schedule the running of this batch file . Also Include one simple application (which will update test result in Excel/Test Cases) as testApp. and call this after every test case/scenario run. create a mailer function to send this result after completing the TC run.

I already using this two methods,

Burly

Answer 8

The simplest solution is to use a batch file to execute testcomplete from command line, and add it into windows scheduler.

Trachytic

Answer 9

You could try Jenkins. At its most basic you could create a project with one build step (batch or bash script) which calls Test Complete or Test Execute from the commend line at the scheduled time. You can then add additional build steps as required. For example, when our tests run we pull the latest version of Test Complete scripts from source control. Jenkins has nice features like archiving of build items (in the case of Test Complete this would be your test logs),email notifications and monitoring of source control repositories. The large plugin library covers most other things you may want to add to your project.

Awake

Answer 10

You can use TestComplete task for Bamboo to run TestComplete tests with TestComplete or TestExecute and parse tests in Bamboo and integrate them worh JIRA. https://marketplace.atlassian.com/plugins/com.mdb.plugins.testcompletetask/server/overview

Gris