Driving a Windows GUI program from a script

G

6

26

I have to use a Windows simulation package to perform a repetitive task with slightly different options each time.

Since I hate repetitive clicking, on grounds of both laziness and the amount of errors that a human introduces, I would like to drive this program automatically. The program in question doesn't support scripting, there is no API, no COM, nada, nyet, nravin. As far as I can tell, the only way to drive this program automatically is to imitate a human (i.e. keyboard and mouse macros.)

I am aware of AutoHotKey but I don't think it does what I want. (Or it might do what I want, but its scripting language is horrible.)

The requirements are:

Must allow time delays between actions, or event detection to trigger actions.

The simulations can take up to ten minutes to run, so the GUI driver would have to wait until the simulation finishes before starting a new one.

One way to do this would be to just wait ten minutes and hope that the simulation has finished. An alternative way is to make it event-driven, i.e. watch for the "Simulation running..." dialog to disappear and be replaced by a "Simulation complete" dialog.
Must allow composition of complex keyboard input.

Some of the keyboard input required is different for each simulation run. For example the simulation description might take the format [Project name][Scenario name][Option 1][Option 2]... and this would have to be entered for each simulation.

I am aware that AutoHotKey allows a basic level of input customisation, but my casual reading of the documentation makes the scripting language look like some kind of eldritch horror.
This is for work, so any solution must be free for commercial use.

I will accept any solution that fits the criteria above, but I have a strong preference for something I can drive from Python. However I would also accept automated GUI-testing tools that I could customise to do what I want - possibly a Win32 GUI equivalent of Selenium for browsers? - keyboard macro recorders that will generate custom output, or anything else that works.

Gyron answered 14/3, 2012 at 8:51 Comment(0)

C

5

Give Autohotkey another look, from you requirements it seems fit for the job.

Alternatively check UI Automation from Microsoft: http://msdn.microsoft.com/en-us/library/ms747327.aspx and also white: http://white.codeplex.com/

Collector answered 14/3, 2012 at 19:55 Comment(0)

P

11

Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API's support. You can programmatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC.

Look at Sikuli, it worked for me.

Predikant answered 14/3, 2012 at 9:53 Comment(0)

D

6

Take a look at Automa - it is written in Python. It can be used either as a standalone tool or as a Python library in your own scripts:

from automa.api import *

It allows automation of any Windows application through commands like click, press, write, etc.

Some examples of the automation scripts can be found at http://www.getautoma.com/blog/category/ui-automation-examples

Disclaimer: I'm one of Automa's developers.

Devilish answered 12/3, 2013 at 13:13 Comment(2)

Please note that this is a python library that costs money – Bogeyman 19/1, 2014 at 21:28

Disclaimer: I'm one of Automa's developers. Please note that this is a python library that costs money No thanks – Cila 23/9, 2016 at 16:31

V

6

Look at this https://pywinauto.github.io/

You can use python script itself to control your windows application.

Advantage is:

no need to learn new language/syntax
integrates easily with other existing script

Viola answered 8/1, 2017 at 3:31 Comment(1)

The correct link is now: pywinauto.readthedocs.io/en/latest – Keramics 9/7, 2021 at 22:47

C

5

Give Autohotkey another look, from you requirements it seems fit for the job.

Alternatively check UI Automation from Microsoft: http://msdn.microsoft.com/en-us/library/ms747327.aspx and also white: http://white.codeplex.com/

Collector answered 14/3, 2012 at 19:55 Comment(0)

T

1

You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.

Must allow time delays between actions.

Example to type with quarter-second pause in between each key:

import pyautogui
pyautogui.typewrite('Hello world!', interval=0.25)

Here is the example to set up a 2.5-second pause after each PyAutoGUI call:

pyautogui.PAUSE = 2.5

Must allow composition of complex keyboard input.

Checkout keyboard control functions where you can use pyautogui.typewrite to type something out. You can pass variables to allow a complex keyboard input.

Event detection to trigger actions.

You can use locate functions to visually find something on the screen and make the condition based on that within a simple loop.

Solution must be free for commercial use.

It is licensed under the BSD which allows commercial use.

Recommended topics

Hot tags