How to simulate keys in game with pywinauto

Question

I've been trying various things for a year. I am a at beginner's level in python. Did the first two questions in Project Euler.

I have tried a few methods to try and simulate keys in games I play. I can easily do this with autohotkey and macro keyboards/mouse. However, I would like to accomplish this through Python or C.

My guess is that win32 api is ignored in video games and I need to simulate key presses through Direct X.

Thank you in advance. Here is my latest attempt... it failed.

I do have to grab/change the handle every time I run a new instance of the game.

My simulated keys would work in the browser and notepad, just not in game. By not working, I mean there is no user input.

The following code would switch to the window but will not simulate an input from the user.

import pywinauto
import time
from pywinauto import application
app = application.Application()
app.connect_(handle = 0x14002a)
dialogs = app.windows_(handle = 0x14002a)
dlg = app.top_window_()
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")

Sundry

although automating a game is considered unethical, you can find a tutorial here : code.tutsplus.com/tutorials/… – Downatheel

Thank you. I can see how automating games is unethical and one would frown to teach me such methods. I can already automate games using tools like autohotkey, but what fascinates me is how people can create software like autohotkey. My next experiment will be using PYAHK to see if it works in games, however, that would not satisfy me. I would like to attempt and create software that can simulate a keystroke as well as autohotkey. Also thank you for that link. I worked through it a while back and learned a lot. However, the key strokes in that example gets ignored in DirectX games. – Sundry

I have just learned that the game has an anti-cheat system that is blocking sendkeys. I am thankful that I have learned to simulate keystrokes in numerous ways. Now I have to figure out how to go around this new/old problem... – Sundry

Found this just now. Maybe this will allow me to make this through C or without AHK!!! autohotkey.com/board/topic/101542-controlsend-buggy – Sundry

Answer 1

We have had a long journey my past self. Though we have much to learn here is what we discovered:

Sending Virtual Keys will be ignored if the game is running on DirectX. Use sendinput and send the scan_codes instead.

# http://www.gamespp.com/directx/directInputKeyboardScanCodes.html

import ctypes
import time

SendInput = ctypes.windll.user32.SendInput

W = 0x11
A = 0x1E
S = 0x1F
D = 0x20
Z = 0x2C
UP = 0xC8
DOWN = 0xD0
LEFT = 0xCB
RIGHT = 0xCD
ENTER = 0x1C 

# C struct redefinitions 
PUL = ctypes.POINTER(ctypes.c_ulong)
class KeyBdInput(ctypes.Structure):
    _fields_ = [("wVk", ctypes.c_ushort),
                ("wScan", ctypes.c_ushort),
                ("dwFlags", ctypes.c_ulong),
                ("time", ctypes.c_ulong),
                ("dwExtraInfo", PUL)]

class HardwareInput(ctypes.Structure):
    _fields_ = [("uMsg", ctypes.c_ulong),
                ("wParamL", ctypes.c_short),
                ("wParamH", ctypes.c_ushort)]

class MouseInput(ctypes.Structure):
    _fields_ = [("dx", ctypes.c_long),
                ("dy", ctypes.c_long),
                ("mouseData", ctypes.c_ulong),
                ("dwFlags", ctypes.c_ulong),
                ("time",ctypes.c_ulong),
                ("dwExtraInfo", PUL)]

class Input_I(ctypes.Union):
    _fields_ = [("ki", KeyBdInput),
                 ("mi", MouseInput),
                 ("hi", HardwareInput)]

class Input(ctypes.Structure):
    _fields_ = [("type", ctypes.c_ulong),
                ("ii", Input_I)]

# Actuals Functions

def pressKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008, 0, ctypes.pointer(extra) )
    x = Input( ctypes.c_ulong(1), ii_ )
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

def releaseKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008 | 0x0002, 0, 
ctypes.pointer(extra) )
    x = Input( ctypes.c_ulong(1), ii_ )
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

if __name__ == '__main__':
    pressKey(0x11)
    time.sleep(1)
    releaseKey(0x11)
    time.sleep(1)

Source: Simulate Python keypresses for controlling a game http://www.gamespp.com/directx/directInputKeyboardScanCodes.html

Thank you Sentdex for awesome tutorials on Machine Learning. I've been wanting to do this to some of my favorite games but couldn't get the keys across (due to DirectX).

Next step: Windows Driver Kit to emulate keypresses...

Sundry

Do you think I can sendkey to a background from this source? Thanks – Rawdan

Answer 2

I don't know if this is too late, but yes for some games you can simulate mouse / keyboard events by using scan codes instead of virtual keys with SendInput

However as for Maplestory specifically, it doesn't respond to both virtual keys and scan codes.

The only possible way to simulate keyboard events within this game that I know of is to use kernel drivers. One example is to use WinIo to write directly to i8042 and i8048 controllers. The command 0xD2 on 8042 is used specifically to simulate hardware level inputs. However, WinIo opens up a big security hole by exposing hardware ports directly to user space programs, so I couldn't say I recommend it.

Nimbostratus

They are progressing and things that work no longer work etc. Another trick was to spam/press_down long enough for polling. – Sundry

In case someone else is reading this, with Nexon games(Maplestory, Kartrider etc.), you need to run your code on admin level. Without admin rights, even the scan codes will be ignored. With admin rights, it works fine. – Trophoblast

@MattYoon Do you know how to simulate a press and hold for maplestory? I've used scan codes on other games besides maplestory for a press and hold, and it works. However, for maplestory specifically, the press and hold just presses the key once. – Sigil