Simulating mouse input programmatically in OS X

Question

Is it possible to simulate the actions of a mouse from a program in OS X? Specifically, the short version is that I'm trying to simulate a touchscreen using two webcams. So assuming I can get X,Y positions, can I send information to the OS as a mouse movement or click?

Edit- Or if it's particularly easy in another operating system I'd be willing to consider that.

Proselytism

Depending on your use case, there's an Instrument called "UI Recorder" that records whatever you do to the UI of your program and replays it when you want. – Darceydarci

Thanks, but what I really want to do is send any arbitrary mouse input to the operating system. – Proselytism

I did this before by running a vnc server and writing a small vnc client to send mouse events to the server. There are open source vnc servers that do that, so a nice last resort would be to read the source of one. – Michellmichella

Good idea Tom, I'll consider that. – Proselytism

You guys are life savers. I made a display wall by connecting a PowerMac to 6 projector displays arranged in a 2x3 grid comprising a single giant desktop. This works well, but with a bug: the OS won't let me programmatically resize a window taller than one row. I can resize it large with the mouse, but not with AppleScript. With this code, I can first use AppleScript to create and position a window, and then use a command-line clicky-drag program to stretch the window to the size I want. I note that there is no special drag event needed, just ordering the dispatch of these four events so as to – Pusey

Answer 1

Yes, it is possible. You can use the Quartz Event Services to simulate input events.

Assuming C, I wrote this quick example:

#include <ApplicationServices/ApplicationServices.h>
#include <unistd.h>

int main() {
    // Move to 200x200
    CGEventRef move1 = CGEventCreateMouseEvent(
        NULL, kCGEventMouseMoved,
        CGPointMake(200, 200),
        kCGMouseButtonLeft // ignored
    );
    // Move to 250x250
    CGEventRef move2 = CGEventCreateMouseEvent(
        NULL, kCGEventMouseMoved,
        CGPointMake(250, 250),
        kCGMouseButtonLeft // ignored
    );
    // Left button down at 250x250
    CGEventRef click1_down = CGEventCreateMouseEvent(
        NULL, kCGEventLeftMouseDown,
        CGPointMake(250, 250),
        kCGMouseButtonLeft
    );
    // Left button up at 250x250
    CGEventRef click1_up = CGEventCreateMouseEvent(
        NULL, kCGEventLeftMouseUp,
        CGPointMake(250, 250),
        kCGMouseButtonLeft
    );
    // Now, execute these events with an interval to make them noticeable
    CGEventPost(kCGHIDEventTap, move1);
    sleep(1);
    CGEventPost(kCGHIDEventTap, move2);
    sleep(1);
    CGEventPost(kCGHIDEventTap, click1_down);
    CGEventPost(kCGHIDEventTap, click1_up);
    // Release the events
    CFRelease(click1_up);
    CFRelease(click1_down);
    CFRelease(move2);
    CFRelease(move1);
        return 0;
}

And assuming GCC, compile with:

gcc -o program program.c -Wall -framework ApplicationServices

Enjoy the magic.

Zlatoust

@Rob: you're welcome. I hope the answer didn't come too late. – Zlatoust

The answer was great but the documentation referenced now state that These functions are still supported, but they are not recommended for new development, they forgot to mention what would be recommended, anybody know? – Trochanter

According to the documentation, these functions refers strictly to the older set of event-related functions declared in the file CGRemoteOperation.h - which aren't used here. – Zlatoust

CGEventCreateMouseEvent method does not seem to exist in recent OSX 10.10 and 10.11 SKD. Compiling from QT IDE I get the error: "no matching function for call to 'CGEventCreateMouseEvent' CGEventRef event = CGEventCreateMouseEvent(NULL, kCGEventMouseMoved, CGPointMake(x, y), 0 /*ignored*/);" – Kristlekristo

@YiannisMpourkelis: The method exists since OS X 10.4 (see my output on 10.10.5). If you use(d) Homebrew to install Qt, the problem seems to be in the Qt5 formula. – Zlatoust

Answer 2

Swift mouse move and click example:

func mouseMoveAndClick(onPoint point: CGPoint) { 
    guard let moveEvent = CGEvent(mouseEventSource: nil, mouseType: .mouseMoved, mouseCursorPosition: point, mouseButton: .left) else {
        return
    }
    guard let downEvent = CGEvent(mouseEventSource: nil, mouseType: .leftMouseDown, mouseCursorPosition: point, mouseButton: .left) else {
        return
    }
    guard let upEvent = CGEvent(mouseEventSource: nil, mouseType: .leftMouseUp, mouseCursorPosition: point, mouseButton: .left) else {
        return
    }
    moveEvent.post(tap: CGEventTapLocation.cghidEventTap)
    downEvent.post(tap: CGEventTapLocation.cghidEventTap)
    upEvent.post(tap: CGEventTapLocation.cghidEventTap)
}

Cicerone

Answer 3

If you don't want to compile things and are looking for a shell-based tool, Cliclick may be the solution.

Totally

Thank you for this recommendation, @ghoti. I have been searching for weeks for such a tool :) – Cello

Answer 4

Here is a working C program based on jweyrick's answer:

// Compile instructions:
//
// gcc -o click click.c -Wall -framework ApplicationServices

#include <ApplicationServices/ApplicationServices.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
  int x = 0, y = 0, n = 1;
  float duration = 0.1;

  if (argc < 3) {
    printf("USAGE: click X Y [N] [DURATION]\n");
    exit(1);
  }

  x = atoi(argv[1]);
  y = atoi(argv[2]);

  if (argc >= 4) {
    n = atoi(argv[3]);
  }

  if (argc >= 5) {
    duration = atof(argv[4]);
  }

  CGEventRef click_down = CGEventCreateMouseEvent(
    NULL, kCGEventLeftMouseDown,
    CGPointMake(x, y),
    kCGMouseButtonLeft
  );

  CGEventRef click_up = CGEventCreateMouseEvent(
    NULL, kCGEventLeftMouseUp,
    CGPointMake(x, y),
    kCGMouseButtonLeft
  );

  // Now, execute these events with an interval to make them noticeable
  for (int i = 0; i < n; i++) {
    CGEventPost(kCGHIDEventTap, click_down);
    sleep(duration);
    CGEventPost(kCGHIDEventTap, click_up);
    sleep(duration);
  }

  // Release the events
  CFRelease(click_down);
  CFRelease(click_up);

  return 0;
}

Hosted at https://gist.github.com/Dorian/5ae010cd70f02adf2107

Aceto

I would use usleep(duration*1000000). Otherwise you will only sleep for an integer number of seconds! – Circumferential

downvoted because it is without timeout and crashes my mac :) – Cleanly