AppleScript click at mouse position

Question

This code works to click on a specific location on the screen but how do you click at the cursor's coordinates?

tell application "System Events"
    click at {10, 10}
end tell

Sagittate

I'm having exactly the same problem right now. Is there a way to 'get' the mouse's coordinates? – Michaels

Currently the best I can find is this: hamsoftengineering.com/codeSharing/MouseTools/MouseTools.html – Michaels

Objec. C version option: discussions.apple.com/thread/3254587 – Unaccountable

Answer 1

below an example with python in applescript , python is living in your system naturaly,no installation to be planned.

set x to 30

set y to 5

set l to 50 -- click same location 50 times



do shell script "

/usr/bin/python <<END

import sys

import time

from Quartz.CoreGraphics import *

def mouseEvent(type, posx, posy):

          theEvent = CGEventCreateMouseEvent(None, type, (posx,posy), kCGMouseButtonLeft)

          CGEventPost(kCGHIDEventTap, theEvent)

def mousemove(posx,posy):

          mouseEvent(kCGEventMouseMoved, posx,posy);

def mouseclick(posx,posy):

          mouseEvent(kCGEventLeftMouseDown, posx,posy);

          mouseEvent(kCGEventLeftMouseUp, posx,posy);

ourEvent = CGEventCreate(None);

currentpos=CGEventGetLocation(ourEvent);             # Save current mouse position

for x in range(0, " & l & "):

          mouseclick(" & x & "," & y & ");

mousemove(int(currentpos.x),int(currentpos.y));      # Restore mouse position

END"

Arpeggio

Hello below an example with python in applescript , python is living in your system naturaly,no installation to be planned. – Arpeggio

Importing Quartz.CoreGraphics is incredibly slow. On my machine, every single scripted mouse operation that calls this from Applescript requires a 10-15 second delay for this to happen. – Nit

Hello, yes the import into python to be used in applescript takes a lot of time to execute. That's why using cliclick by Carsten Blüm is much easier in applescript. this depends on your need for scripting. f https:/github.com/BlueM/cliclick – Arpeggio

Ah, thanks, yeah, I saw that, but I'm trying to minimize dependencies, I was hoping for a native solution. – Nit

Answer 2

You can also in applescrit using command-tool cliclick

dowload https://github.com/BlueM/cliclick

unzip cliclick-master.zip

in a Terminal

cd  /Users/yourname/Downloads/cliclick-master

make

now you have in /Users/yourname/Downloads/cliclick-master/

cliclick , command tool for using a mouse in applescript with do shell script

copy cliclick in /usr/local/bin folder

in a Terminal

cp -f /Users/yourname/Downloads/cliclick-master/cliclick  /usr/local/bin/

    in applescript for example

    do shell script "/usr/local/bin/cliclick  " & quoted form of "c:12,34"

    will click at the point with x coordinate

              12 and y coordinate 34.

    in a Terminal for LIST OF COMMANDS

    cliclick -h

 LIST OF COMMANDS

  c:x,y   Will CLICK at the point with the given coordinates.
          Example: “c:12,34” will click at the point with x coordinate
          12 and y coordinate 34. Instead of x and y values, you may
          also use “.”, which means: the current position. Using “.” is
          equivalent to using relative zero values “c:+0,+0”.

  dc:x,y  Will DOUBLE-CLICK at the point with the given coordinates.
          Example: “dc:12,34” will double-click at the point with x
          coordinate 12 and y coordinate 34. Instead of x and y values,
          you may also use “.”, which means: the current position.

  dd:x,y  Will press down to START A DRAG at the given coordinates.
          Example: “dd:12,34” will press down at the point with x
          coordinate 12 and y coordinate 34. Instead of x and y values,
          you may also use “.”, which means: the current position.

  du:x,y  Will release to END A DRAG at the given coordinates.
          Example: “du:112,134” will release at the point with x
          coordinate 112 and y coordinate 134.

  kd:keys Will trigger a KEY DOWN event for a comma-separated list of
          modifier keys. Possible keys are:
          “alt”, “cmd”, “ctrl”, “fn”, “shift”
          Example: “kd:cmd,alt” will press the command key and the
          option key (and will keep them down until you release them
          with another command)

  kp:key  Will emulate PRESSING A KEY (key down + key up). Possible keys are:
          “arrow-down”, “arrow-left”, “arrow-right”, “arrow-up”, “delete”, “end”,
          “esc”, “f1”, “f2”, “f3”, “f4”, “f5”, “f6”, “f7”, “f8”, “f9”, “f10”, “f11”,
          “f12”, “f13”, “f14”, “f15”, “f16”, “fwd-delete”, “help”, “home”, “mute”,
          “page-down”, “page-up”, “return”, “space”, “tab”, “volume-down”, “volume-up”
          Example: “kp:return” will hit the return key.

  ku:keys Will trigger a KEY UP event for a comma-separated list of
          modifier keys. Possible keys are:
          “alt”, “cmd”, “ctrl”, “fn”, “shift”
          Example: “ku:cmd,ctrl” will release the command key and the
          control key (which will only have an effect if you performed
          a “key down” before)

  m:x,y   Will MOVE the mouse to the point with the given coordinates.
          Example: “m:12,34” will move the mouse to the point with
          x coordinate 12 and y coordinate 34.

  p[:str] Will PRINT the given string. If the string is “.”, the
          current MOUSE POSITION is printed. As a convenience, you can skip
          the string completely and just write “p” to get the current position.
          Example: “p:.” or “p” will print the current mouse position
          Example: “p:'Hello world'” will print “Hello world”

  tc:x,y  Will TRIPLE-CLICK at the point with the given coordinates.
          Example: “tc:12,34” will triple-click at the point with x
          coordinate 12 and y coordinate 34. Instead of x and y values,
          you may also use “.”, which means: the current position.

  t:text  Will emulate typing the given text into the frontmost application.
          If the text includes space(s), it must be enclosed in quotes.
          Example: “type:Test” will type “Test” 
          Example: “type:'Viele Grüße'” will type “Viele Grüße”

  w:ms    Will WAIT/PAUSE for the given number of milliseconds.
          Example: “w:500” will pause command execution for half a second

Arpeggio

Version 3.0.3, released 10/29/2014 Author: Carsten Blüm, <[email protected]> List of contributors: github.com/BlueM/cliclick/graphs/contributors Website: www.bluem.net/jump/cliclick/ – Arpeggio

Version 4.0.1, 10/04/2018 Author: Carsten Blüm, <[email protected]> Bugfix: in version 4.0, it was not possible to press several keys at once using the kd action. Added missing options to the built-in help (cliclick -h). – Arpeggio

Binary releases are also available. – Caducity

Answer 3

    use framework "Foundation"


    property NSEvent : a reference to current application's NSEvent
    property NSScreen : a reference to current application's NSScreen


    # Get display size
    set display to NSDeviceSize ¬
        of deviceDescription() ¬
        of item 1 ¬
        of NSScreen's screens() as record

    # Get mouse location (relative to the bottom-left of the screen)
    set mouseLocation to {x, y} of (NSEvent's mouseLocation as record)

    # Calculate mouse y-coordinate so it's relative to the top of the screen
    set mouseLocation's item 2 to (display's height) - (mouseLocation's item 2)


    tell application "System Events" to click at the mouseLocation

Charie

Not working, I'm getting "error "System Events got an error: Script Editor is not allowed assistive access." number -25211" – Durst

Then turn on assistive access, @AntonMalmygin! It’s not appropriate to downvote an answer simply because you don’t understand what’s caused the error. Ask for help, and I’ll be happy to provide. – Charie

Hmm… this is a stretch, but could the Foundation framework be used to simulate mouse click-and-drags instead of just clicks? Or is it read-only? I found Apple's doc for AppKit but as I'm not an ObjC programmer I find it all too impenetrable to apply to my AS. Are there clearer docs on it somewhere? @AntonMalmygin He answered the top post, and answered it well. You shouldn't downvote good answers to what someone else asks just because you yourself don't understand the issues discussed in the original question. If your question is broader than the top one, you should post it separately. – Nit

@MichaelKupietz It definitely used be possible to simulate mouse clicks with AS-ObjC using the Quartz Core Graphics Framework. This is no longer possible for whatever reason. The NSEvent class from the Foundation framework I don't think has had the necessary methods bridged for AppleScript use. This page offers some clues as to implementation; this answer is educational. I expect it'll be possible eventually. – Charie

Answer 4

I created a command line tool in Xcode using the Swift programming language to produce an executable:

    // Location of Mouse pointer
    var ml = NSEvent.mouseLocation
    ml.y = NSHeight(NSScreen.screens[0].frame) - ml.y
    let location = CGPoint(x: ml.x, y: ml.y)

    // Single mouse click.
    var e = CGEvent(mouseEventSource: nil, mouseType: .leftMouseDown, mouseCursorPosition: location, mouseButton: .left)!
    e.post(tap: .cghidEventTap)
    e = CGEvent(mouseEventSource: nil, mouseType: .leftMouseUp, mouseCursorPosition: location, mouseButton: .left)!
    e.post(tap: .cghidEventTap)

    e = CGEvent(mouseEventSource: nil, mouseType: .leftMouseDown, mouseCursorPosition: location, mouseButton: .left)!
    e.setIntegerValueField(.mouseEventClickState, value: 1)
    e.post(tap: .cghidEventTap)

    e = CGEvent(mouseEventSource: nil, mouseType: .leftMouseUp, mouseCursorPosition: location, mouseButton: .left)!
    e.setIntegerValueField(.mouseEventClickState, value: 1)
    e.post(tap: .cghidEventTap)

I then created a shell script in Script Editor to trigger the executable:

do shell script "/Users/bobby/Documents/AppleScripts/Mouse/Executables/Click"

Change the location based on where you saved the executable.

Sagittate

Answer 5

You should definitely grab the AppleScript toolbox. It has scripting commands to get your mouse location and mouse clicking. AppleScript Toolbox is a scripting addition/AppleScript-plugin (a.k.a. “OSAX”) providing commands not available in regular AppleScript.

Makedamakefast

Answer 6

Install cliclick with Homebrew(https://brew.sh/):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

then look at ==> Next steps:

• Run these two commands in your terminal to add Homebrew to your PATH:

Run these two commands

then

brew install cliclick

To click at the current mouse location:

cliclick c:.

allow terminal or Scrip Editor

on Click(x, y, k)
  repeat k times
    do shell script "/opt/homebrew/Cellar/cliclick/5.1/bin/cliclick c:" & x & "," & y
  end repeat
end Click

Christoforo