Robot.mouseMove does not work at all in Mac OS X

Question

I created java file in IntelliJ IDE, with that code

import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Robot;

public class JavaRobotExample {
    public static void main(String[] args) {
        int x = 12,
                y = 300;
        try {
            Robot robot = new Robot();
            robot.mouseMove(x, y);

            int xAct = (int)    MouseInfo.getPointerInfo().getLocation().getX(),
                    yAct = (int) MouseInfo.getPointerInfo().getLocation().getY();

            String sPred = String.format("Predicted mouse location : %, d, %, d", x, y),
                    sAct = String.format("Actual mouse location : %, d, %, d", xAct, yAct);

            System.out.println(sPred);
            System.out.println(sAct);
        } catch (AWTException e) {
            e.printStackTrace();
        }
    }
}

That example works fine in Windows 7 environment, but the same code on Mac OS 10.14 doesn't move mouse using method mouseMove (but reads position with MouseInfo class).

Also I don't receive any Exception e.g.

Did someone had earlier similar problem? Any ideas how to get that code to work in Mac OS?

Best Regards,

Suttee

You're going to have to be more specific with "it doesnt work". What does the console say when you run it?/what is it supposed to say? Also more to the point how are you running it? – Editheditha

Hi, so in console I receive only these two lines (example from Mac): "Predicted mouse location : 12, 300" and "Actual mouse location : 730, 428" - what is correct, because the second line shows mouse position (because mouse does not change position for some reason). In Windows 7 I can see the same values in both lines (because predicted mouse position is equal to actual position after mouse moved). Do I need to provide something more? Thanks for reply – Suttee

I ran your code with JDK 1.8 and the actual and predicted do not match, but my mouse cursor was moved to the correct location. The same code with JDK 11 works. – Tropine

The code example will not always work. For it to work reliably, there needs to be a SwingUtilities.invokeAndWait used to get the MouseInfo. – Tropine

Answer 1

I'm having the same exact issue too. I have a statement "robot.mouseMove(100, 100);" and the mouse doesn't move at all. I have an idea to research. I noticed a warning or informational type of message from MacOS while I was running my code. It was asking me something about giving permission for my program to control the screen or system or something. I tried to click the "allow" checkbox, but it wouldn't let me. I'll research this some more. I think it's a MacOS thing. You (and I) have to tell MacOS that it's ok for our Java program to control the mouse.

I think this is the solution . . .

In System Preferences (the "gear" icon), under Security & Privacy, click the Privacy tab toward the top, then choose Accessibility on the left. This lists all the programs that can "control your computer". I'm using STS, not IntelliJ. I see STS listed along with BetterSnapTool and KeyCastr. Both BetterSnapTool and KeyCastr are checked. STS is not checked. And, all of these are grayed out so that I can't change any of the checks. There's a Lock icon at the bottom left of the window. I clicked on the icon, and a prompt came up asking for my password (I have some level of sysadmin privileges). I put in my password, and I am now allowed to "check" STS. I "checked" STS (you will "check" IntelliJ). After "checking" STS, I clicked the Lock icon at the bottom left again. This "closed" the lock. Now when I run my program, the robot command moves my mouse.

Maryjanemaryjo

I already had IntelliJ in the list of authorized apps for "Accessibility". And I'm also having the problem where my Java app, launched within IntelliJ, cannot move the mouse. I wonder if the JVM must be the same for my app and IntelliJ. This is not currently the case. – Tropine

So indeed, each JVM that needs to run the above code must be given permission using the method described here. IntelliJ embeds a JVM, but I don't use that JVM to compile and run code. I use Gradle which launches yet another JVM. In addition, Gradle has a daemon worker which must be restart after giving the JVM permission. Note: when I say "JVM" I mean the java command. – Tropine

Ok, it's worse than my previous comment. I had java for JDK 11.0.6 in the list of permitted apps and things still did not work. To resolve the problem I had to remove IntelliJ and all java commands from the list of permitted apps (remove, not uncheck). Then I had to run my program using gradle on the command line (not in IntelliJ). Doing so gave a macOS window prompting for security permissions. I clicked "Open System Preferences" and java was there. I checked it. Now it works. This java is the same as I added earlier manually and did not work. I verified this with "Open in Finder". – Tropine

The above method also made things work when running from IntelliJ. – Tropine

I'm still having difficulties with this. Every once in a while I have to remove IntelliJ and all "java" entries from "Accessibility" under "Privacy" in System Preferences and then run gradle from the command line to provoke macOS to prompt me to allow. Then everything works after that for a while. Very strange. – Tropine

Thanks a lot for your comment! I ran my java autoclicker from iTerm2 so i had to enable iTerm in Security & Privacy / Privacy / Accessibility to make it work. – Sardonyx

It was useful for me. It was necessary to remove in security and put it again – Offside

Fortunately, I can fiddle with the Accessibility controls while my application is running. When I set the right application, the effect is immediate. So if you're not sure which application needs the permission, just launch your java application that moves the mouse, and try enabling different apps, one at a time, and watch if your mouse starts moving. Also, if you have multiple versions of Java installed, they'll all show up as "java" with no version number. – Sandstrom

Answer 2

I had the same problem on mac os. But finally I found a way to do it.

1. Create a .jar file from your source. (It can be done by simply doing Clean & Build in netbeans)

2. After that go to system preference > Privacy & Security > Accessibility > Select "Terminal" and turn it ON (as below) If "Terminal" is not visible in Accessibility you can add pressing "+" button

3.Run the .jar file via terminal. Then robot function will be execute ! Use this command to run .jar file via terminal java -jar /Users/<username>/Documents/MyApp.jar

Enjoy !

Ayurveda

Please give more information. I tried this and it did not work. What java version are you using? Do you need to restart Terminal? Which macOS do you have? I have macOS 13.2, Java 11.0.6 LTS from Oracle (because that's the LTS version that works for moving the mouse cursor in previous releases of macOS). I removed Java from accessibility to be sure it wasn't interfering (using the minus button at the bottom of Preferences. It was there because it had to be in the previous macOS version). Thanks for any more info you can provide. – Tropine

Fortunately, I can fiddle with the Accessibility controls while my application is running. When I set the right application, the effect is immediate. So if you're not sure which application needs the permission, just launch your java application that moves the mouse, and try enabling different apps, one at a time, and watch if your mouse starts moving. Also, if you have multiple versions of Java installed, they'll all show up as "java" with no version number. – Sandstrom

This solution does not work for all versions of macOS. It never worked for me in the past... but as of today... it works! Lol. I'm running macOS 14.2.1. This solution depends on the version of macOS and whether Java is launched from the command line. – Tropine

Answer 3

The answer is "you can" and "you can't". Let's start with the "can't".

Can't:

It seems macOS incorrectly detects the attempt to move the mouse as coming from IntelliJ, when in fact, it's coming from java. You can give IntelliJ all the permissions you want, but java will never be able to move the cursor when the permissions are given to IntelliJ.

IntelliJ (which is a Java application) is running a child process to start your program. The child process is the java command line and not the same java as is bundled with IntelliJ. This child process needs the permissions.

Can:

This is a bit round about.

1. Use Gradle to build your project, IntelliJ recognizes Gradle projects and interacts nicely with them
2. Remove all occurrences of IntelliJ and Java from Accessibility permissions in macOS system preferences -> Security and Privacy.
3. Be sure all Gradle daemons are stopped (or if you haven't run Gradle yet, then be sure the next step is the first time gradle runs since you last rebooted your computer)
4. Run you application from the command line using Gradle
5. macOS will prompt to give permissions, open system preferences and give java permission.
6. Run IntelliJ and run your app as a gradle project through IntelliJ, and voila!

What happens: IntelliJ, in step 6, uses the already running gradle daemon process to execute your application. This process is using java and java has permission to move the mouse cursor in macOS system preferences.

It's not pretty, but it works. If you're like me, you use gradle anyway for java projects in IntelliJ, and all you have to do is remember to run your project from the command line before running it from IntelliJ. This way, the gradle daemon (a java process) will be responsible for running the application and proper permissions will be detected by macOS.

I tested this with the following code (note a few modifications, as the OP code had some "bugs"):

import javax.swing.SwingUtilities;
import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Robot;
import java.lang.reflect.InvocationTargetException;

public class RunRobot {
    public static void main(String[] args) {
        int x = 12,
            y = 300;
        final int[] xAct = new int[1],
                    yAct = new int[1];
        try {
            Robot robot = new Robot();
            robot.mouseMove(x, y);
            robot.waitForIdle();

            SwingUtilities.invokeAndWait(()->{
                xAct[0] = (int) MouseInfo.getPointerInfo().getLocation().getX();
                yAct[0] = (int) MouseInfo.getPointerInfo().getLocation().getY();
            });

            String sPred = String.format("Predicted mouse location : %, d, %, d", x, y),
                   sAct  = String.format("Actual    mouse location : %, d, %, d", xAct[0], yAct[0]);

            System.out.println(sPred);
            System.out.println(sAct);
        } catch (InterruptedException | AWTException | InvocationTargetException e) {
            e.printStackTrace();
        }
    }
}

Tropine

I've been using this approach for two years now successfully. Note that you don't actually have to start your application with Gradle before using IntelliJ to run the application. You just need to start Gradle from the command line and this can be done with a simple command like ./gradlew tasks from your project directory. That's how I've been doing it. It's a simple little step that must be done before doing anything in IntelliJ at the start of your work day. Furthermore, if you forget, just run kill on each gradle process. – Tropine

I see now that the success of Robot.mouseMove depends on the version of the JDK. With JDK 11.0.6 it works. With JDK 11.0.14 it does not. It seems it doesn't work with JDK 1.8 either. I did indeed at the specific java binary for JDK 11.0.14 per my instructions in this answer to no avail. – Tropine

Fortunately, I can fiddle with the Accessibility controls while my application is running. When I set the right application, the effect is immediate. So if you're not sure which application needs the permission, just launch your java application that moves the mouse, and try enabling different apps, one at a time, and watch if your mouse starts moving. Also, if you have multiple versions of Java installed, they'll all show up as "java" with no version number. – Sandstrom

@MiguelMunoz, that's a good idea to have an app that's constantly moving the mouse and start fiddling with the accessibility controls. Thanks! Also, the approach I outlined here no longer works for macOS 14.2.1. I now have to add Terminal to accessbility according to another answer to this question. It's hard to keep this working over the years, but I have automated tests that mimic user interactions... Java does not allow all GUI controls to be manipulated without a mouse, so I need to move the mouse cursor. – Tropine