Java Robot class simulating human mouse movement

Question

I am working on a project about remote control, send conrdinate x and y of cursor from client to server.

But

robot.mouseMove(x,y);

will only move the cursor to the particular point without moving the cursor form origional point

I have find this simple algorthim to simulate the continuing movement of mouse

for (int i=0; i<100; i++){
   int x = ((end_x * i)/100) + (start_x*(100-i)/100);
 int y = ((end_y * i)/100) + (start_y*(100-i)/100);
 robot.mouseMove(x,y);
} 

But this algorthim still too simple, it just move from one point to other point slowly, which still unlike human behave.

I have read some open soruce code about remote control from web, and I find this project http://code.google.com/p/java-remote-control/ is using the method call MosueMovement from MouseListener class, which they use to perform the "dragging".

I like to know is any one know the better way of doing this?

Moravia

you may want to gather some data from human users and then use that in various ways to replicate human simulation. – Oneiromancy

From your post it appears that your main complaint is that the mouse movement speed is wrong. If that's the main issue, then you could adjust your code to make the mouse movement a constant speed, and then adjust that speed until it is close to optimal. – Idaho

Maybe you could [Alt]+[Tab] a couple of times and check e-mail and facebook on the way from A to B. Seriously, though, I think recording mouse movement of a couple of users could be an interesting experiment to make this more realistic. It would be cool to take into account a couple of variables, such as X, Y position, speed, time, current direction, run Rapid Miner and see what you can extract from it. Perhaps you'd be able to come up with a more sophisticated algorithm. – Bawbee

Answer 1

There are a few things to consider if you want to make the artificial movement natural, I think:

1. Human mouse movement is usually in a slight arc because the mouse hand pivots around the wrist. Also that arc is more pronounced for horizontal movements than vertical.
2. Humans tend to go in the general direction, often overshoot the target and then go back to the actual target.
3. Initial speed towards the target is quite fast (hence the aforementioned overshoot) and then a bit slower for precise targeting. However, if the cursor is close to the target initially the quick move towards it doesn't happen (and neither does the overshoot).

This is a bit complex to formulate in algorithms, though.

Multiplicity

Thanks for adive, I am working on the project to remote control from Android phone to desktop, most people done it well but looks use java to write the function is little bit difficult, I will keep work on this – Moravia

Answer 2

For anyone in the future: I developed a library for Java, that mimics human mouse movement. The noise/jaggedness in movement, sinusoidal arcs, overshooting the position a bit, etc. Plus the library is written with extension and configuration possibilities in mind, so anyone can fine tune it, if the default solution is not matching the case. Available from Maven Central now.

https://github.com/JoonasVali/NaturalMouseMotion

Shorts

Hmm, why the deletion request and downvote, if you don't mind me asking? (To whoever requested this.) – Shorts

Answer 3

Take a look in this example that I wrote. You can improve this to simulate what Joey said. I wrote it very fast and there are lots of things that can be improved (algorithm and class design). Note that I only deal with left to right movements.

import java.awt.AWTException;
import java.awt.MouseInfo;
import java.awt.Point;
import java.awt.Robot;

public class MouseMoving {

    public static void main(String[] args) {
        new MouseMoving().execute();
    }

    public void execute() {
        new Thread( new MouseMoveThread( 100, 50, 50, 10 ) ).start();
    }

    private class MouseMoveThread implements Runnable {

        private Robot robot;
        private int startX;
        private int startY;
        private int currentX;
        private int currentY;
        private int xAmount;
        private int yAmount;
        private int xAmountPerIteration;
        private int yAmountPerIteration;
        private int numberOfIterations;
        private long timeToSleep;

        public MouseMoveThread( int xAmount, int yAmount,
                int numberOfIterations, long timeToSleep ) {

            this.xAmount = xAmount;
            this.yAmount = yAmount;
            this.numberOfIterations = numberOfIterations;
            this.timeToSleep = timeToSleep;

            try {

                robot = new Robot();

                Point startLocation = MouseInfo.getPointerInfo().getLocation();
                startX = startLocation.x;
                startY = startLocation.y;

            } catch ( AWTException exc ) {
                exc.printStackTrace();
            }

        }

        @Override
        public void run() {

            currentX = startX;
            currentY = startY;

            xAmountPerIteration = xAmount / numberOfIterations;
            yAmountPerIteration = yAmount / numberOfIterations;

            while ( currentX < startX + xAmount &&
                    currentY < startY + yAmount ) {

                currentX += xAmountPerIteration;
                currentY += yAmountPerIteration;

                robot.mouseMove( currentX, currentY );

                try {
                    Thread.sleep( timeToSleep );
                } catch ( InterruptedException exc ) {
                    exc.printStackTrace();
                }

            }

        }

    }

}

Universe

Thanks for the code, I will keep work and see what I can do on it – Moravia