how to obtain mouse click coordinates outside my window in Java

Question

I need to implement a class, using Swing, which can obtain the mouse coordinates when the user clicks anywhere on the screen. if I wanted to obtain the mouse coordinates inside my own window, I'd use a MouseListener, but I want it to work even when the user clicks outside my program.

I want my class to behave just like KColorChooser: the user clicks on the drop button and he can click anywhere on the screen to obtain the color of that spot. but I don't know if that's possible using pure Java.

Confocal

Answer 1

It is possible though limited:

Add an AWTEventListener for focus events. As long as your app has focus before the button is clicked you'll receive a focus lost event. Then query for the pointer position.

The limitation is that, of course, your app loses focus. So depending on what you are ultimately trying to achieve this might not be useful.

If you don't want to lose focus then you will have to temporarily take a screenshot of the whole screen and display that in a screen filling window which listens for a mouse click as usual.

Proof of first method:

import java.awt.AWTEvent;
import java.awt.MouseInfo;
import java.awt.Toolkit;
import java.awt.event.AWTEventListener;

import javax.swing.JFrame;

public class Application1 {
    public static void main(String[] args) {
        Toolkit.getDefaultToolkit().addAWTEventListener(
          new Listener(), AWTEvent.MOUSE_EVENT_MASK | AWTEvent.FOCUS_EVENT_MASK);
        JFrame frame = new JFrame();
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);
    }

    private static class Listener implements AWTEventListener {
        public void eventDispatched(AWTEvent event) {
            System.out.print(MouseInfo.getPointerInfo().getLocation() + " | ");
            System.out.println(event);
        }
    }
}

Clicking outside of the app produced:

java.awt.Point[x=198,y=59] | java.awt.event.MouseEvent[MOUSE_EXITED, ...
java.awt.Point[x=976,y=503] | java.awt.FocusEvent[FOCUS_LOST, ...

The second point is outside of the app.

Irinairis

This is actually pretty clever, but it will of course only report the first click outside the application, the one that actually causes the loss of focus. After that, no other click will be reported unless the application regains focus again. Now I wonder if it's possible to react on a FOCUS_LOST event with focus request...?! – Niles

I don't think so, the focus request or window.toFront() requests have no guarantee of affecting things outside of the VM. Testing on OSX I see this doesn't work. – Irinairis

Another downside is that you won't be able to distinguish focus losses that are due to clicks outside the window and other types, such as e.g., CTRL-Tabbing to another running application. – Niles

I have adapted this solution to my self and now it does what I need :-) – Confocal

Answer 2

Forget about GlassPane, there's another 100% native Java way to do it that works both on OS X and on Windows.

Java has always supported translucency for its windows on OS X and Java now supports translucency for its windows on Windows too (since Java 1.6.0_10 or so, needs to be checked).

So the trick is: upon clicking on the "pick a color" tool, you create a nearly transparent borderless Java window covering the entire screen. You set its alpha to 10 (alpha goes from 0 to 255). That alpha is so low the user won't notice that there's a very thin "nearly transparent but only very very very translucent" borderless window covering the entire screen.

Now when the user clicks on your "alpha set to 10 translucent borderless window" covering the entire screen, you get your (x,y).

Discard the borderless Java window.

Use Robot's getRgb(x,y) and you're done.

Why set the alpha to 10 and not 0? Because otherwise clicks aren't intercepted by Java but go directly to the OS (at least that's how it works for a fact on OS X). There's a treshold and I know it's not set at '1', nor '2', it's around 10 or so.

EDIT I just realized you know need to pick several colors, this is trickier but can still be done using 100% Java. Either you can live with "slightly off" colors (affected by the "nearly transparent" 'invisible' layer) or upon getting a click you must remove the layer, get the correct pixel color, and put again a "nearly transparent" layer. Now of course that is one heck of a hack but it can be done in 100% Java.

Semipalmate

it's very desirable that the class can be executed on Linux too. – Confocal

Answer 3

Use

import java.awt.MouseInfo;
import java.awt.Point;
import java.awt.PointerInfo;

PointerInfo inf = MouseInfo.getPointerInfo();
Point p = inf.getLocation();

p.x and p.y will give you co-ordinates outside your window.

Gonzales

This is absolutely perfect! – Mccammon

Yeah but it doesn't show you when a click occurred. – Plasmolysis

Answer 4

I don't know if that's possible using pure Java.

Its not possible using pure Java, since Java is only aware of MouseEvents on Windows belonging to Java.

Gumption

It is possible (see my answer). – Plasmolysis

Answer 5

These events are directed to the window which has the focus, from all events on the desktop you can only get the mouse position.

As already shown by Keilly it's only possible to get the mouse postion.

You need to include a native lib

Cachucha

With permision of Jotschi (the original author of the Keyboard/Mouse Hook), I took his library years ago and created a new version. Please find the latest release and examples on GitHub. As well as a description of the library on the blog post. – Inconsumable

Answer 6

I haven't tried this myself, but maybe you could create a full-screen, transparent panel/frame/etc, and add a MouseListener to that.

Gonzales

@Chris: it is possible but there are gotchas (see my answer) because not every version of Windows support fully transparent JFrame etc. Then the other issue is that on OS X a really "fully transparent" JFrame doesn't intercept the click. – Semipalmate

Answer 7

It is possible with a little trick. Should be 100% cross-platform (tested on Linux & Windows). Basically, you create a small JWindow, make it "alwaysOnTop" and move it around with the mouse using a timer.

For details, see my answer here.

Plasmolysis

Answer 8

The location (x,y) and the time interval (d) between each click is supplied thru command line arguments. Here is the program

import java.awt.* ;
import java.util.* ;

public final class ClickMouse extends TimerTask {
    public static int x, y, d ;

    public static void main(String[] args) {
        TimerTask clikMouse = new ClickMouse();
        Timer t = new Timer();
/*  
    x = Integer.parseInt(args[0]) ;
    y = Integer.parseInt(args[1]) ;
    d = Integer.parseInt(ares[2]) ;
*/
        x = 500;
        y = 200;
        d = 5;
        t.schedule(clikMouse,1000,d*1000);
    }

    public void run() {
        try 
        {
            Robot bot = new Robot();

            bot.mouseMove(x,y);
            bot.mousePress(java.awt.event.InputEvent.BUTTON1_MASK );
            bot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);
        }
        catch (Exception e)
        {
            System.out.println("Exception occured :" + e.getMessage());
        }
    }
}

Vane

Answer 9

Look, I understand I am 7 years late...

This is a re-make of Keilly's answer, which allows to get when the mouse button is clicked, anywhere. The main problem is that fullscreen games are always unfocused, and it becomes annoying to handle.

Here is the code:

import java.awt.AWTEvent;
import java.awt.MouseInfo;
import java.awt.Point;
import java.awt.Toolkit;
import java.awt.event.AWTEventListener;

import javax.swing.JFrame;

public class Main {

    public static JFrame frame = new JFrame();

    public static void main(String[] args) {
        Toolkit.getDefaultToolkit().addAWTEventListener(
          new Listener(), AWTEvent.MOUSE_EVENT_MASK | AWTEvent.FOCUS_EVENT_MASK);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);
        frame.setAlwaysOnTop(true);
        frame.setLocation(1, 1);
    }

    private static class Listener implements AWTEventListener {
        public void eventDispatched(AWTEvent event) {

            // We do not want the event to show twice,
            // as it shows for focusing and unfocusing

            if(event.getID() == 1004) {
                Point p = MouseInfo.getPointerInfo().getLocation();
                System.out.println("Mouse Clicked at " + p.x + ", " + p.y);
            }

            // The frame was just unfocused! To make
            // sure we get the next mouse click, we
            // need to focus it again!

            frame.setVisible(true);

        }
    }
}

Wesle

Answer 10

https://github.com/kwhat/jnativehook JNativeHook: Global keyboard and mouse listeners for Java.

Pinafore

Answer 11

I don't have enough rep yet to leave comments, but here are my comments on the other techniques:

• Use a native lib: will work, but has obvious distribution limitations

• Use GlassPane to fill entire screen: GlassPanes must be contained within a Window.

• Create a Window containing a picture of the desktop and fill the entire screen: Will work, but it will suddenly make the desktop static. The cursor will no longer change, any animations or video in other windows or desktop will become eerily static.

Alternative solution: A refinement of the screen filling window, if you are using Java 6u10 or later is to make the window completely transparent. Put this window in front of all others and listen for mouse clicks. It still has shortcomings, such as no cursor changes, but it depends on what you want to do.

Irinairis

Looks like WizardOfOdds has the same idea about translucent windows. Just remember that the implementation of them varies by platform each with various limitations, as he demonstrates on OSX, and in fact is not guaranteed to work on all platforms. Still, looks like we've found a few different techniques to try. – Irinairis

Answer 12

Based on SyntaxT3rr0r's answer I created a sample color picker in groovy which shows how it can work.

import java.awt.*
import java.awt.datatransfer.*
//import com.sun.awt.AWTUtilities;
import javax.swing.WindowConstants as WC;
import javax.swing.SwingConstants as SWC
import groovy.swing.SwingBuilder

class ColorPicker {

    SwingBuilder swb = new SwingBuilder()
    def window;
    def overlayWindow
    def mainPanel;
    def mainLabel;
    def menu;
    def transparent = new Color(0, 0, 0, 0);
    def nearlyTransparent = new Color(0, 0, 0, 26);

    Color color = new Color(150, 150, 255);
    def colorHex = { col ->
        col = col?: color;
        "#"+Integer.toHexString(col.getRGB())[2..-1]
    }
    def getTextColor = { baseColor ->
        baseColor = baseColor?: color;
        (baseColor.red*1.5 + baseColor.green*1.5 + baseColor.blue > 400) ? Color.BLACK : Color.WHITE;
    }
    def setDisplayColor = {newColor ->
        mainPanel.background = newColor
        mainLabel.foreground = getTextColor(newColor)
        mainLabel.text = colorHex(newColor)
    }

    def show(){
        menu = swb.popupMenu { // invoker: mainPanel
            menuItem(text: "Pick Color", actionPerformed: capturePixelColor)
            menuItem(text: "Copy to Clipboard", actionPerformed: {
                Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
                clipboard.setContents(new StringSelection(colorHex()), null);
            })
            separator()
            menuItem(text: "Close", actionPerformed: {dispose()})
        }
        window = swb.frame(
            title: "Color Picker",
            location:[50,50],
            size:[60, 60],
            resizable: false,
            undecorated: true,
            alwaysOnTop: true,
            defaultCloseOperation:WC.EXIT_ON_CLOSE
        ){
            def textColor = getTextColor()
            mainPanel = panel( constraints: BorderLayout.CENTER,
                    border: lineBorder(color: Color.BLACK),
                    componentPopupMenu: menu){
                borderLayout()
                mainLabel = label(text: "--",
                    constraints: BorderLayout.CENTER,
                    horizontalAlignment: SWC.CENTER)
            }
        }
        setDisplayColor(color);
        window.show();
    }

    def capturePixelColor = {
        def screenSize = Toolkit.getDefaultToolkit().screenSize
        overlayWindow = swb.frame(
            location:[0,0],
            size: screenSize,
            resizable: false,
            undecorated: true,
            alwaysOnTop: true,
            defaultCloseOperation:WC.DISPOSE_ON_CLOSE,
            show: true,
            background: nearlyTransparent, // AWTUtilities.setWindowOpacity(overlayWindow, 0.1f);
            cursor: Cursor.CROSSHAIR_CURSOR,
            mouseClicked: {event -> 
                int x = event.getXOnScreen() // or maybe getX() is enough
                int y = event.getYOnScreen()
                overlayWindow.dispose()
                overlayWindow = null
                color = new Robot().getPixelColor(x, y)
                setDisplayColor(color)
            }
        )
    }

    public static void main(String...args){
        println "Welcome to ColorPicker"
        def picker = new ColorPicker()
        picker.show()
    }
}

Brookins