Getting Window Number through OSX Accessibility API

Question

I am working on an application that moves windows of third party applications around on the screen.

To get an overview of all currently open windows, I use

CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly | kCGWindowListExcludeDesktopElements, kCGNullWindowID);

This returns an array of dictionaries defining every open window. Here's an exemplary dictionary returned:

{
    kCGWindowAlpha = 1;
    kCGWindowBounds =         {
        Height = 442;
        Width = 475;
        X = 3123;
        Y = "-118";
    };
    kCGWindowIsOnscreen = 1;
    kCGWindowLayer = 0;
    kCGWindowMemoryUsage = 907184;
    kCGWindowName = Untitled;
    kCGWindowNumber = 7328;
    kCGWindowOwnerName = TextEdit;
    kCGWindowOwnerPID = 20706;
    kCGWindowSharingState = 1;
    kCGWindowStoreType = 2;
    kCGWindowWorkspace = 3;
},

The dictionary is full of good information used elsewhere but lacks an accessibility object that could be used to modify the windows' positions. Windows are clearly identified by the Window Number.

I am now using the PID (kCGWindowOwnerPID) to create an accessibility object for the window's application:

AXUIElementRef app = AXUIElementCreateApplication(pid);

Followed by retrieving a list of all windows the application has opened using AXUIElementCopyAttributeValues:

NSArray *result;

AXUIElementCopyAttributeValues(
                               (AXUIElementRef) app, 
                               kAXWindowsAttribute,
                               0,
                               99999,
                               (CFArrayRef *) &result
                               );

This works and returns an array of AXUIElements. This is where I am stuck. There seems to be no API call to retrieve the Window Number of an accessibility object. Is there any way to either

a) Find the accessibility object's Window Number (to ultimately iterate over the array and find the right window)

or

b) Otherwise clearly match a window described in the array returned by CGWindowListCopyWindowInfo to the Accessibility Objects returned by AXUIElementCopyAttributeValues?

Bodily

Answer 1

We ended up hiring a dedicated Accessibility Developer for this task.

It turns out there is no way to do this without using undocumented APIs (a no go in our case).

Luckily, there is a practical workaround:

Loop over all open windows of the app. Get their position, size and title:

AXUIElementCopyAttributeValue(target, kAXPositionAttribute, (CFTypeRef*)&posValue);
AXUIElementCopyAttributeValue(target, kAXSizeAttribute, (CFTypeRef*)&sizeValue);
AXUIElementCopyAttributeValue(target, kAXTitleAttribute, (CFTypeRef*)&titleValue);

Next, convert the position and size into actual CGPoint and CGSize values:

AXValueGetValue(posValue, kAXValueCGPointType, &point);
AXValueGetValue(sizeValue, kAXValueCGSizeType, &size);

Compare the size, position and title against the values returned by the object in CGWindowListCopyWindowInfo(). If they match, you can safely assume it's the window you were looking for and use the already open AXUIElement (target in our case) to work it.

The overhead for looping through all open windows turns out to be negligible on OSX. There is a pretty low cap on how many windows are open at the same time.

Also, while this is not 100% accurate (it is possible that 2 windows have the same position, size and title), we haven't encountered any situation in real usage where this happens so far.

Bodily

For those that can use undocumented APIs, how would it be done? – Hanni

This is the best solution I ever seem. But still has risk I think. Anyway thanks for your answer! – Muscovite

Answer 2

There is a private function for obtaining CG window number for a given AX object for window: _AXUIElementGetWindow . More details in SO discussion Uniquely identify active window on OS X It looks like there is no public API to do the task with 100% probability. Identifying windows by title and frame (as described in answer above) will works in 99.9% of cases.

Puto

I can't find _AXUIElementGetWindow on macOS 10.15 SDK – Muscovite

Answer 3

Clarifying other answers that suggest the undocumented call of _AXUIElementGetWindow.

In Swift, do this:

1. Create Bridged-Header.h with the following contents:

#import <AppKit/AppKit.h>

AXError _AXUIElementGetWindow(AXUIElementRef element, uint32_t *identifier);

2. Reference this file in your build settings: (your path may vary, it is relative to project root)

3. Call like this:

// variable 'window' is your AXUIElement window
var cgWindowId = CGWindowID()
if (_AXUIElementGetWindow(window, &gcWindowId) != .success) {. 
  print("cannot get CGWindow id (objc bridged call)")
}

Congrats, you succesfully called objective C functions via a bridge!

Unwind