Mac OS X Window Server vs. X11: the insane task

Question

Dedicated to all who likes low-level Window Server (CoreGraphicsPrivate.h, etc), X11 on Mac, SIMBL and other crazy stuff :)

There's a simple X11-emulated application on Mac (like xterm, xeyes and so on) with one window. While running, X11 somehow creates a native Quartz window to represent this emulated application, and this window is accessible via Quartz Window Services so that I can get its CSWindowID, title, position, size and owner's PID (PID of X11.app). But it does not support Accessibility API, so there's no way to control it (except, maybe Core Graphichs private functions from the same process).

---

Now, here's the task:

I need to host an additional NSView (or just draw something) upon such a window. I mean a native Quartz window, which appeared as a result of X11 emulating some application. I know, to manipulate windows on Mac I must be in the same process, i.e. X11.app.

---

I wrote a SIMBL plug-in which intrudes into X11.app process.

There I can call [NSApp windows], but all the time I'm getting exactly 2 NSWindows which have nothing in common with real applications' windows. They are not even visible on the screen.

Nevertheless, when I call NSWindowList(), I get anything I need (window IDs for X11 windows) and even more (window IDs from other applications).

When I've got CSWindowIDs for X11-emulated windows, I call [NSApp windowWithWindowNumber: ] (Cocoa) and HIWindowFromCGWindowID() (Carbon), but they both return nil! From the very same process!

BTW, all this actions work perfectly when I intrude into Safari process and others...

---

So, the questions are:

• How did X11 create such windows which are not accessible from the very same process?

• How can I get pointers to X11 windows (NSWindow *, CGContextRef, or, at least, anything...) and host my graphics (I don't even speak about NSViews) upon them?

---

Thanks a lot in advance!

Black

Answer 1

All the X11.app sources and other stuff (Xquartz) are available at Apple's official site (current version 2.3.5 (server 85.2)). The core of windows creation lies in xpr subdirectory.

To manipulate windows Xquartz uses Xplugin library (/usr/lib/libXplugin.dylib). Its header, /usr/include/Xplugin.h, defines functions like xp_create_surface() and others, which create windows using private CoreGraphics API, like CGSNewWindowWithOpaqueShape(). Undocumented CoreGraphicsPrivate.h or CSGPrivate.h, the result of reverse engineering, can be found over the Web. Xplugin remembers ids of such Quartz windows in its own hash and returns an opaque integer (i.e., xp_resource_id) for them. Then Xquartz associates a particular XID with this xp_resource_id and returns it to a client.

Xplugin is closed source and has no API to return native Quartz drawable by xp_resource_id or XID.

In order to draw upon a window which was created with private CoreGraphics API you have to use those private API. There's a function, named CGWindowContextCreate(), which returns CGContextRef for a particular native window by its Quartz id. It is possible to draw on the window using this context. But to receive the real context instead of NULL, you must be in a process, which created the window.

Black

If this is the answer to your question you should hit the checkmark so the system knows it's been answered. – Shiller

Answer 2

It is my understanding that X11 uses its own windows server and general stack. That is why it can run X11 apps without special ports.

It only has a layer of responses that mimics that of Cocoa windows such that it can communicate with the general interface. Its not a Cocoa stack in disguise, its an X11 stack superficially disguised as Cocoa. As such, it only responds to only a subset of the Cocoa related messages.

I think to do anything serious in X11 you have to use the X11 API from the start. In other words, write as if it was not intended to run on top of the Mac OS.

Shiller

So, it's possible to create a real Quartz window but even have no WindowRef for it? How then X11 moves its windows when gets NSEvents, collapse them and so on? You mean it stores WindowsRefs somewhere deeply in its code? – Black

Seems like there're private API to create windows... X11 uses libXplugin, in it in turn uses functions like CGSNewWindow()... – Black

CGSNewWindow() is part of the SDL (Simple DirectMedia Library) lib which is a cross platform media/gaming library. see libsdl.org/index.php Like, I said, its not really Cocoa. – Shiller

OK, then look at: CGError CGSNewWindowWithOpaqueShape(CGSConnectionID cid, int always2, float x, float y, CGSRegionRef shape, CGSRegionRef opaqueShape, int unknown1, void *unknownPtr, int always32, CGSWindowID *outWID); – Black

What am I supposed to look at? Rather, what am I supposed to infer from this code? – Shiller

Open /usr/lib/libXplugin.1.dylib with IDA Pro and look at _xp_create_window subroutine. You'll see it uses CGSNewWindowWithOpaqueShape() there and then remembers its result in own hash and returns an opaque ID, known as xp_window_id. Xquartz (X server bindings for Mac) uses Xplugin to create windows, and in its turn associate x_window_id with XID. – Black

So, in the end it has it's pretty much like I thought. It has custom internal window management and maps that to Cocoa. – Shiller