So in that case I think it's Gtk that applies the window decorations. How does this work?
Correct. GTK apps tell the window manager not to decorate them by setting the border width to 0. For now my suggestion would be to only implement that: if the window sets a border width of 0, ignore decorations for it. I wouldn't bother with anything else in the beginning. In fact you could even ignore this hint for now.
I read that the EWMH window property […]
Don't bother with EWMH for now. Just decorate all managed windows which don't set the border to 0. Besides, I don't see a good reason for why other window types like dialogs shouldn't be decorated; I don't think window managers really use this property to determine that, but I can only say for sure for a couple.
Is this how you should draw your window decorations normally? Or can I use Gtk (or something else) for this?
While you didn't explicitly ask for this, the last sentence in this quote tells me you might not fully understand how decorations work. The most common way, and I strongly recommend you do it like this, is called reparenting.
Reparenting means that when you manage a window, you create a new window (which of course you should not manage like a normal client window) called the frame window and then reparent the client window into your frame window. So the actual top-level window is the frame window owned by the window manager; the client window (the window the user interacts with) is the direct child of it.
Now you simply make the frame window slightly larger than the client window and position the client window correctly within it. Of course you need to track resizes of the client window and act upon them.
So why did we create this frame window? Simple! Because you can create a pixmap that you use for it and on which you draw your titlebar. This is better than drawing directly onto the child window because you don't mess with a window you don't actually own.
Drawing can be done with "raw" and simple calls like xcb_poly_fill_rectangle
or you can use a more sophisticated approach by, e.g., using a library like cairo (which I would recommend). The i3 window manager, for example, uses a simple abstraction which supports both via compile flag (libi3/draw_util.c).
This reparenting approach is the reason why tools such a xwininfo
or xprop
have a -frame
option. By default, these tools actually ignore the frame window and descend into the client window, pretty much hiding the fact that there is a frame window. Just try xprop
and xprop -frame
on the same window and you'll see that the frame window has much less information attached to it.
Once you have reparenting and drawing in place you can think more about cases where you don't need / want to decorate a window. Given that there's quite a bit to keep track of here I think implementing this at first will keep you busy for a while. I strongly recommend studying the code of other simple window managers which reparent.