How can I share an OpenGL context/texture between 2 processes (linux)

Question

I am trying to build 2 applications running in separate processes. One application will display live video from a camera (server) and the other will overlay UI (client) on top of that video. The solution requires low latency and therefore I would like to render both without going thru the OS compositor. The solution I am trying to implement involves creating a shared OpenGL context or texture so that the UI can render its part to some off screen buffer/texture. After every live image frame is rendered the server can take the information from the off-screen buffer/texture and render it on top. This way there is no latency added due to synchronization of the processes. The server will take the latest image from the UI if one is ready. In case it is not ready it shouldnt wait for it, and use a previous image instead.

How can i pass a texture or context between processes? The CreateContext function can take a pointer of another context and make it shared but the address as far as I understand will not be valid outside the process space.

Jenelljenelle

I ended up solving this using CUDA. Cuda provides interop with openGL which allows mapping of openGL buffer to CUDA memory space. In addition CUDA allows sharing of CUDA memory between processes using its own IPC functions. – Jenelljenelle

Answer 1

These days the "cleanest" way to share GPU resources between processes is to create those resources using Vulkan, export them into file descriptors (POSIX) or HANDLEs (Win32) and import those into OpenGL contexts created at either side. The file descriptors you can pass by the usual methods (sendmsg with SCM_RIGHTS, or pidfd_getfd, or open("/proc/${PID}/fd/${FD}").

Exporting from Vulkan:

https://www.khronos.org/registry/vulkan/specs/1.2-khr-extensions/html/chap46.html#VK_KHR_external_memory_fd (ff.)

Importing into OpenGL:

https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt

https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects_fd.txt

https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects_win32.txt

Doing with just "pure" OpenGL requires a lot of hacks. One way would be to force indirect contexts (at the cost of modern capabilities, due to the lack of GLX support for those) and sharing the X11 IDs for that. Another method is to use ptrace to access to mapped buffers in the other process. Either is quite daunting and a lot more work to properly implement (BT;DT.), than setting up a Vulkan instance, creating all the textures therein and then importing them to OpenGL.

Formaldehyde

How this can be done, when HANDLEs are just pointers (void*), so they can cannot be shared between different processes? I need to do Vulkan + IPC (interprocess communication), I used the shared memory object and wanted to avoid GL's PBO using Vulkan's interop and importing the mem. through handle in other process. But it seems impossible to me, as still I need to transfer whole texture data through the shared memory and not just a mem. handle, that could be reused in other process. Is there a way how to do IPC without transferring the whole pixel data chunk? – Chrystel

@Chrystel handles can be made accessible to a different process using the DuplicateHandle Win32 function: learn.microsoft.com/en-us/windows/win32/api/handleapi/… – you'd then use whatever form of IPC to pass the value of the pointer (which is relative to the destination process address space) to the other process. Could be shared memory, or using WriteProcessMemory if you know the destination address learn.microsoft.com/en-us/windows/win32/api/memoryapi/… – Formaldehyde