X11 forwarding of a GUI app running in docker

Question

First off: I have read the answers to similar questions on SO, but none of them worked.

IMPORTANT NOTE: The answer below is still valid, but maybe jump to the end for an alternative.

The situation:

• App with GUI is running in a docker container (CentOS 7.1) under Arch Linux. (machine A)
• Machine A has a monitor connected to it.
• I want to access this GUI via X11 forwarding on my Arch Linux client machine. (machine B)

What works:

• GUI works locally on machine A (with /tmp/.X11-unix being mounted in the Docker container).
• X11 forwarding of any app running outside of docker (X11 forwarding is set up and running properly for non-docker usage).
• I can even switch the user while remotely logged in, copy the .Xauthority file to the other user and X11 forwarding works as well.

Some setup info:

• Docker networking is 'bridged'.
• Container can reach host (firewall is open).
• DISPLAY variable is set in container (to host-ip-addr:10.0 because of TCP port 6010 where sshd is listening).
• Packets to X forward port (6010) are reaching the host from the container (tcpdump checked).

What does not work:

• X11 forwarding of the Docker app
• Errors:

X11 connection rejected because of wrong authentication.

xterm: Xt error: Can't open display: host-ip-addr:10.0

Things i tried:

• starting client ssh with ssh -Y option on machine B
• putting "X11ForwardTrusted yes" in ssh_config on machine B
• xhost + (so allow any clients to connect) on machine B
• putting Host * in ssh_config on machine B
• putting X11UseLocalhost no in sshd_config on machine A (to allow non-localhost clients)
• Adding the X auth token in the container with xauth add from the login user on machine A
• Just copying over the .Xauthority file from a working user into the container
• Making shure .Xauthority file has correct permissions and owner

How can i just disable all the X security stuff and get this working?

Or even better: How can i get it working with security?

Is there at least a way to enable extensive debugging to see where exactly the problem is?

Alternative: The first answer below shows how to effectively resolve this issue. However: I would recommend you to look into a different approach all together, namely VNC. I personally switched to a tigerVNC setup that replaces the X11 forwarding and have not looked back. The performance is just leagues above what X11 forwarding delivered for me. There might be some instances where you cannot use VNC for whatever reason, but i would try it first.

The general setup is now as follows: -VNC server runs on machine A on the host (not inside a docker container). -Now you just have to figure out how to get a GUI for inside a docker container (which is a much more trivial undertaking). -If the docker container was started NOT from the VNC environment, the DISPLAY variable maybe needs ajdusting.

Sandalwood

Answer 1

Ok, here is the thing:

1) Log in to remote machine

2) Check which display was set with echo $DISPLAY

3) Run xauth list

4) Copy the line corresponding to your DISPLAY

5) Enter your docker container

6) xauth add <the line you copied>*

7) Set DISPLAY with export DISPLAY=<ip-to-host>:<no-of-display>

*so far so good right?

This was nothing new...however here is the twist: The line printed by xauth list for the login user looks something like this (in my case):

<hostname-of-machine>/unix:<no-of-display> MIT-MAGIC-COOKIE-1 <some number here>

Because i use the bridged docker setup, the X forwarding port is not listening locally, because the sshd is not running in the container. Change the line above to:

<ip-of-host>:<no-of-display> MIT-MAGIC-COOKIE-1 <some number here>

In essence: Remove the /unix part.

<ip-of-host> is the IP address where the sshd is running.

Set the DISPLAY variable as above.

So the error was that the DISPLAY name in the environment variable was not the "same" as the entry in the xauth list / .Xauthority file and the client could therefor not authenticate properly.

I switched back to an untrusted X11 forwarding setting.

The X11UseLocalhost no setting in the sshd_config file however is important, because the incomming connection will come from a "different" machine (the docker container).

Sandalwood

Answer 2

Thanks so much @Lazarus535
I found that for me adding the following to my docker command worked:
--volume="$HOME/.Xauthority:/root/.Xauthority:rw"
I found this trick here
EDIT:
As Lazarus pointed out correctly you also have to set the --net=host option to make this work.

Ulbricht

Hello! Great find! But i suspect this only works if you add the --net=host flag as well? Because then the application "thinks" it runs locally networkwise. However i do not like the host network option and prefer if the docker have their own IP in a virtual network. – Sandalwood

You are absolutely right --net=host has to be added as well – Ulbricht

I never had this problem before running GUI in Docker. This time I was chaining the X11 through two servers and got this permission error. Attaching full command for others docker run -it --network=host --env DISPLAY=127.0.0.1:11.0 --privileged --volume="$HOME/.Xauthority:/root/.Xauthority:rw" -v /tmp/.X11-unix:/tmp/.X11-unix --rm <idockerimage> – Avocet

Note that the location of your Xauthority file can be overruled by $XAUTHORITY. So you could do --volume="${XAUTHORITY:-$HOME/.Xauthority}:/root/.Xauthority:rw" – Handle

Answer 3

Install xhost if you don't have it. Then, in bash,

export DISPLAY=:0.0
xhost +local:docker

After this run your docker run command (or whatever docker command you are running) with -e DISPLAY=$DISPLAY

Anapest

Getting a GUI from an app inside docker was already solved, my question specifically dealt with an added ssh X11 forwarding step. I will try this now. – Sandalwood

Did not work for me. You have not mounted /tmp/X11-unix (the X11 domain socket) and a new docker container per default is usually attached via a virtual network. I can not see how an app would find your X11 instance running outside of docker. Please clarify. Thanks. – Sandalwood

If your network for the container is set to host, the sockets are shared and the Unix socket shouldn't need to be explicitly called out – Inventive

After adding everything if it does not work yet, the solution above will solve it! – Coaction

This solved it for me, although display was :1. Nicely xhost +local:docker throws an error message if the DISPLAY variable is incorrect, so you can try different values until it works – Bellied

What is the role of the :docker part? The above works for me for anything after the colon. – Handle

@Handle It is ignored, but the colon needs to be there. You can use xhost +local:. If you just type xhost you see what is stored (LOCAL:). – Palpebrate

Answer 4

It works usually via https://mcmap.net/q/472626/-docker-cannot-connect-to-x-server

But if you are running docker with a different user than the one used for ssh -X into the server with; then copying the Xauthority only helped along with volume mapping the file.

Example - I sshed into the server with alex user.Then ran docker after su -root and got this error

X11 connection rejected because of wrong authentication.

After copying the .XAuthoirty file and mapping it like https://mcmap.net/q/468829/-x11-forwarding-of-a-gui-app-running-in-docker made it work

cp /home/alex/.Xauthority .

docker run  -it --network=host --env DISPLAY=$DISPLAY  --privileged  \
 --volume="$HOME/.Xauthority:/root/.Xauthority:rw"  \
-v /tmp/.X11-unix:/tmp/.X11-unix --rm <dockerimage>

More details on wiring here https://unix.stackexchange.com/a/604284/121634

Avocet

Answer 5

Some clarifying remarks. Host is A, local machine is B

Ive edited this post to note things that I think should work in theory but haven't been tested, vs things I know to work

Running docker non-interactively

If your docker is running not interactively and running sshd, you can use jumphosts or proxycommand and specify the x11 client to run. You should NOT volume share your Xauthority file with the container, and sharing -e DISPLAY likely has no effect on future ssh sessions

Since you essentially have two sshd servers, either of the following should work out of the box

if you have openssh-client greater than version 7.3, you can use the following command

ssh -X -J user-on-host@hostmachine,user-on-docker@dockercontainer xeyes

If your openssh client is older, the syntax is instead (google says the -X is not needed in the proxy command, but I am suspicious)

ssh -X -o ProxyCommand="ssh -W %h:%p user-on-host@hostmachine" user-on-docker@dockermachine xeyes

Or ssh -X into host, then ssh -X into docker.

In either of the above cases, you should NOT share .Xauthority with the container

Running docker interactively from within the ssh session

The easiest way to get this done is to set --net=host and X11UseLocalhost yse. If your docker is running sshd, you can open a second ssh -X session on your local machine and use the jumphost method as above. If you start it in the ssh session, you can either -e DISPLAY=$DISPLAY or export it when you're in. You might have to export it if you attach to an exiting container where this line wasn't used.

Use these docker args for --net host and x11uselocalhost yes ssh -X to host

   -e DISPLAY=$DISPLAY
   -v $HOME/.Xauthority:/home/same-as-dash-u-user/.Xauthority
   -u user

What follows is explanation of how everything works and other approaches to try

About Xauthority

ssh -X/-Y set up a session key in the hosts Xauthority file, and then sets up a listen port on which it places an x11 proxy that uses the session key, and converts it to be compatible with the key on your local machine. By design, the .Xauthority keys will be different between your local machine and the host machine. If you use jumphosts/proxycommand the keys between the host and the container will yet again be different from each other. If you instead use ssh tunnels or direct X11 connection, you will have to share the host Xauthority with the container, in the case of sharing .Xauthority with the container, you can only have one active session per user, since new sessions will invalidate the previous ones by modifying the hosts .Xauthority such that it only works with that session's ssh x11 proxy

X11UserLocalhost no theory##

Even Though X11UseLocalhost no causes the x server to listen on the wildcard address, With --net host I could not redirect the container display to localhost:X.Y where x and why are from the host $DISPLAY

X11UseLocalhost yes is the easy way

If you choose X11UseLocalhost yes the DISPLAY variable on the host becomes localhost:X:Y, which causes the ssh x11 proxy to listen only on localhost port x.

If X11UseLocalhost is no, the DISPLAY variable on the host becomes the host's hostname:X:Y, which causes the xerver to listen on 0.0.0.0:6000+X and causes xclients to reach out over the network to the hostname specified.

this is theoretical, I don't yet have access to docker on a remote host to test this

But this is the easy way. We bypass that by redirecting the DISPLAY variable to always be localhost, and do docker port mapping to move the data from localhost:X+1.Y on the container, to localhost:X.Y on the host, where ssh is waiting to forward x traffic back to the local machine. The +1 makes us agnostic to running either --net=host or --net=bridge setting up container ports requires specifying expose in the dockerfile and publishing the ports with the -p command.

Setting everything up manually without ssh -X

This works only with --net host. This approach works without xauth because we are directly piping to your unix domain socket on the local machine

ssh to host without -X

ssh -R6010:localhost:6010 user@host
start docker with -e DISPLAY=localhost:10.1 or export inside

in another terminal on local machine

socat -d -d TCP-LISTEN:6010,fork UNIX-CONNECT:/tmp/.X11-unix/X0

In original terminal run xclients

if container is net --bridged and you can't use docker ports, enable sshd on the container and use the jumphosts method

Inventive

where does the X11UseLocalhost setting go, I don't see it mentioned in any of the commands? is it part of the configuration of the host or the container? – Hance

Answer 6

This is one of those problem where if you don't know what's happening and one link is not satisfied - there's no way it'll work. There is no magic or voodoo here. I'll try to explain it clearly.

There is logically only

1. 1 X server (the one with a screen you are looking at on your localhost with the keyboard plugged into it)
2. the remote host you are connecting to
3. the docker container which runs the app
4. the app running in the container (on the remote host)

Local Listener <---- Remote Listener <--- X Client

The app (say xclock from x11-apps) connects to whatever the docker environments $DISPLAY is set to - usually a file socket (inside the container) at /tmp/.X11/# where # is the :#.0 in the docker environments $DISPLAY

Remember a container is just a process on the Remote host running in a fancy chroot with filesystem/network from CGROUPS. But really it's just a fancy process on the Remote host.

when you ssh -X remoteuser@remotehost - ssh will set up a remote tunnel back to your local X server and sets a DISPLAY env (on the remote host) that points to the remote hosts side of that tunnel.

Unfort your app running in the container can't just connect to that - I don't know why.

You can do netstat -latnp and echo $DISPLAY to see what it is.

What's a xauth MAGIC_COOKIE - it's shared password/token that's it - simple and the hostname:# part is like a username.

It will also create a MAGIC_COOKIE and add it to the remote hosts users .Xauthority file

Every time you ssh -X into your remote host you get a MAGIC_COOKIE and DISPLAY and listening TCP socket.

Try it. ssh in multiple times at the same time and run in each

echo $DISPLAY
xauth list 
netstat -latnp

X on your localhost usually listens on 6010 you will see something like

netstat -latnp
tcp        0      0 127.0.0.1:6013          0.0.0.0:*               LISTEN      -

echo $DISPLAY
localhost:13.0

So you need to set up a forwarder in the remote host from the container to the remote hosts X listener socket (that SSH gave you) using socat.

Maybe there's other ways of doing this but socat is fine - just run it in the background before you start the container.

socat TCP:localhost:60${DISPLAY_NUMBER} UNIX-LISTEN:${DISPLAY_DIR}/socket/X${CONTAINER_DISPLAY} &

You can mount that $DISPLAY_DIR into the container at the mount point that the xclock app will connect.

You'll see the error in the docker run change from

Error: Can't open display: :1.0

to

X11 connection rejected because of wrong authentication.

You also need to let the xclock app pick a MAGIC_COOKIE to use by mounting the .Xauthority file.

So this goes into your docker command or you docker-compose.yml

-v ./${DISPLAY_DIR}/.Xauthority:/root/.Xauthority -v ./${DISPLAY_DIR}/socket:/tmp/.X11-unix

So mount your remote hosts .Xauthority file into the container at the default location $HOME/ where $HOME is the containers environment

You should see the error change to

Authorization required, but no authorization protocol specified

So you need to sort authorization.

use xauth list on the remote host to see what's going on.

One way to circumvent xauth host/cookie (user/password) checking is by running xauth + on the local host.

If you had run ssh -v (with verbose) and you instead run xhost + or xhost on the remote host then you'll see activity on the tunnel and it will be rejected

xhost:  must be on local machine to enable or disable access control.

This is because X is inherently TCP network capable it's perfectly legitimate to have network client connections just as unix domain sockets to connect to it.

You don't need a --privileged container either or --network either AFAIK

I hope I've cleared up a few misconceptions (and probably introduced a few of my own) I wanted to keep this a brief clean theory read without all the dry doco. But here is my full working example.

DISPLAY_NUMBER=$(echo $DISPLAY | cut -d. -f1 | cut -d: -f2)
CONTAINER_DISPLAY=1
DISPLAY_DIR=.display
AUTH_COOKIE=$(xauth list | grep "^$(hostname)/unix:${DISPLAY_NUMBER} " | awk '{print $3}')

mkdir -p ${DISPLAY_DIR}/socket
xauth -f ${DISPLAY_DIR}/.Xauthority add ${CONTAINER_HOSTNAME}/unix:${CONTAINER_DISPLAY} MIT-MAGIC-COOKIE-1 ${AUTH_COOKIE}

sudo socat TCP:localhost:60${DISPLAY_NUMBER} UNIX-LISTEN:${DISPLAY_DIR}/socket/X${CONTAINER_DISPLAY} &

docker run --privileged --name 3dspecan --hostname ${CONTAINER_HOSTNAME} --rm  -it -v ./${DISPLAY_DIR}/.Xauthority:/root/.Xauthority -v ./${DISPLAY_DIR}/socket:/tmp/.X11-unix -v .:/app 3dspecan ${ARGS[@]}
   
echo $?

Hushaby

Answer 7

Below is a solution using socat for redirecting the port. It creates the XAUTH cookie file for authentication as well. Assuming docker0 interface IP is set to 172.17.0.1.

#!/bin/bash

trap 'kill $(jobs -p)' EXIT

XAUTH=.docker.xauth
echo > $XAUTH
xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f $XAUTH nmerge -

X11PORT=`echo $DISPLAY | sed 's/^[^:]*:\([^\.]\+\).*/\1/'`
TCPPORT=`expr 6000 + $X11PORT`
socat TCP-LISTEN:$TCPPORT,fork,bind=172.17.0.1 TCP:127.0.0.1:$TCPPORT &

DISPLAY=`echo $DISPLAY | sed 's/^[^:]*\(.*\)/172.17.0.1\1/'`

sudo docker run -ti --rm \
        --user="$(id -u):$(id -g)" \
        -e DISPLAY=$DISPLAY \
        -e XAUTHORITY=/tmp/$XAUTH \
        --volume=$(pwd)/$XAUTH:/tmp/$XAUTH \
        --volume="/etc/group:/etc/group:ro" \
        --volume="/etc/passwd:/etc/passwd:ro" \
        opensuse

Have fun!

Prepuce