I am trying to use an SSL connection with x11vnc (VNC server) and noVNC (VNC client). Whenever I try to connect, I get the error "Unsupported Security Types: 19,18" from noVNC and "SSL: ssl_helper[2957]: exit case 2 (ssl_init failed) SSL: accept_openssl: cookie from ssl_helper[2957] FAILED. 0" from x11vnc. If I turn off SSL for x11vnc, the client is able to connect with no problem.
I do realize that x11vnc does come packaged with a java based applet viewer. However, I'm more interested in the HTML5 based noVNC.
Both x11vnc and noVNC both run on the same machine, using the commands:
x11vnc: x11vnc -forever -shared -unixpw_cmd [cmd] -ssl [pem]
noVNC: ./utils/websockify --ssl-only --cert=[pem] --web=./ 6080 localhost:5900
Note: Both are pointing to the same pem. I use websockify instead of noVNC's launch.sh to have more options such as --ssl-only.
The encrypt option for noVNC seems to have no effect on connecting to the server (the results are the same whether it's on or off).
I am most concerned about a secure connection. As of now, it does not seem like turning on noVNC's encrypt option does much if x11vnc's SSL must be turned off (the encrypt option makes noVNC use wss:// instead of ws://). If this does create a secure connection, please let me know. Else, how can I get noVNC and x11vnc working with SSL?