Kubernetes ingress websockets connection issue

Question

Deployed k8s on AWS with KOPS. I have created nginx ingress https://github.com/kubernetes/ingress-nginx nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0

Everything is up and running and I could able to access applications externally using aws classic load balancer which is created by nginx service.

Recently we started working on websockets. I deployed my services in k8s and trying to access externally.

I created service and ingress for my application. Ingress is now pointing to loadbalancer(below json file).

I created route53 entry in aws and trying to connect to that but I am getting below error when I am trying to connect from my client application through chrome browser

WebSocket connection to 'wss://blockchain.aro/socket.io/?EIO=3&transport=websocket' failed: Error during WebSocket handshake: Unexpected response code: 400

I tried creating Application load balancer but I could not able to connect to wss://<host>

Error:

WebSocket connection to 'wss://blockchain.aro/socket.io/?EIO=3&transport=websocket' failed: Error during WebSocket handshake: Unexpected response code: 400

const config: SocketIoConfig = { url: 'wss://blockchain.aro',
  options: { autoConnect: false, transports: ['websocket']} };


Ingress:

   "annotations": {
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"extensions/v1beta1\",\"kind\":\"Ingress\",\"metadata\":{\"annotations\":{},\"name\":\"blockchain\",\"namespace\":\"adapt\"},\"spec\":{\"rules\":[{\"host\":\"blockchain.aro\",\"http\":{\"paths\":[{\"backend\":{\"serviceName\":\"blockchain\",\"servicePort\":8097},\"path\":\"/\"},{\"backend\":{\"serviceName\":\"blockchain\",\"servicePort\":8097},\"path\":\"/socket.io\"},{\"backend\":{\"serviceName\":\"blockchain\",\"servicePort\":8097},\"path\":\"/ws/\"}]}}],\"tls\":[{\"hosts\":[\"blockchain.aro\"],\"secretName\":\"blockchain-tls-secret\"}]}}\n",
      "nginx.ingress.kubernetes.io/proxy-read-timeout": "3600",
      "nginx.ingress.kubernetes.io/proxy-send-timeout": "3600"
    }

included tls and secretname and rules in ingress file. I tried creating ApplicationLoadbalancer but I could not able to connect with that too.

Galang

Answer 1

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: core-prod
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/secure-backends: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/websocket-services: core-service
    nginx.org/websocket-services: core-service
  name: core-ingress
spec:
  rules:
  - host: test.io
    http:
      paths:
      - backend:
          serviceName: core-service
          servicePort: 80
  tls:
  - hosts:
    - test.io
    secretName: core-prod

Psychogenesis

I updated asp per your inputs. gettting handshake error as . WebSocket connection to 'wss://blockchain.aro/socket.io/?EIO=3&transport=websocket' failed: Error during WebSocket handshake: Unexpected response code: 502 – Galang

@Galang try adding anotation if you are on AWS service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" – Psychogenesis

Or you can also do it like : "service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp" " – Psychogenesis

@Galang worked? Please update the status of question if it's helped you can upvote and if worked mark it as answer. This is how it's work – Psychogenesis

Sure Harsha, I updated and looks like it is connecting, but I could see that websockets information in pod logs like "websocket connection received from client" in server, but getting same error in client "failed: WebSocket is closed before the connection is established.. I am checking if this is code issue. I will check and update. Again appreciate your support, I will revert to this and update without fail. – Galang

Thank you, googled for 30 minutes before finding this – Recommendation

Specifically, for deploying an R Shiny app I had to set the nginx websocket-services annotations – Recommendation

I found this article that explains why it happens and an alternate solution, here – Lanctot