K8S: Error running load balancer syncing routine

Question

Trying to get ThingsBoard running on google cloud.

I am now seeing the following error:

Error during sync: error running load balancer syncing routine: loadbalancer thingsboard-tb-ingress--013d7ab9087175d7 does not exist: CreateUrlMap: googleapi: Error 400: Invalid value for field 'resource': '{ "name": "k8s-um-thingsboard-tb-ingress--013d7ab9087175d7", "hostRule": [{ "host": ["*"], "...'. Invalid path pattern, invalid

kubectl describe ingress gives me the following:

Name:             tb-ingress
Namespace:        thingsboard
Address:
Default backend:  default-http-backend:80 (10.52.0.5:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *
        /api/v1/.*            tb-http-transport:http (<none>)
        /static/rulenode/.*   tb-node:http (<none>)
        /static/.*            tb-web-ui:http (<none>)
        /index.html.*         tb-web-ui:http (<none>)
        /                     tb-web-ui:http (<none>)
        /.*                   tb-node:http (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/proxy-read-timeout":"3600","nginx.ingress.kubernetes.io/ssl-redirect":"false","nginx.ingress.kubernetes.io/use-regex":"true"},"name":"tb-ingress","namespace":"thingsboard"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"tb-http-transport","servicePort":"http"},"path":"/api/v1/.*"},{"backend":{"serviceName":"tb-node","servicePort":"http"},"path":"/static/rulenode/.*"},{"backend":{"serviceName":"tb-web-ui","servicePort":"http"},"path":"/static/.*"},{"backend":{"serviceName":"tb-web-ui","servicePort":"http"},"path":"/index.html.*"},{"backend":{"serviceName":"tb-web-ui","servicePort":"http"},"path":"/"},{"backend":{"serviceName":"tb-node","servicePort":"http"},"path":"/.*"}]}}]}}

  nginx.ingress.kubernetes.io/proxy-read-timeout:  3600
  nginx.ingress.kubernetes.io/ssl-redirect:        false
  nginx.ingress.kubernetes.io/use-regex:           true
Events:
  Type     Reason  Age               From                     Message
  ----     ------  ----              ----                     -------
  Warning  Sync    3m (x28 over 1h)  loadbalancer-controller  Error during sync: error running load balancer syncing routine: loadbalancer thingsboard-tb-ingress--013d7ab9087175d7 does not exist: CreateUrlMap: googleapi: Error 400: Invalid value for field 'resource': '{  "name": "k8s-um-thingsboard-tb-ingress--013d7ab9087175d7",  "hostRule": [{    "host": ["*"],    "...'. Invalid path pattern, invalid

What am I missing here?

Furculum

Hi, check the content of kubectl describe ingress $INGRESS_NAME – Luthanen

Updated my answer with that information – Furculum

there need to be some value for the host key, as its wildcard here – Luthanen

Without a hostname, how we can access it curl -kv https://HOSTNAME/PATH – Luthanen

Can you post an answer and point out the error in the yaml file (github.com/thingsboard/thingsboard/blob/release-2.3/k8s/…) So I can accept it? – Furculum

Hi @ThomasVdBerge, can you accept the answer and close the question, If it resolved the issue – Luthanen

Answer 1

I forgot to specify kubernetes.io/ingress.class: "nginx" annotation. If you don't specify any kubernetes.io/ingress.class - GKE will consider using its own ingress which does not support regexps and gives the error mentioned.

Furculum

Are you exposing your ingress controller using a LoadBalancer service? I have similar problem which you had but if you configure ingress.class: "nginx" you can't reserve a Global static address for it :-/ – Vocoid

@Vocoid You have to reserve the IP yourself. If you're using GCP, you can do that with gcloud as described in the Nginx Ingress docs kubernetes.github.io/ingress-nginx/examples/static-ip. You could also use some infrastructure automation like terraform to get the static IP address reserved – Drunken

Answer 2

The error occurs when using the default gke loadbalancer and using the wrong path expression. From the documentation: https://cloud.google.com/kubernetes-engine/docs/how-to/load-balance-ingress

• The only supported wildcard character for the path field of an Ingress is the * character. The * character must follow a forward slash (/) and must be the last character in the pattern. For example, /, /foo/, and /foo/bar/* are valid patterns, but , /foo/bar, and /foo/*/bar are not.
• A more specific pattern takes precedence over a less specific pattern. If you have both /foo/* and /foo/bar/, then /foo/bar/bat is taken to match /foo/bar/. For more information about path limitations and pattern matching, see the URL Maps documentation.

You don't need to set a host entry per se, this error is actually very confusing and unclear.

From the same page, this is a valid configuration:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    # If the class annotation is not specified it defaults to "gce".
    kubernetes.io/ingress.class: "gce"
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: hello-world
          servicePort: 60000
      - path: /kube
        backend:
          serviceName: hello-kubernetes
          servicePort: 80

To use regular expression you need to use another ingress controller, like nginx or haproxy:

https://kubernetes.github.io/ingress-nginx/deploy/#gce-gke

https://github.com/haproxytech/kubernetes-ingress#readme

Chop