Exposing Redis with Ingress Nginx Controller
Asked Answered
N

2

7

Hello when I use node port to expose my redis service it works fine. I am able to access it. But if I try switch to Ingress Nginx controller it refuse to connect.. Other apps work fine with ingress.

Here is my service:

apiVersion: v1
kind: Service
metadata:
  name: redis-svc
spec:
#  type: NodePort
  ports:
    - name: http
      port: 6379
      targetPort: 6379
      protocol: TCP
#      nodePort: 30007
  selector:
    app: redis

And here is ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: redis-ing
  annotations:
     kubernetes.io/ingress.class: "nginx"
     ingress.kubernetes.io/ssl-redirect: "true"
     nginx.ingress.kubernetes.io/ssl-redirect: "true"
     nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
     cert-manager.io/cluster-issuer: "letsencrypt-prod"
    #  nginx.ingress.kubernetes.io/enable-cors: "true"
    #  nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
    #  nginx.ingress.kubernetes.io/cors-allow-origin: "https://test.hefest.io"
    #  nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
spec:
  tls:
  - secretName: letsencrypt-prod
    hosts:
      - redis-dev.domain.com
  rules:
  - host: redis-dev.domain.com
    http:
      paths:
      - path: /
        backend:
          serviceName: redis-svc
          servicePort: 6379

Any idea what can be an issue?

I am using this ingress controller: https://github.com/nginxinc/kubernetes-ingress

Noletta answered 16/7, 2020 at 17:1 Comment(0)
A
6

Redis works on 6379 which is non HTTP port(80,443). So you need to enable TCP/UDP support in nginx ingress controller. The minikube docs here shows how to do it for redis.

Update the TCP and/or UDP services configmaps

Borrowing from the tutorial on configuring TCP and UDP services with the ingress nginx controller we will need to edit the configmap which is installed by default when enabling the minikube ingress addon.

There are 2 configmaps, 1 for TCP services and 1 for UDP services. By default they look like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: udp-services
  namespace: ingress-nginx

Since these configmaps are centralized and may contain configurations, it is best if we only patch them rather than completely overwrite them.

Let’s use this redis deployment as an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  namespace: default
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - image: redis
        imagePullPolicy: Always
        name: redis
        ports:
        - containerPort: 6379
          protocol: TCP

Create a file redis-deployment.yaml and paste the contents above. Then install the redis deployment with the following command:

kubectl apply -f redis-deployment.yaml

Next we need to create a service that can route traffic to our pods:

apiVersion: v1
kind: Service
metadata:
  name: redis-service
  namespace: default
spec:
  selector:
    app: redis
  type: ClusterIP
  ports:
    - name: tcp-port
      port: 6379
      targetPort: 6379
      protocol: TCP

Create a file redis-service.yaml and paste the contents above. Then install the redis service with the following command:

kubectl apply -f redis-service.yaml

To add a TCP service to the nginx ingress controller you can run the following command:

kubectl patch configmap tcp-services -n kube-system --patch '{"data":{"6379":"default/redis-service:6379"}}'

Where:

6379 : the port your service should listen to from outside the minikube virtual machine

default : the namespace that your service is installed in

redis-service : the name of the service

We can verify that our resource was patched with the following command:

kubectl get configmap tcp-services -n kube-system -o yaml

We should see something like this:

apiVersion: v1
data:
  "6379": default/redis-service:6379
kind: ConfigMap
metadata:
  creationTimestamp: "2019-10-01T16:19:57Z"
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
  name: tcp-services
  namespace: kube-system
  resourceVersion: "2857"
  selfLink: /api/v1/namespaces/kube-system/configmaps/tcp-services
  uid: 4f7fac22-e467-11e9-b543-080027057910

The only value you need to validate is that there is a value under the data property that looks like this:

"6379": default/redis-service:6379

Patch the ingress-nginx-controller

There is one final step that must be done in order to obtain connectivity from the outside cluster. We need to patch our nginx controller so that it is listening on port 6379 and can route traffic to your service. To do this we need to create a patch file.

spec:
  template:
    spec:
      containers:
      - name: ingress-nginx-controller
        ports:
         - containerPort: 6379
           hostPort: 6379

Create a file called ingress-nginx-controller-patch.yaml and paste the contents above.

Next apply the changes with the following command:

kubectl patch deployment ingress-nginx-controller --patch "$(cat ingress-nginx-controller-patch.yaml)" -n kube-system
Appreciate answered 16/7, 2020 at 17:40 Comment(3)
I am using this repository: github.com/nginxinc/kubernetes-ingress ... It is example for another one github.com/kubernetes/ingress-nginx ....Noletta
So you followed the approach in my answer and it did not work?Appreciate
It worked for me, I'm using Microk8sFacile
E
1

The way I made it work is by enabling ssl-passthrough on nginx-ingress controller. Once my nginx-ingress controller was patched, I was able to connect.

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: redis-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-passthrough: 'true'
spec:
  tls:
    - hosts:
      - <host_address>
      secretName: <k8s_secret_name>
  rules:
    - host: <host_address>
      http:
        paths:
          - path: "/"
            pathType: Prefix
            backend:
              service:
                name: redis-service
                port:
                  number: 6380

python snippet to connect

import redis

r = redis.StrictRedis(host='<host_address>',
                port=443, db=0, ssl=True,
                ssl_ca_certs='server.pem')

print(r.ping())

I do use redistls in my case.

Executory answered 9/3, 2022 at 15:48 Comment(3)
I don't think its a good idea to use predefined port i.e 443 which is used for HTTPS protocolValarievalda
nginx ingress only works with ports 80/443. So, in that particular scenario, that's the only available port.Executory
This is also how redis states to configure access through ingress in their docs. docs.redis.com/latest/kubernetes/networking/ingressArel

© 2022 - 2024 — McMap. All rights reserved.