Redis master/slave on GCP MemoryStore

Question

In our current architecture, we are using Redis from MemoryStore, as a simple fast cache and pub-sub system.

For one of our services, we need to have a local very fast cache for a replicated service. The service would be hosted on GKE (Google Kubernetes Engine). The idea would be to have a Redis slave per node which would be the endpoint of any pod of our services.

Is it possible to use our MemoryStore Redis as a master of this system? If yes, is there a documentation or example somewhere for that? Or should we have a master Redis instance on our K8S cluster?

Therrien

Answer 1

Not sure you can do it or not with MemoryStore. As much as i know it's managed service not provide a feature like that.

Generally what i have seen and used is Redis HA Kubernetes Deployment with Sentinel.

Helm : https://docs.bitnami.com/tutorials/deploy-redis-sentinel-production-cluster/

What is sentinel?

Consider it like a sidecar proxy that manages the Master and Slave connections. it will return the Master and Slave IPs to you if you send request to it.

By default helm of Redis will deploy the Master POD with 2 slave POD and sentinel as a sidecar.

When you say, The idea would be to have a Redis slave per node which would be the endpoint of any pod of our services.

On a note of Deploying Redis slave on each node might be easy to configure but connecting POD directly to that Slave on the same Node would be weird as all traffic goes using Kubernets service.

Yes, you can do it keeping the slave on each node but not sure how your service will connect to those slaves ?

Your application will be talking with Single Service of Kubernetes, that service will return the IP of Current Masters and Slave as per sentinel document.

Sentinel Documentaion : https://redis.io/topics/sentinel

Extra note (Clustering Vs Sentinel):

Redis clustering is distributed option while sentinel is good for HA and Replication as new Master will be always ready to handle.

Here one application example in Python :

from redis import Sentinel
sentinel = Sentinel([('<**Single K8s service Name**>', 26379)], socket_timeout=0.1)
sentinel.discover_master('mymaster')
('127.0.0.1', 6379)
sentinel.discover_slaves('mymaster')
[('127.0.0.1', 6380)]

Reference : https://github.com/redis/redis-py#sentinel-support

Frumpy

Thanks for your clear answer. However are you sure it is not possible to force communication with the slave on the same node? I'd say it would be possible using NodePort type for slaves? – Therrien

for using of NodePort you will use Node IP along port right ? or service name with node port ? – Frumpy

I think the best solution would be the first one: deploying a pod on each node (I know it's possible, Datadog does it for its agents for instance), and then exposing a port and talking to it through node DNS name and port. Am I wrong? – Therrien

yes, you can give it try if everything works well replication backup and all. – Frumpy