Kubernetes: in-memory shared cache between pods

Question

I am looking for any existing implementation of sharing a read only in-memory cache across pods on the same node. This setup would allow fast access without the need to load the entire cache into each pods memory.

Example: 1GB lookup dictionary is kept up to date, each pod has read access to the data allowing fast lookup without effectively cloning the data into memory. So end result would be just 1GB of memory utilized on the node, and not 1GB * N(number of pods)

Imagined solution for k8s:

1. A single (Daemon) pod that has the tmpfs volume RW, maintaining an up to date cache data
2. Multiple pods that have the same tmpfs volume R(only), mapping the data file(s) to read data out
   1. Naturally reading out values and operating on them is expected to create transitive memory usage

Notes:

• I have found multiple entries regarding volume sharing between pods, but no complete solution reference for the above
• tmpfs is ideal for cache speed of R/W, however obviously it can be over regular fs
• Looking for solutions that can be language specific or agnostic for reference
  • Language specific would be utilizing a specific language to map a file data as a Dictionary / other KV lookup
  • Language agnostic and more generalized solution could be utilizing sqlite where processes in our case here would be pods

Brannen

i got your issue or we can say plan it's good no doubt just one thought in mind if anyhow we run the daemon on K8s it will schedule the POD on each Node with tmpfs but how your service will make sure to connect to same node POD read only instance ? ideally traffic flow from service to PODs doesn't matter read of write replica. inside your application you can't use POD ip or so. Read more same scenario discussion : #70452838 – Unwholesome

there is an option in Redis for client-side cache but again as you mentioned 1GB * N(number of pods) : redis.io/topics/client-side-caching – Unwholesome

@HarshManvar assumption here is that the writer pod is setup as daemonset, making sure each Node has a valid file cache - while rest of pods mount it and "see" it as their local readonly copy – Brannen

hi adding for reference rbranson.medium.com/… and their open source project I am exploring github.com/segmentio/ctlstore – Brannen

Answer 1

You could use hostIPC and/or hostPath mounted on tmpfs, but that comes with a swathe of issues:

1. hostPath by itself poses a security risk, and when used, should be scoped to only the required file or directory, and mounted as ReadOnly. It also comes with the caveat of not knowing who will get "charged" for the memory, so every pod has to be provisioned to be able to absorb it, depending how it is written. It also might "leak" up to the root namespace and be charged to nobody but appear as "overhead"
2. hostIPC is a part of Pod Security Policies , which are depracted as of 1.21, and will be removed in the future

Generally, the best idea is to use Redis, it is one of the most used tools in such scenarios.

Wreckage

Using Redis is fairly reasonable suggestion. I think user is even trying to reduce the network round trip to Redis. That's why I landed on this page :) – Councilor