how to set different environment variables of Deployment replicas in kubernetes

Question

I have 4 k8s pods by setting the replicas of Deployment to 4 now.

apiVersion: v1
kind: Deployment
metadata:
  ...
spec:
  ...
  replicas: 4
...

The POD will get items in a database and consume it, the items in database has a column class_name.

now I want one pod only get one class_name's item. for example pod1 only get item which class_name equals class_name_1, and pod2 only get item which class_name equals class_name_2...

So I want to pass different class_name as environment variables to different Deployment PODs. Can I define it in the yaml file of Deployment?

Or is there any other way to achieve my goal?(like something other than Deployment in k8s)

Uncouple

Answer 1

I would not recommend this approach, but the closest thing to do what you want is using the stateful-set and use the pod name as the index.

When you deploy a stateful set, the pods will be named after their statefulset name, in the following sample:

apiVersion: v1
kind: Service
metadata:
  name: kuard
  labels:
    app: kuard
spec:
  type: NodePort
  ports:
  - port: 8080
    name: web
  selector:
    app: kuard
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: kuard
spec:
  serviceName: "kuard"
  replicas: 3
  selector:
    matchLabels:
      app: kuard
  template:
    metadata:
      labels:
        app: kuard
    spec:
      containers:
      - name: kuard
        image: gcr.io/kuar-demo/kuard-amd64:1 
        ports:
        - containerPort: 8080
          name: web

The pods created by the statefulset will be named as:

kuard-0
kuard-1
kuard-2

This way you could either, name the stateful-set according to the classes, i.e: class-name and the pod created will be class-name-0 and you can replace the _ by -. Or just strip the name out to get the index at the end.

To get the name just read the environment variable HOSTNAME

This naming is consistent, so you can make sure you always have 0, 1, 2, 3 after the name. And if the 2 goes down, it will be recreated.

Like I said, I would not recommend this approach because you tie the infrastructure to your code, and also can't scale(if needed) because each service are unique and adding new instances would get new ids.

A better approach would be using one deployment for each class and pass the proper values as environment variables.

Diminished

Sounds like a good usecase for snowflake ID generator github.com/bwmarrin/snowflake ? Or is there a drawback ? – Ole

Answer 2

For distributed job processing Deployments are not very good, because they don't have any type of ordering or consistent pod hostnames. You'd better use StatefulSet for it, because they have consistent naming, like pod-0, pod-1, pod-2. You can rely on that hostname index.

For example, if your class_name_idx - is the index of class name in class names list, num_replicas - is the number of replicas in StatefulSet and pod_idx - is the index of pod in StatefulSet, then pod should run the job only if: class_name_idx % num_replicas == pod_idx.

Unfortunately number of StatefulSet replicas cannot be obtained within the pod dynamically using Downward API, so you can either hardcode it or use Kubernetes API to obtain it from cluster.

Tallula

Answer 3

Neither Deployment nor anything else won't help to achieve your goal. Your goal is some kind of logic and it should be implemented via code in your application.

Since the Deployment is some instances of the same application the only thing that might be useful for you is: using multiple deployments, each for its own task. The first could get class_name_1 item, while other class_name_2, class_name_3 etc. But it is not a good idea

Sweitzer