Handling PersistentVolumeClaim in DaemonSet

Question

I have a DaemonSet that creates flink task manager pods, one per each node.

Nodes

Say I have two nodes

• node-A
• node-B

Pods

the daemonSet would create

• pod-A on node-A
• pod-B on node-B

Persistent Volume Claim

• I am on AKS and want to use azure-disk for Persistent Storage
• According to the docs : [https://learn.microsoft.com/en-us/azure/aks/azure-disks-dynamic-pv ]
  • an azure disk can be associated on to single node

say I create

• pvc-A for pv-A attached to node-A
• pvc-B for pv-B attached to node-B

Question

How can I associate pod-A on node-A to use pcv-A ?

UPDATE:

After much googling, i stumbled upon that it might be better/cleaner to use a StatefulSet instead. This does mean that you won't get the features available to you via DaemonSet like one pod per node.

https://medium.com/@zhimin.wen/persistent-volume-claim-for-statefulset-8050e396cc51

Kadner

you need a cloud native storage backend where the PVC is independent of where the real disk is. – Downcome

Answer 1

If you use a persistentVolumeClaim in your daemonset definition, and the persistentVolumeClaim is satisfied with PV with the type of hostPath, your daemon pods will read and write to the local path defined by hostPath. This behavior will help you separate the storage using one PVC.

This might not directly apply to your situation but I hope this helps whoever searching for something like a "volumeClaimTemplate for DaemonSet" in the future.

Using the same example as cookiedough (thank you!)

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: x
  namespace: x
  labels:
    k8s-app: x
spec:
  selector:
    matchLabels:
      name: x
  template:
    metadata:
      labels:
        name: x
    spec:
      ...
      containers:
      - name: x
        ...
        volumeMounts:
        - name: volume
          mountPath: /var/log
      volumes:
      - name: volume
        persistentVolumeClaim:
          claimName: my-pvc

And that PVC is bound to a PV (Note that there is only one PVC and one PV!)

apiVersion: v1
kind: PersistentVolume
metadata:
  creationTimestamp: null
  labels:
    type: local
  name: mem
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 1Gi
  hostPath:
    path: /tmp/mem
    type: Directory
  storageClassName: standard
status: {}

Your daemon pods will actually use /tmp/mem on each node. (There's at most 1 daemon pod on each node so that's fine.)

Omasum

Answer 2

The way to attach a PVC to your DaemonSet pod is not any different than how you do it with other types of pods. Create your PVC and mount it as a volume onto the pod.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: my-pvc
  namespace: x
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

This is what the DaemonSet manifest would look like:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: x
  namespace: x
  labels:
    k8s-app: x
spec:
  selector:
    matchLabels:
      name: x
  template:
    metadata:
      labels:
        name: x
    spec:
      ...
      containers:
      - name: x
        ...
        volumeMounts:
        - name: volume
          mountPath: /var/log
      volumes:
      - name: volume
        persistentVolumeClaim:
          claimName: my-pvc

Stepsister

an azure disk can be associated on to single node! so i need a way to have multiple pvcs – Kadner