How to access PersistentVolume files on docker-for-desktop?

Question

I'd like to access and edit files in my Kubernetes PersistentVolume on my local computer (macOS), but I cannot understand where to find those files!

I'm pointing my hostPath to /tmp/wordpress-volume but I cannot find it anywhere. What is the hidden secret I'm missing

I'm using the following configuration on a docker-for-desktop cluster Version 2.0.0.2 (30215).

PersistentVolume

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  hostPath:
    path: /tmp/wordpress-volume

PersistentVolumeClaim

kind: PersistentVolumeClaim
metadata:
  name: wordpress-volume-claim
# ...

Deployment

kind: Deployment
metadata:
  name: wordpress
# ...
spec:
  containers:
  - image: wordpress:4.8-apache
    # ...
    volumeMounts:
    - name: wordpress-volume
      mountPath: /var/www/html
  volumes:
  - name: wordpress-volume
    persistentVolumeClaim:
      claimName: wordpress-volume-claim

Octofoil

Answer 1

Thanks to @aman-tuladhar and some hours lost on the internet I've found out that you just need to make sure storageClassName is set for you PersistentVolume and PersistentVolumeClaim.

As per documentation if you want to avoid that Kubernetes dynamically generetes PersistentVolumes without considering the one you statically declared, you can just set a empty string " ".

In my case I've set storageClassName: manual.

PersistentVolume

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  storageClassName: manual
  hostPath:
    path: /tmp/wordpress-volume

PersistentVolumeClaim

kind: PersistentVolumeClaim
metadata:
  name: wordpress-volume-claim
spec:
  storageClassName: manual
  # ...

This works out of the box with docker-for-desktop cluster (as long as mountPath is set to a absolute path).

References:

• Kubernetes: Binding PersistentVolumes and PersistentVolumeClaims
• Storing data into Persistent Volumes on Kubernetes

Octofoil

This answer doesn't work in the latest version of Docker Desktop for macOS (v4.1.1 as on date). It ends up using the said folder from within the Docker VM. That said, the local-storage seems to be doing the same thing. – Callista

Correcting my partially incorrect comment. The solution works, however, does not work for any non-user owned directory. Docker Desktop silently mounts the folder from Docker VM if it is not a user owned directory chain (example: /tmp/xyz). However, works for /Users/$(whoami)/. It is extremely frustrating that this mechanism is not documented anywhere. – Callista

For anyone surprised they can’t find the docs on this: manual can be pretty much any string, e.g., foo, and this still works. As well you’ll probably want to add some selectors to ensure the PVC will only get attached to the intended PV. – Pelagias

PS: the pvc spec volumeName only ‘works’ when specified at resource creation time if the pv spec claimRef.name and claimRef.namespace for the intended pv match. But if you leave it blank you can read the pv name from that field once it is bound. – Pelagias

Answer 2

First this you need to remember is that Kubernetes is running on minikube cluster. minikube itself run on Virtual Machine. So that path won't be on you host machine, rather it is the path in Virtual Machine.

But with minikube we have easy way to do this. First you have to mount host directory to minikube.

(If you are using cloud providers you will have some way to create a storage. For GCE you have gcePersistentDisk)

minikube mount /path/to/dir/to/mount:/vm-mount-path

Now

kind: PersistentVolume
metadata:
  name: wordpress-volume
spec:
  # ...
  hostPath:
    path: /vm-mount-path

If you create this resource this should save file in your host machine.

Follow this minikube documentation for more detail

Renascent

I was trying to use docker-for-desktop, but at this point I'll switch to Minikube which seems to have more documentation on the matter. – Octofoil

Uh, do you remember where you saw it? I found out this article that says: "in order for the pvc to actually bind to the pv, you need to give both of them the same storageClassName in the yaml definition." – Octofoil

I don't remember where but, if you are not using StorageClass Resource may be you need to add spec.storageClassName: "" in a yaml where you are creating PVC. It ensures to use existing PV instead of creating new one. – Renascent

Yes, I've found another article talking about the same thing here – Octofoil

Answer 3

In case of MacOS and Kubernetes inside Docker for Mac. How to find a real location of dir-based local volume into VM

1) Create a new PersistentVolume with unique path:

blablabla.yml:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: blablabla
spec:
  storageClassName: manual
  capacity:
    storage: 1G
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/blablabla"

kubectl apply -f blablabla.yml

2) Log into VM:

screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
# then press Enter

3) Find your volume:

find / -name blablabla
/containers/services/docker/rootfs/mnt/blablabla # <= got it!
/containers/services/docker/tmp/upper/mnt/blablabla

4) Exit from screen: Ctrl-a k y, Detach from screen: Ctrl-a d

Note

Sometimes you have a chance to get a broken screen session, it seems like a garbaged stdout with messed symbols, stdin still works fine. In that case, try to terminate all screen sessions and reconnect to the first one. Or just restart you docker for mac.

Deputy

In case of the question path (/tmp/wordpress-volume) it could be find on /containers/services/docker/rootfs/tmp/wordpress-volume. Having a K8S IDE like Lens allows you to log into the VM/Node without using screen. – Pittel

In latest macOS versions tty (from Step 2) will not work. Alternate is; docker run -it --privileged --pid=host justincormack/nsenter1 – Dawdle

Answer 4

I create a PersistentVolume and use the storageClassName (local-storage in my example below). Make sure to replace the path (/Users/user/data-pv1) with an actual path on your Mac.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-pv1
spec:
  capacity:
    storage: 30Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /Users/user/data-pv1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - docker-desktop

---

If you'd like to access the hostPath, you need to use nsenter:

docker run -it --rm --privileged --pid=host alpine:edge nsenter -t 1 -m -u -n -i sh

or nsenter1

docker run -it --rm --privileged --pid=host justincormack/nsenter1

and navigate to the following directory:

/var/lib/k8s-pvs

Grillage

Thank you sir, this answer fixed it for me. Pay close attention to /spec/local/path Most examples use a hostPath/path setting, which only created the folder on the docker container host. – Quadrireme

Correcting my partially incorrect comment. The answer (and the answer marked as the accepted, works, however, does not work for any non-user owned directory. Docker Desktop silently mounts the folder from Docker VM if it is not a user owned directory chain (example: /tmp/xyz). However, works for /Users/$(whoami)/. It is extremely frustrating that this mechanism is not documented anywhere. – Callista