How to properly setup hostPath persistent volume on Minikube?

Question

I'm currently working on a Lumen project where we are using Minikube as our dev environment. Our host machine's /Users/development/<project name> is mounted at /var/www/html and is working fine. However, I'm facing this Storage issue where file writes are not working in the /var/www/html/storage/framework due to the fact that the entire /var/www/html directory has the 1001:1001 ownership.

This is my deployment spec:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: apiserver
  namespace: development
  labels:
    app: sample-app-name
spec:
  selector:
    matchLabels:
      app: sample-app-name
      tier: apiserver
  replicas: 1
  template:
    metadata:
      labels:
        app: sample-app-name
        tier: apiserver
    spec:
      containers:
      - name: php-app
        image: my-image:latest
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        imagePullPolicy: Never
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: "/var/www/html"
            name: host-mount
      imagePullSecrets:
        - name: dockercred
      volumes:
      - name: host-mount
        hostPath:
          path: "/Users/development/<app directory>"

I tried moving the persistent volume onto its own separate file, and had created a pvc, but still, it's not working. I also tried multiple ways on how to change the directory permissions using both init containers and security context, but it still the permissions are always set to **1001:1001* I'm really desperate here, so any help would be appreciated.

Host specs:

• OS: Ubuntu 18.04.3 LTS
• Minikube Version: v0.30.0
• Kubectl Client Version: Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
• Kubectl Server Version: Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
• Virtualbox Version: 5.2.34 r133893 (Qt5.9.5)

EDIT: (Here's docker file of the image i'm using in the Deployment)

FROM phpearth/php:7.1-nginx

RUN apk add --no-cache php7.1-redis php7.1-pdo php7.1-pdo_pgsql php7.1-xdebug composer bash

COPY ./nginx-default.conf /etc/nginx/conf.d/default.conf
COPY ./xdebug.ini /etc/php/7.1/conf.d/xdebug.ini
COPY ./www.conf /etc/php/7.1/php-fpm.d/www.conf
RUN mkdir -p /var/www/storage/import
RUN mkdir -p /var/www/storage/import/files
RUN mkdir -p /var/www/storage/import/templates
RUN mkdir -p /var/www/storage/logs
RUN mkdir -p /var/www/storage/framework/sessions
RUN mkdir -p /var/www/storage/framework/views
RUN touch /var/www/storage/logs/lumen.log
RUN chown -Rf 1000:1000 /var/www/
# Install the blackfire client
RUN version=$(php -r "echo PHP_MAJOR_VERSION.PHP_MINOR_VERSION;") \
    && curl -A "Docker" -o /tmp/blackfire-probe.tar.gz -D - -L -s http://packages.blackfire.io/binaries/blackfire-php/1.23.1/blackfire-php-alpine_amd64-php-71.tar.gz \
    && mkdir -p /tmp/blackfire \
    && tar zxpf /tmp/blackfire-probe.tar.gz -C /tmp/blackfire \
    && mv /tmp/blackfire/blackfire-*.so $(php -r "echo ini_get('extension_dir');")/blackfire.so \
    && printf "extension=blackfire.so\nblackfire.agent_socket=tcp://blackfire:8707\n" > /etc/php/7.1/conf.d/blackfire.ini \
    && rm -rf /tmp/blackfire /tmp/blackfire-probe.tar.gz

Summerlin

Answer 1

Turns out, this wasn't a mounting issue. I kept blaming the hostPath mount because when I try running ls -lah on /var/www, it kept showing the html directory's permissions as 1001:1001 instead of www-data.

In the end, it was PHP's user that wasn't running on the correct UID. Dumping posix_getpwuid(posix_geteuid()) shows the following result:

array:7 [
  "name" => "www-data"
  "passwd" => "x"
  "uid" => 82
  "gid" => 82
  "gecos" => "www-data"
  "dir" => "/var/www"
  "shell" => "/sbin/nologin"
]

But after adding this line in my Dockerfile: RUN apk add shadow && usermod -u 1000 www-data && groupmod -g 1000 www-data, it now shows this:

array:7 [
  "name" => "www-data"
  "passwd" => "x"
  "uid" => 1000
  "gid" => 1000
  "gecos" => "www-data"
  "dir" => "/var/www"
  "shell" => "/sbin/nologin"
]

I'm not having any permission issues now on my APIs.

Summerlin

Answer 2

From what you wrote I understand that your host machine's /Users/development/<project name> is mounted at /var/www/html in your VM so in kubernetes you should reference it with

volumes:
  - name: host-mount
    hostPath:
       path: "/var/www/html"

and also specifying security context like following should make it work

spec:
  securityContext:
    runAsUser: 1001
    runAsGroup: 1001

Let me know if it helped.

Tope

I just tried this, but still, it didn't work and my pod's status became Error. kubectl logs pod/<pod id> return this: /sbin/runit-wrapper: line 5: can't create /env: Permission denied By the way, I added my Dockerfile to see if it helps. – Summerlin

I can see that ` /sbin/runit-wrapper` is trying to create /env in the root tree so it needs root permissions to write to it, so modifying securityContext won't work because the container needs to start as root to be able to modify these files. I also noticed that you are using kubernetes v1.10 which is no longer supported version. Anyway try changing files owner on your host: chown 82:82 -R /Users/development/<project name>, this might not be the best solution but at least will make us sure what's the issue. – Tope

Also, what are you trying to achieve by running RUN chown -Rf 1000:1000 /var/www/ in you dockerfile as there is no user of id=1000 in the container? – Tope

Hi @HelloWorld, thank you so much for your assistance. I feel bad that this is very far from a Kubernetes issue. Please see my posted answer – Summerlin