Docker build failing when using gcsfuse to mount google storage

Question

I have been trying to mount SQL and a storage bucket to my docker WordPress container. It appears to succeeding in mounting SQL, but failing mounting the bucket. The instance is based of of this post.

I have attached the Docker file and error below, as well as my build command.

Build command:

docker build -t ic/spm .

Dockerfile:

FROM wordpress
MAINTAINER Gareth Williams <[email protected]>

# Move login creds locally
ADD ./creds.json /creds.json

# install sudo, wget and gcsfuse
ENV GCSFUSE_REPO=gcsfuse-jessie
RUN   apt-get update && \
      apt-get -y install sudo && \
      apt-get install -y curl ca-certificates && \
      echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" > /etc/apt/sources.list.d/gcsfuse.list && \
      curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
      apt-get update && \
      apt-get install -y gcsfuse wget && \
      apt-get remove -y curl --purge && \
      apt-get autoremove -y && \
      rm -rf /var/lib/apt/lists/*

# Config fuse
RUN chmod a+r /etc/fuse.conf
RUN perl -i -pe 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf

# Setup sql proxy
RUN sudo mkdir /cloudsql
RUN sudo chmod 777 /cloudsql
ADD https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 cloud_sql_proxy.linux.amd64
RUN mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy && chmod +x ./cloud_sql_proxy
RUN ./cloud_sql_proxy -dir=/cloudsql -fuse -credential_file=/creds.json &
# mysql -u icroot -S /cloudsql/[INSTANCE_CONNECTION_NAME]

# Perform Cloud Storage FUSE mounting for uploads folder
RUN mkdir /mnt/uploads
RUN chmod a+w /mnt/uploads
#RUN chown www-data:www-data -R /mnt && groupadd fuse && gpasswd -a www-data fuse && chmod g+rw /dev/fuse
USER www-data
RUN gcsfuse --key-file /creds.json \
  --debug_gcs --debug_http --debug_fuse --debug_invariants \
  --dir-mode "777" -o allow_other spm-bucket /mnt/uploads

Error:

Step 17 : RUN gcsfuse --key-file /creds.json   --foreground --debug_gcs --debug_http --debug_fuse --debug_invariants   --dir-mode "777" -o allow_other spm-bucket /mnt/uploads
 ---> Running in 7e3f31221bee
Using mount point: /mnt/uploads
Opening GCS connection...
Opening bucket...
gcs: Req              0x0: <- ListObjects()
http: ========== REQUEST:
GET http://www.googleapis.com/storage/v1/b/spm-bucket/o?maxResults=1&projection=full HTTP/1.1
Host: www.googleapis.com
User-Agent: gcsfuse/0.0
Authorization: Bearer ya29.ElrQAw8oxClKt8YGvtmxhc7z2Y2LufvL0fBueq1UESjYYjRrdxukNTQqO1qfM8e8h-rqfbOWNSjVK2rCRXVrEDla-CiUVhHwT6X71Y1Djb0jDJg7z3KblgNQPrc
Accept-Encoding: gzip

http: ========== RESPONSE:
HTTP/2.0 200 OK
Content-Length: 31
Alt-Svc: quic=":443"; ma=2592000; v="35,34"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Wed, 11 Jan 2017 09:19:05 GMT
Expires: Wed, 11 Jan 2017 09:19:05 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: AEnB2UpTqXhtHW906FFDTRsz4FjHjFu_E84wYhvt0zhaVFuMpqSY1fsd1XcrEcpsYBBwX1mqf0ZXRVWJH05ThtDQIfFKHd4PFw

{
 "kind": "storage#objects"
}
http: ====================
gcs: Req              0x0: -> ListObjects() (1.793169206s): OK
Mounting file system...
mountWithArgs: mountWithConn: Mount: mount: running fusermount: exit status 1

stderr:
fusermount: failed to open /dev/fuse: Operation not permitted

Polymath

You have an authentication problem. I have posted a similar q with a bit more detail. Be helpful to see the results of gcloud info, have you authenticated your service account? Can you simply add gcloud auth activate-service-account --key-file <path to the .json> #41685851 – Soembawa

@Soembawa I just tried and that appears to be fine, I can auth. I can only think that it is something to do with fuse, but I have no idea what it could be. – Polymath

On what platform are you running your docker? – Campaign

@Campaign I'm using Mac OSX, and then hoping to push up the image to Google and use via kubernetes – Polymath

Your Dockerfile has a commented-out RUN line that appears like it should address the issue? What happens when you uncomment it? – Blooper

Docker machine or Docker for Mac? If it's the former, what VM driver? – Campaign

@GrishaLevit The response is the same. The commented line was something I tried to resolve fuse permissions, but it seems 'docker build' does not run with sufficient privileges to run fuse. You need to use 'docker run --privileged' – Polymath

@Campaign docker for mac. – Polymath

Answer 1

If you're running your container on GKE, and you want to use gcsfuse, permissions should automatically be inherited in your account locally. Also...there is a caveat that you need to make sure that the cluster your running needs to have storage access. So make sure your cluster has the storage permissions set to full access. That way gcsfuse can mount your buckets on GCS within the container without having to worry about passing credential files and all that stuff...making the implementation pretty straight forward.

In your docker file...make sure you're doing your apt commands to get and install the gcsfuse application.

I personally made a shell script that I call once the instance is up, that mounts my directories that I needed.

Something like this...

Docker Entry

ENTRYPOINT ["/opt/entry.sh"]

entry.sh script example

gcsfuse [gcs bucket name] [local folder to mount as]

When generating your GKE cluster, make sure to add the storage scope

gcloud container clusters create [your cluster name] --scopes storage-full

Hope this helps you.

Niello

Answer 2

Docker won't allowed to mount with other storages(like GCP) by default. What you can do is when running the container with privileged option you can mount with the storage.

Put this command in script file(gcp.sh) and build the docker image.

RUN gcsfuse --key-file /creds.json \
  --debug_gcs --debug_http --debug_fuse --debug_invariants \
  --dir-mode "777" -o allow_other spm-bucket /mnt/uploads

gcp.sh:

gcsfuse --key-file /creds.json --debug_gcs --debug_http --debug_fuse --debug_invariants --dir-mode "777" -o allow_other spm-bucket /mnt/uploads

and the Dockerfile:

FROM wordpress
MAINTAINER Gareth Williams <[email protected]>

# Move login creds locally
ADD ./creds.json /creds.json

# install sudo, wget and gcsfuse
ENV GCSFUSE_REPO=gcsfuse-jessie
RUN   apt-get update && \
      apt-get -y install sudo && \
      apt-get install -y curl ca-certificates && \
      echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" > /etc/apt/sources.list.d/gcsfuse.list && \
      curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
      apt-get update && \
      apt-get install -y gcsfuse wget && \
      apt-get remove -y curl --purge && \
      apt-get autoremove -y && \
      rm -rf /var/lib/apt/lists/*

# Config fuse
RUN chmod a+r /etc/fuse.conf
RUN perl -i -pe 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf

# Setup sql proxy
RUN sudo mkdir /cloudsql
RUN sudo chmod 777 /cloudsql
ADD https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 cloud_sql_proxy.linux.amd64
RUN mv cloud_sql_proxy.linux.amd64 cloud_sql_proxy && chmod +x ./cloud_sql_proxy
RUN ./cloud_sql_proxy -dir=/cloudsql -fuse -credential_file=/creds.json &
# mysql -u icroot -S /cloudsql/[INSTANCE_CONNECTION_NAME]

# Perform Cloud Storage FUSE mounting for uploads folder
RUN mkdir /mnt/uploads
RUN chmod a+w /mnt/uploads
#RUN chown www-data:www-data -R /mnt && groupadd fuse && gpasswd -a www-data fuse && chmod g+rw /dev/fuse
USER www-data
COPY gcp.sh /home
RUN chmod +x /home/gcp.sh
CMD cd /home && ./gcp.sh

and finally after build the image run the container with --privileged option docker run --privileged

Cultch

Answer 3

your www-data have permission problem in the dockerfile:

#RUN chown www-data:www-data -R /mnt && groupadd fuse && gpasswd -a www-data fuse && chmod g+rw /dev/fuse

uncomment this line

Haroldson

Than line had previously been used and reinstating has not work. The issue appears to be to do with docker privileges. There is no flag to run as --privileged when building – Polymath

this is been an issue for long time, check this out, in summary, you can only work around by running a container with increased capabilities, installing required software, and then creating an image from that container. – Haroldson

This should be the actual answer. – Kero