How can I mount an S3 volume with proper permissions using FUSE

Question

I have an Amazon S3 bucket (let's call it static.example.com) that I need to mount on an EC2 instance (Ubuntu 12.04.2). I've installed s3fs. I'm able to mount the volume, but I can't write to the bucket. I have tried:

sudo s3fs static.example.com -o use_cache=/tmp,allow_other,uid=33,gid=33 /mnt/static.example.com

I can then cd /mnt and ls -la to see:

drwxr-xr-x  5 root     root      4096 Mar 28 18:03 .
drwxr-xr-x 25 root     root      4096 Feb 19 19:22 ..
lrwxrwxrwx  1 root     root         7 Feb 21 19:19 httpd -> /httpd/
drwx------  2 root     root     16384 Oct  9  2012 lost+found
drwxr-xr-x  1 www-data www-data     0 Jan  1  1970 static.example.com

This all looks good, but when I cd static.example.com and mkdir test, I get:

mkdir: cannot create directory `test': Permission denied

The only way I can actually create a directory or touch a file is to force it with sudo. This is not a viable option, however, because I want to write files to the bucket from Apache. My Apache server runs as user:group www-data. Running mount yields:

s3fs on /mnt/static.example.com type fuse.s3fs (rw,nosuid,nodev,allow_other)

How can I mount this bucket in a manner that will allow me to write to the bucket?

Dylane

Ok, may be a silly question and I'd not ask it if you clearly said so, but you did run cd/mkdir as the actual user www-data, not as another user in the www-data group, right? – Crispi

@JoachimIsaksson Yes, I have tried sudo su www-data followed by the mkdir command. I got the same mkdir: cannot create directory `test': Permission denied – Dylane

It seems to me that you are doing it the right way. Please try this command: sudo -u www-data mkdir /mnt/static.example.com/test – Congressman

Answer 1

I'm the lead developer and maintainer of Open source project RioFS: a userspace filesystem to mount Amazon S3 buckets.

Our project is an alternative to “s3fs” project, main advantages comparing to “s3fs” are: simplicity, the speed of operations and bugs-free code. Currently the project is in the “beta” state, but it's been running on several high-loaded fileservers for quite some time.

We are seeking for more people to join our project and help with the testing. From our side we offer quick bugs fix and will listen to your requests to add new features.

Regarding your issue:

if'd you use RioFS, you could mount a bucket and have a write access to it using the following command (assuming you have installed RioFS and have exported AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables):

riofs  -o allow_other http://s3.amazonaws.com bucket_name /mnt/static.example.com

(please refer to project description for command line arguments)

Please note that the project is still in the development, there are could be still a number of bugs left.

If you find that something doesn't work as expected: please fill a issue report on the project's GitHub page.

Hope it helps and we are looking forward to seeing you joined our community !

Indreetloire

@Wizzard This makes all the files show up as rw-r--r--, owned by my user (not www-data). How can I get the files to allow write for the group, or be owned by www-data? – Dardanelles

After reading the README more carefully (duh), I saw the --uid and --gid options. Due to a bug in my application, I'm not sure if using --fmode 0664 and --dmode 0775 were needed, but I used them. – Dardanelles

Last update: 6y ago... Well then, s3fs it is. – Shields

Answer 2

This works for me:

sudo s3fs bucketname /mnt/folder -o allow_other,nosuid,use_cache=/mnt/foldercache

If you need to debug, just add ,f2 -f -d:

sudo s3fs bucketname /mnt/folder -o allow_other,nosuid,use_cache=/mnt/foldercache,f2 -f -d

Leesa

Answer 3

Try this method using S3Backer:

mountpoint/
    file       # (e.g., can be used as a virtual loopback)
    stats      # human readable statistics

Read more about it hurr: http://www.turnkeylinux.org/blog/exploring-s3-based-filesystems-s3fs-and-s3backer

Ruelle