Is it not prefer to use :local/:disk option on production environment when using ActiveStorage? Will it make me not able to backup the files?

Question

I am working on a project using Rails as my backend API server. Uploading clients' files will be one of the most important part of this application (it is something like a CRM/ERP system). However, my client prefer storing all the data and files into their own server due to security and privacy issues of their clients.

However, while I am reading the documents of ActiveStorage, it sounds like :disk option is just used for test and development environment. I understand using cloud storage like s3 would benefit on scalability and backup stuffs that are much secure and flexible for web development, but after all, you know, client's requirement.

1) Therefore, would like to know is it not preferred to use :disk on any production environment? What will be the cons that I may miss out?

Also, will it be hard for me to do backup for the files, as I saw in the /storage path, the files are all saved not into the same names of the original files.

I am having a guess that, could I just backup the whole sites by just doing pg_dump and a clone of the entire site directory including the /storage file (they will be gitignore, so I need to backup them by myself and do some git clone git pull stuff while doing recovery or server transition). Will this workflow work flawlessly?

2) What should be the actual backup and recover flow if I use :disk option in ActiveStorage?

Thanks for your help, and am appreciate any of your helps!

Bergerac

Answer 1

Disk and local are indeed not recommended for production.

If you lose the table contents or some of the files in storage/ you may not be able to recover your data.

The growing storage/ directory will make it difficult to move your application somewhere else as you'd have to copy all of the content along with the code.

It will also make it difficult scaling horizontally as the storage/ directory must be present on all instances of your applications and always in sync. You may counter this issue by setting up a NFS share somewhere and mounting under storage/ but that can come with some reliability issues - a timeout or permissions error when writing a file for instance will generate the ActiveStorage table entry but without the associated file => lots of annoying errors.

I believe it may also be rather difficult making incremental backups, you'd have to dump the table and zip it along with all of storage/ files, if something changes while you're backing up or restoring your data you'll experience all kinds of errors.

These are not really impossible to work around, just rather unfeasible.

You may want to check out Minio or a similar application. It gives you ActiveStorage support with none of the S3 costs and data privacy concerns. Just drop it on a docker instance somewhere on your network, set up persistence and backups/RAID and you're kind of done.

Vizierate

May also ask, if I would like to transfer all files from Minio to a real Amazon S3, is it possible? Is there a convenient way to keep the ActiveStorage association and do the transfer? – Bergerac

In theory it should be as simple as copying the directory into a bucket and updating your yml. I however have a hunch it can't be that simple, there always has to be something that doesn't work with these open source projects, so you might need to set up a script to iterate through the assets, upload them to S3 and update the active_storage_ table entries. It's not impossible, just a bit of a hassle. AS was developed with a very specific purpose in mind (Basecamp) so it may not cater exactly to every use case out there, nor be hassle-free to manage despite all the associated hype. – Vizierate

Besides that the files are not necessarily referenced by their filename only (and the additional step of redirecting to the uploaded images instead of just serving them), the downsides of using disk is what most "traditional" (few dependency) web applications have to deal with, imho. People have this "problems" since ever. Besides, I do not see where having the files in an external storage service makes the backup/restore process less tricky. And of course you could recover, but you will likely have to write some scripts to do so - a plain copy might not be enough. – Ockham