To have truly off-site and durable backups of my ZFS pool, I would like to store zfs snapshots in Amazon Glacier. The data would need to be encrypted locally, independently from Amazon, to ensure privacy. How could I accomplish this?
Storing locally encrypted incremental ZFS snapshots in Amazon Glacier
Asked Answered
An existing snapshot can be sent to a S3 bucket as following:
zfs send -R <pool name>@<snapshot name> | gzip | gpg --no-use-agent --no-tty --passphrase-file ./passphrase -c - | aws s3 cp - s3://<bucketname>/<filename>.zfs.gz.gpg
or for incremental back-ups:
zfs send -R -I <pool name>@<snapshot to do incremental backup from> <pool name>@<snapshot name> | gzip | gpg --no-use-agent --no-tty --passphrase-file ./passphrase -c - | aws s3 cp - s3://<bucketname>/<filename>.zfs.gz.gpg
This command will take an existing snapshot, serialize it with zfs send, compress it, and encrypt it with a passphrase with gpg. The passphrase must be readable on the first line in the ./passphrase file.
Remember to back-up your passphrase-file separately in multiple locations! - If you lose access to it, you'll never be able to get to your data again!
This requires:
- A pre-created Amazon s3 bucket
- awscli installed (
pip install awscli) and configured (aws configure). - gpg installed
Lastly, S3 lifecycle rules can be used to transition the S3 object to glacier after a pre-set amount of time (or immediately).
For restoring:
aws s3 cp s3://<bucketname>/<filename>.zfs.gz.gpg - | gpg --no-use-agent --passphrase-file ./passphrase -d - | gunzip | sudo zfs receive <new dataset name>
I'd recommend using
gzip -9 or bzip2. Depending on the data, that might compress better. The compression won't be the bottleneck here - the network copy will. –
Gouty please, fix the mistake in For restoring: section, in s3 cp src and targert –
Elvinaelvira
Sorry about that, fixed now. –
Iridium
© 2022 - 2024 — McMap. All rights reserved.
0.8, ZFS On Linux supports native encryption. Perhaps thezfs send -Rwcommand is exactly what you'd want – Soubise