Syncing Azure BLOB Storage to Amazon S3

Question

We're storing about 4 million files (4 TB or so) of miscellaneous files, mainly Word and PDF, in Azure BLOB storage. I'm looking to replicate this data in a different cloud for disaster recovery and peace of mind, and Amazon S3 seems as good a candidate as any.

Trouble is, I don't have a local server large enough to hold a local copy of these files. Ideally, I'd want to sync right from Azure Blob to S3. We're adding new files continually, so the sync would need to be frequent as well (multiple times per day).

I see lots of options for download from Azure to local => upload from local to S3, but very little for direct Azure => S3 sync. What are some good options here?

Crossway

Are you able to create VM (e.g. EC2 instances) in the cloud? If so, Skyplane (github.com/skyplane-project/skyplane) can automatically create ephemeral VMs to execute the cross-cloud transfer. – Closestool

Answer 1

We can migrate the azure storage data to amazon s3 by node.js package. You can see the full description provided here.

You can also use azure data factory to replicate as it provides a copy tool which can be modified according to your needs and settings for transferring data . You can refer to this document on Azure data factory and copy tool.

Tripodic