How to use AWS CLI with Digital Ocean Spaces?

Question

How to use AWS CLI with Digital Ocean Spaces? Is it possible?

I tried to add my key id and secret to .aws/credentials but I got:

An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our record

Sharkey

Answer 1

In order to use the AWS CLI with object storage service that provide an S3-compatible API like DigitalOcean Spaces, you must also configure a custom endpoint.

This can be done on the command line using the --endpoint flag:

aws s3 ls --endpoint=https://nyc3.digitaloceanspaces.com

Catcall

Worth pointing out that there's no way currently to have a default endpoint. You gotta specify it every time. I usually alias it for convenience: alias awsdo='aws --endpoint=https://nyc3.digitaloceanspaces.com' – Gorgerin

Also, commands like aws s3 sync do not work with spaces endpoints, so the working alternative is using rclone. – Utah

@Niloct's comment may have been true at the time of posting, but it is no longer true as I am able to use aws s3 sync with spaces endpoints. – Gideon

@OnelHarrison thanks for your update. Anyway rclone has a very nice feature that only copies files changed in last n minutes without traversing the spaces file structure, and that is REALLY fast. I'm using it with success. – Utah

Answer 2

If you don't want to set the flag in each command, you can simply:

Set the AWS_ENDPOINT_URL_S3 environment variable as https://nyc3.digitaloceanspaces.com.

Then you can run the commands without the --endpoint flag, e.g. aws s3 ls.

Calamint