The AWS Access Key Id does not exist in our records

I

37

155

I created a new Access Key and configured that in the AWS CLI with aws configure. It created the .ini file in ~/.aws/config. When I run aws s3 ls it gives:

A client error (InvalidAccessKeyId) occurred when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

AmazonS3FullAccess policy is also attached to the user. How to fix this?

Item answered 20/8, 2016 at 7:5 Comment(7)

is there a file ~/.aws/credentials and does it contain the correct id and key? – Hardship 20/8, 2016 at 7:29

No only config file is created with aws configure – Item 20/8, 2016 at 7:31

I am facing the same issue mentioned here. I have got credentials file created and the credentials are verified for validity. Couldn't find anything wrong. Any help? – Enteric 17/1, 2020 at 15:53

I've faced this issue on a Lambda function uploading files to S3 and managed to solve by configuring S3 object using the endpoint instead of the AWS credentials. – Amesace 21/4, 2021 at 13:55

@Amesace could you explain how you solve in detail – Mohawk 8/12, 2021 at 11:9

@HimalAcharya I did something like this: aws --endpoint-url=https://<your_s3_endpoint_url> s3 ls, please check Abhinav Sureka response. – Amesace 20/12, 2021 at 9:28

I've encountered the same error with S3cmd CLI because I have the wrong bucket_host address – Danielldaniella 20/11, 2023 at 9:22

A

86

It looks like some values have been already set for the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

If it is like that, you could see some values when executing the below commands.

echo $AWS_SECRET_ACCESS_KEY
echo $AWS_ACCESS_KEY_ID

You need to reset these variables, if you are using aws configure

To reset, execute below commands.

unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY

Athanasian answered 16/8, 2017 at 1:53 Comment(3)

In my case, I had to configure AWS credentials using cli (All problems came after I revoked IAM credentials and added new credentials. Even after adding and modifying .aws/credentials file, I wasn't able to perform as before. ) Solution: configure credentials as below https://docs.aws.amazon.com/cli/latest/userguide/welcome-examples.html -------------------------------------

$ aws configure AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: us-west-2 Default output format [None]: ENTER

– Barrick 18/1, 2023 at 0:23

This was my problem! I had up to date credentials in ~/.aws/credentials, but needed to update the values in my ~/.zshrc file (Mac OS X) and then run source ~/.zshrc to pick up the changes. – Gewirtz 19/7, 2023 at 17:6

Running aws configure list helped me to identify which credentials are being used (env/shared-credentials-file). In case of credentials are being picked from the env variable, the type column in the output of the aws configure list would be env, otherwise, it would be shared-credentials-file. – Cormorant 19/8, 2023 at 11:47

W

79

It might be happening that you have the old keys exported via env variables (bash_profile) and since the env variables have higher precedence over credential files it is giving the error "the access key id does not exists".

Remove the old keys from the bash_profile and you would be good to go.

Happened with me once earlier when I forgot I have credentials in bash_profile and gave me headache for quite some time :)

Withdrawal answered 25/1, 2017 at 11:18 Comment(2)

this should be the accepted answer. in my case, I was mounting host keys but there was already empty variables define an ENV for AWS keys. – Psychopathy 27/12, 2019 at 14:47

how to check bash_profile ? If wehave any keys there? please help – Procto 22/4, 2020 at 5:2

O

77

Need to add aws_session_token in credentials, along with aws_access_key_id,aws_secret_access_key

Once answered 16/4, 2020 at 11:17 Comment(8)

Can u provide an example for aws_session_token. – Outguard 2/3, 2021 at 5:0

assumeRoleResult = AssumeRole(role-arn); tempCredentials = new SessionAWSCredentials(    assumeRoleResult.AccessKeyId,     assumeRoleResult.SecretAccessKey,     assumeRoleResult.SessionToken); s3Request = CreateAmazonS3Client(tempCredentials);

docs.aws.amazon.com/IAM/latest/UserGuide/… – Cochard 17/6, 2021 at 20:15

This worked for me. Thanks!! A little strange though, since access key and secret key used to be enough. – Stidham 22/9, 2022 at 5:38

Worked for me. After add aws_session_token, the request headers will have a X-Amz-Security-Token, and everything goes fine. – Unbelievable 20/10, 2022 at 3:48

To configure aws_session_token in the aws configure method, try: 1) run aws configure 2) edit file in ~/.aws/credentials 3) add a new line with aws_session_token=<YOUR_SESSION_TOKEN> 4) close the terminal and open it again – Aerodrome 17/2, 2023 at 22:25

I got a temporary token using aws sts assume-role-with-saml and it came back with aws_access_key_id, aws_secret_access_key along with aws_session_token. I put those credentials in Config file like- [TempToken] aws_access_key_id = your-key aws_secret_access_key = your-secret aws_session_token = your-session-token region=us-east-1 Then I ran the command s3 and s3api with profile and both worked: aws s3 ls --profile TempToken aws s3api list-buckets --profile TempToken – Redress 26/4, 2023 at 4:34

Thank you. The official web is: boto3.amazonaws.com/v1/documentation/api/latest/guide/… – Hawkshaw 13/7, 2023 at 22:37

Worked for me as well. I added the session token to ~/.aws/credentials. That is of course temporary (needs to be updated when the session token changes) but at least I don't have to worry about setting headers. – Arianism 17/4 at 9:12

P

18

None of the up-voted answers work for me. Finally I pass the credentials inside the python script, using the client API.

import boto3
client = boto3.client(
's3',
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
aws_session_token=SESSION_TOKEN)

Please notice that the aws_session_token argument is optional. Not recommended for public work, but make life easier for simple trial.

Patinated answered 24/11, 2017 at 19:35 Comment(6)

If you're using EC2/Lambda/etc. you'll get temporary security credentials which means you'll need to use AWS_SESSION_TOKEN environment variable and pass it to boto3. I'm pretty sure it's not optional, because omitting it immediately throws InvalidAccessKeyId. – Honky 10/12, 2017 at 16:8

The solution works in my local jupyter notebook, and it has not been tested for a EC2/Lambda environment. Thanks for the experiments you have done, @HendyIrawan, do you test for both EC2 and Lambda(or other environments if any)? – Patinated 11/12, 2017 at 18:40

Yep the session token must be used in this case. – Chloroform 30/1, 2020 at 18:24

using okta and gimme-aws-creds. this is what I needed – Brazzaville 20/3, 2020 at 12:10

Setting up the "aws_session_token" in credential file also solved my problem. – Kare 14/1, 2021 at 8:37

Helped me developing lambda code in VSCode – Jackinthepulpit 30/6, 2021 at 2:6

D

13

You may have configured AWS credentials correctly, but using these credentials, you may be connecting to some specific S3 endpoint (as was the case with me).

Instead of using:

aws s3 ls

try using:

aws --endpoint-url=https://<your_s3_endpoint_url> s3 ls

Hope this helps those facing the similar problem.

Doth answered 30/11, 2020 at 9:4 Comment(2)

worked for me after setting up localstack – Primitivism 25/2, 2021 at 5:54

--endpoint-url is not supported (any more). Try aws s3 ls s3://your-bucket-resource --profile <profile_name> instead – Gaultiero 2/1, 2023 at 20:58

S

9

For me, I was relying on IAM EC2 roles to give access to our machines to specific resources.

I didn't even know there was a credentials file at ~/.aws/credentials, until I rotated/removed some of our accessKeys at the IAM console to tighten our security, and that suddenly made one of the scripts stop working on a single machine.

Deleting that credentials file fixed it for me.

Spillman answered 8/10, 2017 at 14:54 Comment(2)

this worked for me in the context of Terragrunt, Terraform, and AWS IAM Identity Center via cli command aws sso login – Straley 1/12, 2022 at 19:19

Thank you! When we ran our laravel code through an artisan command it seems to have used those instead of the IAM role – Spirit 12/5, 2023 at 9:43

A

7

I made the mistake of setting my variables with quotation marks like this:

AWS_ACCESS_KEY_ID="..."

Angio answered 1/11, 2020 at 17:39 Comment(1)

For me, it worked with the quotation marks. – Stroll 2/9, 2021 at 7:8

H

5

I have been looking for information about this problem and I have found this post. I know it is old, but I would like to leave this post in case anyone has problems.

Okay, I have installed the AWS CLI and opened:

It seems that you need to run aws configure to add the current credentials. Once changed, I can access

Highjack answered 20/8, 2016 at 7:6 Comment(1)

This was my problem, I had previously exported the environment variables, when I ran aws configure list it listed the old credentials I had. Hope this helps someone else. export AWS_ACCESS_KEY_ID=[access_id] && export AWS_SECRET_ACCESS_KEY=[secret_key] – Fredia 13/1, 2022 at 4:24

P

5

You can configure profiles in the bash_profile file using:

<profile_name>
aws_access_key_id = <access_key>
aws_secret_access_key = <acces_key_secret>

if you are using multiple profiles. Then use:

aws s3 ls --profile <profile_name>

Pushy answered 17/11, 2017 at 7:7 Comment(1)

Hmm... this did it for me. But, I'm curious why as I have logged in with the profile using: aws-azure-login --profile profilename --mode gui --enable-chrome-seamless-sso false. Unsure why the --profile profilename is required after the aws command – Boatload 15/9, 2022 at 21:36

M

3

You may need to set the AWS_DEFAULT_REGION environment variable.

Myo answered 15/1, 2020 at 5:2 Comment(0)

P

3

TRY THIS

The AWS Access Key Id you provided does not exist in our records.

One of the reasons for getting the above error is when the aws default region setted through the aws configure is different from the actual account region.

So check the default region in the aws configure and the account region on the concole.

Personality answered 30/10, 2022 at 15:42 Comment(1)

Just saved me a long morning of trying to debug this..! Thank you :-) – Chairborne 7/8, 2023 at 12:1

E

2

In my case, I was trying to provision a new bucket in Hong Kong region, which is not enabled by default, according to this: https://docs.aws.amazon.com/general/latest/gr/s3.html

It's not totally related to OP's question, but to topic per se, so if anyone else like myself finds trapped on this edge case:

I had to enable that region manually, before operating on that AWS s3 region, following this guide: https://docs.aws.amazon.com/general/latest/gr/rande-manage.html

Epiblast answered 10/9, 2021 at 14:29 Comment(0)

H

1

Looks like ~/.aws/credentials was not created. Try creating it manually with this content:

[default]
aws_access_key_id = sdfesdwedwedwrdf
aws_secret_access_key = wedfwedwerf3erfweaefdaefafefqaewfqewfqw

(on my test box, if I run aws command without having credentials file, the error is Unable to locate credentials. You can configure credentials by running "aws configure".) Can you try running these two commands from the same shell you are trying to run aws:

$ export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
$ export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

and then try aws command.

Hardship answered 20/8, 2016 at 7:40 Comment(2)

@DusanBajic the accesskeyid "does not exist in our records" is an error I have encountered during testing when I deliberately modified the value of the key in a signed URL while trying to simulate various failures. It implies that a key was indeed used but that IAM claims it doesn't exist. Assuming no copy/paste error, this response suggests -- to me -- that since the credentials were newly created, then it may be the case that IAM is/was experiencing a transient replication delay that will sort itself out by waiting, rather than the config file being missing. – Tail 20/8, 2016 at 10:30

So it must be replication delay. When I changed the AWS_DEFAULT_REGION to us-east-1 it started working. – Item 21/8, 2016 at 9:11

H

1

another thing that can cause this, even if everything is set up correctly, is running the command from a Makefile. for example, I had a rule:

awssetup:
        aws configure
        aws s3 sync s3://mybucket.whatever .

when I ran make awssetup I got the error: fatal error: An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: The AWS Access Key Id you provided does not exist in our records.. but running it from the command line worked.

Headwaiter answered 25/1, 2017 at 5:23 Comment(0)

F

1

Adding one more answer since all the above cases didn't work for me.

In AWS console, check your credentials(My Security Credentials) and see if you have entered the right credentials.

Thanks to this discussion: https://forums.aws.amazon.com/message.jspa?messageID=771815

Flattery answered 12/9, 2019 at 8:34 Comment(0)

B

1

This could happen because there's an issue with your AWS Secret Access Key. After messing around with AWS Amplify, I ran into this issue. The quickest way is to create a new pair of AWS Access Key ID and AWS Secret Access Key and run aws configure again. I works for me. I hope this helps.

Bruell answered 21/4, 2020 at 16:11 Comment(1)

for me, I had to delete the amplify backend folder, re-configure and then it worked – Brest 10/3 at 10:5

B

1

To those of you who run aws s3 ls and getting this exception. Make sure You have permissions to all regions under the provided AWS Account. When running aws s3 ls you try to pull all the s3 buckets under the AWS Account. therefore, in case you don't have permissions to all regions, you'll get this exception - An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.

Follow Describing your Regions using the AWS CLI for more info.

Buxtehude answered 21/7, 2020 at 9:19 Comment(0)

A

1

I had the same problem in windows and using the module aws-sdk of javascript. I have changed my IAM credentials and the problem persisted even if i give the new credentials through the method update like this

s3.config.update({
    accessKeyId: 'ACCESS_KEY_ID',
    secretAccessKey: 'SECRET_ACCESS_KEY',
    region: 'REGION',
});

After a while i found that the module aws-sdk had created a file inside the folder User on windows with this path

C:\Users\User\.aws\credentials

. The credentials inside this file take precedence over the other data passed through the method update.

The solution for me was to write here

C:\Users\User\.aws\credentials

the new credentials and not with the method s3.config.update

Author answered 16/2, 2021 at 9:39 Comment(0)

S

1

Kindly export the below variables from the credential file from the below directory.

path = .aws/
filename = credentials

export aws_access_key_id = AK###########GW
export aws_secret_access_key = g#############################J

Smear answered 6/3, 2021 at 19:7 Comment(0)

A

1

In my case, I was using aws configure

However, I hand-edited the .aws/config file to export the KeyID and key environment variables.

This apparently caused a silent error and saw the error listed above.

I solved this by destroying the .aws directory and running aws configure again.

Antecedence answered 1/10, 2021 at 19:8 Comment(1)

"I solved this by destroying the .aws directory and running aws configure again." - this solved the problem for me, except that I did not have to run aws configure. After deleting the directory aws s3 ls just worked. – Samekh 29/1 at 16:46

R

1

Hopefully this saves others from hours of frustration:

call aws.config.update({ before initializing s3.

const AWS = require('aws-sdk');

AWS.config.update({
    accessKeyId: 'AKIAW...',
    secretAccessKey: 'ptUGSHS....'
});

const s3 = new AWS.S3();

Credits to this answer:

https://mcmap.net/q/152946/-invalidaccesskeyid-while-trying-to-upload-to-s3

Roxana answered 16/11, 2021 at 9:57 Comment(0)

L

1

If you happen to use JupyterLab and get this error, after setting up the environment variables for AWS_PROFILE, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY, be sure to Restart Kernel... from the Kernel menu of your notebook.

Leong answered 13/2, 2023 at 19:9 Comment(0)

B

0

I tries below steps and it worked: 1. cd ~ 2. cd .aws 3. vi credentials 4. delete aws_access_key_id = aws_secret_access_key = by placing cursor on that line and pressing dd (vi command to delete line).

Delete both the line and check gain.

Boaz answered 31/5, 2018 at 20:50 Comment(1)

is this on the server or the local pc? – Plight 18/1, 2019 at 11:51

C

0

If you have an AWS Educate account and you get this problem:

An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records".

The solution is here:

Go to your C:/ drive and search for .aws folder inside your main folder in windows.
Inside that folder you get the "credentials" file and open it with notepad.
Paste the whole key credential from AWS account to the same notepad and save it.
Now you are ready to use you AWS Educate account.

Counterpoison answered 20/12, 2019 at 10:37 Comment(0)

M

0

Assuming you already checked Access Key ID and Secret... you might want to check file team-provider-info.json which can be found under amplify/ folder

"awscloudformation": {
      "AuthRoleName": "<role identifier>",
      "UnauthRoleArn": "arn:aws:iam::<specific to your account and role>",
      "AuthRoleArn": "arn:aws:iam::<specific to your account and role>",
      "Region": "us-east-1",
      "DeploymentBucketName": "<role identifier>",
      "UnauthRoleName": "<role identifier>",
      "StackName": "amplify-test-dev",
      "StackId": "arn:aws:cloudformation:<stack identifier>",
      "AmplifyAppId": "<id>"
    }

IAM role being referred here should be active in IAM console.

Metts answered 19/10, 2020 at 17:40 Comment(0)

R

0

If you get this error in an Amplify project, check that "awsConfigFilePath" is not configured in amplify/.config/local-aws-info.json

In my case I had to remove it, so my environment looked like the following:

{
  // **INCORRECT**
  // This will not use your profile in ~/.aws/credentials, but instead the
  // specified config file path
  // "dev": {
  //  "configLevel": "project",
  //  "useProfile": false,
  //  "awsConfigFilePath": "/Users/dev1/.amplify/awscloudformation/cEclTB7ddy"
  // },
  // **CORRECT**
  "dev": {
    "configLevel": "project",
    "useProfile": true,
    "profileName": "default",
  }
}

Ria answered 22/10, 2020 at 10:45 Comment(0)

D

0

Maybe you need to active you api keys in the web console, I just saw that mine were inactive for some reason...

Disembarrass answered 19/12, 2020 at 18:12 Comment(0)

R

0

Thanks, everyone. This helped to solve.

Something somehow happened which changed the keys & I didn't realize since everything was working fine until I connected to S3 from a spark...then from the command line also error started coming even in AWS s3 ls

Steps to solve

Run AWS configure to check if keys are set up (verify from last 4 characters & just keep pressing enter)
AWS console --> Users --> click on the user --> go to security credentials--> check if the key is the same that is showing up in AWS configure
If both not the same, then generate a new key, download csv
run --> AWS configure, set up new keys
try AWS s3 ls now

Change keys at all places in my case it was configs in Cloudera.

Revanche answered 2/4, 2021 at 16:17 Comment(0)

N

0

I couldn't figure out how to get the system to accept my Vocareum credentials so I took advantage of the fact that if you configure your instance to use IAM roles, the SDK automatically selects the IAM credentials for your application, eliminating the need to manually provide credentials.

Once a role with appropriate permissions was applied to the EC2 instance, I didn't need to provide any credentials.

Nightcap answered 8/5, 2021 at 10:53 Comment(0)

S

0

Open the ~/.bash_profile file and edit the info with the new values that you received at the time of creating the new user:

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_DEFAULT_REGION=us-east-1

Afterward, run the command:

source ~/.bash_profile

This will enable the new keys for the local machine. Now, we will need to configure the info in the terminal as well. Run the command -

aws configure

Provide the new values as requested and you are good to go.

Satchel answered 31/8, 2021 at 7:49 Comment(0)

B

0

I have encountered this issue when trying to export RDS Postgres data to S3 following this official guide.

TL;DR Troubleshooting tips:

Reset RDS credentials using:

DROP EXTENSION aws_s3 CASCADE;
DROP EXTENSION aws_commons CASCADE;
CREATE EXTENSION aws_s3 CASCADE;

Delete and add DB instance role used for s3Export feature. Optionally reset RDS credentials (previous action point) once again after that.

Below you will find more details on my case.

In particular, I have encountered:

[XX000] ERROR: could not upload to Amazon S3
Details: Amazon S3 client returned 'The AWS Access Key Id you provided does not exist in our records.'.

To be able to perform export to S3, RDS DB instance should be configured to assume a role with permission to write to S3 bucket, the guide describes these steps.

The reason of an error was in aws_s3.query_export_to_s3 Postgres procedure using some (cached?) invalid assumed credentials. I am still not aware which credentials has it been using but I have managed to achieve the same behaviour using AWS CLI:

I have assumed a role (aws sts assume-role),
And then tried to perform another action (aws s3 cp in particular) with this credentials without session token (only AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY without AWS_SESSION_TOKEN).

This resulted in the same error from AWS CLI: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.

In short: hard resetting RDS credentials helped.

Bugbear answered 28/12, 2021 at 8:46 Comment(0)

A

0

I just found another cause/remedy for this error/situation. I was getting the error running a PowerShell script. The error was happening on an execution of Write-S3Object. I have been working with AWS for a while now and have been running this script with success, but had not run it in a while.

My usual method of setting AWS credentials is: Set-AWSCredential -ProfileName <THE_PROFILE_NAME>

I tried the "aws configure" command and every other recommendation in this forum post. No luck.

Well, I am aware of the .aws\credentials file and took a look in there. I have only three profiles, with one being [default]. Everything was looking good, but then I noticed a new element in there, present in all 3 profiles, that I had not seen before:

 toolkit_artifact_guid=64GUID3-GUID-GUID-GUID-004GUID236
                (GUID redacting added by me)

Then I noticed this element differed between the profile I was running with and the [default] profile, which was the same profile, except for that.

On a hunch I changed the toolkit_artifact_guid in the [default] to match it to my target profile, and no more error. I have no idea why.

Adown answered 11/4, 2022 at 21:55 Comment(0)

N

0

I found a useful tip which worked for me for the above mentioned issue. You can navigate to

Control Panel\System and Security\System
Click on "Advanced system settings"
Under "advanced" tab click on "Environment Variables"
Delete the envieronment variable for all AWS_* related
COnfigure the AWS via AWS Configure
It would start working ( check it via AWS configure list)

Naughty answered 19/12, 2022 at 5:46 Comment(0)

K

0

Check which keys are used using below command:

Look at the Type column
If Type is shared-credentials-file then you can try updating credentials file located in .aws folder with latest working keys
If Type is env then you can try updating environment variables AWS_ACCESS_KEY_ID , AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION

Kerry answered 22/2 at 8:56 Comment(0)

T

0

If you're organization is using federated logins for console and CLI, then option 2 for all three tabs is "Paste the following text in your AWS credentials file (typically located in ~/.aws/credentials)". If this applies to your org, then option 2 includes this Learn More link. If you paste the text into your credentials file, you may need to remove an existing set of default cred, and you may need to rename the copied credential name to [default].

Note: These credentials are temporary, so I don't feel bad showing it here.

Tran answered 12/3 at 1:46 Comment(0)

N

0

When you setup the aws cli with aws configure it doesn't write aws_session_token in the credentials file. to do that manually you can access the credentials file and add the aws_session_key by:

vi ~/.aws/credentials

Nitrous answered 1/4 at 10:40 Comment(0)

T

-1

open aws account ->> goto iam -->> goto users --> select the user -->> goto credentials file ->> create access key and copy them to a text file.

then got to .aws file through command line and check the userid and access key replace them with the access key and secretkey taken from iam account to the appropriate user.

it should work

Tradition answered 27/3, 2022 at 13:23 Comment(0)

Recommended topics

Hot tags