Retrieving RSA key from AWS Secrets Manager in CodeBuild corrupts key "invalid format"

Question

During a CodeBuild run I am retrieving a rsa key from SecretsManager, which is the private key to use to access private sources in BitBucket. To do this I have copied the private key into a secret, then in my buildspec file I have the following snippet:

  "env": {
    "secrets-manager": {
      "LOCAL_RSA_VAR": "name-of-secret"
    }
  },

In the install portion of the buildspec:

"install": {
  "commands": [
    "echo $LOCAL_RSA_VAR" > ~/.ssh/id_rsa,
    "chmod 600 ~/.ssh/id_rsa",
    "yarn install"
  ]
},

HOWEVER, this always ends up with an error:

Load key "/root/.ssh/id_rsa": invalid format
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

To determine if the key was wrong I tried uploading the rsa_id file into S3 and then download it from there and used it that way using these commands instead:

"install": {
  "commands": [
    "aws s3 cp s3://the-bucket-name/id_rsa ~/.ssh/id_rsa",
    "chmod 600 ~/.ssh/id_rsa",
    "yarn install"
  ]
},

This works fine.

So I guess the question is... Has anyone tried this and had better success? Is there something that I am not doing correctly that you can think of?

Stannary

Can you rename 1 of the ~/.ssh/id_rsa files and do a diff and/or an od -ax? The echo could be messing it up or the permissions?(e.g. try swaping the echo (add a touch) and chmod); just some thoughts--hope it helps – Budweis

cat the ~/.ssh/id_rsa and make sure it's what you think it is - make sure it's in plain text – Acrolein

Thanks for the suggestions. Event though 'cat'ing the secret manager env variable wont work (it just prints ***) I was able to diff the contents of the file vs the env var which in turn DID print the contents since they differed. And this led me to make adjustments which ultimately solved the problem. Thanks Again! – Stannary

Answer 1

I was able to get an answer by diff'ing the output of the Env Var vs the File contents from the S3 file. ('cat' will not print out the content of a secret mgr env variable) It ends up content of the env var was altered by the 'echo' command.

The solution that ended up working for me was:

printenv LOCAL_RSA_VAR > ~/.ssh/id_rsa

this command didn't alter the content of the rsa and I was able to successfully use the certificate.

As a recap this is what I was successful doing:

1. Generate the new key
2. Used command "pbcopy < id_rsa" to get local key into clipboard
3. Pasted that into a new secret in Secret Manager
4. Used the first set of code above to have the buildspec file retrieve the content into a env variable and then the 'printenv' command above, in the install command portion of the buildspec file, to save that to the default ssh location.

Hope this helps anyone that runs into the same issue.

UPDATE: I found that this works if the RSA is stored as its own secret as one big block of text. If you try and add this as part of a json object, ie:

{
  "some": "thing",
  "rsa_id": "<the rsa key here>"
}

this does not seem to work. I found that the content is altered with spaces in place of the newline. This is what i found when running an 'od -ax' on each and comparing them:

own secret:
R   I   V   A   T   E  sp   K   E   Y   -   -   -   -   -  nl

json secret:
R   I   V   A   T   E  sp   K   E   Y   -   -   -   -   -  sp

Stannary

Answer 2

I have encountered the same issue. Copying the id_rsa generated from the the command echo $LOCAL_RSA_VAR > ~/.ssh/id_rsa in S3 I have noticed that the new lines have not been preseved.

I have resolved putting the var env between double quote "":

echo "$LOCAL_RSA_VAR" > ~/.ssh/id_rsa

Siskin

Right answer is at the bottom. – Cisneros

Answer 3

I has the same issue, fixed it my NOT Copy-Paste my private key to SecretManager, but use AWS CLI to upload my private key to SecretManager:

aws secretsmanager put-secret-value --secret-id AWS_CODECOMMIT_SSH_PRIVATE --secret-string file://myprivatekey.pem

And then CloudBuild worked fine:

version: 0.2

env:
  secrets-manager:
    AWS_CODECOMMIT_SSH_ID     : AWS_CODECOMMIT_SSH_ID
    AWS_CODECOMMIT_SSH_PRIVATE: AWS_CODECOMMIT_SSH_PRIVATE

phases:
  install:
    commands:
      - echo "Setup CodeCommit SSH Key"
      - mkdir ~/.ssh/
      - echo "$AWS_CODECOMMIT_SSH_PRIVATE"          > ~/.ssh/id_rsa
      - echo "Host git-codecommit.*.amazonaws.com"  > ~/.ssh/config
      - echo " User $AWS_CODECOMMIT_SSH_ID"        >> ~/.ssh/config
      - echo " IdentityFile ~/.ssh/id_rsa"         >> ~/.ssh/config
      - echo " StrictHostKeyChecking no"           >> ~/.ssh/config
      - chmod 600 ~/.ssh/id_rsa
      - chmod 600 ~/.ssh/config

Vienna

omg I wanna hug you right now this is the only answer that worked between AI and abt twenty different answers.. THANK YOU!!! – Cisneros