How to manage multiple ssh keys in the ~/.ssh directory

Question

I'm sure we all get this error from time to time:

$ git push origin master
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The typical remedy is to simply create a public/private key pair and share it with your git host (in my case bitbucket, with their instructions)

The thing is though, I have many accounts that require that I have a public/private key pair (for example i need to save a key to connect to AWS.. etc).. so what I do is that i create these keys and save them in separate directories ie

~/.ssh $ find .
./awskeys
./awskeys/id_rsa
./awskeys/id_rsa.pub
./bitbucket
./bitbucket/id_rsa
./bitbucket/id_rsa.pub

but then this error pops up every now and then.. to solve it I have to move the relevant keys back to the root ~/.ssh. this doesn't seem right to me. How can I reliably do this?

Densimeter

You could just use the same keypair for all of these services (as long as none of them ever gets to know your private key). – Izak

I have multiple keys in my .ssh folder, when I'm pushing git commits, it asks which one is to be used. Just give them different names. – Galligaskins

@Galligaskins well that's inconvenient – Densimeter

Answer 1

You can have them anywhere you want, but their permission and the permission of the parent folders need to be strict:

• no writable access for the parent folder (for others and all)
• 644 for a public key
• 600 for a private key.

You then:

• declare those different keys in ~/.ssh/config (example here)
• change the remote url in order to use the appropriate entry of the ~/.ssh/config file which described the right ssh key to use.

That means an entry like:

Host mygithub
    User           git
    IdentityFile   ~/.ssh/mypath/mykey # wherever your "new" key lives
    IdentitiesOnly yes

Allows you to replace an url like [email protected]:username/repo with:

git remote set-url origin mygithub:username/repo

Fanelli

mmm interesting.. i gotta look into this – Densimeter

@Densimeter I have added an example. – Fanelli

@Densimeter this really is the standard way to manage multiple ssh keys. – Fanelli

good stuff.. I updated the title to better reflect this discussion – Densimeter

Answer 2

When I tried using subfolders, I noticed that it WASN'T enough to just list the full paths to the keys in the ~/.ssh/config file.

After some research, I can confirm that the problem with subfolders not being scanned is:

1. Intentional by the OpenSSH team.
2. Not a bug

The ssh-agent and all the other SSH tools scan for keys in ~/.ssh but never any subfolders.

You can verify this behavior easily:

mkdir ~/.ssh/testfolder
ssh-keygen # give it the path /home/yourname/.ssh/testfolder/id_rsa_test
ssh-add -l # list keys. your new key is not listed
mv ~/.ssh/testfolder/id_rsa_test* ~/.ssh/
ssh-add -l # list keys, it is now listed since it's in the correct folder.

So the conclusion is:

NEVER store SSH keys in subfolders. The SSH tools don't support it by default, so you'll have to manually start ssh-agent and register the keys with ssh-add in each new session.

Keep the keys inside ~/.ssh without subfolders as intended instead. That is the only path which allows the SSH tools to automatically register and use the keys.

My own personal strategy for separating keys is to add a suffix with a description of the host or key purpose. Like this:

~/.ssh/config
~/.ssh/id_ed25519.storagebox
~/.ssh/id_ed25519.storagebox.pub
~/.ssh/id_rsa.mygithub
~/.ssh/id_rsa.mygithub.pub
~/.ssh/known_hosts

This keeps keys cleanly separated alphabetically and also provides info about the type of each key at a glance.

So that's what I do instead of using subfolders!

Flout

Answer 3

I faced this problem when I was installing python packages through requirements.txt from multiple private github repositories, each had its own deploy key.

As already stated within accepted answer - host config needs to be added to ~/.ssh/config, my example:

Host myfirstproject github.com
    HostName github.com
    IdentityFile ~/.ssh/myfirstprojectkey
    User git

Host mysecondproject github.com
    HostName github.com
    IdentityFile ~/.ssh/mysecondprojectkey
    User git

Then in my requirements I install packages like this:

myfirstpackagename@git+ssh://git@myfirstproject/myaccount/[email protected]#egg=myfirstpackagename
mysecondpackagename@git+ssh://git@mysecondproject/myaccount/[email protected]#egg=mysecondpackagename

I originally was hoping that following This SO answer would solve my problem, but in the end I landed with solution inspired by answer to this question.

Nary