sshtunnel doesn't recognize private key password

Question

I'm using sshtunnel to connect to connect to a remote server and on to a database from there. Everything is working fine, except that sshtunnel doesn't seem to recognize my ssh_private_key_password unless I've already entered it elsewhere (or more likely I'm making a mistake when providing it).

with SSHTunnelForwarder(
    ('my.server', 22),
    ssh_username="myUsername",
    ssh_pkey="~/.ssh/id_rsa", 
    ssh_private_key_password="myPassword",
    remote_bind_address=("other.server", 3306)
) as server:
{do some stuff}

If I log into "my.server" separately and enter my private key password in the dialog for storage by ssh-agent, and then run the code above, it works as expected.

If I run the code without having already logged into the server at some point earlier, I receive this error: ValueError: No password or public key available! (But the password is there - ssh_private_key_password - no?)

This happens whether ssh_pkey is pointing to the public key or the private key.

What might be happening here? Am I misunderstanding something about the expected arguments for SSHTunnelForwarder? Something more fundamental about the key and password?

Kravits

Hi, did you work out what the issue was? We're having the same problem! – Gattis

Is the user that you run this script same with the user that owns the key? – Borsch

Maybe the problem is the format of the private key. The private key file id_rsa should be start like -----BEGIN RSA PRIVATE KEY-----, not with the title of -----BEGIN OPENSSH PRIVATE KEY-----. – Officer

Answer 1

I have worked through this problem.

In my case, my macOS ~/.ssh/id_rsa start with -----BEGIN OPENSSH PRIVATE KEY----- which is not the proper start line.

In fact, the ssh_pkey should be started with -----BEGIN RSA PRIVATE KEY-----. According to the source code of sshtunnel model:

ssh_pkey (str or paramiko.PKey):
    **Private** key file name (``str``) to obtain the public key
    from or a **public** key (:class:`paramiko.pkey.PKey`)

The key is a paramiko.pkey.PKey, that means, the ~/.ssh/id_rsa should be convert to paramiko.pkey correctly.

import paramiko
pkey='~/.ssh/id_rsa'
key=paramiko.RSAKey.from_private_key_file(pkey)

# id_rsa with `-----BEGIN OPENSSH PRIVATE KEY-----`
# raise SSHException: not a valid RSA private key file

So, I generate a new key in a linux OS, with command of ssh-keygen -t rsa, then I authorize access to the key with ssh-copy-id -i ~/.ssh/id_rsa.pub username@jump_server_host -p ssh_port.

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub username@jump_server_host -p ssh_port

now the ~/.ssh/id_rsa on my linux OS looks like:

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAuFSEXjLMwyAJ54d5hQQjyIE+4l5pZw7GuYFDgNCzXjl6CNwV
XXXXXXXXXXXXXXX
XXXXXXXXXXXXXXX
XXXXXXXXXXXXXXX
-----END RSA PRIVATE KEY-----

Copy it to my local mac, and test it.

pkey='/path/to/the/new/id_rsa'
key=paramiko.RSAKey.from_private_key_file(pkey) 

# id_rsa with `-----BEGIN RSA PRIVATE KEY-----`
# No SSHException now.

Now, it works without Exception ValueError: No password or public key available!.

with SSHTunnelForwarder(
        ('jump_server_host', jump_ssh_port),  # jump server
        ssh_username='username',
        ssh_pkey="/path/to/new/id_rsa",
        remote_bind_address=('remote_host', remote_ssh_port),  # remote_server
        local_bind_address=('0.0.0.0', 30023)                  # local_bind_port set to 30023, your can select new one 
) as server:
    server.start()
    print(server.local_bind_port)
    server.close() 

It outputs 30023.

Officer

Thank you Ferris, changing the key format to PEM format (i.e. what gives you that opening line of BEGIN RSA PRIVATE KEY) is what solved this problem for me as well. If you don't have access to a Linux OS, you can actually change an existing OPENSSH key in-place to PEM with a one-liner: ssh-keygen -p -N "new passphrase" -m pem -f /path/to/key. This will leave your public key unchanged so you don't need to add or change anything on the remote machine. Via #54995141. – Kirman

Answer 2

Maybe you should add the private key

ssh-add ~/.ssh/id_rsa

Ilion

Answer 3

I faced a similar issue, by running below command on my machine. I am able to resolve this issue.

$ssh-add ~/.ssh/id_rsa

Appetite

Welcome to Stack Overflow! We encourage all users of the platform to be as informative as possible when answering a question. Please include a reason why your approach resolves the issue and when possible provide resource links for further understanding of the topic. These are just a couple of different approaches to making sure your answer is clear and understandable. Thank you for your participation! – Isothere

Answer 4

In case you haven't found a solution, in my case, the problem was the path to the private key. Instead of using:

ssh_pkey="~/.ssh/id_rsa",

Use (for example) the absolute path:

ssh_pkey="/Users/<your-user>/.ssh/id_rsa",

From my analysis, the problem was the check that the key exists in sshtunnel: https://github.com/pahaz/sshtunnel/blob/master/sshtunnel.py#L1018

os.path.exists(ssh_pkey) is False when using ~/.ssh/id_rsa path

Selfimmolation

Answer 5

Please be aware that this error can also show up if you provide an invalid password to the private key. I just tested now with paramiko==2.8.1 and I provided a gibberish password and got the same error message. The initial password was from a password manager and had many special characters so it's also worth to look into this as well.

Ilmenite

Answer 6

I met the same issue yesterday. The code I used worked well before. But after three months it can not work with "No password or public key available" exception. I found my paramiko version was 2.5, so I update it to 2.7. Then the code can work well again.

Solis