Remote tcp connection in python with zeromq

Question

I have a python client that needs to talk to a remote server I manage. They communicate using zeromq. When I tested the client/server locally everything worked. But now I have the client and server deployed on the cloud, each using a different provider. My question is, what's the simplest way (that is safe) to make the connection? I'm assuming I can't pass the password over, and even if I could I'm guessing there are safer alternatives.

I know how to set an ssh connection without a password using ssh-keygen. Would that work? Would the client need to make an ssh connection with the server before sending the tcp req? If there's a python library that helps with this it'd be a big help.

Thanks!

Update: So more than 24 hours passed and no one replied/answered. I think I'm getting closer to solve this, but not quite there yet. I added my client's key to .ssh/authorized_key on the server, and now I can ssh from the client to the server without a password. Next, I followed this post about "Tunneling PyZMQ Connections with SSH". Here's what I have in my client code:

1    context = zmq.Context()
2    socket = context.socket(zmq.REQ)
3    socket.connect("tcp://localhost:5555")
4    ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser@remote-server-ip:5555")
5    socket.send_string(some_string)
6    reply = socket.recv()

This doesn't work. I don't really understand lines 3 & 4 and I assume I do something wrong there. Also, my server (hosted on linode) has a "Default Gateway" IP and a "Public IP" -- in the tunnel connection I only specify the public ip, which is also the ip I use to ssh to the machine.

Pavilion

Answer 1

Indeed, ZMQ way is - tunnelling connection with the SSH. Your example is exactly what needs to be done, except that one should either use connect or tunnel_connection, not both.

Also, when specifying server to connect to, make sure to define the SSH port, not the ZMQ REP socket port. That is, instead of myuser@remote-server-ip:5555 you might try myuser@remote-server-ip or myuser@remote-server-ip:22.

import zmq
import zmq.ssh
context = zmq.Context()
socket = context.socket(zmq.REQ)
zmq.ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser@remote-server-ip")
socket.send(b"Hello")
reply = socket.recv()

Finally, make sure you've installed either pexpect or paramiko - they will do the tunnelling actually. Note that if you're using Windows, paramiko is the only solution which will work - pexpect openssh tunnelling won't work on Windows.

If you use paramiko instead of pexpect, make sure to set paramiko=True in the tunnel_connection arguments.

Ruching

It worked!! You're the man :) I added "import pexpect" (before only had "from zmq import ssh") and changed the server connection to specify the ssh port number. And also removed the socket.connect line. This is the actual line I'm using: (you might want to update your answer) ssh.tunnel_connection(socket, "tcp://localhost:5555", "users@server-ip-address:ssh-port"). In my case the ssh port is not the default 22 (for security purposes). Many thanks! – Pavilion

Thanks. Just note - I think you may even remove "import pexpect" line, as this is what the zsh.ssh does by its own in any case. You just need to ensure pexpect is installed. – Ruching

Answer 2

I have found ssh in Python to be iffy at best, even with paramiko and fabric libraries, so to debug, you might try setting up a tunnel separately, just to see if that's the issue with the broken connection.

For example:

ssh myuser@remote-server-ip -L 5050:localhost:5555 -N

This says: connect to myuser@remote-server-ip, and whenever I request a connection to localhost:5050 on my machine, forward it across the ssh connection so that the server at remote-server-ip thinks it's receiving a connection from localhost:5555.

-L constructs the tunnel, and -N means don't do anything else on the connection.

With that running in another shell, e.g., a different Terminal window, on your local development machine, try to connect to a zeromq server at localhost:5050, which will actually be the zeromq running on the remote server.

You could use 5555:localhost:5555 in the ssh command above, but I find that can be confusing and often conflicts with a local copy of the same service.

Pessa

I tried this with a very simple client/server from the zeromq guide. It works when I run both on the same computer but not when I try it over the internet. I think your solution is right, it should work, I might have a configuration problem with my server. Thanks for your detailed answer. – Pavilion