Python Connect over HTTP proxy with pysftp

Question

Currently, I am doing SFTP transfer using Python subprocess.POPEN and PuTTY psftp.exe.

It is working, but not really clean nor transportable.

I would like to reproduce the same behavior using Python pysftp, but I do not know where to input all the parameters. I have in PuTTY the following configuration:

• Server IP : 123.123.123.255
• Server Port : 22
• Connection Type : SSH
• AutoLogin UserName : MyUser
• Proxy type : HTTP
• Proxy Hostname : gw.proxy.fr
• Proxy port : 1234
• Proxy Username : ProxyUser
• Proxy Password: ProxyPass

How should I input all these parameters in pysftp so I can retrieve my files?

EDIT: Using the answer from Martin Prikryl, I found some new stuffs to explore. If I understand well, I need to use a socket. Put I still have some problem to input all the information I need.

import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
proxy = ("gw.proxy.fr",1234)
sock.connect(proxy)
target=("123.123.123.255",23)
cmd_connect = "CONNECT {}:{} HTTP/1.1\r\n\r\n".format(*target)
sock.sendall(cmd_connect)

The respond that I receive from this is HTTP/1.0 407 Proxy Authentication Required, which is kind of normal because I did not use the Proxy authentication information anywhere. So, do you have any idea how I can use them and input them into my socket ?

Mingmingche

i struggled a lot with the line ´sock.sendall(cmd_connect)'. as it gives TypeError : bytes expected. It should be ´sock.sendall(bytes(cmd_connect),'UTF-8')'. – Mammal

Answer 1

I do not think that the pysftp supports proxies. Though note that the pysftp is just a wrapper around Paramiko library, which does support proxies.

So I suggest you to use Paramiko directly.

For a start see How to ssh over HTTP proxy in Python Paramiko?, particularly the answer by @tintin.

---

To authenticate to the proxy, after the CONNECT command, add a Proxy-Authorization header like:

Proxy-Authorization: Basic <credentials>

where the <credentials> is base-64 encoded string username:password.

auth = 'Basic ' + base64.encodebytes("username:password".encode()).decode()
args = ("123.123.123.255", 23, auth)
cmd_connect = "CONNECT {}:{} HTTP/1.1\r\nProxy-Authorization: {}\r\n\r\n".format(*args)

Twist

OMG, it is working. You did it. Now, just need to deal with the key – Mingmingche

Answer 2

In my case, I do this:

import pysftp
import paramiko

hostname, prot = 'some.host.name', 22
proxy = paramiko.proxy.ProxyCommand('/usr/bin/nc --proxy proxy.foobar:8080 %s %d' % (hostname, port))
t = paramiko.Transport(sock=proxy)
t.connect(username='abc', password='123')

sftp = paramiko.SFTPClient.from_transport(t) # back to pysftp wrapper
sftp.listdir('.')

Chondrule

what does /usr/bin/nc do? @Aziz Alto – Nonoccurrence

Oh NetCat. This got me a friendly call from Cybercommand. They locked me out of my computer, scanned my computer, and asked many questions. – Nonoccurrence

Does this work if the sshd you are connecting to is configured for sftp only? Or is there a problem with t.connect? If so ... is there a workaround? – Continuant

Answer 3

try this:

import paramiko

use_proxy = True

# PROXY
host_proxy = ''
port_proxy = ''

# SFTP
host = ''
port = 22
username = ''
password = ''

try:
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    
    if host_proxy and use_proxy:
    # find the path to nc on the command line: "which nc"
        proxy = paramiko.ProxyCommand(f"/bin/nc --proxy {host_proxy}:{port_proxy} {host} {port}") 
        trans = paramiko.Transport(proxy)
    else:
        trans = paramiko.Transport((host, port))
    trans.connect(username=username, password=password)
    sftp = paramiko.SFTPClient.from_transport(trans)
except Exception as e:
    print(e)


sftp.listdir('.')

sftp.close()
trans.close()

Remember to install ncat if you run in docker, in debian. Not install netcat, it's the older version and does not work well.

apt-get -y update && apt-get -y install ncat

Presentment

There are native Python implementations of proxies (linked from my answer), so why relying on an external command-line tool? – Twist

Also and implemented in a similar way as you did, but unfortunately the connection was unstable. I shared the optimal solution I found for my case, and you should consider that ncat is a highly used functionality on a unix basis. – Presentment