How can I fetch emails via POP or IMAP through a proxy?

Question

Neither poplib or imaplib seem to offer proxy support and I couldn't find much info about it despite my google-fu attempts.

I'm using python to fetch emails from various imap/pop enabled servers and need to be able to do it through proxies.

Ideally, I'd like to be able to do it in python directly but using a wrapper (external program/script, OSX based) to force all traffic to go through the proxy might be enough if I can't find anything better.

Could anyone give me a hand? I can't imagine I'm the only one who ever needed to fetch emails through a proxy in python...

** EDIT Title edit to remove HTTP, because I shouldn't type so fast when I'm tired, sorry for that guys **

The proxies I'm planning to use allow socks in addition to http.

Pop or Imap work through http wouldn't make much sense (stateful vs stateless) but my understanding is that socks would allow me to do what I want.

So far the only way to achieve what I want seems to be dirty hacking of imaplib... would rather avoid it if I can.

Eyehole

i'm curious: how would you tunnel imap through an http proxy even without python? – Monoploid

Removed Http from the title, my mistake, meant socks... – Eyehole

Answer 1

You don't need to dirtily hack imaplib. You could try using the SocksiPy package, which supports socks4, socks5 and http proxy (connect):

Something like this, obviously you'd want to handle the setproxy options better, via extra arguments to a custom __init__ method, etc.

from imaplib import IMAP4, IMAP4_SSL, IMAP4_PORT, IMAP4_SSL_PORT
from socks import sockssocket, PROXY_TYPE_SOCKS4, PROXY_TYPE_SOCKS5, PROXY_TYPE_HTTP

class SocksIMAP4(IMAP4):
    def open(self,host,port=IMAP4_PORT):
        self.host = host
        self.port = port
        self.sock = sockssocket()
        self.sock.setproxy(PROXY_TYPE_SOCKS5,'socks.example.com')
        self.sock.connect((host,port))
        self.file = self.sock.makefile('rb')

You could do similar with IMAP4_SSL. Just take care to wrap it into an ssl socket

import ssl

class SocksIMAP4SSL(IMAP4_SSL):
    def open(self, host, port=IMAP4_SSL_PORT):
        self.host = host
        self.port = port
        #actual privoxy default setting, but as said, you may want to parameterize it
        self.sock = create_connection((host, port), PROXY_TYPE_HTTP, "127.0.0.1", 8118)
        self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile)
        self.file = self.sslobj.makefile('rb')

Latea

Nice find MattH, that's exactly the kind of thing I was looking for (didn't know SocksiPy). I'll give it a go. Thanks. – Eyehole

@rafael-t what is the create_connection() method in your code? Where is it implemented? – Grigg

@ВсеволодТимченко: that fleshed out ssl implementation was added by Rafael T, I'm not sure where he's implying create_connection comes from – Latea

@Latea yeah, i tried to address Rafael, but SOF engine obviously doesn't support addressing someone who has made edits. I figure this thing comes from PySocks (line 150). This method should be socks.create_connection. Thanks for the reply though. There's also this thing about self.sock closing after being wrapped. – Grigg

there is no sockssocket in socks – Pantie

create_connection parameters does not match function create_connection in socks – Pantie

Answer 2

Answer to my own question... There's a quick and dirty way to force trafic from a python script to go through a proxy without hassle using Socksipy (thanks MattH for pointing me that way)

import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4,proxy_ip,port,True)
socket.socket = socks.socksocket

That global socket override is obviously a bit brutal, but works as a quick fix till I find the time to properly subclass IMAP4 and IMAP4_SSL.

Eyehole

didn't work for me , i got "IOError: [Errno socket error] ('timed out',)" , any suggestion ? – Performative

Answer 3

If I understand you correctly you're trying to put a square peg in a round hole.

An HTTP Proxy only knows how to "talk" HTTP so can't connect to a POP or IMAP server directly.

If you want to do this you'll need to implement your own server somewhere to talk to the mail servers. It would receive HTTP Requests and then make the appropriate calls to the Mail Server. E.g.:

How practical this would be I don't know since you'd have to convert a stateful protocol into a stateless one.

Rachmaninoff

You're 100% right, I forgot to mention socks capability. I removed HTTP from the title, sorry for that, I'm tired ;p – Eyehole

The 'CONNECT' method of an HTTP proxy would let you run any tcp client->server protocol. IF the HTTP proxy allows it, most default configurations restrict 'CONNECT' requests to port 443. – Latea

I understand what you mean but how does thunderbird connect through http proxy then? – Heriberto