How do I get the IP address from a http request using the requests library?

Question

I am making HTTP requests using the requests library in python, but I need the IP address from the server that responded to the HTTP request and I'm trying to avoid making two calls (and possibly having a different IP address from the one that responded to the request).

Is that possible? Does any python HTTP library allow me to do that?

PS: I also need to make HTTPS requests and use an authenticated proxy.

Update 1:

Example:

import requests

proxies = {
  "http": "http://user:[email protected]:3128",
  "https": "http://user:[email protected]:1080",
}

response = requests.get("http://example.org", proxies=proxies)
response.ip # This doesn't exist, this is just an what I would like to do

Then, I would like to know to which IP address requests are connected from a method or property in the response. In other libraries, I was able to do that by finding the sock object and using the getpeername() function.

Antonyantonym

"The server that responded" is ambiguous, because there may be all sorts of proxies, load balancers etc between you and the machine that actually generates the response. And a given domain name may have multiple IP addresses in DNS. – Nil

Answer 1

It turns out that it's rather involved.

Here's a monkey-patch while using requests version 1.2.3:

Wrapping the _make_request method on HTTPConnectionPool to store the response from socket.getpeername() on the HTTPResponse instance.

For me on python 2.7.3, this instance was available on response.raw._original_response.

from requests.packages.urllib3.connectionpool import HTTPConnectionPool

def _make_request(self,conn,method,url,**kwargs):
    response = self._old_make_request(conn,method,url,**kwargs)
    sock = getattr(conn,'sock',False)
    if sock:
        setattr(response,'peer',sock.getpeername())
    else:
        setattr(response,'peer',None)
    return response

HTTPConnectionPool._old_make_request = HTTPConnectionPool._make_request
HTTPConnectionPool._make_request = _make_request

import requests

r = requests.get('http://www.google.com')
print r.raw._original_response.peer

Yields:

('2a00:1450:4009:809::1017', 80, 0, 0)

---

Ah, if there's a proxy involved or the response is chunked, the HTTPConnectionPool._make_request isn't called.

So here's a new version patching httplib.getresponse instead:

import httplib

def getresponse(self,*args,**kwargs):
    response = self._old_getresponse(*args,**kwargs)
    if self.sock:
        response.peer = self.sock.getpeername()
    else:
        response.peer = None
    return response


httplib.HTTPConnection._old_getresponse = httplib.HTTPConnection.getresponse
httplib.HTTPConnection.getresponse = getresponse

import requests

def check_peer(resp):
    orig_resp = resp.raw._original_response
    if hasattr(orig_resp,'peer'):
        return getattr(orig_resp,'peer')

Running:

>>> r1 = requests.get('http://www.google.com')
>>> check_peer(r1)
('2a00:1450:4009:808::101f', 80, 0, 0)
>>> r2 = requests.get('https://www.google.com')
>>> check_peer(r2)
('2a00:1450:4009:808::101f', 443, 0, 0)
>>> r3 = requests.get('http://wheezyweb.readthedocs.org/en/latest/tutorial.html#what-you-ll-build')
>>> check_peer(r3)
('162.209.99.68', 80)

Also checked running with proxies set; proxy address is returned.

---

Update 2016/01/19

est offers an alternative that doesn't need the monkey-patch:

rsp = requests.get('http://google.com', stream=True)
# grab the IP while you can, before you consume the body!!!!!!!!
print rsp.raw._fp.fp._sock.getpeername()
# consume the body, which calls the read(), after that fileno is no longer available.
print rsp.content  

---

Update 2016/05/19

From the comments, copying here for visibility, Richard Kenneth Niescior offers the following that is confirmed working with requests 2.10.0 and Python 3.

rsp=requests.get(..., stream=True)
rsp.raw._connection.sock.getpeername()

---

Update 2019/02/22

Python3 with requests version 2.19.1.

resp=requests.get(..., stream=True)
resp.raw._connection.sock.socket.getsockname()

---

Update 2020/01/31

Python3.8 with requests 2.22.0

resp = requests.get('https://www.google.com', stream=True)
resp.raw._connection.sock.getsockname()

Airwoman

It's not pretty but it works. Thanks! That helped a lot! – Antonyantonym

There's a "fine" tradition of monkey patching... The requests library is a high level abstraction of HTTP, it doesn't consider the IP address that served a response to be needful at its abstraction level. The minimal monkey patch was an interesting challenge. If you want pretty you'd have to use a lower level HTTP library, but do more HTTP logic yourself. – Airwoman

I've just realized that it doesn't work in all cases. Is not working for wheezyweb.readthedocs.org/en/latest/…. Can you tell a reason? – Antonyantonym

It didn't work for me. But I've managed to find the sock by changing your code to, response.fp._sock: response.peer = response.fp._sock.getpeername() – Antonyantonym

Perhaps a difference between versions. – Airwoman

Great solution: thanks. I had a problem with HTTPS connections: the "WrappedSocket" object didn't have the getpeername() method. I eventually found the method on self.sock.socket, so a two-step try/except for self.sock.getpeername() then self.sock.socket.getpeername() does the job for me: working on my web monitor service. – Heparin

Thanks. This helped me show that inconsistent results from an API were not due to load balancing. – Boothe

@Airwoman I found a new way without the need to monkeymatch: rsp=requests.get(..., stream=True);ip=rsp.raw._fp.fp._sock.getpeername();print rsp.content see github.com/kennethreitz/requests/issues/2158 – Detector

@Detector Interesting! Added it into the body here. Don't know if it was possible back with 1.2.3, but certainly works with 2.5.1 that I have to hand. – Airwoman

I've just needed this and I have discovered that rsp=requests.get(..., stream=True);rsp.raw._connection.sock.getpeername() works. I have requests 2.10.0 and am using Python 3 – Pew

@RichardKennethNiescior - your solution works but when requesting via a proxy the IP address returned is the proxy's ip address but not the origin server IP address. Any idea how to get the origin server's ip address? – Martinet

@SureshKumar: If you're using a HTTP proxy then your client never sees the IP address of the service beyond the proxy, your client asks the proxy by URL. If you're using an HTTPS proxy, then your client resolves the IP address and asks the proxy for a CONNECT to the resolved address and the HTTPS port. The proxy may add a header for an HTTP request, but may be a feature / setting. – Airwoman

It seems to be resp.raw._connection.sock.socket.getsockname() now with requests version '2.19.1'. Note that in my case, resp.raw._connection.sock is of type urllib3.contrib.pyopenssl.WrappedSocket. – Nellynelms

How would I go about adding a proxy IP to the solution for Python 3? – Almost

on python 3.8 requests 2.22.0 call.raw._connection.sock does not have socket property even i'm using stream=True. for example getta = requests.get("google.it", stream=True) print(getta.raw._connection.sock.socket.getsockname()) returns AttributeError: 'SSLSocket' object has no attribute 'socket' i'm not able to find where socket is. – Mammalogy

@AndreaBisello Updated. sock was the socket. – Airwoman

@Airwoman getsockname() is the local name, getpeername() is the remote name, you're mixing both in your reply with your updates. – Blame

You are doing a great job keeping this answer updated! – Liquidity

Answer 2

Try:

import requests

proxies = {
  "http": "http://user:[email protected]:3128",
  "https": "http://user:[email protected]:1080",
}

response = requests.get('http://jsonip.com', proxies=proxies)
ip = response.json()['ip']
print('Your public IP is:', ip)

Parson

That's great and all, but how does this work when trying to obtain that local and/or remote ip when pulling a different url which 99.9% of people using requests will be doing? This works in a testing environment when you're testing basic code. But in production, you would never do this. – Desdee

Answer 3

Update 2023/08/08:

For Python 3.11.3 & requests 2.30.0, use the following monkey-patch:

def patch_http_and_https_connection():
    import http.client

    def getresponse(self, *args, **kwargs):
        response = self._old_getresponse(*args, **kwargs)
        if self.sock:
            response._local = self.sock.getsockname()
            response._remote = self.sock.getpeername()
        else:
            response._local = None
            response._remote = None
        return response

    http.client.HTTPConnection._old_getresponse = http.client.HTTPConnection.getresponse
    http.client.HTTPConnection.getresponse = getresponse

Then for this example:

rsp = requests.get('https://www.google.com', stream=True)

Use the following:

Remote IP Address/Port = rsp.raw._original_response._remote
Local IP Address/Port = rsp.raw._original_response._local

Purism