urllib3 debug request header

Question

I'm using urllib3 and I want to see the headers that are send.

I've found this in documentation but it doesn't print the headers:

urllib3.add_stderr_logger(1)

Is there any way of doing this?

Taejon

Answer 1

Right now, the best way to achieve really verbose logging that includes headers sent in urllib3 is to override the default value in httplib (which is used internally).

For Python 3:

# You'll need to do this before urllib3 creates any http connection objects
import http.client
http.client.HTTPConnection.debuglevel = 5

# Now you can use urllib3 as normal
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', ...)

In Python 2, the HTTPConnection object lives under the httplib module.

This will turn on verbose logging for anything that uses httplib. Note that this is not using the documented API for httplib, but it's monkeypatching the default value for the HTTPConnection class.

The goal is to add better urllib3-native logging for these kinds of things, but it hasn't been implemented yet. Related issue: https://github.com/shazow/urllib3/issues/107

Grata

This works on in Python 2.7. In Python 3.x, you'll need to do import http; http.client.HTTPConnection.debuglevel=5 – Burdensome

@DanLenski I'm trying to get your solution to work with log file but it only works with stdout. More here: https://mcmap.net/q/151600/-python-http-request-and-debug-level-logging-to-the-log-file/1090360 – Mctyre