Python requests & urllib3 Retry - How may retries were made?

Question

Given following example usage:

adapter = HTTPAdapter(max_retries=Retry(
    total=5,
    backoff_factor=0.1,
    status_forcelist=[429, 500, 502, 503, 504],
    method_whitelist=["HEAD", "GET", "OPTIONS"]
))
session = requests.Session()
session.mount("http://", adapter)
session.mount("https://", adapter)
rsp = session.post(url, json=my_json, params=my_params)

How do I tell how many retries were made? I'm trying to debug/diagnose/resolve an issue posted in this related question

Alternatively, is there a different usage of these libs that provides this?

Stocktonontees

Answer 1

You can get the list of retries by accessing the retries.history property of the underlying urllib3.response.HTTPResponse object used by the requests library.

In your case, that would be:

rsp.raw.retries.history

Irresolvable

If you could show a simple example of how to access the history of retried attempts, then that would be great. Because, if you try to access raw.retries.history either it will throw an exception or if the first attempt gets successfully done then it will give you an empty list. With try/except it will then skip to the except block. – Guilty

Answer 2

In a worse case (if the first request will get status code in response 429, 500, 502, 503, 504), there will be 5 retries with a backoff factor of 0.1. You can look there the explanation of each parameter. https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#module-urllib3.util.retry

Gertudegerty

Using wall clock timings to blackbox feels like a pretty rough last-resort. I'd probably sooner implement retries myself. – Stocktonontees