How to use urllib3.Retry?

Question

I have found next example

def prepare_retry_requester(retries: int = 5, forcelist: List = (503,)) -> requests.Session:
    requester = requests.Session()
    retry = urllib3.Retry(total=retries, backoff_factor=1, status_forcelist=forcelist)
    for protocol in 'http://', 'https://':
        requester.mount(protocol, requests.adapters.HTTPAdapter(max_retries=retry))
    return requester


with prepare_retry_requester(forcelist=[502, 503, 413]) as requester:
    response = requester.post(url, data=serialized)

But it still fails if i get 502 errors for a while (server is restarting for 10 seconds).

Odious

You are passing max_retries to HTTPAdapter which does not recover from 502 errors (by design) since it's not a connection error (but an applicative one). You can instead pass retries to http.request. See the docs – Calender

@BenjaminGruenbaum This should be an answer. :) – Systemize

If you need a custom timeout algorith, then subclass urllib3.util.retry.Retry and override method get_backoff_time(). For example: You can return a fixed number of fraction seconds, e.g., 5.75 – Graces

Answer 1

You are passing max_retries to HTTPAdapter which does not recover from 502 errors (by design) since it's not a connection error (but an applicative one). You can instead pass retries to http.request. See the retrying docs.

Calender