Permanent 'Temporary failure in name resolution' after running for a number of hours

Question

After running for a number of hours on Linux, my Python 2.6 program that uses urllib2, httplib and threads, starts raising this error for every request:

<class 'urllib2.URLError'> URLError(gaierror(-3, 'Temporary failure in name resolution'),)

If I restart the program it starts working again. My guess is some kind of resource exhaustion but I don't know how to check for it. How do I diagnose and fix the problem?

Blockhouse

Are you closing earlier HTTPRequests (or whatever you are using)? – Madalene

Trying to. I just noticed I have a large number of CLOSE_WAIT connections that must be related to the issue. – Blockhouse

Answer 1

This was caused by a library's failure to close connections, leading to a large number of connections stuck in a CLOSE_WAIT state. Eventually this causes the 'Temporary failure in name resolution' error due to resource exhaustion.

Blockhouse

When exactly was this issue fixed? I assume that error was caused by httplib library? – Affront

Seconded, please provide more details on your findings, thanks! – Urinate

@2371: How did you find out it is due to resource exhaustion? – Iridic

I encountered a similar problem with python requests. The solution was to use connection pooling via session objects docs.python-requests.org/en/master/user/advanced/… – Bonin

Answer 2

Was experiencing the same issue, in my case it wasnt resource exhaustion, the problem for me happened when my dhcp server changed the nameserver address, libc did not want to play ball and reload the new resolv.conf file, maintaining the cached one and forcing me to restart the script every time it changed.

All my python socket connections attempts fail after this, so I found this code that solved the situation:

import ctypes
try:
    libc = ctypes.CDLL('libc.so.6')
    res_init = getattr(libc, '__res_init')
    res_init(None)
except:
    pass

Use it before calling the socket.connect, hope this helps

Fairtrade