python TCPServer address already in use but I close the server and I use `allow_reuse_address`

Question

Here is my code to run the server:

class MyRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
    #....

PORT = 8089

httpd = SocketServer.TCPServer(("", PORT), MyRequestHandler)
httpd.allow_reuse_address = True

print "Serving forever at port", PORT
try:
    httpd.serve_forever()
except:
    print "Closing the server."
    httpd.server_close()
    raise

Yet this is what happens:

^CClosing the server.
Traceback (most recent call last):
  File "server.py", line 118, in <module>
    self.send_error(400, "Unimplemented GET command: %s" % (self.path,))
  File "/home/claudiu/local/lib/python2.6/SocketServer.py", line 224, in serve_forever
    r, w, e = select.select([self], [], [], poll_interval)
KeyboardInterrupt
(.virtualenv)claudiu@xxx:~/xxx$ python server.py
Traceback (most recent call last):
  File "server.py", line 122, in <module>
    httpd = SocketServer.TCPServer(("", PORT), MyRequestHandler)
  File "/home/claudiu/local/lib/python2.6/SocketServer.py", line 402, in __init__
    self.server_bind()
  File "/home/claudiu/local/lib/python2.6/SocketServer.py", line 413, in server_bind
    self.socket.bind(self.server_address)
  File "<string>", line 1, in bind
socket.error: [Errno 98] Address already in use

Why? I close the server and set allow_reuse_address to True... Using python 2.6.8.

Dav

Answer 1

Thanks to the other answers, I figured it out. allow_reuse_address should be on the class, not on the instance:

SocketServer.TCPServer.allow_reuse_address = True
httpd = SocketServer.TCPServer(("", PORT), MyRequestHandler)

I'm still not sure why closing the socket didn't free it up for the next run of the server, though.

Dav

Is there a way to change the port and try again? I tried using expcept OSError: port+=1 ... but python does not catch the port error (ON python 3.7 it gives OSError Exception rather than socket.error exception) – Spitfire

Answer 2

The [Err 98] Address already in use is due to the fact the socket was .close() but it's still waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request (see TIME_WAIT). By default you are not allowed to bind a socket if there a socket bound to that port but you can override that with allow_reuse_address (SO_REUSEADDR)

Although is possible to mutate TCPServer.allow_reuse_addr (as proposed in this other answer), I think is more clean to your own subclass of TCPServer where allow_reuse_address is set to True:

import SocketServer
import SimpleHTTPServer
import time

class MyRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
    def do_GET():
        time.sleep(60)
        self.request.sendall("I'm here!")

class ReuseAddrTCPServer(SocketServer.TCPServer):
    allow_reuse_address = True

PORT = 8089

httpd = ReuseAddrTCPServer(("", PORT), MyRequestHandler)
httpd.daemon_threads = True


print "Serving forever at port", PORT
try:
    httpd.serve_forever()
except:
    print "Closing the server."
    httpd.server_close()
    raise

You can use definitely set allow_reuse_address on the instance itself (without messing with classes) but you need to use TCPServer(..., bind_and_activate=False), otherwise the socket will be bound before you have a chance to change the allow_reuse_address setting. Then you need to manually call .server_bind() and .server_activate() before serve_forever():

...
httpd = SocketServer.TCPServer(("", PORT), MyRequestHandler, bind_and_activate=False)
httpd.allow_reuse_address = True
httpd.daemon_threads = True
...
httpd.server_bind()
httpd.server_activate()
httpd.serve_forever()

Rattat

Answer 3

It is because TCP TIME_WAIT.

Somebody discovered this exact problem.

However, if I try to stop and start the server again to test any modifications, I get a random “socket.error: [Errno 98] Address already in use” error. This happens only if a client has already connected to the server.

Checking with netstat and ps, I found that although the process it self is no longer running, the socket is still listening on the port with status “TIME_WAIT”. Basically the OS waits for a while to make sure this connection has no remaining packets on the way.

Scabrous

Not a great citation. The port is still present in TIME_WAIT state. There is no socket, and no listening either. – Classis

I understand, but I wanted to know why my solutions (set allow_reuse_address and close the socket on shutdown) did not fix it. – Dav

Answer 4

It is because you have to set SO_REUSEADDRESS before you bind the socket. As you are creating and binding the socket all in one step and then setting it, it is already too late.

Classis

Ah so allow_reuse_address basically does nothing for the default TCPServer implementation? Also shouldn't closing the socket at the end of the program prevent this error? – Dav

Answer 5

Address already in use error happened when SocketServer.TCPServer(("", PORT), MyRequestHandler), so you cannot catch below.

Empanel

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Gemstone

Answer 6

In "httpd = SocketServer.TCPServer(("", PORT), MyRequestHandler)", will reset allow_reuse_address to False, so you set allow_reuse_address behind it is useless, regardless of closing the socket at the end of the program.

Empanel