Shutdown for socketserver based Python 3 server hangs

Question

I am working on a "simple" server using a threaded SocketServer in Python 3.

I am going through a lot of trouble implementing shutdown for this. The code below I found on the internet and shutdown works initially but stops working after sending a few commands from the client via telnet. Some investigation tells me it hangs in threading._shutdown... threading._wait_for_tstate_lock but so far this does not ring a bell.

My research tells me that there are ~42 different solutions, frameworks, etc. on how to do this in different python versions. So far I could not find a working approach for python3. E.g. I love telnetsrv (https://pypi.python.org/pypi/telnetsrv/0.4) for python 2.7 (it uses greenlets from gevent) but this one does not work for python 3. So if there is a more pythonic, std lib approach or something that works reliably I would love to hear about it!

My bet currently is with socketserver but I could not figure out yet how to deal with the hanging server. I removed all the log statements and most functionality so I can post this minimal server which exposes the issue:

# -*- coding: utf-8 -*-
import socketserver
import threading

SERVER = None


def shutdown_cmd(request):
    global SERVER
    request.send(bytes('server shutdown requested\n', 'utf-8'))
    request.close()
    SERVER.shutdown()
    print('after shutdown!!')
    #SERVER.server_close()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        while True:
            try:
                msg = str(self.request.recv(1024).strip(), 'utf-8')
                if msg == 'shutdown':
                    shutdown_cmd(msg, self.request)
                else:
                    self.request.send(bytes("You said '{}'\n".format(msg), "utf-8"))
            except Exception as e:
                pass


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    global SERVER
    SERVER = ThreadedTCPServer(('', 1520), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    SERVER.shutdown()


if __name__ == '__main__':
    run()

It would be great being able to stop the server from the handler, too (see shutdown_cmd)

Absorption

please post a minimal reproducible example. – Nutgall

@Nutgall I added more code so you have a complete example that exposes the issue – Absorption

@Nutgall I am using the solution I posted below and so far it works fine; shutdown with the socketserver based solution never worked for me – Absorption

have you tried two solutions in my answer? what is the problem with it? – Nutgall

like I said in my comment above your solution does not fix the original issues from my question – Absorption

wait I see no comment above my answer? am I miss something here? I would like to know how it fails and improve it. – Nutgall

you asked stackoverflow.com/help/mcve and I added the complete code. You solution does not fix the issues. Why? This is a very good question. Basically looking for this answer is what this whole issue is about. Don't know how to be more explicit without reposting the whole question, sorry. – Absorption

would you show the code how the solution in the answer get applied? – Nutgall

you see the information is not symmetry here, I havn't see your new code. – Nutgall

and I proposed two solutions, not one, I don't know which one gets applied by you. – Nutgall

hmm, this sounds overly complicated. I tried the solution you advertised as ideal solution. If it works on your side then why don't you post the complete working solution? like I said I am still looking for a working solution for the stated problem - not a different broken one. Besides I can not add my solution as a comment. But you could post a complete solution as an answer. – Absorption

I do not think your solution implements a shutdown command – Absorption

Answer 1

I tried two solutions to implement a tcp server which runs on Python 3 on both Linux and Windows (I tried Windows 7):

• using socketserver (my question) - shutdown is not working
• using asyncio (posted an answer for that) - does not work on Windows

Both solutions have been based upon search results on the web. In the end I had to give up on the idea of finding a proven solution because I could not find one. Consequently I implemented my own solution (based on gevent). I post it here because I hope it will be helpful for others to avoid stuggeling the way I did.

# -*- coding: utf-8 -*-
from gevent.server import StreamServer
from gevent.pool import Pool


class EchoServer(StreamServer):

    def __init__(self, listener, handle=None, spawn='default'):
        StreamServer.__init__(self, listener, handle=handle, spawn=spawn)

    def handle(self, socket, address):
        print('New connection from %s:%s' % address[:2])
        socket.sendall(b'Welcome to the echo server! Type quit to exit.\r\n')

        # using a makefile because we want to use readline()
        rfileobj = socket.makefile(mode='rb')
        while True:
            line = rfileobj.readline()
            if not line:
                print("client disconnected")
                break
            if line.strip().lower() == b'quit':
                print("client quit")
                break
            if line.strip().lower() == b'shutdown':
                print("client initiated server shutdown")
                self.stop()
                break
            socket.sendall(line)
            print("echoed %r" % line.decode().strip())
        rfileobj.close()


srv = EchoServer(('', 1520), spawn=Pool(20))
srv.serve_forever()

Absorption

Answer 2

shutdown() works as expected, the server has stopped accepting new connections, but python still waiting for alive threads to terminate.

By default, socketserver.ThreadingMixIn will create new threads to handle incoming connection and by default, those are non-daemon threads, so python will wait for all alive non-daemon threads to terminate.

Of course, you could make the server spawn daemon threads, then python will not waiting:

The ThreadingMixIn class defines an attribute daemon_threads, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is False, meaning that Python will not exit until all threads created by ThreadingMixIn have exited.

class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    daemon_threads = True

But that is not the ideal solution, you should check why threads never terminate, usually, the server should stop processing connection when no new data available or client shutdown connection:

import socketserver
import threading


shutdown_evt = threading.Event()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        self.request.setblocking(False)
        while True:
            try:
                msg = self.request.recv(1024)
                if msg == b'shutdown':
                    shutdown_evt.set()
                    break
                elif msg:
                    self.request.send(b'you said: ' + msg)
                if shutdown_evt.wait(0.1):
                    break
            except Exception as e:
                break


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    SERVER = ThreadedTCPServer(('127.0.0.1', 10000), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    shutdown_evt.set()
    SERVER.shutdown()


if __name__ == '__main__':
    run()

Nutgall

Using the above suggested overriding class ThreadedTCPServer solves it ― the server will no longer wait for open connections to terminate by the client side, when using it, rather it will shut down when asked to (using python 3.11). The official docs do touch near, but do not explicitly discuss the relation to open connections nor explicitly guarantee this. – Alanalana

I only beg to defer on one point made in the answer, I think this alone is the ideal solution, when you want your server not to leave control over its shutdown to its clients. Happy to learn any differently. – Alanalana

Answer 3

after more research I found a sample that works using asyncio:

# -*- coding: utf-8 -*-
import asyncio

# after further research I found this relevant europython talk:
#     https://www.youtube.com/watch?v=pi49aiLBas8
# * protocols and transport are useful if you do not have tons of socket based code
# * event loop pushes data in
# * transport used to push data back to the client
# found decent sample in book by wrox "professional python"


class ServerProtocol(asyncio.Protocol):
    def connection_made(self, transport):
        self.transport = transport
        self.write('Welcome')

    def connection_lost(self, exc):
        self.transport = None

    def data_received(self, data):
        if not data or data == '':
            return
        message = data.decode('ascii')
        command = message.strip().split(' ')[0].lower()
        args = message.strip().split(' ')[1:]

        #sanity check
        if not hasattr(self, 'command_%s' % command):
            self.write('Invalid command: %s' % command)
            return

        # run command
        try:
            return getattr(self, 'command_%s' % command)(*args)
        except Exception as ex:
            self.write('Error: %s' % str(ex))

    def write(self, msg):
        self.transport.write((msg + '\n').encode('ascii', 'ignore'))

    def command_shutdown(self):
        self.write('Okay. shutting down')
        raise KeyboardInterrupt

    def command_bye(self):
        self.write('bye then!')
        self.transport.close()
        self.transport = None


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    coro = loop.create_server(ServerProtocol, '127.0.0.1', 8023)
    asyncio.async(coro)
    try:
        loop.run_forever()
    except KeyboardInterrupt:
        pass

I understand that this is the most useful way to do this kind of network programming. If necessary the performance could be improved using the same code with uvloop (https://magic.io/blog/uvloop-blazing-fast-python-networking/).

Absorption

note: this works fine on linux but not on windows since on windows single characters are processed – Absorption

for a working solution see my answer that is using gevent – Absorption

Answer 4

Another way to shut down the server is by creating a process/thread for the serve_forever call.

After server_forever is started, simply wait for a custom flag to trigger and use server_close on the server, and terminate the process.

streaming_server = StreamingServer(('', 8000), StreamingHandler)

FLAG_KEEP_ALIVE.value = True

process_serve_forever = Process(target=streaming_server.serve_forever)
process_serve_forever.start()

while FLAG_KEEP_ALIVE.value:
    pass

streaming_server.server_close()
process_serve_forever.terminate()

Venavenable