redis-py not closing threads on exit

Question

I am using redis-py 2.10.6 and redis 4.0.11.

My application uses redis for both the db and the pubsub. When I shut down I often get either hanging or a crash. The latter usually complains about a bad file descriptor or an I/O error on a file (I don't use any) which happens while handling a pubsub callback, so I'm guessing the underlying issue is the same: somehow I don't get disconnected properly and the pool used by my redis.Redis object is alive and kicking.

An example of the output of the former kind of error (during _read_from_socket):

redis.exceptions.ConnectionError: Error while reading from socket: (9, 'Bad file descriptor')

Other times the stacktrace clearly shows redis/connection.py -> redis/client.py -> threading.py, which proves that redis isn't killing the threads it uses.

When I star the application I run:

self.redis = redis.Redis(host=XXXX, port=XXXX)
self.pubsub = self.redis.pubsub()
subscriptions = {'chan1': self.cb1, 'chan2': self.cb2}  # cb1 and cb2 are functions
self.pubsub.subscribe(**subscriptions)
self.pubsub_thread = self.pubsub.run_in_thread(sleep_time=1)

When I want to exit the application the last instruction I execute in main is a call to a function in my redis using class, whose implementation is:

self.pubsub.close()
self.pubsub_thread.stop()
self.redis.connection_pool.disconnect()

My understanding is that in theory I do not even need to do any of these 'closing' calls, and yet, with or without them, I still can't guarantee a clean shutdown.

My question is, how am I supposed to guarantee a clean shutdown?

Marillin

Running into the same issue. Also seems to be a problem reconnecting to the server when I restart the app, I have to restart the server along with the app, very inconvenient! – Durstin

Answer 1

I ran into this same issue and it's largely caused by improper handling of the shutdown by the redis library. During the cleanup, the thread continues to process new messages and doesn't account for situations where the socket is no longer available. After scouring the code a bit, I couldn't find a way to prevent additional processing without just waiting.

Since this is run during a shutdown phase and it's a remedy for a 3rd party library, I'm not overly concerned about the sleep, but ideally the library should be updated to prevent further action while shutting down.

self.pubsub_thread.stop()
time.sleep(0.5)
self.pubsub.reset()

This might be worth an issue log or PR on the redis-py library.

Nganngc

Answer 2

PubSubWorkerThread class check for self._running.is_set() inside the loop.

To do a "clean shutdown" you should call self.pubsub_thread._running.clean() to set the thread event to false and it will stop.

Check how it work here: https://redis.readthedocs.io/en/latest/_modules/redis/client.html?highlight=PubSubWorkerThread#

Ready

Answer 3

In case someone comes upon this question, I had the same issue and added daemon=True as an additional parameter to the run_in_thread call.

self.pubsub_thread = self.pubsub.run_in_thread(sleep_time=1, deamon=True)

This ensures that the thread doesn't survive the program end and may give you the clean shutdown you are looking for.

Inexcusable