How to cleanly sleep indefinitely?

Question

A few threads are started in my code and I need at the end of the script to sleep indefinitely, without this sleep being a major hit on the performance¹.

One possibility can be to loop indefinitely with a short sleep:

while True:
    time.sleep(1)

or sleep for a long time

time.sleep(4000000)

or

import signal
signal.pause()

But:

• I did not manage to find the largest time sleep would accept (sys.maxint is too large)

• signal.pause() is implemented in Unix only

• and the first "sleep loop" does not look clean to me (why 1 second and not 10, or 0.1?)

Is there a clean, pythonic way to sleep indefinitely?

---

¹ I do not control the threads directly, otherwise I would have gone for threading.Thread.join() as the threads themselves will not end.

Finagle

why? you want a job scheduler perhaps? – Bank

How about you join() the threads or the thread pool? – Bustamante

"cleanly sleep" looks like an oxymoron... – Jutland

@Chris_Rands: the long story is that I use a messaging library (Paho MQTT) which starts its own threads (which in turn never stop, they are responsible for listening to incoming and outgoing events). I do not have control over these threads so I just would like to suspend the main thread (or sleep) while the other threads do their work. I could make use of the indefinite loop in the library in a separate thread, but I am afraid of border effects of such "thread in thread" approach (maybe overly afraid) – Finagle

@KlausD. please see the footnote in my question – Finagle

There's nothing wrong with sleeping for a long time in an infinite loop. Every N seconds, the scheduler will wake your thread, loop around to the next sleep and then move on. No harm done; very quick. Your MQTT library probably has a method to do this for you. The Eclipse Paho MQTT Python docs show client.loop_forever() as an example. – Errhine

threading.enumerate gives you the list of all running threads including the main one, so you could wait (join) them. – Rapping

to me sleep indefinitely == stop running. What is the condition that stops sleeping? If there is none at all then why aren't you just letting the process end? – Germanic

Given that something like while True: i+= 1 will increment i millions of times per second, calling time.sleep(1) introduces a relatively tiny amount of overhead, and sleeping longer as your granularity allows introduces even less. – Ankylosaur

@TadhgMcDonald-Jensen: because there are other threads which must be running and I do not want them to die because the main thread is over – Finagle

@CristianCiupitu: that's a great idea - would you mind turning this into an answer (or just bootstrap it so that I can edit with actual code), I would be glad to accept it. It seems to be the most natural one in my case and I did not know that i could enumerate all running thread, even the ones I did not start explicitly. – Finagle

Answer 1

threading.enumerate gives you the list of all running threads including the main one, so you could do this:

main_thread = threading.main_thread()
while True:
    L = threading.enumerate()
    L.remove(main_thread)  # or avoid it in the for loop
    for t in L:
        t.join()

The while True is needed in case your library creates new threads while you wait for the current ones to finish.

Assuming that no threads are created while enumerate is running, you can check if L has only one element (the main thread) and if so, break the loop. This combined with Tadhg McDonald-Jensen's suggestion of using iter with a sentinel, results in:

main_thread = threading.main_thread()
main_threads = [main_thread, ]  # WARN: can't have more than one thread here
for threads in iter(threading.enumerate, main_threads):
    for t in threads:
        if t == main_thread:
            continue
        t.join()

enumerate returns a list in undefined order, so if you have more than one "main" thread, order starts to matter. A solution would be to use sets, i.e. main_threads = {main_thread, } and iter(lambda : set(threading.enumerate()), main_threads).

If you prefer the EAFP approach of asking for forgiveness instead of permission and all your threads are started when you reach the end of your script, you can also do this:

for thread in threading.enumerate():
    try:
        thread.join()
    except RuntimeError:
        # trying to join the main thread, which would create a deadlock (see https://docs.python.org/3/library/threading.html#threading.Thread.join for details)
        pass

Rapping

you may want to rewrite it slightly to have while L instead of while True so when there are no more threads you can let the thing stop. you can also do for L in iter(threading.enumerate, []): which basically reads "keep calling threading.enumerate() and keep running this loop with the result until it gives an empty list" – Germanic

@TadhgMcDonald-Jensen, I've expanded my answer based on your suggestion. Thanks for the idea. – Rapping

That makes more sense than checking for empty list and I really like the way you used it, although I do need to point out that if you wanted to have more than one "main thread" I'm not sure you can guarantee the order they appear in the list so I'm not sure it would directly scale to multiple main threads. Either way very good solution. – Germanic

This does not look very exactly pythonic but due no better answer upvoting. – Carruth