Semaphores on Python

Question

I've started programming in Python a few weeks ago and was trying to use Semaphores to synchronize two simple threads, for learning purposes. Here is what I've got:

import threading
sem = threading.Semaphore()

def fun1():
    while True:
        sem.acquire()
        print(1)
        sem.release()

def fun2():
    while True:
        sem.acquire()
        print(2)
        sem.release()

t = threading.Thread(target = fun1)
t.start()
t2 = threading.Thread(target = fun2)
t2.start()

But it keeps printing just 1's. How can I intercale the prints?

Capriola

Answer 1

It is working fine, its just that its printing too fast for you to see . Try putting a time.sleep() in both functions (a small amount) to sleep the thread for that much amount of time, to actually be able to see both 1 as well as 2.

Example -

import threading
import time
sem = threading.Semaphore()

def fun1():
    while True:
        sem.acquire()
        print(1)
        sem.release()
        time.sleep(0.25)

def fun2():
    while True:
        sem.acquire()
        print(2)
        sem.release()
        time.sleep(0.25)

t = threading.Thread(target = fun1)
t.start()
t2 = threading.Thread(target = fun2)
t2.start()

Murrumbidgee

Thx for the help, but I found out the real problem, since I'm using the same semaphore in both threads the first one is finishing almost instantly so the second cannot get the lock and execute. – Capriola

@VictorTurrisi Instead of while True if you put a big range and run your program, and then redirect the output to a file and then check the file, you may be able to see that 2 does get printed in between but its like , lots of 1 and then lots of 2 then again lots of 1, etc. This is because it is executing too fast and you need to put a time.sleep() between them to see them executing one after the other. – Murrumbidgee

Answer 2

Also, you can use Lock/mutex method as follows:

import threading
import time

mutex = threading.Lock()  # is equal to threading.Semaphore(1)

def fun1():
    while True:
        mutex.acquire()
        print(1)
        mutex.release()
        time.sleep(.5)

def fun2():
    while True:
        mutex.acquire()
        print(2)
        mutex.release()
        time.sleep(.5)

t1 = threading.Thread(target=fun1).start()
t2 = threading.Thread(target=fun2).start()

Simpler style using "with":

import threading
import time

mutex = threading.Lock()  # is equal to threading.Semaphore(1)

def fun1():
    while True:
        with mutex:
            print(1)
        time.sleep(.5)

def fun2():
    while True:
        with mutex:
            print(2)
        time.sleep(.5)

t1 = threading.Thread(target=fun1).start()
t2 = threading.Thread(target=fun2).start()

---

[NOTE]:

The difference between mutex, semaphore, and lock

Grani

Answer 3

In fact, I want to find asyncio.Semaphores, not threading.Semaphore, and I believe someone may want it too.

So, I decided to share the asyncio.Semaphores, hope you don't mind.

from asyncio import (
    Task,
    Semaphore,
)
import asyncio
from typing import List


async def shopping(sem: Semaphore):
    while True:
        async with sem:
            print(shopping.__name__)
        await asyncio.sleep(0.25)  # Transfer control to the loop, and it will assign another job (is idle) to run.


async def coding(sem: Semaphore):
    while True:
        async with sem:
            print(coding.__name__)
        await asyncio.sleep(0.25)


async def main():
    sem = Semaphore(value=1)
    list_task: List[Task] = [asyncio.create_task(_coroutine(sem)) for _coroutine in (shopping, coding)]
    """ 
    # Normally, we will wait until all the task has done, but that is impossible in your case.
    for task in list_task:
        await task
    """
    await asyncio.sleep(2)  # So, I let the main loop wait for 2 seconds, then close the program.


asyncio.run(main())

output

shopping
coding
shopping
coding
shopping
coding
shopping
coding
shopping
coding
shopping
coding
shopping
coding
shopping
coding

16*0.25 = 2

Conjoint

Answer 4

I used this code to demonstrate how 1 thread can use a Semaphore and the other thread will wait (non-blocking) until the Sempahore is available.

This was written using Python3.6; Not tested on any other version.

This will only work is the synchronization is being done from the same thread, IPC from separate processes will fail using this mechanism.

import threading
from  time import sleep
sem = threading.Semaphore()

def fun1():
    print("fun1 starting")
    sem.acquire()
    for loop in range(1,5):
        print("Fun1 Working {}".format(loop))
        sleep(1)
    sem.release()
    print("fun1 finished")



def fun2():
    print("fun2 starting")
    while not sem.acquire(blocking=False):
        print("Fun2 No Semaphore available")
        sleep(1)
    else:
        print("Got Semphore")
        for loop in range(1, 5):
            print("Fun2 Working {}".format(loop))
            sleep(1)
    sem.release()

t1 = threading.Thread(target = fun1)
t2 = threading.Thread(target = fun2)
t1.start()
t2.start()
t1.join()
t2.join()
print("All Threads done Exiting")

When I run this - I get the following output.

fun1 starting
Fun1 Working 1
fun2 starting
Fun2 No Semaphore available
Fun1 Working 2
Fun2 No Semaphore available
Fun1 Working 3
Fun2 No Semaphore available
Fun1 Working 4
Fun2 No Semaphore available
fun1 finished
Got Semphore
Fun2 Working 1
Fun2 Working 2
Fun2 Working 3
Fun2 Working 4
All Threads done Exiting

Hick

Answer 5

Existing answers are wastefully sleeping

I noticed that almost all answers use some form of time.sleep or asyncio.sleep, which blocks the thread. This should be avoided in real software, because blocking your thread for 0.25, 0.5 or 1 second is unnecessary/wasteful - you could be doing more processing, especially if your application is IO bound - it already blocks when it does IO AND you are introducing arbitrary delays (latency) in your processing time. If all your threads are sleeping, your app isn't doing anything. Also, these variables are quite arbitrary, which is why each answer has a different value they sleep (block the thread for).

The answers are using it as a way to get Python's bytecode interpreter to pre-empt the thread after each print line, so that it alternates deterministically between running the 2 threads. By default, the interpreter pre-empts a thread every 5ms (sys.getswitchinterval() returns 0.005), and remember that these threads never run in parallel, because of Python's GIL

---

Solution to problem

How can I intercale the prints?

So my answer would be, you do not want to use semaphores to print (or process) something in a certain order reliably, because you cannot rely on thread prioritization in Python. See Controlling scheduling priority of python threads? for more. time.sleep(arbitrarilyLargeEnoughNumber) doesn't really work when you have more than 2 concurrent pieces of code, since you don't know which one will run next - see * below. If the order matters, use a queue, and worker threads:

from threading import Thread
import queue

q = queue.Queue()

def enqueue():
    while True:
        q.put(1)
        q.put(2)
      
def reader():
    while True:
        value = q.get()
        print(value)

enqueuer_thread = Thread(target = enqueue)
reader_thread_1 = Thread(target = reader)
reader_thread_2 = Thread(target = reader)
reader_thread_3 = Thread(target = reader)
enqueuer_thread.start()
reader_thread_1.start()
reader_thread_2.start()
reader_thread_3.start()
...

Unfortunately in this problem, you don't get to use Semaphore.

---

*An extra check for you

If you try a modification of the top voted answer but with an extra function/thread to print(3), you'll get:

1
2
3
1
3
2
1
3
...

Within a few prints, the ordering is broken - it's 1-3-2.

Earnest

Answer 6

You need to use 2 semaphores to do what you want to do, and you need to initialize them at 0.

import threading
SEM_FUN1 = threading.Semaphore(0)
SEM_FUN2 = threading.Semaphore(0)


def fun1() -> None:
    for _ in range(5):
        SEM_FUN1.acquire()
        print(1)
        SEM_FUN2.release()


def fun2() -> None:
    for _ in range(5):
        SEM_FUN2.acquire()
        print(2)
        SEM_FUN1.release()


threading.Thread(target=fun1).start()
threading.Thread(target=fun2).start()
SEM_FUN1.release()  # Trigger fun1

Output:

Hersch