Does new implementation of GIL in Python handled race condition issue?

Question

I've read an article about multithreading in Python where they trying to use Synchronization to solve race condition issue. And I've run the example code below to reproduce race condition issue:

import threading 

# global variable x 
x = 0

def increment(): 
    """ 
    function to increment global variable x 
    """
    global x 
    x += 1

def thread_task(): 
    """ 
    task for thread 
    calls increment function 100000 times. 
    """
    for _ in range(100000): 
        increment() 

def main_task(): 
    global x 
    # setting global variable x as 0 
    x = 0

    # creating threads 
    t1 = threading.Thread(target=thread_task) 
    t2 = threading.Thread(target=thread_task) 

    # start threads 
    t1.start() 
    t2.start() 

    # wait until threads finish their job 
    t1.join() 
    t2.join() 

if __name__ == "__main__": 
    for i in range(10): 
        main_task() 
        print("Iteration {0}: x = {1}".format(i,x)) 

It does return the same result as the article when I'm using Python 2.7.15. But it does not when I'm using Python 3.6.9 (all threads return the same result = 200000).

I wonder that does new implementation of GIL (since Python 3.2) was handled race condition issue? If it does, why Lock, Mutex still exist in Python >3.2 . If it doesn't, why there is no conflict when running multi threading to modify shared resource like the example above?

My mind was struggling with those question in these days when I'm trying to understand more about how Python really works under the hood.

Yttriferous

Using Python 3.8, I get different, less than 200000 results almost every time. – Lepore

The GIL has no idea what is a race condition for your code. It did and still does only protect individual byte-code instructions. Race conditions triggering only sometimes is exactly why they are dangerous -- e.g. success of your test is misleading precisely because it has a race condition. – Nuncupative

Answer 1

The change you are referring to was to replace check interval with switch interval. This meant that rather than switching threads every 100 byte codes it would do so every 5 milliseconds.

Ref: https://pymotw.com/3/sys/threads.html https://mail.python.org/pipermail/python-dev/2009-October/093321.html

So if your code ran fast enough, it would never experience a thread switch and it might appear to you that the operations were atomic when they are in fact not. The race condition did not appear as there was no actual interweaving of threads. x += 1 is actually four byte codes:

>>> dis.dis(sync.increment)
 11           0 LOAD_GLOBAL              0 (x)
              3 LOAD_CONST               1 (1)
              6 INPLACE_ADD         
              7 STORE_GLOBAL             0 (x)
             10 LOAD_CONST               2 (None)
             13 RETURN_VALUE        

A thread switch in the interpreter can occur between any two bytecodes.

Consider that in 2.7 this prints 200000 always because the check interval is set so high that each thread completes in its entirety before the next runs. The same can be constructed with switch interval.

import sys
import threading 

print(sys.getcheckinterval())
sys.setcheckinterval(1000000)

# global variable x 
x = 0

def increment(): 
    """ 
    function to increment global variable x 
    """
    global x 
    x += 1

def thread_task(): 
    """ 
    task for thread 
    calls increment function 100000 times. 
    """
    for _ in range(100000): 
        increment() 

def main_task(): 
    global x 
    # setting global variable x as 0 
    x = 0

    # creating threads 
    t1 = threading.Thread(target=thread_task) 
    t2 = threading.Thread(target=thread_task) 

    # start threads 
    t1.start() 
    t2.start() 

    # wait until threads finish their job 
    t1.join() 
    t2.join() 

if __name__ == "__main__": 
    for i in range(10): 
        main_task() 
        print("Iteration {0}: x = {1}".format(i,x)) 

Recede

Answer 2

The GIL protects individual byte code instructions. In contrast, a race condition is an incorrect ordering of instructions, which means multiple byte code instructions. As a result, the GIL cannot protect against race conditions outside of the Python VM itself.

---

However, by their very nature race conditions do not always trigger. Certain GIL strategies are more or less likely to trigger certain race conditions. A thread shorter than the GIL window is never interrupted, and one longer than the GIL window is always interrupted.

Your increment function has 6 byte code instructions, as has the inner loop calling it. Of these, 4 instructions must finish at once, meaning there are 3 possible switching points that corrupt the result. Your entire thread_task function takes about 0.015s to 0.020s (on my system).

With the old GIL switching every 100 instructions, the loop is guaranteed to be interrupted every 8.3 calls, or roughly 1200 times. With the new GIL switching every 5ms, the loop is interrupted only 3 times.

Nuncupative