PyEval_InitThreads in Python 3: How/when to call it? (the saga continues ad nauseam)

Question

Basically there seems to be massive confusion/ambiguity over when exactly PyEval_InitThreads() is supposed to be called, and what accompanying API calls are needed. The official Python documentation is unfortunately very ambiguous. There are already many questions on stackoverflow regarding this topic, and indeed, I've personally already asked a question almost identical to this one, so I won't be particularly surprised if this is closed as a duplicate; but consider that there seems to be no definitive answer to this question. (Sadly, I don't have Guido Van Rossum on speed-dial.)

Firstly, let's define the scope of the question here: what do I want to do? Well... I want to write a Python extension module in C that will:

1. Spawn worker threads using the pthread API in C
2. Invoke Python callbacks from within these C threads

Okay, so let's start with the Python docs themselves. The Python 3.2 docs say:

void PyEval_InitThreads()

Initialize and acquire the global interpreter lock. It should be called in the main thread before creating a second thread or engaging in any other thread operations such as PyEval_ReleaseThread(tstate). It is not needed before calling PyEval_SaveThread() or PyEval_RestoreThread().

So my understanding here is that:

1. Any C extension module which spawns threads must call PyEval_InitThreads() from the main thread before any other threads are spawned
2. Calling PyEval_InitThreads locks the GIL

So common sense would tell us that any C extension module which creates threads must call PyEval_InitThreads(), and then release the Global Interpreter Lock. Okay, seems straightforward enough. So prima facie, all that's required would be the following code:

PyEval_InitThreads(); /* initialize threading and acquire GIL */
PyEval_ReleaseLock(); /* Release GIL */

Seems easy enough... but unfortunately, the Python 3.2 docs also say that PyEval_ReleaseLock has been deprecated. Instead, we're supposed to use PyEval_SaveThread in order to release the GIL:

PyThreadState* PyEval_SaveThread()

Release the global interpreter lock (if it has been created and thread support is enabled) and reset the thread state to NULL, returning the previous thread state (which is not NULL). If the lock has been created, the current thread must have acquired it.

Er... okay, so I guess a C extension module needs to say:

PyEval_InitThreads();
PyThreadState* st = PyEval_SaveThread();

Indeed, this is exactly what this stackoverflow answer says. Except when I actually try this in practice, the Python interpreter immediately seg-faults when I import the extension module.    Nice.

---

Okay, so now I'm giving up on the official Python documentation and turning to Google. So, this random blog claims all you need to do from an extension module is to call PyEval_InitThreads(). Of course, the documentation claims that PyEval_InitThreads() acquires the GIL, and indeed, a quick inspection of the source code for PyEval_InitThreads() in ceval.c reveals that it does indeed call the internal function take_gil(PyThreadState_GET());

So PyEval_InitThreads() definitely acquires the GIL. I would think then that you would absolutely need to somehow release the GIL after calling PyEval_InitThreads().   But how? PyEval_ReleaseLock() is deprecated, and PyEval_SaveThread() just inexplicably seg-faults.

Okay... so maybe for some reason which is currently beyond my understanding, a C extension module doesn't need to release the GIL. I tried that... and, as expected, as soon as another thread attempts to acquire the GIL (using PyGILState_Ensure), the program hangs from a deadlock. So yeah... you really do need to release the GIL after calling PyEval_InitThreads().

So again, the question is: how do you release the GIL after calling PyEval_InitThreads()?

And more generally: what exactly does a C-extension module have to do to be able to safely invoke Python code from worker C-threads?

Scutari

related: Python code calls C library that create OS threads, which eventually call Python callbacks. See the example c_extension module there (its purpose is to trigger an error in threading while being "correct" to expose a bug in threading implementation. It fails to trigger an error on Python 3). – Rubiaceous

Did you manage to solve this? I'm having the exact same problem and my c application keeps giving segfaults no matter what I do – Encephalograph

Answer 1

Your understanding is correct: invoking PyEval_InitThreads does, among other things, acquire the GIL. In a correctly written Python/C application, this is not an issue because the GIL will be unlocked in time, either automatically or manually.

If the main thread goes on to run Python code, there is nothing special to do, because Python interpreter will automatically relinquish the GIL after a number of instructions have been executed (allowing another thread to acquire it, which will relinquish it again, and so on). Additionally, whenever Python is about to invoke a blocking system call, e.g. to read from the network or write to a file, it will release the GIL around the call.

The original version of this answer pretty much ended here. But there is one more thing to take into account: the embedding scenario.

When embedding Python, the main thread often initializes Python and goes on to execute other, non-Python-related tasks. In that scenario there is nothing that will automatically release the GIL, so this must be done by the thread itself. That is in no way specific to the call that calls PyEval_InitThreads, it is expected of all Python/C code invoked with the GIL acquired.

For example, the main() might contain code like this:

Py_Initialize();
PyEval_InitThreads();

Py_BEGIN_ALLOW_THREADS
... call the non-Python part of the application here ...
Py_END_ALLOW_THREADS

Py_Finalize();

If your code creates threads manually, they need to acquire the GIL before doing anything Python-related, even as simple as Py_INCREF. To do so, use the following:

// Acquire the GIL
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

... call Python code here ...

// Release the GIL. No Python API allowed beyond this point.
PyGILState_Release(gstate);

Gee

PyGILState_Ensure()/PyGILState_Release() should be used to invoke Python from a C thread instead of PyEval_{Save,Restore}Thread() See Thread State and the Global Interpreter Lock – Rubiaceous

@J.F.Sebastian Agreed, I've now edited the answer to advertise the correct API. – Gee

But, as I mentioned in the question, if the main thread is holding the GIL (which it will be after it calls PyEval_InitThreads(), then when a worker thread calls PyGILState_Ensure, it's going to deadlock. – Scutari

@Scutari Why would it deadlock? It will only wait as long as necessary to acquire the GIL. The main thread must relinquish the GIL sooner or later—it will either entering a blocking syscall that releases the GIL, or the interpreter will release it automatically after executing a number of bytecode instructions. – Gee

@Gee - It would deadlock if the thread that calls PyEval_InitThreads doesn't actually do anything with Python. It happened in an Init call of some sort, so it didn't necessarily need to call Python at that point. So does that thread just go about its business and ignore the GIL lock for the rest of the program execution? – Entrust

@Entrust How can it "not do anything with Python"? One doesn't just call PyEval_InitThreads at random. In fact, the answer explicitly recommends to call PyEval_InitThreads in the extension's <module>init. If this is adhered to, the init function will return to the Python caller that triggered the import. Python will happily continue on its way, eventually releasing the GIL, as described in the answer in some detail. – Gee

For me doing this still leads to a crash on the C++ side. I get an Access violation ... – Diabetic

@Gee Regarding your question "why would it deadlock?": As I understand it, PyGILState_Ensure(), which tries to retrieve the GIL, WILL deadlock (specifically, get stuck in an infinite loop within the function take_gil() in the Python source code) if the GIL is not retrievable. To make it retreivable, following a call to PyEval_InitThreads(), the code calling that function must also call PyEval_SaveThread(). (Furthermore, PyEval_RestoreThread() will again make the GIL unavailable for retrieval by PyGILState_Ensure().) – Gord

@Gord But why would that situation arise, other than facing malicious or patently incorrect code? The extension code that calls PyEval_InitThreads is obviously Python-aware. So its thread will either keep calling Python, which will result in the GIL being released sooner or later (by the interpreter loop or by a blocking-IO call from Python); or it will call into C code, in which case it is its responsibility to release the GIL before doing so (and reacquire it afterwards). I honestly don't understand where all the confusion lies - when I do, I will amend the answer to address it. – Gee

I'm dealing with a different case than the OP (see this answer or my SO question): I'm initializing the interpreter from C++ code (i.e. not from a Python extension) that won't interact with the Python API further, except to finalize the interpreter. So if I don't explicitly release the GIL, it'll never get released, and any call to PyGILState_Ensure() from a thread started from the C++ code will get stuck in an infinite loop trying to acquire the GIL. – Gord

@Gord I think I now understand the confusion. I've amended the answer to address it. – Gee

To summarize everything above: The older PyEval_InitThreads functionality has been integrated into Py_Initialize. Therefore, Py_Initialize is essentially holding the GIL hostage. The GIL must then be explicitly released via Py_BEGIN_ALLOW_THREADS or PyEval_SaveThread to prevent PyGILState_Ensure from deadlocking other threads. – Marianomaribel

Answer 2

There are two methods of multi threading while executing C/Python API.

1.Execution of different threads with same interpreter - We can execute a Python interpreter and share the same interpreter over the different threads.

The coding will be as follows.

main(){     
//initialize Python
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
    "print 'In Main, Today is',ctime(time())\n");

//to Initialize and acquire the global interpreter lock
PyEval_InitThreads();

//release the lock  
PyThreadState *_save;
_save = PyEval_SaveThread();

// Create threads.
for (int i = 0; i<MAX_THREADS; i++)
{   
    hThreadArray[i] = CreateThread
    //(...
        MyThreadFunction,       // thread function name
    //...)

} // End of main thread creation loop.

// Wait until all threads have terminated.
//...
//Close all thread handles and free memory allocations.
//...

//end python here
//but need to check for GIL here too
PyEval_RestoreThread(_save);
Py_Finalize();
return 0;
}

//the thread function

DWORD WINAPI MyThreadFunction(LPVOID lpParam)
{
//non Pythonic activity
//...

//check for the state of Python GIL
PyGILState_STATE gilState;
gilState = PyGILState_Ensure();
//execute Python here
PyRun_SimpleString("from time import time,ctime\n"
    "print 'In Thread Today is',ctime(time())\n");
//release the GIL           
PyGILState_Release(gilState);   

//other non Pythonic activity
//...
return 0;
}

2. Another method is that, we can execute a Python interpreter in the main thread and, to each thread we can give its own sub interpreter. Thus every thread runs with its own separate , independent versions of all imported modules, including the fundamental modules - builtins, __main__ and sys.

The code is as follows

int main()
{

// Initialize the main interpreter
Py_Initialize();
// Initialize and acquire the global interpreter lock
PyEval_InitThreads();
// Release the lock     
PyThreadState *_save;
_save = PyEval_SaveThread();


// create threads
for (int i = 0; i<MAX_THREADS; i++)
{

    // Create the thread to begin execution on its own.

    hThreadArray[i] = CreateThread
    //(...

        MyThreadFunction,       // thread function name
    //...);   // returns the thread identifier 

} // End of main thread creation loop.

  // Wait until all threads have terminated.
WaitForMultipleObjects(MAX_THREADS, hThreadArray, TRUE, INFINITE);

// Close all thread handles and free memory allocations.
// ...


//end python here
//but need to check for GIL here too
//re capture the lock
PyEval_RestoreThread(_save);
//end python interpreter
Py_Finalize();
return 0;
}

//the thread functions
DWORD WINAPI MyThreadFunction(LPVOID lpParam)
{
// Non Pythonic activity
// ...

//create a new interpreter
PyEval_AcquireLock(); // acquire lock on the GIL
PyThreadState* pThreadState = Py_NewInterpreter();
assert(pThreadState != NULL); // check for failure
PyEval_ReleaseThread(pThreadState); // release the GIL


// switch in current interpreter
PyEval_AcquireThread(pThreadState);

//execute python code
PyRun_SimpleString("from time import time,ctime\n" "print\n"
    "print 'Today is',ctime(time())\n");

// release current interpreter
PyEval_ReleaseThread(pThreadState);

//now to end the interpreter
PyEval_AcquireThread(pThreadState); // lock the GIL
Py_EndInterpreter(pThreadState);
PyEval_ReleaseLock(); // release the GIL

// Other non Pythonic activity
return 0;
}

It is necessary to note that the Global Interpreter Lock still persists and, in spite of giving individual interpreters to each thread, when it comes to python execution, we can still execute only one thread at a time. GIL is UNIQUE to PROCESS, so in spite of providing unique sub interpreter to each thread, we cannot have simultaneous execution of threads

Sources: Executing a Python interpreter in the main thread and, to each thread we can give its own sub interpreter

Multi threading tutorial (msdn)

Citrus

Answer 3

I have seen symptoms similar to yours: deadlocks if I only call PyEval_InitThreads(), because my main thread never calls anything from Python again, and segfaults if I unconditionally call something like PyEval_SaveThread(). The symptoms depend on the version of Python and on the situation: I am developing a plug-in that embeds Python for a library that can be loaded as part of a Python extension. The code needs therefore to run independent of whether it is loaded by Python as main.

The following worked for be with both python2.7 and python3.4, and with my library running within Python and outside of Python. In my plug-in init routine, which is executed in the main thread, I run:

  Py_InitializeEx(0);
  if (!PyEval_ThreadsInitialized()) {
    PyEval_InitThreads();
    PyThreadState* mainPyThread = PyEval_SaveThread();
  }

(mainPyThread is actually some static variable, but I don't think that matters as I never need to use it again).

Then I create threads using pthreads, and in each function that needs to access the Python API, I use:

  PyGILState_STATE gstate;
  gstate = PyGILState_Ensure();
  // Python C API calls
  PyGILState_Release(gstate);

Crowded

I'm using Python 2.6 and if I run this solution in the main thread it hangs the main thread. – Diabetic

I guess the proper way to go is to eventually retrieve the GIL in the main thread again via PyEval_RestoreThread(mainPyThread), somewhere before Py_FinalizeEx(). Make sure to wait until all threads have finished their execution before doing this though, or any call to PyGILState_Ensure() in any thread will deadlock (infinite loop within the function take_gil() in the Python source code), because the GIL is now unavailable again. – Gord

@Gord Indeed, if the thread that calls PyEval_InitThreads really doesn't do anything Python-related for the rest of its lifetime, then it is correct to immediately call PyEval_SaveThread, and only call PyEval_RestoreThread at shutdown. I have a final nit - you might be misusing the term "deadlock", which means "waiting for a condition that can never be met". take_gil will in fact succeed to take the GIL as soon as it is made available. – Gee

@Gee I agree, I'm misusing the term; the thread is stuck in a loop which continually tries to acquire the GIL, so it may well eventually succeed if something changes in another thread to make it available. But what do you mean when you say "anything Python-related"? Isn't releasing the GIL something that has to be done explicitly? Or do you mean that any interaction with the Python API has implications for the GIL? – Gord

@Gord Yes, exactly. Whenever you interact with the Python interpreter in any way, you must hold the GIL, and Python will release it for you in two ways: 1) in regular intervals as Python code is executing (this allows other threads that need the GIL to run), and 2) around the call to a blocking or potentially blocking function (e.g. anything to do with IO). It is only when you write your own C code having nothing to do with Python that you need to release the GIL manually - and even then the GIL must be reacquired before doing anything with Python in that thread again. – Gee

@Gee That last example, that's exactly what I'm doing :). Thanks for the information, the picture is much clearer! – Gord

Answer 4

To quote above:

The short answer: you shouldn't care about releasing the GIL after calling PyEval_InitThreads...

Now, for a longer answer:

I'm limiting my answer to be about Python extensions (as opposed to embedding Python). If we are only extending Python, than any entry point into your module is from Python. This by definition means that we don't have to worry about calling a function from a non-Python context, which makes things a bit simpler.

If threads have NOT be initialized, then we know there is no GIL (no threads == no need for locking), and thus "It is not safe to call this function when it is unknown which thread (if any) currently has the global interpreter lock" does not apply.

if (!PyEval_ThreadsInitialized())
{
    PyEval_InitThreads();
}

After calling PyEval_InitThreads(), a GIL is created and assigned... to our thread, which is the thread currently running Python code. So all is good.

Now, as far as our own launched worker "C"-threads, they will need to ask for the GIL before running relevant code: so their common methodology is as follows:

// Do only non-Python things up to this point
PyGILState_STATE state = PyGILState_Ensure();
// Do Python-things here, like PyRun_SimpleString(...)
PyGILState_Release(state);
// ... and now back to doing only non-Python things

We don't have to worry about deadlock any more than normal usage of extensions. When we entered our function, we had control over Python, so either we were not using threads (thus, no GIL), or the GIL was already assigned to us. When we give control back to the Python run-time by exiting our function, the normal processing loop will check the GIL and hand control of as appropriate to other requesting objects: including our worker threads via PyGILState_Ensure().

All of this the reader probably already knows. However, the "proof is in the pudding". I've posted a very-minimally-documented example that I wrote today to learn for myself what the behavior actually was, and that things work properly. Sample Source Code on GitHub

I was learning several things with the example, including CMake integration with Python development, SWIG integration with both of the above, and Python behaviors with extensions and threads. Still, the core of the example allows you to:

• Load the module -- 'import annoy'
• Load zero or more worker threads which do Python things -- 'annoy.annoy(n)'
• Clear any worker threads -- 'annon.annoy(0)'
• Provide thread cleanup (on Linux) at application exit

... and all of this without any crashes or segfaults. At least on my system (Ubuntu Linux w/ GCC).

Recitation

Answer 5

The suggestion to call PyEval_SaveThread works

PyEval_InitThreads();
PyThreadState* st = PyEval_SaveThread();

However to prevent crash when module is imported, ensure Python APIs to import are protected using

PyGILState_Ensure and PyGILState_Release

e.g.

PyGILState_STATE gstate = PyGILState_Ensure();
PyObject *pyModule_p = PyImport_Import(pyModuleName_p);
PyGILState_Release(gstate);

Paugh

Answer 6

I feel confuse on this issue too. The following code works by coincidence.

Py_InitializeEx(0);
    if (!PyEval_ThreadsInitialized()) {
    PyEval_InitThreads();
    PyThreadState* mainPyThread = PyEval_SaveThread();
}

My main thread do some python runtime initial work, and create other pthread to handle tasks. And I have a better workaround for this. In the Main thread:

if (!PyEval_ThreadsInitialized()){
    PyEval_InitThreads();
}
//other codes
while(alive) {
    Py_BEGIN_ALLOW_THREADS
    sleep or other block code
    Py_END_ALLOW_THREADS
}

Oaken

Answer 7

You don't need to call that in your extension modules. That's for initializing the interpreter which has already been done if your C-API extension module is being imported. This interface is to be used by embedding applications.

When is PyEval_InitThreads meant to be called?

Pyxidium

no, you need to call PyEval_InitThreads if you are planning to do Python callbacks from multiple non-Python threads (as the answer that you've linked says) – Rubiaceous

After looking at the source a bit, it would appear that you are technically correct. However, the documentation makes no such suggestion for use, so I would avoid using the interface in such a way. In fact, docs.python.org/3.3/c-api/init.html#PyEval_InitThreads states that subsequent calls are a no-op. – Pyxidium

Okay, I guess C-API extension modules that are demanding Python threads would desire to call this to make sure that the gil is initialized. Personally, if an embedding application did not initialize Python threading, I would think twice about running threads. – Pyxidium