Simple multiprocessing Pool hangs in Jupyter notebook

Question

I'm trying to run some multiprocessing in a Jupyter notebook, using python version 3.7.0. However, even a really simple example seems to hang indefinitely. After reading this answer I tried explicitly calling .close and .join on the pool, but it still hangs. Example code is below, can anyone tell me what's wrong?

import multiprocessing as mp

def fun(x):
    return 2*x

with mp.Pool() as pool:
    args = list(range(10))
    res = pool.map(fun, args)
    pool.close()
    pool.join()

Suiter

multiprocessing has problems working in an interactive prompt. save your code as a .py file, and run it with the python interpreter. (it can be made to work, but it's much simpler to just run the file rather than execute code interactively) – Sobel

If you look at the console where you started up Jupyter Notebook you will see a bunch of errors. You need to put function fun in an external file, for example, workers.py in the same directory as your ipynb file and then add from workers import fun. In other words, worker functions need to be imported. – Outsoar

Does this answer your question? Jupyter notebook never finishes processing using multiprocessing (Python 3) – Outsoar

Answer 1

Another solution is to use the multiprocess module, which is identical to the multiprocessing library while working with Jupyter.

Chalcography

This is the correct answer – Tiger

Does not work for me - VS Code Jupyter / win 10 / Python 3.8.18. Above code hangs without an error on pool.map(). Also does not work with code as an external function.... – Urania

Answer 2

For me, it worked the solution proposed by @Booboo.

• Write your function in an external file
• import it to your .ipynb file

Gee