I have Python 3.8 and cannot upgrade (I have dependencies that only work with Python 3.8), so this SO post does not answer my question. How do I prevent the OSError: handle is closed error? This happens when returning a result to the main thread. Here is a very trivial example:
from concurrent.futures import ProcessPoolExecutor
def my_func(num):
return num + 1
def worker(data):
with ProcessPoolExecutor(5) as executor:
result = executor.map(my_func, data)
return result
if __name__ == "__main__":
arr = [1, 2, 3, 4, 5]
print(arr)
arr = worker(arr)
print(tuple(arr))
This gives
[1, 2, 3, 4, 5]
(2, 3, 4, 5, 6)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Users\hendra11\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\process.py", line 102, in _python_exit
thread_wakeup.wakeup()
File "C:\Users\hendra11\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\process.py", line 90, in wakeup
self._writer.send_bytes(b"")
File "C:\Users\hendra11\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py", line 183, in send_bytes
self._check_closed()
File "C:\Users\hendra11\AppData\Local\Programs\Python\Python38\lib\multiprocessing\connection.py", line 136, in _check_closed
raise OSError("handle is closed")
OSError: handle is closed
Does anyone know what's causing this bug and how I can manually fix it?
If it matters, I'm on Windows 10 running Visual Studio Code.
UPDATE: This error goes away when I use ThreadPoolExecutor AND it is much faster, but I don't know why.