process-pool Questions
0
I am trying to capture all output from a ProcessPoolExecutor.
Imagine you have a file func.py:
print("imported") # I do not want this print in subprocesses
def f(x):
return x
then you ...
Electromagnetism asked 6/7, 2022 at 12:42
4
Solved
I was using this answer in order to run parallel commands with multiprocessing in Python on a Linux box.
My code did something like:
import multiprocessing
import logging
def cycle(offset):
# ...
Coupling asked 14/8, 2017 at 0:17
4
Solved
I am doing some parallel processing, as follows:
with mp.Pool(8) as tmpPool:
results = tmpPool.starmap(my_function, inputs)
where inputs look like:
[(1,0.2312),(5,0.52) ...]
i.e., tuples of an...
Miscellanea asked 5/8, 2019 at 8:20
1
Context:
A Python application server that uses a concurrent.futures.process.ProcessPool to execute code
We sometimes want to hot reload imported code without restarting the entire server process
...
Jeremiahjeremias asked 16/10, 2020 at 16:27
3
Solved
So, I know the difference between the two methods in the title, but not the practical implications.
From what I understand: If you use more NUM_WORKERS than are cores actually available, you face b...
Confiscate asked 3/10, 2020 at 21:26
2
Solved
I have a method inside a class that needs to do a lot of work in a loop, and I would like to spread the work over all of my cores.
I wrote the following code, which works if I use normal map(), but...
Constant asked 10/9, 2018 at 20:35
1
if __name__ == '__main__':
MATCH_ID = str(doc_ref2.id)
MATCH_ID_TEAM = doc_ref3.id
with concurrent.futures.ProcessPoolExecutor(max_workers=30) as executor:
results = list(executor.map(ESPNla...
Splendent asked 25/11, 2019 at 21:23
1
Solved
1. Why does the following Python code using the concurrent.futures module hang forever?
import concurrent.futures
class A:
def f(self):
print("called")
class B(A):
def f(self):
...
Handicap asked 15/6, 2019 at 11:16
0
I want to monitor progress across multiple workers which are different processes. For each subprocess I have its own progress bar but it doest work properly with ProcessPoolExecutor executor.
def ...
Kreplach asked 4/3, 2019 at 14:43
1
Solved
Similar Question (but answer does not work for me): How to cancel long-running subprocesses running using concurrent.futures.ProcessPoolExecutor?
Unlike the question linked above and the soluti...
Serrulation asked 22/10, 2018 at 1:24
1
Solved
What's the proper way of aborting multiprocessing when one of the child aborts and/or throw an Exception?
I found various questions around that (generic multiprocessing error handling, how to clos...
Hairy asked 11/9, 2018 at 8:59
4
Solved
This code runs fine under regular CPython 3.5:
import concurrent.futures
def job(text):
print(text)
with concurrent.futures.ProcessPoolExecutor(1) as pool:
pool.submit(job, "hello")
But if y...
Tomkins asked 12/1, 2018 at 2:55
1
© 2022 - 2024 — McMap. All rights reserved.