multiprocessing Questions
10
Solved
What's the best way to wait (without spinning) until something is available in either one of two (multiprocessing) Queues, where both reside on the same system?
Optometrist asked 14/7, 2009 at 6:58
2
Solved
We all know that using asyncio substantially improves the performance of a socket server, and obviously things get even more awesome if we could take advantage of all cores in our cpu (maybe via mu...
Tyndall asked 1/6, 2019 at 18:49
14
Solved
In the example code below, I'd like to get the return value of the function worker. How can I go about doing this? Where is this value stored?
Example Code:
import multiprocessing
def worker(procn...
Inept asked 2/5, 2012 at 13:44
4
Solved
I am trying to create a class than can run a separate process to go do some work that takes a long time, launch a bunch of these from a main module and then wait for them all to finish. I want to l...
Samarasamarang asked 5/1, 2013 at 7:10
11
Solved
To make my code more "pythonic" and faster, I use multiprocessing and a map function to send it a) the function and b) the range of iterations.
The implanted solution (i.e., calling tqdm ...
Islean asked 29/1, 2017 at 10:58
3
Say I have a set of 20 CPU heavy tasks (~1 hour each, but some take a lot longer) that are run by calling a function via e.g. Pool.apply_async(function, task_list) etc. PC has 12 Cores. So I can sp...
Meiny asked 24/1, 2019 at 16:59
2
Solved
I'm using the multiprocessing module in Python 3 but for some reason, it keeps throwing TypeError: 'int' object is not iterable when I run the program. This is what I did:
def main(i):
global url...
Pita asked 7/8, 2015 at 18:13
5
Solved
I am trying to solve a big numerical problem which involves lots of subproblems, and I'm using Python's multiprocessing module (specifically Pool.map) to split up different independent subproblems ...
Campbell asked 19/11, 2012 at 1:13
6
Solved
Is there a way to log the stdout output from a given Process when using the multiprocessing.Process class in python?
Conquistador asked 1/10, 2009 at 2:44
1
Solved
According to this SO answer, when a multiprocessing.Queue is passed to a child process, what is actually sent is a file descriptor (or handle) obtained from pipe, instead of pickling the Queue obje...
Sacculus asked 28/10, 2023 at 16:11
2
Question related to the topic of Parallelism in Self-Hosted Runners:
One self-hosted runner can only run one job at a time, when no available runners are idle, the subsequent jobs will be in que...
Tellurion asked 9/4, 2020 at 15:11
4
Solved
I'm using Python 2.7.3. I have parallelised some code using subclassed multiprocessing.Process objects. If there are no errors in the code in my subclassed Process objects, everything runs fine. Bu...
Adnate asked 20/3, 2013 at 22:51
3
Solved
I run the code in parallel in the following fashion:
grouped_data = Parallel(n_jobs=14)(delayed(function)(group) for group in grouped_data)
After the computation is done I can see all the spawned ...
Israelite asked 11/5, 2021 at 22:50
9
Solved
I am trying to run multiple functions in parallel in Python.
I have something like this:
files.py
import common #common is a util class that handles all the IO stuff
dir1 = 'C:\folder1'
dir2 = 'C...
Vannoy asked 26/8, 2011 at 15:46
2
I'm running a spell correction function on a dataset I have. I used from pathos.multiprocessing import ProcessingPool as Pool to do the job. Once the processing is done, I'd like to actually access...
Nematic asked 9/1, 2019 at 18:36
5
Solved
I'm looking for a possibility to use uvicorn.run() with a FastAPI app but without uvicorn.run() is blocking the thread. I already tried to use processes, subprocessesand threads but nothing worked....
Hooper asked 3/5, 2020 at 16:8
5
Solved
I've read through the documentation, but I don't understand what is meant by:
The delayed function is a simple trick to be able to create a tuple (function, args, kwargs) with a function-call synt...
Shalna asked 14/2, 2017 at 7:44
3
I'm using multiprocessing to create a sub-process to my Python app.
I would like to share data between my parent process and the child process.
it's important to mention that I need to share this a...
Ushaushant asked 2/2, 2016 at 15:25
3
Solved
I currently have a MQTT code that can subscribe to a topic, print out the messages received, then publish further instructions to a new topic. The subscribing/printing is completed in one Goroutine...
Calcutta asked 28/6, 2018 at 21:33
2
I am looking to use the multiprocessing module to speed up the run time of some Transport Planning models. I've optimized as much as I can via 'normal' methods but at the heart of it is an absurdly...
Lipscomb asked 3/11, 2011 at 0:50
1
Solved
I'm starting to include multiprocessing into my code, as the task I'm trying to automate is rather costly computationally speaking. The module structure in the code I have is the following, based o...
Excitant asked 17/3, 2021 at 10:1
1
I took multiprocessing example for Stable Baselines 3 and everything was fine.
https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/multiprocessing_rl.ipynb#sc...
Miche asked 29/4, 2022 at 13:56
4
The following code hangs without doing anything in Python 3.2.2 in Linux:
import tkinter
from multiprocessing import Process
def f():
root = tkinter.Tk()
label = tkinter.Label(root)
label.pack(...
Crave asked 25/5, 2012 at 13:54
11
Solved
Is there a simple way to track the overall progress of a joblib.Parallel execution?
I have a long-running execution composed of thousands of jobs, which I want to track and record in a database. H...
Ayer asked 27/7, 2014 at 17:20
3
Solved
I just noticed that my program is using more and more memory as it processes a large file. It's only processing one line at a time, though, so I couldn't figure out why it would keep using more mem...
Sacken asked 1/12, 2016 at 23:55
© 2022 - 2024 — McMap. All rights reserved.