I have some subprocesses (using multiprocessing) and when they stop, each of them need do some final work. Something like the following, which did not work though... import multiprocessing import...

Python concurrent.futures and ProcessPoolExecutor provide a neat interface to schedule and monitor tasks. Futures even provide a .cancel() method: cancel(): Attempt to cancel the call. If the ca...

I have more than 10 tasks to execute, and the system restrict that there at most 4 tasks can run at the same time. My task can be started like: myprog taskname How can I write a bash shell script...

We have been told that Google Chrome runs each tab in a separate process. Therefore a crash in one tab would not cause problems in the other tabs. AFAIK, multi-processes are mostly used in program...

I am facing the problem with collecting logs from the following script. Once I set up the SLEEP_TIME to too "small" value, the LoggingThread threads somehow block the logging module. The script fre...

I instantiate one grpc server per subprocess by multiprocess pool. When I use multiple clients to access the server, I found the following two problems： all clients access same server subprocess c...

I added a progress bar to my 2.7 python code using tqdm but it has slowed down my code significantly. Without the progress bar for one example it takes 12 seconds while with the progress bar it tak...

This is the code I use to parrellize an apply function on the lines of a pandas.DataFrame object: from multiprocessing import cpu_count, Pool from functools import partial def parallel_applymap_d...

I am trying to use forkserver and I encountered NameError: name 'xxx' is not defined in worker processes. I am using Python 3.6.4, but the documentation should be the same, from https://docs.python...

I am using python's Multiprocess.Pool to plot some data using multiple processes as follows: class plotDriver: def plot(self, parameterList): numberOfWorkers = len(parameterList) pool = Pool(n...

I have a small pool of workers (4) and a very large list of tasks (5000~). I'm using a pool and sending the tasks with map_async(). Because the task I'm running is fairly long, I'm forcing a chunks...

I am currently trying to use python multiprocessing. The library I use is multiprocess (NOT multiprocessing). I have the following code, which creates a number of computing jobs, and runs it thro...

I am using sqlite in a multi process scenario. The sqlite library was compiled with the threadsafe serialized mode (-DSQLITE_THREADSAFE=1). I want to be notified on datachanges and found sqlite3_u...

I have a large pandas dataframe with multiple "records" consisting of 2 or more line items. I'm trying to efficiently perform a CPU intensive calculation on each record using multiprocessing. Here'...

I was about to write a shell with C language. Here is the source code below: #include <unistd.h> #include <stdio.h> #include <string.h> #include <sys/wait.h> #include <s...

I am fairly new to Python, and my experience is specific to its use in Powerflow modelling through the API provided in Siemens PSS/e. I have a script that I have been using for several years that r...

I have a multidimensional array (result) that should be filled by some nested loops. Function fun() is a complex and time-consuming function. I want to fill my array elements in a parallel manner, ...

I am implementing a multithreaded program that uses different cores, and many threads are executed simultaneously. Each thread makes a printf() call, and the result is not readable. How can I make...

I'm working on a mathematical model that uses data generated from XFOIL, a popular aerospace tool used to find the lift and drag coefficients on airfoils. I have a Perl script that calls XFOIL rep...

Yesterday i asked a question: Reading data in parallel with multiprocess I got very good answers, and i implemented the solution mentioned in the answer i marked as correct. def read_energies(mo...

I want to pull all the items currently in a queue. There is another thread constantly putting items in the other end, and every period I want to get all the items currently in the queue. Is there ...

Let's say I want to paralelize some intensive computation (not I/O bound). Naturally, I do not want to run more processes than available processors or I would start paying for context switching (a...

I was writing a program that starts two processes. The first process, the "client" sends two type of messages. The first type increases a shared resource (int). The second type sets the resource ...

Am new to multi threaded/processs programming. So here's what I need to clarify. Process A code pthread_mutex_lock() pthread_create(fooAPI(sharedResource)) //fooAPI creates another thread with s...

(Please note: There is a question called "SQLite3 and Multiprocessing" but that question is actually about multithreading and so is the accepted answer, this isn't a duplicate) I'm implementing a ...