python-multiprocessing Questions
3
Solved
I'm taking my first foray into the python mutliprocessing module and I'm running into some problems. I'm very familiar with the threading module but I need to make sure the processes I'm executing ...
Merissa asked 8/6, 2016 at 2:22
0
I want to run julia functions/scripts from within python. I managed to call julia scripts via the library juliacall. Now I want to parallelize this. Therefore I created a python multiprocessing Poo...
Oria asked 14/2, 2023 at 17:18
1
Solved
Problem Statement
After booting the GUnicorn worker processes, I want the worker processes still be able to receive data from another process. Currently, I'm trying to use multiprocessing.Queue to ...
Charlet asked 31/1, 2023 at 7:1
1
I want to test a thread safety of a function using py.test
My efforts:
def function_to_be_tested(arg1,arg2):
some functionality
class Test:
def setup()
def teardown()
def test_conc()
p1=Pro...
Nimble asked 14/4, 2017 at 9:3
4
From what I understand, the GIL makes it impossible to have threads that harness a core each individually.
This is a basic question, but, what is then the point of the threading library? It seems...
Kenosis asked 25/9, 2018 at 22:46
2
System information
python 3.8.7
OS 11.1 (Big Sur)
Python installed via brew install [email protected]
To reproduce on Big Sur and most probably older versions:
import multiprocessing as mp
...
Ticket asked 7/1, 2021 at 9:13
2
Solved
Say I have the below code, a function that does something, which is initiated in a Process, and returns a value.
from multiprocessing import Process
def my_func(arg):
return 'Hello, ' + arg
p1 =...
Godber asked 10/2, 2019 at 10:33
3
Solved
I am attempting to create a program in python that runs multiple instances (15) of a function simultaneously over different processors. I have been researching this, and have the below program set ...
Sluff asked 27/3, 2014 at 22:2
7
Solved
Background
I have a very small network which I want to test with different random seeds.
The network barely uses 1% of my GPUs compute power so i could in theory run 50 processes at once to try man...
Jibheaded asked 14/11, 2020 at 18:42
2
I'm trying to run some multiprocessing in a Jupyter notebook, using python version 3.7.0.
However, even a really simple example seems to hang indefinitely.
After reading this answer I tried explici...
Suiter asked 1/3, 2021 at 10:51
4
Solved
I have not been able to implement the suggestion here: Applying two functions to two lists simultaneously.
I guess it is because the module is imported by another module and thus my Windows spawns...
Thorstein asked 4/3, 2017 at 23:29
1
Solved
I have the following code, all pretty simple:
from multiprocessing import Process, Lock
from multiprocessing.managers import BaseManager
class DatabaseConnection:
def __init__(self, conn_id):
s...
Cory asked 16/9, 2022 at 6:38
4
Solved
I have been told that logging can not be used in Multiprocessing. You have to do the concurrency control in case multiprocessing messes the log.
But I did some test, it seems like there is no prob...
Baobaobab asked 25/12, 2017 at 12:55
2
On a Windows 10 system running Python 3.6, when trying to use multiprocessing.Process to create a new rq worker,
multiprocessing.Process(target=Worker(qs).work, kwargs={'burst': True}).start()
...
Prehistory asked 14/9, 2018 at 17:3
1
Solved
I have created a nested dictionary inside a multiprocessing.Manager.dict object. Dictionary methods like update, clear, etc., does not work when I apply them to the nested dictionary. This is an ex...
Actable asked 18/8, 2022 at 20:37
1
I'm new to multi-processing and I am trying to write a program that gets the top 10 results for a search query on google. In this example, I just want to run 2 search queries simultaneously. Here i...
Sandbag asked 6/4, 2022 at 0:1
1
Solved
I want to write a service that launches multiple workers that work infinitely and then quit when main process is Ctrl+C'd. However, I do not understand how to handle Ctrl+C correctly.
I have a foll...
Weisburgh asked 13/7, 2022 at 14:13
2
Solved
An Exception is raised in threading._wait_for_tstate_lock when I transfere hugh data between a Process and a Thread via multiprocessing.Queue.
My minimal working example looks a bit complex first -...
Opiate asked 27/5, 2019 at 7:44
2
Solved
I am using a multiprocessing.Pool which calls a function in 1 or more subprocesses to produce a large chunk of data.
The worker process creates a multiprocessing.shared_memory.SharedMemory object ...
J asked 6/7, 2020 at 3:3
4
Solved
import subprocess
def my_function(x):
return x + 100
output = subprocess.Popen(my_function, 1) #I would like to pass the function object and its arguments
print output
#desired output: 101
I ...
Lipcombe asked 12/1, 2010 at 3:49
4
Solved
Sometimes I need to use multiprocessing with functions with no arguments. I wish I could do something like:
from multiprocessing import Pool
def f(): # no argument
return 1
# TypeError: f() tak...
Roark asked 29/12, 2014 at 13:28
2
Solved
So I'm trying to use multiprocessing Manager on a dict of dicts, this was my initial try:
from multiprocessing import Process, Manager
def task(stat):
test['z'] += 1
test['y']['Y0'] += 5
if _...
Trichloride asked 18/6, 2019 at 3:35
2
Solved
I am learning python multiprocessing, and I am trying to use this feature to populate a list with all the files present in an os. However, the code that I wrote is executing sequentially only.
#!/...
Gig asked 8/10, 2015 at 10:5
10
Solved
I am sorry that I can't reproduce the error with a simpler example, and my code is too complicated to post. If I run the program in IPython shell instead of the regular Python, things work out well...
Ad asked 10/1, 2012 at 14:28
1
I have some questions about using the torch.multiprocessing module. Let’s say I have a torch.nn.Module called model and I call model.share_memory() on it.
What happens if two threads call the forwa...
Amerce asked 3/12, 2020 at 18:50
© 2022 - 2024 — McMap. All rights reserved.