Python multiprocessing manager class object thread/process safe
Asked Answered
M

2

8

I have the following class that is shared between multiple consumers (using producer/consumer methodology). My question involves the methods called on this class. Do I need to implement locks or is the manager class thread safe?

import multiprocessing as mp
from multiprocessing.manager import BaseManager

class SampleClass(object):

    def __init__(self):
        self._count = 0

    # Does locking need to be implemented here?
    def increment(self):
        self._count += 1

BaseManager.register('SampleClass', SampleClass)
manager = BaseManager()
manager.start()

instance = manager.SampleClass()

jobs = []
for i in range(0, 5):
    p = mp.Process(target=some_func, args=(instance,))
    jobs.append(p)
    p.start()

for p in jobs:
    p.join()
Miosis answered 26/7, 2017 at 3:10 Comment(0)
P
1

Prior answers are incomplete as they don't address thread safety. Poster asked about threads.
SyncManager section of docs don't mention Queues and threads. Without the code for some_func we don't know whether child processes spawn multiple threads.

It appears that multiprocessing queues are both thread and process safe. According to this section of Python multiprocessing docs:

Queues are thread and process safe.

That's for multiprocessing.Queue not multiprocessing.manager.Queue or SyncManager.Queue. However answers to this question indicate that manager.Queue is just a multiprocessing.Queue run by a manager. So manager.Queue and SyncManager.Queue are thread safe too.

Pyrone answered 19/6 at 10:28 Comment(0)
S
0

I think so. As said as a comment on this other question:

  1. multiprocessing: How do I share a dict among multiple processes?

Is manager.dict() this process safe?

@LorenzoBelli, if you're asking whether access to the manager is synchronized, I believe the answer is yes. multiprocessing.Manager() returns an instance of SyncManager, the name of which suggests as much!

Scrumptious answered 9/9, 2019 at 21:51 Comment(1)
Incomplete answer. Poster asked about thread safety. SyncManager is process safe; docs don't say whether it's also thread safe. Without the implementation of some_func we don't know whether each child process spawns multiple threads to access SampleClass.Pyrone

© 2022 - 2024 — McMap. All rights reserved.