Python Conditional "With" Lock Design

Question

I am trying to do some shared locking using with statements

def someMethod(self, hasLock = False):
     with self.my_lock:
         self.somethingElse(hasLock=True)


def somethingElse(self, hasLock = False):
    #I want this to be conditional...
    with self.my_lock:
          print 'i hate hello worlds"

That make sense? I basically only want to do the with if I don't already have the lock.

On top of being able to accomplish this, is it a bad design? Should I just acquire/release myself?

Saenz

Not any more, i am going to use RLock – Saenz

Answer 1

Just use a threading.RLock which is re-entrant meaning it can be acquired multiple times by the same thread.

http://docs.python.org/library/threading.html#rlock-objects

For clarity, the RLock is used in the with statements, just like in your sample code:

lock = threading.RLock()

def func1():
    with lock:
        func2()

def func2():
    with lock: # this does not block even though the lock is acquired already
        print 'hello world'

As far as whether or not this is bad design, we'd need more context. Why both of the functions need to acquire the lock? When is func2 called by something other than func1?

Seedtime

@Saenz “All of the objects provided by this module that have acquire() and release() methods can be used as context managers for a with statement.” – Misread

Keep in mind that you will have to keep track of how many times lock has been acquired as release will not fully unlock the lock until the count reaches 0. – Proudman

Answer 2

The Python or is short circuiting so you can make the locking conditional:

def somethingElse(self, hasLock = False):
    #I want this to be conditional...
    with hasLock or self.my_lock:
          print 'i hate hello worlds'

Unfortunately it's not quite that easy, because a boolean isn't a valid return from a with statement. You'll need to create a class with the __enter__ and __exit__ to wrap the boolean True value.

Here's one possible implementation that I haven't tested.

from contextlib import contextmanager

@contextmanager
def withTrue():
    yield True

def withbool(condition):
    if condition:
        return withTrue()
    return False

def somethingElse(self, hasLock = False):
    with withbool(hasLock) or self.my_lock():
          print 'i hate hello worlds'

This is a lot of boilerplate for something so simple, so the RLock solution looks like a winner. This solution might be useful in a different context though.

Harriott

This would be a great time to use contextmanager. – Misread

I think this would work, but RLock does what i need out of the box. Thanks. – Saenz

@jleedev, thanks, I forgot about contextmanager. Unfortunately I don't think it would work in this context since you only want to wrap the True value, not the False. – Harriott

Better use docs.python.org/3/library/… – Proudman

Answer 3

Why not:

def someMethod(self):
     with self.my_lock:
         self.somethingNoLock()

def somethingElse(self):
    with self.my_lock:
         self.somethingNoLock()

def somethingNoLock(self):
    print 'i hate hello worlds"

Note that while someMethod and somethingElse are identical in my solution, in general they would be different. You could put another wrapper around somethingNoLock so that the lock acquisition and release is not repeated multiple times.

This is far simpler and straightforward. Just because the re-entrant lock hammer is available, I wouldn't recommend using it when there is a more straightforward, less fragile way to nail it.

The more specific criticism of rlock is that the line that creates the re-entrant lock is far away from the code that is acquiring the lock in a re-entrant way. This is slightly fragile if someone say coalesces the re-entrant lock with another lock that is not re-entrant or otherwise changes the line that creates the lock.

Halfhardy

Answer 4

Using with statement is better than just acquire() and release() functions. This way, if an error occurs, the locks will be released.

Cirrus

Can you do with conditionally? – Saenz

You can't do conditional statements, but some objects support assignment via with statement. For example with open("x.txt") as f: print f.read() – Cirrus

Answer 5

The with statement is a great way to implement locking, as locking is a perfect resource acquisition pattern. Though your current example won't work. You'll need an if statement around the with statement within somethingElse().

Internationale

Answer 6

Since this is the first result in Google about "using threading.Lock() with with / context_manager" here is the answer: yes, you can use both Lock, RLock, or any other object in python threading lib that supports acquire()/release() as with threading.smth():.

All of the objects provided by threading that have acquire and release methods can be used as context managers for a with statement.

Blurt