Is shelve in Python thread safe?

Question

Is shelve in Python used for data persistence thread safe? If not, what's a good alternative?

Vitiligo

Answer 1

From the standard library documentation about the Shelve module, under the heading Restrictions:

The shelve module does not support concurrent read/write access to shelved objects. (Multiple simultaneous read accesses are safe.)

I would assume that it's probably implementation dependent and in which case, in order to be sure, I would conclude that it certainly is not thread safe.

Ribosome

Thanks for the comment. In the next parts of the documentation, it says: “When a program has a shelf open for writing, no other program should have it open for reading or writing. Unix file locking can be used to solve this, but this differs across Unix versions and requires knowledge about the database implementation used.” - this suggests that the shelve module is not thread safe unless you implement your own locking mechanism. Do you have an example that demonstrates otherwise? – Ribosome

Answer 2

Alternatives: ZODB

http://www.zodb.org/

Fabozzi

(from link) (ZODB...) Transactions provide isolation Transactions allow multiple logical threads (threads or processes) to access databases and the database prevents the threads from making conflicting changes. – Mayest

zodb allows conflicts with concurrent access: zodb.org/en/latest/articles/ZODB2.html#zodb-and-concurrency – Guyette

Answer 3

Threads = # amount of threads

thread_moment = [False for _ in range(Threads)] 

def job(x):  # x would be the index of the thread  

    lock.aquire()

    # open/edit/update/close your shelve file

    thread_moment[x] = True 

    lock.release()

    while True:
        if all(thread_moment) == True:
            thread_moment = [False for _ in range(threads)]
            break
        else:
            time.sleep(1)

    # carry on with your script

Vestige

the question is about whether it's safe.. how does your code answer the question? – Overpowering