zc.lockfile.LockError in ZODB

Question

I am trying to use ZODB 3.10.2 on my web server which is running Debian and Python 2.7.1. It seems like every time I try to access the same database from 2 different processes, I get a mysterious exception. I tried accessing a database from an interactive Python session and everything seemed to work fine:

>>> import ZODB
>>> from ZODB.FileStorage import FileStorage
>>> storage = FileStorage("test.db")
>>> 

But then I tried the same series of commands from another session running at the same time and it didn't seem to work:

>>> import ZODB
>>> from ZODB.FileStorage import FileStorage
>>> storage = FileStorage("test.db")
    No handlers could be found for logger "zc.lockfile"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/ZODB3-3.10.2-py2.7-linux-x86_64.egg/ZODB/FileStorage/FileStorage.py", line 125, in __init__
    self._lock_file = LockFile(file_name + '.lock')
  File "/usr/local/lib/python2.7/site-packages/zc.lockfile-1.0.0-py2.7.egg/zc/lockfile/__init__.py", line 76, in __init__
    _lock_file(fp)
  File "/usr/local/lib/python2.7/site-packages/zc.lockfile-1.0.0-py2.7.egg/zc/lockfile/__init__.py", line 59, in _lock_file
    raise LockError("Couldn't lock %r" % file.name)
zc.lockfile.LockError: Couldn't lock 'test.db.lock'
>>>

Why is this happening? What can be done about it?

Anorthic

Thanks for the answers. I didn't know that ZODB didn't support multi-process access. I was looking for something like shelve, but with better concurrency support. I want to be able to reliably write to the database from one process while reading from it from another. I may try ZEO, or I may look elsewhere. – Anorthic

This looks like a misunderstanding. ZODB supports multi-process access just fine. Just the file storage mode doesn't. Clearly with an RDBMS you are going to have multi-process access when hit from multiple ZODB sources. Same when using it with ZEO. – Fincher

Answer 1

The ZODB does not support multi-process access. This is why you get the lock error; the ZODB file storage has been locked by one process to prevent other processes altering it.

There are several ways around this. The easiest option is to use ZEO. ZEO extends the ZODB machinery to provide access to objects over a network, and you can easily configure your ZODB to access a ZEO server instead of a local FileStorage file:

<zodb>
    <zeoclient>
    server localhost:9100
    </zeoclient>
</zodb>

Another option is to use RelStorage, which stores the ZODB data in a relational database. RelStorage supports PostgreSQL, Oracle and MySQL backends. RelStorage takes care of concurrent access from different ZODB clients. Here is an example configuration:

<zodb>
  <relstorage>
    <postgresql>
      # The dsn is optional, as are each of the parameters in the dsn.
      dsn dbname='zodb' user='username' host='localhost' password='pass'
    </postgresql>
  </relstorage>
</zodb>

RelStorage requires more up-front setup work but can outperform ZEO in many scenarios.

Introspect

Answer 2

You can not access the same database files from two processes at the same time (which is obvious). That's why you get this error. If you need to perform actions on the same data.fs file from two or more processes: use ZEO.

Pule

> (which is obvious) sqlite supports this (albeit with a massive lock) – Boggs

Answer 3

## Let the program run once (if it's already running, don't run it again)
## Run the program, open the form
import sys
import zc.lockfile
try:
    lock = zc.lockfile.LockFile('lock', content_template='{pid};{hostname}')
    if __name__ == '__main__':
        mainForm()
except zc.lockfile.LockError:
    sys.exit()

## zc.lockfile thanks
## https://pypi.org/project/zc.lockfile/#detailed-documentation

Monogyny