ZODB In Real Life [closed]

Question

Writing an app in Python, and been playing with various ORM setups and straight SQL. All of which are ugly as sin.

I have been looking at ZODB as an object store, and it looks a promising alternative... would you recommend it? What are your experiences, problems, and criticism, particularly regarding developer's perspectives, scalability, integrity, long-term maintenance and alternatives? Anyone start a project with it and ditch it? Why?

Whilst the ideas behind ZODB, Pypersyst and others are interesting, there seems to be a lack of enthusiasm around for them :(

Furl

-1: Value Judgement: "beginning to hate the RDBMS/Object mismatch" Could you please eliminate the value judgement from the question? Is there a way to focus on the facts and reduce the emphasis on your personal hates? – Bursar

@Bursar - A reluctant edit. Not because you are incorrect, but because the -1 came inline with a request which is just rude. This isn't the bayeux tapestry. – Furl

Answer 1

I've used ZODB for more than ten years now, in Zope and outside. It's great if your data is hierarchical. The largest data store a customer operates has maybe. I don't know, 100GB in it? Something on that order of magnitude anyway.

Here is a performance comparison against Postgres.

If you're writing a WSGI web app, these packages may be useful:

• repoze.tm2 (docs)

• repoze.zodbconn (docs)

Pensionary

Accepted. Not just because of the answer, but I keep coming across your posts/articles/presentations when googling ZODB stuff I need. I know who to direct further questions at! ;) – Furl

This docs.zope.org/zope3/Code/BTrees/index.html is also useful – Geraint

Really interesting link about the performance there. Worth a read! – Genetics

"performance comparison against Postgres" the links expired, you can see cached page version at web.archive.org/web/20090218122454/http://… – Tarrance

Answer 2

Compared to "any key-value store", the key features for ZODB would be automatic integration of attribute changes with real ACID transactions, and clean, "arbitrary" references to other persistent objects.

The ZODB is bigger than just the FileStorage used by default in Zope:

• The RelStorage backend lets you put your data in an RDBMS which can be backed up, replicated, etc. using standard tools.
• ZEO allows easy scaling of appservers and off-line jobs.
• The two-phase commit support allows coordinating transactions among multiple databases, including RDBMSes (assuming that they provide a TPC-aware layer).
• Easy hierarchy based on object attributes or containment: you don't need to write recursive self-joins to emulate it.
• Filesystem-based BLOB support makes serving large files trivial to implement.

Overall, I'm very happy using ZODB for nearly any problem where the shape of the data is not obviously "square".

Royall

Answer 3

I would recommend it.

I really don't have any criticisms. If it's an object store your looking for, this is the one to use. I've stored 2.5 million objects in it before and didn't feel a pinch.

Finish

Answer 4

ZODB has been used for plenty of large databases

Most ZODB usage is/was probably Zope users who migrated away if they migrate away from Zope

Performance is not so good as relatonal database+ORM especially if you have lots of writes.

Long term maintenance is not so bad, you want to pack the database from time to time, but that can be done live.

You have to use ZEO if you are going to use more than one process with your ZODB which is quite a lot slower than using ZODB directly

I have no idea how ZODB performs on flash disks.

Rencontre

Thanks for the answer! I am wondering if ZODB will fit in place of MySQL using ZEO in a webfarm setup. Serious caching will hopefully negate the slow performance of the store. – Furl

ZODB usually smokes relational databases for repeated queries when the data doesn't change, even with ZEO in the mix, because there's already a client-side cache. – Pensionary

Answer 5

With pickling you should be able to use any key value database in a similar fashion.

Quite

Unless you pickle both 'a' and 'b' which each reference 'c' surely? – Furl

@Aiden, how does zodb handle that case? – Quite

ZODB handles it transparently. See faassen.n--tree.net/blog/view/weblog/2008/06/20/0 – Pensionary