How to properly handle denormalized data synchronization in Doctrine MongoDB ODM
Asked Answered
C

0

25

Denormalization of referenced data seems a pretty common practice when using MongoDB. Yet, I do not see any built-in way to handle that with Doctrine MongoDB ODM.

Let's say I have a social network where users can follow each other, here are two example users :

{
  _id : id1,
  name: "Malcolm Reynolds",
  followed: []
}

{
  _id : id2,
  name: "Zoe Alleyne",
  followed: [
    { _id: id1, name: "Malcolm Reynolds" }
  ]
}

As you can see I want the 'name' property to be denormalized. As I said there seems to be no built-in way in Doctrine ODM to do that. Since the latest issue on the subject is a year old, I would try to do it myself.

Whereas I found plenty of articles on the internet explaining in what case denormalization is useful and mentioning how it can be a pain to keep denormalized data consistent, I didn't find one explaining how implement the dernomalized data update process.

In my case eventually consistent data is enough, there can be few hours between the update of the user name and the update on the denormalized data. I can see 3 different ways to do it :

1 - Consistency checker: Have a task running in background that regularly updates denormalized data.

2 - Update trigger: Each update on a name field updates all of the associated denormalized data in a single flush.

3 - Hybrid solution For each user, when the name is updated, an entry is added to a queue with the updating info (the user update and the insert in the queue would be made in a single flush), and have a task running in background to do the actual updates.

The first solution seems to be the easiest to implement, but as I see it it may be very resource consuming. The second solution would make the update requests ridiculously long, even with a high read/write ratio I this might be a problem. I think the third solution is the way to go, am I right to think so ?

Also I would like to do it in a DRY way, e.g. I would like not to have to rewrite the same code in a preUpdate callback for each Document whose data is denormalized. I am thinking about making a custom annotation, is it a good idea ?

Chlorophyll answered 15/7, 2015 at 13:3 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.