MongoDB / Mongoose Schema for Threaded Messages (Efficiently)

Question

I'm somewhat new to noSQL databases (I'm fairly good with relational databases though), and I'm wondering what the most efficient way to handle an inbox system with threaded messages would be.

Each 'message' will have a single sender and recipient. The number of received / sent messages will vary widely between users. This system should scale well to over 1k+ users.

I've read up on fan out on write / read but I'm not sure how well this would work for threaded messages.

Since I'm new to MongoDB / NoSQL in general, I'm not really used to structuring data efficiently this way.

I'm guessing there's going to be nested objects in any sort of efficient way of handling this...but I can't settle on a design that seems both efficient and convenient for threaded conversations between 2 users.

I thought of storing data with an array of the 2 users, combined with an array of 'message' objects. But then there's the issue of the order of the 2 user's usernames. (ex. [UserA, UserB] and [UserB, UserA] are both possible and would be problematic, so that seemed like a bad idea).

I thought of doing the whole fan out on read / write thing, but that doesn't seem efficient for threaded messages (since if grabbing messages by recipient is convenient, grabbing messages by sender won't be and vice versa).

I'm leaning towards favoring grabbing messages by recipient (since the inbox loads multiple messages, and sending only involves one [albeit with a longer look-up time]). But I'd really like to grab a threaded conversation in one go, as well as the list of users that a user has threaded conversations with (for the list of threads).

If someone could give me an efficient schema for threaded conversations I'd be very grateful. I've been researching this and trying to settle on a design for hours, and I'm exhausted. I keep finding flaws in my designs and scrapping them and I'd really just like some input from someone more experienced with NoSQL databases / MongoDB so I can avoid making a huge design flaw and/or writing logic that could've been handled with a better database design.

Thanks in advance for any and all help.

Multiracial

Answer 1

On this particular topic you are in luck, there is a great post discussing the various approaches to the schema here (it's a slight twist on what you are looking at, but not much different):

http://blog.mongodb.org/post/65612078649/schema-design-for-social-inboxes-in-mongodb

Then, this topic was also covered in detail at MongoDB World 2014 in three parts by Darren Wood and Asya Kamsky:

Part 1 Outline and Video

Part 2 Outline and Video

Part 3 Outline and Video

Also at MongoDB World the guys at Dropbox talked about the lessons they learned when building their Mailbox:

http://www.mongodb.com/presentations/mongodb-mailbox

And then, to round it off, there is a full reference architecture with code called Socialite on Github written by the aforementioned Darren Wood:

https://github.com/10gen-labs/socialite

Voltaic

Thanks for all the helpful links. I came across the first and last ones during my research. I guess what my real question is, for efficiencies sake, where should I split the database schema and the logic for it. Should I be storing it in a way that allows me to grab it from the database already sorted in a threaded-conversation way....or should I just have a message schema and sort it with server-side logic? – Multiracial