A limit clarification for the new Firestore

Question

So in the limits section (https://firebase.google.com/docs/firestore/quotas) of the new Firestore product from Firebase it says:

Maximum write rate to a collection in which documents contain sequential values in an indexed field: 500 per second

We're pretty confused as to what that actually entails.

If we have, say, a root-level collection called users with 10 million entries in it, will this rate affect this collection in such a way, so only 500 users can update their data in any given second?

Can anyone clarify?

Loam

I recommend you ask this question at groups.google.com/forum/#!forum/firebase-talk and/or reddit.com/r/Firebase instead of StackOverflow, you'll get better insight there. StackOverflow is best for inquiries regarding specific code. Also, Google is planning to lift the limits they have posted once they get more data on Firestore performance in Beta. – Middlebreaker

Answer 1

Sorry for the confusion; an example might help.

If your user documents contained a last-updated timestamp and you index on that timestamp then each new write would end up clustering around the same value (now) creating a hotspot in the index.

Similarly if you somehow assigned users a sequential value like a place in line or something like that this would also create a hotspot.

Incidentally this is why generated document IDs are random strings. This evenly distributes the writes on the primary key index.

If you avoid these kinds of patterns the sky's the limit, though during beta you'd hit the database-wide limit.

Oireachtas

This is a bit blurry, since it is unknown how the indexes are built and hotspots are created. Also, any number field in objects can be considered as “sequential” since you can sort by that field. I believe by sequential you meant the fields of the updated/written objects during some period of time. Imagine I have 10000 objects and they all have a number field which “enumerate” all the objects - from 0 to 9999. Suppose I created those objects in random order so there should not be a hotspot, however if I created all 10K randomly in less than 1 second will that create a hotspot? – Alcoholism

And what about the case when those 10K objects are randomly numbered between 0 - 1000’000’000? Does updating all of them in less than a second will mean touching all index positions which means hotspots? Does this effectively mean that one should not update all the objects in a collection in less than a second if there are more than 500 of them or am I getting wrong the idea behind indexes and hotspots? I believe this is worth explaining in details also in the documentation. – Alcoholism

What happens in the case where you reach that rate limit? Do things just slow down for users or do the requests get rejected? – Outpour

If you're just over the limit, writes will just take a little bit longer. If you go way over the limit, you'll eventually start to see writes fail with contention errors. If you're using our SDKs, they handle this for you: they retry these kinds of errors with exponential backoff. – Oireachtas

I'm guessing the same applies for a collection group scope indexes? – Orang

Correct. It's a write rate limit on an index, and collection group scope indexes aren't any different. – Oireachtas

@GilGilbert does the limit apply to composite indexes? For example, say I have a collection of calendar events (containing all events for all users): I disable single-field indexing on the event start_date, but create a composite index on [user_id, start_date]. Does the rate limit still apply across the entire collection of events or just for a specific user_id? – Jog

This applies to composite indexes too. For any sequentially valued index component the limit will apply to the prefix of that component in the index. In your example that any given user_id would see the limit of 500/sec and the collection as a whole would see a limit of users * 500/sec. – Oireachtas

Would adding an index exemption on all sequential fields for documents in a collection mean the "sky is the limit" for writes to that collection?.. we would not hit the 500 qps cap? – Parve

Yes. Sequentially-valued fields only matter if they’re part of an index. – Oireachtas

@GilGilbert I'm storing scores in a collection. As expected, i'm indexing by score value. Once at the end of the week, I want to create the ranks and store inside the collection as it's not going to change further. If i have 1 million score documents, is it like it takes more than 30minutes to update? (1,000,000/500 = 2000 secs) ? – Beaubeauchamp

Also, it's quite common that every document has a created timestamp or updated timestamp by default. I'm not sure why this limit is not affecting much anyone. – Beaubeauchamp

@Beaubeauchamp re 30 minutes to update, you'll probably see it go faster, especially if score data is not uniform, but yes, for highly clustered values you'll contend on the index and won't be able to write faster than 500/sec. Update times aren't a problem unless you're indexing on them. Even then, getting to 500 writes/second sustained doesn't happen until your app is doing substantial traffic. – Oireachtas

We see its very quick to achieve. Consider case where multiple scores per player is allowed. And its quite common that score will be indexed for sure as we need to order based on it for a leaderboard. Even if not multiple scores, is it like firestore is not a right db for 1Mil userbase? At 1million userbase i will be limited by 500/sec once I need to update status of players or scores or any bulk operation:| – Beaubeauchamp

@GilGilbert : I made an another post to explain the problem in detail. Please have a look - #62360616 – Beaubeauchamp

Answer 2

A quick additional note : for the moment all properties are indexed by default, so if you had a last-updated timestamp it would necessarily be indexed - so you would not be able to avoid the hotspoting.

Index disablement will be available down the road though.

Amaras