Maximum size of a document in firestore?

Question

I want create a document that containing about 20 million objects.

The structure like that:

documentID
---- key1
-------- object1
-------------name: "test1"
-------------score: 123

I don't know the limitation of a document size in firestore, so can you help me any reference or information about that? Thanks!

Holocaine

You're mostly looking at Google Cloud Storage rather than Firestore. – Br

Answer 1

The maximum size is roughly 1 Megabyte, storing such a large number of objects (maps) inside of a single document is generally a bad design (and 20 million is beyond the size limit anyway).

You should reconsider why they need to be in a document, rather than each object being their own document.

Cloud Firestore's limits are listed in the documentation.

Consciencestricken

@nard did you find an answer to this? I am basically asking myself the same question. Simple example: Tracking visited urls per user. As each user can have in theory unlimited visited urls it makes no sense to save it within their profile document. So instead I create a "pageview" collection which has a document for each url. The document contains an array with users which have visited this page. This way I can easily query all users which have visited a certain page without having to read potentially thousands of documents for a simple query. – M16

Also best to ask new questions as questions on SO and not in the comment section. – Consciencestricken

@DanMcGrath how do I know if its reaching close to 1megabyte? – Sheffie

@simpleton easiest way is adding up the size of all its fields and values. firebase.google.com/docs/firestore/storage-size#document-size – Toque

Note that writing 20M documents on the Blaze plan will cost $36. If this is a problem, you could consider "batching" your original objects into more than one, but fewer than 20M separate documents. Or avoid Firestore altogether... – Transmittance

Answer 2

Have you looked at Firestore sub-collections?

You can store the main item as a document in one top-level collection, and all of its underlying data could be in a sub-collection of that document.

There is no limit to how many object records a sub-collection can contain when those objects are stored as child documents of that sub-collection.

So 20M records should not be an issue.

Allow

Answer 3

If you want to save objects bigger than 1mb you should use cloud storage, the limit is 5tb per object:

There is a maximum size limit of 5 TB for individual objects stored in Cloud Storage. There is an update limit on each object of once per second, so rapid writes to a single object won't scale. Google cloud storage

Talton

Answer 4

If you want to check the size of a document against the maximum of 1 MiB (1,048,576 bytes), there is a library that can help you with that:

• https://github.com/alexmamo/FirestoreDocument-Android/tree/master/firestore-document

In this way, you'll be able to always stay below the limit.

Landfall

Answer 5

i think you should try using firebase_storage instead and send the data as a json and retrieve later on as needed (downloading it, and adding more content if needed and sending the update to override the current one like that, like that), firestore has a limit of 1 mb which is something like 1 million bytes. this is something like 900k characters(on normal characters anyway, not including other languages like chinese characters and so on ,valid: A upto Z and 1-9 and symbols included) so firebase firestore can not handle 20 million characters it is not possible. but on storage it is very possible cause a json with those characters is something like less than 20 mb.

Parkinson