Maximum number of fields for a Firestore document?

Question

Right now I have a products collection where I store my products as documents like the following:

documentID:

title: STRING,
price: NUMBER,
images: ARRAY OF OBJECTS,
userImages: ARRAY OF OBJECTS,
thumbnail: STRING,
category: STRING

NOTE: My web app has approximately 1000 products.

I'm thinking about doing full text search on client side, while also saving on database reads, so I'm thinking about duplicating my data on Firestore and save a partial copy of all of my products into a single document to send that to the client so I can implement client full text search with that.

I would create the allProducts collection, with a single document with 1000 fields. Is this possible?

allProducts: collection

Contains a single document with the following fields:

Every field would contain a MAP (object) with product details.

document_1_ID: {  // Same ID as the 'products' collection
  title: STRING,
  price: NUMBER,
  category: STRING,
  thumbnail
},
document_2_ID: {
  title: STRING,
  price: NUMBER,
  category: STRING,
  thumbnail
},
// AND SO ON...

NOTE: I would still keep the products collection intact.

QUESTION

Is it possible to have a single document with 1000 fields? What is the limit?

I'm looking into this, because since I'm performing client full text search, every user will need to have access to my whole database of products. And I don't want every user to read every single document that I have, because I imagine that the costs of that would not scale very well.

NOTE2: I know that the maximum size for a document is 1mb.

Estevan

let me tell you how I solved it, I have 35k docs on a collection, in order to implement text search we moved to algolia, probably you want to check that, right now firebase does not support text search, if you keep in sync firebase with algolia you can delegate the search feature to it... for us it cost around 30usd a month – Avraham

Thanks @Avraham ! That's what Firestore recommends. But for now my project can't afford that. Do you know if it's possible to share the same Algolia account for three different Firebase/Firestore projects? Imagine that I have 3 websites with 12k documents in each. Can I use the same Algolia fee to provide full text search for those 3 websites? Or do I need to set up an account for each? – Estevan

algolia works by index which is equivalent to a collection, they provide you an sdk that you can use, each project can have it owns cloud functions to sync, it is perfectly possible :) – Avraham

Thanks a lot. I'll take another look into that! – Estevan

Answer 1

According to the documentation, there is no stated limit placed on the number of fields in a document. However, a document can only have up to 40,000 index entries, which will grow as documents contain more fields that are indexed by default.

Darwinism

Answer 2

According to this document, in addition to the 1MB limit per document, there is a limit of index entries per document, which is 40,000. Because each field appears in 2 indexes (ascending and descending), the maximum number of fields is 20,000.

I made a Node.js program to test it and I can confirm that I can create 20,000 fields but I cannot create 20,001.

If you try to set more than 20,000 fields, you will get the exception:

INVALID_ARGUMENT: too many index entries for entity

  // Setting 20001 here throws "INVALID_ARGUMENT: too many index entries for entity"
  const indexPage = Array.from(Array(20000).keys()).reduce((acc, cur) => {
    acc[`index-${cur}`] = cur;
    return acc;
  }, {});
  await db.doc(`test/doc`).set(indexPage);

Demandant

Answer 3

I would create the allProducts collection, with a single document with 1000 fields. Is this possible?

There isn't quite a fixed limitation for that. However, the documentation recommends having fewer than 100 fields per document:

Limit the number of fields per document: 100

So the problem isn't the fact that you duplicate data, the problem is that the documents have another limitation that you should care about. So you're also limited to how much data you can put into a document. According to the official documentation regarding usage and limits:

Maximum size for a document: 1 MiB (1,048,576 bytes)

As you can see, you are limited to 1 MiB total of data in a single document. When we are talking about storing text, you can store pretty much but as your documents get bigger, be careful about this limitation.

If you are storing a large amount of data in your documents and those documents should be updated by lots of admins, there is another limitation that you need to take care of. So you are limited to 1 write per second on every document. So if you have a situation in which the admins are trying to write/update products in that same document all at once, you might start to see some of these writes fail. So, be careful about this limitation too.

And the last limitation is for index entries per document. So if you decide to get over the first limitation, please note that the maximum limit is set to 40,000. Because each field has associated two indexes (ascending and descending), the max number of fields is 20,000.

Is it possible to have a single document with 1000 fields?

It is possible up to 40,000 properties but in your case with no benefits. I say that because every time you perform a query (get the document), only a single document will be returned. So there is no way you can implement a search algorithm in a single document and expect to get Product objects in return.

And I don't want every user to read every single document that I have, because I imagine that the costs of that would not scale very well.

Downloading an entire collection to search for fields client-side isn't practical at all and is also very costly. That's the reason why the official documentation recommends a third-party search service like Algolia.

For Android, please see my answer in the following post:

• Is it possible to use Algolia query in FirestoreRecyclerOptions?

Fa

What he actually meant is storing the products as a map with the product's Id as the key and the name of the product as the value. That way he can fetch that single document, perform the search query, collect matched document ids, create DocumentReferences and then use the getall() function without wasting reads and requests on the built in query. – Shrewish

Answer 4

Firebase has a limit of 20k fields per document. https://www.youtube.com/watch?v=o7d5Zeic63s

Gingili