Search by pattern on Cloud Firestore collection

Question

I'm trying to perform a filter by pattern over a Firestore collection. For exemple, in my Firestore database I have a brand called adidas. The user would have an search input, where typing "adi", "adid", "adida" or "adidas" returns the adidas document. I pointed out several solutions to do this :

1. Get all documents and perform a front-end filter

var brands = db.collection("brands");
filteredBrands = brands.filter((br) => br.name.includes("pattern"));

This solution is obviously not an option due to the Firestore pricing. Moreover it could be quite long to perform the request if the number of documents is high.

2. Use of Elasticsearch or Algolia

This could be interesting. However I think this is a bit overkill to add these solutions' support for only a pattern search, and also this can quickly become expensive.

3. Custom searchName field at object creation

So I had this solution : at document creation, create a field with an array of possible search patterns:

{
    ...
    "name":"adidas",
    "searchNames":[
        "adi",
        "adida",
        "adidas"
    ],
    ...
}

so that the document could be accessed with :

filteredBrands = db.collection("brands").where("searchNames", "array-contains", "pattern");

So I had several questions:

• What do you think about the pertinence and the efficiency of this 3rd solution? How far do you think this could be better than using a third party solution as Elasticsearch or Algolia?
• Do you have any other idea for performing pattern filter over a firestore collection?

Enucleate

Whether storing search terms is better is subjective. But it can definitely work. In fact, one of our engineers recently created a search storing trigrams in Firestore. Btw: your search terms can actually also be found with the build in search operators of Firestore: db.collection("brands").where("name", ">=", "adi").where("name", "<=", "adidas") – Bogosian

@FrankvanPuffelen Hi Puf! Where can we find this search storing trigrams? – Benzofuran

Went down this exact same path, skipped 1 & 3 (because we need total control and it has to scale), tried both Algolia and ElasticSearch. Ultimately, ran into issues w/ Algolia on SSR, then gave the ElasticSearch approach a shot - and it's gone really well so far. I'm not sure on the costs yet as we are still doing low volume in development, but I did a write up if you're interested in the Algolia approach: link.medium.com/NNg0I3Usi1 – Joacima

Answer 1

IMHO, the first solution is definitely not an option. Downloading an entire collection to search for fields client-side isn't practical at all and is also very costly.

The second option is the best option considering the fact that will help you enable full-text search in your entire Cloud Firestore database. It's up to you to decide if it is worth using it or not.

What do you think about the pertinence and the efficiency of this 3rd solution?

Regarding the third solution, it might work but it implies that you create an array of possible search patterns even if the brand name is very long. As I see in your schema, you are adding the possible search patterns starting from the 3rd letter, which means that if someone is searching for ad, no result will be found. The downside of this solution is the fact that if you have a brand named Asics Tiger and the user is searching for Tig or Tige, you'll end up having again no results.

Do you have any other ideas for performing pattern filters over a Firestore collection?

If you are interested to get results only from a single word and using as a pattern the staring letters of the brand, I recommend you a better solution which is using a query that looks like this:

var brands = db.collection("brands");
brands.orderBy("name").startAt(searchName).endAt(searchName + "\uf8ff")

In this case, a search like a or ad will work perfectly fine. Besides that, there will be no need to create any other arrays. So there will be less document writing.

I have also written an article called:

• How to filter Firestore data cheaper?

That might also help.

Benzofuran

Thanks for your answer. I think search by starting letters is enough for my project, that's why I though about this alternative. What does + "\uf8ff" do? – Enucleate

You're welcome! The character \uf8ff used in the query is a very high code point in the Unicode range (it is a Private Usage Area [PUA] code). Because it is after most regular characters in Unicode, the query matches all values that start with searchName. – Benzofuran

Is there anyway to make it case-insensitive? – Afresh

@Afresh Usually we store both, like explained in my answer from this post. – Benzofuran

@MadMac Good to hear that ;) – Benzofuran

@Alex Mamo Great solution. But may I know what if I want to have multiple orderBy and the first orderBy is a date? – Acquaint

@Acquaint It's possible but it will require an index. – Benzofuran