Correctly import data from FireStore to Algolia

Question

I am trying to import existing data from firestore to Algolia but can not make it work. I installed the *Firebase Algolia Extension` and tried to follow the documented steps:

1. running this:

npx firestore-algolia-search

Below are the questions that will be asked:

What is the Region? europe-west3

What is the Project Id? wishlist-88d58

What is the Algolia App Id? 15W2O5H8ZN

What is the Algolia Api Key? { unspecified parameter }

What is the Algolia Index Name? allUsers

What is the Collection Path? allUsers What are the Fields to extract? { unspecified parameter }

What is the Transform Function? { unspecified parameter }

What is the path to the Google Application Credential File? wishlists_key.json

For the Algolia API Key I added a key:

I did not specify Fields to extracts and Transform Functions.

For path to the Google Application Credential File I created a private key in Firebase and located it on the my desktop as wishlists_key.json, which is where I ran the command above from.

I got a response which also contained the data but said at the beginning there was an error:

{"severity":"WARNING","message":"Warning, FIREBASE_CONFIG and GCLOUD_PROJECT environment variables are missing. Initializing firebase-admin will fail"} {"location":"europa-west3","algoliaAppId":"15W205H8ZN","algoliaAPIKey":"********","algoliaIndexName":"allUsers","collectionPath":"allUsers","fields":"","transformFunction":"","projectId":"wishlist-88d58","severity":"INFO","message":"Initializing extension with configuration"} {"severity":"INFO","message":"[ 'Sending rest of the Records to Algolia' ]"} {"severity":"INFO","message":"[ 'Preparing to send 20 record(s) to Algolia.' ]"} {"name":"RetryError","message":"Error when performing Algolia index","transporterStackTrace":[{"request":{"data":"{"requests":[{"action":"partialUpdateObject","body":{"signInMethod":"mail","username":"user662572"

...

The command does not finish running but get's stuck in this.

What am I doing wrong here? How do I correctly import data from FireStore to Algolia?

Also, later I will need to import a collection with about 24k documents. Is the documented way also capable of handling these amount of documents?

Oshea

Can you have a look at this thread1 & thread2 – Slaughter

@RoopaM I tried running it on the Cloud Shell but same error. I don't know how to properly setup up the variables or run it correctly. Everytime the same error. Did you make it work? – Oshea

I tried on my linux terminal, Initially I also faced same issue as I was using default Algolia API key. Then I created new API key with the same ACLs you are using, after that I can able to import firestore collection to Algolia – Slaughter

Answer 1

I was able to make it work through Google Cloud Shell. I had to import my Google Application Credential File to it, ran the command above again.

I still got the same Warning that it will fail, but it worked anyway and all data was correctly imported.

Oshea

Answer 2

Alternatively to running the backfill script, you can write an admin script to update each document and the extension's code will sync each document to Algolia

Emit