How to use Google Cloud Firestore local emulator for python and for testing purpose

Question

I tried to find out how to use firestore local emulator for python and for testing purpose. But I can not find out how-to document.

Could somebody help me?

Heartfelt

Answer 1

Welcome to SO :)

The primary purpose of the Cloud Firestore Emulator (at the moment) seems to be to test security rules, as documented here. This section states: "The only SDK that currently supports the emulator is the Node.js SDK."

Confusingly there are also these Ruby docs for the Google Cloud Client Libraries. The same thing does not seem to be available in Python yet.

Here are instructions for running the emulator as part of the Google Cloud SDK.

---

Consider using Cloud Firestore in Datastore mode, which has better tooling (it's probably just had more time to mature). You can find instructions for running its emulator on the Running the Datastore mode Emulator page.

Use the Choosing between Native Mode and Datastore Mode page to decide which direction you want to take. If you feel you need the additional 'Native mode' features, it'll probably be easiest to connect straight to a real Firestore instance in the cloud.

Interlaken

"Cloud Firestore in Datastore mode" hardly sounds like Firestore at all - it doesn't provide real-time updates or offline persistence. The only advantage of "datastore mode" over "native mode" seems to be that it can handle more writes per second. – Whitechapel

@Whitechapel Not sure what you mean when you say that Cloud Firestore in Datastore mode "doesn't provide offline persistence". Cloud Firestore in Datastore mode (formerly known as just Cloud Datastore) was around long before Firestore, so you should think of it as a legacy product that's been rolled into Firestore. The advantage is that older applications built to work with Datastore can switch to Cloud Firestore in Datastore mode without modification and take advantage of most of the Firestore benefits (all except the real-time capabilities). – Interlaken

I was referring to the feature comparison chart in the "Choosing between Native Mode and Datastore Mode" page linked above. It says that "Offline data persistence" is "Not supported" in Datastore mode. I believe it's referring to the mobile & web clients' ability to keep allowing local updates when disconnected from the network. – Whitechapel

I think renaming the new version of Datastore to "Cloud Firestore in Datastore mode" was a branding mistake. I've thought of Firestore as fundamentally a real-time database ever since it was announced (it was billed as a more sophisticated alternative to Firebase RTDB, and the Firebase brand has been strongly associated with real-time updates from the start). It took me quite a while to understand that "Datastore mode" does not do real-time updates. – Whitechapel

It's not even clear to me whether there is an emulator for Firestore in Native mode (I think there is, but it somehow only works with the Javascript client library). – Whitechapel

Answer 2

Using the firebase_admin python module, follow the standard setup documented in the Cloud Firestore Docs

This will involve calling initialize_app with a credentials context and then creating a traditional Firestore Client with firestore.client()

For example:

from firebase_admin import credentials, firestore, initialize_app

firebase_credentials_file_path = ...
cred = credentials.Certificate(firebase_credentials_file_path)
initialize_app(cred)
db = firestore.client()

Next, you will need to install and run the Firestore Emulator, which will host the a local Firestore instance over localhost:8080.

npx firebase setup:emulators:firestore
npx firebase --token $FIREBASE_TOKEN emulators:start --only firestore --project $PROJECT_KEY

Finally, inject a redirect in the already instantiated firestore.client instance to interact with the local emulator host/port using an insecure GRPC channel:

import grpc
from google.cloud.firestore_v1.gapic import firestore_client
from google.cloud.firestore_v1.gapic.transports import firestore_grpc_transport

channel = grpc.insecure_channel("localhost:8080")
transport = firestore_grpc_transport.FirestoreGrpcTransport(channel=channel)
db._firestore_api_internal = firestore_client.FirestoreClient(transport=transport)

Now, your db object will interact with the local emulator without any problems.

Acknowledgements to John Carter for figuring this out on the gcloud internal api

Allochthonous

Note: grpc just doesn't work in multiprocessing environments. See github.com/googleapis/google-cloud-python/issues/… – Allochthonous

Hah I just stumbled over this answer :) – Patchy

For anyone looking to get around the grpc mutliprocessing limitations, look into extending the multiprocessing.managers.BaseManager as a facade to a singleton grpc instance that is process safe. – Allochthonous

The firestore_client and firestore_grpc_transport are now in google.cloud.firestore_v1.client (using firebase-admin==5.2.0 and google-cloud-firestore==2.6.0). – Ossicle

Answer 3

As of now, one can connect any SDK of firebase_admin to firebase emulator by simply setting these two environment variables. I have tested this personally on Python SDK and it works like a charm.

export FIRESTORE_EMULATOR_HOST="localhost:8080"
export GCLOUD_PROJECT="any-valid-name"

Documentation Link

Trawl

I just tested this when trying to write a document, without luck. When I set FIRESTORE_EMULATOR_HOST="localhost:8080" the set-promise never resolves resulting in the error "503 failed to connect to all addresses". – Latarsha

Have you started the emulator before running your test? – Trawl

Yes it's running and I've checked that the correct port is set in firebase.json – Latarsha

Any link to a minimal reproducible example? I'd like to get to the bottom of this. – Trawl

No, sorry. For more details look here: #68272614 – Latarsha

Apologies for downvoting. It actually worked for me. Thanks for the simple solution. Unable to upvote now; – Ballarat

Just like Fanchen said, it works like a charm. Just need to also add : os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path_to_my_service_account.json' – Heteropterous

Answer 4

I was running this in a pytest fixture, however you should be able to use it anywhere:

from firebase_admin import credentials, delete_app, firestore, initialize_app
from google.auth.credentials import AnonymousCredentials


class _OfflineGCPCredentials(credentials.Base):
    def get_credential(self):
        return AnonymousCredentials()


app = initialize_app(credential=_OfflineGCPCredentials(), options={'projectId': 'demo-projectname'})

# ... run your code that includes
client = firestore.client()

delete_app(app)

Pantywaist