Firestore client in python (as user) using firebase_admin or google.cloud.firestore

Question

I am building a python client-side application that uses Firestore. I have successfully used Google Identity Platform to sign up and sign in to the Firebase project, and created a working Firestore client using google.cloud.firestore.Client which is authenticated as a user:

import json
import requests
import google.oauth2.credentials
from google.cloud import firestore

request_url = f"https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key={self.__api_key}"
headers = {"Content-Type": "application/json; charset=UTF-8"}
data = json.dumps({"email": self.__email, "password": self.__password, "returnSecureToken": True})
response = requests.post(request_url, headers=headers, data=data)
try:
    response.raise_for_status()
except (HTTPError, Exception):
    content = response.json()
    error = f"error: {content['error']['message']}"
    raise AuthError(error)

json_response = response.json()
self.__token = json_response["idToken"]
self.__refresh_token = json_response["refreshToken"]

credentials = google.oauth2.credentials.Credentials(self.__token,
                                                    self.__refresh_token,
                                                    client_id="",
                                                    client_secret="",
                                                    token_uri=f"https://securetoken.googleapis.com/v1/token?key={self.__api_key}"
                                                    )

self.__db = firestore.Client(self.__project_id, credentials)

I have the problem, however, that when the token has expired, I get the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.UNAUTHENTICATED
    details = "Missing or invalid authentication."
    debug_error_string = "{"created":"@1613043524.699081937","description":"Error received from peer ipv4:172.217.16.74:443","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Missing or invalid authentication.","grpc_status":16}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/my_app/src/controllers/im_alive.py", line 20, in run
    self.__device_api.set_last_updated(utils.device_id())
  File "/home/my_app/src/api/firestore/firestore_device_api.py", line 21, in set_last_updated
    "lastUpdatedTime": self.__firestore.SERVER_TIMESTAMP
  File "/home/my_app/src/api/firestore/firestore.py", line 100, in update
    ref.update(data)
  File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/document.py", line 382, in update
    write_results = batch.commit()
  File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/batch.py", line 147, in commit
    metadata=self._client._rpc_metadata,
  File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/gapic/firestore_client.py", line 1121, in commit
    request, retry=retry, timeout=timeout, metadata=metadata
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
    on_error=on_error,
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.Unauthenticated: 401 Missing or invalid authentication.

I have tried omitting the token and only specifying the refresh token, and then calling credentials.refresh(), but the expires_in in the response from the https://securetoken.googleapis.com/v1/token endpoint is a string instead of a number (docs here), which makes _parse_expiry(response_data) in google.oauth2._client.py:257 raise an exception.

Is there any way to use the firestore.Client from either google.cloud or firebase_admin and have it automatically handle refreshing tokens, or do I need to switch to the manually calling the Firestore RPC API and refreshing tokens at the correct time?

Note: There are no users interacting with the python app, so the solution must not require user interaction.

Cambist

I'm still learning about some stuff on Google Cloud but in imho if you expect to work without user interaction then you should use a service account as client authentication. Those are treated differently i believe, and should have some tweak about the token expiration handled by the IAM – Misfile

If there is a way for a service account to be treated exactly like a normal Firebase user, then this is definitely a possiblity. However, I have not seen anything like that – Cambist

That sounds super intersting to me as well! I see you used some class for it. Do you think you could wrap this in a pipy package something like firestore-cloud-client ? and do something smiliar as in the older firestore python API? Would be really helpful! – Bryon

The class is mainly there for other reasons, but there isn't much to it. The only addition to the above code to make it work is: from google.oauth2 import _client _client._parse_expiry = parse_expiry Where parse_expiry is a function that just casts expires_in from the response to an int before calculating the datetime: expires_in = response_data.get("expires_in", None) return datetime.datetime.utcnow() + datetime.timedelta(seconds=int(expires_in)) if expires_in else None – Cambist

Answer 1

Can't you just pass the string cast as integer _parse_expiry(int(float(response_data))) ?

If it is not working you could try to make a call and refresh token after getting and error 401, see my answer for the general idea on how to handle tokens.

Wasson

That is actually my temporary hack. Because _parse_expiry is inside the Google Oauth2 library, I have to overwrite the function as follows: from google.oauth2 import _client _client._parse_expiry = my_parse_expiry. Can you explain why you are casting the value to a float first? – Cambist

I was not sure what would be the response.data. if there is no decimal you can just ignore it and only cast to int. – Wasson

Answer 2

As mentioned by @Marco, it is recommended that you use a service account if it's going to be used in an environment without user. When you use service account, you can just set GOOGLE_APPLICATION_CREDENTIALS environment variable to location of service account json file and just instantiate the firestore Client without any credentials (The credentials will be picked up automatically):

import firestore
client = firestore.Client()

and run it as (assuming Linux):

$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
$ python file.py

Still, if you really want to use user credentials for the script, you can install the Google Cloud SDK, then:

$ gcloud auth application-default login

This will open browser and for you to select account and login. After logging in, it creates a "virtual" service account file corresponding to your user account (that will also be loaded automatically by clients). Here too, you don't need to pass any parameters to your client.

See also: Difference between “gcloud auth application-default login” and “gcloud auth login”

Salomie

Is it possible to configure the service account to be treated as a regular Firebase user? I am not looking for a server-side solution, so it has to be possible to get a Firebase user ID, and the reads/writes from/to Firebase and Firestore must be checked against the security rules – Cambist

What if I need to deploy this in a container and can't deploy a creds.json file into the container. How would I create the Client object? – Illimitable

Load in the creds from another source such as Google Secret Manager – Teasley