Does Firestore in "Datastore mode" have any advantages "Native mode"

Question

Google Cloud Firestore is going to replace the legacy Google Cloud Datastore soon. One then has the choice between using Cloud Firestore in "native mode" or in "datastore mode". The former allows access to Firestore through the usual Firestore SDK while the latter allows usage of the old Cloud Datastore SDK (which has no Web/Mobile APIs).

I am not yet familiar with Firestore. My question is: Apart from porting things to a new API, are there actually any things that can not be done with Firestore in "native mode" which could be done with the old Cloud Datastore (or its replacement: Firestore in "datastore mode")? Or any other advantage of using "datastore mode" (like costs, for example)?

If not, then it seems there is actually no advantage of using Firestore in "datastore mode" other than compatibility for older code using the old Cloud Datastore.

Am I right in my assumption that Firestore "datastore mode" has absolutely no advantage besides being able to use the legacy Datastore API (at the cost of not being able to use the newer and probably more feature-rich Firestore APIs, including mobile and web APIs)?

Giacomo

Answer 1

Cost.

Firestore in Datastore mode supports key-only and projection queries just like the original datastore. That means that the result set of those queries counts towards "Cloud Firestore Small Operations" which are free. We accumulate billions of those small operations per day and not having projection queries would effectively 10-fold our datastore cost which would be unbearable.

As this feature is not available in Firestore native mode - and also with the strong consistency added in - we expected it to be less performant then in the original datastore, but in our tests this was not the case. Firestore in datastore mode consistently performed about twice as fast for our application across all types of operations.

Sieracki

For those services which do not rely on key-only queries... cost should relatively be similar. However, I want to emphasize that 10,000 write tps limit per database in 'Firestore mode' can be a problem for those systems which requires more than 10,000 inserts into DB every second. – Frodin

@Sieracki Can you please list in detail what operations are considered small and for free in Firestore with Datastore mode? Also Firestore in Datastore mode is best to choose when you don't want any mobile sdk's? – Denominational

Choice points are covered in detail here: cloud.google.com/datastore/docs/firestore-or-datastore – Cheju

@Sieracki is Datastore mode going to stay in the market or its not getting any attention as native mode? I see datastore mode as more scalable considering only server side usage. – Denominational

Answer 2

There are advantages of using "datastore mode", even for a new project.

I am evaluating the two modes of firestore "Datastore mode" and "Native mode" for a migration project from MySQL.

On one hand, I consider using the "Datastore mode" for one back-end global repository because:

1. Server side only
2. Strong performance expectation on search queries across all entities
3. Query and sort on several properties altogether
4. Structured data model with one root kind and few second level kinds
5. A lot of write with limited transaction requirement, a huge number of read with projection within an entity group

On the other hand, the "Native mode" seems to fit some requirements for a user facing specific application because:

6. Web, iOS, Android, API interface with bi-directional sync
7. Several occasionally connected use cases
8. Few large polymorphic objects to sync and to persist
9. Mostly simple query on one properties (parent object)

Though, there is one reason which advocate for Datastore mode for the second project

10. Multi-tenancy with namespace

There are also common needs fulfilled by both mode, which support the decision to migration to NoSQL technologies

11. Scalability
12. No admin
13. Availability
14. Speed of development

The items 2 to 5 and 10 are based on features specific for the Datastore Mode, not possible in Native Mode. The items 6 to 9 are specific to Native Mode.

Update : March, 21st 2019

Six months after the evaluation described in my first answer, my team is using both Firestore (native) mode and Datastore mode.

• 2 projects based on Firestore. We are using a lot the concepts of collection, sub-collection and documents and the undelying segregation of data. We also have implemented listeners in iOS and Android apps for sub-collections selected accordung to strong business and security rules, which is not possible with Datastore.

• 9 projects based on Datastore. For three 3 of them, we are are using a lot the concepts of namespace and kind. We also use the global indexing of kind's properties and the projection of properties server-side, which is not possible with Firestore.

PS: we are considering to open-source our python library for fast development of a common API running either on Firestore and on Datastore.

Miyokomizar

Can you please explain what you mean by "Strong performance expectation on search queries across all entities"? Also do you have comparison among the two modes w.r.t cost? – Denominational

am in the same boat, also want to use both database. Creating 2 projects, one which contains 90% of my business logic, etc & the rtdb is used for mobile caching and realtime updates such as chat features, etc – Marcmarcano

Is Datastore mode going to exist or no updates on it? – Denominational

Answer 3

According to official documentation, although Cloud Firestore is backwards compatible with Cloud Datastore, the new data model, real-time updates, and mobile and web client library features are not.

Cloud Firestore in Datastore mode uses Cloud Datastore system behavior but accesses Cloud Firestore's storage layer, removing the following Cloud Datastore limitations:

• Eventual consistency, all Cloud Datastore queries become strongly consistent.
• Transactions are no longer limited to 25 entity groups.
• Writes to an entity group are no longer limited to 1 per second.

Datastore mode disables Cloud Firestore features that are not compatible with Cloud Datastore:

• The project will accept Cloud Datastore API requests and deny Cloud Firestore API requests.
• The project will use Cloud Datastore indexes instead of Cloud Firestore indexes.
• You can use Cloud Datastore client libraries with this project but not Cloud Firestore client libraries.
• Cloud Firestore real-time capabilities will not be available.
• In the GCP console, the database will use the Cloud Datastore viewer.

Ozell

Thanks for this concise summary. So, to directly address the question: I gather from this answer that there is not a single piece of functionality (apart from the different API, obviously) that is missing from Firestore in "native mode", compared with Firestore in "datastore mode" (or the old Cloud datastore), because Firestore in "datastore mode" is just a kind of compatibility wrapper that is using Firestore native functions under the hood. Am I correct? – Giacomo

Yes, it is. They just merge into one API. In lower layer it is still Cloud Datastore – Ozell

Okay, thanks. (I guess what you mean is "At a lower layer it is still Cloud Firestore :P) – Giacomo

Is Firestore going to cost the same as Datastore – Too

@Too It is same. Just that Datastore mode has some free offer for small operations. – Denominational

Answer 4

Because Firestore in Datastore mode is optimized for server use cases and for App Engine, we recommend using Firestore in Datastore mode for databases that will be used primarily by App Engine apps. Firestore in Native mode is most useful for mobile and real-time notification use cases.

https://cloud.google.com/appengine/docs/flexible/go/using-cloud-datastore

When looking further into the docs after reading this post, I came across this official GCP page. It explains in plain English that Firestore "Datastore mode" recommended for primarily server-side use cases.

This answers my question perfectly, because my use case is only server-side. Note: I don't have any first hand experience with Firestore yet, but I appreciate the answers that do.

Sympathin