Google Firestore a subset or superset of Google Cloud Datastore?

Question

Google announced Firestore, the new document datastore on the block.

I have been developing an application using Google Cloud Datastore for over six months now and after reading the blog, I feel Firestore seems to be a better choice.

The concept of the alternate collection-document-subcollection looks excellent to me because while designing schema for datastore I was aware I will be unable to query nested fields. Now with firestore subcollections, I get full query capabilities which is a game changer for me (I can get maximum data with minimum queries).

As a counter argument, the flowchart suggests me to use datastore because I do not have any mobile clients.

Will it be a good idea to use Firestore just like Datastore ? (I will conveniently ignore the mobile client/realtime updates/syncing features!)

Osy

Answer 1

Update 2 (01/31/19)

As of today, Cloud Firestore is no longer in Beta and is Generally Available: https://cloud.google.com/blog/products/databases/announcing-cloud-firestore-general-availability-and-updates

This means that Cloud Datastore is no longer an option for new projects (you can keep using it on existing projects). New projects that want to use the Datastore API can use Cloud Firestore in Datastore mode.

Update 1

As you we have noticed, we've expanded Cloud Firestore since this question was posted.

This means Cloud Firestore now has 2 modes:

1. The original launch was 'Native mode'
2. The new launch adds 'Datastore mode'

'Datastore mode' is the 3rd gen of Cloud Datastore. 1st was called Master/Slave Datastore, 2nd was High-Replication Datastore (HRD) that was rebranded as Cloud Datastore in 2013.

The below answer is still largely relevant since both modes are currently mutually exclusive, so you need to pick one or the other.

The main differences are the improves of Cloud Firestore in Datastore mode over Cloud Datastore. The biggest ones are:

• Write through-put per entity group now unbounded (was 1 write/second)
• Transactions no longer limited to 25 entity groups
• All queries now strongly consistent.

Also note Cloud Firestore regardless of mode is beta, so the new Service-Level Agreement (SLA) doesn't go into effect until the product reaches General Availability (GA).

Original Answer

Cloud Datastore (CD) and Cloud Firestore (CF) are similar, however different in significant ways.

CF is mobile-centric with direct from mobile client functionality with the Firebase SDKs and Rules functionality. CD is server-centric with a wider range of server client libraries, as well as some mature frameworks on App Engine Standard that bundle in memcache functionality.

CF has a newer storage layer that is strongly consistency in the same way as Cloud Spanner, however, it's still in beta without an SLA. CD's storage layer is only strongly consistent within entity-groups and eventually consistent across entity-groups, however, it is GA with a 99.95% SLA for the Multi-Region locations.

CF is only available in the US Multi-Region at this time. CD is available Cloud across a dozen locations including places in the Americas, Europe, Asia, and Australia.

CF during beta has a guideline limit of 2500 writes/second while we build experience monitoring and tuning the system prior to GA, whereas CD will happily handle >1M writes/second (please reach out to your account rep first though).

CF and CD's set of query capabilities are overlapping but not the same. Overall CD has a broader set of query capabilities we haven't built in CF yet, so you'd have more flexibility in CD.

Overall, I'd consider this list to see if any of the differences make or break what you're trying to build then pick the DB that fits closest to your needs.

Piaffe

Thank you for the clarification, Dan. I am enticed by Firestore just by the fact that I can query nested objects using the sub-collections. I want to reduce the data redundancy in CD (which was created by the limitation of CD). – Osy

Is Firestore being positioned as a replacement for Cloud Datastore? imgur.com/a/EWx5v – Custumal

@DimuDesigns Great answer from Dan the man: #48992825 – Hafner

This is now a dated answer. See mine below – Inveigle

The use of insider abbreviations makes this hard to decipher for me. What is "GA"? What is "SLA"? – Androecium

@Somo - updated, got a little busy during the launch period :) – Piaffe

@Androecium - they are standard software terms for the last few decades, but you're still right. I expanded and linked to details for both GA and SLA upon first usage. – Piaffe

CF is now available in 12 locations. and extended to more features in Native mode. perhaps the answer need to be updated – Encircle

on this page: cloud.google.com/products/databases Google does not even list Datastore. that makes me think for new projects, they want us to use Firestore. – Replacement

Answer 2

Firestore is the 3rd generation architecture and replacement for Datastore, essentially available in 2 modes: Native mode and Datastore mode.

• Documentation regarding the choices: https://cloud.google.com/datastore/docs/firestore-or-datastore
• Video overview: https://www.youtube.com/watch?v=SYG-BgXoJFQ

S

Naice! I was attending Next 18 but I think I missed this session. Thank you for the update. – Osy

That's @DanMcGrath in the video (same guy with the accepted answer). It appears Google has changed its stance and Firestore is now Datastore's impending replacement. – Custumal

Answer 3

I'd say that Datastore is now a subset of Firestore:

Cloud Firestore is the next major version of Cloud Datastore and a re-branding of the product. See Choosing between Cloud Firestore and Cloud Datastore

Cloud Firestore can operate in "Datastore mode", making it backwards- compatible with Cloud Datastore. Some time after Cloud Firestore is released for general availability, Google will begin contacting owners of existing Cloud Datastore databases to schedule an automatic upgrade to Cloud Firestore in Datastore mode. See auto upgrade

Inveigle

To a first approximation that seems to be the case. But there are some additional Cloud Firestore Native Mode limitations that don't apply to Datastore mode: "No collection group queries and projection queries" and "Up to 500 documents per transaction across any number of collections." (vs "Transactions can access any number of entity groups") and no namespacing. cloud.google.com/datastore/docs/firestore-or-datastore – Sauls

Also I'm curious to know if there are significant latency differences. – Sauls

Answer 4

Google documentation says:

Firestore is the new version of Datastore and removes several Datastore limitations.

Jigging

Answer 5

I think cloud firestore also has nodejs client and its not centric to mobile. Actually thats the difference between Firebase realtime database which was mobile-centri vs Cloud Firestore which is anything centric.

Laurynlausanne

There is a Node.js SDK for both Firestore and "legacy" Firebase RTDB. So there is no (and has never, or at least not as long as I'm aware, been any) difference here. Both are "anything centric" in that sense, whereas Cloud Datastore (and, by extension, Firestore in "Datastore mode") is server-centric. – Androecium