Offline Firebase

Question

I am implementing an AngularJS web app with Firebase as a backend; it should work offline, too; multi-user sync issues should be very limited, since the app - by design - will only allow new data entries when offline.

I understand Firebase has offline capabilities, in the sense that a client can withstand temporary network connection failures: any write operation will be delayed and cached until network comes up again.

I ask if any possibility does exist (or does any plan to implement it) to extend Firebase offline capabilities to enable clients to locally cache a snapshot of (some of) the data on the server, to be able to offer clients a complete offline experience, with read operations available, too.

I see a third part Firebase wrapper exists, but it's documentation is quite 'limited' (to be kind... :-). A native solution should be preferred...

UPDATE: After Frank van Puffelen comment, I better qualify my question:

*Does Firebase natively support offline data access in its Web API, or will it any time soon?*

Philippine

So is your question: does Firebase natively support offline data access in its Web API? Because if so, the answer is "no". – Shorter

Yes. More precisely it is: "Does Firebase natively support offline data access in its Web API, or it will any time soon?" – Philippine

In the foreseeable future offline persistence will be released on both Android and web. It's available in beta on iOS. – Pull

@Kato: for "web" may I read "AngularFire", too? Could you anticipate what the cacheing space limits will be, for the web? – Philippine

@Pull Could Firebase please make their intentions here clear? What options are you working on for local storage, how far are they along, when are you planning release. Many months have past and it's very concerning to see your moderators deleting posts on Google Groups which ask questions about this.... – Dillion

Our intentions are perfectly clear: Roll out offline persistence as soon as possible. It's currently our top priority and looks to be targeted for some time next year. I can't be more specific than that as our internal road map is governed by a large number of policies and people, nor will more questions on the mailing list or here get it done any faster. – Pull

@Pull can you make a real response below? Right now this question reads more like an ad for PouchDb. :-) – Woad

Answer 1

An alternative to Firebase that solves this problem for JS apps is CouchDb (server) <=> PouchDb (JS client). If you've implemented a nice clean Service layer for your AngularJS app then porting to PouchDb should be fairly straight forward since both are NoSQL/JSON databases.

PouchDb is a Javascript API that implements a fully offline CouchDb client. It can auto detect and use either _local storage_, _IndexDb_ or _WebSQL_ to permanently persist local data while online or offline. The PouchDb API can be used to access either your local or remote databases (just change the URL) and wire up full syncing or filtered syncing between the two. There are many useful PouchDb plugins, code samples and a small wrapper library to support AngularJS's Q promises API.

Using PouchDb, you can safely start up your app while offline and then days later restart your app and sync all your CUD data changes to the server. This can result in update collisions so CouchDb supports record versioning that is designed to detect and track this. Consequently, you'll likely need server side logic to resolve these collisions. This is unavoidable for distributed systems with offline synchronization and a key feature of CouchDb (not quite true ... see comments)

PouchDb is basically a reimplementation of Apache CouchDb including it's efficient synchronization protocol. Both CouchDb and PouchDb are well tested, free and open source. Being open source means that a CouchDb server can also be deployed as an Intranet service - optionally syncing to an external cloud service. There are a number of CouchDb hosting providers.

The Cloudant hosting team recently added their BigCouch clustering features to Apache CouchDb 2.0 project so now you can scale from Micro Db (PouchDb) => Single Server => Multi-Master (Replicated) => Big Couch Clustered / Geo Clustered. Unlike MongoDb, CouchDb safely supports single server deployment.

NOTE: PouchDb can also sync to CouchBase using the same protocol but Couchbase !== CouchDb. It's a commercial product.

• How to code your CouchDb and PouchDb replication for mobile or web apps.

Links:

• Apache CouchDb Project
  • Couch Db - The Definitive Guide - Free book.
• Pouch Db
  • AngularJS wrapper for PouchDB
• Couchbase

CouchDb Hosters:

• Cloudant
• Iris Couch
• Smileupps

DIY

• How To Install CouchDB and Futon on Ubuntu 12.04 - Digital Ocean
• Secure CouchDB by using SSL/HTTPS

Docker + CouchDb:

• Dockerizing a CouchDB Service
• GitHub: Yet Another Dockerized CouchDB
• Dockerized CouchDB 1.6.0 with stud SSL terminator

Truck load of Plugins

PouchDb has a number of extension points and growing list plugins (37 last count):

• Plugins and External Projects

Security Model

One issue you'll need to consider when migrating to CouchDb is that it has a more limited access control model. This is partly due to it's replication algorithm. This blog post covers this in detail (better than the real definitive guide).

• Matt Woodward's Definitive Guide to CouchDB Authentication and Security
• CouchDB Security Overview
• Superlogin - Client and server authentication APIs based on PassportJS to use OAuth2 or local signup/logins to a CouchDb server. Optional Angular API and a nice Demo reference app. Replaces weaker basic authentication in CouchDb with a randomly generated username and password for each login session (like an auth token) after authentication via PassportJS which supports over 300 auth plugins.

Cabalism

I said "Consequently, you'll likely need server side logic to resolve these collisions. This is unavoidable for distributed systems with offline synchronization and a key feature of CouchDb." ... not quite true You can just create "delta" docs that record changes and thus avoid updates and ensuing collisions. Delta docs act like an audit trail. To collapse the change docs you add an indexed map/reduce view. Couch views update incrementally by applying new changes since last read so it's very efficient. You can also create multiple views over the audit (e.g. different summary levels). – Berar

Answer 2

According to the Firebase docs, it does: https://www.firebase.com/docs/web/guide/offline-capabilities.html

From the site: Firebase provides some simple primitives that allows data to be written when a client disconnects from the Firebase servers. These updates will occur whether the client disconnects cleanly or not, so we can rely on them to clean up data even if a connection is dropped or a client crashes. All Firebase write operations, including setting, updating, and removing, can be performed upon a disconnection.

Am I interpreting the question incorrectly ?

I almost deleted my post but when I clicked on the link OP had given, I saw that the third party package does exactly the same thing as what Firebase itself does, maybe it was done before Firebase improved sync ?

When I deleted the post, I thought the OP might have wanted a selective set of data only, not 'active data' which is what FB stores locally until connection is restored

Benedic

OP, here... :-) I wasn't clear enough, in my question... The point is: does (or will) FB offer real offline capabilities? I.e. being able to initiate a connection while offline, non just withstand a network failure when a connection is up, and data already on the client. To this purpose it would need some kind of client caching (appcache?). I spoke about 'partial' data since any browser's cache will be limited in size... I heard some rumors FB is going to implement something similar, so I'd like some anticipation about implementation and expected "time to market"... :-) – Philippine

I jumped on the FB bandwagon thinking the same. Was duly disappointed to learn it cannot cache data for offline use. To their benefit we can say how can we define what should be 'initial data'. But the puzzling question is, what will happen when we do initiate an FB session whilst offline, will the app (say AngularFire) let us add records or will we get a dino (offline message) ? Marco, you might wanna check out appery.io, they claim their generated apps can do this, albeit not to a 'large extent'. No clue what they mean; they say you write your own js code to sync if it's too complex. – Benedic

thanks for pointing me to appery.io, but going with it would mean to restart my app from scratch or so, it's not an option, for me... :-) Offline usage is not an imperative requirement for me... I'd better think about swapping from Firebase to a custom database solution (deployable to customer's lan/device). However, thanks ti Kato's anticipation, I am going to wait for the release of offline web capabilities by Firebase folks... :-). P.S.: Sorry I'm not accepting this answer, it's not resolutive, for me... – Philippine

That offline capabilities page left me with the impression that it cached data locally. It does not. The data only exists in RAM and any refresh or relaunch of the app results in no data being present. I believe the onDisconnect stores some javascript on the server to be executed when the client goes away, obviously data can't be sent to the server when you're offline. I'm left with a bad taste by Firebase being deliberately evasive and using double-speak on this issue. – Dillion

The crucial word is "... even if your app loses its network connection temporarily". In other words you must be online to perform FB authentication and then they can handle temporary drop outs. Yes IMHO they were not very clear on this. This may change in the future but not at present. – Berar