Offline apps with Node.JS and CouchDB

Question

I have an app that I would like to create. But I am not sure how to go about it. I am using node.js and would like to use couchdb, but if something like mongodb or riak would be a better choice them im willing to hear ideas. But, i have a site, say

cool.com

and on there is a couchdb instance, as well as a site to manage a store. say a shopping cart. the db houses all the store's items and data. The app itself has an admin backend to manage that data and can change items. What i would like to be able to do, is have the ability to have the user be disconnected from the internet, and still have the admin backend work. I realize for this to work I need to use a client side framework with my models/routes/controllers/whatever. But what I am not sure of, is how to let the site function while offline. couchdb if installed locally can sync the data from local to remote when back online, and if the admin user is on the computer, i could have them install couch. but that could be messy.

Also, what if the admin user is on a tablet or a phone? Would I need to have an actual mobile app and a desktop app to do this? is there some way I can set this up so it is seamless the the end user. I would also like this to be offline for end users too, but the bigger audience is the admin.

Another use case, instore POS system. and the power goes out. But the POS system can be loaded from the web onto a tablet and they can still make card based sales if the wifi is out, because the app is available offline.

Im just not sure how to do this. lets assume i need a client framrwork that can handle the data as well as the backend. something like ember, or angular. theres also all in one stacks like meteor and derby js, but those arent fully offline,but are for the appearance of real time. though meteor does have mini mongo so it might be worth looking into.

I was hoping someone could help me figure out how I would get this setup to work, preferrably with couch, but other nosql's would work too if I can have a way to sync the data.

Diction

What did you mean by card based sales? Did you mean payment by credit card? – Chak

Yes. So an in store point of Sale system. It will be taking multiple payment methods. – Diction

What did you end up going with? – Pediment

I'm still looking into it. most of my weekend was away from my computer. I'm currently looking at pouchdb, which should be able to be integrated into my app. But there's not a lot of docs on how to get it going. ill be sure to reply back here with what I found. – Diction

Answer 1

There's an interesting project out there called AppJs (http://appjs.com/), which packages Node.JS and Chrominium as a desktop environment. It's currently very fresh (very little documentation), but it appears to be straight forward enough (you'll be using the same tools as you would for your online application).

As for synchronising the offline and online environments. I doubt you can rely on CouchDB in the way that you envisage. CouchDB mobile support is not as comprehensive as some of the documentation suggests. So in this sense, it would be no different to using SQL/Mongo/Punchcards.

You might have more luck with designing a suitable serialisation scheme based on XML or JSON (or just plain text), and passing files between the online and offline installations.

Edit - Since writing this, Node Webkit - http://nwjs.io/ - is clearly the most obvious replacement for App.js. It has a very simple API, and some great features.

Ruttish

hrmm, app.js looks like it could very well fit. do you know if it has the capability to bundle the server as well? If I could get it to bundle the couch instance it would be perfect. – Diction

just to note, I will be using app.js for a desktop instance of my app. and something like phonegap for mobile. Though I know i have other issues to bettle for a local database. But for the time since it will only be installed on a few computers, i can simply create a command script to install the db manually, and later on find a solution to package it properly. – Diction

Answer 2

I'm not sure if it would work for you, but I have been thinking of such an application for quite a long time now and been doing some research on what's possible. The best solution I could come up with is using a server with a couchdb and writing the application clientside based. Then for the data storage use pouchdb and synchronize the pouchdb regularly with your serverside couchdb if the app is online. I know pouch is in an early stage and not production ready but if you are willing to put some work into it I'd say it's doable.

Gold

pouchdb looks promising as well. I really like this. if it can be bundled with app.js that @charlie mentioned, that would be a plus. – Diction

@luckysmack you don't actually need AppJs for PouchDB, it runs on IndexedDB in the browser (or Chromium Embedded or wherever you run it), with WebSQL and LocalStorage support also planned for backwards compatibility. – Autocracy

is there any documentation for accomplishing this? what if in the end I don't use app.js and just use a users built in browser? or something else? – Diction

talking with someone at couch (irc), pouchdb would not work for my use case. I would need to actually embed the db. But in looking around and researching, I will be using either mongo, or orienddb, which i explained in another comment. – Diction

Answer 3

If you want clients that work seemless as they go offline and come online (like a POS with the power out) then I would recommend making the app primarily work off local storage with a background publishing or synchronization to the cloud.

Local storage options could be everything from something light like sqlite, sqlexpress, firebird to no sql options like mongo, couchdb etc...

But for the client or device, consider the ease of configuration and weight of the option. You also need to consider the type of clients - do you have many platforms varying from devices to PCs? You don't want something that has a heavy config and runtime footprint. That's fine on the service side.

On the service side, consider the nature of your data and whether it's fitted better for transactional/relational systems (banking etc...) or eventually consistent/non transactional (no-sql) documents. Don't forget hybrid as an option. Also consider the service platform - for example, node goes well with mongodb (json objects front to back) ...

The device and service storage options can be different (and likely should be) separate by service interfaces (soap, rest/http, sockets etc...).

It's hard to have a one size fits all solution but often something light weight like sqlite on the device or client makes for ease of installation/config while scalability on the service side with something like sqlserver/mysql or couchdb/mongodb makes sense.

Some links to read:

• http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB
• http://www.sqlite.org/
• http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx

You're question is pretty wide open and there's no one size fits all solution. Hopefully I provided some options to think about.

Pediment

I had considered local first as well. essentially making 2 apps. the site, and the server. both could use very similar api's which would make it easier. Actually, part of the system will be a POS interface that * should * be available on an tablet. thought I may be able to force them to use the online version. the desktop needs to be able to have offline (again, POS). Alhtough if i could get offline for mobile that would be great as well (phonegap?). If i need to build a couple separate apps I will, I just need it to be seamless as possible overall. – Diction

You could consolidate the device/PC around sqlite (it's multi-platform - share sql code) and the service side around whatever storage option makes sense there (mongo/couchdb). I found mongo to fit with node but haven't played with couchdb as much. – Pediment

i was thinking, only offer the offline app to admins, and still require credentials to setup the sync. then on the site it could check locally for the db and wait for it to sync. when its done it uses the local one. if there isnt a local copy, it uses the online db. this way its one app, and i only provide some kind of package for the local db. but this still leaves to question how I would tackle mobile. force them to be online? or make a mobile app. couch apparently has a mobile client in the works for couch, which they say they have working, which may be an answer as well. – Diction

are there any libraries to take data from sqlite and push it to the server? if thats the case i could use that for real local storage, and find a way to push it up when online and push the data to monogo/couch. u would have to look into that as a solution. as the local db could be sizable as well. – Diction

I think it's tough to have a one size fits all general data/sync model. there's also value in abstracting the server via http services. The data and app rules vary greatly - good luck. – Pediment

yea i have a planned out rest api for communicating with the server. so I should be able to separate the front ends with the server. I appreciate the input, ill have to look more into sqlite. – Diction

I would also advise to be wary of sometimes local is primary store, sometimes server - it can get complex with multi-master. IMHO, pick a pattern like local is primary with publishing or server is master with client caching etc...and stick with it. – Pediment

yea in this case the server is always the true master. but this kind of setup is the main purpose behind couch. its made to keep local in sync with remote. you can even specify what db's or parts of a db will sync. – Diction

Currently, I will be having a core api that will be on whichever platform it is on. and have a different view/UI layer for desktop and web, if something like app.js does not work. It bundles in a portable web kit instance so I will be able to control my UI, as if it was html/js, since, it is. as for using localdb first, when the dektop client it made, it will use the local db before remote. and background processes will then sync the data. – Diction

I also will be using mongo for my database, unless I can find a way to use orientdb, which is graph based, but can be like document stores, and it has ACID compliance. this was a great answer though. I would makr it as answer if I could mark 2. But using app.js is technically the true answer to the question. I appreciate the feedback. – Diction