Best way to synchronize local HTML5 DB (WebSQL Storage, SQLite) with a server (2 way sync) [closed]
Asked Answered
S

2

153

I am developing a mobile web application (for iPhone & Android) with a local database (using html5 webstorage) so my app is still usable when the user is offline.

This is working perfectly, but I want to save the local data on a server. So I need to synchronize the local DB with a DB on a server. The synchronisation can only be one way, but in the future, I would like to synchronize it in the both way (server <-> local DB).

This requierement looks very common (or will be common in the future for mobile web app), but I can't find a library doing that.

I know google is doing that in their mobile web app (ex. gmail), and I found the WSPL project a google project but without source to download.

If I can't find a solution, I will create a library to do that, as one way sync doesn't look difficult, but I wonder if there are other solutions.

Subclinical answered 16/11, 2009 at 20:2 Comment(11)
I don't know if there are any libs, but easiest way to do this seems to be storing modification timestamp, and transferring changes to records which are newer than the records on other side, and also transferring additions and deletions since last sync. It may go crazy if local and server clocks are not in sync, but you'll think of something. -- Posting as comment since it's probably not very helpful and doesn't provide you with an answer.Epithelioma
Thanks Ivan. You are right, if the local and server clocks are not in sync, it could be messy... I just found that : quickconnect.pbworks.com/Using-Enterprise-Synchronization It says it can synchronize a local HTML 5 DB with a DB in a server. I need to have a deeper look on that, and see if it can run outside the QuickConnect Framework...Subclinical
I found another solution : impel.simulacre.org/blog/… Looks great, but you will need to use Mootools library, and the Impel ORM...Subclinical
How about CouchDB? couchdb.apache.orgC
I don't think it provides WebSQL synchronisation...Subclinical
Topics are for discussions, Stack Exchange is for questions. At one point posts like this were accepted on Stack Exchange, but no longer.Globeflower
Microsoft Access has a way of synchronizing rows by using unique hash type ID. That could be an additional layer including a transaction date. If it is just one table this is an easy task. When you get into multiple tables and dependencies, this is when can be messy.Atheroma
You can mark all offline changes or new records that are done in the HTML5 (local) db. The sync process will just effect on synchronizing the changes to server which are marked as "offline".Rev
These answers are out of date, check out pouchdb.comNapoleon
@JChrisA Your comment is out of the subject, we are talking about WebSQL DB hereSubclinical
Is there any framework to Sync IndexDB database to DB?Sundin
S
70
  • I created a small JS lib named WebSqlSync to synchronize a local WebSql DB with a server (client <-> server). Very easy to use and to integrate in your code :

https://github.com/orbitaloop/WebSqlSync

  • The open source project QuickConnect contains a JS library to synchronize the local HTML5 SQLite DB to a server DB (MySQL or other) :

http://quickconnect.pbworks.com/Using-Enterprise-Synchronization

To use this lib, you need to use the DataAccessObject of the framework to access your DB. It works by storing all the SQL request applied to the DB (except select of course) , and sending them to the server. It's great to manage deletion, but it's a little heavy if you have a lot of updates, and the server need to use the same SQL language...

  • Another project from QuickConnect is a native SQLite synch (in Objective C for iOS or Mac OS and in Java for Android) :

http://www.quickconnectfamily.org/qcdbsync/ (I think it store also the history of all the SQL requests)

  • And i just found another promising JS library : persistenceJS

https://github.com/zefhemel/persistencejs

"persistence.js is an asynchronous Javascript object-relational mapper library. You can use it in the browser, as well on the server (and you can share data models between them)."

They have a DB synch module: DOC of persistence.synch.js

(works with HTML5 DB SQLite or Google Gears on the client, and MySQL on the server)

  • And there is also Impel.inTouch. It looks very easy to use (with php files included), but you must use the Mootools framework in the client side :

http://impel.simulacre.org/api/Impel.inTouch

  • Sencha has also a synchronisation service: Sencha.io. Looks great, but it's dependent of the Sencha Touch framework:

http://www.sencha.com/products/io/

Subclinical answered 16/11, 2009 at 20:2 Comment(10)
Hi Samuel, did the js lib work out for you?Longicorn
DB sync is not my priority for now, so I just give up, waiting for a more standard and robust solution...Subclinical
as the WebSQL Standard is already deprecated (link) and work will not be continued, I would rather search for some alternatives, as WebSQL will probably not be supported in future releases!Lilialiliaceous
WebSQL is being used in thousands of webapps (even Apple and Google use it), so I don't think it will be removed soon...Subclinical
After testing all of them, I think I will develop my own small JS lib to synchronize WebSQL with a server DB. It will be a double synch (local <-> server) and will not have any dependency. I'll post here the link to the code once finishedSubclinical
I had the similar situation to sync mobile data with enterprise database. I have used persistencejs and written custom sync logic based on sync/not-synced row property on server as @samuel mentioned.Ringnecked
I have commited the first version of my own sync solution named WebSqlSync : github.com/orbitaloop/WebSqlSync (cf below answer)Subclinical
Hi Guys, I've started a persistencejs plugin for restful synchronization. It still in development, but if anyone wants to check it out: github.com/robertokl/persistencejs and a working example on server/client side with ruby on rails: github.com/robertokl/persistencejs-restfulSync-exampleMapping
Do you have anything similar but for localStorage and not WebSQL?Unbind
A quite recent alternative is replicache.devConcessive
S
18

I have developed a generic sync solution called WebSqlSync.

It's not dependant of any framework. It's available here : https://github.com/orbitaloop/WebSqlSync

Extract of the README file :

WebSqlSync

Automatically synchronize a local WebSql database (SQLite in the navigator) to a server. (2 way sync : client <-> server)

Very easy to integrate to your existing app and very easy to use (2 functions to call : initSync and syncNow)

Usage

Initialize

You need to initialize the lib (at each startup for example).

It will automatically create 2 tables (if they don't already exists, one to store all the new or modified elements (table new_elem) and one to store the date of the last sync (table sync_info). It will also create SQLite triggers in order to watch the INSERT or UPDATE on the tables you want to synchronize (to automatically insert the modified elements in the new_elem table):

DBSYNC.initSync(TABLES_TO_SYNC, webSqlDb, sync_info, 'http://www.myserver.com', callBackEndInit);

Where TABLES_TO_SYNC is the list of table that you want to sync with the server, ex :

TABLES_TO_SYNC = [
    {tableName : 'table1', idName : 'the_id'},
    {tableName : 'table2'} //if idName not specified, it will assume that it's "id"
];

Synchronize

To start the synchronization, you need to call the syncNow function. You can call it every X seconds, or after some changes for example :

DBSYNC.syncNow(callBackSyncProgress, function(result) {
     if (result.syncOK === true) {
         //Synchronized successfully
     }
});

And that's all you need to do on the client. On the server side, you will need to code your own solution (but it's not complicated). And there are some example inPHP & Java. Again, contributions are welcome.

Subclinical answered 2/2, 2012 at 14:54 Comment(4)
How well would you say this have worked for you, summarizing it one year later? I'm looking for a good client-side DB that works for browsers and mobile units.Yehudi
WebSQLSync is working very well with more than 25 apps in production (iOS and Android). WebSQL is really great and fast. It's working on iOS, Android, Blackberry (the latest version I think) and of course chrome and safari. But it's not working on IE and firefox, because the API has been depreciated by W3C..Subclinical
Alright, both positives and negatives there. Thanks for the recap!Yehudi
Do you have anything similar but for localStorage and not WebSQL?Unbind

© 2022 - 2024 — McMap. All rights reserved.