How to access Google Chrome's IndexedDB/LevelDB files?

Question

I want to use Google Chrome's IndexedDB to persist data on the client-side.

Idea is to access the IndexedDB outside of chrome, via Node.JS, later on. The background is the idea to track usage behaviour locally and store the collected data on the client for later analysis without a server backend.

From my understanding, the indexedDB is implemented as a LevelDB. However, I cannot open the levelDB with any of the tools/libs like LevelUp/LevelDown or leveldb-json.

I'm always getting this error message:

leveldb-dump-to-json --file test.json --db https_www.reddit.com_0.indexeddb.leveldb

events.js:141
    throw er; // Unhandled 'error' event
        ^   OpenError: Invalid argument: idb_cmp1 does not match existing   comparator : leveldb.BytewiseComparator
      at /usr/local/lib/node_modules/leveldb-  json/node_modules/levelup/lib/levelup.js:114:34 Christians-Air:IndexedDB 

Can anybody please help? It seems as if the Chrome implementation is somehow special/different.

Perichondrium

I've ended up with using localstorage in Chrome, which uses a regular SQLite database, which was pretty easy to work with. – Perichondrium

Another option is perhaps using a service worker and do the later processing in the browser itself. – Mechanize

Answer 1

Keys in leveldb are arbitrary binary sequences. Clients implement comparators to define ordering between keys. The default comparator for leveldb is something equivalent to strncmp. Chrome's comparator for Indexed DB's store is more complicated. If you try and use a leveldb instance with a different comparator than it was created with you'll observe keys in seemingly random order, insertion would be unpredictable or cause corruption - dogs and cats living together, mass hysteria. So leveldb lets you name the comparator (persisted to the database) to help detect and avoid this mistake, which is what you're seeing. Chrome's code names its comparator for Indexed DB "idb_cmp1".

To inspect one of Chrome's Indexed DB leveldb instances outside of chrome you'd need to implement a compatible comparator. The code lives in Chrome's implementation at content/browser/indexed_db/indexed_db_backing_store.cc - and note that there's no guarantee that this is fixed across versions. (Apart from backwards compatibility, of course)

Shorten

IndexedDB inside nodejs enviroment will be awesome! – Neckwear

thanks Joshua for this detailed description of the matter! For what I'm trying to achieve, this is a bit too much effort for me to reimplement the needed comparator. But thanks for providing a well founded insight! – Perichondrium

Any idea if anything has changes during the last three years? – Gift

Answer 2

It's implemented and public available on github now

• C# example
• Python example

Almallah

Except I can't even install leveldb or plyvel on windows 10, python 3.8.10 so all those examples are useless – Galloromance

@MikaC. I no longer use this solution. I simply rebuild Electron and added custom node modules, and the problem is resolved on my side. – Almallah

Cool... thanks for sharing that with us. Do you think maybe you would care to elaborate? A link or code example of "custom node modules" would be nice. You might imagine that comments like these are pretty frustrating otherwise. – Sidero

Answer 3

maven: https://github.com/hnuuhc/often-utils code: Map<String, String> storages = LocalStorage.home().getForDomain("pixiv.net");

Vadnais

Welcome to SO! Please add some explanation. Imparting the underlying logic is more important than just giving the code, because it helps the OP and other readers fix this and similar issues themselves. – Mettlesome