Import and Export Indexeddb data

Question

I have an Epub annotation plugin where user can annotate the Epub, but the annotated text is stored in the browser in browser's indexeddb database, want to export those annotated text to an file and in other system i want to import those file to my indexeddb database.

How can I do this , can any one suggest any idea along with some URLS , examples, code.

Endometriosis

Answer 1

You can get anything out of an IndexedDB database as a JavaScript object. Then you can use JSON.stringify to turn it into a JSON string. Then you can turn that into a downloadable file by using data URIs. I did something like this in one of my projects; you can see the git diff here.

For import, you can just accept a JSON file, parse it, and add it to the database.

You probably won't have this problem with your application, but I'm just writing it here for completeness... Data URIs only perform reasonably when the amount of data is small (< 1 MB roughly, although it's browser dependent). Downloadify performs better with large amounts of data, but I still ran into trouble with more than ~5MB of data. I'm still not sure what a good solution would be for larger files than that.

Thurstan

I've written a module indexeddb-export-import which helps with importing/exporting IndexedDB to/from JSON. – Ravi

i have tryin something same, but am getting error in JSON.stringify() , i tried with a CSV with 1 million data (150MB) size, breaking in JSON.stringyfy how to tackle this issue? please help me – Toback

@JustinEmery, have you done any testing on large datasets for 100s of thousands of rows or maybe a million ? – Boutin

@Boutin no. The module uses JSON.stringify() under the hood so that may be a limiting factor given that another commenter hit a limit with that. – Ravi

For extremely large data (like so large that reading it into memory will crash the browser tab) I think dumbmatter.com/2021/06/streaming-data-from-indexeddb is the only way to export it from IndexedDB without splitting it into multiple files. Currently only works in Chrome. – Thurstan

Answer 2

You can import/export IndexedDB data using leveldb-cli:

To install:

go install github.com/cions/leveldb-cli/cmd/leveldb@latest

To dump all data in text form:

leveldb -i -d <path/to/my/indexedDB> show -w > dump.txt

Command-line usage:

$ leveldb -h
NAME:
   leveldb - A command-line interface for LevelDB

USAGE:
   leveldb [global options] command [command options] [arguments...]

VERSION:
   1.1.0

COMMANDS:
   init, i    initialize a database
   get, g     get the value for the given key
   put, p     set the value for the given key
   delete, d  delete the value for the given key
   keys, k    list all keys
   show, s    show all entries
   dump       dump the database as MessagePack
   load       load MessagePack into the database
   repair     repair the database
   compact    compact the database
   destroy    destroy the database
   help, h    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --dbpath dir, -d dir  path to the database directory (default: ".") [$DBPATH]
   --indexeddb, -i       open Chromium's IndexedDB database (default: false)
   --help, -h            show help
   --version, -v         print the version

Shadow