pouchdb replicate from couchdb : very slow

Question

I got a ~10k entries(~30Mo, no attachment) database in my couchDB.

Using Pouchdb browser-side, when replicating from couch, it takes really a while to complete...

What surprise me is the amount of requests my couch receives during it (thousands!, I guess as many as documents) — is that normal ?

Is there a way to "bulk" those requests and generally accelerate the replicating process ?

Thank you.

Schnorkle

Not strictly related to your question, but explicitly using websql as the local adapter made a big difference to the speed of my applications: new PouchDB('name', { adapter: 'websql' }) – Sanyu

My comment is not strictly related to couchdb. In many applications that potentially need to retrieve a lot of data, it is advisable just to retrieve headers of the data instead of the details in the beginning. Most users just look at the headers without looking at the details. – Prismatic

Answer 1

I assume you're using the PouchDB.replicate function

In that case, try modifying the batch_size option:

PouchDB.replicate('mydb', 'http://localhost:5984/mydb', {batch_size: large_val})

where large_val is higher than the default of 100. The higher the value, the faster the replication should go, but the more memory it will use, so be careful.

See the API reference

Edit: Also note the option batches_limit which defaults to 10. This is how many requests may run in parallel at any time, so the number of documents in memory equals batch_size * batches_limit.

Dygal

Assuming my 10k entries, if I set bacth_size to 10000, will I have only 1 request ? – Schnorkle

@Schnorkle That's the idea. One of the best ways to minimize the time it takes for a database replication like this is to minimize the number of requests, so put it as high as you safely can. Every one of those 10000 entries will be loaded into memory at the same time so make the device you are uploading to has enough memory. Note the edit in my answer referring to batches_limit. – Dygal