PouchDB delete data on device without affecting remote sync

Question

Right now I am replicating my entire device database over to my remote database.

Once that is complete, I grab all my data that is not older than 1 month from my remote database, using a filter, and bring it to my device.

FILTER

{
  _id: '_design/filters',
  "filters": {
    "device": function(doc, req) { 
      if(doc.type == "document" || doc.type == "signature") { 
        if(doc.created >= req.query.date) return true;
        else return false;
      } 
      else return true;
    }
  }
}

REPLICATION

device_db.replicate.to(remote_db)
.on('complete', function () {

  device_db.replicate.from(remote_db, {

    filter: "filters/device", 
    query_params: { "date": (Math.floor(Date.now() / 1000)-2419200) }

  })
  .on('complete', function () {

    console.log("localtoRemoteSync replicate.to success");
    callback(true);

  });

});

My question:

I want to be able to periodically delete data from my device that is older than 3 months (old enough data where I already know it's been sync'd)

But just because I delete it from my device, when I replicate the data back to my remote_db, I don't want it to be deleted on there too.

How can I delete specific data on my device but not have that deletion translated when I replicate?

Denti

According to your workflow, will it happen that you will delete documents in your localDB and it needs to be replicated to the remotedb ? – Spirula

@AlexisCôté the goal is to only delete items that have already been replicated. Once a replication is complete with the remote database, I want to do a check to see how old the data is and delete it on the device. – Denti

I totally understand this part. My point is that you can filter deleted documents and block them. So except the "old data delete", will you do any delete operation on your local database ? – Spirula

I'm a bit confused but to try and answer your question, for now, the only time I want to delete something is on my device_db. My remote_db will always retain all data. @AlexisCôté – Denti

And last question, when, in your workflow, you will do the delete? As I can see, you do the following : ->push to remote->getLatestWithoutOld-> – Spirula

@AlexisCôté I want to do the delete once all the replication is complete, right before the callback(true). So the workflow would be push to remote, grab latest from remote, delete old from device. – Denti

Answer 1

FILTERS

Here, we have 2 filters:

noDeleted : This filter doesn't push _deleted documents.

device : Filter to get the latest data only.

{
  _id: '_design/filters',
  "filters": {
      "device": function(doc, req) {
          if (doc.type == "document" || doc.type == "signature") {
              if (doc.created >= req.query.date) return true;
              else return false;
          }
          return true;
      },
       "noDeleted": function(doc, req) {
          //Document _deleted won't pass through this filter.
          //If we delete the document locally, the delete won't be replicated to the remote DB 
          return !doc._deleted;
      }
  }
}

---

REPLICATION

device_db.replicate.to(remote_db, {
      filter: "filters/noDeleted"
  })
  .on('complete', function() {
      device_db.replicate.from(remote_db, {
              filter: "filters/device",
              query_params: { "date": (Math.floor(Date.now() / 1000) - 2419200) }
          })
          .on('complete', function() {
              console.log("localtoRemoteSync replicate.to success");
              callback(true);
          });
  });

Workflow

1. You push all your documents without pushing the deleted document.
2. You get all the updates for the latest data
3. You delete your old documents
   • You could either query the remote DB to get the ids of the documents that are too old and delete them locally. Note that the documents will still be there as _deleted. To completely remove them, a compaction will be required.
   • You could also totally destroy your local database after step1 and start from scratch.
4. callback(true);

Sphenogram

Thank you! If I have auto compaction do I still need a filter? – Denti

According to the doc (auto_compaction: This turns on auto compaction, which means compact() is called after every change to the database. Defaults to false.). So if the compaction is run after your documents deleted, you shouldn't need a filter to push your changes. Just test it! – Spirula

FYI: For me, it replicated the deleted doc with auto_compaction on (without using filters). – Happ

Answer 2

Add a one-way filtered replication. However anything you need back on the server you will need to use a put request with the server's _rev.

For example

1. Replicate from server to client, then add a filter mechanism, like transfer:true to the docs you want to replicate. replication

    db.replicate.from(remoteDB, {
    live: true,
    retry: true,
    selector: {transfer:true}// or any other type of selector
    });
   

2. To delete a doc on the client, set transfer to false, then delete it on the client. it won't meet your filter criteria so it won't replicate.

3. Anything you want to put back to the server use a put request instead of replicate.

4. If you want the document back on the client just set transfer to true in the doc.

Messenia