Rethinkdb atomic operations

Question

Let's say I have a document

{
    id: 1,
    fruits: []
}

fruits here acts as a SET

Now I want to atomically add a value to fruits array for document with primary key = 1 OR create such document if it does not exist(i.e. use SetInsert ReQL under the hood)

I also need to do the same for increment(ReQL .Add)

Obviously this can't be done in client code as it breaks atomicity and I end up with inconsistent data

I wish something like this was possible

r.table('results').insert({
  id: '62c70132-6516-4279-9803-8daf407ffa5c',
  counter: r.row('counter').add(1).default(0)
}, {conflict: "update"})

but it dies with "RqlCompileError: r.row is not defined in this context in"

Any help/guidance is appreciated, thanks!

Herbartian

r.row basically means "the current row for each row you will operate on" in ReQL-speak. In an insert, r.row doesn't make sense which is why that error occurs. However, if you're looking for a particular document then there is no reason you can't get the document's field with a subquery, see my answer below on that :) – Launalaunce

Answer 1

This is not possible with insert at the moment. The other mentioned solution is not atomic because it uses a subquery. We're working on a solution for this in https://github.com/rethinkdb/rethinkdb/issues/3753 .

You can however use replace to perform an atomic upsert:

r.table('results').get('62c70132-6516-4279-9803-8daf407ffa5c')
 .replace({
  id: '62c70132-6516-4279-9803-8daf407ffa5c',
  counter: r.row('counter').add(1).default(0)
})

replace will actually perform an insert if the document doesn't exist.

Arst

Thanks for the help Daniel! – Herbartian

Answer 2

In RethinkDB, all single-query updates are atomic. If Rethink doesn't think a particular update/replace operation will be atomic, it will throw an error and require you to add a non-atomic flag to the query. So normally, you don't have to worry too much about it. However, this is only with update and replace queries. It isn't possible to do this atomically with insert.

You are correct that if you retrieve that document, update it client side, and then put it back in the DB that it would be a non-atomic update by nature as well.

In a single query, though, you could do the following which is effectively an upsert in the same manner you used insert for, but using replace:

r.table('FruitStore')
.get(1)
.replace({
  id : 1, 
  fruits : r.row('fruits').default([]).append('orange') 
})

...which would be atomic. Similarly, to use the add operation:

r.table('FruitStore')
.get(1)
.replace({
  id : 1, 
  count : r.row('count').default(0).add(1) 
})

Launalaunce

Thanks so much Chris for fast answer and explanation! – Herbartian

The only my worry here is rethinkdb.com/docs/architecture "How does the atomicity model work?". They state this: "Operations that cannot be proven deterministic cannot update the document in an atomic way. Currently, values obtained by executing JavaScript code, random values, and values obtained as a result of a subquery (e.g. incrementing the value of an attribute by the value of an attribute in a different document) cannot be performed atomically". Here it uses subquery – Herbartian

@sfireman You're right, the documentation does appear conflicting on this. This subquery utilizes the same document as the one it is updating, so theoretically this should be atomic. However, you could also update this document based on the result of another document's attribute without rethink throwing an error, contrary to the docs. I'm having trouble finding a definitive answer for you right now... – Launalaunce

Just tried the command with get(2) inside and indeed rethinkdb does not throw an error but it goes contrary to the docs... – Herbartian

@sfireman Based on some experimentation, it's not clear at all if this is actually atomic. I've created a Github issue on Rethinkdb to address this. When I hear back from them I will update the issue as appropriate :) – Launalaunce

I see here 2 operations: get and insert. And there are not atomic. Am I wrong? – Boaz

Unfortunately this solution isn't atomic because of the subquery. The documentation in the FAQ is misleading in this respect. I'm going to open an issue so we can improve it. Sorry for the confusion! – Arst

@sfireman I've updated my answer to be actually atomic based on Daniel's answer on Github. Sorry about being initially misleading! – Launalaunce

Thanks for the help! Using replace is undesirable as I will most likely have to stick to fixed data structure within the document to be able to reuse old fields... but at least it will be atomic :) – Herbartian

@sfireman you can use a merge inside the replace to emulate an update. E.g.: table.get(...).replace(r.branch(r.row.eq(null), {id: ..., counter: 0}, r.row.merge({counter: r.row('counter').add(1)}))) – Arst

Daniel, just checked using branch results in "Could not prove function deterministic. Maybe you want to use the non_atomic flag?"... So there is currently no way to atomically upsert dynamic documents – Herbartian