Google Datastore queries and eventual consistency
Asked Answered
H

1

7

I would like to confirm my understanding of eventual consistency in the Google datastore. Suppose that I have an entity defined as follows (using ndb):

class Record(ndb.Model):
    name = ndb.StringProperty()
    content = ndb.BlobProperty()

I think I understand Scenarios 1, but I have doubts about Scenarios 2 and 3, so some advice would be highly appreciated.

Scenario 1: I insert a new Record with name "Luca" and a given content. Then, I query the datastore:

qry = Record.query(name=="Luca")
for r in qry.iter():
    logger.info("I got this content: %r" % r.content)

I understand that, due to eventual consistency, the just-inserted record might not be part of the result set. I know about using ancestor queries in order to over come this if needed.

Scenario 2: I read an existing Record with name "Luca", update the content, and write it back. For instance, assuming I have the key "k" of this record:

r = k.get()
r.content = "new content"
r.put()

Then, I run the same query as in Scenario 1. When I get the results, assume that the record is part of the result set (for instance, because the index already contained the record with name "Luca" and key k). Am I then guaranteed that the field content will have its new value "new content"? In other words, if I update a record, leaving its key and indexed fields alone, am I guaranteed to read the most recent value?

Scenario 3: I do similarly to Scenario 2, again where k is the key of a record with name "Luca":

r = k.get()
r.content = "new content"
r.put()

but then I run a modified version of the query:

qry = Record.query(name=="Luca")
for k in qry.iter(keys_only=True):
    r = k.get()
    logger.info("I got this content: %r" % r.content)

In this case, logic tells me I should be getting the latest value of the content, because reading by key guarantees strong consistency. I would appreciate confirmation.

Humpbacked answered 9/11, 2013 at 22:47 Comment(0)
P
9

Scenario 1. Yes, your understanding is correct.

Scenario 2. No, same query, so still eventually consistent.

Scenario 3. Yes, your understanding is correct.

Also you can avoid eventual consistency by doing everything in the same transaction, but of course this may not be applicable.

Prudenceprudent answered 9/11, 2013 at 22:52 Comment(2)
Aha, thanks, very interesting. So there is a difference between doing a query normally, and doing it with keys_only=True and then reading from the key.Humpbacked
yes, going over the key should be consistent... however, i had problems with the rollback of transactions... (if something fails in the commit statement) its not always working (with JDO - Java)... i think i am not doing it wrong .. but think with native google code it should work!Precede

© 2022 - 2024 — McMap. All rights reserved.