Using @ndb.tasklet or @ndb.synctasklet in Google App Engine

Question

I have a POST method which calls a few tasklets. These tasklets do have yields in them, and I do have some x.put_async() in my code. So I don't want it to return before all the async stuff is done. So I decorated all my tasklets, which are just small functions with @ndb.tasklet. Also, on top of my POST method, I have:

@ndb.toplevel
def post(self):

However, in the documentation it states:

But if a handler method uses yield, that method still needs to be wrapped in another decorator, @ndb.synctasklet; otherwise, it will stop executing at the yield and not finish.

Indeed my method has a yield. It's already wrapped in @ndb.tasklet. Do I replace this with @ndb.synctasklet or do I use both (if so how would I use both)?

Also, see this thread which has some relevance. I too noticed an issue where my request would return without any output, but is un-reproducible. It happens every 15 minutes or so of constant use. I had app = ndb.toplevel(webapp2.WSGIApplication([..]) only, but now I've added @ndb.toplevel to the main POST methods, but the issue still persists.

Should I put @ndb.tasklet on top of methods that have just put_async()'s too? (Should I put it on top of every method just to be safe? What are the downsides to this?)

Grenada

Answer 1

Regarding the handler and using @ndb.toplevel and @ndb.synctasklet: The way I understood it was that you need to use both @ndb.synctasklet and @ndb.toplevel on the handler. All the sub-tasklets only need the @ndb.tasklet decorator. e.g.

class Foo(ndb.Model):
    name = ndb.StringProperty()

    @ndb.tasklet
    def my_async(self):
        ....
        #do something else that yields
        raise ndb.Return("some result")   


@ndb.toplevel
@ndb.synctasklet
def post(self):
    foo = Foo(name="baz")
    yield foo.put_async()
    yield foo.my_async()
    ....

However. looking at the source, it appears that @ndb.toplevel is actually a synctasklet anyway:

def toplevel(func):
  """A sync tasklet that sets a fresh default Context.

  Use this for toplevel view functions such as
  webapp.RequestHandler.get() or Django view functions.
  """

Running a small test with yields in the handler and decorated with @ndb.toplevel still seems to work, and appears that you can remove @ndb.synctasklet from the handler.

Regarding whether you should include @ndb.tasklet on methods that call put_async(): If you're not yielding on the put_async(), then you don't need to include @ndb.tasklet on the surrounding method (@ndb.toplevel will handle getting the results from the put_async())

Unwinking

Good reply! The rules could be summarized as follows: (1) if a function uses "yield" it should ndb.tasklet, ndb.synctasklet, or ndb.toplevel. (2) a function wrapped in ndb.tasklet returns a Future (and you can either yield it or call get_result() explicitly). (3) ndb.synctasklet is like wrapping it in ndb.tasklet but implicitly calling get_result(). (4) ndb.toplevel is just like ndb.synctasklet but also waits for all pending operations to complete. – Rhodian