How to flip to previous page with ndb cursors?

Question

I cant manage to get to 'previous page' in ndb paging.

I have checked the documentation and also this similar question here without success.

 def show_feedback(kind, bookmark=None):
    """Renders returned feedback."""
    cursor = None    
    more_p= None
    if bookmark:
        cursor = Cursor(urlsafe=bookmark)

    q = Feedback.query()
    q_forward = q.filter(Feedback.kind==Feedback.KINDS[kind]).order(-Feedback.pub_date)
    q_reverse = q.filter(Feedback.kind==Feedback.KINDS[kind]).order(Feedback.pub_date)

    feedbacks, next_cursor, more = q_forward.fetch_page(app.config['FEEDBACK_PER_PAGE'], start_cursor=cursor)
    if cursor:
        rev_cursor = cursor.reversed()
        feedbacks2, prev_cursor, more_p = q_reverse.fetch_page(app.config['FEEDBACK_PER_PAGE'], start_cursor=rev_cursor)

    next_bookmark = None
    prev_bookmark = None
    if more and next_cursor:
        next_bookmark = next_cursor.urlsafe()
    if more_p and prev_cursor:
        prev_bookmark = prev_cursor.urlsafe()
    return render_template_f11('show_feedback.html', kind=kind, feedbacks=feedbacks, next_bookmark=next_bookmark, prev_bookmark=prev_bookmark)

html:

  {% if prev_bookmark %}
        <a href="{{ url_for(request.endpoint, bookmark=prev_bookmark) }}">Previous</a>
  {% endif %}
  {% if next_bookmark %}
    <a href="{{ url_for(request.endpoint, bookmark=next_bookmark) }}">Next</a>
  {% endif %}

I can page forwards correctly until the end. But I can't page backwards until the last page and even then I can't page back until the first page neither.

What am I missing please?

UPDATE:

Changed code with Faisal's suggestions. It works better I must admit. But still the paging doesn't work correctly:

I have 7 entries. PAGE_SIZE in config is 3. Hence we get three pages:

When clicking on Next I get 7,6,5 -> 4,3,2 -> 1 Perfect. Now when clicking on previous: 1 -> 3,4,5 (?) -> 5,6,7 (?)

Thanks for your help

def show_feedback(kind, bookmark=None):
    """Renders returned feedback."""
    is_prev = request.args.get('prev', False)
    cursor = None        
    if bookmark:
        cursor = Cursor(urlsafe=bookmark)

    q = Feedback.query()
    q_forward = q.filter(Feedback.kind==Feedback.KINDS[kind]).order(-Feedback.pub_date)
    q_reverse = q.filter(Feedback.kind==Feedback.KINDS[kind]).order(Feedback.pub_date)

    qry = q_reverse if is_prev else q_forward

    feedbacks, cursor, more = qry.fetch_page(app.config['FEEDBACK_PER_PAGE'], start_cursor=cursor)

    if is_prev:
        prev_bookmark = cursor.reversed().urlsafe() if more else None
        next_bookmark = bookmark
    else:
        prev_bookmark = bookmark
        next_bookmark = cursor.urlsafe() if more else None
    return render_template_f11('show_feedback.html', kind=kind, feedbacks=feedbacks, next_bookmark=next_bookmark, prev_bookmark=prev_bookmark)

UPDATE 2:

It seems now its nearly working with reverse().

7,6,5 -> next -> 4,3,2 -> next -> 1

1 -> prev -> 2,3,4 -> 5,6,7 (Order is no longr latest date first)

Dana

Did you get the ordering issue figured out? I'm also doing date ordering and the pattern in the docs mess that up. – Morph

Answer 1

So what I do here is use the current bookmark to for navigating for next or previous and removed the other query so it doesn't query twice for each request. (Edited the old description/answer was wrong when I tested it. This one works on my localhost).

Try:

is_prev = self.request.get('prev', False)
if is_prev:
    qry = q_reverse
    cursor = cursor.reversed()
else:
    qry = q_forward

feedbacks, cursor, more = qry.fetch_page(app.config['FEEDBACK_PER_PAGE'], start_cursor=cursor)

if is_prev:
    prev_bookmark = cursor.reversed().urlsafe() if more else None
    next_bookmark = bookmark
else:
    prev_bookmark = bookmark
    next_bookmark = cursor.urlsafe() if more else None

html

{% if prev_bookmark %}
    <a href="{{ url_for(request.endpoint, bookmark=prev_bookmark, prev=True) }}">Previous</a>
{% endif %}
{% if next_bookmark %}
  <a href="{{ url_for(request.endpoint, bookmark=next_bookmark) }}">Next</a>
{% endif %}

Openandshut

Faisal, thank you so much for your help on this. I have now posted the updated code and the result. It works better than before, however the previous steps are still not correct. So strange... – Dana

updated the answer, I added the cursor.reversed() when is_prev I think that's what I missed. – Openandshut

Yes Faisal, that nearly solved the problem fully. Quite complicated. But the order is still not fully correct. Please see the updated question. Thank you – Dana

you're right, just tested it. I would think that cursor.reversed() would handle the result in reverse order as well. Right now the only thing I can think of is just if is_prev: feedbacks = reversed(feedbacks), not sure if its intended or could be a bug or maybe for us to handle since we could be representing/showing data in different ways like ajax load previous or paginated like right now. – Openandshut

You might be right. It would be interesting to know if this is a bug or its intended. But now it works. I think this was the most complicated paging in history of writing code. :) +1 many thanks. I don't think I could have solved this by myself. – Dana

Answer 2

Here you have a complete working solution. There's something wrong in your code.

They key thing is to reverse the results when going backwards. It's tricky.

Here you have:

def return_query_page(query_class, size=10, bookmark=None, is_prev=None, equality_filters=None, orders=None):
    """
    Generate a paginated result on any class
    Param query_class: The ndb model class to query
    Param size: The size of the results
    Param bokkmark: The urlsafe cursor of the previous queries. First time will be None
    Param is_prev: If your requesting for a next result or the previous ones
    Param equal_filters: a dictionary of {'property': value} to apply equality filters only
    Param orders: a dictionary of {'property': '-' or ''} to order the results like .order(cls.property)
    Return: a tuple (list of results, Previous cursor bookmark, Next cursor bookmark)
    """
    if bookmark:
        cursor = ndb.Cursor(urlsafe=bookmark)
    else:
        is_prev = None
        cursor = None

    q = query_class.query()
    try:
        for prop, value in equality_filters.iteritems():
            q = q.filter(getattr(query_class, prop) == value)

        q_forward = q.filter()
        q_reverse = q.filter()

        for prop, value in orders.iteritems():
            if value == '-':
                q_forward = q_forward.order(-getattr(query_class, prop))
                q_reverse = q_reverse.order(getattr(query_class, prop))
            else:
                q_forward = q_forward.order(getattr(query_class, prop))
                q_reverse = q_reverse.order(-getattr(query_class, prop))
    except:
        return None, None, None
    if is_prev:
        qry = q_reverse
        new_cursor = cursor.reversed() if cursor else None
    else:
        qry = q_forward
        new_cursor = cursor if cursor else None

    results, new_cursor, more = qry.fetch_page(size, start_cursor=new_cursor)
    if more and new_cursor:
        more = True
    else:
        more = False

    if is_prev:
        prev_bookmark = new_cursor.reversed().urlsafe() if more else None
        next_bookmark = bookmark
        results.reverse()
    else:
        prev_bookmark = bookmark
        next_bookmark = new_cursor.urlsafe() if more else None

    return results, prev_bookmark, next_bookmark

This is the link to the github project: https://github.com/janscas/ndb-gae-pagination

Hotien