Objectify paging with Cursors

Question

I have this method in my RPC service:

@Override
public Entrata[] getEntrate(int from, int to) {
    List<Entrata> data = entrateDao.list();
    return data.toArray(new Entrata[0]);
}

As you can see, I am not using the two parameters, which, in a SQL world, I would use as LIMIT and OFFSET.

It's not completely clear what I have to do now, I started reading this: http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Cursors

I think I have to do a query.startCursor(<my_"from"_parameter>)

Then iterate for "TO" times, the page size.

All right? Can you help me with some snippets? :)

Stockjobber

Answer 1

From docs: Cursors let you take a "checkpoint" in a query result set, store the checkpoint elsewhere, and then resume from where you left off late

As you need just limit/offset, you have to use limit() and offset() method of Objectify Query. Like:

ob.query(Entrata.class).limit(to - from).offset(from)

Or, when you have cursor:

String cursor = // get it from request
Query<Entrata> query = ob.query(Entrata.class);
Query q = query.startCursor(Cursor.fromWebSafeString(cursor));
q.limit(x);
QueryResultIterator<Entrate> iterator = query.iterator()
List<Entrate> data = // fetch data
String newCursor = iterrator.getStartCursor().toWebSafeString()
return new EntrataListWithCursor(data, cursor);

Diley

Not efficient, sorry, I already know that https://mcmap.net/q/1173776/-objectify-paging-closed I'm trying to build an app which can handle bilions of record not a few :-\ – Stockjobber

If you know cursors, is there a way to have a limit-offset behaviour with them? – Stockjobber

You can just start from cursor (cursor is just a serialized position), if you already have one. How you planning to use this method? – Diley

What you mean "already have one" ? So, let's suppose my page-size is 15. The first request, the cursor is null, so I iterate the first 15 elements of my dataset. Then I grab the cursor and I save it somewhere, e.g. the info for the client. The next time I read that cursor, right? – Stockjobber

ok, so the first time I will perform the "complete" query and iterate only 15 times, then save the cursor and get it back with the data to the client. The second time... something similar to your code. Thank you :) I'm going to try it soon! – Stockjobber

Answer 2

I just want make sure you don't have any errors in your code since you can copy and past the Igor Artamonov code. Here is a cleaner code from Objectify Wiki with less errors and some documentation:

// create the query and set the limit to 1000
Query<Car> query = ofy().load().type(Car.class).limit(1000);

// Here you get the cursor (if exists) from the request
// For the first request, i-e the first page, this parameter(cursor) will be null
String cursorStr = request.getParameter("cursor");

// Here you check if cursor is not null and not empty
// If so, we start our query from the last check point
if (cursorStr != null && !cursorStr.isEmpty())
    query = query.startAt(Cursor.fromWebSafeString(cursorStr));

// We need this variable to know when we have been loaded all the entries
boolean remaining = false;
QueryResultIterator<Car> iterator = query.iterator();
while (iterator.hasNext()) {
    Car car = iterator.next();

    ... // your code here

    // We have found entries, so we set this variable to true.
    // That means, we have probably another page to fetch
    remaining = true;
}

// If we have found entries, we send the last check point
if (remaining) {
    // we take the last check point by calling "toWebSafeString()" from the iterator's cursor
    Cursor cursor = iterator.getCursor();
    Queue queue = QueueFactory.getDefaultQueue();
    queue.add(url("/pathToThisServlet").param("cursor", cursor.toWebSafeString()));
}

Gonococcus