Bigtable node client: How to set read offset for pagination?

Question

I'm using createReadStream to read rows from Bigtable. I want to do some pagination with the results so limit and offset are necessary. There's a limit option so that's great. However, I can't find a way to set offset. How should I go about it?

Gladygladys

Answer 1

Using https://cloud.google.com/nodejs/docs/reference/bigtable/0.13.x/Table#createReadStream start is an option. As you retrieve the first page. you can use the last row key to set the start for the next page (will need to exclude the first result for all page counts > 1)

Bryology

Hmm, that's one way to do it. But then I'd need page n-1 to get page n. It's not possible to get an arbitrary page in the middle, which I think is a poor experience for users. – Gladygladys

Bigtable doesn't have a row offset index, it only has a single primary row key index. If this functionality is important to you, you can build the secondary index on top of Bigtable that maps row positions to keys. – Tyner

@IgorBernstein, could you please expand a bit on what you commented and post an answer explaining how to build the secondary index you suggest? Thank you. – Verbality

I simply meant that the OP can create a separate table to track offsets of rows. In the simplest case the application code can store the page number as the key and the starting key as the value. I don't think I can expand much on this without knowing the schema, the data distribution or the usage patterns (is it append only? random insert?) – Tyner