How to use (opaque) cursors in GraphQL / Relay when using filter arguments and order by
Asked Answered
I

3

5

Imagine the following GraphQL request:

{
  books(
    first:10,
    filter: [{field: TITLE, contains: "Potter"}],
    orderBy: [{sort: PRICE, direction: DESC}, {sort: TITLE}]
  )
}

The result will return a connection with the Relay cursor information.

Should the cursor contain the filter and orderBy details?

Meaning querying the next set of data would only mean:

{
  books(first:10, after:"opaque-cursor")
}

Or should the filter and orderBy be repeated?

In the latter case the user can specify different filter and/or orderBy details which would make the opaque cursor invalid.

I can't find anything in the Relay spec about this.

Inexistent answered 20/8, 2018 at 9:7 Comment(0)
C
4

I've seen this done multiple ways, but I've found that with cursor-based pagination, your cursor exists only within your dataset, and to change the filters would change the dataset, making it invalid.

If you're using SQL (or something without cursor-based-pagination), then, you would need to include enough information in your cursor to be able to recover it. Your cursor would need to include all of your filter / order information, and you would need to disallow any additional filtering.

You'd have to throw an error if they sent "after" along with "filter / orderBy". You could, optionally, check to see if the arguments are the same as the ones in your cursor, in case of user error, but there simply is no use-case to get "page 2" of a DIFFERENT set of data.

Churchman answered 21/8, 2018 at 0:26 Comment(0)
E
2

I came across the same question / problem, and came to the same conclusion as @Dan Crews. The cursor must contain everything you need to execute the database query, except for LIMIT.

When your initial query is something like

SELECT *
FROM DataTable
WHERE filterField = 42
ORDER BY sortingField,ASC
LIMIT 10
-- with implicit OFFSET 0

then you could basically (don't do this in a real app, because of SQL Injections!) use exactly this query as your cursor. You just have to remove LIMIT x and append OFFSET y for every node.

Response:

{
  edges: [
    {
      cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 0",
      node: { ... }
    },
    {
      cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 1",
      node: { ... }
    },
    ...,
    {
      cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 9",
      node: { ... }
    }
  ]
  pageInfo: {
    startCursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 0"
    endCursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 9"
  }
}

The next request will then use after: CURSOR, first: 10. Then you'll take the after argument and set the LIMIT and OFFSET:

  • LIMIT = first
  • OFFSET = OFFSET + 1

Then the resulting database query would be this when using after = endCursor:

SELECT *
FROM DataTable
WHERE filterField = 42
ORDER BY sortingField,ASC
LIMIT 10
OFFSET 10

As already mentioned above: This is only an example, and it's highly vulnerable to SQL Injections!


In a real world app, you could simply encode the provided filter and orderBy arguments within the cursor, and add offset as well:

function handleGraphQLRequest(first, after, filter, orderBy) {
  let offset = 0; // initial offset, if after isn't provided

  if(after != null) {
    // combination of after + filter/orderBy is not allowed!
    if(filter != null || orderBy != null) {
      throw new Error("You can't combine after with filter and/or orderBy");
    }

    // parse filter, orderBy, offset from after cursor
    cursorData = fromBase64String(after);
    filter = cursorData.filter;
    orderBy = cursorData.orderBy;
    offset = cursorData.offset;
  }

  const databaseResult = executeDatabaseQuery(
    filter,  // = WHERE ...
    orderBy, // = ORDER BY ...
    first,   // = LIMIT ...
    offset   // = OFFSET ...
  );

  const edges = []; // this is the resulting edges array
  let currentOffset = offset; // this is used to calc the offset for each node
  for(let node of databaseResult.nodes) { // iterate over the database results
    currentOffset++;
    const currentCursor = createCursorForNode(filter, orderBy, currentOffset);
    edges.push({
      cursor = currentCursor,
      node = node
    });
  }

  return {
    edges: edges,
    pageInfo: buildPageInfo(edges, totalCount, offset) // instead of
        // of providing totalCount, you could also fetch (limit+1) from
        // database to check if there is a next page available
  }
}

// this function returns the cursor string
function createCursorForNode(filter, orderBy, offset) {
  return toBase64String({
    filter: filter,
    orderBy: orderBy,
    offset: offset
  });
}

// function to build pageInfo object
function buildPageInfo(edges, totalCount, offset) {
  return {
    startCursor: edges.length ? edges[0].cursor : null,
    endCursor: edges.length ? edges[edges.length - 1].cursor : null,
    hasPreviousPage: offset > 0 && totalCount > 0,
    hasNextPage: offset + edges.length < totalCount
  }
}

The content of cursor depends mainly on your database and you database layout.

The code above emulates a simple pagination with limit and offset. But you could (if supported by your database) of course use something else.

Elinoreeliot answered 9/2, 2019 at 16:50 Comment(0)
E
2

In the meantime I came to another conclusion: I think it doesn't really matter whether you use an all-in-one cursor, or if you repeat filter and orderBy with each request.

There are basically two types of cursors:

(1.) You can treat a cursor as a "pointer to a specific item". This way the filter and sorting can change, but your cursor can stay the same. Kinda like the pivot element in quicksort, where the pivot element stays in place and everything around it can move.

Elasticsearch's Search After works like this. Here the cursor is just a pointer to a specific item in the dataset. But filter and orderBy can change independently.

The implementation for this style of cursor is dead simple: Just concat every sortable field. Done. Example: If your entity can be sorted by price and title (plus of course id, because you need some unique field as tie breaker), your cursor always consists of { id, price, title }.

(2.) The "all-in-one cursor" on the other hand acts like a "pointer to an item within a filtered and sorted result set". It has the benefit, that you can encode whatever you want. The server could for example change the filter and orderBy data (for whatever reason) without the client noticing it.

For example you could use Elasticsearch's Scroll API, which caches the result set on the server and though doesn't need filter and orderBy after the initial search request.

But aside from Elasticsearch's Scroll API, you always need filter, orderBy, limit, pointer in every request. Though I think it's an implementation detail and a matter of taste, whether you include everything within your cursor, or if you send it as separate arguments. The outcome is the same.

Elinoreeliot answered 14/10, 2020 at 12:36 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.