How to paginate react-admin lists when the total is unknown?

Question

Summary: I can't get a total number of records from my GraphQL endpoint. I only know if I have reached the end of my GraphQL records list when I am parsing the response from the endpoint. How can I make my custom pagination component aware that it's on the last page?

Details: I'm using React Admin with AWS AppSync (GraphQL on DynamoDB) using ra-data-graphql. AppSync can't tell you the total number of records available to a list query, and it also limits the number of records you can return to a 1MB payload. Instead, it includes a nextToken value if there are more records to query, which you can include in subsequent list queries.

I have created a custom pagination component that only uses "prev" and "next" links, which is fine. But I need to know when the last page is displayed. Right now, I only know this in the parseResponse() function that I'm passing in to buildQuery() for the list query. At this point, I have access to the nextToken value. If it's empty, then I have fetched the last page of results from AppSync. If I could pass this value, or even a boolean e.g. lastPage to the custom pagination component, I'd be all set. How can I do this in React Admin?

Nudnik

Looking at the docs, using customReducers might be the way to go. – Nudnik

Answer 1

There is also a way to adapt AppSync resolver to work with page and perPage native react-admin parameters.

It's a bad practice because query response is limited by 1MB and also full dynamodb query response needs to be parsed and transformed for each page query, however it does the trick.

VTL AppSync resolver Request Mapping Template:

{
    "version" : "2017-02-28",
    "operation" : "Query",
    "query" : {
        "expression": "userId = :userId",
        "expressionValues" : {
            ":userId" : $util.dynamodb.toDynamoDBJson($context.identity.sub)
        }
    }
}

VTL AppSync resolver Response Mapping Template:

#set($result = {})
#set($result.items = [])
#set($result.length = $ctx.result.items.size())
#set($start = $ctx.arguments.perPage * ($ctx.arguments.page - 1))
#set($end = $ctx.arguments.perPage * $ctx.arguments.page - 1)
#if($end > $result.length - 1)
 #set($end = $result.length - 1)
#end

#if($start <= $result.length - 1 && $start >= 0 )
  #set($range = [$start..$end])
  #foreach($i in $range)
     $util.qr($result.items.add($ctx.result.items[$i]))
  #end
#end 

$util.toJson($result)

dataProvider.js

...
const buildQuery = () => (
  raFetchType,
  resourceName,
  params
) => {
  if (resourceName === "getUserToDos" && raFetchType === "GET_LIST") {
    return {
      query: gql`
        query getUserToDos($perPage: Int!, $page: Int!) {
          getUserToDos(perPage: $perPage, page: $page) {
            length
            items {
              todoId
              date
              ...
            }
          }
        }
      `,
      variables: {
        page: params.pagination.page,
        perPage: params.pagination.perPage
      },
      parseResponse: ({ data }) => {
        return {
          data: data.getUserToDos.items.map(item => {
            return { id: item.listingId, ...item };
          }),
          total: data.getUserToDos.length
        };
      }
    };
  }
...

Pyrochemical

I changed the accepted answer to this. The solution I posted was awkward and yielded unpredictable results when moving from create/edit back to list. That said, I opted to not paginate at all, as the number of records was not too great and the RA filters provide an adequate UX. – Nudnik

Answer 2

To achieve this I created a custom reducer, nextTokenReducer that looks for React Admin's CRUD_GET_LIST_SUCCESS action, the payload of which is the entire response from the AppSync GraphQL endpoint. I can pull the nextToken value out of that:

import { CRUD_GET_LIST_SUCCESS } from "react-admin";

export default (previousState = null, { type, payload }) => {
  if (type === CRUD_GET_LIST_SUCCESS) {
    return payload.nextToken;
  }
  return previousState;
};

I passed this custom reducer to the Admin component in my main App component:

import nextTokenReducer from "./reducers/nextTokenReducer";
...
class App extends Component {
...
  render() {
    const { dataProvider } = this.state;

    if (!dataProvider) {
      return <div>Loading</div>;
    }

    return (
      <Admin
        customReducers={{ nextToken: nextTokenReducer }}
        dataProvider={dataProvider}
      >
        <Resource name="packs" list={PackList} />
      </Admin>
    );
  }
}

I then connected the nextToken store to my custom pagination component. It will display "next", "prev", or nothing based on whether nextToken is in its props:

import React from "react";
import Button from "@material-ui/core/Button";
import ChevronLeft from "@material-ui/icons/ChevronLeft";
import ChevronRight from "@material-ui/icons/ChevronRight";
import Toolbar from "@material-ui/core/Toolbar";

import { connect } from "react-redux";

class CustomPagination extends React.Component {
  render() {
    if (this.props.page === 1 && !this.props.nextToken) {
      return null;
    }
    return (
      <Toolbar>
        {this.props.page > 1 && (
          <Button
            color="primary"
            key="prev"
            icon={<ChevronLeft />}
            onClick={() => this.props.setPage(this.props.page - 1)}
          >
            Prev
          </Button>
        )}
        {this.props.nextToken && (
          <Button
            color="primary"
            key="next"
            icon={<ChevronRight />}
            onClick={() => this.props.setPage(this.props.page + 1)}
            labelposition="before"
          >
            Next
          </Button>
        )}
      </Toolbar>
    );
  }
}

const mapStateToProps = state => ({ nextToken: state.nextToken });

export default connect(mapStateToProps)(CustomPagination);

Finally, I passed the custom pagination component into my list component:

import React from "react";
import { List, Datagrid, DateField, TextField, EditButton } from "react-admin";
import CustomPagination from "./pagination";

export const PackList = props => (
  <List {...props} pagination={<CustomPagination />}>
    <Datagrid>
    ...
    </Datagrid>
  </List>
);

Nudnik

How can I increase the query limit on a resource? – Oxidation

@Berni, you can't. "A single Query operation can retrieve a maximum of 1 MB of data. This limit applies before any FilterExpression is applied to the results. If LastEvaluatedKey is present in the response and is non-null, you will need to paginate the result set (see Paginating the Results)." – Nudnik

This is the best answer for appsync/dynamodb/amplify. messing with VTL, scanning, trying to do what dynamodb wasn't built to do, is smell, IMO. Cheers for the great solution. – Sphenic