Pagination in Cosmos DB using Page Size and Page Number
Asked Answered
G

4

18

I am trying to return items from cosmosDB using PageSize and PageNumber. I know we can set the page size in MaxItemCount, but how do we put the page number in this function?

Here's what I got so far:

  public async Task<IEnumerable<T>> RunSQLQueryAsync(string queryString, int pageSize, int pageNumber)
        {
            var feedOptions = new FeedOptions { MaxItemCount = pageSize, EnableCrossPartitionQuery = true };
            IQueryable<T> filter = _client.CreateDocumentQuery<T>(_collectionUri, queryString, feedOptions);
            IDocumentQuery<T> query = filter.AsDocumentQuery();
            var currentPageNumber = 0;
            var documentNumber = 0;
            List<T> results = new List<T>();
            while (query.HasMoreResults)
            {
                foreach (T t in await query.ExecuteNextAsync())
                {
                    results.Add(t);
                    documentNumber++;
                }
                currentPageNumber++;
                return results;

            }
            return null;
        }
Goddaughter answered 4/7, 2018 at 0:34 Comment(0)
G
23

Currently, the pagination support is based on continuation token only.

Find below some interesting discussion and feature request about this limitation:


--- Continuation Token Example ---

The following example illustrates a method (very similar to yours) that queries documents based on the desired page number, page size and continuation token:

    private static async Task<KeyValuePair<string, IEnumerable<CeleryTask>>> QueryDocumentsByPage(int pageNumber, int pageSize, string continuationToken)
    {
        DocumentClient documentClient = new DocumentClient(new Uri("https://{CosmosDB/SQL Account Name}.documents.azure.com:443/"), "{CosmosDB/SQL Account Key}");

        var feedOptions = new FeedOptions {
            MaxItemCount = pageSize,
            EnableCrossPartitionQuery = true,

            // IMPORTANT: Set the continuation token (NULL for the first ever request/page)
            RequestContinuation = continuationToken 
        };

        IQueryable<CeleryTask> filter = documentClient.CreateDocumentQuery<CeleryTask>("dbs/{Database Name}/colls/{Collection Name}", feedOptions);
        IDocumentQuery<CeleryTask> query = filter.AsDocumentQuery();

        FeedResponse<CeleryTask> feedRespose = await query.ExecuteNextAsync<CeleryTask>();

        List<CeleryTask> documents = new List<CeleryTask>();
        foreach (CeleryTask t in feedRespose)
        {
            documents.Add(t);
        }

        // IMPORTANT: Ensure the continuation token is kept for the next requests
        return new KeyValuePair<string, IEnumerable<CeleryTask>>(feedRespose.ResponseContinuation, documents);
    }

Now, the following example illustrates how to retrieve documents for a given page by calling the previous method:

    private static async Task QueryPageByPage()
    {
        // Number of documents per page
        const int PAGE_SIZE = 3;

        int currentPageNumber = 1;
        int documentNumber = 1;

        // Continuation token for subsequent queries (NULL for the very first request/page)
        string continuationToken = null;

        do
        {
            Console.WriteLine($"----- PAGE {currentPageNumber} -----");

            // Loads ALL documents for the current page
            KeyValuePair<string, IEnumerable<CeleryTask>> currentPage = await QueryDocumentsByPage(currentPageNumber, PAGE_SIZE, continuationToken);

            foreach (CeleryTask celeryTask in currentPage.Value)
            {
                Console.WriteLine($"[{documentNumber}] {celeryTask.Id}");
                documentNumber++;
            }

            // Ensure the continuation token is kept for the next page query execution
            continuationToken = currentPage.Key;
            currentPageNumber++;
        } while (continuationToken != null);

        Console.WriteLine("\n--- END: Finished Querying ALL Dcuments ---");
    }
Galatians answered 4/7, 2018 at 1:12 Comment(9)
Thanks for the sample code. I am not sure why, but when I set the page size to be 2, and the total items in cosmosDB is 5. For the first page, it returns 2 items - which is fine; then the next page, it returns 1 item for some weird reason; the 3d page, it returns 1 item again.. Any idea why? Thanks again!Goddaughter
{"token":null,"range":{"min":"05C1C9CD673390","max":"05C1D399CD672C"}} - it shows the token is null after the line continuationToken = currentPage.Key;Goddaughter
When the continuation token is null, it means there is no more documents available to read based on the query. I'll update the sample code to reflect that.Galatians
Shouldn't this skip if it's not on the correct page?Alla
How to go to Previous Page ?Lavinia
There is a limitation with using continuation tokens that they cannot be with cross partition queries. The query runs but header is not returned. github.com/Azure/azure-cosmosdb-node/issues/….Intrigue
I have added Order by, getting this token : [{\"compositeToken\":{\"token\":\"+RID:~WllDAJf2Rs4EAgAAAAAAAA==#RT:1#TRC:20#RTD:nReGcfWnOHBG6GZpH1ceBUJtY2pvAA==#ISV:2#IEO:65567#QCF:3#FPC:AgEAAAAqAE+AZADg7wEQEPj/AVEA8N8SQP+//v8RQLX/QUD3/xFA/98RQP7/EUD/Bw==\",\"range\":{\"min\":\"\",\"max\":\"FF\"}},\"orderByItems\":[{\"item\":\"Albin\"}],\"rid\":\"WllDAJf2Rs4EAgAAAAAAAA==\",\"skipCount\":0,\"filter\":\"true\"}] But when unable to get next page getting error as " Reason: (ParallelContinuationToken is missing field: 'token': {\"compositeToken\":{\"token\":\"Sexology
Where does the pageNumber come into play in QueryDocumentsByPage?Inconsonant
Chech this out: OFFSET LIMIT clause in Azure Cosmos DB (posted on 10/12/2022)Niu
P
22

Skip & take is now available in Cosmos DB via a new OFFSET LIMIT clause: https://learn.microsoft.com/en-us/azure/cosmos-db/sql-query-offset-limit

Pretorius answered 13/5, 2019 at 12:49 Comment(3)
This doesn’t seem to be in the linked doc... try here insteadSwinton
Thanks, @Matt. Microsoft apparently moved the documentation.Pretorius
The offset-limit loads everything before applying pagination which defies the very purpose of pagination. We paginate to reduce loading time by not loading everything at once.Croatia
L
1

There is a workaround, but it is not optimal:

...
int pageNumber = 1;
int pageSize = 10;
...
    var query = Client.CreateDocumentQuery<T>(
                    UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
                    new FeedOptions { MaxItemCount = pageNumber * pageSize }) // Limit count or returned items
                    .Where(predicate)
                    .AsDocumentQuery();

                var results = new List<T>();
                int resultsToSkip = (pageNumber - 1) * pageSize;

                while (query.HasMoreResults)
                {
                    var result = await query.ExecuteNextAsync<T>();

                    // Skip pages, not optimal way, the best solution is to use RequestContinuation token.
                    if (resultsToSkip > 0)
                    {
                        resultsToSkip--;
                        continue;
                    }

                    results.AddRange(result);
                }

                return results;
    ...
Longevous answered 28/5, 2019 at 13:18 Comment(0)
T
0

public static List pagination(int pageNo=1,int pageSize=20) { List ArticlesList = new List(); var collection = UriFactory.CreateDocumentCollectionUri(databaseName, loginCollectionId);

    using (client = new DocumentClient(new Uri(endpointUrl), primaryKey))
    {

        var optionss = new FeedOptions
        {
            MaxItemCount = (pageNo!=1)?((pageNo-1)*pageSize): ((pageNo) * pageSize)
        };

        var query1 = client.CreateDocumentQuery<ArticlesListDetails>(collection, optionss).OrderByDescending(x => x._ts).AsDocumentQuery();

        var res = query1.ExecuteNextAsync<ArticlesListDetails>().Result;
        if (pageNo == 1)
        {
            return   ArticlesList = res.ToList();
        }
        else
        {
            var options = new FeedOptions
            {
                MaxItemCount = pageSize,
                RequestContinuation = res.ResponseContinuation
            };

            var query = client.CreateDocumentQuery<ArticlesListDetails>(collection, options).OrderByDescending(x => x._ts).AsDocumentQuery();

            while (query.HasMoreResults)
            {
                return  ArticlesList = query.ExecuteNextAsync<ArticlesListDetails>().Result.ToList();
            }
        }

        return ArticlesList;
    }

}
Terriss answered 18/3, 2019 at 12:16 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.