Pagination with mongoTemplate

Question

I have a Query with Pageable:

Query query = new Query().with(new PageRequests(page, size))

How can I execute it with MongoTemplate ? I don't see a single method returning Page<T>.

Pivot

Answer 1

It's true that the MongoTemplate doesn't have findXXX with Pageables.

But you can use the Spring Repository PageableExecutionUtils for that.

In your example it would look like this:

Pageable pageable = new PageRequests(page, size);
Query query = new Query().with(pageable);
List<XXX> list = mongoTemplate.find(query, XXX.class);
return PageableExecutionUtils.getPage(
                       list, 
                       pageable, 
                       () -> mongoTemplate.count(Query.of(query).limit(-1).skip(-1), XXX.class));

Like in the original Spring Data Repository, the PageableExecutionUtils will do a count request and wrap it into a nice Page for you.

Here you can see that spring is doing the same.

Volva

and where is class PageableExecutionUtils ? – Certified

From Spring Data commons: github.com/spring-projects/spring-data-commons/blob/master/src/… – Volva

Doesn't this cause the query to run twice on MongoDB side with the added overhead over the net? – Stellite

@Stellite yes, the query runs twice. Since the query should run against an indexed collection it shouldn't be a big deal. Okay, if you are facebook or twitter it could be a big deal ;D. I don't have another solution – Volva

@checklist: Yes, exactly like mongodb: github.com/spring-projects/spring-data-mongodb/blob/… – Mcmichael

This is interesting but if your query returns hundreds of thousands or millions of rows w/o pagination, this is going to cripple your server when it tries to read all those documents into memory. Much better to use aggregation framework – Brinkley

@Volva would you plz advise why you cannot use list.size() to get total elements? – Disaster

What about query.with(Pageable.unpaged()) instead of query.limit(-1).skip(-1)? – Incurrence

Answer 2

Based on d0x's answer and looking at the spring code. I'm using this variation which works off the spring-boot-starter-data-mongodb dependency without needing to add spring data commons.

@Autowired
private MongoOperations mongoOperations;

@Override
public Page<YourObjectType> searchCustom(Pageable pageable) {
    Query query = new Query().with(pageable);
    // Build your query here

    List<YourObjectType> list = mongoOperations.find(query, YourObjectType.class);
    long count = mongoOperations.count(query, YourObjectType.class);
    Page<YourObjectType> resultPage = new PageImpl<YourObjectType>(list , pageable, count);
    return resultPage;
}

Flesh

just fyi: for getting correct count value - the query should reset: skip, limit values. The count call should be: mongoOperations.count(query.skip(-1).limit(-1), YourObjectType.class). Otherwise it will return incorrect count for the query. – Kerosene

@Kerosene you're right. I spent a whole day trying to understand and fix it. It actually returns the count of the current page instead of the total count of the results. Adding skip and limit solves that problem. – Bagworm

Answer 3

MongoTemplate does not have methods to return Page. The find() methods return an ordinary List.

with(new PageRequests(page, size) is used internally to adjust skip and limit with a MongoDB query (proceeded by a count query I think)

Page can be used in conjunction with MongoDB repositories which is a specialized case of Spring data repositories.

Thus, you'll have to use MongoRepository's Page findAll(Pageable pageable) for paginated results (actually inherited from PagingAndSortingRepository).

Shawana

Answer 4

All answers so far are executing the query twice. This is not normally what you want. Here's a solution using aggregations to run it only once.

public class Result<T> {
    int totalCount;
    List<T> objects;
}

public <T, R extends Result<T>> R executePaged(Class<T> inputType,
                                               Criteria criteria,
                                               Pageable pageable,
                                               Class<R> resultType) {
    var match = Aggregation.match(criteria);
    var facets = Aggregation
            .facet(
                    Aggregation.sort(pageable.getSort()),
                    Aggregation.skip(pageable.getOffset()),
                    Aggregation.limit(pageable.getPageSize())
            ).as("objects")
            .and(
                    Aggregation.count().as("count")
            ).as("countFacet");
    var project = Aggregation
            .project("objects")
            .and("$countFacet.count").arrayElementAt(0).as("totalCount");

    var aggregation = Aggregation.newAggregation(match, facets, project);
    return mongoTemplate.aggregate(aggregation, inputType, resultType)
            .getUniqueMappedResult();
}

Notes:

• Extend and parameterize Result<T> to pass as resultType, otherwise the deserializer doesn't know what class to use
• pageable must be created with a Sort
• If desired, you can still create a PageImpl afterwards

Example:

class Person {
    String name;
    Date birthdate;
}

class PersonResult extends Result<Person>{}

PersonResult result = executePaged(Person.class,
      Criteria.where("name").is("John"),
      PageRequest.of(3, 50).withSort(Sort.by("birthdate")),
      PersonResult.class);

System.out.println("Total #Johns: " + result.totalCount);
System.out.println("Johns on this page: " + result.objects);

Shelve

The problem with this is it will not make use of the index for sorting. $facets do not make use of indexes forcing this whole query use disk for sorting – Varioloid

Answer 5

By default, spring mongo template has no method to find by page. It searches, and returns the whole list of records. I Tried this, and It worke:

Pageable pageable = new PageRequests(0, 10);                              
Query query = new Query(criteria); 
query.with(pageable);   
List<User> lusers = mt.find(query, User.class);   
Page<User> pu = new PageImpl<>(lusers, pageable, mongoTemplate.count(newQuery(criteria), User.class));

Brachyuran

Answer 6

None of the solutions provided here worked in my own case. I tried using this solution below from a medium post and it has never returned the paged results but returns ALL the results which is not what I am expecting

return PageableExecutionUtils.getPage(
        mongoTemplate.find(query, ClassName.class),
        pageable,
        () -> mongoTemplate.count(query.skip(0).limit(0), ClassName.class)
);

So I found out a better way to go about it and it worked in my case:

return PageableExecutionUtils.getPage(
            mongoTemplate.find(query.with(pageable), ClassName.class),
            pageable,
            () -> mongoTemplate.count(query, ClassName.class));

Cantrip

you need to add pageRequest in the query eq. query.with(pageRequest) in the first snippet of your code. rest is fine. skip,limit are required to configure the total page count logic. – Unharness

Answer 7

To execute a Query with Pageable using MongoTemplate, you can use the find(Query, Class, String) method and then convert the result to a Page. Here's how you can do it:

public Page<YourEntity> executeQuery(Query query, Pageable pageable) {
    long totalCount = mongoTemplate.count(query, YourEntity.class);
    query.with(pageable);
    List<YourEntity> resultList = mongoTemplate.find(query, YourEntity.class);
    return PageableExecutionUtils.getPage(resultList, pageable, () -> totalCount);
}

Fluctuation

This is already in accepted answer – Pivot

Answer 8

return type Mono<Page<Myobject>>...

return this.myobjectRepository.count()
        .flatMap(ptiCount -> {
          return this.myobjectRepository.findAll(pageable.getSort())
            .buffer(pageable.getPageSize(),(pageable.getPageNumber() + 1))
            .elementAt(pageable.getPageNumber(), new ArrayList<>())
            .map(ptis -> new PageImpl<Myobject>(ptis, pageable, ptiCount));
        });

Recidivism