Spring Data PageImpl not returning page with the correct size?

Question

I am trying to create a new Page using a list of objects retrieved from the database. First I get all the elements from the DB, convert it to a Stream and then use lambda to filter the results. Then I need a Page with a set number of elements, however, instantiating a new PageImpl doesn't seem to return a page with the correct size.

Here is my code:

List<Produtos> listaFinal;
Stream<Produtos> stream = produtosRepository.findAll().stream();
listaFinal = stream.filter(p -> p.getProdNome().contains("uio")).collect(Collectors.toList());

long total = listaFinal.size();
Page<Produtos> imp = new PageImpl<>(listaFinal,pageable,total);

Here's a screenshot from debugging:

Note the size in the Pageable object is set to 20 and it understands that it needs 4 pages to render the 70 elements, but it returns the whole list.

What am I missing?

Edit answering the comment made by Thomas:

I understand how to use Page to return just a slice of the data. The code I showed was my attempt to use a lambda expression to filter my collection. The problem for me is I want to use Java 8's lambda to query the database via Spring Data JPA. Im used to VB.NET's and Entity function(x) query expressions and was wondering how to do the same with Spring JPA.

In my repository, Im using extends JpaRepository<Produtos, Integer>, QueryDslPredicateExecutor<Produtos> which gives me access to findAll(Predicate,Pageable). However, the Predicate is not typed so I cant simply use p -> p.getProdNome().contains("uio") in the query. I'm using SQL Server and Hibernate.

Pierre

Answer 1

After learning more about how Spring Data works I ended up using @Query annotations on my methods inside the JpaRepository implementations to properly query the DB and filter the results, eliminating the need to use a stream and then convert back to Page.

Here's how the code above would look in case anyone needs an example:

@Query("select p from Produtos p where p.prodNome = ?1")
public Page<Produtos> productsListByName(String prodNome, Pageable pageable)

Im aware of Spring's findBy methods but sometimes the method names become really difficult to read depending on the amount of parameters so I just stuck to JPQL.

Doing it this way the Page's content will always have up to the maximum amount of elements defined by you in the Spring configuration.

I also use a custom implementation of PageImpl, I'm not at work right now and don't have access to the code, but I'll post it whenever I can.

Edit: Custom implementation can be found here

Pierre

Nice! I will give this a shot with what I'm working on. – Thanksgiving

Can you provide me the code you have been using for PageImpl. I am facing same problem now! – Pathic

Code added. Note that this implementation was needed in my case because I was dealing with a REST service, and the default implementation wasn't working for me, might be different in your case. – Pierre

Moved custom implementation to another website since the other one was down. – Pierre

@Pierre what if the query is typedQuery ... how it will work , in my code it is not working ..#62486092 – Wrightson

Interesting, have you implemented another subclass of Page? Because as far as I have seen the TotalPage Property of PageImpl always returns the number of elements passed in the collection seem not right for me and would be simple to solve as I think – Evolutionary

Answer 2

To extend stites' answer, a PagedListHolder is the way to go and here is how:

List<String> list = // ...

// Creation
PagedListHolder page = new PagedListHolder(list);
page.setPageSize(10); // number of items per page
page.setPage(0);      // set to first page

// Retrieval
page.getPageCount(); // number of pages 
page.getPageList();  // a List which represents the current page

If you need sorting, use another PagedListHolder constructor with a MutableSortDefinition.

Wrathful

Answer 3

PageImpl is not intended to perform any kind of pagination of your list. From the docs you can see that it's just the "basic Page implementation" which almost sounds like what you want, but it's really misleading.

Use PagedListHolder which is a simple state holder for handling lists of objects, separating them into pages.

Thanksgiving

Answer 4

If I understood your code right, then your intent is to load all records from the database and and split them into x buckets that are collected in the PageImpl, right?

Thats not how it used to work. The actual intent of the Pageable and Page abstraction is NOT having to query all the data but just the "slice" of data that is needed.

In your case you could query the data via Page<X> page = repository.findAll(pageable); and simply return that. Page holds the records for the current page alongside some additional information like e.g., the total number of records and whether there is a next page.

In your client code you can use that information to render a list of records and generating next / prev links appropriately. Note that a query with Page<X> as result type issues 2 queries (1 to determine the overall total count for the query and 1 for the actual page data).

If you don't need the information about the total number of results but still want to be able to generate a next link you should use Slice<X> as a return type - since it only issues 1 query.

Moderate

I edited my original question with my comment since it was too long to write here, take a look please! Thanks. – Pierre

I am facing the same issue. I have to use a DTO to map the output of the query to remove some hidden columns. So I first get all the results and then map the result to new output as per DTO. then I use PageImpl on the new result . But its showing all the records instead of paged info. – Wulfe

Answer 5

I was also facing the same issue and found the way for it.

The SimpleJpaRepository has the method:

public Page<T> findAll(Specification<T> spec, Pageable pageable) {
    TypedQuery<T> query = getQuery(spec, pageable);
    return pageable == null ? new PageImpl<T>(query.getResultList())
                : readPage(query, getDomainClass(), pageable, spec);
}

Which is used to return Page<T> in case you are extending JpaRepository. So we can use the same functionality here (need to rewrite the code, as Spring doesn't give you public method to have full pagination support).

If you look at the method PageImpl<>(List<T> content, Pageable pageable, long total); it just set's the value whatever you give in pageable. Here you are sending content as full list, but spring doesn't do it for it's internal purpose.

Need to replace Page<Produtos> imp = new PageImpl<>(listaFinal,pageable,total);

Following code:.

TypedQuery<User> query = entityManager.createQuery(criteriaQuery); 
// Users type can be replaced with any other entity

query.setFirstResult(pageable.getOffset());
query.setMaxResults(pageable.getPageSize());

List<User> users = query.getResultList();
Page<User> result = PageableExecutionUtils.getPage(users,pageable, 
                    () -> getCountForQuery(User.class));

Method getCountForQuery:

private Long getCountForQuery(Class<?> t) {

    CriteriaBuilder criteriaBuilder=entityManager.getCriteriaBuilder();

    CriteriaQuery<Long> countQuery = criteriaBuilder
            .createQuery(Long.class);
    countQuery.select(criteriaBuilder.count(
            countQuery.from(t)));
    Long count = entityManager.createQuery(countQuery)
            .getSingleResult();

    return count;
}

You can find the usage of PageableExecutionUtils.getPage in:

readPage(TypedQuery<S> query, final Class<S> domainClass, 
         Pageable pageable, final Specification<S> spec) 

Method in SimpleJpaRepository which is mostly used by findAll internal method.

Ectoderm

Assume that in the data query you have more roots or joins, so predicates could involve more entities. How should be the count query?...You can't do a count query only from a single root...how should be the count query?...I have to redefine all roots and joins also for the count query?...Any idea ? – Hynda

Answer 6

After applying a lot of methodes, this was the working solution in my case:

int pageSize = pageable.getPageSize();
long pageOffset = pageable.getOffset();
long total = pageOffset + list.size() + (list.size() == pageSize ? pageSize : 0);
Page<listType> page = new PageImpl<listType>(list, pageable,total)

Psalms

Answer 7

previously my code was written like this

    Pageable pageable = PageRequest.of(pageNo,size);
    Query query = new Query().with(pageable);

and i was getting this for

"pageNumber": 0, "pageSize": 5, "size": 5, "numberOfElements": 5,

"pageNumber": 0, "pageSize": 10, "size": 10, "numberOfElements": 8,

8 is the total actual element i have in my DB

I changed that to this

    Pageable pageable = PageRequest.of(pageNo,size);
    Query query = new Query();

now i'm getting the actual number of item for any size of page

"pageNumber": 0, "pageSize": 5, "size": 5, "numberOfElements": 8,

Inaccessible

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Blackandwhite