Spring Boot & JPA: Implementing search queries with optional, ranged criteria
Asked Answered
T

4

21

This is an SSCCE, shows research, isn't a dupe and is on topic!!!


Spring Boot REST service and MySQL here. I have the following Profile entity:

@Entity
@Table(name = "profiles")
public class Profile extends BaseEntity {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "profile_given_name")
    private String givenName;

    @Column(name = "profile_surname")
    private String surname;

    @Column(name = "profile_is_male")
    private Integer isMale;

    @Column(name = "profile_height_meters", columnDefinition = "DOUBLE")
    private BigDecimal heightMeters;

    @Column(name = "profile_weight_kilos", columnDefinition = "DOUBLE")
    private BigDecimal weightKilos;

    @Column(name = "profile_dob")
    private Date dob;

    // Getters, setters & ctor down here
}

I also have a ProfileController and I want to expose a GET endpoint that provides a really flexible/robust way to search for Profiles based on a large range of criteria:

# Search for women between 1.2 and 1.8 meters tall.
GET /v1/profiles?isMale=0&heightMeters={"gt": 1.2, "lt": 1.8}

# Search for men born after Jan 1, 1990 who weigh less than 100 kg.
GET /v1/profiles?isMale=1&dob={"gt" : "1990-01-01 00:00:00"}&weightKilos={"lt": 100.0}

etc.

So here's my controller:

@RestController
@RequestMapping("/v1/profiles")
public class ProfileResource {
  @Autowired
  ProfileRepository profileRepository;

  @GetMapping
  public ResponseEntity<Set<Profile>> searchProfiles(@RequestParam(value = "isMale", required = false) String isMaleVal,
                                              @RequestParam(value = "heightMeters", required = false) String heightMetersVal,
                                              @RequestParam(value = "weightKilos", required = false) String weightKilosVal,
                                              @RequestParam(value = "dob", required = false) String dobVal) {

      Integer isMaleVal;
      BooleanCriteria isMaleCriteria;
      if(isMaleVal != null) {
        // Parse the value which could either be "0" for female, "1" for male or something like
        // ?isMale={0,1} to indicate

        // BooleanCriteria would store which values male, female or both) to include in the search
      }

      BigDecimal heighMeters;
      BigDecimalCriteria heightCriteria;
      if(heightMetersVal != null) {
        // Parse the value which like in the examples could be something like:
        // ?heightMeters={"gt" : "1.0"}

        // BigDecimalCriteria stores range information
      }

      BigDecimal heighMeters;
      BigDecimalCriteria weightCriteria;
      if(weightKilosVal != null) {
        // Parse the value which like in the examples could be something like:
        // ?weightKilos={"eq" : "100.5"}

        // BigDecimalCriteria stores range information
      }

      // Ditto for DOB and DateCriteria

      // TODO: How to pack all of these "criteria" POJOs into a
      // CrudRepository/JPQL query against the "profiles" table?
      Set<Profile> profiles = profileRepository.searchProfiles(
        isMaleCriteria, heightCriteria, weightCriteria, dobCriteria);
    }
}

My thinking for, say, BigDecimalCriteria would be something like:

// Basically it just stores the (validated) search criteria that comes in over the wire
// on the controller method
public class BigDecimalCriteria {
  private BigDecimal lowerBound;
  private Boolean lowerBoundInclusive;
  private BigDecimal upperBound;
  private Boolean upperBoundInclusive;

  // Getters, setters, ctors, etc.
}

Since all of these search criteria are optional (and thus can be null), I'm stuck on how to write the JPQL query in the ProfileRepository:

public interface ProfileRepository extends CrudRepository<Profile,Long> {
  @Query("???")
  public Set<Profile> searchProfiles();
}

How can I implement the @Query(...) for ProfileRepository#searchProfiles in such a way that enables all of my search criteria (given all the permissible ranges and criteria values to search for), and allows any criteria to be null/optional?

Of course, if there are any nifty little libraries or if Spring Boot/JPA already has a solution for this, I'm all ears!

Taction answered 19/1, 2018 at 17:19 Comment(6)
Have you looked into querydsl querydsl.comGelignite
The easiest way to handle a query with complex, conditional criteria is to use a CriteriaQuery. Criteria queries can be integrated smoothly into Spring Data repositories using a Specification (see example here: spring.io/blog/2011/04/26/…)Almswoman
Why the downvote? This is an SSCCE, shows research, isn't a dupe and is on topic!Taction
It is very much a dupe and shows no research, also contains tons of irrelevant cruft. #32729343 #45717423 and many more deal with variations of this topic.Matelda
@JensSchauder many more you too big boy! Sounds like Spring Data doesn't have the capability to address this type of problem and someone is upset that its being called out....Taction
I'm sorry that I left the impression of being upset about the limitations you see in in Spring Data. I'm not, but thanks for worrying. My intention was just to help you solve the problem stated in the question by pointing out that variations of it are solved many times on this site, and to answer your question in the comment why it was downvoted.Matelda
A
35

You can achieve complex queries with specifications by JpaSpecificationExecutor in spring data. Repository interface must extend the JpaSpecificationExecutor<T> interface so we can specify the conditions of our database queries by creating new Specification<T> objects.

The trick is in the use of the Specification interface in combination with a JpaSpecificationExecutor. here is the example:

@Entity
@Table(name = "person")
public class Person {

 @Id
 @GeneratedValue(strategy = GenerationType.AUTO)
 private Long id;

 @Column(name = "name")
 private String name;

 @Column(name = "surname")
 private String surname;

 @Column(name = "city")
 private String city;

 @Column(name = "age")
 private Integer age;

        ....

}

Then we define our repository:

public interface PersonRepository extends JpaRepository<Person, Long>, JpaSpecificationExecutor<Person> {

}

As you can see we have extended another interface the JpaSpecificationExecutor. This interface defines the methods to perform the search via a Specification class.

What we have to do now is to define our specification that will return the Predicate containing the constraints for the query (in the example the PersonSpecification is performing the query select * from person where name = ? or (surname = ? and age = ?) ):

public class PersonSpecification implements Specification<Person> {

    private Person filter;

    public PersonSpecification(Person filter) {
        super();
        this.filter = filter;
    }

    public Predicate toPredicate(Root<Person> root, CriteriaQuery<?> cq,
            CriteriaBuilder cb) {

        Predicate p = cb.disjunction();

        if (filter.getName() != null) {
            p.getExpressions()
                    .add(cb.equal(root.get("name"), filter.getName()));
        }

        if (filter.getSurname() != null && filter.getAge() != null) {
            p.getExpressions().add(
                    cb.and(cb.equal(root.get("surname"), filter.getSurname()),
                            cb.equal(root.get("age"), filter.getAge())));
        }

        return p;
    }
}

Now it is time to use it. The following code fragment shows how to use the Specification we just created:

...

Person filter = new Person();
filter.setName("Mario");
filter.setSurname("Verdi");
filter.setAge(25);

Specification<Person> spec = new PersonSpecification(filter);

List<Person> result = repository.findAll(spec);

Here is full example present in github

Also you can create any complex queries using Specification

Aeneas answered 2/2, 2018 at 6:22 Comment(6)
Great answer, suppose I want to filter the records based on 3 parameters and each are optional to be passed to filter records, so do I need to write 6 cases in toPredicate right?Rauwolfia
it should be 3 if you have just needed 3, means each has the individual condition and don't need to group with others, if you have the group-based condition on you parameter then you can create any number of cases with the predicate.Aeneas
yes, as parameters are optional, so there might be group condition and might be not if only one parameter is passed. So I have to check for all the condition and write cases for 6 conditions as I have 3 parametersRauwolfia
Thank you @bhushan for the time, just one more doubt, I have to filter on entity which has manytomany relationship, how can I pass parameter for that. can you suggest on that pleaseRauwolfia
you can Join with your entity "Join<SourceEntity, TargetEntity> join = root.join("targetEntiiesVaribaleNameInSourceClass");" and add the predicate with itAeneas
Thank you for your time @bhushanuniyal, it is working as expectedRauwolfia
A
9

Almost what you need is already implemented in Spring Data with help of Querydsl and Web support Spring Data extensions.

You should extend your repo as well from QuerydslPredicateExecutor and, if you are using Spring Data REST, you can query your repo data right 'from the box' with base filtering, paging and sorting support:

/profiles?isMale=0&heightMeters=1.7&sort=dob,desc&size=10&page=2

To implement more complex filter you should extend your repo from the QuerydslBinderCustomizer and use its customize method (right in your repo).

For example you can implement 'between' filter for heightMeters and 'like' filter for surname:

public interface ProfileRepository extends JpaRepository<Profile, Long>, QuerydslPredicateExecutor<Profile>, QuerydslBinderCustomizer<QProfile> {

    @Override
    default void customize(QuerydslBindings bindings, QProfile profile) {

      bindings.excluding( // used to exclude unnecessary fields from the filter
          profile.id,
          profile.version,
          // ...
      );

      bindings.bind(profile.heightMeters).all((path, value) -> {

          Iterator<? extends BigDecimal> it = value.iterator();
          BigDecimal from = it.next();
          if (value.size() >= 2) {
              BigDecimal to = it.next();
              return path.between(from, to)); // between - if you specify heightMeters two times
          } else {
              return path.goe(from); // or greter than - if you specify heightMeters one time
          }
      });

      bindings.bind(profile.surname).first(StringExpression::containsIgnoreCase);        
    }
}

Then you can query your profiles:

/profiles?isMale=0&heightMeters=1.4&heightMeters=1.6&surename=doe

i.e. - find all females which height is between 1.4 and 1.6 meters and surename contains 'doe'.

If you are not using Spring Data REST you can implement your own rest controller method with QueryDSL support:

@RestController
@RequestMapping("/profiles")
public class ProfileController {

    @Autowired private ProfileRepository profileRepo;

    @GetMapping
    public ResponseEntity<?> getAll(@QuerydslPredicate(root = Profile.class, bindings = ProfileRepository.class) Predicate predicate, Pageable pageable) {

        Page<Profile> profiles = profileRepo.findAll(predicate, pageable);
        return ResponseEntity.ok(profiles);
    }
}

Note: don't forget to add QueryDSL dependency to you project:

<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-jpa</artifactId>
</dependency>

<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-apt</artifactId>
    <scope>provided</scope>
</dependency>

<build>
    <plugins>
        <plugin>
            <groupId>com.mysema.maven</groupId>
            <artifactId>apt-maven-plugin</artifactId>
            <version>1.1.3</version>
            <executions>
                <execution>
                    <goals>
                        <goal>process</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>target/generated-sources/annotations</outputDirectory>
                        <processor>com.querydsl.apt.jpa.JPAAnnotationProcessor</processor>                                                       
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Then compile your project (for example mvn compile) to let it make 'Q' classes.

Amimia answered 3/2, 2018 at 10:6 Comment(4)
Can you also give the gradle dependencies for query dsl?Mesentery
@JagratiGogia No, but I hope these posts can help you: discuss.gradle.org/t/…, #22774139Amimia
Small note: in your customize method, there is Optional.of( missing on both return statements.Cataract
@Graslandpinguin Yes, it has been changed since Spring Boot 2 (see my related 'how-to' post)Amimia
C
4

The answer is quite eaiser and you can use the query-by-example in spring.

and even more you do not need to list all the Profile properties in your controller, you just take the Profile as the parameter, spring will take care of it.

And as you want to validate the request params, here is eaier to integrate with bean validator, take "givenName" as an example. add the NotNull in the entity, and add @Valid in the controller, in case the "givenName" is not in the request params, you will get the "Bad Request" response.

Here are the working codes:

@Entity
@Table(name = "profiles")
public class Profile {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "profile_given_name")
    @NotNull
    private String givenName;

    @Column(name = "profile_surname")
    private String surname;

    @Column(name = "profile_is_male")
    private Integer isMale;

    @Column(name = "profile_height_meters", columnDefinition = "DOUBLE")
    private BigDecimal heightMeters;

    @Column(name = "profile_weight_kilos", columnDefinition = "DOUBLE")
    private BigDecimal weightKilos;

    @Column(name = "profile_dob")
    private Date dob;
}

ProfileResource

@RestController
@RequestMapping("/v1/profiles")
public class ProfileResource {
    @Autowired
    ProfileRepository profileRepository;

    @GetMapping
    public ResponseEntity<List<Profile>> searchProfiles(@Valid Profile profile) {
        List<Profile> all = profileRepository.findAll(Example.of(profile));
        return ResponseEntity.ok(all);
    }
}

ProfileRepository

public interface ProfileRepository extends JpaRepository<Profile, Long> {
}

Then send the GET /v1/profiles?isMale=0 HTTP method as you wanted.

Cristencristi answered 30/1, 2018 at 10:8 Comment(11)
Thanks @Liping (+1) - a few followup questions, if you don't mind: (1) is this documented anywhere? (2) This is a GET endpoint that takes an argument in the controller method...does the user have to send anything in the request entity/body? (3) Your example shows isMale=0, what are all the different types of supported comparison operators? For instance, how could I search for Profiles with heightMeters > 1.5 AND heightMeters <= 2.0?Taction
@Taction for (1) , yes, you can refer to the link. for (2) I just update the answer to add the the bean validator sample, you do not need to list all the Profile properties with the GET method. for (3) spring will convert the path vars to the correspondind Object, you can refer to spring type convestion part.Cristencristi
Thanks again so much @Liping!!! For #3 above, I just looked at your link and searched for "type conversion" and nothing turned up. Any chance you could modify your example to show me how I could search for Profiles with 1.5 < heightMeters <= 2.0 (as an example)? Thanks again!Taction
@Taction Sorry, for (3) my mistake, you can impmement it with the bean validator, add @Range(min=, max=) in your entity.Cristencristi
@Taction Here is a good example for the bean validator - baeldung.com/javax-validationCristencristi
Thanks again @Liping, but that's only for validation, not querying...I need to be able to query for Profiles whose heightMeters property is between, say, 1.5 and 2.0 meters. I don't believe that your Query By Example approach above allows me to do that, correct?Taction
Yes, this is the validator for the entity, so will impact the save behavior. for query, one is you can add the logic into the controller section.Cristencristi
Or add the property with the corresponding annotation just like what your already did.Cristencristi
Thanks again :-) Last question (I promise!) Does Spring/QueryByExample natively support query string params like ?heightMeters={"gt" : "1.0"} or are you saying that I'll need to add support for that and add an ExampleMatcher into the controller logic myself?Taction
it ok smeeb, let's say JPA/Hibernate, as the search by example is actually the hibernate behavior, so basis it will parse to native sql then query it from the database. so for you, the query string is a json string, so you need to do the logic self.Cristencristi
According to the reference document between queries are not possible using this method. > >Query by Example also has several limitations: > No support for nested or grouped property constraints, such as firstname = ?0 or (firstname = ?1 and lastname = ?2). > Only supports starts/contains/ends/regex matching for strings and exact matching for other property types.Unalloyed
P
3

Check out “query by example” in spring data. Seems to fit the bill for what you need...

https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example

Pleochroism answered 20/1, 2018 at 2:18 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.