Design RESTful query API with a long list of query parameters [closed]

Question

I need to design a RESTful query API, that returns a set of objects based on a few filters. The usual HTTP method for this is GET. The only problem is, it can have at least a dozen filters, and if we pass all of them as query parameters, the URL can get quite long (long enough to be blocked by some firewall).

Reducing the numbers of parameters is not an option.

One alternative I could think of is to make use of the POST method on the URI and send the filters as part of the POST body. Is this against being RESTfull (Making a POST call to query data).

Anyone have any better design suggestions?

Pontefract

Use short (1-char, etc) parameter names? – Hymnist

It may not be truly RESTful, but I think you have to be practical when it comes to GETs and POSTs. If you have that many variables to send and you can't reduce them, I'd POST them. I don't like overstuffing the URL, but that's just me. – Shreveport

Thanks. Even though this question is closed, it is EXACTLY the question to which I needed an answer. I am glad you asked. – Festination

Answer 1

Remember that with a REST API, it's all a question of your point of view.

The two key concepts in a REST API are the endpoints and the resources (entities). Loosely put, an endpoint either returns resources via GET or accepts resources via POST and PUT and so on (or a combination of the above).

It is accepted that with POST, the data you send may or may not result in the creation of a new resource and its associated endpoint(s), which will most likely not "live" under the POSTed url. In other words when you POST you send data somewhere for handling. The POST endpoint is not where the resource might normally be found.

Quoting from RFC 2616 (with irrelevant parts omitted, and relevant parts highlighted):

9.5 POST

The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. POST is designed to allow a uniform method to cover the following functions:

• ...
• Providing a block of data, such as the result of submitting a form, to a data-handling process;
• ...

...

The action performed by the POST method might not result in a resource that can be identified by a URI. In this case, either 200 (OK) or 204 (No Content) is the appropriate response status, depending on whether or not the response includes an entity that describes the result.

If a resource has been created on the origin server, the response SHOULD be 201 (Created)...

We have grown used to endpoints and resources representing 'things' or 'data', be it a user, a message, a book - whatever the problem domain dictates. However, an endpoint can also expose a different resource - for example search results.

Consider the following example:

GET    /books?author=AUTHOR
POST   /books
PUT    /books/ID
DELETE /books/ID

This is a typical REST CRUD. However what if we added:

POST /books/search

    {
        "keywords": "...",
        "yearRange": {"from": 1945, "to": 2003},
        "genre": "..."
    }

There is nothing un-RESTful about this endpoint. It accepts data (entity) in the form of the request body. That data is the Search Criteria - a DTO like any other. This endpoint produces a resource (entity) in response to the request: Search Results. The search results resource is a temporary one, served immediately to the client, without a redirect, and without being exposed from some other canonical url.

It's still REST, except the entities aren't books - the request entity is book search criteria, and the response entity is book search results.

Weigel

Could you suggest some classes naming conventions for the DTO? – Josefinejoseito

Personally I would go with BooksSearchCriteriaDTO and BooksSearchResultsDTO. – Weigel

what would be the best HTTP response code for this case of POST /books/search? 201 still applies? – Tempera

201 is the opposite - it implies that a resource has been created. A resource which is expected to have it own unique URI somewhere. 201 is suitable when the POST is used for the C part of CRUD. I would go with plain old 200, optionally maybe with 204 for empty search results. – Weigel

Keep in mind that GET is in theory perfectly fine for that now too- "The RFC2616 referenced as "HTTP/1.1 spec" is now obsolete. In 2014 it was replaced by RFCs 7230-7237. Quote "the message-body SHOULD be ignored when handling the request" has been deleted. It's now just "Request message framing is independent of method semantics, even if the method doesn't define any use for a message body" The 2nd quote "The GET method means retrieve whatever information ... is identified by the Request-URI" was deleted. " - from https://mcmap.net/q/40417/-http-get-with-request-body – Edric

Answer 2

A lot of people have accepted the practice that a GET with too long or too complex a query string (e.g. query strings don't handle nested data easily) can be sent as a POST instead, with the complex/long data represented in the body of the request.

Look up the spec for POST in the HTTP spec. It's incredibly broad. (If you want to sail a battleship through a loophole in REST... use POST.)

You lose some of the benefits of the GET semantics ... like automatic retries because GET is idempotent, but if you can live with that, it might be easier to just accept processing really long or complicated queries with POST.

(lol long digression... I recently discovered that by the HTTP spec, GET can contain a document body. There's one section that says, paraphrasing, "Any request can have a document body except the ones listed in this section"... and the section it refers to doesn't list any. I searched and found a thread where the HTTP authors were talking about that, and it was intentional, so that routers and such wouldn't have to differentiate between different messages. However, in practice a lot of infrastructure pieces do drop the body of a GET. So you could GET with with filters represented in the body, like POST, but you'd be rolling the dice.)

Kuth

See also this question for more discussion about HTTP GET with body. – Justitia

A lot of people have invented something different from RESTful and HTTP-specification. The author of RESTful Fielding regrets that. – Stinnett

Answer 3

In a nutshell: Make a POST but override HTTP method using X-HTTP-Method-Override header.

Real request

POST /books

Entity body

{ "title": "Ipsum", "year": 2017 }

Headers

X-HTTP-Method-Override: GET

On the server side, check if header X-HTTP-Method-Override exists then take its value as the method to build the route to the final endpoint in the backend. Also, take the entity body as the query string. From a backend point of view, the request became just a simple GET.

This way you keep the design in harmony with REST principles.

Edit: I know this solution was originally intended to solve PATCH verb problem in some browsers and servers but it also work for me with GET verb in the case of a very long URL which is the problem described in the question.

Robrobaina

IETF deprecated X- prefixed HTTP headers: tools.ietf.org/html/rfc6648 – Crony

Google translate API POST request with X-HTTP-Method-Override – Vinic

@Crony The RFC you links stays 1.4. it makes no recommendation on existing X- removal and 1.5. it does not override existing specifications. ... X- will IMO remain here. – Resolvent

Answer 4

If you are developing in Java and JAX-RS I recommend you use @QueryParam with @GET

I had the same question when I needed to go through a list.

See example:

import java.util.List;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Response;

@Path("/poc")
public class UserService {

    @GET
    @Path("/test/")
    @Produces(MediaType.APPLICATION_JSON)
    public Response test(@QueryParam("code") final List<Integer> code) {
                Integer int0 = codigo.get(0);
                Integer int1 = codigo.get(1);

        return Response.ok(new JSONObject().put("int01", int0)).build();
    }
}

URI Pattern: “poc/test?code=1&code=2&code=3

@QueryParam will convert the query parameter “orderBy=age&orderBy=name” into java.util.List automatically.

Unburden

It would be better if you explain your example. In what programming language it's written? – Lareine

Hi @AleksAndreev. Thank you for your opinion. It got better? tks – Unburden

This question is about the design of the RESTful service, not about the implementation. This answer does not answer the question. – Cyanamide

@user1331413 IMHO yes, now it is better. Thank you for your effort.. However, as Mike McCaughan said, question is about REST concept, rather than implementation – Lareine