spring resttemplate url encoding

Question

I try to do a simple rest call with springs resttemplate:

private void doLogout(String endpointUrl, String sessionId) {
    template.getForObject("http://{enpointUrl}?method=logout&session={sessionId}", Object.class,
            endpointUrl, sessionId);
}

Where the endpointUrl variable contains something like service.host.com/api/service.php

Unfortunately, my call results in a org.springframework.web.client.ResourceAccessException: I/O error: service.host.com%2Fapi%2Fservice.php

So spring seems to encode my endpointUrl string before during the creation of the url. Is there a simple way to prevent spring from doing this?

Regards

Katabatic

I hope it would be better solution - https://mcmap.net/q/102518/-spring-resttemplate-url-encoding It shows how to avoid Spring's RestTemplate URL encoding issues. – Subcommittee

Answer 1

There is no easy way to do this. URI template variables are usually meant for path elements or a query string parameters. You're trying to pass a host. Ideally, you'd find a better solution for constructing the URI. I suggest Yuci's solution.

If you still want to work with Spring utilities and template expansion, one workaround is to use UriTemplate to produce the URL with the URI variables as you have them, then URL-decode it and pass that to your RestTemplate.

String url = "http://{enpointUrl}?method=logout&session={sessionId}";
URI expanded = new UriTemplate(url).expand(endpointUrl, sessionId); // this is what RestTemplate uses 
url = URLDecoder.decode(expanded.toString(), "UTF-8"); // java.net class
template.getForObject(url, Object.class);

Blancmange

this solved the issue with the escaped / in my url. But unfortunately, it does not solve my whole problem. There are cases, where "payload" contains a JSON-string. This json String is encoded somewhere inside the rest template code. The String URL that is passed to the templates getForObject method looks as expected. But then I receive an exception when spring tries to expand the json part of my url... – Katabatic

@Katabatic You would have to ask another question with all the details, but I recommend against putting JSON as a query string request parameter. Send it in the body of a POST or PUT. – Blancmange

You doing unnecessary work: 1) you encoding uri with expanded.toString(), 2) then you decoding it back to url, 3) and after that it will be encoded again in template.getForObject(). It would be better to use clear approach (use URI object instead of String). See explanation in my answer to this question - https://mcmap.net/q/102518/-spring-resttemplate-url-encoding – Subcommittee

@DmitryTrifonov I tried to clarify that such a transformation can be used to generate the URI through template variable resolution ("there is no easy way to do this). If they can construct the URL correctly without it, then that is a better solution. – Blancmange

Answer 2

Depends on which version of Spring you're using. If your version is too old, for example, version 3.0.6.RELEASE, you'll not have such facility as UriComponentsBuilder with your spring-web jar.

What you need is to prevent Spring RestTemplate from encoding the URL. What you could do is:

import java.net.URI;

StringBuilder builder = new StringBuilder("http://");
builder.append(endpointUrl);
builder.append("?method=logout&session=");
builder.append(sessionId);

URI uri = URI.create(builder.toString());
restTemplate.getForObject(uri, Object.class);

I tested it with Spring version 3.0.6.RELEASE, and it works.

In a word, instead of using restTemplate.getForObject(String url, Object.class), use restTemplate.getForObject(java.net.URI uri, Object.class)

See the rest-resttemplate-uri section of the Spring document

Urias

Answer 3

Looks like I found best native way (up-to-date) solution:

1. Do not pass encoded url string as parameter to RestTemplate.exchange()
2. Use URI object instead. Use UriComponentsBuilder to construct URI.

See (simplified) example below:

    String instanceUrl = "https://abc.my.salesforce.com"
    HttpEntity<String> entity = new HttpEntity<>(headers);
    UriComponents uriComponents =
            UriComponentsBuilder.fromHttpUrl(instanceUrl)
                    .path("/services/data/v45.0/query/")
                    .queryParam("q", String.format(sqlSelect, id))
                    .build();

    ResponseEntity<OpportunityLineItem> responseEntity =
            restTemplate.exchange(
                    uriComponents.toUri(), HttpMethod.GET,
                    entity, OpportunityLineItem.class);

// Wrong! URI string will be double encoded
/*
ResponseEntity<OpportunityLineItem> responseEntity =
            restTemplate.exchange(
                    uriComponents.toUriString(), HttpMethod.GET,
                    entity, OpportunityLineItem.class);
*/

This way you will not get issue with double encoding.

Solution was found while debugging SalesForce REST client, based on Spring RestTemplate client (including SOQL queries).

Subcommittee

They wouldn't have an encoding problem if the URI string passed to getForObject was correctly constructed. So the gist of their question is how to construct a correct URI. They have part of the URI in the endpointUrl and sessionId parameters and part of it in the String literal passed to getForObject. They need something that can reconcile the parts. You've skipped all that and just assumed there was an instanceUrl. What is that in the context of their question? Where does its value come from? How did you build it? – Blancmange

The query parameters aren't the problem here. The problem is the resolution of the host service.host.com and path /api/service.php in their endpointUrl variable. – Blancmange

I fixed example again so it will be more clear and easy to understand. My intent was to show correct usage of UriComponentsBuilder to avoid URL encoding issues. – Subcommittee

Answer 4

You can use the overloaded variant that takes a java.net.URI instead public T getForObject(URI url, Class responseType) throws RestClientException

From Spring's own documentation

UriComponents uriComponents =
    UriComponentsBuilder.fromUriString("http://example.com/hotels/{hotel}/bookings/{booking}").build()
        .expand("42", "21")
        .encode();

URI uri = uriComponents.toUri();

Impatient

Answer 5

Apparently there is a better way to do this by calling build(true) of class UriComponentsBuilder:

private void doLogout(String endpointUrl, String sessionId) {
    String url = "http://" + endpointUrl +"?method=logout&session=" + + URLEncoder.encode(sessionId, "UTF-8");
    URI uri = UriComponentsBuilder.fromUriString(url.toString()).build(true).toUri();
    template.getForObject(uri, Object.class,
            endpointUrl, sessionId);
}

This method tells URIComponentsBuilder not to encode while creating URI.

Bik

This solution worked best for me, all other solutions had either no encoding or wrong encoding. – Surrounding

Answer 6

Full example with headers, body, for any HttpMethod and ResponseType could look like:

String url = "http://google.com/{path}?param1={param1Value}&param2={param2Value}";
Object body = null;
HttpEntity request = new HttpEntity(body, new HttpHeaders());

Map<String, String> uriVariables = new HashMap<>();
uriVariables.put("path", "search");
uriVariables.put("param1Value", "value1");
uriVariables.put("param2Value", "value2");

ResponseEntity<Void> responseEntity = restTemplate.exchange(url, HttpMethod.POST, request, Void.class, uriVariables)
//responseEntity.getBody()

Actually, it will use the same UriTemplate and expand method

Muttonhead

uri variables will be encoded, which is the opposite of this question – Abolition

Answer 7

Best way to do is with UriComponentsBuilder:

UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url);
builder.queryParam("some_param", "param with space for encoding");
template.getForObject(builder.encode().build().toUri(), Object.class, headers);

Baggywrinkle