How to best get a byte array from a ClientResponse from Spring WebClient?

Question

I'm trying out the new WebClient from Spring 5 (5.0.0.RC2) in a codebase that uses reactive programming and I've had success mapping the JSON response from an endpoint to a DTO in my app, which works very nice:

WebClient client = WebClient.create(baseURI);
Mono<DTO> dto = client.get()
        .uri(uri)
        .accept(MediaType.APPLICATION_JSON)
        .exchange()
        .flatMap(response -> response.bodyToMono(DTO.class));

However, now I'm trying to the response body from an endpoint which uses Protocol Buffers (binary data served as application/octet-stream), so I'd like to get the raw bytes from the response, which I'll then map to an object myself.

I got it to work like this using Bytes from Google Guava:

Mono<byte[]> bytes = client.get()
        .uri(uri)
        .accept(MediaType.APPLICATION_OCTET_STREAM)
        .exchange()
        .flatMapMany(response -> response.body(BodyExtractors.toDataBuffers()))
        .map(dataBuffer -> {
            ByteBuffer byteBuffer = dataBuffer.asByteBuffer();
            byte[] byteArray = new byte[byteBuffer.remaining()];
            byteBuffer.get(byteArray, 0, bytes.length);
            return byteArray;
        })
        .reduce(Bytes::concat)

This works, but is there an easier, more elegant way to get these bytes?

Penuche

Answer 1

ClientResponse.bodyToMono() in the end uses some org.springframework.core.codec.Decoder which claims to support the specified class.

So we should check the class hierarchy of the Decoder, in particular where and how the decodeToMono() method is implemented.

There is a StringDecoder which supports decoding to String, a bunch of Jackson-related decoders (used in your DTO example under the hood), and there is also a ResourceDecoder which is of particular interest.

ResourceDecoder supports org.springframework.core.io.InputStreamResource and org.springframework.core.io.ByteArrayResource. ByteArrayResource is essentially a wrapper around byte[], so the following code will provide an access to the response body as a byte array:

Mono<byte[]> mono = client.get()
            ...
            .exchange()
            .flatMap(response -> response.bodyToMono(ByteArrayResource.class))
            .map(ByteArrayResource::getByteArray);

Nepali

Note that this will buffer the response and load the entire byte array in to memory. – Presentable

I tried the above option for getting pdf file byte array but post writing the byte array into a file, I get "file is corrupted" error. Am I missing something? – Dorice

Better options here https://mcmap.net/q/346602/-how-to-correctly-read-flux-lt-databuffer-gt-and-convert-it-to-a-single-inputstream/839733 – Midships

Given .exchange() is now marked as deprecated what would be the updated code for Spring 6? – Purveyor

Answer 2

Oleg Estekhin's answer gives the OP what he asked for, but it's loading the entire response content in the memory, which is an issue for large responses. To get a chunk of bytes at a time instead, we can do the following:

client.get()
  .uri("uri")
  .exchange()
  .flatMapMany { it.body(BodyExtractors.toDataBuffers()) }

The size of these buffers would be 8192 kb by default; see this answer for changing that if needed.

Note that attempting to do dataBuffer.asByteBuffer().array() thrwos an exception if the ByteBuffer is not backed by an array.

Midships