How to perform a zero-copy upload and download with WebClient?

Question

Can you perform a zero-copy upload and download with Spring 5 WebFlux using org.springframework.web.reactive.function.client.WebClient?

Silverstein

Based on the documentation of BodyInserters.fromResource, it sounds like the following will do a zero-copy upload: client.post().body(BodyInserters.fromResource(new FileSystemResource(new File("file.txt")))). Is that accurate? What about downloads? – Silverstein

Can only assume a zero-copy download is achievable through: client.post().exchange().doOnNext(r -> DataBufferUtils.write(r.body(BodyExtractors.toDataBuffers()), channel, 0).subscribe(DataBufferUtils.releaseConsumer())). However, how can I then block until it is finished consuming? Is this even going to perform a zero-copy download? – Silverstein

what is zero-copy? – Conchiferous

Answer 1

You're right, zero-copy is supported for now when posting data from a File-based Resource.

So the following looks right:

client.post()
      .body(BodyInserters.fromResource(new FileSystemResource(new File("file.txt"))));

Now for the reading part, zero-copy is not supported on the reading side right now in Spring Framework; you could create an enhancement issue on jira.spring.io for that.

Your code sample should look like:

Flux<DataBuffer> incoming = client.post()
      .retrieve().bodyToFlux(DataBuffer.class);
Mono<Void> writeOperation = DataBufferUtils.write(incoming, channel)
      .map(DataBufferUtils::release)
      .then();
// subscribe to the returned value, which will complete when writing is done

Unfortunately, reading data to DataBuffer won't do zero-copy as data will be copied in memory. I don't think zero-copy is supported right know on the reading side, so this could be an enhancement request on https://jira.spring.io.

Biphenyl

Does it buffer the entire response? Doesn't buffering cause a block? What's the purpose of DataBufferUtils.write(Publisher<DataBuffer>, AsynchronousFileChannel, long) if it buffers the response? – Silverstein

The entire response is not buffered, it’s written as demand is requested by the reactive pipeline. – Biphenyl

Shouldn't it be bodyToFlux? – Jaipur

Please mind that in case of retry you will start writing bytes starting from the beginning of the channel which correlates to the beginning of the file which is basically overwriting anything that you have already downloaded. – Schaller