How can I return a Flux<Order> when my response is wrapped in a json pagination object with Spring 5?

Question

I've got a web service that I'm trying to consume with the new Spring 5 WebClient.

Working example

# GET /orders/
[
    { orderId: 1, ... },
    { orderId: 1, ... }
]

And the java code for the call

// Java
Flux<Order> ordersStream = webClient.get()
    .uri("/orders/")
    .exchange()
    .flatMap(response -> response.bodyToFlux(Order.class));

Problem

The response from the web service is paginated and therefore doesn't contain the list of items directly as in the example above.

It looks like this

# GET /orders/
{
    "error": null,
    "metadata": {
      "total": 998,
      "limit": 1000,
      "offset": 0
    },
    "data": [
       { orderId: 1, ... },
       { orderId: 2, ... },
    ]
}

How can I get the sub key "data" as a Flux<Order>?

Possible solution, but I don't know if it's the best approach...

Create a wrapper class and convert the wrappers .data to a flux.

But now we need to deserialize the whole response at once, potentially running out of memory.

// Java
Flux<Order> ordersStream = webClient.get()
    .uri("/orders/")
    .exchange()
    .flatMap(response -> response.bodyToMono(PageWrapper.class))
    .flatMapMany(wrapper -> Flux.fromIterable(wrapper.data));

Is there a better way?

Lazes

Could you provide more context please? What exactly do you want to map to the Order type? Each individual item in data? – Cloraclorinda

@Cloraclorinda I've updated the question with some more context. – Lazes

I don't think there's an out-of-the-box solution to this. You would need to work on a Flux<DataBuffer> level to parse the json on a low-level and then emit Order instances as soon as enough data is accumulated. In fact this is how webclient is doing it: see org.springframework.http.codec.json.JsonObjectDecoder. This decoder is used to rechunk the raw data so that the boundaries of the chunks are aligned with valid json objects – Cloraclorinda

I am also confused about this. The spring-webflux and spring data ReactiveXXXRepsoitory do not include pageable APIs as general. And Flux stands for the live data in the stream, Is there a good way to let Flux to emit a limit of data one time from subscriber? – Camise