spring webclient: retry with backoff on specific error

Question

i'd like to retry the request 3 times after waiting 10sec when response is 5xx. but i don't see a method that I can use. On object

WebClient.builder()
                .baseUrl("...").build().post()
                .retrieve().bodyToMono(...)

i can see methods:

retrying on condition with retry count but no delay

.retry(3, {it is WebClientResponseException && it.statusCode.is5xxServerError} )

retrying with backoff and number of times but no condition

.retryBackoff 

there is also a retryWhen but i'm not sure how to use it

Charge

Answer 1

With reactor-extra you could do it like:

.retryWhen(Retry.onlyIf(this::is5xxServerError)
        .fixedBackoff(Duration.ofSeconds(10))
        .retryMax(3))

private boolean is5xxServerError(RetryContext<Object> retryContext) {
    return retryContext.exception() instanceof WebClientResponseException &&
            ((WebClientResponseException) retryContext.exception()).getStatusCode().is5xxServerError();
}

Update: With new API the same solution will be:

    .retryWhen(Retry.fixedDelay(3, Duration.ofSeconds(10))
            .filter(this::is5xxServerError));

//...

private boolean is5xxServerError(Throwable throwable) {
    return throwable instanceof WebClientResponseException &&
            ((WebClientResponseException) throwable).getStatusCode().is5xxServerError();
}

Galahad

@RobertW. , I've never used coroutines before. I'll try to investigate your problem this weekend. – Galahad

fixed it, will answer my own question #60827683 – Judaea

this is deprecated. – Quaff

Answer 2

You can do this taking the following approach:

• Use the exchange() method to obtain the response without an exception, and then throw a specific (custom) exception on a 5xx response (this differs from retrieve() which will always throw WebClientResponseException with either a 4xx or 5xx status);
• Intercept this specific exception in your retry logic;
• Use reactor-extra - it contains a nice way to use retryWhen() for more complex & specific retries. You can then specify a random backoff retry that starts after 10 seconds, goes up to an arbitrary time and tries a maximum of 3 times. (Or you can use the other available methods to pick a different strategy of course.)

For example:

//...webclient
.exchange()
.flatMap(clientResponse -> {
    if (clientResponse.statusCode().is5xxServerError()) {
        return Mono.error(new ServerErrorException());
    } else {
        //Any further processing
    }
}).retryWhen(
    Retry.anyOf(ServerErrorException.class)
       .randomBackoff(Duration.ofSeconds(10), Duration.ofHours(1))
       .maxRetries(3)
    )
);

Snowfall

Inside the retryWhen() can we use different Retry policies for different Exception ? Let's say for some exception we'd like a higer maxRetries for example ? – Armpit

retryWhen(reactor.retry.Retry) method is deprecated and to be removed in v3.4. But use retryWhen(reactor.util.retry.Retry) – Tundra

Answer 3

the retryWhen with Retry.anyOf and Retry.onlyIf are deprecated I assume. I found this approach useful, and it allows us to process and throw a User defined exception.

for example :

retryWhen(Retry.backoff(3, Duration.of(2, ChronoUnit.SECONDS))
                        .filter(error -> error instanceof UserDefinedException/AnyOtherException)
                        .onRetryExhaustedThrow((retryBackoffSpec, retrySignal) ->
                                new UserDefinedException(retrySignal.failure().getMessage())))

Auvergne

Answer 4

// ...
.retryWhen(
    backoff(maxAttempts, minBackoff)
        .filter(throwable -> ((WebClientResponseException) throwable).getStatusCode().is5xxServerError()))
// ...

Trial

Answer 5

Adding only withThrowable to your existing code can make it work. This has worked for me. You can try something like this :

For example :

.retryWhen(withThrowable(Retry.any()
    .doOnRetry(e -> log
        .debug("Retrying to data for {} due to exception: {}", employeeId, e.exception().getMessage()))
    .retryMax(config.getServices().getRetryAttempts())
    .backoff(Backoff.fixed(Duration.ofSeconds(config.getServices().getRetryBackoffSeconds())))))

Perfect

Answer 6

here's how i do it:

 .retryWhen(retryBackoffSpec())

private RetryBackoffSpec retryBackoffSpec() {
        return Retry.backoff(RETRY_ATTEMPTS, Duration.ofSeconds(RETRY_DELAY))
                .filter(throwable -> throwable instanceof yourException);
    }

Hankypanky

Answer 7

The class reactor.retry.Retry from reactor-extra is deprecated and should be avoided. Use the reactor.util.Retry class from reactor-core.

There is a convenient way to map response statuses to concrete exceptions using the onStatus method.

So, if you want to retry on 5xx status codes, a simple solution would be to throw a CustomException on 5xx and retry only when exception is CustomException.

// Retry only on 5xx
webClient.post()
    .retrieve()
    .onStatus(HttpStatusCode::is5xxClientError, clientResponse -> Mono.error(new CustomException()))
    .bodyToMono(...)
    .retryWhen(Retry.max(3).filter(t -> t instanceof CustomException))

// Alternatively if you don't want to fail/retry on certain status code you can also return an empty `Mono` to ignore the error and propagate the response
webClient.post()
    .retrieve()
    .onStatus(httpStatusCode -> httpStatusCode.value() == 404, clientResponse -> Mono.empty())
    .bodyToMono(...)
    .retryWhen(Retry.max(3).filter(t -> t instanceof CustomException))

Mistassini