Micronaut ReadTimeoutException

Question

I have a Grails 4 application providing a REST API. One of the endpoints sometimes fail with the following exception:

io.micronaut.http.client.exceptions.ReadTimeoutException: Read Timeout
    at io.micronaut.http.client.exceptions.ReadTimeoutException.<clinit>(ReadTimeoutException.java:26)
    at io.micronaut.http.client.DefaultHttpClient$10.exceptionCaught(DefaultHttpClient.java:1917)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:426)
    at io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:92)
    at io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:147)
    at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
    at io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:233)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
    at io.netty.handler.timeout.ReadTimeoutHandler.readTimedOut(ReadTimeoutHandler.java:98)
    at io.netty.handler.timeout.ReadTimeoutHandler.channelIdle(ReadTimeoutHandler.java:90)
    at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:505)
    at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:477)
    at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)

The endpoint uses micronaut http client to call other systems. The remote system takes a very long time to respond, causing the ReadTimeOutException.

Here is the code calling the remote Service:

class RemoteTaskService implements GrailsConfigurationAware {

    String taskStepperUrl

    // initializes fields from configuration
    void setConfiguration(Config config) {
        taskStepperUrl = config.getProperty('services.stepper')
    }

    private BlockingHttpClient getTaskClient() {
        HttpClient.create(taskStepperUrl.toURL()).toBlocking()
    }

    List<Map> loadTasksByProject(long projectId) {
        try {
            retrieveRemoteList("/api/tasks?projectId=${projectId}")
        } catch(HttpClientResponseException e) {
            log.error("Loading tasks of project failed with status: ${e.status.code}: ${e.message}")
            throw new NotFoundException("No tasks found for project ${projectId}")
        }
    }

    private List<Map> retrieveRemoteList(String path) {
        HttpRequest request = HttpRequest.GET(path)
        HttpResponse<List> response = taskClient.exchange(request, List) as HttpResponse<List>
        response.body()
    }
}

I've tried resolving it using the following configuration in my application.yml:

micronaut:
    server:
        read-timeout: 30

and

micronaut.http.client.read-timeout: 30

...with no success. Despite my configuration, the timeout still occurs around 10s after calling the endpoint.

How can I change the read timeout duration for the http rest client?

Towill

It isn't clear what you are using to initiate the REST call. Can you show that code? – Asafetida

@JeffScottBrown It found a use of a rest client in the code, calling a remote microservice that take a very long time to respond. The code calling the remote service has been posted. Since I now know where the exception is coming from, the question is now how to change the read timeout duration of my client. – Towill

Answer 1

It seems that the configuration values are not injected in the manually created http clients.

A solution is to configure the HttpClient at creation, setting the readTimeout duration:

private BlockingHttpClient getTaskClient() {
    HttpClientConfiguration configuration = new DefaultHttpClientConfiguration()
    configuration.readTimeout = Duration.ofSeconds(30)
    new DefaultHttpClient(taskStepperUrl.toURL(), configuration).toBlocking()
}

Towill

I'd assume that the default client config (the one you define with micronaut.http.client) can be injected too. – Lobachevsky

Yes, using the Config object I could inject it the same way I do for the 'taskStepperUrl' – Towill

Answer 2

micronaut.http.client.read-timeout takes a duration, so you should add a measuring unit to the value, like 30s, 30m or 30h.

Mannos

Answer 3

In my case I was streaming a file from a client as

  @Get(value = "${service-path}", processes = APPLICATION_OCTET_STREAM)
  Flowable<byte[]> fullImportStream();

so when I got this my first impulse was to increase the read-timeout value. Though, for streaming scenarios the property that applies is read-idle-timeout as stated in the docs https://docs.micronaut.io/latest/guide/configurationreference.html#io.micronaut.http.client.DefaultHttpClientConfiguration

Hallagan