Reducing connection wait time for APIs in Flutter

Question

I'm trying to optimize API response time for my Flutter app. I have an API where I've added server level caching and I'm using Dio for my API needs.

The reponse time when tested on Postman is ~100ms. Getting similar metrics on Chrome developer tools.

The problems is when I measure the response time on Flutter DevTools, all responses are >250ms. Upon checking, I find that there's a connection wait time of ~200ms for every request.

How can I reduce the above wait time for Flutter?

I tried switching libraries and reusing the same https client as suggested here and in the docs of http but the wait time for subsequent requests remained similar.

var client = http.Client();
final res = await client.get(...);

How can I avoid these connection delays and get the optimum performance? If caching the SSL, DNS Lookup etc is the solution, how do I do that in flutter?

Diffusivity

Please refer to this answer here: stackoverflow.com/a/69268170. You have to create only one single httpClient object & then reuse it. You shouldn't be creating http client just before every API call. The first http client will take time & then subsequent requests will be faster. – Concerned

Hi, thanks for the suggestion. Unfortunately, I've already tried the approach from the link - it does help bring down the wait time for subsequent requests but does not reduce it further after 150ms. I'm trying to reduce it even further, just like Postman is able to reduce it to zero. The link you shared is mentioned as a ref in the question as well. – Diffusivity

Answer 1

I have figured out the exact cause of the above issue.

TLDR: Dio uses socket-based dart:io HttpClient which has a default idleTimeout of 15sec. But this timeout was overridden to 3s in the lib and this is why all requests with a delay of more than 3s between them had to open another socket connection. This led to a significant connection wait time before each request.

---

Code solution:

Add package dio_http2_adapter

final dio = Dio()
    ..options.baseUrl = 'https://example.com'
    ..httpClientAdapter = Http2Adapter(
      ConnectionManager(idleTimeout: Duration(seconds: 15)),
    );

---

I wrote a Dart program to repetitively hit an endpoint. I noticed that after the first call, all requests had a connection wait time of <1ms. Also, there was just one entry of type ws for the 13 GET requests I made. See the image below.

I added a delay to 4s between each request and noticed that now there was a connection wait time for each request and that the number of socket connections(type ws) were equal to the number of requests i.e 13. See image below.

Upon digging more, I found out that by default, Dio uses socket-based dart:io HttpClient under the hood for making all the requests. This means the socket connection above was used for making requests by Dio and the connection closed after 3s.

Looking through the code of HttpClient and Dio lib, I found a parameter - idleTimeout. Its default value was 15s but was overridden to 3s in Dio and this is why all requests with a delay of more than 3s between them had to open a new socket connection. This led to a significant connection wait time before each request.

Diffusivity

You're right! On applying this fix i noticed a significant improvement to my api calls. Now I can't understand why the package maintainers reduced the idleTimeout to less than 15secs – Francenefrances

You mentioned you'd like to reduce the wait time to <150ms (just like postman) instead of creating a single httpClient object & then reuse it. Does this solution work for this as well? – Francenefrances

I've used a mix of both approaches. I'm reusing the client as well as overriding the idleTimeout parameter via Http2Adapter adapter as a plugin. – Diffusivity

Answer 2

In case of Dio we can override the keep alive time using custom httpClient

_dio = Dio(baseOptions)
      ..httpClientAdapter = Http2Adapter(
        ConnectionManager(idleTimeout: 150000),
      );

We also have to add dio_http2_adapter package

dio_http2_adapter:latest

by default Dio has 3sec timeout

Carmoncarmona