Reverse proxy with keepalive and re-resolve DNS according to TTL in nginx

Upstream blocks in nginx open source allow you to enable keepalive connections to reverse-proxied servers (docs: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive).

Syntax: keepalive connections;
Default: —
Context: upstream
Activates the cache for connections to upstream servers.

The connections parameter sets the maximum number of idle keepalive connections to upstream servers that are preserved in the cache of each worker process.

However, using upstream blocks in nginx open source has some caveats, notably: DNS is cached until restart or reload, regardless of TTL. This means that if you used upstream blocks with keepalives to reverse proxy to i.e. an AWS load balancer, you would experience downtime when the load balancer IPs change unless a human/service reloaded or restarted nginx. "Setting the Domain Name in a Variable" (see below), allows DNS to be re-resolved at the TTL interval or any other desired interval, but does not appear to let you enable keepalive connections.

Is it possible to enable keepalive connections while reverse proxying to another server by domain name in nginx open source and also re-resolve DNS according to TTL (without server reload/restart)? (Nginx Plus supports re-resolving DNS in upstream blocks). If this isn't possible using nginx open source alone, would it be best to move DNS resolution to another layer like HAProxy and point to that layer in an nginx upstream block?

nginx keepalive and dns resolver is pretty much the same question with the following difference: here, using a plugin or a different layer is fine if this can't be accomplished in nginx itself. One answer from that question mentions using https://github.com/wdaike/ngx_upstream_jdomain, but this plugin doesn't appear to have been changed in years, so I suspect moving DNS resolution to another layer would be easier to maintain going forward.

A similar question came up in the following comment on a related question: Why does nginx proxy_pass close my connection?

In https://www.nginx.com/blog/dns-service-discovery-nginx-plus/, three methods of DNS service discovery are mentioned for nginx, and it looks like none of them both allow for re-resolving DNS without nginx restart/reload and using upstream params such as keepalive:

Using the Domain Name in the proxy_pass Directive
server {
    location / {
        proxy_pass http://backends.example.com:8080;
    }
}
...

This method is the least flexible way to do service discovery and has the following additional drawbacks:

If the domain name can’t be resolved, NGINX fails to start or reload its configuration.

NGINX caches the DNS records until the next restart or configuration reload, ignoring the records’ TTL values. [emphasis mine]

We can’t specify another load‑balancing algorithm, nor can we configure passive health checks or other features defined by parameters to the server directive, which we’ll describe in the next section.

Using a Domain Name in an Upstream Server Group
upstream backends {
    least_conn;

    server backends.example.com:8080 max_fails=3;
}

server {
    location / {
        proxy_pass http://backends;
    }
}
...

Though this method enables us to choose the load‑balancing algorithm and configure health checks, it still has the same drawbacks with respect to start, reload, and TTL as the previous method. [emphasis mine]

Setting the Domain Name in a Variable
resolver 10.0.0.2 valid=10s;

server {
    location / {
        set $backend_servers backends.example.com;
        proxy_pass http://$backend_servers:8080;
    }
}
...

This method eliminates two drawbacks of the first method, in that the NGINX startup or reload operation doesn’t fail when the domain name can’t be resolved, and we can control how often NGINX re‑resolves the name. However, because it doesn’t use an upstream group, you can’t specify the load‑balancing algorithm or other parameters to the server directive (as we did in the second method). [emphasis mine]

Using the Domain Name in the proxy_pass Directive

Using a Domain Name in an Upstream Server Group

Setting the Domain Name in a Variable

Recommended topics

Hot tags