HAProxy dynamic server addresses

Question

We have Similar setup to this diagram

Where request arrives to HAProxy, it get's roundrobin balanced to any servers, backend server checks its cache and if resource is not on that server it issues redirect with header set to the correct server IP.

Second time request arrives to HAProxy, it detects that the header with backend server is there, but how can I take that IP and direct request directly to it?

For example, second time request arrives to haproxy it has header X-BACKEND-IP=10.0.0.5

So instead haproxy trying to load balance that request, I want it to read the header, take that IP and go directly to that backend.

Is that possible? If not, would it be possible with nginx ?

Drumhead

So your original behavior is: 1) the HAProxy receives the original HTTP request -> 2) detects that the requested resource is not available -> 3) Forwards the original request with an additional HTTP header field X-BACKEND-IPwith an available server to itself -> 4) detects that the requested resource is available -> 5) sends request to resource. Your goal is to eliminate step 3 and 4? – Ironwood

@LiamKelly Not really, the HAProxy does not set the header nor does the redirection. That is all done by the backend service itself. 1) HAPRoxy receives request -> 2) "blindly" round-robin it to any backend next in line -> 3) the backend service itself decides if it can serve that request or what other backend it should go to. If it decides it should go somewhere else, it sets the X-BACKEND-IP header and redirects back to haproxy (all fine up to this point) -> 4) HAProxy will take that header and route directly to the IP from the header – Drumhead

If I am understanding this correctly, it seems most efficient for the backend service to send the request to the final server directly. The first backend service knows where the request should go, so why burden HAProxy? If the two backend servers are not directly routable, you can put an ip-forwarding rule (OSI Layer 3, not a request proxy rule) to still use the HAProxy machine as a central routing point. – Ironwood

Looking for sticky sessions? blog.haproxy.com/2012/03/29/… – Saintmihiel

@DmitryMiksIr Unfortunately sticky sessions in this case won't work – Drumhead

@LiamKelly Any idea if thats possible with websockets? #43318077 – Drumhead

@Tom a kernel, IP-forwarding route will work with Web Sockets. At a lower level, Web Sockets are just TCP/IP connections. – Ironwood

@LiamKelly I will need to have a think about that, I think forwarding would occur too early to be able to get some logic in – Drumhead

Answer 1

Assuming you're happy with trusting the IP in the header of the second request, then yes, you can do it with use-server:

backend bk_foo
  [...]
  server srv_0a_00_01_05 10.0.1.5:80 weight 100
  server srv_0a_00_02_05 10.0.2.5:80 weight 100
  use-server %[req.hdr(x-backend-ip),lower,map_str(/etc/haproxy/hdr2srv.map,srv_any)] if { req.hdr(x-backend-ip),lower,map_str(/etc/haproxy/hdr2srv.map) -m found }

Contents of /etc/haproxy/hdr2srv.map:

#ip srv_name
# hex of IP used for names in this example
10.0.1.5  srv_0a_00_01_05
10.0.2.5  srv_0a_00_02_05

If you need to down one of the servers, you should dynamically update the map to remove it, so that the requests with the header set get redirected again.

If you have multiple backends, you can do similar with use_backend.

Jute

Yes use-server is exactly what I was looking for. In the end I won't use this unfortunately but it is the correct answer to my question – Drumhead

Actually.. when I put code from your answer to haproxy config it gives error unable to find server '%[req.hdr(x-backend-address),lower,map_str(/etc/haproxy/hdr2srv.map,srv_any)]' referenced in a 'use-server' rule. – Drumhead

@Tom What version of HAProxy are you using? I'll try and replicate the issue here. I know i've deployed with use_backend before. – Jute

Using 1.6.3, yes use_backend works but I would prefer to make user-server work – Drumhead

Hmm, you're right, digging in the codebase. Presently use-server does not expand a logformat string like use_backend does. Poke the haproxy mailing list and it should be fairly easy to get implemented, but will probably be limited to the 1.7 or 1.8 series to avoid major changes in the old codebase. – Jute