What's "tcp-backlog" in redis.conf

Question

I'm confused by tcp-backlog in redis.conf:

# TCP listen() backlog.
#
# In high requests-per-second environments you need an high backlog in order
# to avoid slow clients connections issues. Note that the Linux kernel
# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
# in order to get the desired effect.
tcp-backlog 511

Is tcp-backlog the size of "complete connection queue" (three-way handshake complete, what is described here) or "incomplete connection queue"?

If it means "complete connection queue" then why should I raise tcp_max_syn_backlog which limits the size of an incomplete connection queue?

Ruberta

Good question. It is the completed connection queue. The SYN backlog is separate. I see no reason why they should need to track. – Lobeline

Answer 1

Is tcp-backlog the size of "complete connection queue" (three-way handshake complete, what is described here) or "incomplete connection queue"?

tcp-backlog is the size of complete connection queue. In fact, Redis passes this configuration as the second parameter of the listen(int s, int backlog) call.

@GuangshengZuo already had a good answer for this question. So I'll focus on the other one.

If it means "complete connection queue" then why should I raise tcp_max_syn_backlog which limits the size of an incomplete connection queue?

Quote from the doc you mentioned:

The implementation uses two queues, a SYN queue (or incomplete connection queue) and an accept queue (or complete connection queue). Connections in state SYN RECEIVED are added to the SYN queue and later moved to the accept queue when their state changes to ESTABLISHED, i.e. when the ACK packet in the 3-way handshake is received. As the name implies, the accept call is then implemented simply to consume connections from the accept queue. In this case, the backlog argument of the listen syscall determines the size of the accept queue.

We can see that items in complete connection queue are moved from the incomplete connection queue.

If you have a large somaxconn with a small tcp_max_syn_backlog, then you might NOT have enough items to be moved to the complete connection queue, and the complete connection queue might never be full. Many requests might have already been dropped from the first queue before they have the chance to be moved to the second.

So only raise the value of somaxconn might NOT work. You have to raise both of them.

Mediterranean

Answer 2

The tcp-backlog is the size of accept queue or complete connection queue.

As the docs you mentioned said:

On Linux, things are different, as mentioned in the man page of the listen syscall:

The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests. The maximum length of the queue for incomplete sockets can be set using /proc/sys/net/ipv4/tcp_max_syn_backlog.

This means that current Linux versions use the second option with two distinct queues: a SYN queue with a size specified by a system wide setting and an accept queue with a size specified by the application.

Redis server use the config of tcp-backlog to specify the size of the accept queue with listen(). And the SYN queue size is determined by the admin of the linux.

If it means "complete connection queue", then why should I raise tcp_max_syn_backlog which limits the size of incomplete connection queue?

Raising the tcp_max_syn_backlog is aimed to avoid the slow client connection issues. If there are some slow client which are doing 3-way handshake with redis server, and these client are slow to read the response and send request, they will took the redis server's SYN queue for a long time because they are slow.

And in some case, the SYN-queue will be filled because these inefficient clients. If the SYN queue is filled, the redis server can not accept the new clients. So you should raise the tcp_max_syn_backlog to deal with this.

Diapedesis

Answer 3

Yesterday , I read Chapter 6 in Redis In Action where Joshua Carlton writes that " One of the drawbacks of the Redis Publish and Subscribe model are that the client must be connected at all times to receive messages, disconnections can cause the client to lose messages, and older versions of Redis could become unusable, crash, or be killed if there was a slow subscriber."

Then, Joshua Carlton states that , "Though push messaging can be useful, we run into problems when clients can’t stay connected all the time for one reason or another. To address this limitation, we’ll write two different pull messaging methods that can be used as a replacement for PUBLISH/SUBSCRIBE. We’ll first start with single-recipient messaging, since it shares much in common with our first-in, first-out queues. Later in this section, we’ll move to a method where we can have multiple recipients of a message. With multiple recipients, we can replace Redis PUBLISH and SUBSCRIBE when we need our messages to get to all recipients, even if they were disconnected" We are interested to know whether it would be more performant to replace Redis PUBLISH and SUBSCRIBE with Joshua Carlton's Section 6.5.2 Multiple-recipient publish/subscribe replacement instead of harnessing the UDP protocol to detect and repair the disconnection loss.

 Could we set a high tcp_max_syn_backlog in redis.conf to prevent either of

Joshua Carlson's single-recipient messaging, and multiple -recipient messaging methods from disconnecting under a load of 20,000 messages per second where each message is 20 bytes?

Phifer

@GuangshengZuo, What tcp_max_syn_backlog in redis.conf to would prevent either of Joshua Carlson's single-recipient messaging, and multiple -recipient messaging methods from disconnecting under a load of 20,000 messages per second where each message is 20 bytes? Thank you. – Phifer