How to set the maximum TCP Maximum Segment Size on Linux?

Question

In Linux, how do you set the maximum segment size that is allowed on a TCP connection? I need to set this for an application I did not write (so I cannot use setsockopt to do it). I need to set this ABOVE the mtu in the network stack.

I have two streams sharing the same network connection. One sends small packets periodically, which need absolute minimum latency. The other sends tons of data--I am using SCP to simulate that link.

I have setup traffic control (tc) to give the minimum latency traffic high priority. The problem I am running into, though, is that the TCP packets that are coming down from SCP end up with sizes up to 64K bytes. Yes, these are broken into smaller packets based on mtu, but this unfortunately occurs AFTER tc prioritizes the packets. Thus, my low latency packet gets stuck behind up to 64K bytes of SCP traffic.

This article indicates that on Windows you can set this value.

Is there something on Linux I can set? I've tried ip route and iptables, but these are applied too low in the network stack. I need to limit the TCP packet size before tc, so it can prioritize the high priority packets appropriately.

Glosseme

I find it hard to believe that packets end up with 64k in the tc queue, tc doesn't work on the TCP level, it cares about packets, not TCP segments – Gauthier

I have to agree. From my reading the TCP segments should be broken up before tc. However, that is not what I see (both in latency and in tcpdump). When a large TCP segment goes out, it delays the high priority channel until all (up to 64K bytes) of the large segment are sent. – Glosseme

Answer 1

Are you using tcp segmentation offload to the nic? (You can use "ethtool -k $your_network_device" to see the offload settings.) This is the only way as far as I know that you would see 64k tcp packets with a device MTU of 1500. Not that this answers the question, but it might help avoid misdiagnosis.

Inclose

Answer 2

ip route command with option advmss helps to set MSS value.

ip route add 192.168.1.0/24 dev eth0 advmss 1500

Tangle

Answer 3

The upper bound of the advertised TCP MSS is the MTU of the first hop route. If you're seeing 64k segments, that tends to indicate that the first hop route MTU is excessively large - are you using loopback or something for testing?

Janettejaneva

Nope. The first hop MTU is 1500. I see the large packets in a tcpdump output (on the transmit side). If I also run a tcpdump on the receive side, I see the large packets broken into many 1500 byte packets. – Glosseme

Answer 4

MSS = MTU – 40bytes (standard TCP/IP overhead of 40 bytes [20+20])

If the MTU is 1500 bytes then the MSS will be 1460 bytes.

Buprestid

Answer 5

You are definitely misdiagnosing the problem; as someone else pointed out, tc doesn't see TCP packets, it sees IP packets, and they'd already be in chunks at that point.

You are probably just experiencing bufferbloat: you're overloading your outbound queue in a totally separate device (probably a DSL modem or cable modem). The only fix is to tell tc to limit your outbound bandwidth to less than the modem's bandwidth, eg. using TBF.

Rocher