What is a good buffer size for socket programming?

Question

We are using .Net and sockets. The server is using the Socket.Sender(bytes[]) method so it just sends the entire payload. On the other side we are clients consuming the data. Socket.Receive(buffer[]). In all the examples from Microsoft (and others) they seem to stick with a buffer size of 8192. We have used this size but every now and then we are sending data down to the clients that exceeds this buffer size.

Is there a way of determining how much data the server's sent method sent us? What is the best buffer size?

Herald

Answer 1

Even if you're sending more data than that, it may well not be available in one call to Receive.

You can't determine how much data the server has sent - it's a stream of data, and you're just reading chunks at a time. You may read part of what the server sent in one Send call, or you may read the data from two Send calls in one Receive call. 8K is a reasonable buffer size - not so big that you'll waste a lot of memory, and not so small that you'll have to use loads of wasted Receive calls. 4K or 16K would quite possibly be fine too... I personally wouldn't start going above 16K for network buffers - I suspect you'd rarely fill them.

You could experiment by trying to use a very large buffer and log how many bytes were received in each call - that would give you some idea of how much is generally available - but it wouldn't really show the effect of using a smaller buffer. What concerns do you have over using an 8K buffer? If it's performance, do you have any evidence that this aspect of your code is a performance bottleneck?

Airfield

I would like to increase the buffer size if it all possible. We were contemplating just making it 32K but I was worried about the above scenario where I the receive command would not get everything even though I sent it in a single Send. – Herald

Sorry i forgot to mention that we are doing this in blocking mode. I don't see it making a difference though. – Herald

@uriDium: You should write your code so that it doesn't rely on receiving everything from a single Send in one Receive. That's not the model that TCP uses - it's a stream model. If you want to segment that stream into distinct messages, you should either use a delimiter or write a length prefix for each message so that the client knows how much to read. – Airfield

We use 0128 as prefix for # of bytes to pull off the buffer. That has helped to establish a nice framing around the messages. Most vendors we interface to are familiar with this practice. You could do something similar. – Splash

@JonSkeet Why don't we use a fixed-length length specifier bytes at the beginning of the "message" which specifies the length of the message, and then append the message? that would work too, right ? – Woodwork

@Tarek: Yes, that's what I suggested in "or write a length prefix for each message so that the client knows how much to read" – Airfield

Answer 2

Jon Skeet's answer unfortunately leaves a big part of the picture out - the buffer sizes for the send and receive buffers, and the bandwidth-delay product of the pipe you're writing to.

If you are trying to send data over a large pipe using a single socket, and you want TCP to fill that pipe, you need to use a send buffer size and receive buffer size that are each equivalent to the bandwidth-delay product of the pipe. Otherwise, TCP will not fill the pipe because it will not leave enough 'bytes in flight' at all times.

TCP handles packet loss for you, which means that it has to have buffers to hold onto the data you give it until it can confirm that data has been received correctly by the other side (by a TCP ACK). No buffer is infinite, therefore there has to be a limit somewhere. That limit is arbitrary, you get to choose whatever you want, but you need to make sure it is large enough to handle the connection's BDP.

Consider a TCP socket that has a buffer size of exactly: 1 byte. And you're trying to send data over a connection that has a bitrate of 1 gbit/sec and a one-way latency of 1 ms.

1. You give the TCP socket your first byte.
2. The socket blocks any further write calls (the send buffer is full).
3. TCP sends the one byte. The gig eth adapter has a bulk transmission rate of 8 ns per byte, so the transmission time is negligible.
4. 1 millisecond later, the receiver gets the 1 byte.
5. 1 millisecond later, you get an ack back from the receiver.
6. TCP removes the first byte from the send buffer, because it has confirmed the receiver correctly got the byte.
7. The send buffer unblocks because it has room.
8. You give the TCP socket your second byte...
9. and so on.

How fast is this connection getting data across? It takes 2 milliseconds to send 1 byte, therefore, this connection is getting a bitrate of 500 bytes/sec == 4 kbit/sec.

Yikes.

Consider a connection that has a speed of 1 gigabit, and has a one-way latency of 10 milliseconds, on average. The round-trip-time (aka, the amount of time that elapses between your socket sending a packet and the time it receives the ack for that packet and thus knows to send more data) is usually twice the latency.

So if you have a 1 gigabit connection, and a RTT of 20 milliseconds, then that pipe has 1 gigabit/sec * 20 milliseconds == 2.5 megabytes of data in flight at all time if it's being utilized completely.

If your TCP send buffer is anything less than 2.5 megabytes, then that one socket will never fully utilize the pipe - you'll never get a gigabit/sec of performance out of your socket.

If your application uses many sockets, then the aggregate size of all TCP send buffers must be 2.5 MB in order to fully utilize this hypothetical 1 gigabit/20 ms RTT pipe. For instance, if you use 8192-byte buffers, you need 306 simultaneous TCP sockets to fill that pipe.

Edit for questions:

Calculating BDP is just multiplying the Bandwidth times the Round-trip Delay and paying attention to units.

So if you have a 1 gigabit/sec connection, and a round-trip time of 20 msecs, then what happens is you're multiplying Bits/Sec * Seconds, so the seconds cancel out and you're left with Bits. Convert to Bytes and you have your buffer size.

• 1 gbit/sec * 20 msec == 1 * gbit/sec * 0.02 sec == (1 * 0.02) gbit
• 0.020 gbit == 20 MBit.
• 20 Mbit * 1 Byte / 8 bits == 20 / 8 MBytes == 2.5 MBytes.

And thus, our TCP buffer needs to be set to 2.5 MB to saturate this made-up pipe.

Scottiescottish

Thanks for the explanations. So a good buffer size is between 8Ko and 2.5 MB. ;-) – Marks

It should be noted, that this really only applies to TCP sockets as here the other side will only send as much data as it assumes to fit into the recipient socket buffer before pausing the data flow and waiting for ACKs to arrive. In case of UDP it is only important that the code will never let the socket buffer run empty, then you will also utilize whatever line speed is available. Of course, if the buffer is bigger, you can produce more data in advance and thus timing will be less critical. – Likable

@Scottiescottish How is this computed? 1 gigabit/sec * 20 milliseconds == 2.5 megabytes of data – Hypothec

@Hypothec - I wrote some edits for you to show the math. Cheers! – Scottiescottish

Answer 3

It depends upon your protocol. If you are expecting messages in excess of 8192 bytes, then you should increase your buffer size accordingly. But keep in mind this buffer size is only for one call to Receive. If you really want/need to, you can loop over Receive multiple times and copy the received data into an arbitrarily large data structure or buffer.

Also keep in mind it is good practice to call Receive repeatedly until you have verified that you have read all of the data for a given message; even if a single message is less than your buffer size, it still might not all be retrieved by a single Receive call.

Instable

Answer 4

Not really Microsoft related, but I am just experimenting with a C++ threaded echo server using a TCP port (not Unix domain socket) to see the throughput. Timing a 4M input with various buffer sizes gave the following results:

1024 - real 0m0,102s; user  0m0,018s; sys   0m0,009s
2048 - real 0m0,112s; user  0m0,017s; sys   0m0,009s
8192 - real 0m0,163s; user  0m0,017s; sys   0m0,007s
 256 - real 0m0,101s; user  0m0,019s; sys   0m0,008s
  16 - real 0m0,144s; user  0m0,016s; sys   0m0,010s

Seems reading in 1024 byte chunks reduces the TCP overhead while the processing time (just echoing the input back) was not affected by the buffer size. 8192 bytes seem high and really low values (like 16) are not good either.

Sparerib

Answer 5

8192 would be ideal. If you have data which exceed this size it would be better of you to send the data in constant length packets.

The size of data that is sent by server can be checked using the recv function in WINSOCK which has a parameter that gives length of buffer.

Matte

8192 is not 'ideal', but a generally useful number. It is not better to send the data in constant-length packets: it is better to stream it all as fast as possible. The size of data that is sent doesn't necessarily have anything to do with the value returned by recv() at the peer. It's a streaming protocol. – Furtek