Why does the socket's buffer fill up sooner than expected?
From Paul W. Nelson (email@example.com):
In the traditional BSD socket implementation, sockets that are atomic
such as UDP keep received data in lists of mbufs. An mbuf is a fixed
size buffer that is shared by various protocol stacks. When you set
your receive buffer size, the protocol stack keeps track of how many
bytes of mbuf space are on the receive buffer, not the number of
actual bytes. This approach is used because the resource you are
controlling is really how many mbufs are used, not how many bytes are
being held in the socket buffer. (A socket buffer isn't really a
buffer in the traditional sense, but a list of mbufs).
For example: Lets assume your UNIX has a small mbuf size of 256
bytes. If your receive socket buffer is set to 4096, you can fit 16
mbufs on the socket buffer. If you receive 16 UDP packets that are 10
bytes each, your socket buffer is full, and you have 160 bytes of
data. If you receive 16 UDP packets that are 200 bytes each, your
socket buffer is also full, but contains 3200 bytes of data. FIONREAD
returns the total number of bytes, not the number of messages or bytes
of mbufs. Because of this, it is not a good indicator of how full
your receive buffer is.
Additionaly, if you receive UDP messages that are 260 bytes, you use
up two mbufs, and can only recieve 8 packets before your socket buffer
is full. In this case, only 2080 bytes of the 4096 are held in the
This example is greatly simplified, and the real socket buffer
algorithm also takes into account some other parameters. Note that
some older socket implementations use a 128 byte mbuf.
6. Advanced Socket Programming