Industry standards for measuring the capacity of an Ethernet network segment are not applied consistently by manufacturers of data networking products or by manufacturers of test equipment. This state of affairs can lead to some odd and misleading results. These unsatisfactory results can often be explained by understanding what has actually been measured.
Frames and datagrams
For some reason lost in the mists of time, Ethernet packets are called "frames". This helps to distinguish them from their payload of IP packets, which are referred to as "datagrams". We'll use that terminology here because it helps to avoid confusion, and to be honest we need all the help we can get.
Ethernet Layer 1
Layer 1 describes the physical layer of the communications medium. In the case of Ethernet, it refers to the electrical signals on the wire, or the flashes of light in the fiber. Ethernet frames on the wire are separated by an inter-frame gap that is never less than 12 bytes, and every frame starts with a preamble of seven bytes and a start of frame delimiter of one byte. This adds up to an additional 20 bytes that make up the Layer 1 overhead of the Ethernet standard. We refer to this as an overhead because the 20 bytes are essential for the transmission and reception of frames, but they don't actually carry useful data.
Let's try to illustrate this with a worked example:
The smallest permitted Ethernet frame is 64 bytes. The highest frame rate we can achieve on Fast Ethernet (100BASE-TX) is 100 x 1,000,000 / ((64 + 20) x 8) = 148,810 frames per second.
Ethernet Layer 2
Layer 2 is the layer concerned with transferring data between adjacent nodes. In the case of Ethernet, we're interested in the Ethernet frame starting at the beginning of the Destination Address, up to the end of the Frame Check Sequence (FCS). The FCS is simply a checksum on the rest of the frame.
It makes a lot of sense to measure the capacity of an Ethernet network at Layer 2. In fact, this is the definition preferred by the Metro Ethernet Forum (MEF) and the International Telecommunication Union (ITU). It's also the definition used to specify capacity in Cambium products.
Back to our worked example:
We have 148,810 frames per second as the maximum for 100BASE-TX. This delivers 148,810 x 64 x 8 = 76,190,467 bits/s at Layer 2. So, we have the surprising result that Fast Ethernet has a capacity as low as about 76.2 Mbit/s using the preferred Layer 2 definition of capacity. Of course, the gap is greatest for small frame size; if we repeat the calculation using a larger 1518 byte frame size we see that Fast Ethernet has a capacity of 98.7 Mbit/s. This is better, but still less than we might expect for a medium advertised as 100 Mbit/s.
The same calculations can be applied to Gigabit Ethernet. We just used 100BASE-TX here to make the numbers easier to read.
IP Layer (Layer 3)
Layer 3 describes the IP datagrams that are carried as the payload in Ethernet frames. The Ethernet frame includes 18 bytes of headers in addition to the IP datagram. In a network with VLANs, that Ethernet overhead increases to 22 bytes.
If we return to our worked example with Fast Ethernet and 64 byte Ethernet frames, the capacity at Layer 3 is 148,810 frames per second where each frame contains 64 – 18 = 46 bytes of payload. This represents a Layer 3 capacity of 148,810 x 46 x 8 = 54,762,080, or about 54.8 Mbit/s.
When reading manufacturers’ specifications for data capacity, it pays to check what the numbers are intended to indicate. Layer 1 and Layer 2 are both common in specifications. Also, when making measurements of data capacity, it pays to check what the test equipment is actually measuring. This could be Layer 1, Layer 2 or Layer 3.
Check out Capacity measurement using iperf
Check out Capacity measurement using RFC2544