Understanding IP Production Networks: Part 12 - Measuring Line Speeds

Broadcast and IT engineers take very different approaches to network speed and capacity; it is essential to reach a shared understanding.

In the analog days, broadcast engineers used frequency response to measure line capability. An audio twisted pair had a bandwidth of at least 20KHz and video had a bandwidth of approximately 8MHz. As we moved to digital audio AES3 and SDI HD these requirements increased to 3Mb/s and 1.485Gb/s respectively.

Digital audio and video systems send high and low voltages to represent the ones and zeros. However, at the higher frequencies we need to take into consideration the analog qualities of the transmission cable and equalizing circuits. Return loss, reflections and phase distortion are all significant factors needing consideration.

When speaking to the IT department, we might think that we should no longer worry about these qualities as signal paths are defined in bits per second. An internet line might be defined as 200Mb/s and an SFP link might be defined as 1Gb/s.

IT engineers tend to be product specialists in their own fields of Microsoft, Linux, and Cisco. Very few of them will study transmission theory in the way broadcast engineers have in the past, especially those who worked in transmitters. This can lead to some very frustrating and confusing conversations between IT and broadcast engineers.

An IT engineer might tell you that the bandwidth of a circuit is 200Mb/s, or the delay is negligible. At this point a broadcast engineers’ blood would boil as they have flashbacks to their Telegrapher’s equations and two port networks. The simple answer is that IT engineers think in terms of service level agreements, if a Telco has provided a circuit with 200Mb/s capacity then they assume and expect it to be true (on average).

Capacity Considerations

IT engineers think of network capacity in terms of bits per second, as opposed to broadcast engineers who think in terms of bandwidth, return loss, phase distortion and reflections. Further problems arise when we start discussing actual data throughput in a system.

Broadcast engineers will assume a 10MHz point to point network will allow them to send signals with 10MHz bandwidths. As IT networks are based on packets of data, there is an inherent loss of data in the system due to the headers and spaced distribution. A 10Mb/s circuit might only have a useful data-rate of 9.5Mbps.

Ethernet frames are generally split into two parts, the header and payload. The header includes information such as send and receive addresses, payload types, packet counts and flags. The payload will be a protocol type such as IP, which will also contain a header and payload.

Drilling down into the Ethernet packet we have the four octets of Cyclic Redundancy Check (CRC), and twelve octets of inter-packet gap appended to the end of the frame.

When discussing networks, engineers use octets instead of bytes to represent eight bits. This is to remove any ambiguity as computer science tells us a byte is “a unit of information”, which is based on the hardware design of the computer, and could just as easily be eight bits, ten bits or one hundred bits.

Figure 1 - Ethernet packets have an overhead from the header and CRC reducing data throughput.

Figure 1 - Ethernet packets have an overhead from the header and CRC reducing data throughput.

Our Ethernet frame generally consists of 1,542 octets, or 12,336 bits. Only the payload is available to us when sending audio and video which is 1,530 octets, or 12,240 bits. If a camera uses UDP/IP to send its data over Ethernet a further 20 octets are lost in the Ethernet payload to the UDP header and CRC information leaving 1,510 octets, or 12,080 bits, all resulting in approximately 98% data throughput.

The theoretical maximum of 98% assumes no congestion and evenly spaced packets. The IT engineer is expecting 200Mb/s, but the Telco is providing 196Mb/s of usable data. Clearly the Telco will dispute this as they will show they are providing a 200Mb/s circuit, and the fact that you are using 2% of it on packet header information is your problem not theirs. To the letter of the contract, they are probably correct.

This may not sound a lot, but if you suddenly find your network has a 2% reduction in capacity, you could find yourself with some tough questions to answer when approaching the Finance Director for more cash.

Protocols

When analyzing network throughput, a thorough understanding of protocols must be achieved.

Transmission Control Protocol (TCP) sits on top of IP/Ethernet and provides a reliable connection-based system to allow two devices to exchange data. Low latency video and audio distribution doesn’t use TCP/IP for standards such as ST2110; its data integrity is very high, but the throughput can be low and highly variable. However, some video and audio distribution systems will use TCP/IP and understanding how it works is critical when considering data throughput and latency.

TCP operates by sending a group of packets and then waits for an acknowledge packet from the receiver, if the “Ack” isn’t received then the same packets are resent, eventually timing-out and sending an error to the user if too many of these errors happen in succession. If the “Ack” is received by the sender, then the next group of packets are sent. Data throughput is reduced and is the price we pay for this data guarantee.

UDP/IP protocols, which ST2022 and ST2110 uses to transport video and audio, is a “fire and forget” system.  The camera outputs the packets and doesn’t use any form of testing to see if it was received at the destination, such as a production switcher. ST2022-5 has forward error correction (FEC) built into it to provide some resilience over the network. 

Figure 2 - IPerf line speed testing.

Figure 2 - IPerf line speed testing.

“IPerf” is used to actively measure the maximum achievable bandwidth in an IP network; it’s the closest tool we have for measuring a network’s capacity and is released under the BSD license. It runs on two computers, a transmit at one end of the network and receive at the other. It works by flooding the link with IP datagrams and using the receiver to check their validity. Consequently the measurement is potentially destructive to other users of the network and must be used in isolation.

Operating as a command line tool IPerf can make many different measurements, from UDP/IP line-speed to TCP throughput. TCP measurements will always be slower as IPerf will be measuring the speed of the protocol, not the network line-speed.

Controlling The Flow

Another system in use in IP networks is the rate-shaper. This is a device or system that caps the amount of sending traffic that can leave a network, router or switch to prevent congestion or to comply with SLAs (Service Level Agreements).

SLAs are generally used by Telco’s and other network providers to guarantee maximum datarates for specific services. Without shaping, devices employing “greedy” protocols could consume the entire network, stopping other services from being able to send data. This is particularly important in packet switched networks such as IP and Ethernet and is something broadcast engineers would probably not have encountered with circuit switched networks where the bandwidth is limited and guaranteed by the electronics employed.

The smoothing effect of the rate-shaper also stops bursts of data from oversubscribing the link and causing buffer overflow, resulting in packet loss. Micro-bursts can overflow buffers as their short term datarate can significantly peak to oversubscribe a link. Buffers in the routers and switches smooth these out, but the buffer has a limited size and cannot be stressed indefinitely.  It’s important to remember that packet switched networks rarely exhibit evenly gapped packets (with the exception of ST2110), and we cannot take for granted that a 50Mb/s service will continually send 50Mb/s – it may peak to 100Mb/s or more, and drop to 0Mb/s. This is a feature of packet switched design as IP networks were originally designed to transfer short bursts of data.

Rate-shaping helps mitigate the issues caused by this type of packet activity and operate at multiple levels such as class based and per service. Class based service rate-shaping is a type of traffic engineering where IP packets are grouped together that share common handling rules, but they are protocol agnostic and work at the queue scheduling level generally within a switch or router. This might be for a specific link such as MPLS or VLAN, where QoS limitations can be enforced to improve packet distribution to operate within the characteristics of those links.

Protecting the network from misbehaving senders is another role for rate-shapers. Broadcasters can easily overcome the inherent latency issues of TCP/IP by using UDP/IP senders. However, when using UDP/IP, the responsibility is on the sender to make sure it is not flooding the network with packets and is behaving in a well-ordered manner. If this is not the case, then congestion can occur, resulting in other devices not being able to send their data. A link may have a capacity of 100Mb/s, but if a video service using UDP/IP, without any form of congestion control it could burst the data and consume the whole bandwidth for significant periods of time. The key here is understanding the difference between the average datarate, and the instantaneous datarate. Rate-shaping will help to reduce the instantaneous datarate while keeping the average datarate the same, thus enabling other devices access to the network link.

There are no hard and fast rules of how UDP/IP devices should behave within a private network, other than being “fair”. If multiple devices are bursting high volumes of data from devices such as cameras, then packet collisions can easily occur resulting in packet drop,causing picture distortion and massively increased latency.

When working with the IT department great care must be taken in understanding what exactly you are measuring.

You might also like...

Understanding IP Production Networks: Part 14 - Delay Monitoring

We use buffers to reassemble asynchronous streams so we must measure how long individual packets take to reliably get to the receiver, and the maximum and minimum delay of all packets at the receiver.

Understanding IP Production Networks: Part 13 - Quality Of Service

How QoS introduces a degree of control over packet prioritization to improve streaming over asynchronous networks.

Understanding IP Production Networks: Part 11 - Network Analyzers

Wireshark is an invaluable tool that enables engineers to examine network traffic in detail. Commercial monitoring platforms provide even deeper observation.

Understanding IP Production Networks: Part 10 - Security

The flexibility of IP and COTS brings with it all of the security dangers of the internet and the need for robust processes. It means new questions need to be asked of broadcast equipment manufacturers.

Understanding IP Production Networks: Part 9 - Ethernet

How Ethernet has evolved to combat congestion and how speeds have increased through the decades.