Understanding IP Production Networks: Part 9 - Ethernet

How Ethernet has evolved to combat congestion and how speeds have increased through the decades.

In the 1970’s Ethernet was in its infancy and competing with two other proprietary networking systems in token ring and token bus. These systems were backed by heavy industry and the concept of each employee having a computer on their desk wasn’t considered.

Token ring allowed computers to be connected to each other with coaxial cable in a ring topology. An empty packet of data held an electronic token and was passed between each computer, and only when the computer received the token could it send data. Various timeout protocols existed to stop a computer hogging the token.

Token bus uses the token ring protocol, and only differs in that token bus can have an open-ended network connection whereas token ring must have the LAN connected in a continuous loop. With token ring, each computer on the network must know the address of the previous and next computers within the LAN to allow them to pass the token.

Token systems could be unreliable, especially if a computer failed, or it was switched off. Adding extra computers to the network was difficult as it stopped all computers on the LAN from communicating. Furthermore, they were highly inefficient as they would have to receive, process, and transmit the empty token packet even if they had no data to send.

Development

Ethernet was adopted by the IEEE in 1980 and given the project number 802. Ethernet originally used coax as its LAN cable, which was limited to either send or receive data but could not do both at the same time. For this reason, it soon moved to twisted pair, providing a cheaper and more flexible alternative with full duplex operation. Hardware buffers within the Network Interface Cards (NIC) allowed the computer to preload memory at CPU speeds and then the NIC would send the data at Ethernet line speeds, thus making the whole operation significantly faster.

Ethernet was originally developed with a shared bus system in mind. All computers on the LAN would be either listening, sending, or receiving data. To send data, a computer would detect other transmissions, and if it found another communication was taking place it would wait.

During transmission, the network interface of each computer would continue to listen for other traffic, and if one or more computers started sending data, they would all detect this and stop their transmissions. An algorithm in each NIC randomly held back the transmission so statistically one NIC would transmit ahead of the others and stop an infinite race-off condition occurring, keeping the system stable.

This collision detection is called “Carriers Sense Multiple Access with Collision Detect”, or CSMA/CD and was adopted by the IEEE as 802.3 and became a full published standard by 1985. CSMA/CD is used on twisted pair networks as multiple computers can be connected through a hub. These essentially connect all the transmits together (through buffer circuits) as well as all the receives, so CSMA/CD was still required.

With one cable or transmission system, packet collisions would increase as the number of computers increased and communicated more. More collisions result in reduced efficiency and hence lower data rates. Layer-2 switches solved this problem as they greatly reduced the number of devices on each segment and could allow a point-to-point connection between the computer and port of the switch.

Intelligent switches took advantage of input buffering and would decide when to schedule the sending of a packet and avoid collision altogether.

Splits

Layer-2 switches can be connected through bridges to keep maintenance easy and reduce complex cable runs.

Figure 1 - Point-to-point requirements for non CSMA/CD CAT8.

Figure 1 - Point-to-point requirements for non CSMA/CD CAT8.

Each Ethernet network interface card has its own unique Media Access Control (MAC) address programmed during manufacture. Ethernet assumes that the manufacturers of the NIC have been assigned a MAC address from the IEEE’s registration authority and that it has been programmed properly. A NIC will only respond to two types of received messages: its own MAC address in the destination address header, and the broadcast address in the same part of the header.

The broadcast address is always “ff-ff-ff-ff-ff-ff” and is used by protocols such as address resolution protocol (ARP) to resolve an IP address to a MAC address. The downside of this is that when one computer sends out a broadcast message, every single device on the network will receive it and must process it. This is inefficient and a waste of valuable bandwidth. The solution to this is to split networks physically using bridges, or VLANs, which coincide with subnets on IP networks.

Standards, Cabling And Connectors

Ethernet was designed to work with many different protocols, IP is only one of them, and it is possible to send IP packets and ARCNET packets at different speeds on the same Ethernet network.

As Ethernet evolved, a series of standards were published (5GBASE-T, 10GBASE-T etc.) that defined the bandwidth supported by new and improved cable types (CAT5, CAT6, CAT8) etc.

In recent years, speeds of Ethernet networks have increased significantly.

In the 1980’s network speeds were running at 100Kbps on CAT1 cabling, CAT5 provided 250Mbps and CAT8 now gives us 40Gbps by splitting the data over multiple pairs within a single cable or using fiber. However, only distances of 30m are achievable over CAT8 cable.  

Figure 2 - Ethernet frame layout.

Figure 2 - Ethernet frame layout.

The most important aspect of 40GBASE-T is that CSMA/CD has been dropped from the IEEE 802 specification and there is no backwards compatibility with CAT6 and earlier. This is fine for cameras as the output of the camera will connect directly to the high-speed non-blocking layer-2 switch. However, we can’t mix other CAT6 devices within the CAT8 segment as they will require CSMA/CD to work correctly.

As speeds increase to 40Gbps and 100Gbps, the RJ45 Ethernet connector has been succeeded by SFP and CXP connectors. The small form-factor pluggable SFP is a hot pluggable sub-module allowing connection for fiber or copper. CXP combines multiple copper pairs to provide 100Gbps with twelve 10Gbps pairs, or three 40Gbps links.

To network multiple 2160P120 cameras would need a seriously fast layer-2 switch to process all the 24Gbps network streams being sent to it. This would be a switch worthy of a Tier-4 datacenter and certainly wouldn’t be Consumer-Off-The-Shelf.

Fiber transmission paths have advanced considerably in recent years, with physical layers now supporting data transfer rates of 400Gbps. Datarates of 25Gbps, 40Gbps, and 100Gbps are now common in broadcast infrastructures, and each operates over different maximum fiber lengths. Standards such as 25GBASE-SR define short reach cabling that achieves between 300m and 400m on OM3 and OM4 fibers, while 25GBASE-LR defines long reach cabling capable of distances around 10km using OS1 and OS2.

Single Mode And Multimode

Fibers used by broadcasters fall into two principal categories, single mode and multimode.

Single mode fiber transmits significantly further than multimode fiber because it supports only one propagation path, otherwise known as a mode. This is accomplished in part by reducing the core diameter, which lowers modal dispersion. Dispersion is the key factor that limits signal transmission bandwidth. As the core of multimode fiber is wider, light enters at many angles forming different modes, and each mode follows a slightly different path length through the fiber. The resulting pulse spreading introduces inter-symbol interference and constrains the maximum transmission distance, especially at higher bit rates. Single mode fiber avoids multi path propagation and exhibits near zero modal dispersion, so bit rates are limited only by attenuation and chromatic dispersion, both of which are far easier to control.

Modern broadcast infrastructures generally use OS2 single mode fiber for building wide cabling, ST2110 core switch uplinks, control networks between buildings, and remote production links such as those between a stadium and an OB truck. OM3 is largely being replaced by OM4 and is often used within machine rooms, patch panels, top of rack switches, and high density ST2110 links where the distance is less than 150m. 

When deciding whether to use OS2 or OM4, connectivity costs must be considered as these can have a significant effect on the overall price. In general, OS2 cabling is cheaper than OM4 because OM4 manufacturing requires tighter tolerances and more complex glass processing. However, the cost of the transceiver optics dominates the link, and OM4 modules are significantly cheaper than OS2. OM4 transceivers are economical, mass produced, and low power, whereas OS2 transceivers are far more complex due to the lasers they employ and therefore substantially more expensive. For short reach OM4 deployments the overall cost is lower when large numbers of ports are required, whereas OS2 demands expensive long reach optics and is best suited to deployments where fewer long-distance links are needed.

You might also like...

Understanding IP Production Networks: Part 14 - Delay Monitoring

We use buffers to reassemble asynchronous streams so we must measure how long individual packets take to reliably get to the receiver, and the maximum and minimum delay of all packets at the receiver.

Understanding IP Production Networks: Part 13 - Quality Of Service

How QoS introduces a degree of control over packet prioritization to improve streaming over asynchronous networks.

Understanding IP Production Networks: Part 12 - Measuring Line Speeds

Broadcast and IT engineers take very different approaches to network speed and capacity; it is essential to reach a shared understanding.

Understanding IP Production Networks: Part 11 - Network Analyzers

Wireshark is an invaluable tool that enables engineers to examine network traffic in detail. Commercial monitoring platforms provide even deeper observation.

Understanding IP Production Networks: Part 10 - Security

The flexibility of IP and COTS brings with it all of the security dangers of the internet and the need for robust processes. It means new questions need to be asked of broadcast equipment manufacturers.