2018 NAB Show Event Channel

The #1 source of technology content in the broadcast & media industry, by the editors of The Broadcast Bridge - filtered by category.

Click here

BCE Going Deeper - Part 2, Choosing Routers

At the start of 2013, BCE at RTL City was a hole in Luxembourg’s ground and in less than four years they were on air broadcasting 35 different channels across Europe and Singapore. Costas Colombus is BCE’s Special Projects Manager and gave The Broadcast Bridge a unique insight into how they made this mammoth installation work, including describing the issues and how they overcame them along the way.

In this second article in the series we look at the IP routers Costas and his team chose, and why.

Thinking IP opens a whole new world to us, no longer are we constrained by point-to-point single direction SDI connections with bulky cable, or esoteric, expensive hardware that is on long delivery times, or trying to predict next year’s technology trend so we can maintain compatibility. IP gives us a reliable hardware platform to work from that is future proof and adaptable.

TCP Adds Delay

IT has been using IP since the 1970’s when it was first introduced into the Arpanet network for the US defense agency. But at that time, it was slow and had limited connections. IP was designed to provide an unreliable, asynchronous, packet delivery system. Slow, because the network speeds in the 1970’s were in the thousands of bits per second, sometimes as fast as one megabit. Unreliable, because you cannot guarantee packets would arrive at their destination. Asynchronous, as IP is not tied to any hardware transport layer.

To overcome unreliability, developers designed the Transmission Control Protocol (TCP); an “n-try” windowing system that uses controls within the protocol to guarantee packets reach their destination, and flag to the user an error if they do not. TCP works efficiently for banking and the internet, but adds delay and reduced bandwidth for broadcasters.

Determine Network Capacity

When deciding which router to choose, BCEs team first determined the required network capacity. To achieve this, they looked at the number of HD services needed and then determined how many would fit into 40Gb fiber ports; the affordable fiber standard at the time. Simplistically, this gives a maximum of 40/1.5 = 26.66 HD services. But BCE allowed for 3G-SDI upgrade, and some overhead for audio, control and timing, resulting in 16 HD services per 40Gb fiber port.

Diagram 1 – Bays are mostly fiber connections, purple multimode fiber @ 10Gb/s, and yellow is single-mode fiber

Diagram 1 – Bays are mostly fiber connections, purple multimode fiber @ 10Gb/s, and yellow is single-mode fiber

SDI has been very successful over the years because of its reliability, in part due to the synchronous nature of operation and dedicated cabling. Broadcast engineers take for granted that the latency and delay through traditional SDI routers is just a few nano-seconds, and they are un-blocking; a term not seen prior to IP in television.

Latency Issues

Few network engineers, other than those working in high end data centers and ISP’s, ever worried about a network routers ability to block, or not. Blocking routers will receive an ethernet or fiber frame of data and store it into a buffer, the processor will determine which port to route that packet to and then schedule it to be sent, this causes indeterminate delay, hence the term blocking.

Routing delay in enterprise IT is irrelevant, and it’s only when we move to high end data centers used in internet hubs or gateways, does it become important. But latency became a big issue when broadcast television started using IP routers.

Web servers and banking systems burst data using TCP, and the response of a system is determined by how long a user is willing to wait for the result to appear after they have clicked a web page button; delays of 400mS are not unreasonable. TCP will mask small packet losses and in these applications, blocking isn’t an issue.

Non-blocking Routers

Uncompressed HD video and audio delivers constant 1.5Gb/s data streams using UDP packets. UDP is a protocol that sits on top of the IP layer to add packet error detection and server port functions. If you think of a telephone number as the IP address of a server, then the UDP port is the telephone extension, it allows one server with a single IP address to provide multiple software services.

Being able to provide a router that is non-blocking was important for BCE, and they insisted vendors explicitly state and demonstrate this.

BCE wanted a fully redundant system using two different manufacturers. They assumed that the routers would require a software upgrade or demonstrate a hardware malfunction, so having two vendors supplying two different routers would reduce the risk of a common bug or malfunction between them. And the probability of both routers needing urgent software patches at the same time was further reduced.

Diagram 2 – Arista switch with QSFP+ and SFP modules

Diagram 2 – Arista switch with QSFP+ and SFP modules

Costas and his team at BCE, after taking advice from their principle technology contractors SAM, chose routers from Arista and Juniper. Both had 40Gb ports and were non-blocking.

Choose Components Carefully

Quad Small Form-factor Pluggable Plus (QSFP+) modules are compact, hot pluggable transceivers, used to interface the fiber cable to the routers. QSFP+’s use Coarse Wavelength Division Multiplexing (CWDM) to give four light channels of 40Gb/s (4 x 10Gb/s) data throughput per fiber.

The procurement department noted that IT were using white-label modules. The specifications looked the same, and so Costas was instructed to use them.

Initially, the modules worked well during installation. But after a few weeks IP circuits would randomly generate lots of packet losses with associated loss or break up of video and audio.

After many hours of in-depth analysis BCE Maintenance engineers determined the QSFP+ modules were randomly failing. To fix this, they installed a set of new QSFP+ modules recommended by the router manufacturers. The packet losses dropped from hundreds of thousands a day, to tens of packets a day.

How Costas and the team could prove the white-label modules were not up to standard, and why they worked in the IT domain but not for broadcast, is the subject of our next article in the series BCE Going Deeper; Testing and Logging IP Circuits.

Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

Live Recording with a Laptop Computer

It is very tempting to take a laptop computer on location for live recording. It can work fine — until it doesn’t. Laptops, by their very nature, are more fragile and prone to failure than other recording devices. Be careful tha…

Audio Over IP - Why It Is Essential

The impact of IP on the design of broadcast equipment and infrastructures is profound. Many broadcasters are replacing existing analog, AES3, MADI and SDI ports with a new class of interface for connecting to standard IT switch infrastructure, together with…

Loudspeaker Technology Part 14: Cables - Behind the Myths and Magic

John Watkinson puts on his snake-oil-proof clothing and looks at speaker cables. Finally, some clarity behind the myths and magic that surround technical aspects of speaker interconnections.

Podcasting Merges with iOS Devices

In this “how small can we make it” world, it was inevitable that podcasting would merge with iPhones, iPads and other iOS devices. Now, it is possible to carry an international radio station in your pocket or shoulder bag. That was…

On Recording the Human Voice

Recording the human speaking voice can be one of the trickiest tasks a professional sound recordist encounters. Even when working with seasoned professional voice artists, problems can creep in. Here are a few of them and how to solve the…