BCE Going Deeper - Part 1 - Cable, Standards and ITIL

At the start of 2013, BCE at RTL City was a hole in Luxembourg’s ground and in less than four years they were on air broadcasting 35 different channels across Europe and Singapore. Costas Colombus is BCE’s Special Projects Manager and gave The Broadcast Bridge a unique insight into how they made this mammoth installation work, including describing the issues and how they overcame them along the way.

Transitioning to IP is a major challenge for any broadcaster, especially when moving an existing facility. Greenfield sites have it difficult too, and learning how they successfully made their installations work will help us all.

BCE’s wish was to be format agnostic and ready for future technologies such as 1080P, 4K and HDR. Preliminary requirements dictated the system should be flexible, scalable, redundant and reliable. Aspirations for us all, but often difficult to achieve. Cable installation was identified as a major area for cost savings so fiber was used extensively. To de-risk reliance on a single supplier, multiple vendors and COTS equipment was used wherever possible.

Waiting for ST2110

During the design phase ST2110 had not been ratified so Costas and his team needed to decide on the specification of the system that was available at the time, but would allow for future upgrade to ST2110. SMPTE 2022-6/-7 was chosen for SDI video and AES67 for audio over IP, and DANTE was used for RTL Radio. 

From a hole in the ground, BCE at RTLCity was on air in less than four years

From a hole in the ground, BCE at RTLCity was on air in less than four years

To reduce the amount of copper cable required, fiber was used wherever possible, to the extent where all connections between buildings, floors and rooms were fiber, and the only cables carrying video that exist are intra connections within rooms and racks.

Fiber Challenges

And this is where the challenges began.

Fiber is an established technology and has been in active use since the 1970’s. However, the combination of interfacing it with SDI, and the variable quality of fiber available, caused Costas no end of issues. It soon became apparent that fiber and the associated connectors, SFP and QSFP’s vary wildly in quality and more attention had to be paid to the procurement stage. Systems could fail in a non-predictable manner.

During investigations into the issues Costas stumbled across another challenge, the differing attitudes and working practices of broadcast and IT engineers. Broadcast engineers are used to working in new, dynamic, fast paced environments where equipment often goes wrong and creative, out-of-the-box solutions must be found. 

Banking Influence

IT engineers have their training and influence in the well-ordered world of the banking industry. Many of the working practices IT engineers work to are well established in the finance market, and their ITIL processes exist for good reason as decimal points can make an enormous difference in the finance sector.

Information Technology Infrastructure Library (ITIL) is a set of detailed procedures used in the IT industry and are designed to bring order to a system to stop engineers jumping into systems and fixing them without proper authorization. This method of working is usually alien to most broadcast engineers, but in the IT world there must be an audit trail for every change or modification. As networks typically affect so many different services, one change on one part of the network could have unintended consequences on another. ITIL aims to stop these actions but the system only really works in an established working environment.

BCE have extensively used fiber throughout their installation.

BCE have extensively used fiber throughout their installation.

IP television is at the cutting edge of technology and driving IT to its limits. Engineers are finding obstacles nobody had predicted or expected. Consequently, when something goes wrong it must be fixed immediately, there is no time to raise a help desk ticket or have five meetings about the proposed fix.

Forward With ITIL

Broadcast television will eventually move to the ITIL model and we have already witnessed this in SDI playout facilities. The last thing anybody wants is an unscheduled software upgrade causing outage on fifty pay-per-view channels. But we must have working systems before these processes can be instigated.

ITIL is used to maintain, install and modify systems that are understood, well-ordered and defined. Broadcast IP is not there yet. But it’s fair to say that if we had stuck with SDI playout systems we would probably be using ITIL extensively now. However, to facilitate the new business models of flexibility, scalability and COTS, we must move to IP and we will continue to find solutions to complications we do not yet know we have.

Improve Equipment Monitoring

One of the areas Costas highlighted for equipment design improvement is monitoring the SDI-IP interface where television turns from copper coax to fiber. He also identified varying quality in SFP/QSFP light sources causing packet loss. Using light power meters from the telco industry, BCE’s engineers could prove that fiber channels were failing, but in a further twist this always wasn’t due to light transmitters. 

Procurement Specifications

Terminating fiber proved to be fraught with the same cost issues affecting fiber. Inferior quality connections don’t tend to just fail, they start by creating unpredictable erratic packet loss. Channels would work one minute but not the next, again the origin of the failure proved difficult to find. Was packet loss due to issues in the SDI domain or IP? A failing SDI connection can cause packet loss on an IP connection due to the amount of erratic data being created.

BCE solved the dilemma by assigning a small team of wiremen (fiber-men) to running and terminating or connecting fiber exclusively, a task requiring great skill, patience and experience, and giving highly specified fiber specifications to his procurement department ensuring the use of high quality fiber. Dirt is the fiber’s greatest enemy as even small amounts of debris can cause major headaches for video-IP installations running at the limits of their capabilities.

In article two of “BCE Going Deeper Part 2 – Choosing IP Routers” we continue our conversations with Costas and his colleagues.

You might also like...

The Big Guide To OTT: Part 10 - Monetization & ROI

Part 10 of The Big Guide To OTT features four articles which tackle the key topic of how to monetize OTT content. The articles discuss addressable advertising, (re)bundling, sports fan engagement and content piracy.

Next-Gen 5G Contribution: Part 2 - MEC & The Disruptive Potential Of 5G

The migration of the core network functionality of 5G to virtualized or cloud-native infrastructure opens up new capabilities like MEC which have the potential to disrupt current approaches to remote production contribution networks.

Standards: Part 8 - Standards For Designing & Building DAM Workflows

This article is all about content/asset management systems and their workflow. Most broadcasters will invest in a proprietary vendor solution. This article is designed to foster a better understanding of how such systems work, and offers some alternate thinking…

Designing IP Broadcast Systems: Addressing & Packet Delivery

How layer-3 and layer-2 addresses work together to deliver data link layer packets and frames across networks to improve efficiency and reduce congestion.

The Business Cost Of Poor Streaming Quality

Poor quality streaming loses viewers at an alarming rate especially when we consider the unintended consequences of poor error reporting on streaming players.