BCE Going Deeper - Part 1 - Cable, Standards and ITIL

At the start of 2013, BCE at RTL City was a hole in Luxembourg’s ground and in less than four years they were on air broadcasting 35 different channels across Europe and Singapore. Costas Colombus is BCE’s Special Projects Manager and gave The Broadcast Bridge a unique insight into how they made this mammoth installation work, including describing the issues and how they overcame them along the way.

Transitioning to IP is a major challenge for any broadcaster, especially when moving an existing facility. Greenfield sites have it difficult too, and learning how they successfully made their installations work will help us all.

BCE’s wish was to be format agnostic and ready for future technologies such as 1080P, 4K and HDR. Preliminary requirements dictated the system should be flexible, scalable, redundant and reliable. Aspirations for us all, but often difficult to achieve. Cable installation was identified as a major area for cost savings so fiber was used extensively. To de-risk reliance on a single supplier, multiple vendors and COTS equipment was used wherever possible.

Waiting for ST2110

During the design phase ST2110 had not been ratified so Costas and his team needed to decide on the specification of the system that was available at the time, but would allow for future upgrade to ST2110. SMPTE 2022-6/-7 was chosen for SDI video and AES67 for audio over IP, and DANTE was used for RTL Radio. 

From a hole in the ground, BCE at RTLCity was on air in less than four years

From a hole in the ground, BCE at RTLCity was on air in less than four years

To reduce the amount of copper cable required, fiber was used wherever possible, to the extent where all connections between buildings, floors and rooms were fiber, and the only cables carrying video that exist are intra connections within rooms and racks.

Fiber Challenges

And this is where the challenges began.

Fiber is an established technology and has been in active use since the 1970’s. However, the combination of interfacing it with SDI, and the variable quality of fiber available, caused Costas no end of issues. It soon became apparent that fiber and the associated connectors, SFP and QSFP’s vary wildly in quality and more attention had to be paid to the procurement stage. Systems could fail in a non-predictable manner.

During investigations into the issues Costas stumbled across another challenge, the differing attitudes and working practices of broadcast and IT engineers. Broadcast engineers are used to working in new, dynamic, fast paced environments where equipment often goes wrong and creative, out-of-the-box solutions must be found. 

Banking Influence

IT engineers have their training and influence in the well-ordered world of the banking industry. Many of the working practices IT engineers work to are well established in the finance market, and their ITIL processes exist for good reason as decimal points can make an enormous difference in the finance sector.

Information Technology Infrastructure Library (ITIL) is a set of detailed procedures used in the IT industry and are designed to bring order to a system to stop engineers jumping into systems and fixing them without proper authorization. This method of working is usually alien to most broadcast engineers, but in the IT world there must be an audit trail for every change or modification. As networks typically affect so many different services, one change on one part of the network could have unintended consequences on another. ITIL aims to stop these actions but the system only really works in an established working environment.

BCE have extensively used fiber throughout their installation.

BCE have extensively used fiber throughout their installation.

IP television is at the cutting edge of technology and driving IT to its limits. Engineers are finding obstacles nobody had predicted or expected. Consequently, when something goes wrong it must be fixed immediately, there is no time to raise a help desk ticket or have five meetings about the proposed fix.

Forward With ITIL

Broadcast television will eventually move to the ITIL model and we have already witnessed this in SDI playout facilities. The last thing anybody wants is an unscheduled software upgrade causing outage on fifty pay-per-view channels. But we must have working systems before these processes can be instigated.

ITIL is used to maintain, install and modify systems that are understood, well-ordered and defined. Broadcast IP is not there yet. But it’s fair to say that if we had stuck with SDI playout systems we would probably be using ITIL extensively now. However, to facilitate the new business models of flexibility, scalability and COTS, we must move to IP and we will continue to find solutions to complications we do not yet know we have.

Improve Equipment Monitoring

One of the areas Costas highlighted for equipment design improvement is monitoring the SDI-IP interface where television turns from copper coax to fiber. He also identified varying quality in SFP/QSFP light sources causing packet loss. Using light power meters from the telco industry, BCE’s engineers could prove that fiber channels were failing, but in a further twist this always wasn’t due to light transmitters. 

Procurement Specifications

Terminating fiber proved to be fraught with the same cost issues affecting fiber. Inferior quality connections don’t tend to just fail, they start by creating unpredictable erratic packet loss. Channels would work one minute but not the next, again the origin of the failure proved difficult to find. Was packet loss due to issues in the SDI domain or IP? A failing SDI connection can cause packet loss on an IP connection due to the amount of erratic data being created.

BCE solved the dilemma by assigning a small team of wiremen (fiber-men) to running and terminating or connecting fiber exclusively, a task requiring great skill, patience and experience, and giving highly specified fiber specifications to his procurement department ensuring the use of high quality fiber. Dirt is the fiber’s greatest enemy as even small amounts of debris can cause major headaches for video-IP installations running at the limits of their capabilities.

In article two of “BCE Going Deeper Part 2 – Choosing IP Routers” we continue our conversations with Costas and his colleagues.

You might also like...

Chris Brown Discusses The Themes Of The 2024 NAB Show

The Broadcast Bridge sat down with Chris Brown, executive vice president and managing director, NAB Global Connections and Events to discuss this year’s gathering April 13-17 (show floor open April 14-17) and how the industry looks to the show e…

Designing IP Broadcast Systems: Part 2 - IT Philosophies, Cloud Infrastructure, & Addressing

Welcome to the second part of ‘Designing IP Broadcast Systems’ - a major 18 article exploration of the technology needed to create practical IP based broadcast production systems. Part 2 discusses the different philosophies of IT & Broadcast, the advantages and challenges…

Essential Guide: Network Observability

This Essential Guide introduces and explores the concept of Network Observability. For any broadcast engineering team using IP networks and cloud ecosystems for live video production, it is an approach which could help combat a number of the inherent challenges…

Audio For Broadcast: Cloud Based Audio

As broadcast production begins to leverage cloud-native production systems, and re-examines how it approaches timing to achieve that potential, audio and its requirement for very low latency remains one of the key challenges.

Designing IP Broadcast Systems: Timing

How adding PTP to asynchronous IP networks provides a synchronization layer that maintains fluidity of motion and distortion free sound in the audio domain.