The theory behind microservices is relatively straightforward: build a system consisting of interconnected small software applications to provide dynamic and highly flexible workflows. We may like to abstract the system design from the underlying hardware, but at some point, broadcast engineers must get their hands dirty and dig deep into the challenges of making their broadcast workflows operate in real-world applications.

Open & Closed Systems

When considering the type of architecture to deploy, it’s worth thinking about the closed and open software philosophies when applied to microservices. In software terms, a closed system is a software application that is effectively a black box. Users have no idea how the internals of the software operate and rely on the interface and its associated specification to provide the necessary control. Also known as proprietary systems, closed systems are usually owned by a company, and they keep the source code and underlying technology under close wraps.

Open systems embrace open-source software to expose how the application and its associated algorithms actually operate. With its philosophy in the Free Software Foundation, open source is free as in free to use and to learn from, but not free as in free beer. Pioneered by Dr. Richard Stallman, open-source software encourages collaboration and education as any software purporting to be open-source makes its source code available so that developers can learn how it works and expand its features as required.

There are both advantages and disadvantages of closed and open systems. For example, closed systems generally have much better-quality control as the software is under the direct management of the company that owns the intellectual property and will have well defined processes in place with test vectors that deliver high quality products. The established view is that open-source code often relies on volunteers to maintain it and provide releases that are delivered when the developers have finished their day jobs.

Secure Systems

Security is often a moot point with open and closed systems. On the one-hand, a closed system should be more secure as only the company employees have access to the code, but with open software, anybody can read and modify the code thus allowing a much greater pool of talented developers to constantly review and respond to any security breaches. But hackers also have access to the source code and have the potential to spot vulnerabilities.

Up until a few years ago, the choice was quite binary: should I use open or closed systems? Broadcast infrastructures have traditionally been built on closed systems where proprietary hardware, firmware and control connectivity was the established method of operation. But as developments outside of broadcasting have continued to influence broadcast facility infrastructures, then the move to open systems is gaining momentum. But there is also another philosophy that is progressing and that is open systems with the maintenance advantages of closed systems.

Companies such as RedHat provide enterprise versions of Linux (Red Hat Enterprise Linux) as the source code is free for anybody to download.  Their commercial model relies on providing support and maintenance services on a subscription basis. Although updates and security patches eventually become available for anybody to download, priority is given to subscribers. Support forms a major part of their subscription service for developers and system integrators and so the business model provides the best of all services for commercial applications. The software is available to learn from and install, but an army of developers and security experts are constantly reviewing and improving the code to make it as secure and reliable as possible.

Open & Closed Collaboration

In broadcasting, NVIDIA’s Holoscan for Media has adopted a similar model where open-source code is available, but they take this one stage further by designing the applications to run on AI hardware architectures. GPUs provide a method of hardware acceleration for graphics computation and processing. Ray tracing, a core method of graphics processing uses vector calculus to determine intersections on surfaces to build the surface texture and shading. The dedicated hardware processing developed for GPUs took this computationally hungry process away from the computers CPU to provide the high precision graphics we find in modern gaming. The same mathematical ray tracing vector calculus is used in machine learning to provide AI systems that take advantage of the hardware and parallel accelerators found in GPUs.

In essence, Holoscan is a software defined infrastructure embracing microservices through containers such as Kubernetes and abstracts the underlying complexity of the hardware away from the user through libraries, SDKs and APIs. Modern GPUs are highly optimized devices with dedicated parallel processing. Although they were originally designed specifically for gaming, the same hardware can be used to process in parallel video pixels and audio samples, thus massively reducing processing latency. Using the SDKs and SDIs, vendors can focus on solving problems specifically for the broadcasters and don’t have to reinvent the wheel in terms of making low latency video and audio streaming systems in IP datacenters. Holososcan provides the processing speeds of FPGAs with the flexibility of software.

Avoiding Vendor Lock-In

At this moment in time, no one vendor has available all the solutions required to build a modern software defined infrastructure. And even if they did, most broadcasters would be nervous about aligning themselves entirely to one vendor as avoiding vendor lock-in has always been a fundamental philosophy for broadcast architects. This is not only to avoid the obvious business risks of putting all your eggs in one basket but also has the downside of closing the door on innovation from other vendors, especially startups and innovators.

The EBU have addressed this by creating their MXL (Media eXchange) Format to address the challenges of interoperability. Without this solution, broadcasters must resort to measures such as converting all signal flows to ST2110, or NDI, or some other format to transfer signals between microservice apps. Not only is this inefficient, but converting to ST2110 is impossible in the public cloud and requires a separate database of video and audio formatting to be administered if this method was adopted on-prem, further adding to the complexity of the system.

MXL describes an abstraction that reduces vendor lock-in and greatly improves interoperability and integration. In part this is achieved by defining the operation of shared memory-based circle buffers that allow the seamless transfer of video and audio signals (flows) between microservice apps, along with the associated flow metadata: audio sampling rate, bit depth, video frame sample rate, and color space etc. This shared-memory approach is a well-defined methodology that is ubiquitous in IT systems and High-Performance Computing.

Signal Flows

Within datacenter infrastructures, two network topology classifications are emerging: north-south and east-west. These refer to two different network flows where east-west generally represents networks within the datacenter and north-south are networks outside of the datacenter.

East-west packet flows are horizontal in their nature and play a very important role in microservice architectures, especially in broadcasting. Even though many broadcasters are moving to COTS infrastructures, the distribution and processing of video and audio still needs special consideration. The relentless datarate and low latency requirements of video and audio demands microservices work within the same east-west network location. Top of Rack (ToR) switches provide fast network routing to other racks as they generally work with Layer-2 frame switching, but once we go beyond the aggregation and core switches, then latencies start to develop.

North-south packet flows are used for video and audio ingest and egress as well as API control from management systems. Depending on the internet link, latencies and packet loss can quickly develop resulting in a seriously negative quality of experience for the end viewer. Although it’s technically possible to use microservices across different datacenters, great caution should be exercised. Preferably, microservices for broadcast infrastructures should be kept within a single rack and if that’s not possible then they should be kept within the east-west network.

Security is another consideration for east-west networks as we often focus attention on the north-south flows. Intrusion Protection Systems and Firewalls are usually installed at the northerly most part of the datacenter just before it connects to the internet thus allowing hostile and rogue actors to be detected quickly. But security vulnerabilities can still manifest themselves in the east-west network, especially if bring-your-own devices are used. A hacker that has infiltrated somebody’s mobile device could gain access to the east-west network once the user is within the building or virtual locality of the east-west network. In these instances, security should be built into the network at the very beginning of the design and not left as an inconvenient amendment later in the design phase.

Zero trust policies should be at the core of any such design.

Microservice Integration

Microservice architectures are continuing to deliver unparalleled opportunities for broadcasters. Keeping applications physically close will keep latency low and reduce the risk of packet dropout leading to the adoption of east-west network topologies. North-south flows still provide important connectivity to the outside world, but latencies will increase. This may not be so much of an issue for media ingress and egress but will be a challenge if broadcasters want to try and build collaborative microservice architectures in different datacenters, especially if low latency and low dropout media flows are needed.

Open systems are further gaining traction within broadcasting, not just within software, but also entire systems. The demarcation between software and hardware is being further blurred as open systems such as NVIDIA’s Holoscan for Media is providing the best of all worlds in terms of the freedom of open source software, but also providing resilience found in closed systems where vendors have much better control of software and hardware quality, while at the same time allowing vendors to focus on building applications that solve specific problems for broadcasters.