Practical Broadcast Storage - Part 2

Broadcast systems are renowned for their high speed and high capacity data demands. Up to recently, they relied on bespoke hardware solutions to deliver the infrastructure required for live real-time uncompressed video. But new advances in IT data storage have now opened the doors for broadcasters to take advantage of this state-of-the-art IT innovation.

High-speed data and capacity requirements are now no longer exclusive to broadcasting, the proliferation of web sites and phone-apps has driven IT speed, capacity, reliability, and resilience way beyond the levels demanded by broadcasters. Big-data processing has further driven innovation and data storage systems capable of processing and streaming real-time video and audio is easily available using high-end IT solutions.

SDI broadcast infrastructures are highly tuned and optimized facilities that are difficult and expensive to maintain. Years of bolted-on tweaks and changes to facilitate new workflows add to the risk of outage and loss of transmission. Engineers are constantly chasing their tails trying to keep up with the changes to the system while maintaining high levels of reliability.

Web-site and phone-app service providers also encounter similar challenges to broadcasters, especially when trying to optimize data workflows, balance workloads, fine tune systems, and reduce the need for their team to constantly firefight.

IT Innovators Solutions

But IT innovators have been working hard to deliver high levels of intelligent automation and provide highly optimized infrastructures and workflows.

Although the first experimental AI systems go back to the 1940’s, recent advances in virtualized computation, networking, and storage has witnessed an explosion in the number of AI systems now available.

But what makes AI so important? Seemingly, we just analyze data collected from sensors and data-sets to help understand systems and provide a level of prediction. This method of statistical analysis has been used by engineers and scientists for hundreds of years. 

Diagram 1 – Although this statistical chart implies a high correlation between humidity and disc parity error, AI provides much greater levels of verification and prediction by parsing many more data sets

Diagram 1 – Although this statistical chart implies a high correlation between humidity and disc parity error, AI provides much greater levels of verification and prediction by parsing many more data sets

By looking at the number of users logging onto a web-site, the designers can set a pre-programmed service to spin up new virtualized servers to meet the increased demand. If analysis indicates the number of users double for four hours on a Friday at 6:00pm, pre-emptive action can be taken to spin up the servers in anticipation of the increased demand.

Systems are Dynamic

For as long as businesses have been able to harness information, managers have analyzed data to help predict the future using ever increasingly complex statistical methods. But this form of analysis assumes a static or very slow-moving system.

Statisticians and Business Intelligence Analysts spend a great deal of time trying to find correlation between different data sets. In the example above, analysis indicated web-site users doubled at 6:00pm on a Friday. Although this is a simple example, it’s difficult to establish with much certainty whether this is a trend or an anomaly. The data used for analysis is out of date before the information can be adequately scrutinized.

Context Aware

To improve the prediction, more data-sets would be needed so better correlations can be established. The prediction can never be 100% accurate, but accuracy is directly proportional to the number of correlated data-sets that can be established. AI comes into its own when we look at context-aware dynamic and fast-changing systems.

AI is a generic term that covers four different disciplines; reasoning, natural language processing, planning, and machine learning. Split into four more categories, machine learning includes supervised learning, unsupervised learning, reinforcement learning, and deep learning (or neural learning).

Traditional computer programming requires a set of logical rules to be provided for the data to be analyzed. Usually providing a series of decision trees, the programmer will parse the data and then provide binary outcomes. For example, if it’s 6:00pm on a Friday, then spin up two more virtual web-servers.

Programming is Too Slow

Changing the data to variables, such as “6:00pm” and “Friday”, allows generic rules to be established so that the time and day can be referenced from a database and easily changed. However, this method soon becomes complex and difficult to administer when the rules need to be adapted. 

Diagram 2 – AI encapsulates four different disciplines and machine learning is further classified into four more classes

Diagram 2 – AI encapsulates four different disciplines and machine learning is further classified into four more classes

It might become clear that Tuesdays also need an extra server at 6:00pm. This would require the programmer to add an addition to the rule or add a completely new rule. The example is trivial, but complexity of the program soon escalates, and efficient software maintenance is further compromised.

The method is slow and potentially problematic as the rules must be constantly adjusted to improve the efficiency of the system.

AI Generates Rules

Instead of manually programming rules for the data, AI takes the opposite approach and the data is used to automatically generate its own rules within the context of the data it’s working with in real-time. And that is the true power of AI.

Algorithms continuously monitor historic as well as real-time data and find accurate correlation between the data sets to help improve the future predictions for the system. This approach further improves efficiencies to take advantage of dynamic systems to constantly balance resource.

In the previous article in this series we looked at how advanced IT storage systems balance the user experience against the different storage resource available. An edit system would require instant low latency access to their edit masters and rushes, and this would be provided by the Solid-State-Device (SSD) and memory storage during the edit session.

Improving Efficiency

It would be uneconomical to continue to keep the media files in the high-speed storage when the edit session had finished so the files would need to be copied back to spinning disks. Although manual rules could be established to copy the files between the two storage mediums, AI provides a far more efficient method of providing this facility.

Algorithms can establish which files to copy based on previous sessions and the editor’s behavior. If the editor was previewing off-line rushes earlier in the day, the AI algorithm would associate these files with them and automatically copy the media files to SSD and memory storage associated with the edit suite for their edit session.

Real-Time Analysis

AI not only provides ever increasing efficient levels of automation, it does it without any human analysis, programming, or intervention. The algorithms learn the behavior of the storage dynamics and accurately predict which files need to be transferred between the different storage medium to improve the user experience and make the most of the resource available.

All this happens in near real-time making the traditional record-analyze-program methods now obsolete. In the next article in this series, we look at how AI has influenced and greatly improved the reliability and resilience of storage. 

Part of a series supported by

You might also like...

The Big Guide To OTT: Part 10 - Monetization & ROI

Part 10 of The Big Guide To OTT features four articles which tackle the key topic of how to monetize OTT content. The articles discuss addressable advertising, (re)bundling, sports fan engagement and content piracy.

Next-Gen 5G Contribution: Part 2 - MEC & The Disruptive Potential Of 5G

The migration of the core network functionality of 5G to virtualized or cloud-native infrastructure opens up new capabilities like MEC which have the potential to disrupt current approaches to remote production contribution networks.

Standards: Part 8 - Standards For Designing & Building DAM Workflows

This article is all about content/asset management systems and their workflow. Most broadcasters will invest in a proprietary vendor solution. This article is designed to foster a better understanding of how such systems work, and offers some alternate thinking…

Designing IP Broadcast Systems: Addressing & Packet Delivery

How layer-3 and layer-2 addresses work together to deliver data link layer packets and frames across networks to improve efficiency and reduce congestion.

The Business Cost Of Poor Streaming Quality

Poor quality streaming loses viewers at an alarming rate especially when we consider the unintended consequences of poor error reporting on streaming players.