Practical Broadcast Storage - Part 1

Broadcast and IT technology collaboration is continuing its journey with the cross-over between them becoming ever clearer. Storage is a major growth area with advanced methods achieving greater efficiencies and resilience. And as broadcast repeatedly demands more and more capacity, IT innovation is taking broadcasting to new levels of productivity.

Broadcast technology developed in the 1950’s to record and playback media is still with us today. Even as broadcasters move to new digital server playback methods, the working practices video tape systems provided are still prevalent. To be more productive, broadcasters must embrace the complex methodologies and working practices of IT data storage now available.

Most mainstream broadcast manufacturers have now stopped manufacturing replacement components for their VTR’s, especially heads. Consequently, there is a massive drive to transfer video tapes to IT storage as there is a real fear that parts will soon become unavailable. But choosing the right storage is not as easy as it may first seem.

Advanced Storage

IT storage has moved on significantly from the simple spinning disk to take advantage of the advanced storage technologies now available. Although it’s difficult to draw any true demarcation between them, three groups of storage type have emerged. LTO (Linear Tape-Open), high data capacity but low speed, spinning hardware disk drives, medium capacity and medium speed, and SSD (Solid State Drive) and memory, lower data capacity but incredibly high speed.

In the ideal world, all our data would be stored in SSD and memory to allow data to be available with the lowest delay and latency possible. However, SSD and memory storage is orders of magnitude more expensive than spinning disk drives and has a limited lifespan. And LTO tape storage is still more reliable than disk storage for long term archiving.

Consequently, data needed “now” is stored in SSD and memory, data needed with a small delay is stored in spinning disks, and data not needed for some time, or archive data, is stored in LTO. Intelligent data management systems constantly balances the need of the user with the technology available by moving information between the relevant storage medium.

Optimize User Experience

Using a bank as an example, SSD and memory would be used to store client information for high users of credit cards. When they present their card to the merchant, the banking system validates the card quickly to give the user the best possible experience. If the client hasn’t used their credit card for some time, then their details may be moved to spinning disks giving a slightly longer delay when validating the sale but providing more regular SSD and memory customers with a consistent experience.

Diagram 1 – Using the IT storage, media assets are moved between technologies to improve efficiency. SSD and memory provides high-speed data to the playout servers and edit suites for transmission and on-line editing, spinning disks are used for general storage for the library search, and long-term archive and library storage uses LTO.

Diagram 1 – Using the IT storage, media assets are moved between technologies to improve efficiency. SSD and memory provides high-speed data to the playout servers and edit suites for transmission and on-line editing, spinning disks are used for general storage for the library search, and long-term archive and library storage uses LTO.

LTO archive would be used to store historic transactions. The bank wouldn’t need access to this information immediately but would still be able to retrieve the data within a few hours. This would be fine for a forensic audit.

This model has developed over many years and has its roots in the 1970’s when mainframe computing systems would move data between memory, disk drives and tape storage. And as new technologies are developed to improve latency, storage capacity, and reliability, the prevailing solutions are upgraded.

New to Broadcasters

However, the differentiated model of storage is new to broadcasters as they have always designed for maximum peak demand with minimum delay delivery for the entire system, especially in live environments such as news and sport.

Efficient information retrieval requires the association of accurate and refined metadata to facilitate searching and indexing of information. Due to the abstract nature of video and audio representation, its proved difficult to label video tape and digital clips with any meaningful information. This is usually restricted to timecode indexing with limited text references.

Video Tape Risk

As technology advanced and better video and audio standards became available, many of the tape formats used proprietary systems requiring obscure manufacturer specific hardware to record and playback the video tape. Nobody knows until the tape is threaded onto or inserted into the machine whether the material is retrievable or not.

Even if a broadcaster has moved to server storage and hasn’t adopted IT best practices, or modern IT storage methodologies, then they will not be leveraging the benefits of advanced data storage. The challenge is the same as it is with video tape, it is just that it exists on server storage and not physical video tapes.

Robust and Resilient

Moving to advanced IT storage gives us three distinct advantages. Firstly, IT storage and retrieval systems are much more efficient, scalable, and cost effective. Secondly, advanced storage systems are much more resilient as they constantly monitor data retrieval and analyze for disk errors and can predict failures. And thirdly, due to block storage methodologies, metadata is a function of the data being stored and is not associated with the physical medium it is stored on.

Live broadcast television requires media assets to be instantly available for real-time playback. Although high-speed data storage is important for the clip being transmitted at that moment in time, and any other media that is needed for the show, all the other media available to the broadcaster does not need to be stored in the high-speed storage.

Diagram 2 - For broadcasters to leverage the benefits of Artificial Intelligence and Deep Learning systems, they must provide fast access to large amounts of data. Advance IT storage system make this possible and scalable

Diagram 2 - For broadcasters to leverage the benefits of Artificial Intelligence and Deep Learning systems, they must provide fast access to large amounts of data. Advance IT storage system make this possible and scalable

Live programming requires a great deal of planning. Video tape assets must be available and even if media is stored on video servers, somebody must make sure the clip is physically on the machine and not on a mapped network drive somewhere. Although primitive, this proves advanced IT storage for media playback is already established in television workflows. For many broadcasters, it’s only a small operational jump to move to the more efficient progressive IT storage systems.

Scalability is key when building any modern system. In the infrastructures of the past, broadcasters would have to calculate and predict how much storage was required, projecting anything up to five years into the future. Advanced storage allows broadcasters to scale both horizontally and vertically. Horizontally to increase storage at a specific technology. And vertically to increase or decrease the different technologies to meet the requirements of the operation.

Artificial Intelligence Storage

For example, if a broadcaster wants to leverage Artificial Intelligence, then they must increase the amount of SSD and memory storage available. Or if they are using international production teams working on a set of media assets, then they may need to be seamlessly moved to storage in a physical location close to their geographical operation.

Improved metadata is key to monetizing media assets. The richness of the metadata will make more of a broadcaster’s library available to a greater audience as well as improving on their own internal workflows and working practices.

In the next two articles in this series we will dig deeper into the technology that underpins advanced IT storage and the highly intelligent systems they employ to simplify and automate workflows to improve efficiencies and scalability.

Part of a series supported by

You might also like...

Audio For Broadcast - The Book

​Audio For Broadcast - The Book gathers together 16 articles into a 78 page eBook which explores the science and practical applications of audio in broadcast.  This book is not aimed at audio A1’s, it is intended as a reference resource for …

Comms In Hybrid SDI - IP - Cloud Systems - Part 1

We examine the demands placed on hybrid, distributed comms systems and the practical requirements for connectivity, transport and functionality.

Designing IP Broadcast Systems: Part 2 - IT Philosophies, Cloud Infrastructure, & Addressing

Welcome to the second part of ‘Designing IP Broadcast Systems’ - a major 18 article exploration of the technology needed to create practical IP based broadcast production systems. Part 2 discusses the different philosophies of IT & Broadcast, the advantages and challenges…

The Streaming Tsunami: Securing Universal Service Delivery For Public Service Broadcasters (Part 2)

This is the second part of our discussion of one of the biggest challenges for national Public Service Broadcasters; how to maintain their obligation for universal service in a future landscape where audiences have migrated to streaming as their primary…

Essential Guide: Network Observability

This Essential Guide introduces and explores the concept of Network Observability. For any broadcast engineering team using IP networks and cloud ecosystems for live video production, it is an approach which could help combat a number of the inherent challenges…