Practical Broadcast Storage - Part 1

Broadcast and IT technology collaboration is continuing its journey with the cross-over between them becoming ever clearer. Storage is a major growth area with advanced methods achieving greater efficiencies and resilience. And as broadcast repeatedly demands more and more capacity, IT innovation is taking broadcasting to new levels of productivity.

Broadcast technology developed in the 1950’s to record and playback media is still with us today. Even as broadcasters move to new digital server playback methods, the working practices video tape systems provided are still prevalent. To be more productive, broadcasters must embrace the complex methodologies and working practices of IT data storage now available.

Most mainstream broadcast manufacturers have now stopped manufacturing replacement components for their VTR’s, especially heads. Consequently, there is a massive drive to transfer video tapes to IT storage as there is a real fear that parts will soon become unavailable. But choosing the right storage is not as easy as it may first seem.

Advanced Storage

IT storage has moved on significantly from the simple spinning disk to take advantage of the advanced storage technologies now available. Although it’s difficult to draw any true demarcation between them, three groups of storage type have emerged. LTO (Linear Tape-Open), high data capacity but low speed, spinning hardware disk drives, medium capacity and medium speed, and SSD (Solid State Drive) and memory, lower data capacity but incredibly high speed.

In the ideal world, all our data would be stored in SSD and memory to allow data to be available with the lowest delay and latency possible. However, SSD and memory storage is orders of magnitude more expensive than spinning disk drives and has a limited lifespan. And LTO tape storage is still more reliable than disk storage for long term archiving.

Consequently, data needed “now” is stored in SSD and memory, data needed with a small delay is stored in spinning disks, and data not needed for some time, or archive data, is stored in LTO. Intelligent data management systems constantly balances the need of the user with the technology available by moving information between the relevant storage medium.

Optimize User Experience

Using a bank as an example, SSD and memory would be used to store client information for high users of credit cards. When they present their card to the merchant, the banking system validates the card quickly to give the user the best possible experience. If the client hasn’t used their credit card for some time, then their details may be moved to spinning disks giving a slightly longer delay when validating the sale but providing more regular SSD and memory customers with a consistent experience.

Diagram 1 – Using the IT storage, media assets are moved between technologies to improve efficiency. SSD and memory provides high-speed data to the playout servers and edit suites for transmission and on-line editing, spinning disks are used for general storage for the library search, and long-term archive and library storage uses LTO.

Diagram 1 – Using the IT storage, media assets are moved between technologies to improve efficiency. SSD and memory provides high-speed data to the playout servers and edit suites for transmission and on-line editing, spinning disks are used for general storage for the library search, and long-term archive and library storage uses LTO.

LTO archive would be used to store historic transactions. The bank wouldn’t need access to this information immediately but would still be able to retrieve the data within a few hours. This would be fine for a forensic audit.

This model has developed over many years and has its roots in the 1970’s when mainframe computing systems would move data between memory, disk drives and tape storage. And as new technologies are developed to improve latency, storage capacity, and reliability, the prevailing solutions are upgraded.

New to Broadcasters

However, the differentiated model of storage is new to broadcasters as they have always designed for maximum peak demand with minimum delay delivery for the entire system, especially in live environments such as news and sport.

Efficient information retrieval requires the association of accurate and refined metadata to facilitate searching and indexing of information. Due to the abstract nature of video and audio representation, its proved difficult to label video tape and digital clips with any meaningful information. This is usually restricted to timecode indexing with limited text references.

Video Tape Risk

As technology advanced and better video and audio standards became available, many of the tape formats used proprietary systems requiring obscure manufacturer specific hardware to record and playback the video tape. Nobody knows until the tape is threaded onto or inserted into the machine whether the material is retrievable or not.

Even if a broadcaster has moved to server storage and hasn’t adopted IT best practices, or modern IT storage methodologies, then they will not be leveraging the benefits of advanced data storage. The challenge is the same as it is with video tape, it is just that it exists on server storage and not physical video tapes.

Robust and Resilient

Moving to advanced IT storage gives us three distinct advantages. Firstly, IT storage and retrieval systems are much more efficient, scalable, and cost effective. Secondly, advanced storage systems are much more resilient as they constantly monitor data retrieval and analyze for disk errors and can predict failures. And thirdly, due to block storage methodologies, metadata is a function of the data being stored and is not associated with the physical medium it is stored on.

Live broadcast television requires media assets to be instantly available for real-time playback. Although high-speed data storage is important for the clip being transmitted at that moment in time, and any other media that is needed for the show, all the other media available to the broadcaster does not need to be stored in the high-speed storage.

Diagram 2 - For broadcasters to leverage the benefits of Artificial Intelligence and Deep Learning systems, they must provide fast access to large amounts of data. Advance IT storage system make this possible and scalable

Diagram 2 - For broadcasters to leverage the benefits of Artificial Intelligence and Deep Learning systems, they must provide fast access to large amounts of data. Advance IT storage system make this possible and scalable

Live programming requires a great deal of planning. Video tape assets must be available and even if media is stored on video servers, somebody must make sure the clip is physically on the machine and not on a mapped network drive somewhere. Although primitive, this proves advanced IT storage for media playback is already established in television workflows. For many broadcasters, it’s only a small operational jump to move to the more efficient progressive IT storage systems.

Scalability is key when building any modern system. In the infrastructures of the past, broadcasters would have to calculate and predict how much storage was required, projecting anything up to five years into the future. Advanced storage allows broadcasters to scale both horizontally and vertically. Horizontally to increase storage at a specific technology. And vertically to increase or decrease the different technologies to meet the requirements of the operation.

Artificial Intelligence Storage

For example, if a broadcaster wants to leverage Artificial Intelligence, then they must increase the amount of SSD and memory storage available. Or if they are using international production teams working on a set of media assets, then they may need to be seamlessly moved to storage in a physical location close to their geographical operation.

Improved metadata is key to monetizing media assets. The richness of the metadata will make more of a broadcaster’s library available to a greater audience as well as improving on their own internal workflows and working practices.

In the next two articles in this series we will dig deeper into the technology that underpins advanced IT storage and the highly intelligent systems they employ to simplify and automate workflows to improve efficiencies and scalability.

Comments:

How do you define AI and Machine Learning algorithms in the figure below?  Both terms are used and abused in equal measure.  Perhaps the green XY plot represents ‘deep learning neural networks and the red represents machine learning. 

Do you also define ML as using supervised, semi-supervised, unsupervised or reinforcement machine learning algorithms? 

link to my annotation of your graphic

https://photos.app.goo.gl/ZJBLhx5XHncpG8ot9

February 2nd 2019 @ 02:03 by Rick Singer

Hello Rick,

please see article 2 in this series for an in-depth answer to your questions - https://www.thebroadcastbridge.com/content/entry/12709/practical-broadcast-storage-part-2

Thanks
Tony

February 5th 2019 @ 09:06 by Tony Orme Editor
Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

ZombieLoad And Other Things That Go Bump In The Night

May 14, 2019 may not have seemed a particularly important date for those who edit and color grade on Mac’s and PC’s. But it was. By chance, that day I went looking for the May Windows 10 Feature Update (1903). I was sur…

AI – ML – DL – The Differences

Artificial Intelligence is more than just one element. In this article, we look at and describe the many parts AI encompasses.

Cyber Security - A DAM Issue

After visiting the recent Henry Stewart DAM (Digital Asset Management) conference in New York, Gary Olson asked some very difficult questions of Cloud vendors regarding security. Their responses may surprise you.

NAB 2019 – Technology vs Consolidation

Giving his unique view on NAB2019, Gary Olson considers and scrutinizes the big moving trends of consolidations, and casts clarity on the cloud, ATSC 3.0, and AI.

Server-Based “At Home” Workflows Provide Efficiency For NASCAR Productions

NASCAR Productions, based in Charlotte NC, prides itself on maintaining one of the most technically advanced content creation organizations in the country. It’s responsible for providing content, graphics and other show elements to broadcasters (mainly Fox and NBC), as w…