Applied Technology: Why Fast Production Storage Is Just Not Enough

UHD, 4K, 6K, 8K…the proliferation of high-resolution formats and files, along with the growing number of skillsets that can collaborate on projects, can push storage bandwidth and capacities beyond their limits. With millions of assets available to potentially hundreds of creatives, resource contention across storage and lack of sophisticated media management tools become a major issue. Efficient, reliable and high-availability media storage, normally reserved for enterprise-class applications, is fast becoming the requirement across all levels for production and post.

The EditShare solution is a scale-out media system that addresses these data demands, not just for the large enterprise operations, but for production and post facilities of all sizes. The core technology is based on the new EditShare File System (EFS), a parallel file system with clever file distribution and smart contention management. Taking it one step further, the solution includes a media asset management application to better manage the increase in content and the growing number of people with access to it.

An intelligent storage environment

To illustrate the benefits, we can consider standard client devices such as non-linear editing systems (NLEs), each equipped with driver software that enables it to communicate with EFS.

When a client device requests to write a file, the software instructs the client to break the file into a number of data blocks, to calculate the parity data associated with those blocks, and to write the data blocks and parity, in parallel, to all of the storage nodes of the system. A key performance benefit is that when distributed in this manner, the resulting bandwidth of the write operation is the sum of the bandwidth available from all the storage elements (i.e. all disk drives) in the system.

Reading stored data

When a client device requests to read a file from storage, it is given the information it needs to locate the stored data associated with the requested file. The client retrieves those blocks and reassembles the file for use in the client application. For media production environments where multiple editors share project spaces and media assets, it is a natural consequence of the environment that contention for access to the storage nodes will increase. And in this environment, contention is unacceptable because of the potential to slow down or delay access resulting in stutter and/or dropped frames.

Swift Read technology

To resolve the issue of contention, the file system monitors the performance of the storage nodes and detects which ones are responding more slowly and may delay read requests. Armed with this critical information, it can instruct a client device to read all of the data blocks associated with the requested file or, if a node containing data is experiencing contention and is likely to delay a read request, to skip that node and reconstruct the requested data using the parity information instead. Plus it can also detect when a storage node has completely stopped working, as in the case of a hardware failure. Thus the file system is able to continue operation even if an entire storage node fails.

Increased storage efficiency

The final benefit of a parallel file system is the efficient way in which it makes use of storage resources and how that efficiency increases as system size increases. Other high performance shared storage systems simply make duplicate copies of data to avoid contention and improve read speeds. This approach, known as mirroring, can at best use just 50% of storage resources.

Look under the hood when selecting storage

So another step change is taking place in storage. A high-availability distributed file system for the masses was once inconceivable, but with the introduction of EFS it seems there is a solution.  Facilities can now grow their internal storage infrastructure securely and profitably. It is important for facilities and engineers to take a closer look under the hood when evaluating their storage infrastructure. A high-performance storage system is not enough. You need smart file system management combined with user and asset management tools to effectively manage the media data road ahead.

You might also like...

2022 NAB Show Review, Part 1

Many annual NAB Shows have become milestones in TV broadcasting history. The presence of the 2022 NAB Show marked the first Las Vegas NAB Show since 2019.

Grass Valley’s New CEO Wants To Lead The Third Video Revolution (The Cloud)

After two years of virtual gathering, broadcasters convening in person for this year’s NAB Show in Las Vegas will see a lot of new faces due to management and staff changes at the various vendors. One notable “new” figure will …

2022 Winter Olympics - Remote Broadcast Infrastructure In A Safety Bubble

This year’s Winter Olympics Games, coming a mere six months after the 2021 Summer Games, is set to begin Feb. 4-20 amidst a persistent pandemic that will once again limit how the television production teams can interact on-site in Beijing, C…

TDM Mesh Networks - A Simple Alternative To Leaf-Spine ST2110: Application - Eurovision Song Contest

With over 4000 signals to distribute, transfer and route, the Eurovision Song Contest (ESC) proved to be this year’s showpiece for Riedel’s TDM based distributed mesh networked system MediorNet. Understanding the intricacies of such an event is key to rea…

TDM Mesh Networks - A Simple Alternative To Leaf-Spine ST2110: Knowing What To Choose

Broadcasters are no longer faced with the binary choice of going down the SDI or IP routes. The hybrid method of using TDM (Time Domain Multiplexing) combines the advantages of distributed networks with IP and SDI to deliver a fully…