Applied Technology: Scale-out 4k+ collaborative workflows - Is your facility ready?

Moving to a 4K production environment requires a network with sufficient performance capability, but what does that mean?

With the constant discussion surrounding 4k and larger acquisition formats, and the ultimate delivery of UHDTV to viewers' homes, there is growing concern about how storage infrastructure factors into the ability to run an efficient post workflow at these more demanding data rates. Since most post houses have spent considerable amounts of time and effort getting their HD workflows to be as efficient as possible, it’s no surprise that talk of 4, 6, and 8k rasters along with the higher frame rates and advanced color spaces that encompass UHDTV’s future, might be cause for some anxiety. After all, how many of us actually believe that budgets and schedules will increase to compensate for these new demanding formats and deliverables?

As media creators tool up for the 4k challenges they’re facing now, it’s equally important to plan for the future. How do you leverage the infrastructure you have to tackle the myriad of transcode and collaboration requirements, while delivering the bandwidth and functionality to stay on schedule and budget?

With the constant pressure of shrinking schedules and budgets, now more than ever, a shared storage network must deliver the performance to handle every scenario that comes in the door without forcing you to buy or rent additional storage to get you through the finishing phases. There’s no reason to think that a shared network would not be able to provide a collaborative environment, while also delivering the capability for uncompressed 4k DPX conform when it’s time to finish.However, when searching for such capability, it’s clear that not all shared storage systems are alike. Many are not designed for the demanding data rates we routinely encounter in post production.

Can your SAN delivery the bandwidth required for High-Resolution Finishing? This image shows the bandwidth required at various production performance levels.

Can your SAN delivery the bandwidth required for High-Resolution Finishing? This image shows the bandwidth required at various production performance levels.

Scaling out and Scaling up

In today’s post production pipeline, more people need access to project media than ever before. “Scale-out” is a term used to describe increased access to network resources for departments and individuals like producers and graphics artists.

However, 4k in an uncompressed or raw format is still the domain of high-end workstations, and high bandwidth connections. While it’s important to maintain the highest quality, the use of compressed 4k codecs in the 444 space like Apple’s Pro Res and Avid’s DNxHR are increasingly popular. The lower data rates of compressed frames make it much more feasible to “scale-out” and allow multiple users to work with those beautiful, if somewhat lossy 4k images. If the final delivery requirement is one of these compressed 4k formats, then it’s completely feasible for some productions to stay in that format throughout the entire workflow.

All storage is not created equal – Media storage requires special considerations

To provide maximum flexibility, an important consideration is the connectivity between clients and the storage server. While Ethernet is the most popular method for collaborative workgroups, there is also a need for high-speed Fibre Channel when maximum bandwidth for finishing at 4k and higher is required. Ideally, you want a storage system that can be easily reconfigured from shared multi-user access to high-bandwidth single-user access as the project evolves. If you find yourself having to completely switch storage systems and transfer data between offline editing and finishing, then it’s costing you valuable time and money.

The importance of optimization

While it’s great that server hardware and disks get faster and bigger every year, it’s still not enough. Maintaining consistency of performance to all clients is a top priority for any post workflow. As such, a shared storage system built for media production needs to be optimized beyond what basic RAID or drive arrays can provide. With large files to read and write, along with the randomization of multiple streams and clients, traditional hard drive systems reach the limit of performance too quickly. Additionally, if things start slowing down as the system reaches full capacity, you can be sure it’s going to happen at the absolute wrong time. So, consistent performance is something you should demand in your SAN, regardless of how many clients attach, or how full it is.

The way Facilis guarantees such performance is by using low-level virtualization. The result of this virtualization is a pool of storage, presented as one large block of storage for use in allocating project-based volumes. In this virtualized mode, a TerraBlock system writes data in a pattern across the entire drive, on all of the disks, all the time, and is not bound by the speed, portion or sector (position) on any given disk.

At a higher level, the optimization of the Facilis shared file system serves to provide the client workstation with as much of the speed available from the virtual pool as possible. The shared file system is the basis for TerraBlock’s collaborative feature set, but at any time, a user can still request full-speed block level access to a volume when maximum bandwidth is required. This gives the system performance, flexibility, and full capacity usability without adding complexity to the end user experience.

Future Proofing

As we march towards 8k, higher frame rates, and advanced color space, one thing is clear: it’s going to take careful planning to avoid roadblocks.Wherever possible, system administrators should leverage the infrastructure they currently have, while ensuring client workstations can maintain flexibility with fully collaborative and ultra-high performance feature sets.

The last thing you want is for your well-oiled postproduction machine to grind to a halt because your storage system runs out of gas.

James McKenna is vice-president of  Product Marketing for Facilis Technology

James McKenna is vice-president of Product Marketing for Facilis Technology

You might also like...

Data Recording: Cyclic Redundancy Checks - Part 19

The CRC (cyclic redundancy check) was primarily an error detector, but it did allow some early error correction systems to be implemented. There are many different CRCs but they all work in much the same way, which is that the…

Data Recording: Modulo Counting - Part 18

The mathematics of finite fields and sequences seems to be a long way from everyday life, but it happens in the background every time we use a computer and without it, an explanation of modern error correction cannot be given.

Selecting A Content Creation Laptop

Computer marketing departments typically do not promote all company products. Rather they focus on high margin products.

Data Recording and Transmission: Error Correction II - Part 17

Here we look at one of the first practical error-correcting codes to find wide usage. Richard Hamming worked with early computers and became frustrated when errors made them crash. The rest is history.

Data Recording: Error Correction - Part 16

Error correction is fascinating not least because it involves concepts that are not much used elsewhere, along with some idiomatic terminology that needs careful definition.