Software Infrastructure Global Viewpoint – July 2017

Video Storage: It’s All About Loss Prevention

Losing media content is a disaster. Yet, there are multiple technologies designed to store content, some for short-term and others for long-term. In the expanding world of content development, everyone has the same question; “How best to store it?”

Whether you are a major movie producer, DP, or a videographer, having a reliable storage technology and workflow are paramount to you keeping your job. Telling your boss, “I lost it” will not endure you to anyone. Fortunately, there are reliable solutions to that challenge. Choosing the best solution for your workflow, well it depends…

There is a growing demand for content that viewers can access on their multiple devices. This content is sometimes used to; avoid work, create presentations, educate/entertain and inform. Before that entertainment can reach the viewer, it must be captured, posted and published. All of which requires storage.

According to Coughlin Associates, a film today with 16,000 x 8,000 pixel resolution, 24 bits/pixel, 300 fps RAW content could require a 115 GB/s data rate and 414 TB/hr of storage.A 360 degree, 4- camera shoot could produce 1.66 PB/hr RAW data. In 10 years, some experts believe professional media projects could easily generate 1 EB of data. That is a lot of data to store.

Typical storage demands

Production–3TB/day

  • Camera RAW output to SSD – 960GB/day
  • On site/on set de-Bayer – 2.1TB/day

Post-production

  • Creative editorial – 1TB per editor
  • VFX – 10-20TB
  • Color, mastering, versioning – 4.4TB per 4K version
  • Copies and back up of each step of the process

Filmmakers calculate that they will need 10X the amount of storage for the original media shot. Moreover, if you are doing VR, storage requirements increase by a factor of 10!


">

With the burgeoning demand for quality video content for marketing, education, information and entertainment, the volume of data that needs to be stored continues to grow. Increasingly, filmmakers use a mixture of local and cloud storage to store and protect their content. Click to enlarge.

Driven

The range of what indie filmmakers shoot/produce includes feature films, series, documentaries, company stories, ads and more.

In addition, filmmakers like Cirina Catania, of The Catania Group, capture content to tell visual stories they feel are best told with visual stories, rather than written words.

Her current project, The Kionte Storey Journey, which has been three years in production, is a documentary of a young Marine who was disabled because of an IED (improvised explosive device).

Cirina Catania directs Sgt Kionte Storey on what she wants to capture from his early morning workout. She has been working on the documentary to help other returning vets, and those around them, to understand the costs of conflict and the way home.

Cirina Catania directs Sgt Kionte Storey on what she wants to capture from his early morning workout. She has been working on the documentary to help other returning vets, and those around them, to understand the costs of conflict and the way home.

Catania, and most filmmakers we know, use 4K cameras that capture 24-60 fps and more to deliver brilliant content. Just how much data must be stored is illustrated in the table below.


">

Today’s high-performance cameras produce a tremendous amount of content, even at 24p. With a growing number of content producers shooting at 30, 60 and more fps, storage demands further increase. Click to enlarge.

While storage cards are reliable and provide a good storage life, they are not cheap. Few filmmakers I have talked with are comfortable using the solid-state devices more than about 20 times for capture.Even fewer will trust their only copy of the original content to the card for long-term storage. Once the content is captured, it is immediately moved to other storage technology.

Move it to the cloud

Storage experts with a slight vested interest (Amazon, Microsoft, Google, IBM, Box, HP, Oracle, Apple and others) say, “Just move it to the cloud.” These same companies remind us how fast, easy and reliable the cloud is.

Yet, much of the media industry does not see the cloud in the same light. It’s more difficult to push 10TB of content to the cloud than it is to download/stream the same volume to a viewer’s screen. It’s even tougher to access the cloud when you are in the middle of the Amazon or some high desert or anywhere the internet connection is a little iffy.

To shoot, produce and air Conan 360, one of the first 360 VR projects done in near real-time, members of Lewis Smithingham’s (foreground right) 30ninjas VR team set up a complete post production facility – including a server and storage rack – up behind the stage. The post team was focused on delivering a complete show in less than an hour after the final shots.  O’Brien and his team have used it as a benchmark for future VR shows.

To shoot, produce and air Conan 360, one of the first 360 VR projects done in near real-time, members of Lewis Smithingham’s (foreground right) 30ninjas VR team set up a complete post production facility – including a server and storage rack – up behind the stage. The post team was focused on delivering a complete show in less than an hour after the final shots. O’Brien and his team have used it as a benchmark for future VR shows.

Even if you are located in the heart of Hollywood and the production team is working from a central server, you’ll see FX, audio, colorists, editors with a strict time budget using the old faithful sneaker net (walking a drive from one workstation to another).

And even when time isn’t the major concern, smart (or burned) filmmakers will immediately copy their RAW files to their computer SSD (solid state drive) and then copy it again to one or more external storage devices.

No one wants to tell the showrunner, producer/executive producer or marketing person that the only copy of probably the best stuff you ever shot went poof but is out there…somewhere.

It is comforting to enter a room (even if it’s your bedroom/post facility) and seeing RAID boxes, stacks of external HDs and piles of SSDs. Especially when reports like Deloitte’s Tech Trends reassure us, “There is no such thing as hacker-proof, given how well-resourced, capable and methodical cyber criminals are these days”.

Award-winning filmmaker and documentarian Leslie Chilcott shoots and directs commercials and documentaries.  No matter how busy her schedule, she moves her content to her computer and external storage soon after capture.

Award-winning filmmaker and documentarian Leslie Chilcott shoots and directs commercials and documentaries. No matter how busy her schedule, she moves her content to her computer and external storage soon after capture.

Distributed workflow—and workers

Today’s workflow is complex and fast-paced. Film production often relies on an array of freelance talent and an entire ecosystem of suppliers handling the specialty work to deliver 4K/HDR or VR content.

Because of the complexity of today’s 4K/HRD and VR film projects, filmmakers use the best production experts available, regardless of their location. Depending on the size of the work, segments are sent over controlled iNet connections or overnight courier services. Click to enlarge.

Because of the complexity of today’s 4K/HRD and VR film projects, filmmakers use the best production experts available, regardless of their location. Depending on the size of the work, segments are sent over controlled iNet connections or overnight courier services. Click to enlarge.

Even the best, most talented filmmaker works globally. Adobe Creative Cloud, Avid Cloud Collaboration, Dolby, Harmonic and many others offer a range of solutions and capabilities for creatives around the globe. With today’s workflow technology, you can be in NYC, working with a VFX house in New Zealand, a colorist in Berlin and audio experts in L.A. It may look messy but notes, photos, video, audio, metadata, spreadsheets, social media plans/tools and home grown databases are constantly in motion during the collaborative production process.

Add to that an equal number of special effects, audio and other post production workstations and suddenly the indie filmmaker and production person becomes pivotal to the success of the M&E ecosystem.

Because time, talent and financial budgets are always tight and work is done globally, filmmakers have to work in the cloud. Even M&E storage salespeople agree the cloud fits in the M&E workflow…just not everywhere. Because along the way, a film project produces a horrendous volume of data. Click to enlarge.

Because time, talent and financial budgets are always tight and work is done globally, filmmakers have to work in the cloud. Even M&E storage salespeople agree the cloud fits in the M&E workflow…just not everywhere. Because along the way, a film project produces a horrendous volume of data. Click to enlarge.

With an industry that relies heavily on a widely dispersed workforce, embracing the ultimate cloud nirvana seems like an absolute must do! The cloud provides data redundancy, protection against hardware failure and reduces CAPEX costs.

That sounds compelling but…

It can be challenging to assess accurately the risks and benefits of cloud storage to a production project, studio or filmmaker. The Internet was never designed to support the type of workload film production requires, a low and fixed latency link.

Of course, there is also that increasing concern about protecting the key components of the project and cyber security. It sucks when an FX, audio or colorized service is hacked and segments are taken. It can be crippling when the completed project is held ransom, posted on the dark web or cannot be found.

The demand for professionally produced content continues to grow; thanks to OTT services like Netflix, Amazon, Vimeo, Hulu and others. The demand for more content by more suppliers means the need for additional production/post production operations. Coughlin Associates estimates there will be more than 954,000 high-, mid- and low-end NLE post production seats next year, an increase of about five percent.

Learn to love the cloud

The cloud is an absolute necessity for the finished project archive as well as the third backup…just in case. Nothing lets you sleep better at night than knowing there is a complete copy on the storage device in your office, in your partner’s office across the country or on other side of the globe.

And if you’re really paranoid (meaning you once lost your absolute greatest work) having a copy next to your bed means you can pet it when you go to sleep at night and again when you wake up in the morning.

Trust me; you will sleep better at night. Moreover, the storage industry will thank you.

Author:


">

Andy Marken, Marken Communications

Let us know what you think…

Log-in or Register for free to post comments…

Related Editorial Content

Media Workflow Forecast: Cloudly With Few Clear Answers

With the push to move media workflow to the cloud, broadcasters and content producers are asking “Is the time right?” Recent surveys across IT pros provide some perspective.

Cloud Post Stalls On Connectivity

While some cloud-first post services are developing a solid business, many conventional facilities remain reluctant to remote host some or all of a client project. Leading facilities share their input into whether cloud is having an impact on their investments…

​Cloud Storage for Exabyte Future

The volume of unstructured data – such as videos, social media posts, images, emails and customer details – is set to constitute 80 per cent of the 125 Exabytes of data generated by organisations in 2017, according to Ovum. Tata Communications has a solution.

Is The Hybrid Cloud Your Best Choice?

Broadcasters and production companies are being encouraged to adopt cloud solutions, with vendors claiming many benefits. Buyers should enter this arena with eyes wide open. Just because the purchase order says “cloud” does not necessarily make it a best choice for…

Cloud Broadcasting - The Hidden Costs

Unless you are a greenfield site, have one vendor to meet all your operational and creative needs, or are incredibly lucky, you will at some point need to integrate your Cloud Software-as-a-Service into the broadcast workflows. This is much easier…