Mega Metadata Experiment May Affect the Future of Hollywood Production

Since the dawn of digital production, many links in the workflow chain have produced their own metadata. Now the “Production in the Cloud” project is making all that metadata useful.

Over the past year, The Entertainment Technology Center (ETC) at USC in collaboration with Disney ABC Television Group has been involved with a massive research undertaking called the “Production in the Cloud” project that used a USC student film, “The Suitcase” as its template.

The implications of this experiment to the future of Hollywood production are so significant that funding was contributed by Universal Studios, Technicolor, Google, Arri, Wipro, and Equinix, with a dozen more companies donating software and hardware.

The project’s purpose was to realize the long-hoped for promise of production metadata. Could they enable all facets of a complex film shoot to not only talk with each other during the production and post production workflow but also retrieve all the metadata created along the way so it could be fed into subsequent applications?

According to Erik Weaver, the program manager of the “Production in the Cloud” project at ETC, four major goals were specified at the outset:

  1. Create a “Next Generation Cloud-Based Workflow” by ingesting metadata directly into the cloud.
  2. Feed this information into a “Cinema Content Creation Cloud”, or C4, framework for accurate identification, called the “C4 ID system”, for tracking.
  3. Evaluate methods for establishing reliable interoperability between data sources, comparing unified metadata storage vs. data silo integration via framework. While doing this, identify metadata inputs and outputs for key production software and hardware.
  4. Maintain High Dynamic Range (HDR) video throughout the workflow and report on production considerations for filming with HDR including departmental training, workflow, color grading and mastering
The production flow chart for the

The production flow chart for the "Production in the Cloud" project

In addition, Weaver told The Broadcast Bridge, they decided to try out the feasibility of having a 360-degree camera on the set constantly streaming Virtual Reality back to the studio offices “In case an executive wanted to pop in on what was going on,” as he put it.

You can actually see this video here.

“To keep the budget from expanding out of hand, we decided to use a student film from the USC School of Cinematic Arts as our working model,” Weaver tells us. “In this case it was a 21 minute graduate thesis short written and directed by Abi Corbin called “The Suitcase”.

Although this cinematic gem cannot be released to the public since it’s headed for the festival circuit, I was granted a private screening. In “The Suitcase”, Ms. Corbin has woven a clever story derived from the murky facts surrounding a piece of luggage that was actually left behind at Boston’s Logan airport by Mohammed Atta, one of the 911 hijackers.

To integrate all the metadata generated during production, ETC turned to MarkLogic who make the only enterprise-level NoSQL (non-relational) database. MarkLogic’s technology has earned high marks by providing the servers that are now running the entire Affordable Care Act’s database after the disastrous launch of Obama Care.

“We learned a whole bunch of lessons from this running of ‘The Production in the Cloud’ project,” Weaver said. “All four of our original goals were accomplished, and we will be producing a White Paper about the results in the near future. We demonstrated interoperability between multiple systems and we learned how to multiplex the bandwidth from the on-set recording cart so we could upload the same copy of a file to our ETC offices that we sent to Technicolor, each with their own proper C4 ID to keep track of them. Our metadata wrangling, thanks to a lot of work from everyone on our team, proved very successful.”

Every sophisticated system used in today’s production and post production chain generates its own metadata, but as Weaver explains to us, the ‘Production in the Cloud’ project’s purpose was to create a Rosetta Stone to let them all talk to each other.

Metadata path in the

Metadata path in the "Production in the Cloud" project

“We’re not trying to replace the metadata generated by each individual system,” Weaver said. “Each one can keep their proprietary standard if they want. We are just adding a layer on top of it that translates everything into a universal identification system that is complimentary to what they are already doing.”

The CTO of Disney ABC , Skarpi Hedinsson, told The Broadcast Bridge that as a result of the project, they could clearly see the benefit of structured and descriptive metadata and semantics to digital applications that will help streamline production processes.

"The ‘Production in the Cloud’ project was an opportunity for Disney ABC Television group working with ETC to explore how sophisticated metadata processes that are common in digital workflows can be used to innovate traditional television and film production,” Hedinsson said.

“We used ‘The Suitcase’ to identify and collect an authoritative set of characters, locations, and storylines, leveraged a video annotation platform we built internally and indexed the authoritative metadata to the film’s time code,” he said. “So we had metadata in every scene describing who was on screen, what locations were featured in the scene, and what storylines were components of the scene.”

All of this technology scooping in many layers of information throughout the filmmaking workflow will help the understanding of the production process, but only if it doesn’t interfere with the creative input from the people who craft the final result.

Chris Witt was the freelance editor who cut “The Suitcase”, and he’s glad to say the gathering and tracking of all that metadata did not interfere with the creative process on his Avid Media Composer running software vers. 8.4.5.

“It didn’t hamper me at all,” Witt said. “I could see the process working in the background, but it was invisible to me.”

However, all that mountain of metadata came to his rescue in the middle of one hectic Saturday night as he was crushing to meet the deadline for an online color grading session.

“We had one vfx shot that was a number of frames too short, even counting in the handles,” he tells us. “We desperately needed to get a new plate over to the vfx team at Technicolor so they could create a new vfx proxy for me to cut into my timeline and a high rez version for the finishing bay. Having instant access to all that metadata let us call up all the elements and fix the shot without having to delay the grading session.”

As the new world of all-digital film production is teaching us, raw data is useless unless it is put into context as metadata. Now, as the ‘Production in the Cloud’ project has demonstrated, once that metadata can be identified, indexed, and recalled it becomes a reference source that can help the lessons of one production meet the challenges of the next.

You might also like...

NAB Show 2024 BEIT Sessions Part 2: New Broadcast Technologies

The most tightly focused and fresh technical information for TV engineers at the NAB Show will be analyzed, discussed, and explained during the four days of BEIT sessions. It’s the best opportunity on Earth to learn from and question i…

The Big Guide To OTT: Part 9 - Quality Of Experience (QoE)

Part 9 of The Big Guide To OTT features a pair of in-depth articles which discuss how a data driven understanding of the consumer experience is vital and how poor quality streaming loses viewers.

Chris Brown Discusses The Themes Of The 2024 NAB Show

The Broadcast Bridge sat down with Chris Brown, executive vice president and managing director, NAB Global Connections and Events to discuss this year’s gathering April 13-17 (show floor open April 14-17) and how the industry looks to the show e…

Essential Guide: Next-Gen 5G Contribution

This Essential Guide explores the technology of 5G and its ongoing roll out. It discusses the technical reasons why 5G has become the new standard in roaming contribution, and explores the potential disruptive impact 5G and MEC could have on…

Designing IP Broadcast Systems: Part 2 - IT Philosophies, Cloud Infrastructure, & Addressing

Welcome to the second part of ‘Designing IP Broadcast Systems’ - a major 18 article exploration of the technology needed to create practical IP based broadcast production systems. Part 2 discusses the different philosophies of IT & Broadcast, the advantages and challenges…