Software Infrastructure Global Viewpoint – January 2018

AI Comes to Media Creation

Artificial Intelligence (AI) is taking center stage in many applications. It already has some history in data analysis. But this year, AI is making gains in the creation of media, especially custom-tailored subscription channels.

Ooyala recently published a report, "A.I. to the Rescue," on the advancing field of artificial intelligence being used in several media-related areas. While those of us over 40 may think of AI as the computer HAL in the movie 2001, the technology is proving to be more easily applied to consumer gear than controlling a complete space station.

The electronic publishing house CNET claims that a single story about Amazon’s Alexa and Google’s Assistant received the most comments and most likes of any article about the CES 2018 Consumer Electronics Show. Frankly I find the idea of having a version of AI setting on my kitchen counter listening to my every comment creepy. Even so consumers are snapping up the technology in all of its forms and this year we will see similar AI capabilities incorporated into many other products including new television sets.

So what does all this have to do with media? If Amazon’s Alexa can understand human language and then stream your favorite songs for playback, and Google’s Assistant can close your window blinds on command, why can’t AI just create my favorite TV channel? Turns out the technology can do that too.

AI in Media and Entertainment

Artificial Intelligence is not some god-like process housed in the cloud. Rather, AI is a rules-based chain of decisions. One key to the process is having sufficient metadata to make good decisions. AI can help here too. Metadata is information about a media project, even down to the frame level. Metadata might cover hundreds of data fields of information. At the uppermost level that could include:

Cast and Crew

Shoot Locations


Series/Episode Info


Licensing Terms

Scene Timecodes

Event Info

Because the data fields must be assigned at the beginning of a project, it is important to consider future options for its use once the program is complete.Think about other applications or channels in which it might be used. A single program on dog obedience training could later be combined with other content on training pets, or a series on dog health. Getting the original show repurposed into new additional outputs is much easier if the metadata is properly defined at the beginning of production.


Ooyala has developed a proof of concept where video files go through an automated workflow from import through to distribution, including advert insertion and recommendations. Click to enlarge.

Automate Metadata Capture

Research by Ooyala said that one hour of content requires four hours of manual labor to capture and enter metadata for that asset. Such a labor-intensive process is not cost effective.

Fortunately, cloud-based platforms like Microsoft Azure Cognitive Services can fast track such tasks by automatically capturing key information, which can then be used to create written transcriptions, provide face, object and text recognition. That and a wealth of other creative information becomes useful later when new versions of the content must be created. Examples might include the ability to produce distribution copies in alternative languages, or with country-specific rating codes. Such powerful tools can help encourage the distribution of even lower-cost content on a world-wide basis.

Creating Content

The Ooyala report notes that even the creative process of editing can be enhanced with artificial intelligence tools. Once content is captured, the next step is for someone to sort through the scenes, identifying key elements, actors, location, time-of-day, etc and then log that into a MAM system.

Such a manual process could take weeks. The video editor cannot start work until the metadata step is completed. But AI can eliminate much of this manual drudgery time in two ways: (1) by automating the capture of a wide range of metadata through face, object and text recognition; and (2) by optimizing search to find key elements within the large pool of content. This way, when an editor needs just the right shot, she can ask AI to find it.

Too Many Choices?

While AI and metadata can enhance the production of entertainment, it can also help viewers find content. Today’s consumers face a bewildering number of choices when to comes to entertainment. Bruce Springsteen nailed this problem way back in 1992 with the song, “57 Channels And Nothing On.” Recognize that today viewers have ten times that number of channels available. How does she find something of interest?

AI to the rescue on two fronts. First, AI will make it easier for viewers to find the content they like. Second, such a selection process encourages higher viewer engagement making the content more valuable to the publisher. For these reasons, personalization is one of the top benefits of AI.

Armed with viewer behavior data, a machine language process can help content distributors better position and serve up the most-desired content to the more valuable viewers. Ooyala calls their recommendation engine, Ooyala Discovery. By leveraging machine learning of viewer behavior with content trends, the viewers can more easily discover new and desired content. AI may reduce the problem of 500 channels with nothing on.


This type solution is being used by ZoneTV. The network uses Ooyala media logistics tools and Microsoft Cognitive Services’ Video Indexer. The two work together to create a first-of-its-kind customizable suite of linear TV channels. Want to see an example, click above on the first video's link.

The channels will initially appear like any traditional linear channel. ZoneTV’s unique service allows consumers to combine linear, on-demand and customized choices into a new offering called ZoneTV Dynamic Channels.

Leveraging both ML and AI components enable ZoneTV to curate 6,000 hours of videos on the fly to create a unique and personalized experience for the consumer.

Combined, these tools offer advanced algorithms that characterize content. The platform then automatically extracts and analyzes metadata to identify video genre and content sentiment, and pulls topics from speech and text. Finally, the software translates captions into multiple languages and integrates subscriber analytics.

This provides quick scalability for ZoneTV as it adds additional content, and reduces manual processes that can slow content curation and introduce errors in metadata translation and application.

Cue The Commercials

Add to the viewer-selected content the known demographics of that household, device being used and even location. The result can be programmatic ad serving. AI can quickly process such data then target an advertisement most likely to get a positive viewer response. Combined with receiver location data, a grocery store ad for eggs might appear on some devices and a restaurant ad on other viewers’ sets—all on the same program at the same time.

The benefits of artificial intelligence are only now beginning to be explored. Expect many more concepts and workflow-enhancing developments soon.

I recall not that long ago, a fellow master control engineer who told me that “automation could never replace a live operator.” He was wrong. Software has removed much of the human decision process.

Broadcast, playout and business software will only get smarter while creating new opportunities for those clever enough to use it.

Editor's Note:

This article used information from an Ooyala white paper, "A.I. to the Rescue." It is available at the link. 

Commenting is not available in this channel entry.

Related Editorial Content

AI Brings New Life To Online Channels Through Personalization

Like the set top box, the traditional TV channel continues to defy predictions of its imminent death and now it appears like it is gaining a further lease of life in the OTT world with the help of AI techniques…

Machine Learning and AI dominate SMPTE 2017 conference

Attendees to Society of Motion Picture and Television Engineers (SMPTE) conference might conclude that Machine Learning (ML) and Artificial Intelligence (AI) have progressed way beyond hype as they start to enter just about every aspect of video production and distribution.

Industry Embraces AI For Streamlined Clip Search Across Petabytes (And Soon Zettabytes) Of Storage

In today’s highly competitive media environment, companies are always looking for ways to streamline their operations and speed up the processes involved in content creation. One of the most critical is post production workflows and the need to find a…