AI in Autodesk Post

Autodesk is using Artificial Intelligence to teach its Flame 2020 software how to help creative artists do their job more efficiently when working on integrated VFX, color correction, look development and final mastering.

Artificial Intelligence (AI) was a leitmotif flowing through many of the introductions at April’s NAB Show 2019 and as usually happens, the most intriguing systems impacted by it were in post-production.

But what AI actually is sometime lies in the inner eye of the beholder, since AI can mean different things to different people.

So the series of articles of which this installment is the opening salvo is going to recognize that when delving into the implementation of AI in post, it’s useful to try to lasso this untamed topic with a definition.

Since we are beginning with some of the software released by Autodesk, I was very grateful when Will Harris, the Flame family product manager at Autodesk, avowed that to him, AI means, “Utilizing established methods for machine learning to produce an algorithm that can predictably produce results.”

In other words, you create a system that can learn, then teach it stuff, and let it autonomously do tasks for you based on what it has learned.

Facial recognition can determine not only identity, but also mood.

Facial recognition can determine not only identity, but also mood.

Harris started with the example of facial recognition, in which a variant of AI can identify different people by tagging their facemasks, but said that Autodesk has progressed the art far beyond that.

“In the Media and Entertainment space at Autodesk, we are taking the intelligence of VFX into the world of post-production,” he began. “For example, if you render something in a c.g. rendering engine you can produce a beauty image, but you can also produce render passes or something that is called AOV’s or Arbitrary Output Variables.

“They are the intelligence of the renderer spat out in different forms so you can post-process and adjust those renderings,” he continued. “A great example would be if in addition to that beauty image you rendered out a diffuse, non-lit version, and also a specular highlight and shadow pass. If you combine those all together, you could create a unique look to that face which did not necessarily exist before.”

Thinking in these terms lead Harris’s Flame 2020 development team at Autodesk toward the first way they decided to implement AI. Manually rotoscoping human images can be a very time consuming operation, but maybe AI could help alleviate this.

So now, with the release of Flame 2020 at the NAB Show last April, this result of AI learning is baked into the software.

Now, if you feed the system a source image and a reference Z-depth, it can produce a new image with the intelligence to identify the desired separation.

Flame lets you see the Z-depth created, and manipulate it.

Flame lets you see the Z-depth created, and manipulate it.

“The system also knows that sky is usually at the top of the image, and ground is conventionally at the bottom,” Harris said. “So we gave the software what is sometimes known as a ‘suck it and see’ tool to give it a help figuring that out.”

You might like to know that The Cambridge University English Dictionary defines “suck it and see” as UK informal for “to try something to find out if it will be successful”.

“What you end up with is sort of an automatic tool in Flame 2020 that you can try and see if you like the results,” Harris said. “If the outcome is acceptable, you keep it. If not, you try something else.”

A second implementation of AI in Flame 2020 has to do with what c.g. renderers call a “normals pass” in which you are trying to re-light a face or any object to create a different effect.

Suppose the original shot was too flat, and you need a more dramatic look. Now you can create a “normal map” in 3D using another AI feature of Flame 2020.

“Again, through the training of scanning 100,00’s of face images, we’ve taught the software how to identify various ethnicities, facial variations, skin tones and expressions,” Harris said. “We actually used many 3D renders and then taught the software how to apply what it had learned to real human faces.”

With this kind of image manipulation control...

With this kind of image manipulation control...

...it is amazing how you can alter the impact of facial expression.

...it is amazing how you can alter the impact of facial expression.

This is all you need to dramatically enhance the lighting on a character’s face.

“Or, you can invoke many different kinds of keyers by using AI as a reference,” Harris said, “using any shape or shot that you choose. It’s all a matter of training the system which shape you want to pattern the key after.”

“Even if it doesn’t work perfectly, it gets you 80% of the way there,” Harris finished up. “Then you can take it from there. This is only the beginning of the ways that Autodesk is planning on incorporating AI in our Flame 2020 and other softwares.”

Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

Sensors and Lenses - Part 1

The push to create the ideal digital cinematography camera has now been going on for, arguably, two decades. There were a couple of standout attempts in the 1980s involving high definition tube cameras, but the introduction of Sony’s HDCAM t…

Real Film Grain For Video

People have been making pictures for both the big and small screens for almost a century. In an industry with a history that long, it’s no surprise that the perpetual search for something new has long been tempered by a…

For DOPs: The Peril Of Larger, Brighter TVs

Each year, as the TVs in our homes grow larger and brighter, DOPs have to wonder how this will affect our craft and the integrity of our images. As it is, HDR is touted as a kind of industry panacea,…

Predicting Editing Performance From Games And Benchmarks: Part 1

In the good old days when you were thinking about upgrading your computer you began by reading printed reviews such as those published by Byte magazine. These reviews usually included industry standard benchmarks. Now, of course, you are far more…

What You Need To Know About Thermal Throttling

With 6K acquisition becoming more common, you may be considering getting ahead of the editing curve by upgrading your computer system. Likely you’ll want a hot system based upon one of the new AMD or Intel 6- or 8-core m…