Camera Matching in Multi-Codec and Multi-Gamut-Colour Space

I remember “painting” the cameras. Riding the CCU’s to make sure that the trees did not change from emerald green to grass green when the fader bar on the switcher was pulled. Colour grading in the post suite has made the crude tools we used to use outdated, but what about live production in the new multi-digital world? When colours change between shots it breaks the illusion. We have become lax about this because the tools are simply not available.

A helmet camera, a DSLR, a mobile phone and your trusty CCD, can we match them?

Since you will probably have to accept the consequences of delay anyway, See Ned Soseman’s see boat racing stories, throw some graphic card processing in each feed and it becomes possible.

If you can get a reference into the shot, “Can you center the logo on the sponsors tent?” capture that, then software can do a lot. The same logo with known lighting will tell you how it is supposed to look but can the software determine if the colour shift is because of the camera or because of location lighting? No, because you have two variables.

One thing that would help would be the logo shot with the same camera under the same lighting as your reference shot, except that we still have the variable settings on the camera. If we can get those settings off the camera (streaming metadata anybody?) then we can automate the process. That’s a lot of ifs! For the time being an operator will still be required. What is the skillset and how much time per camera/shot will be required?

Camera Matching or Colour Grading?

One of the big differences between camera matching and colour grading is the goal. Live camera matching implies that you are trying to reproduce the scene accurately, and compromising this accuracy is only done reluctantly in order to get all cameras to match. Colour grading is about the “look” and “tone” of the image achieved. As long as the systems were completely different, chemical reactions for film grading and analogue circuitry for camera control, there was no overlap. Now that the underlying hardware, graphics engines, is the same, it may be time to rethink.

This implies that the needed skill set will be the same whether grading or camera matching, except that we have to do it on a live stream—which means we got one chance at getting it right—no re-do’s

So how much image manipulation can be done within today’s graphic systems? Tons.

In this image the performer is initially show prior to image enhancements. Next, we'll let the magic of software and expertise dazzle you.

In this image the performer is initially show prior to image enhancements. Next, we'll let the magic of software and expertise dazzle you.

Now, let’s see this process in action. In this video a good graphics system has been combined with the skills of an excellent operator. You will see Figure 1 was changed into Figure 2, all with live video.

Final image enhancement.If you watch the video, you will notice the artistic skills used to manipulate the image.  Images courtesy Boggie - Nouveau Parfum, via Vimeo.

Final image enhancement.If you watch the video, you will notice the artistic skills used to manipulate the image. Images courtesy Boggie - Nouveau Parfum, via Vimeo.

OK so live Photoshop may not be want we want. Even though the final image is unreal, the point is that today’s graphic processing engines can do this live.

One grading manufacturer offers to automatically give you a primary base grade by analyzing shots containing standard colour charts. “This lets you set the source gamma, target gamma and target colour space for the chart used in your shot. Simply use the chip grid to identify the colour chip chart and the images will automatically balance, even if they come from different cameras, under different lighting conditions and with different colour temperatures”.

So if I am shooting from a helicopter I should throw one of these charts into the air and hope to get it in the shot?All kidding aside, this is a good idea, and should be part of the solution. Another part is the ability to pick an object in two shots and have the gamut etc. of the whole scene shift to match. Doing this on the center of attention will go a long way to alleviating colour shifts that may take place elsewhere. Still, running this sort of thing without some kind of operator oversight is not a good idea.

A camera control unit (CCU) on the left and a grading system on the right. Appearances alone tell you different operating skills are needed.<br />

A camera control unit (CCU) on the left and a grading system on the right. Appearances alone tell you different operating skills are needed.

Neither of these interfaces really works for what we want to do. The evolution of these interfaces, in order to achieve the results required, took decades. I have spoken to software developers and they say they are working on a solution that would not require an additional operator to achieve the simultaneous adjustment of all feeds for a specific look.

Technical image feedback, which is currently supplied by waveform monitors and vectorscopes, may disappear just as oil pressure and ampere gauges have disappeared from the dashboard. The manufacturers are looking for input from the user community, so let’s look at a typical use case.

Again from the boat racing scenario: “One unit went up in a helicopter with an iPhone camera source. It was on-air live for nearly 10 minutes, flying over of the race course lined with thousands of spectator boats tied together.”

I am not going to write a formal use case but the informal version looks like this: The cameraman notifies the TD that she is ready to transmit. The TD gives the OK. The camera person starts transmitting. The TD checks the signal and makes any necessary adjustments or asks the shooter to change something. The signal goes live when it can or must.

Legacy CCUs allow the operator to control iris, gain, etc., as if we were working on RAW data. An iPhone doesn’t give us this so we need the camera operator to intervene. The software engineering required to allow for Wi-Fi control of these features is not a major effort. Even if we cannot have a RAW stream from the camera, Wi-Fi control will give us most of the advantages.

Any required image signal processing should take place before format conversion, but include a feedback loop with the converter so that parameters can be automatically optimised. The ergonomics of the TD’s interface are open to discussion, but a simple indicator of degree of match with an optional “area of interest” should be implemented.

Manufacturers see this as an opportunity. It is up to us, the users, to tell them what we want, otherwise they will modify what they have as they see fit.

You might also like...

5G Service’s Use Of C-Band Frequency Causes Interference With Aviation

Although 5G service providers AT&T and Verizon have agreed to delay the launch of more powerful wireless transmissions sent within the C-band spectrum obtained in government auctions from broadcasters around airports—due to concerns over interference with airplane a…

Getting The Whole Picture With Volumetric Video Capture

With talk of an impending Metaverse and augmented reality graphics increasingly becoming part of mainstream television broadcasts and live eSports events, top graphics artists are looking past traditional 3D animation and virtual environments and using fully rendered 360-degree 2K and …

HDR: Part 44 - Creative Technology - 2021 Cinematography Roundup

Filmmaking is now an artform with a long history, and that means a lot of received knowledge. Things like primes being better than zooms, and characterful primes best of all. The public attitude to that orthodoxy, though, is starting to…

HDR: Part 43 - Creative Technology - Color Balancing Different Light Sources

Creating an LED light that looks reasonable, both on camera and to the eye, is no longer that difficult.

2021: Remote Tech Challenge Met

If 2020 was considered a disruptive one for the television production community, 2021 was a year where trial and error and the lessons learned became real-world REMI deployments to keep live sports and entertainment content on the air. Production studios too learned…