A Video Tinkerer Solves a Perplexing Audio Problem

The short history of video is full of working videographers who solve major problems in their craft and go on to build companies to share their inventions with others in the industry. Ross Lowell of Lowel Lighting and Anton Wilson of Anton-Bauer are two well-known examples working image-makers turned inventors. Now, add Bruce Sharpe to the list.

In 2000, Bruce Sharpe and his wife shot videos of live events like stage presentations and dances. To get higher production quality, they bought a pair of camcorders. To achieve better audio, they often took the sound feed from the venue’s audio board or placed a small portable audio recorder closer to the performers.

Bruce Sharp had a nine-year quest to sync audio with video

As Sharpe graduated to more professional video jobs, he found the manual syncing of outboard sound with his video to be a time consuming, laborious job. He needed to find a way to automate the sync process. So he began shopping for software that would do the job. To his surprise, none was available.

Thus began Sharpe’s nine-year quest to create his own. “Because of my technical background, which was analyzing satellite imagery and visual data, I figured it was a solvable problem,” Sharpe said. “So I started tinkering with it.”

Sharpe knew that sync could be achieved by viewing and matching audio waveforms. A manual method was to match the peaks and valleys of the audio from the camera and the separate audio track with the eye. Then he would replace the poor audio with the good. But that was way too difficult. A computer could do it much better and faster, Sharpe surmised.

However, the problem was far trickier than Sharpe at first thought. He realized that even though the sound was the same, the waveforms often looked different. The waveform from the camera — which was further away than the better sound source — often didn’t look the same to the eye.

So the tinkering began. He set a goal to create a software application that would automatically sync the higher-quality sound with the picture. The user would not have to tweak it or be bothered with prep in the field. It would just happen quickly and reliably.

From the original concept to finished product took over eight years. In 2009, Sharpe started a company called Singular Software and at that year’s NAB, he introduced two products: PluralEyes and DualEyes. PluralEyes worked with multiple sound sources and video, while DualEyes worked with one sound source and the video.

Sharpe’s timing couldn’t have been more perfect. The previous September, Canon had introduced the EOS 5D Mark II, the first Canon DSLR camera to have high-quality video recording capabilities.

The Canon 5D MkII was a revolution for the cost-conscious videographer

In the design, Canon paid far more attention to the video than audio, and this left a wide window of opportunity for double system sound software like Sharpe had created. “Canon’s introduction blew the whole DSLR thing wide open,” Sharpe said. “They created the need to do separate audio. It broadened the market immediately.”

In July, 2012, Sharpe’s Singular Software merged with Red Giant, a maker of tools for digital video makers based in Beaverton, Oregon. Today, Sharpe is Business Unit Manager of Red Giant’s Shooter Suite. PluralEyes 3 is one of the products in the suite. “It’s all the tools you need as a shooter to get everything ready for post-production,” Sharpe said.

Today, PluralEyes is the market leader for syncing software. A number of smaller competitors have come and gone. One of the newcomers, Pixelynx Labs’ DreamSync, is taking on PluralEyes with a simpler, lower-cost product.

Jon Acosta, designer of DreamSync, said the goal of his company was to create the easiest to use alternative to PluralEyes. He targeted his potential customers as users of basic editing applications like iMovie, Windows Movie Maker and Sony Vegas, which do not have built-in sound sync capability.

DreamSync is ideal for television news, or anyone seeking a very simple way to do double system sound with their video,” Acosta said.

The major competition to PluralEyes comes from Adobe and Apple. Double system audio sync software is now deemed so important that both companies have added basic audio sync capability to their main professional editing products.

In Apple’s Final Cut Pro, the feature is called automatic audio sync. “Final Cut Pro compares waveforms from the camera audio and second audio source to sync sound automatically — down to the sample level,” Apple wrote about its editing feature.

In Adobe Premiere Pro, the audio sync feature is called Merge Clips. “This will streamline the process by which users can sync audio and video which have been recorded separately (a process sometimes called double-system recording),” wrote Adobe. “You’ll be able to select a video clip and synchronize it with up to 16 channels of audio by using the ‘Merge Clips’ command.”

Apple and Adobe’s sync technology, said Sharpe, are very basic. “It works on a few clips at a time,” he noted. “PluralEyes covers a much wider variety of projects and formats. You get a lot of visual feedback. You can see how things are progressing. You can do quality control afterwards. It's much more convenient to use than the very basic functions in those editing applications.”

As to the competition, Sharpe is not worried. “When people hear the idea of sound sync software, some say 'I can do that.' So they try. So far none have been very successful in the marketplace. That's because this is actually a very hard problem to solve.”

PluralEyes is now in beta for version 3.5, which adds “Drift Correction,” a new feature that corrects audio in long clips that can drift out of sync over time. It also adds functionality for more media formats and offers additional playback capabilities.

“We also continue to improve the basic algorithm,” Sharpe said. “It’s now faster and more accurate than in earlier versions.”

For the future, Sharpe said PluralEyes will get greater integration into the Red Giant Shooters Suite. The suite features not only PluralEyes, but BulletProof, an on-set footage prep and delivery solution that takes the user from backup through export. There’s also LUT Buddy for sharing color decisions made on set, Instant HD for up-rezzing and Frames for deinterlacing to integrate new footage with older video formats sharing the edit.

For Sharpe, he picked the right problem to solve at the right moment and has created an essential application for the age of DSLR videography.

You might also like...

An Introduction To Network Observability

The more complex and intricate IP networks and cloud infrastructures become, the greater the potential for unwelcome dynamics in the system, and the greater the need for rich, reliable, real-time data about performance and error rates.

2024 BEITC Update: ATSC 3.0 Broadcast Positioning Systems

Move over, WWV and GPS. New information about Broadcast Positioning Systems presented at BEITC 2024 provides insight into work on a crucial, common view OTA, highly precision, public time reference that ATSC 3.0 broadcasters can easily provide.

Next-Gen 5G Contribution: Part 2 - MEC & The Disruptive Potential Of 5G

The migration of the core network functionality of 5G to virtualized or cloud-native infrastructure opens up new capabilities like MEC which have the potential to disrupt current approaches to remote production contribution networks.

Designing IP Broadcast Systems: Addressing & Packet Delivery

How layer-3 and layer-2 addresses work together to deliver data link layer packets and frames across networks to improve efficiency and reduce congestion.

Next-Gen 5G Contribution: Part 1 - The Technology Of 5G

5G is a collection of standards that encompass a wide array of different use cases, across the entire spectrum of consumer and commercial users. Here we discuss the aspects of it that apply to live video contribution in broadcast production.