The Sponsors Perspective: Capturing Immersive Audio

Strategies for capturing immersive audio for scene and object-based audio.


This article was first published as part of Essential Guide: Immersive Audio Pt 3 - Immersive Audio Objects

For a truly immersive experience, cinematic virtual reality needs spatial sound - 360° spatial audio will make or break the immersive illusion. Just as with any video production, the key to success is correct recording. While traditional microphones still play an important role in virtual reality productions, they need to be augmented with spatial microphones that capture the full 360° ambience.

This is what the Sennheiser AMBEO VR Mic does - a single compact microphone that operates on the Ambisonics principle, allowing you to capture complete spherical audio from a single point in space via its four matched KE 14 condenser capsules in a tetrahedral arrangement. For playback, the audio is rendered binaurally, allowing you to virtually rotate the orientation of the perspective in all directions. Ambisonics is supported by all major post-production and playback tools on the market today. This makes Ambisonics the appropriate tool for Virtual Reality and all other applications involving immersive sound. Basically, you capture exactly what a listener would hear if he or she was standing in that position.

AMBEO on set.

AMBEO on set.

Location Recording

Some care should be taken to record the Ambisonics signal correctly with regard to position and level, as certain errors cannot be corrected during post-production. A field recorder that has an AMBEO VR Mic mode such as the Zoom F8 will help to make this task easier. On location, the AMBEO VR Mic – fitted with the appropriate windshield or hairy cover – should be positioned as close as possible to the 360° camera as you need to patch the microphone out later in post-production, the same goes for the sound bag that accommodates the recorder. The VR Mic will usually be combined with additional conventional spot micrcophones such as wireless lavalier mics. This allows for increased flexibility during post-production, giving the mixing engineer greater control over the final experience.

Mixing

Mixing for cinematic virtual reality can be done in most standard DAWs as long as they support multichannel tracks, i.e. a minimum of four channels in a track. To support you in the mixing process, you should select an Ambisonics tool chain because most deliveries for cinematic virtual reality – including the AMBEO VR Mic recordings – are in Ambisonics. There are many tools to choose from, such as DearReality’s dearVR tool chain or the free Spatial Audio Workstation available from Facebook. As all Ambisonics tool chains operate in B-format, you first need to convert the AMBEO VR Mic’s raw 4-channel output signal to B-format using Sennheiser’s free A-to-B format converter. The converter is available as free download for VST, AU and AAX format for your preferred Digital Audio Workstation for both PC and Mac. B-format is a W, X, Y, Z representation of the sound field around the microphone. W being the sum of all 4 capsules, whereas X, Y and Z are three virtual bi-directional microphone patterns representing front/back, left/right and up/down. Thus, any direction from the microphone can be auditioned by the listener during playback of Ambisonics B.

AMBEO on drumkit.

AMBEO on drumkit.

In mixing, use the recorded Ambisonics signal as the base ambience, then add signals from conventional microphones such as wireless mics and foley sound to emphasize and build your final mix. These additional conventional sound sources need to be spatialized so that they come from the correct point in space and match the video image and the ambience. This step is accomplished by your selected Ambisonics tool chain.

During mixing, it is important to monitor your Ambisonics mix. However, before you are able to listen to it, Ambisonics must be decoded. As cinematic virtual reality will in most cases be delivered over headphones, use a binaural renderer. Best practice is to monitor via the binaural renderer that is used on the platform or device that you will deliver your content to.

It should be noted that Ambisonics is the description of a sound field. Therefore, you should never work on any of the constituent tracks of an Ambisonics signal on its own. Always use Ambisonics mixing and editing tools if you want to modify an Ambisonics signal. When using a standard multichannel mixing tool, you must make sure that changes are applied equally to all four channels – otherwise you risk altering the spatial image of the Ambisonics signal.

Delivery

Ambisonics B-format is the audio format of choice for cinematic virtual reality. All major cinematic VR distribution platforms support Ambisonics B-format, including YouTube and Facebook.

To ensure that a file is viewed properly as a 360 video with 3D immersive audio, every platform requires its own metadata and has its own file format specifications. Please check the documentation of your targeted service. If you plan to distribute to your own app or custom platform, make sure to include support for Ambisonics decoding.

AMBEO end to end workflow.

AMBEO end to end workflow.


Ambisonics In Practice

  • Recording - You can use the AMBEO VR Mic to record full Ambisonics audio, or the AMBEO Smart Headset and the KU 100 dummy head to capture binaural audio. Simple mono microphone recordings or library sounds can also be used but require special encoding.
  • Encoding -  For further processing, the various input formats need to be converted to Ambisonics B-format and rendered binaurally for headphone monitoring. For these purposes, the dearVR AMBI MICRO includes the AMBEO A-to-B and AMBEO Ambisonics-to-binaural conversion libraries. Mono sources can be encoded to Ambisonics with dearVR PRO or dearVR MUSIC.
  • Mixing - dearVR SPATIAL CONNECT enables the user to mix virtual sound sources in VR and to control their position and levels in the dearVR PRO plug-in when in Ambisonics output mode.
  • Integration - By adding dearVR AMBI MICRO to the Ambisonics master bus in the DAW, the user can binaurally monitor the Ambisonics mix with headtracking using a VR headset. This makes for an easy assessment of the Ambisonics track on 360° video platforms or in game engines.
  • Playback - Use any pair of stereo headphones to monitor the binaural soundfield, or use the AMBEO Soundbar, AMBEO Smart Headset or loudspeakers for playback.

Supported by

You might also like...

Essential Guide: Protecting Premium Content OTT & VOD Distribution

The complexity of modern OTT and VOD distribution has increased massively in recent years. The adoption of internet streaming gives viewers unparalleled freedom to consume their favorite live and pre-recorded media when they want, where they want, and how they…

Adaptive Monitoring Keeps An Eye Out For Quality While Saving Cost

As the number of channels for OTT delivery continues to grow, monitoring these channels in a highly automated way has become paramount to ensuring a good Quality of Experience for the viewer. To deliver QoE that’s as good as l…

Protected Transport Bolsters Signal Monitoring For Live Broadcasts Over IP

Any experienced master control operator or quality control manager will tell you that monitoring hundreds of feeds requires that each individual channel is delivered reliably, on time and to the exact location it was meant to go. When these signals…

New Compliance Tools Bring New Revenue Opportunities

Compliance solutions have rapidly transformed a once operator-intensive legal necessity into suites of automatic processes for new revenues.

Don’t Make An Ass Of aaS

Media companies were already gradually moving towards as-a-service business models before Coronavirus hit. However, X-as-a-Service is not a simple transition and it is not for all product or all companies.