High Efficiency AAC Audio Coding

High Efficiency AAC (HE-AAC) refines the original AAC coding techniques but remains compatible with MPEG-2 Part 7. It can deliver almost uncompressed CD-quality sound at 32 Kbps.

HE-AAC is known by a variety of other names:

These are some important features that HE-AAC introduces:

• SpectralBand Replication (SBR).
• Parametric Stereo (SSC).
• Perceptual Noise Substitution (PNS).
• Long Term Predictor (LTP).
• Low Delay (AAC-LD).
• MPEG-4 Scalable to Lossless (SLS).
• AAC Scalable Sample Rate (SSR).
• Structured Audio (SA).
• Text To Speech (TTSI).
• Error Resilience (ER).

Structured Audio and text to speech are extremely compact because they describe a sound that the player renders entirely in the receiving client. This can deliver performance as low as 100bps.

There are many historical versions. Each additional profile will add more variants:

AAC Tools & Technologies

The AAC coding tools were reorganized in MPEG-4 to make them more flexible. New tools were added and older tools were refactored into separate items. They are now described as Audio Objects, each one having a specific identity and purpose. Some of them are containers for descriptive information. This is an additional layer of abstraction facilitating profile definitions.

These are the basic AAC audio-objects and the technologies that are used inside them. Refer to the MPEG-4 Part 3 numbered sub-parts in the ISO MPEG-4 part 3 official standard for functional descriptions of these tools and how they are mapped into the profiles. The sub-part references are all in the MPEG-4 Part 3 standard:

Spectral Band Replication (SBR)

Spectral Band Replication discards redundant harmonic components in the encoder but reconstructs them by replicating the lower frequencies to derive suitable replacements in the player. This can be used with any codec.

A typical stream of audio might be coded to a target maximum 128kbps bitrate. This would reproduce all frequencies up to 15kHz with a small reduction in the frequency response at the top end.

SBR cuts off the incoming frequencies at around 7.5kHz. This loses a lot of the detail but reduces the bitrate to 64kbps.

The higher band from 7.5kHz to 15kHz is processed through a more aggressive compression tool. This generates a description of the high frequency sounds that can be used in the decoder to reconstruct them from the lower order harmonics. The description is carried in auxiliary segments within the stream and only adds 1.5kbps to the bitstream (65.5 kbps in total).

The player transposes the lower frequencies into the upper band where it can filter and mix them in using the descriptions in the auxiliary segments derived from the higher frequencies. This is practical because the upper frequencies are likely to be harmonics of the lower band with a different amplitude envelope.

Perceptual Noise Substitution (PNS)

The bitrate gains from using PNS are often not worth the computational workload when the audio is of a high quality.

For noisy audio sources, the noise can be filtered out and described as control parameters for a pseudorandom noise generator in the player where they can be recreated.

Parametric Stereo (PS) & SinuSoidal Coding (SSC)

Parametric stereo exploits the similarity between the left and right channels to code them more efficiently.

The two channels are mixed down into a single monophonic channel and coded at full resolution. This is a base from which two differential channels can be derived. Those differences can be coded to a 3-kbps bitrate using SinuSoidal Coding.

The player decodes the mono channel and applies the differences to make the left and right outputs.

Instead of delivering two full bitrate channels, the encoder delivers one full bitrate channel and two very low bitrate differential channels.

Scalable Sample Rate (SSR)

Scaling the audio coding by splitting at the sample level is an interesting alternative to using base and enhancement layers.

If we de-interleave CD audio into three scalable streams then stream one carries the first sample, stream two the second, and stream three carries the third and perhaps the fourth. The next sample is added to stream one and so on. This yields two 11 kHz sample streams and one 22 kHz stream which can be used by the target device in any combination.

A low-quality service can be reconstructed from one stream or all of them can be combined to reconstruct the original sample stream.

Error Resilience (ER)

Patent Licenses

Patents for MPEG-4 Audio coding are managed by Via Licensing. Contact them for a license if you design and sell an Encoder or Decoder (Player) of your own.

Content owners do not need a license to distribute their MPEG Audio content. They have implicitly paid for it when purchasing the encoder or decoder.

Patents for AAC baseline technologies expire in 2028 and some newer extensions will have active patents until 2031.

Profiles & Audio Objects

A profile could use a single Audio Object while other profiles stack the tools hierarchically to make more efficient and sophisticated coders. Complexity requires more computational effort and increases the latency:

• MPEG-2 AAC-LC profile only uses the Low Complexity AAC-LC audio object.
• MPEG-4 AAC-LC adds Perceptual Noise Substitution (PNS).
• MPEG-4 HE-AAC v1 adds Spectral Band Replication (SBR).
• MPEG-4 HE-AAC v2 Adds Parametric Stereo (PS).

Because the specification is hierarchical, HE-AAC v2 players can decode any of the lower stacked levels.

These are the standardized profiles. Organizations such as Fraunhofer create their own proprietary profiles.

The Fraunhofer Scalable Lossless Coding (HD-AAC) is not the same as the SLS support defined by the MPEG-4 standard.

Refer to section 1.5 of the MPEG-4 Part 3 standard for a detailed description of the Audio Objects and how they are mapped to the profiles.

Media Type Identifiers

Because HE-AAC is coded differently to classic AAC, a new media type is needed so that browsers can distinguish between the two formats:

Relevant Standards

The vintage column indicates the most recent base standard, corrigenda or amendment. Although the latest versions are indicated, earlier versions may contain relevant information that is removed from later standards. Some devices may be compatible only with an earlier version and you should use that if necessary when developing your services for them.

There is a gradual refactoring of the MPEG standards underway so they can benefit from reusing supporting technologies without needing to repeat them. The MPEG-D and Coding Independent Code Points standards are examples of that as are the ISO 23XXX group of MPEG relevant standards, which provide additional infrastructural support outside of the individual coding specifications.

Applying HE-AAC

Audio and video compression is a complex subject. We balance it here at a level sufficient to explain the fundamentals whilst avoiding a deep dive down the rabbit hole. Consult the MPEG-4 Part 3 standard if you need to explore MPEG Audio coding in greater detail.

The AAC audio standard is increasingly being used with High-Definition TV services (HDTV). This is supported by the DVB standards that are distributed by ETSI. HE-AAC is particularly relevant for mobile TV using DVB-H.

Digital radio services such as DAB+ and Digital Radio Mondiale are also adopting HE-AAC.

These Appendix articles contain additional information you may find useful: