About HEVC

The HEVC codec is based on the earlier work on AVC and prior coding technologies. New techniques are employed to reduce the coded output size even further.

Video content now delivers much higher resolution than H.264 was designed for. Early trials revealed that H.264 could encode 8K video better than expected, but AVC was never optimized for 8K content. It was recognized that new ideas could halve the resulting output file size and improve performance.

Eventually a new codec is mandated. The higher resolution allowed many more redundant macro-blocks to be eliminated. HEVC was designed to achieve compression ratios of 1000:1 and supports resolutions up to 16K.

The HEVC codec is optimized for 8K and can operate up to 16K. Prototype 32K cameras are already being designed that will require very high-performance network support.

About The ISO Standard

Like the preceding AVC standard, the HEVC/MPEG-H Part 2 standard (ISO 23008-2) is large and complex. The core of HEVC has extensions for Scalable, Multiview and Stereoscopic 3D viewing support.

The document structure is similar to AVC and the same section numbering scheme has been retained. Having studied the AVC standard, much of this content will be familiar but there are important and subtle differences to take note of throughout.

The Network Abstraction Layer packets (NAL units) described in Section 7 have a slightly different format. The payload carries Profile, Tier and Level parameters which are more logically defined than they were for AVC. These are described in the annexes as before.

The latest edition was published in 2025. It adds more profile specifications and some new SEI messages.

HEVC Performance Factors

HEVC differs in performance and CPU processing workload compared with AVC encoders:

• Frame rates - HEVC extends the maximum to 300 fps. This is significantly higher than the nominal 60 fps supported by AVC. Note that level 6.2 in AVC is rated at 120 fps as an exception.
• Image quality - Performance evaluations verify that the image quality of HEVC is superior to AVC at any given bitrate. Noise levels, color spaces and dynamic range are all improved.
• Image sizes - The operating range is optimal when dealing with 8K video. The latest revisions have increased this to 16K at extremely high bitrates.
• Size of coded output - HEVC was targeted to halve the bitrate of AVC coded output. This is comfortably achieved with SD content and exceeded with higher definitions.
• Encoding speed - HEVC needs to do significantly more work than AVC to reduce the output size.
• Decoding performance - HEVC delivers content that is easier to decode. The increased coding workload is offset by improvements in the receiving client-player performance.
• CPU utilization - Increasing the coding workload requires more CPU capacity. HEVC can be parallelized more efficiently and makes better use of multiple CPU cores to spread the workload.
• Interlacing - There is no support for interlaced video in HEVC. Interlaced content can be managed by coding individual fields as separate images and passing SEI messages to the player.

HEVC Feature Advancements

These are some of the improvements that HEVC offers when compared with AVC coding:

• Video coding layer - Better de blocking and edge preserving filters.
• Color spaces - More alternatives supported.
• Sample sizes - Allow greater ranges of colors.
• Coding tree units - Can be sub divided more flexibly than macro blocks.
• Lossless - HEVC supports truly lossless coding which improves on the region constrained partial lossless coding in AVC.
• Still picture support - This facilitates HEIC image compression. This is useful in situations where the video is displaying a stationary image.
• Screen content - Coding support for text and graphics images. This may provide better compression for traditional 2D animation as well.
• SEI - Supplemental Enhancement Information supports the delivery of additional metadata signals.

Improved Video Coding Layer

The Video Coding Layer works conventionally by splitting the first picture into blocks. Intra-prediction is used to eliminate redundant identical or similar blocks within that first picture. Subsequent pictures are assimilated, and Inter-prediction looks for redundancy by comparing new pixel rectangles with the previously processed blocks.

Note the similarity in terminology naming. Intra and Inter are different in scope. Intra works within a single frame. Inter works across several frames.

The loop filters are applied after the block prediction phase is completed. There are two loop filters in HEVC:

• DBF - The Deblocking Filter reduces artefacts at the block boundaries. This is simpler than the equivalent filter in AVC. It is designed to be easily parallelized across multiple processors as a result.
• SAO - The Sample Adaptive Offset reduces sample distortions. This helps process hard edges and reduces ringing artefacts. It also helps reduce contouring artefacts.

Coding Tree Units (CTU)

AVC divides pictures into macro-blocks. These are based on 16 x 16 pixels but can be sub-divided into smaller (4 x 4) rectangles.

HEVC uses Coding Tree Units up to 64 x 64 pixels in size to improve redundancy detection. Internally, they implement a Quadtree data structure to support variable block sizes. A Quadtree is a node-based nested tree structure where every node has exactly four child nodes attached to it. The Quadtree can be recursively sub-divided and nested until the block size reduces to 4 x 4 pixels. Each layer must have exactly four leaf nodes stemming from the parent node. Each branch can support a further level if that helps resolve fine detail. Large areas of similar pixels do not require such deeply nested trees.

Here is an illustration showing progressively deeper Quadtree nesting. The third level is only omitted to reduce the number of examples.

The source image may not be an exact integer multiple of the Coding Tree Unit (64 x 64) pixel size. Because the origin of the image is in the top left corner, the extreme right and bottom edges of the image may not result in completely filled blocks. The encoder can work around this by partially filling a square block. The decoder can crop the additional empty pixels when the content is unpacked and rendered into the display rectangle.

HEVC can group multiple CTU blocks into tiles and slices if this helps to find redundant duplicated parts of the image.

Slices can be used in various Intra and Inter modes or clone other slices from the same or other images and compute the residual deltas from them.

Tiles can be decoded independently of the rest of the picture.

I-slices behave like inter frames in AVC.

P-slices behave like predictive frames.

B-slices behave like bi-predictive frames.

Motion vector prediction support is improved by having 33 intra-prediction modes. This is significantly better than the 8 that were implemented in AVC. Better support for the DC coefficient in the Discrete Cosine Transform (DCT) also helps reduce the output size.

More Diverse Color Spaces

Many alternative color spaces are supported in HEVC. These are optimized for various content sources and other standards:

• NTSC video.
• PAL video.
• Generic film stock.
• Rec 601.
• Rec 709.
• Rec 2020.
• Rec 2100.
• ST 170.
• ST 240.
• sRGB.
• sYCC.
• xvYCC.
• XYZ.
• RGB.
• YCbCr.
• YCoCg.
• Externally defined color spaces.

Pixel Sample Sizes

HEVC supports more pixel sample sizes. The 8-bit sampling is similar to AVC, but HEVC adds 10, 12, 14 and 16-bit alternatives depending on the profile selected. This will yield more vibrant color rendition and facilitates HDR support. Monochrome images are supported natively rather than desaturating color pictures to simulate them. This improves bitrate and coding efficiency because the redundant chroma samples are not included.

Supplemental Enhancement Information (SEI)

Metadata is passed from the encoder to the receiving client-player in SEI messages. The player uses them to apply post-processing to the decoded images.

These are a few of the SEI messages that apply to HEVC:

• Remapping color spaces from one to another.
• Hints for defining transfer functions to convert from SDR (Standard Dynamic Range) to HDR (High Dynamic Range) renditions.
• Support for Hybrid Log Gamma (HLG) for open-source HDR applications.
• Describing the color primaries, white point and maximum-minimum luminance to define the originating display color volume.
• Giving time-code values relating to the content for archival purposes.
• Providing details of the ambient lighting environment where the video was authored.
• Offering support for 3D displays.

ISO standard 23002 part 7 (2024) describes Supplemental Enhancement Information (SEI) messages and Video Usability Information (VUI) parameters. It is cross-referenced from video coding standards to avoid repetition therein. This standard is particularly relevant to the Versatile Video Coding (VVC) video format.

Profiles

Profile selection determines how the encoder operates and selects a sub-set of the available coding tools. This affects the coding efficiency and size of the output bitstream. Profile and level signaling is less complex than it was with H.264.

HEVC profiles are grouped in a similar way to the AVC profiles but can be gathered into different categories. The core profiles are extended in different ways according to the support you need:

Note that profiles can simultaneously belong to the High Throughput and Screen Content Coding Extension categories.

The HEVC profile names are more descriptive than AVC profiles. Color sampling and bit depths are both clearly indicated.

Color sampling values range from 8 to 16-bits depending on the profile. Some profile names include an integer value to indicate the number of color-sampling bits. Assume that the sample size is 8-bits unless it is specified otherwise.

Chroma sampling formats are also implied by the profile name. Assume the default 4:2:0 format unless the profile describes a monochrome picture in which case use 4:0:0. Profiles operating in 4:2:2 and 4:4:4 mode will have the appropriate description embedded in their name. Profiles using 4:4:4 chroma sampling can also support delivery of 4:0:0, 4:2:0 and 4:2:2 content.

Note that profiles can simultaneously belong to the High Throughput and Screen Content Coding Extension categories.

Tiers

HEVC introduces two Tiers of operation which are related to the Profiles and the corresponding Levels:

• Main tier - Designed for most applications, this tier is available across all levels but constrains the maximum bitrate to a much lower level than the High tier. Optimized for SD picture sizes and smaller. It is suitable for most consumer applications.
• High tier - Cannot be used for levels 1 to 3. This is designed for HD and higher picture sizes with levels 4 to 7 and where the application demands better performance.

Levels

Levels describe the picture sizes in the receiving client-player and are similar to AVC levels. An additional level (7) increases the scope to support 16K images. The level signaling is much simpler in HEVC than it was in AVC.

These are the main level groups and picture sizes:

The encoder trades off frame-rates and picture sizes to remain within a given bitrate defined by the level. This affects the decoding speed and picture buffering limits. Maximum values for picture size and frame rates for the different levels are listed here as examples. Consult the standard for more detail:

Levels 6.3 to 7.2 were defined in a later edition of the standard. They introduce the 12K and 16K sizes for use with future display technologies beyond the current 8K formats.

Container Files

Most of the containers that were compatible with AVC can be used for storing HEVC content with a couple of exceptions. They are summarized here:

Because HEVC can be carried in the same MPEG container files, the file extensions that applied to AVC content also work for HEVC.

Market Penetration

HEVC is a format for the future. As technology platforms advance, HEVC might achieve the same market penetration as AVC eventually. It may benefit from the viewing platforms deploying larger screen sizes (4K and 8K). HEVC must offer sufficient advantages to offset the additional cost of coding systems and patent licensing fees to succeed. The biggest challenge is the sheer number of AVC decoders embedded in the deployed viewing platforms.

The open-source X265 codec project provides a similar patent free design to the earlier X264 project. If you use these open-source codecs, you should verify that they produce 100% compliant bitstreams to avoid problems with the viewing experience.

Media Type Identifiers

The HEVC codec is carried in the same container types as AVC. Therefore, the media type descriptions are broadly similar. The codec type suffix, however, is described differently. That allows the receiving player to distinguish between AVC and HEVC content:

Note that the support for these codec parameters in media type identifiers is not yet fully stabilized in all browsers. Some browsers may require additional descriptions of profiles and levels. Ensure your QA processes test on all the target browsers you intend to support.

Relevant Standards

The HEVC standards are covered by MPEG-H which is published as ISO 23008. Only five parts of that standard are directly relevant to HEVC. The rest describe MPEG Media Transport (MMT) and 3D Audio. ISO 23002 part 7 is also relevant.

Further Evolution

Caveats regarding profile and level compatibility apply to HEVC as they did to AVC. The encoder and client-player must both support the same configuration.

We may still need yet another cycle of codec innovation. HEVC is good for 8K and is scoped to support 16K at the upper end of its designed performance range. Research work and prototypes are already in hand for 32K video systems.

We thought that 8K might be too extreme for consumers to have at home, but compatible displays are now affordable. Will 16 and 32K be feasible? If they are, we need even better compression technologies. There are no displays of that size yet, but there are several prototype camera designs in the pipeline.

These Appendix articles contain additional information you may find useful: