ISO Base Media File Format (ISOBMFF)

The reason that the Apple QuickTime File Format (QTFF) was adopted by ISO/MPEG is that QTFF was already very close to what ISO/MPEG were looking for as a candidate technology when they were designing the ISO Base Media File Format (ISOBMFF). It is published as ISO 14496-12 (MPEG-4) for video, and ISO 15444-12 (JPEG 2000) for images.

The ISOBMFF supports all the features and benefits that QTFF provided for timing, structure and metadata. This extensible format carries any kind of time-based media elements you might ever need.

The specification allows external ancillary files of any kind to be called in to use by reference. The primary timing, and framing information must be carried in the main ISO base media file.

Since ISOBMFF and QTFF are based on the same DNA, applications designed to manage either kind of file can share a lot of common code. This is good for application developers because less code = easier maintenance.

QuickTime File Format (QTFF)

Apple developed the QuickTime file format as an object-oriented storage container. The media is stored in many small chunks called atoms. The data type of those atoms is indicated with a four-character (FourCC) code. The format is extensible because new atom types can be added to the repertoire at any time. Older file readers simply ignore the atoms they don’t recognize. New atom descriptions are introduced via a registry process.

The atom payload can carry Unicode characters. This allows URLs and internationally localized text to use the full range of the character set.

QTFF supported very sophisticated VR techniques for creating panoramas and object movies by rapidly switching frames in an entirely non-linear fashion. Apple called them Navigable Movies. This has been inherited by the ISOBMFF and points towards some future Metaverse and VR applications which MPEG is developing standards for.

The Apple QuickTime File Format is described here:

FourCC Codes

The FourCC concept was designed by Apple in 1984 when the original Macintosh was released. These identifiers were widely used in the file system to identify resource data types. Apple describes them as OSTypes but the FourCC nomenclature is more popular. In the registry and some standards documents, they are also described as Code Points.

In 1985, Electronic Arts developed the IFF media file format for use on the Amiga personal computer. It used the same FourCC concept to identify the chunks of media data within the IFF file. The IFF technical documents credit Apple with the original innovation. IFF files are the foundation for many other formats (AIFF, RIFF, AVI and WAV). The Apple QuickTime files emerged in 1991 using the same idea but must surely have been in development for some time before that.

FourCC values are 32-bit integers composed of four separate 8-bit ASCII characters. The data is arranged in big-endian fashion to render the characters left to right. The big-endian format allows them to be assembled into a viable 32-bit value without needing any transformation.

Normally the codes are constructed with ASCII printable characters. Spaces are also allowed. This can cause problems for the unwary because the lack of a space character will not match the atoms correctly. In rare cases, non-printing control characters are used, which makes them even harder for humans to read without special formatting software. By convention, most of the codes are spelled with lower-case characters. Upper case variants define a completely different atom type to their lower case counterpart.

Serialization Into Files

Essence data needs to be serialized so it can be written to a file. The receiving application reverses this process (de-serializing) when it opens the file and reconstructs the original format.

The earliest media file formats only contained raw data. The internal structure was very simple with a header preceding the essence data.

The header contains a few metadata items to describe the format and scope of the raw data.

The next evolution splits the stored media into chunks. An optional trailer might be added to carry ancillary metadata.

After editing, the entire content is written to a new file. This takes a long time, leading to an unsatisfactory user experience.

An index to list the active chunks allows them to be rearranged, omitted or appended. Access to the index is easy when it is placed at the end of the file. The file manager provides the file length for your application code. If the last item in the file is a negative offset (N) back to the start of the index, then a simple file pointer seek function can locate it.

This is effective and quick but after many edits there might be some redundant chunks left in the file. These waste space. On poorly implemented solutions, the old redundant indexes are left intact as well but are ignored. Flatten the file to its optimal size by rewriting it without the garbage. Software engineers describe this as ‘Garbage-Collection’.

Run the garbage collecting file flattening process offline in the background as a scheduled workflow task running at a low priority so as not to consume an annoying amount of CPU resource. Read about the nice command which can manage the process priority from the command line shell.

A properly designed file editor will add the new chunks and replace the old index with the new one. That will only leave the redundant blocks in the main body to be flattened.

If the index or the chunks themselves are enhanced to identify a distinct data type, then metadata can be stored in a chunk as well. This facilitates mixed media and multi-channel storage in a single file.

Editing the file removes redundant chunks from the index leaving them embedded in the file and appends new ones to the end of the file. The index is updated when the file is saved.

ISO MPEG-4 Containers

The ISO Base Media File Format is similar to the Apple MOV file format developed for QuickTime. Apple originally designed this file structure to support multiple simultaneous media tracks of any type.

Additional features that support MPEG-4 coded media are described in MPEG-4 Part 14. Part 15 addresses the storage of AVC video content packaged for transmission on a network.

Combine MPEG-4 Parts 12, 14 and 15 and RFC 6381 to fully understand the MP4 container file structure. Then examine the MP4 registry to find descriptions of the internal chunk storage ATOMs.

Base Media File Formats

Base Media File Formats are foundations that provide a basic structure for storing arbitrary data of any kind. Other container formats are derived from them either by extending them or by using profiles.

Registering Atom & Box Type Codes

QTFF describes the media objects inside the container as Atoms and ISO calls them boxes. Each box has a size parameter, type code specifier and a payload. The entire contents of the file are managed in boxes with no other data allowed.

The Box and Atom type codes are managed by the MP4 Registration Authority. This is administered by Apple on behalf of ISO and the wider QuickTime community. The complete list of type codes is available for public access at their web site:

Debugging ISO Media Container Files

Before QuickTime was revised to use the AVFoundation library, Apple developers had a variety of tools for disassembling movie files for inspection. Those tools are long extinct now that QuickTime 7 is deprecated.

This is an example output from an ancient Apple QuickTime de-compiler tool which illustrates the internal atom structure. ISOBMFF files would look very similar.  (see above)

Clearly, atoms are nested at multiple levels contained within one another. This tree structure is reflected into the object graph cached in memory when the file is loaded by an application.

You will encounter similar tree structures in many scenarios in a software based production environment. They can be expressed in many different forms.

MP4 and ISOBMFF formatting problems can be traced with these more recent inspection tools:

MPEG Application Formats

More recently, MPEG has standardized specific ISOBMFF file formats for different target applications as the ISO 23000 MPEG Application Format.

The various parts of ISO 23000 collate media storage requirements under various categories. Collectively, these are described as MPEG-A.

File Name Extensions

These file name extensions are based on standards for the ISOBMFF containers:

Applying ISOBMFF

The MPEG containers probably have the best support for end-users across all the client-player applications.

MPEG containers can carry a variety of different coded media elements but are designed around the MPEG codecs. These have been around for some time and are being overtaken by better performing and newer codecs. Those new codecs are often stored in a Matroška file container.

MPEG containers are not likely to disappear. They may become even more popular when the patents expire and there are no more license fees incurred. However, by then the Matroška format may be dominant.

These Appendix articles contain additional information you may find useful: