Grace Hopper was one of the early pioneers of computing. She is quoted as saying "The nice thing about standards is that there are so many of them to choose from". The quote is also attributed to Professor Andrew Tanenbaum. Nevertheless, it is an accurate observation. There are a lot of standards!

We encounter standards everywhere, every day, from the power sockets on the wall to the way our mobile devices interact with one another. We should expect to see new standards describing how to interface with Artificial Intelligence (AI) and other smart systems in due course.

These articles focus on important standards for broadcast production and delivery in modern IP driven enterprises.

What Are Standards?

Standards describe the best practices for making compatible products from different manufacturers. For example, the dimensions of a connector and its pin connections, a data structure or an Applications Programming Interface (API). A standard could describe an entirely ephemeral entity such as a video stream or a file containing audio material.

Product manufacturing can be scaled up and distributed across many suppliers with standardized designs. This also facilitates spare parts provision and maintenance. The alternative would require every piece of equipment or software to be a bespoke and unique design. The costs of constantly re-engineering and re-inventing the same wheels every time would be prohibitive.

A Little Bit Of History

Standards have been around for a very long time. Much longer than you might think.  Here are some early examples:

Who Creates The Standards?

International standards such as those produced by ISO are the result of many industry experts collaborating through a process of consensus and approval. Each country supports its own committees covering specific areas of expertise feeding into ISO via a national standards organization. The standards are detailed and high quality but the preparation might take a long time.

Special interest organizations that operate internationally or regionally such as AES, SMPTE and EBU also collaborate to bring a high level of expertise to the process.

Smaller special interest groups (AIMS Media Alliance, AMWA and DTG) promote good practice with technical reports and recommendations.

Expertise is drawn from participating companies whose work informs the standards. A standard can be adopted informally by those companies to avoid any delay while it is being ratified.

Independent research laboratories and universities run projects to examine the viability of new technical approaches. The Fraunhofer Institute is a good example where the work they did on AAC audio compression facilitated the standards process.

Standards may evolve in an entirely different way by a large-scale adoption of proprietary technology. These de-facto standards are initially created by companies that make products to exploit them. Here are two examples:

• The Rich Text Format (RTF) document standard could only be used with Microsoft word processors at first. The format was easy to reverse engineer and is now widely supported by other applications. Microsoft have since published the specification for general use but it remains their intellectual property.
• Adobe PostScript code can be wrapped in a container to become a Portable Document Format (PDF) file. This has now been adopted by ISO as an international standard (ISO 32000) which is available free of charge.

A Taxonomy Of Standards

Organizing the standards into a taxonomy helps to see how they interact. Some large standards are divided into multiple parts, each of which might fall under different categories. This diagram illustrates our major areas of interest.

Video and audio standards are developed independently and integrated when a container describes how to embed and synchronize multiple tracks for streaming or broadcast transport.

Why Doesn't It Work Straight Out Of The Box?

Large and complex standards provide many alternatives for configuration with a lot of parameters each having a wide range of possible values.

Two companies might engage in building products based on a standard and intend that their products will integrate seamlessly with each other when they are deployed. If the standard allows any leeway for interpretation, these completely compliant products might not interoperate as expected if they use mutually exclusive implementations of the standard. Their customers cannot make them work together when they conduct the conformance testing.

This problem is caused by incomplete standards implementation on both sides. It is driven by pragmatism and budgetary controls on what is deemed to be a reasonable level of functionality.

Profiles & Levels

Complex technologies such as the AVC codecs introduced the concept of profiles and levels to simplify the interpretation of the standard.

An encoder does not always need to use all of the tools defined in the standard. Profiles describe a sub-set of the available coding tools that the encoder could use to compress the video. These profiles are designed around particular use cases to eliminate unnecessary features in the encoder implementation. The profile also provides hints to the decoder in the client player.

Levels define the picture size, frame-rate and bandwidth limits for the encoded bitstream to reduce complexity in the decoder.

Check that the profile and level support in your encoder settings is compatible with the decoder in your receiving device. Using higher profiles and levels may result in a better compression ratio but may also limit the range of devices that your content can be viewed on.

Standards Wars

Standards are sometimes weaponized for commercial gain. Perhaps by omitting support for a standard in a new product or by requiring customers to buy licenses to use certain features.

Organizations may build products that adhere very strictly to an accepted standard. Then they implement additional proprietary features and encourage developers to exploit them in their projects.

This serves several purposes. Firstly, they can rightly claim the moral high ground by declaring that they are wholly standards compliant - which is true. Once the proprietary extensions are deployed by developers, they are locked in to using that product. When a single maintenance release introduces a proprietary technology, removing it can take between 3 and 5 subsequent updates to replace it with your own solution.

The potential for using standards to lock customers into proprietary technologies still exists and needs to be anticipated to avoid it.

Glue-Ware

As an end-user faced with two incompatible products, you might still need to implement a work around and introduce bridging mechanisms.

Hardware interfacing devices to transform the signal from one format to another are widely used. For example, converting computer video output to a useful format so it can be integrated into the rest of the studio infrastructure.

IP based studios may still require a similar kind of virtual 'glue' to convert data streams or AV files from one format to another. The concept is similar but the implementation is quite different using apps and processes instead of hardware.

Classical hardware glue solutions tend to be instantaneous while software conversion may introduce some latency and delay the output somewhat. Hybrid solutions using the best of both approaches are also possible.

Overlapping Standards

Some regulatory bodies collaborate on specific standards topics. For example, CENELEC, ISO, BSI, EBU, ITU-T, JVT, MPEG and JPEG all share specifications for moving and still images. The output from these organizations leads to an ISO standard being published. Sometimes, a new standard is published and other organizations release essentially the same specification under a different name or number. The namespace is already crowded with acronyms and this can be confusing.

Risk Factors With Patents & Licensing

Check carefully whether a standard depends on patents before you commit to it. The authors may not have known which patents apply.

A useful standard might have patent licensing terms that make it very expensive to deploy. An open standards group might take the opportunity to create a compatible and patent-free alternative. This then becomes a de-facto standard that displaces the original. It is a major cost saving.

The standards development process tries to avoid this by not including patented technologies without an undertaking from the patent owner to license it for minimal costs to the implementer.

Submarine patents surface sometime after a standard has been widely adopted. This leads to a lot of unpleasantness. Be well prepared before committing to the use of a standard.

A potentially expensive situation arises where the patent holder attempts to monetize the usage (instead of the purchase) of equipment or software that includes their patent. This is a major problem for broadcasters who cannot be expected to pay a fee per viewer or per programme.

Careful architectural design can mitigate this risk factor. Web browsers such as Firefox rely on the operating system to provide support for playing AVC video. This is necessary because Firefox is an open source application with limited funding and the licensing fees are prohibitive. Delegating this to the OS means that only one shared license is necessary for all apps on that platform.

Conclusion

Standards are widely available but they are sometimes expensive to buy. They are often written in a deeply technical fashion which is hard to decipher. In the following articles we will drill down and examine the standards we depend on in broadcasting to unravel this complexity.

About The Standards Article Series

There are 26 parts planned for this series – due to publish throughout 2024. Here is a list of forthcoming articles with simple descriptions of their content:

Part 1 : An Introduction To Standards
The introduction and anchor article for the series covering the following topics: What are standards? Who creates them? Why do they exist? History, Benefits, Advantages, Disadvantages, Proprietary Extensions, and Profiles.

Part 2 : Standards For Broadcasting & Deployment
A section level article giving an overview of the standards relating to production and transmission or playout. This article prepares the ground for subsequent more detailed articles which will explore the following subject areas: ST 2110, higher bit rate codecs and profiles that are non-lossy, standards relating to exchanging material between different NLEs and Vƒx, and the AMWA workflow standards.

Part 3 : Standards For Video Coding
A section level article giving an overview of codec specifications. ISO and non-ISO standards will be covered alongside SMPTE 2110 elements to contextualise all the different video coding standard alternatives. Codec efficiency will be compared - AVC is roughly twice as efficient as MPEG-2 and HEVC is better still.

Part 4 : Standards For Media Container Files
A section level article giving an overview of files for transporting media essence. Certain kinds of files and video/audio formats go together but some also prohibit the use of other codecs. AVI and QuickTime are multimedia platforms as well as file containers and video codecs. Most of these are described by ISO standards but there are some de-facto proprietary formats that are important.

Part 5 : Standards For Audio Coding
A section level article giving an overview of MPEG & AES standards. The AES standards are referred to by ST 2110 which mainly defines how to use them. We will briefly cover AES 10 and 31 here.

Part 6 : MPEG-4 Overview  
Here we describe all the different parts of this standard with a little of its history and coverage of the parts that are interesting but which have not gained any traction (such as BIFS). This article prepares the ground for subsequent more detailed articles where the important parts will be covered in greater depth, such as MPEG-4 Part 10 - AVC.

Part 7 : ST 2110 - A Review Of The Current Standard
This is an outline of the different parts of this standard. How they use prior work to extend the SDI and related other SMPTE standards. As the series builds this article links to more detailed articles.

Part 8 : Standards For Producing Video Content  COMING SOON
These are supporting standards that help with workflow construction and management. This is scaffolding for building out your workflows; AAF, MXF, NMOS etc.

Part 9 : Broadcasting Video & Audio  COMING SOON
Where earlier discussions were about retaining full uncompressed quality for production and archival reasons, these standards allow for compression of the assets so they can be delivered through limited bandwidth conduits. This is where streaming services apply compression to video and audio assets.

Part 10 : Streams, Multiplexing & Embedding  COMING SOON
Standards that describe video and audio compression only deal with single elementary streams. Audio needs to be embedded with video to make a viewable programme. Multiple channels need to be combined to make a transport stream for delivery over DSat, DCable and DTT, DVB & ATSC 1 & 3. Here there are opportunities to apply statistical multiplexing to use the available bandwidth efficiently when multiple channels are multiplexed together.

Part 11 : Streaming Video & Audio Over IP  COMING DURING 2024
Delivering multiple streams on internet services viewed in web browsers is done quite differently to broadcast. This is covered by MPEG-4, Parts 8, 29, 31 and 33. MPEG-DASH. ST 2110 parts 20, 21 and 22. AES 3 & 67 are part of this as are HLS, SRT, RIST, RTP and RTSP protocols.

Part 12 : ST2110 Part 10 - System Level Timing & Control  COMING DURING 2024
This standard is all about timing and synchronisation and holds the entire ST 2110 environment together.

Part 13 : MPEG4-Part 10 - AVC  COMING DURING 2024
This codec supersedes MPEG-4 Part 2 and is commonly known as AVC or H.254.

Part 14 : MPEG-H Part 2 - HEVC  COMING DURING 2024
Although HEVC is not yet in as wide use as AVC, it could become dominant as the viewing platforms deploy larger screen sizes (4K and 8K).

Part 15 : ST2110 - Video  COMING DURING 2024
ST 2110-20 is a production video coding standard where quality is more important than bandwidth efficiency. Generally speaking, the bandwidth within a studio environment is quite high because for production, we need to move uncompressed video around the network. We also cover ST 2110-22:2019 and ST 2110-22:2022 which are different editions of the same standard with slightly different coding strategies. This is a potential gotcha when integrating systems. Specifying ST 2110-22 alone is not enough without indicating which edition is being deployed.

Part 16 : MP3 Audio  COMING DURING 2024
This is still in wide use as an audio codec for delivering content to end users. It also carries metadata for identifying the content. The full name for this is MPEG-2 Layer 3 audio. We will observe that this codec is sub-optimal for primary recording or archival purposes and is strictly a delivery codec for deploying audio to end users.

Part 17 : AAC Audio  COMING DURING 2024
MPEG-2 part 7 is known as AAC. There are several enhancement derivatives of this and it is widely used in the same sort of situations as MP3 was. It can be used at a higher bit-rate for better quality but it is still a lossy codec. MPEG-2 part 7 AAC audio is often paired with MPEG-4 part 14 AVC video and stored in MPEG-4 container files. Another oddity is that the audio specification is part of MPEG-2 while the video is covered by MPEG-4.

Part 18 : High Efficiency Audio Coding (HE-AAC)  COMING DURING 2024
The AAC audio coding has been improved and is known as HE-AAC. This delivers superior audio quality. This is a specific profile for using MPEG-4 part 3.

Part 19 : ST 2110-10 & AES 67 - PCM Audio  COMING DURING 2024
PCM audio is described by AES67 and deployed as ST2110 part 30. They are closely related and will be covered in the same article.

Part 20 : ST 2110-31 & AES 3  COMING DURING 2024
This is a standard for exchanging audio data between systems. It includes S/PDIF for optical transmission and is also available as an ISO standard. Because these standards are so closely related, we will cover them together.

Part 21 : The MPEG Containers  COMING DURING 2024
These are largely described by the MPEG-4 standard. Parts 12, 14 and 15 describe the container structure. It is based very much on the Apple QuickTime movie container format and can potentially carry anything that you would have used with QuickTime. A selection of other proprietary non-MPEG containers formats which will also be covered.

Part 22 : AIFF files  COMING DURING 2024
AIFF files contain raw audio samples that can be edited with audio workstation tools. There are a range of sample rates, track numbers and bit-depths for this data. It is an old format but it is actually a rather good solution for uncompressed audio, especially for archiving purposes.

Part 23 : MIME Types  COMING DURING 2024
This standard is important because it defines how a receiving client application will treat the incoming content whether it is delivered in a container file or as a stream. Your workflow will likely use this information to control how assets are processed. It is an important piece of metadata. The standards are managed by the IETF as RFC documents. This was originally defined for mail attachments in RFC 2045 but it is also used in Web Server response headers when delivering downloads. Since that technology is also used within a production environment based on IP, it becomes important in that context too.

Part 24 : Timed Text & Subtitles  COMING DURING 2024
MPEG-4 parts 17 and 30 describe synchronised text streams. Similar coverage is described in SMPTE ST 2110 parts 40 and 43 all running on an ST 2110-part 10 foundation for timing and control. This is reflected into web-based media players as VTT tracks embedded in < track > tags that are contained within < video > and < audio > tags in HTML-5 web pages. A timed text item, triggers a JavaScript event which can be trapped by an event handler to call some active code to action. This can alter the appearance and content of the user interface or simply present a sub-title text. Visual material can be triggered for presentation within strictly audio presentations which blurs the distinction between ‘TV’ and ‘Radio’ like user experiences. There isn’t really any difference from a technical point of view other than the amount of bandwidth required for delivery.

Part 25 : Client-side Players  COMING DURING 2024
Building client-side web browser-based players is not just for adding A/V to web pages. It is also used to create entire user experiences in TV receivers and set top boxes. There are a variety of alternative approaches which are becoming de-facto standards. Some are built as apps using Java or Android environments. Others are proprietary such as Apple i-Devices and TV platforms. Smart TV sets support a variety of HTML-5 or TVML authoring environments. Delivery of these user experiences tends to be via a proprietary conduit either over the air or via an IP connection. Wired vs. WiFi also becomes an issue for performance, QoS and reliability.

Part 26 : Other Standards  COMING DURING 2024
An overview of some of the additional standards covering content management, interactive media and internet protocols which might be of value to broadcasters.

These Appendix articles contain additional information you may find useful: