The media industry is evolving ever more rapidly. Very advanced and complex media formats are on the horizon. Architectural design of the infrastructure we depend on is also changing at a rapid pace. This becomes less frightening if we break things down into individual technologies and standards. It facilitates a better understanding of the system as a whole.

Grow your knowledge-base with documentation on relevant topics as a learning process. Different standards organizations allow access to their document repositories in different ways. Discover which ones are freely available vs. those that are fee based to equip your development projects with the resources they need.

There are a lot of standards bodies, ad-hoc groups and proprietary organizations that provide technical documents. The open-source community is also developing important solutions. Some organizations operate globally while others work within a confined territory.

Standards Development Organizations

Prominent standards organizations are well known and often collaborate on joint endeavors. There is not much overlap between them because they focus on different areas of competency. Some input comes from external groups of specialists in a particular field.

Standards are often harmonized between regional or international organizations with substantially the same specification being published in multiple forms. This promotes a regionally constrained standard to be internationally important.

These are short profiles of the organizations that create standards for us:

JPEG - Joint Photographic Experts Group

Their main focus is on compressing still-images. Recent work on storing multiple-images in the same file extends the capabilities. This facilitates stereoscopic views and more complex visual formats such as Light Field Cameras. New work on using Artificial Intelligence to steer the compression is also on the way. Typically, any still-image compression format can be used to create video by assembling a series of frames. Motion-JPEG is an alternative to MPEG coding for example. The specifications are reviewed by national standards bodies and then published as ISO standards.

MPEG - Motion Picture Experts Group

Audio and Video coding is developed by the MPEG group. After High-Efficiency AAC, the main focus has been video coding. Stereoscopic video requires two simultaneous synchronized sequences to be coded. Some redundancy is evident because both images will be similar. New work on immersive video will be an important aspect of Metaverse support. Point-Clouds and vectors encode immersive environments for use in VR displays. Coding efficiency improvements continue with hybrid coders that choose optimum techniques on a segment-by-segment basis. AI technologies introduce Neural Network coding which separately requires a means of compressing the Neural Networks themselves. This is very advanced technology. These specifications are also published as ISO standards.

ISO - International Standards Organization

The material contained in ISO standards is compiled by expert members of the working groups. Many ISO standards are subject to patent licensing for proprietary technology. A few ISO standards are royalty-free and not subject to these patents. Patents can encumber a standard which reduces its attractiveness in the marketplace. AVC (H.264) and AAC audio have become very successful even with patent encumbrance but HEVC is still working hard to gain traction whilst facing competition from open-standards codecs. Patents eventually expire which then leads to open-source implementations.

ISO standards are quite expensive and you will need to pay for the standards document and the patent license fees separately if you are implementing hardware or software that incorporates them. These purchase costs make open-standards that are free to use more attractive. Occasionally a standard is important enough for ISO to release it as a Publicly Accessible Standard (PAS) which can be downloaded free of charge. The fundamentals of some standards have also been distilled and published in other forms but these may be incomplete, out-of-date and are certainly not authoritative. The only reliable source of these standards is ISO itself or its distribution agents.

Fraunhofer Institute

Fraunhofer is a leading independent institute for innovation research in Europe. This is where the MP3 audio codec originated and where AAC and HE-AAC prototypes were developed. If you visit broadcasting industry trade shows such as the IBC, NAB or NAMM, Fraunhofer is often in attendance and demonstrating their latest research. It is always worthwhile visiting them at the show to see what is emerging as new technology. They contribute a great deal of very high-quality research expertise to the standards development process.

IEC - International Electrotechnical Commission

IEC publishes international standards for electrical and electronic technologies in use worldwide. This includes energy distribution as well as some telecommunications work. Many IEC standards are developed collaboratively with and published by ISO. Those that are published directly by the IEC are numbered in the range 60000–79999. Some very old standards were renumbered to conform to this regime. IEC works closely with CENELC on European standards. IEC standards are not free and the revenues go towards funding the development of new standards.

ETSI - European Telecommunications Standards Institute

ETSI develops technical standards for Information and Communications Technology (ICT) in Europe. They also publish standards on behalf of other organizations such as DVB. ETSI standards are especially relevant to mobile devices and will be a useful source of 5G related knowledge. PDF versions of the standards can be downloaded for free.

DVB - Digital Video Broadcasting

This is a collection of freely available standards that describe the physical formats for transmitting digital media. This is the foundation of digital TV broadcasting in Europe, Africa and Australia. The DVB-I standard is being developed to describe interactive services. Earlier versions of the DVB standards were published as a free CD-ROM disk. Now, the DVB standards are distributed by ETSI. Electronic versions of the standards are easy to find online and PDF versions are free to download. Printed copies are available for a modest cost.

DTG - Digital TV Group

DTG publishes the D-Book which is the mandatory standard describing the Freeview Digital Terrestrial TV service in the UK. A product for Freeview cannot be shipped without it being validated against the D-Book specification. Membership is quite expensive and the D-Book is not available without becoming a member.

EBU - European Broadcasting Union

A collection of over 100 public service media organizations from more than 50 countries. EBU has members worldwide and not just in Europe. This explains why countries such as Israel and Australia are able to participate in the Eurovision Song Content. The EBU publishes many informative and useful technical documents. These are free to download at the EBU website.

ITU - International Telecommunications Union

Formed as part of the United Nations. The ITU focuses on telecoms topics worldwide under separate groupings:

ITU-R - Radio communications (spectrum allocation etc).
ITU-T - Standards development for telecoms and multimedia.
ITU-D - Facilitates development of ICT technologies on an affordable basis.

PDF versions of many (but not all) ITU documents are available to download free-of-charge.

ATSC- Advanced Television Standards Committee

This is the counterpart to DVB that serves the North American continent. ATSC version 3 is the newest and most important emerging standard. Technical documents are free to download as PDF files.

SMPTE - Society of Motion Picture and Television Engineers

The ST standards are concerned with the application of the various audio and video coding standards in a working studio and broadcast context. ST 2110 has become prominent. Become familiar with the other ST standards from SMPTE. They provide foundation technologies that are important. SMPTE standards tend to be easier to understand than ISO standards. These standards have largely evolved out of the classic analogue TV systems. Additional supplementary documentation from SMPTE helps when you are studying the ST documents. Recent innovations in the membership structure allow you to pay an annual subscription fee to gain access to the entire library of standards for less than $100.

AES - Audio Engineering Society

The AES has been publishing technical reports and guidelines for a long time. There are many quite old standards relating to connectivity that are still very relevant with digital systems. They publish standards documents with additional supporting guidelines that describe how to apply the standards. The AES has innovated the same membership approach that gives access to all the standards for less than $100 per year. This makes the standards available to a much wider audience which can only be a good thing.

IETF - Internet Engineering Task Force

The RFC documents are fundamental to making the best use of Internet Protocol (IP) networks and services. They cover transmission and format descriptions in sufficient detail to facilitate building software to interact with the Internet. RFC documents have always been free to download in a variety of formats.

VSF - Video Services Forum

VSF maintains some freely available technical recommendations on RIST, IPMX and Cloud Computing. Read these in conjunction with the relevant AMWA documents.

AMWA - Advanced Media Workflow Association

AMWA manages a variety of specifications that augment and supplement other TV production and broadcast standards. In particular the NMOS specifications augment SMPTE ST 2110 architectures. Their specifications are all free to download as PDF documents.

ECMA - European Computer Manufacturers Association

ECMA publishes useful standards for IP based media production and delivery. The ECMA 262 standard is the core foundation of the JavaScript language implemented in every web-browser. ECMA standards are free to download. Join as a subscribing member if you want to be involved in creating new standards. Membership dues are expensive but they are free for non-profit organizations.

W3C - World Wide Web Consortium

All matters pertaining to the standardization of web content implemented in web-browsers are under the supervision of the W3C. There are various parts of the organization. For example, WhatWG develops new prototypes for web technologies. Some web-browser manufacturers implement the new ideas from WhatWG in prototype versions of their software for download and testing as an early adopter.

Open-Standards

When it becomes expensive to buy standards and pay for licenses to use them in your own products, it naturally creates a degree of friction that that impedes their adoption. If the standard is particularly important, the costs become irrelevant and are factored into the price of the product or service. This is the case for MPEG-2 video compression for digital TV broadcast and DVD disks for example. The same is true of the H.264/AVC codec and the emerging HEVC codec. Audio is very often delivered using MP3 or AAC codecs. These are all sufficiently dominant that the return on investment into patent licensing fees is worthwhile.

There is a perception by the general public that all things on the Internet should be free to access. This evolved out of the original market share wars where companies would offer their products for free with the view that later they might be able to charge a premium once the end-users were locked-in. This is also evidenced in downloadable 'Freemium' apps that are free to install but have vital features deactivated unless an in-app-payment unlocks them.

There is a demand for technologies that are completely free of royalties. Open-technologies are sometimes offered by large organizations as an incentive to adopt other products. A free video codec that is only supported by one web browser confers a commercial advantage over the other web browsers that do not (yet) support it. Wider support is then often implemented quite soon to eliminate that advantage and we then have another de-facto standard emerging. Google VP7, 8 & 9 are examples.

A completely different approach is community supported open-source software. The X.264 codec and the Matroška container are examples. Because the source code is available, end-users can make their own modifications. Their improvements can be contributed back to the main repository so that everyone else can benefit. It is possible for open-source projects to enjoy superior technical quality because the distributed 'development team' is so much larger than a commercial organization would ever deploy on a project.

These are examples of open-standards that are competing against ISO MPEG & JPEG standards:

Other open-standards are not in competition with ISO MPEG/JPEG but are supplementary and useful augmentations to other standards such as AES & SMPTE. The specifications are free to download.

International Moving Image Society (IMIS)

The IMIS organization was previously known as the British Kinematograph Sound & Television Society (BKSTS). It was founded in 1931 by UK based members of the Society of Motion Picture Engineers (SMPE) which itself was established in 1916. These two organizations are possibly the oldest established collaborations in the media industry. BKSTS was renamed as IMIS in 2017.

The IMIS store has a wonderful collection of resources that distil knowledge into a compact format that is very easy to assimilate. One of their wall-charts presents a range of film formats visually for comparison and another illustrates television aspect ratios. There are many others in the series. These collections of knowledge are unique and not found anywhere else in such an accessible format. The prices are modest and are reduced for subscribing members. Membership fees are very affordable and provide access to a large archive of historical resources.

Conclusion

Multiple synchronized video and audio streams will become more important as Virtual and Augmented Reality (VR/AR) and the Metaverse is deployed.

Encoding the spatial arrangement of sound and vision assets is complex when layered on top of the multiple stream support. The player in a head mounted display needs to offset those spatial co-ordinates in an opposing direction as the user moves their head. A sound that was mapped to the left ear must travel across the soundscape to the right ear as the viewing orientation rotates. Everything else must also stay spatially coherent. This is very complex but the standards to support it are already being worked on. This involves six axes of movement to be correlated:

• Rotate in X, Y and Z affects the position of media assets.
• Translate in X, Y and Z affects the sound level and resolution of the video.

Separate to the immersive technologies, Artificial Intelligence will provide assistance with coding choices and configurations as well as all the other areas where it is useful for search and pattern matching.

I would highly recommend joining the AES, SMPTE and IMIS as a member. This allows access to a huge collection of useful reference material for very little cost. Concessions may be offered to non-profit or educational organizations.

Many other documents are available free of charge from W3C, DVB, AMWA, ATSC ECMA ETSI etc.

The most expensive resources are the ISO and IEC standards. Bear in mind that these often contain multiple parts. Select the parts you need and purchase them for your team when necessary. The others can go on your shopping list for later.

In the 12 months since I started writing here about standards a huge amount of new material has arrived. ISO, MPEG and JPEG have been particularly active in the areas of AI and Immersive media. There is still so much more to explore.

These Appendix articles contain additional information you may find useful: