Everything Is Changing

At the time of writing, we are celebrating the 100th anniversary of John Logie Baird successfully transmitting his mechanical Televisor service from London to Scotland over the ancient telephone lines on January 26th 1926. He would be astonished to see what we are doing now!

The evolution of the media industry is accelerating. Advanced and complex media formats are already on the horizon, while architectural design strategies for the infrastructure we depend on are also evolving at a rapid pace. Examining the individual technologies and standards one at a time is less daunting than trying to take it all in at once.

The Traditional Approach

Conventional broadcasters are migrating to an IP based workflow by incrementally replacing the real-time critical parts of their SDI infrastructure with a functionally similar software solution. Currently, the industry is building systems that use SMPTE ST 2110, NMOS and their related technologies to replace their conventional SDI based hardware.

This is an important step forward for some organizations but it is not the ultimate destination. Nor does it apply across the whole enterprise.

The Modern Approach

The popular streaming services have always used entirely file-based workflows and storage. The streaming server transmits the file contents directly to the receiving devices. Performance is improved by adding Content Delivery Networks and Edge Servers but it is conceptually very simple and eliminates all the real-time streaming from the production workflow processes.

These enterprises rarely deployed any SDI based architectures and have no need for ST 2110 to replace a classic analog TV architecture. They might still benefit from some of the features that NMOS offers with discovery and annotation of resources.

The foundation architecture resembles a traditional IT system with file servers and commercial off the shelf (COTS) workstations.

The Task In Hand

Introducing new technologies and the imperative to give up long accepted traditions is always painful, stressful and prone to conflict.

Data delivery via an IP network is subject to delays and interruptions. This is partly due to the encoding latency as the source material arrives. Routing and transport are dependent on routes that may change during the transmittal. Timely arrival is more likely if you are prepared to accept an occasional packet loss with UDP transport. Packets may arrive out of sequence or not at all, due to the way the data is routed through the network. Buffering the data can alleviate this at the cost of additional delay by using the TCP protocol instead of UDP.

The asynchronous nature of IP networking is optimal for file-based transfers but is challenging for real-time video delivery. Implementing ST 2110 reliably across an entire enterprise must work within these constraints.

A Hybrid Media Architecture

Perhaps there is a way to reorganize the architectural design to yield the benefits of both ideologies if you are migrating from a classical approach.

Create smaller self-contained pools of real-time sub-systems using ST 2110 where low latency performance is mandated, then bridge them together with IP technology enabled solutions. The bridging could be file-based or use services and events. This would split the problem into manageable pieces and also allows redundancy to be provided as a back-up. A hybrid approach might yield benefits to the media asset life cycle. The above diagram illustrates a small-scale hybrid solution.

Only the section inside the Real-Time shaded box is time-critical. The rest can exploit the benefits of an IP based network. Asset files are moved intact without worrying about how long it takes. Choose the standards-based codecs and file containers carefully throughout to avoid unnecessary transcoding.

A smart IP enabled receiving device can also request content on demand. That content can be delivered from the primary asset store. Optimize the downstream performance by using a Content Delivery Network (CDN) and edge serving buffers (are not shown on the diagram.)

Arguably, an entirely COTS and IT based approach might be cheaper in the long run. There is a great deal of work in hand to leverage ST 2110 to successfully make the transition away from classic SDI based installations.

How Do Standards Help?

Standards are vital for TV and radio broadcasting as they converge with streaming services and IP based production studio workflows.

Standards describe agreed solutions for integrating compatible products from different manufacturers. For example, a hardware specification for the dimensions of a connector and the individual pin connections. Software standards might describe a shared data structure or an Applications Programming Interface (API). A standard could describe an entirely ephemeral entity such as an encoded video stream or a file containing audio material. Another important standard would describe the metadata used to steer and manage all of these resources.

Standards facilitate the scaling up of manufacturing processes distributed across many suppliers to enable them to deliver predicable results. They also facilitate spare parts provision and maintenance. The alternative would require every piece of equipment or software to be a bespoke and unique design, and the costs of constantly re-engineering and re-inventing the same wheels every time would be prohibitive.

A Little Bit Of History

Admiral Grace Hopper was one of the early pioneers of computing and popularized the term ‘bug’ in the context of computing when her colleagues found a large moth in the computer system she was working on. The term was originally coined by Thomas Edison and used by engineers thereafter.

She is also quoted as saying “The nice thing about standards is that there are so many of them to choose from,” a quote also attributed to Professor Andrew Tanenbaum. Whoever said it, it is an accurate observation. There are a lot of standards and there are many more on the way!

Standards have been around for a very long time. Much longer than you might think. Here are some early examples:

Computing Platform History

There was a time when the Apple platforms were the preferred solutions for video and audio production and we eagerly anticipated their product launches at the NAB and IBC conferences. MacOS supported every available format at that time either as a native feature or by extension plugin components. The QuickTime platform was a foundation on which many powerful media tools could be built. In support of that, Apple was very active in developing tools, techniques data structures and standards for AV and media. That inventiveness still resonates in the industry as we continue to benefit from their efforts. Consequently, you will find that work being used to illustrate various concepts.

This is not to say other organizations did not also contribute valuable work too. Microsoft have provided important resources in the standards arena too and the Linux community continues to thrive as the optimum platform for open-source projects.

We stand on the shoulders of giants!

A Taxonomy Of Media Standards

Organizing the standards into a taxonomy helps to see how they interact. Some large standards are divided into multiple parts, each of which might fall under different categories. This diagram illustrates our major areas of interest for broadcasting applications.

Video and audio standards are developed independently and integrated when a container specification describes how to embed and synchronize multiple tracks for streaming or broadcast transport. Likewise any ancillary metadata and text tracks are also included but their content is delivered asynchronously with large gaps between each event.

Low latency delivery of multiple synchronized video and audio streams will become more important as Virtual and Augmented Reality (VR/AR) – and the Metaverse – is deployed.

Standards Organizations

Standards are created by many different organizations. There are a lot of standards bodies, ad-hoc groups, special interest communities and proprietary organizations that provide technical documents. The open-source community is also developing important solutions.

Standards bodies don’t compete with each other. In fact, they collaborate very closely and share their expertise through liaisons at the working group level. Each standards body has a unique pool of experienced engineers and scientists who specialize in a particular area.

The standard documents themselves are sometimes (but not always) a very hard read. They may use arcane descriptive techniques and unfamiliar notations. Read them through several times if necessary to grasp the fine points.

AMWA, IETF, SMPTE and AES documents are more straightforward and written in a practical and applied style.

Because there are a lot of different organizations, knowing what their area of expertise is focused on helps to locate the right document repository for the reference material you need for a project.

Refer to Appendix A for a description of the different standards bodies.

Standards Identification Numbers & Editions

Standards organizations use different serial number formatting rules to identify their documents. Sometimes the organization name is used as part of the reference.

Standards are referred to by number. This describes a single document when there is only one part. If the standard has been split into multiple parts, then this would describe the family of related documents. The individual parts might be identified with a numeric suffix. The ITU adds similar numeric suffixes to the document identifiers but they indicate a version number of the standard instead of a sub-part.

Occasionally, you will see the publication year following the standard number separated from it with a colon (:) character.

The AES, SMPTE and IETF maintain several separate collections of documents. If you are only aware of the main standards documents collection you may not discover important helpful advice. Refer to the supplementary document collections for supporting information. The key information you are searching for may be included in a related document with a different identifier.

Version Numbers & Vintage

Where standards are enumerated here in listings relevant to each topic, the vintage or publication date is included. This is somewhat of a moving target. Whenever this material is edited, the vintages are checked but new editions are arriving all the time. Always check to ensure you are using the most up to date version unless a specific vintage of the standard is required.

Normative & Informative References

In most standards documents, there are two kinds of reference lists to external documents:

• Normative - These are documents that are essential and necessary. You should read these in conjunction with the rest of the document. In a sense, they do form an active part of the specification.
• Informative - These are assistive. They may help to explain obscure topics but are not vital parts of the standard.

Beware of diving down the rabbit hole of normative specifications and encountering even more documents to read as you delve deeper into the essential items.

There are normative and informative references in some standards documents and delivery specifications that describe SMPTE standards using an out-of-date nomenclature such as ST 436M. In this case, the letter ‘M’ suffix is obsolete and these standards have been re-issued with a numeric suffix denoting one of several parts. ST 436M has been replaced by ST 436-1 and ST 436-2 which enhance the specification to describe the encapsulation of VANC data in MXF files. The older ST 436M file only addressed television applications.

Nomenclature And Terminology

The media landscape is so diverse and distributed that acronyms are being duplicated and mean different things depending on how they are used. Always be aware of the context when using terminology that your readers may be confused by.

Conversely, some technical ideas have several names. Video codecs for example described as MPEG-4, MPEG-4-Part 10, H.264 and AVC all describe the same thing.

When delivering content across the network, Media Type Identifiers describe what that content is and how to interpret it. The receiving device is then able to engage the correct handling mechanism. There are different variants of these media type identifiers and some have additional qualifying parameters that are not always included in descriptions of them. They have evolved out of the earlier Multi-purpose Internet Mail Extensions. You may find media type identifiers described as MIME types but they are largely the same thing. I have chosen to call them Media Type Identifiers as consistently as I can here unless the context demands that they are called MIME types.

Open Standards

There is a perception by the general public that all things on the Internet should be free to access. This evolved out of the original market share wars where companies would offer their products for free with the view that they might be able to charge a premium later on, once the end-users were locked in. This is also evidenced in downloadable ‘Freemium’ apps that are free install but have vital features deactivated unless an in-app-payment unlocks them.

Technology companies offer patented technologies to the standards developers in the hope that they will be included and a license fee can then provide a steady stream of royalty payments. Some standards organizations are resistant to this and the open-source community deplores patents and actively develops work-around solutions to avoid them.

There is an increasing appetite for technology standards that are completely free of royalties. Open technologies are sometimes offered by large organizations as an incentive to adopt other products.

A free video codec that is only supported by one popular web browser with a significant market share confers a commercial advantage over the other web browsers that do not support it. Wider support is then often implemented quite soon in the competing browsers to eliminate that advantage and we have another de-facto standard emerging. Google VP7, VP8 & VP9 are examples.

Community supported open-source software is a completely different approach. The X.264 video codec and the Matroška container are examples. Because the source code is available, end-users can make their own modifications. Their improvements can be contributed back to the main repository so that everyone else can benefit. It is possible for open-source projects to enjoy superior technical quality because the distributed ‘development team’ is so much larger than a commercial organization would ever deploy on a project.

Locking-in

Occasionally a company might add proprietary and premium features for an extra charge to an otherwise open standard. This is frowned on and actively discouraged through the licensing model which requires enhancements to be shared on the same terms as the original code.

Using proprietary extensions locks customers in which might make commercial sense but fosters resentment on the part of the customer.

Locking in typically takes one release cycle of your software. Disengaging can take between three and five releases to completely remove the dependency and deploy a replacement.

Cost Factors

Most documents are widely available and easy to find and obtain. Proprietary standards are sometimes expensive to buy or license. That cost creates a degree of friction that impedes the adoption of a standard. This is sometimes described as gaining traction or market penetration.

If the standard is particularly important, the costs become irrelevant and are factored into the cost of the consumer product or service. This is the case for MPEG-2 video compression for digital TV broadcast and DVD disks for example. The same is true of the H.264/AVC codec. Audio is very often delivered using MP3 or AAC codecs. These are all so dominant that the return on the investment into patent licensing fees is worthwhile.

Arguably these licensing costs are holding up the adoption of the emerging HEVC codec which is already competing with a huge installed base of already deployed AVC decoders. Those will all need to be upgraded or replaced to use HEVC if they are embedded in hardware.

Overlapping Standards

Some regulatory bodies collaborate on specific standards development. For example, CENELEC, ISO, BSI, EBU, ITU-T, JVT, MPEG and JPEG all share specifications for moving and still images. The output from these organizations eventually leads to an ISO standard being published. The contributors may also all individually republish the material as their own versions but the ISO document should be considered to be the canonical version.

It is also possible for a specification to be revised, published by researchers and adopted widely before the official canonical standard is updated to include the new material. There are no hard and fast rules and it is important to be clear about which document you are basing your project work on.

Sometimes, a new standard is published by other unconnected organizations who release essentially the same specification under a different name or number. This is not necessarily the same as the original contributors republishing their work. The standards identifier namespace is already crowded with acronyms and it can be confusing.

If the standards are released by several organizations, be careful that you acquire an up-to-date edition and that it is complete and consistent with an official version. Check the provenance and use the authoritative version if you can.

What Do You Do When There Are No Standards?

You could write your own! Your guidelines should be complete, clear and unambiguous. These documents are most likely a specification for a data model which manages metadata or content. If you are describing a part of the life cycle without existing formal standards, you could present this as a foundation for a standards body to build on.

Here are the things you need to ask:

• What is the purpose of your standard?
• Where it is targeting in the workflow or life cycle?
• Does it rely on other pre-existing standards?
• What are the inputs to it when it is a process?
• What are the outputs?
• Are there any controlling parameters or configurations?
• What are the units of measure and ranges of values?
• Are there profiles or preset configurations that work well together?

Here are the things you need to address:

• Define the default assumed values.
• Anticipate the future and design your standard to be extended.
• Allow for errors to be handled gracefully without invalidating the entire piece.
• Provide clear usage examples. This is a major failing in international standards.

Share your own standards with collaborators and also with your competitors. If the guidelines are adopted more widely, this will benefit all of you in the long run.

The Diverse Range Of Standards Is A Good Thing

Many useful standards are created internationally and used worldwide. Others are only used within a geographical region. They will likely have similar counterparts with regional specific features in other territories. Some are defined by industry collaborations and special interest groups. Where there is no standard, create one specifically for your own enterprise. Perhaps it can be shared with other organizations and in doing so might eventually become adopted as a widely used standard.

Occasionally you may read reviews about standards that criticize them for being incomplete. This is based on a fundamental misunderstanding. No single standard can possibly cover every conceivable requirement. The diversity of standards development organizations and processes brings many different areas of expertise into play. Standards have failed to gain traction in the past because the editors tried to develop a standard that was outside of their core competency. Celebrate the diversity of standards groups and endorse the way that they collaborate and interact to bring us the best practices in each discipline.

These Appendix articles contain additional information you may find useful: