Standards Vs Open Source & Adopted Protocols

The broadcast industry is built on standards. For decades, standards have literally held everything together; they have encouraged interoperable behavior, told us what we can and can’t do, and kept us all on the same page. But today? Maybe not so much.

More recently we’ve seen that standards are not the only way. AMWA’s suite of NMOS specifications developed outside of the standards bodies as a collaborative industry response to the need for control and management in IP-based media productions. Suddenly IS-05 and IS-06 were in every RFP.

Similarly, the adoption of microservices and cloud compute infrastructures in the IT industries are rarely based on standards. More likely to be developed in collaborative open source environments and making use of already established protocols, these systems have built on existing technologies with the goal to fail fast and iterate, treating failure as a learning opportunity and getting to market faster. So as broadcasting adopts many of the same software-based technologies, the question is whether the media industry should try to form standards as part of its own adoption pathway.

What Is A Standard Anyway?

There is an argument to say that standards have always been a technology implementation that slowly becomes ratified. Appear CTO Andy Rayner thinks this is a good place to start.  

“SDI was first developed by Sony before it was adopted by the world and became a standard. Similarly, ST 2110 started its gestation within the Video Services Forum as TR03 before it became standardized as ST 2110, and it took around four years to get to ST 2110. Standards evolution in a traditional world takes a long time,” he says.

“The way that AMWA developed NMOS was seen as a more agile way to work, and the way MXL is being developed using the Linux Foundation is to embrace and leverage the way IT is now developing technologies”.

Riedel’s Chief Executive Officer (Product Division) Jan Eveleens is somewhat more pragmatic about it.

“These protocols do not need to be standards per se; for me they can also be de facto standards,” he says. “I don’t think anybody really cares about whether it has an official stamp from a standardization body or not, just as long as it works and it is available to everybody. Working in a hardware environment you need to have a specification where everyone agrees a way of working so that when you connect, it all works together.

“In software, I think that is more of a blurred line because once you use a library or an open source software that everybody uses, that agreement is actually inside the software, so the software becomes the way it is defined. Do we then still need a stamp on top of that? It could be helpful.

“But I’m not religious about it.”

As well as being co-chair of both the Technical Steering Committee of the MXL open-source project and the JT-DMF working group, Director, Global Collaborations / Innovation Hub at CBC/ Radio-Canada, Felix Poulin is also the only broadcaster on our panel.  And he is fully on board.

“In software, since we can fail fast and prototype quickly, achieving interoperability can be much faster if you just play with it and make it work,” says Poulin. “Maybe then you document it, and that documentation looks like a standard, but that implementation-first principle brings real benefits. You can try it early, you can learn from it, discover what works and what doesn’t, and improve things even before version one is finalized.

“At some point you stabilize and converge, and that source code becomes the reference implementation against which everything else is tested for interoperability. The goal is exactly the same as a traditional standard, but the process is somewhat reversed: implementation first, and then, once it’s stable, you can call it a form of standard.”

Like software-based workflows themselves, this way of working is definitely a more agile way to create working practices than what we’re used to. In 2020, SMPTE published a five step process outlining the “creation of Standards (STs), Recommended Practices (RPs) and Engineering Guidelines (EGs)”. This process involves starting a project; securing project approval; writing a document; gaining committee support; and finally obtaining verification.

It is a time consuming and lengthy process, and while there is a convergence from the entire panel about moving away from such lengthy standards processes for the development of software-based workflows, Matrox Video Product Manager Daniel Robinson thinks standards still have a place for formats, if not for architecture.

“I think there’s probably a place for both standards and specifications as well as open-source initiatives like MXL, and it doesn’t necessarily have to be one or the other,” he says. “Standards have clearly been important to the industry; without them it would be a complete free-for-all.  I think there’s still a very strong place for standards in defining formats, but for large, conceptual, and architectural initiatives, I think they move too slowly and I don’t think you would ever get enough consensus on what it should be.”

Speed & The Technology Landslide

Standards definitely have their place, but Lawo CTO Phil Myers isn’t sure whether they are always appropriate either.

“I think broadcast standards have served us very well in the video and audio domain. The work that SMPTE and others have done has been remarkable, and it’s what has enabled the industry to do what it does today. But we also have to recognize when technology shifts and changes.

“This might be a slightly controversial statement, but would SMPTE be able to standardize data center technology? Probably not. Will we still need SDI and different transport mechanisms? Probably yes. It doesn’t have to be one or the other.

“The downside is that standardization processes can take a long time. If you look at ST 2110, it took many years to get all of them in place as a working system, and then you also needed NMOS and other specifications for the control layer. In today’s environment, where customers can’t forecast two years ahead and the technology landslide is moving so quickly, we simply can’t wait five to seven years for a fully described standard.”

This rapid speed of change across the industry is a great motivator for all our panel, and the development of the EBU’s Dynamic Media Facilities (DMF) is already paying dividends: 

“To give a sense of the speed, we first discussed the idea with the industry at IBC 2024, on the back of which we brought everyone together and launched an open-source project,” says Poulin (CBC). “At IBC one year later we had multi-vendor demonstrations, just one year from idea to working implementations. That’s much faster than any standards process.”

Given broadcasting’s development history, from analog to HD to IP to where we are now, this speed certainly is unusual. Riedel’s Eveleens argues that it’s more efficient from an engineering perspective.

“I think the way the open source community works is very different from standardization,” says Eveleens. “In open source, I think engineers are always looking for the most efficient way to solve a problem, whereas in standardization there is often some company politics involved. I’m not saying that doesn’t occur in open source developments, but I think there is more of a focus on people working on the most efficient solution.

“Standards are not very flexible in terms of changes because they have to go through a whole process of modifying and ratification. With open source you have a similar thing, where people need to validate that a new software version doesn’t break anything, but it’s more a technical verification whereas in standards there are also legal and commercial factors at play.”

Meanwhile, open source might not just be better – it might even be necessary, especially given the existential consequences of moving too slowly.

“My worry is that even five or ten years from now, we’ll still be holding on to SDI for dear life,” says Myers (Lawo). “What if we can’t build it anymore? If the volumes of components go down, prices go up, and at some point those products may become unaffordable to manufacture. We need to be allowing our customers to migrate at their own pace, rather than arriving at a cliff edge one day and realizing they have no options.”

There is some additional caution to note however, which is that agility – although efficient – needs careful management. Because while open source development might be agile enough to keep pace with industry needs – and perhaps because of this – it needs robust governance, but the rewards are substantial.

“The challenge with code is that it’s a moving target, unlike a standard which is set in stone,” says Robinson (Matrox Video). “Open source relies on contributors to continue to evolve and support it, which means that resources can be pooled across the industry. With a standards-based approach, once the standard is published, everyone implements it individually and at some point everyone tries to make their implementations work together.

“Open source initiatives cut straight through that; you either support the implementation or you don’t, and if there’s a patch to fix a problem, everyone benefits from it.”

Lessons Learned

As we mentioned at the beginning, the broadcast industry has form, and perhaps there are lessons to be learned from the NMOS development. Whilst not quite open source code, this AMWA initiative was not following a due process as a standard and it is surely not a coincidence that AMWA is working with the EBU as part of the JT-DMF working group.

Neither a traditional standard or pure open source development, NMOS represents something of a middle ground and something which the MXL development may be able to build on.

“NMOS sits somewhere in between.  It’s still a specification, but within its governance, no NMOS specification has ever been published without first being implemented,” says Poulin (CBC).

“There’s the theoretical work of writing the spec, but then there are workshops where multiple vendors come together and make their own implementation to test interoperability. They’ll find small issues, tweak things, and improve the final spec before it’s published.

“So there’s still the principle that we don’t publish a theoretical document that hasn’t been implemented. This is quite possible in software, but much more difficult in hardware.”

Of course, there were other reasons why NMOS was adopted by the broadcast industry so readily, and this too could have a positive spin on the development of software-based workflows, says Robinson (Matrox Video): “The AMWA model is a good example and it’s been widely adopted, although I’d argue that if Sony’s open-source NMOS CPP library didn’t exist I’m not sure NMOS would have had the same adoption – it played a significant role in driving it and because it was available as a reference implementation, people really used it.

“For something like DMF, which is a huge cross-cutting initiative, a standards body simply couldn’t take it on. ST 2110 has multiple standards and all it covers is moving video, audio, data and timing. It says nothing about control, deployment, orchestration, security or authentication, all of which are in scope for DMF. I just can’t imagine a scenario where a standards body could take on that and build us a standard for DMF, it’s just not practical.”

Robinson’s point is that NMOS gained greater momentum partly through the availability of a working reference implementation. The fact that developers had this common library to refer to is something that the broadcast industry has seen many times before, including during its last big technological shift to IP. But whatever the development cycle, what we’ve done in the past often sets a precedent as to what we will do in the future, and achieving a critical mass for adoption is one of those things.

“End user drive and adoption is vital to make this happen,” says Rayner (Appear). “If we cast our minds back to the beginning of ST 2110, there were a number of proprietary, IP-based interfaces for production which were perfectly valid within the space that they’d created. We formed AIMS as a way of helping end users and vendors that were already on board to create a critical mass of adoption for a single way forward, and ST 2110 was born out of that.

“The reason NMOS has been so successful is because end users have been mandating it on RFPs. It means that products need to have NMOS control to be valid, and as we move forward we need to see end users mandating a JT-DMF way of working, and as part of that, adopting MXL as the compute handoff.

“I’m not sure MXL ever needs to become a standard. I think many people interchange the word standards with specifications and recommendations and for software-based infrastructures, moving from standards to specifications is more akin to living in the IT world we are in. There were questions about whether NMOS should become a SMPTE standard rather than an NMOS specification, and the Video Services Forum’s TR07 and TR08 are both specifications rather than standards. Within VSF we call them technical recommendations, but we still refer to them in language that mandates what should happen. And the industry is living quite effectively with all of these.”

Looking Forward

Ultimately, it’s not really about defining standards as much as it is about being able to adapt to developing conditions, and if the infrastructure itself can accommodate change then it’s worth doing, whether it’s a standard, a specification or a recommendation.

“I think we need to follow the same philosophy we have in agile development which is to fail fast and iterate, and use the right technology for the task in hand and make technology choices that are more widely adopted,” says Myers (Lawo). “If you look at containerization, for example, you can run it on a simple Docker environment on a single box, or on Kubernetes infrastructure. You don’t have to make that decision up front; the artefacts can exist in both systems. There are plenty of standards in that space, but that community is very focused on white papers, documentation, sharing, and open source, because everyone wants to move forward quickly.

“That’s a different approach to the broadcast industry, where we’ve historically done technology shifts in five-to-ten-year blocks. What if the infrastructure can accommodate change, and it’s only the software layer above that needs to adapt when requirements change? That’s a real mind shift, and I think as a vendor community we need to be more open and transparent in educating customers about the benefits of it.”