Test, QC & Monitoring Global Viewpoint – December 2016

Do The Right QC, Early

The days of looking at bars and tone and calling it a quality check are long gone. The digital world makes it easier to playout and produce high quality. It can also be more difficult to identify errors. With this in mind, where in the playout and production chain should one perform quality-control checks?

In the electronics industry, it is commonly held that, as an item progresses through each stage of the design-delivery-support cycle, the cost of correcting an error increases 10 fold. If an error is corrected by improving design, then the cost in the final product may be 1x. If the error has to be fixed in manufacturing, the cost is 10x. If the error has to be fixed once the product is in the hands of the customer, the cost to correct is 100x. The same principle applies - perhaps with even greater multipliers - in the car industry. (VW diesel car anyone?)

Fix it early

I would argue that it also applies in the broadcast industry: fixing a problem as early as possible is the most cost-effective way to correct an error. This has worked extremely well for UK broadcasters, in their implementation of the DPP specification. Suppliers of media files for transmission are required to provide a test certificate, proving that the file has been QC'ed by an approved QC tool and that it has passed the requisite tests.

In theory, therefore, the broadcaster need not do any QC at all on the incoming file, other than to check for file corruption during receipt. In practice, of course, depending upon the value (reputational or otherwise) of the incoming file, the broadcaster is likely to do some amount of QC - perhaps using the same tool with which the file was tested by the file supplier, ensuring that the same pass result is achieved. But it is clear that 'earlier' QC has been highly beneficial to UK broadcasters.

Figure 1. In this example, a QC is made first at ingest and again after post production is completed (click image to enlarge).

Figure 1. In this example, a QC is made first at ingest and again after post production is completed (click image to enlarge).

This, then, raises two questions: how much QC should be done at each stage; and what should this QC comprise?

Figures 1 and 2 show the possible stages for doing QC during media file production/post, and transmission. (We are looking only at files here, ignoring real-time media delivery.)

QC during production

During production, some QC can be done on-set, but this would more typically include only things such as histogram measurements to ensure proper exposure , or perhaps white balance checks using an eye-diagram to check skin color, with a subsequent check by the DIT to ensure files are not corrupted. (This QC is not considered further here.)

During post-production, the real first point at which QC can reasonably be done is close to a final color grade and edit, checking technical aspects such as legal colors (if appropriate) and audio levels, plus the 'editorial' QC checks: is this suitable for pre-9pm transmission, for example?

Figure 2. In this example of a broadcast chain, QC can be completed at three points: at ingest, on files destined for real-time playout and on files destined for on-demand and over the top delivery.

Figure 2. In this example of a broadcast chain, QC can be completed at three points: at ingest, on files destined for real-time playout and on files destined for on-demand and over the top delivery.

The real 'heavy lifting' of the technical QC starts when a file is rendered out, as only then are the technical features finally set. The question, then, is what QC to do here? Perhaps unhelpfully, the answer is 'it depends'.

If a file is to be transmitted directly by a high-profile broadcaster (and I am including in the definition of 'broadcaster' the likes of Amazon and Netflix), where it is important for value, reputational, or other reasons that it be correct, then the level of QC should clearly be much higher than for a short, transitory web item with low value.

For many file deliveries there will be a delivery specification which covers many technical aspects such as video codecs, lead in/out, audio codecs, audio track layouts, video quality, video and audio levels as well as legal constraints, e.g. CALM Act, captions, etc. Often these file delivery specifications are extensive and complex - so much so that it is often hard for an editor doing post-production to understand all the details. (And, to be fair, why should they? The editor is generally employed for their editing skill and flair, not as an engineer.)

The simplest solution is to use a QC tool which already has built-in test templates which match the delivery specifications. It is then a case of simply selecting the right test template and seeing if the file passes or not.

Automatic test template generation

Software systems which do a comprehensive QC are extensive and sophisticated - they have to be able to cover all the aspects in the delivery specifications.

So the problem arises if there is no pre-existing test template: building a test template from scratch can take many hours and a lot of trial and error.

One of the QC test systems has an elegant solution: use the 'auto-template' feature. This uses a known-good media file as the basis to automatically build a test template, and takes less than one minute to complete.

QC by the broadcaster

The broadcaster receiving the file needs to ensure either that it is good-to-go for transmission, or to be used as the master file from which transmission versions can be created. In either case it is a question of the same delivery specification being applied, again with the overlay of value and importance to determine what QC should be done.

And the same issue arises, i.e. that of generating an appropriate test template, which means that the same answer applies.

Not at which point, but what purpose

So the answer regarding what QC to do at what point is determined by the purpose of the file at that point, not by the actual point in the process.

Figure 3. If the file is high value, as indicated in Figure 3, then comprehensive QC is demanded, both automatic and manual (click image to enlarge).

Figure 3. If the file is high value, as indicated in Figure 3, then comprehensive QC is demanded, both automatic and manual (click image to enlarge).

Always do auto QC first

It is often tempting for those sending or receiving content to take a 'quick QC look' first, before doing anything else. In general this is a mistake. The reason is fairly obvious: people are expensive - far more expensive than running a file through an auto QC system for 15 minutes. It may be that the first minute of an auto QC check reveals a fundamental problem with the file - wrong bit-rate, wrong audio layout, wrong codec - none of which would be obvious when taking that 'quick look'. So it would be easy to waste your most expensive resource looking at a file that is simply not fit for purpose: avoid this by always doing the auto QC first, then only do any manual QC after.

An alternative view on this is when considering master files which are used as the source for transcode. Figure 4 indicates a good approach: the master 'mezzanine' file is comprehensively checked before the many output versions are generated. The rationale is obvious: there is no point doing transcodes on a file which is bad to begin with.

And, depending upon the confidence in the transcode process, it may be that only an automatic QC is required after transcode, even for 'standard' content.

Figure 4. In this QC model, comprehensive QC takes place at three points; master file output and again at the output of a linear playout server and other playout/streaming channel outputs (click image to enlarge).

Figure 4. In this QC model, comprehensive QC takes place at three points; master file output and again at the output of a linear playout server and other playout/streaming channel outputs (click image to enlarge).

Thomas Dove, Telestream, Director of Technology, QC Products

Thomas Dove, Telestream, Director of Technology, QC Products

Let us know what you think…

Log-in or Register for free to post comments…