Applied Technology: Video Clarity RTM Scheduler Expands QC Measurement Capability

The economics of the media and entertainment industry require that broadcasters and CDNs deliver a quality product every time to every screen so that viewers are provided with the best possible experience. This means engineers must be able to easily measure key signal parameters to be sure the signal meets delivery requirements.

Today's viewers demand a lot. They expect to get their content on a variety of platforms in almost any location and with the utmost quality. For broadcasters and CDNs, that can be a high bar to reach. An important tool in meeting viewer demands is an automated quality control (QC) measurement system. 

Measurement speed is important

Because many content delivery platforms must deliver multiple channels, the quicker and easier the measurements can happen, the better. But with a high number of channels, formats, and bit rates to manage, traditional 'one-of' QC solutions may be inadequate. Originators and distributors need a way to measure all those content streams quickly, efficiently, and automatically. Ultimately, automated measurement and QC can help ensure a quality viewing experience, with greater retention rates while meeting SLA requirements.

Figure 1. Example of delivering multiple OTT streams per program, each with a different bit rate.

Figure 1. Example of delivering multiple OTT streams per program, each with a different bit rate.

Why care about QC?

As viewers increasingly demand their content on any device and anywhere, streamed and downloaded programs to TVs, laptops, tablets, or phones must be properly provisioned for each such device. See Figure 1. Here the streamed programming is being delivered at three different bit rates so that any device can adapt its playback for changes in network conditions.

Quality control versus quality measurement

Measuring quality in any type of entertainment delivery network, means checking multiple criteria, many of which are IT-centric. The testing process traditionally checked network performance and programming quality, but only after it went to air. That is the basic definition of QC. Today this process should include quality measurement as well. That is, measuring the true quality of any given content throughout the entire processing chain. 

IT testing for network performance versus testing for A/V quality are different and they require different tools and procedures. It is easier to perform an objective QC test to make sure the infrastructure is functioning properly. Measuring A/V quality is more subjective because it is based on human perception.

The once laborious and lengthy human perception measurement process can now be managed by accurate test equipment and procedures. Quality issues such as lip sync errors, excessive loudness or caption alignment accuracy can now be measured in real time using scientifically proven algorithms.

PQA--key to accurate measurement

In-depth A/V quality analysis is highly subjective and relies on the human Perceptual Quality Analysis (PQA) model. What makes electronic PQA analysis possible are test methods that use a numeric quality score that is correlated to standardized, subjective databases. The databases are derived from human perceptual studies based on Recommendation ITU-R BT.500-13 (01.12) — Methodology for the subjective assessment of the quality of television pictures. The studies yielded two databases of mean opinion scores (MOS).

Three types of algorithms have been developed to estimate perceived quality in a precise way. The results of these algorithms are then correlated against the correctly produced subjective data described above. The result becomes a measurement of subjective quality, PQA, that closely approximates a human’s perception of the picture quality.

Full-reference algorithms yield the most accurate results because the equipment performing the test is supplied with two copies of the content: a source (or reference) version of the video content, and a post-broadcast version that has been processed through some type of network or equipment. The powerful full-reference technique allows measurements to be made in service, so that normal video delivery can continue uninterrupted while tests are being performed.

Multi-stream testing: The next step in quality measurement

The next step in the evolution of quality measurement is to develop tools that can do it with maximum accuracy on multiple content streams. The challenge for broadcasters and CDNs is to test in-network quality and customer QoE across multiple screens and devices at different data rates, using a range of platforms, resolutions and bandwidth. All of these factors make multichannel quality measurement incredibly complex.

Figure 2. TV network and CDN example of quality test points (click to enlarge).

Figure 2. TV network and CDN example of quality test points (click to enlarge).

Multi-stream quality measurement

There are two key types of service providers that may benefit from multi-stream quality measurement technology.

Content originators

This group could include those who produce content and then rely on delivery by: broadcast networks, cable, satellite, IPTV and OTT methods. Content originators have an ever-increasing number of delivery requirements and must ensure their content is delivered at both constant and adaptive bit rates within set parameters for video quality, audio quality, loudness, and lip sync.

Secondary delivery service providers

This category includes companies that provide the actual delivery of content to the viewer. This category includes broadcasters, cable, IPTV, direct broadcast, and satellite providers, as well as ISPs, CDNs, and hybrid organizations that operate as both originator and delivery service. 

These groups will be concerned with additional signal issues such as audio loudness. Broadcasters must meet certain FCC-mandated CALM act requirements. Finally, these companies will be concerned about used bandwidth. Proper testing can ensure compliance and log all of these factors.

Existing options are not enough

Some existing solutions do a fairly good job of identifying syntax and/or data and IP issues in both the file and the IP domains. Those well-deployed solutions can tell engineers whether the data packets/streams are arriving at their destinations intact, but they can’t identify with a high degree of accuracy the quality that’s being delivered. They cannot monitor what comes out of the endpoint decoder or define the true, reference-based quality consistently throughout the chain.

Today, “eyeballing it” is the norm, but as more revenue becomes tied to OTT content, such a practice will insufficient. Users must be able to accurately measure the quality of program signal as it flows from source to receive points. The only way to do that is with a solution that relies on full-reference test methods for perceptual quality.

Figure 3. RTM Scheduler Example (click to enlarge).

Figure 3. RTM Scheduler Example (click to enlarge).

Video Clarity RTM Scheduler

One solution that solves the source-to-viewer quality measurement dilemma is provided by Video Clarity's RTM Scheduler. The tool relies on the company’s RTM real-time audio and video monitoring solution. The RTM Scheduler allows operators to use a single RTM unit to monitor, sequentially, multiple programs or channels on a user-defined schedule.

RTM Scheduler generates a sequence of server commands according to a user-generated schedule. The input file provides the test start time, target's IP address and a list of the tests to be completed.

To run the scheduler, users select the tool in Windows Explorer, which opens a command window that also captures log status. Alternatively, users can start it directly from a command window. In either case, status for all runs is maintained in a log. If users modify the input file while the tool is running, the program immediately interprets the changes and regenerates the schedule while it is executing.

The benefits

Although the RTM solution can monitor only one channel at a time, Video Clarity's RTM Scheduler makes it easy to perform multiple sequential tests. One RTM device can perform quality measurements and fault-monitoring sessions on several channels in sequence automatically. This capability is especially useful when the RTM unit is monitoring an SDI source against an IP-processed source.

Given today’s complex and dynamic content delivery environment, the ability to measure the quality of multiple streams automatically in a highly accurate way is critically important. It helps content originators and distributors ensure that viewers get the best possible QoE, that advertisers are satisfied, and that regulations are met. RTM Scheduler is a tool designed for use with Video Clarity’s RTM real-time monitoring system.

Adam Schadle, Vice President, Video Clarity

Adam Schadle, Vice President, Video Clarity

Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

Rohde & Schwarz Embraces IP Standards at IBC 2017

Rohde & Schwarz, like many companies in this sector, were once known as hardware manufacturers but are now evolving to offer new solutions. In a future where IP connectivity reigns and resolution increases to 4K and beyond it is vital…

Ensuring Performance in Hybrid IP/SDI Video Networks

The broadcast equipment industry is in the process of making the transition to IP based transport for video, audio and data. This has led to development of a suite of standards including SMPTE ST 2022-6 for encapsulation of uncompressed SDI…

Documentation Part 2: Designing and Documenting an IP Architecture

Some engineers can maintain their current SDI systems armed with little more than a foggy memory of how things are interconnected. But with IP networks, such a philosophy guarantees panic if something fails. When it comes to properly documenting an…

Understanding IP Networks - Forward Error Correction

In the last article, we looked at the monitoring packet delay in real-time. In this article, we continue the theme of looking at a network from a broadcast engineers’ point of view so they can better communicate with the IT d…

Articles You May Have Missed

Broadcast and IT network engineers view the same datalink quite differently. Neither viewpoint is wrong, but confusion between terms and performance set the stage for misunderstanding. Tony Orme explains terminology and how both types of engineers can better understand the…