The EBU (European Broadcasting Union) BroadThinking 2017 event highlighted work still to be done for high quality streaming.
Online streaming quality has improved greatly over the last decade through increases in bandwidth and better optimization around transmission and playback. But significant challenges remain, including latency, platform fragmentation and failure to rally round a single codec capable of delivering Ultra HD (UHD) content cost effectively over available bandwidth.
Such challenges were discussed at the European Broadcasting Union (EBU) Broad Thinking 2017, where Thierry Fautier, VP Video Strategy at Harmonic and President of the Ultra HD Forum, identified key areas where progress was needed and highlighted the way forward in each case. Broadly these areas cover compression, delay, user experience, mobile video and live UHD.
On the compression front, HEVC (High Efficiency Video Codec) has been emerging for years as the industry’s preferred successor to H.264, roughly doubling compression efficiency, but has been bogged down in a licensing quagmire which, despite efforts by the patent holders, has not been satisfactorily resolved, according to Fautier. At the same time HEVC has been criticized for failing to deliver a sufficient improvement in efficiency to cope with the vastly increased bandwidth demands of UHD services, which quadruple resolution while also being set to increase frame rate.
On this basis, the doubling in compression performance every decade that the industry has been accustomed to under the longstanding MPEG regime is no longer fit for purpose, which has led to emergence of alternative codecs such as V-Nova’s Perseus, which deliver superior performance but with questions over compatibility, standardization and lack of full independent verification.
While these issues may be resolved there is growing momentum behind the idea of content-aware encoding, which can dovetail with existing codecs to yield further improvements in compression. The idea in simple terms is that some content, such as talking heads, can be compressed a lot more than others, such as fast-moving nature scenes, yielding overall savings in bandwidth and efficiency while improving the experience in many cases.
Traditionally compression has been equated to resolution by bitrate ladders, which fail to take account of the massive variations in amount of detail within frames and rate of variation between frames. Netflix has used such a static bit ladder which for example specifies a bit rate of 5.8 Mbps for “full HD” at 1920 x 1080 pixels, 1.05 Mbps for standard definition at 640 x 480 and 235 Kbps for 320 x 240 resolution sufficient only for small screens at low quality.
Netflix uses a set of bitrate-resolution pairs (referred to as a bitrate ladder). They were selected such that the bitrates were sufficient to encode the stream at that resolution without significant encoding artifacts. This “one-size-fits-all” fixed bitrate ladder achieves, for most content, good quality encodes given the bitrate constraints.
Under Content Aware Encoding (CAE), video encoding is no longer determined by bit rate ladders but instead by analyzing the characteristics of each video, which can be done either to optimize bit rate for a given video quality, or alternatively to deliver the best experience possible at all times at a given bit rate. This is what ABRS (Adaptive Bit Rate Streaming) is broadly set to do, but allied to content aware encoding there is scope for much further performance or efficiency gains.
CAE is already employed within some OTT platforms, such as Kaltura’s, where it is applied to almost all content ingested into the system. Fautier though is calling for CAE to become a universal standard more tightly integrated across the video ecosystem, which would bring greater scope for optimization.
Different objectives or use cases could then be supported. A service provider wanting to save on bandwidth globally might encode content at different bitrates to match varying complexities. The aim would be to yield a consistent quality as perceived by the user while exploiting differences in content complexity for maximum compression efficiency at all times. On the other hand, to deliver the highest average quality across all content, then all video would be encoded at the same resolution. Viewers of simpler content would then benefit from extra quality, bringing up the average.
There would also be scope for taking account of the viewing device. Given that the human eye has a given ability to resolve angles in field of vision, small defects in the picture quality are not visible on screens such as smartphones where the pixel density is high, but much more noticeable on big TV screens, especially at close viewing distance. So device type can be combined with content complexity to optimize resolution and bit rate.
Thierry Fautier, VP Video Strategy at Harmonic and President of the Ultra HD Forum, advocates content aware encoding as the most attractive replacement for HEVC with its licensing nightmare.
The second area - latency - discussed by Fautier has gained prominence with the rise of live OTT services, especially sports, driving initiatives to bring delay in line with broadcast. There have been some proprietary products such as Net Insight’s Sye that achieve this in many situations, but again there are advantages in having a standard approach, which has emerged through CMAF (Common Media Access Framework), whose primary purpose is to unite the industry behind a common format for OTT distribution and playback on all devices, in particular unifying Apple HLS with MPEG DASH. But CMAF also includes a low latency mode, which can be set to limit delay by bypassing plug-ins that exceed a given threshold to execute. Only plug-ins with latencies within the budget are allowed to run with others being bypassed. Incorporation of this CMAF low latency mode has been shown to cut end-to-end delays down to five seconds, comparable or lower than many broadcast services. There can admittedly be a price to pay bypassing plug-ins that after all have some purpose such as optimizing audio, so care must be taken to ensure that the other aspects of the experience are not sacrificed on the altar of low latency.
Indeed, delay is just one aspect of experience. In many cases, it is more important that the video plays out smoothly at the highest quality possible and without buffering even if there is a delay. At the EBU event Fautier cited the role of transparent caching, sometimes called open caching, in delivering a smooth buffering-free streaming experience. The Streaming Video Alliance (SVA) set up in November 2014 laid the foundations for a new more efficient content caching layer in April 2015, aiming to make optimal use of network capacity. This has led to development of Open Caching Request Routing and HTTPS Delegation Technical Specification, which has been trialed by members of a working group comprising Charter, Limelight Networks, Major League Baseball Advanced Media, Qwilt, Verizon, Viacom, ViaSat, and Yahoo.
The Streaming Video Alliance was formed to develop, publish, and promote open standards, policies and best practices for streaming media.
The basic idea is simple enough, to improve quality and indeed minimize latency for on-demand services by distributing content closer to users. But because it is not feasible to place all but the most popular content in its entirety in edge caches near users, transparent caching has been developed as a compromise. It allows business rules to be set that determine how much of each piece of content should be moved to the network edge in order to make best use of the available storage and core network bandwidth for the optimum overall experience.
This allows content to start playing out from this edge cache, which then in the background refreshes itself over the network from the center. The idea is that the most popular content then resides entirely in the edge cache layer to avoid consuming too much core bandwidth through excessive refreshing. By contrast only a small amount of the least popular “long tail” content is held in edge cache to avoid consuming too much storage while still ensuring smooth playout as the cache refreshes, noting that this will not use so much bandwidth because it doesn’t happen very often.
Mobile delivery is another huge area for streaming, given that by 2020 video will account for 70% of all traffic over cellular networks, according to Cisco. Scalability is a big issue because Mobile Network Operators (MNOs) naturally want to minimize the deployment of extra infrastructure to cope with the video proliferation. A related issue is the cost to users of data associated with video, which is much greater per unit time than that consumed by other applications and services. As a result, there has been growth both in data throttling, where MNOs cap the quality of video to say standard definition 480p, and zero rating, where the cost of data generated by a particular service such as Netflix is not counted against a user’s data allowance.
Fautier highlighted T-Mobile USA’s Binge On service as an example, copied by AT&T Verizon and Sprint, involving commercial agreements with OTT video providers which limit resolution to a fixed value in return for unlimited streaming, at least within certain bounds. Fautier anticipated that zero rating would become more popular and so OTT service providers should be prepared.
T-Mobile's Binge On permits users on a qualifying plan to stream free, unlimited video from their favorite services like YouTube, Netflix, HBO NOW, all without using their high-speed data.
Finally Fautier considered the advent of UHD streaming, for both live and on-demand content. A key challenge for this year is delivery of HDR (High Dynamic Range) content, given that this has been shown to yield the biggest uplift in perceived quality for a given hike in bandwidth. One problem has been a proliferation in HDR variants for different use cases, but this is no longer a barrier to adoption because it is possible to convert between these HDR versions at the head end, in the network or on the device. Fautier presented Hybrid Log Gamma (HLG) as the preferred version for live UHD content. This was developed by the BBC in the UK and Japanese national broadcaster NHK as a more convenient HDR package for the broadcast world than the HDR10 system deployed in most current content sources and displays. This is because it combines standard dynamic range and high dynamic range images in a single video signal that can play out as SDR (Standard Dynamic range) on legacy TVs and HDR on HLG-compatible screens. Its advantage for OTT is that it avoids consuming the extra bandwidth needed to carry separate SDR and HDR signals.
For broadcasters, it scores by being accommodated within current 10-bit production workflows.
Related Editorial Content
Is the rush of interest in Ultra HD a flash in the pan, or will we see complete broadcast chains from the camera to the home? If you attended NAB this year you would be forgiven for thinking that every…
Not even a success yet, 4K video is now getting some enhancements and is being rebranded as Ultra HD Premium. Engineering experts have long said that 4K alone does not have the “wow” factor to jump out at consumers. Premium add…
The Ultra HD Forum released the first phase of industry Guidelines on end-to-end workflows for creating and delivering live and pre-recorded UHD content. The Ultra HD Forum is a global organization defining industry best practices for the introduction of technologies…