Augmented Reality the Next Revolution

While the consumer electronics industry focuses on bigger displays and ever higher resolution, there are pioneers developing a more compatible technology, augmented reality television. How close is it and what are the benefits?

Television is old. Around 70 years ago television began the march to become mainstream and commonly found in most modern households, but little changed until more recently. There have been, of course, a number of technical exploits that have improved the quality of the TV immensely. Today viewers can enjoy images in 4k HDR color when before it was UHF black & white. Yet the broadcast industry has kept the same paradigm of watching something behind a screen in a very passive way.

Not to be pretentious but, Augmented Reality (AR) will make today’s television displays a museum artifact. If we can have a virtual screen wherever we want with the size that we want, why would need to have a real one? Once viewers begin consuming AR-enabled television, broadcasters and content owners will realize that they are not limited to a rectangular flat shape anymore and will create new innovative storytelling techniques to entertain us.

“You want to watch TV? We could put a digital TV on that wall and instead of being a piece of hardware, it's a $1 app, instead of a $500 piece of equipment.”
Mark Zuckerberg, Facebook's F8 conference, April 20, 2017

Television and AR 

There should be little doubt that AR is going to be the main way of interacting with information in the coming years. We don’t know yet when, but it’ll happen. Apple and Google are leading this revolution,and taking the necessary steps towards this change. In the near future we are going to have socially-acceptable AR and VR glasses that will let us see through lenses, mixing virtual and real worlds. It will be as revolutionary as the move from print magazines and newspapers to the web, or from the web to mobile apps. The next user experience will be a leap to AR.

The television and cinema industries can be expected to take a dominant position on the AR wave. While television could someday be completely holographic, we are not yet there in terms of production technology and device availability. However, it is time to begin taking the necessary steps in this direction, creating mixed experiences with television and augmented reality, what we call Augmented Television.

Augmented Television is a state where the watching experience is enhanced by augmented reality experiences in a way where the glass separation between the content and viewer is removed.

Augmented Television is a state where the watching experience is enhanced by augmented reality experiences in a way where the glass separation between the content and viewer is removed.

How to do AR

It is not necessary to wait for fancy AR glasses to interact with AR experiences. It is already possible today with smartphones and tablets. This part is solved by the AR frameworks created by Apple and Google. They just require a regular smartphone or tablet to be able to use AR. There are literally billions of devices, already in our hands, already compatible with AR.

On the content production side, we already have a solid foundation, essentially thanks to the animation and video gaming industries, which have been developing 3D modelling and animation for the last 30 years. It is cheap and easy to create or reuse 3D models. For testing, there are libraries like Poly and Sketchfab. The formats used in AR are compatible with existing models, so it is easy to reuse old 3D models in AR .

But of course, new television cannot be created just with 3D models. The development of a unique video format native for AR will be required. Some technology is already capable of recording actors and scenes in three dimensions. Not two stereoscopic images put together to simulate 3D images in our brains, but pure 3D video recordings. This technology is called holographic video or volumetric video.

Some companies are developing this capability including Microsoft with its Mixed Reality Capture Studios.. They are the pioneers in professional holographic production, but today they require multiple cameras and many hours of post-processing. Jaunt, a company known for their VR productions, is focusing now on AR and have recently developed a production set that works with six cameras and can broadcast holograms live.

As technology progresses, expect future mobile phones to be able to record a simplified version of AR video. At least today's professional sets enable the recording of video holograms for integration into AR experiences.

Sports

There are several applications well suited for the use of augmented television. Many are being developed by companies including Facebook, Microsoft and others.

Sport programming is already experimenting with AR-enhanced imagery.

Sport programming is already experimenting with AR-enhanced imagery.

Augmented Reality for sport may be one of the first broadcast applications. Even though each sport has its own particularities, all of them can have an enhanced experience by making use of different cameras, statistics, maps etc.

An early AR sport application was developed to follow a cycling race. While watching the regular broadcast on TV, viewers could interact with the map of the race. Each participant’s location was identified on the map, complete with extra layers of information such as the slope level. Individual cameras would display exact locations on the map, even when they are moving like helicopters or motorbikes. The attention gathered and the intuitiveness of the experience make a compelling argument that this is the model to follow for interactive television.

Documentaries

Documentaries and education may also drive this new technology, syncing the regular video with, for example, 3D models of the objects or places being showcased.

Perhaps a viewer is watching a fantastic documentary about the ecosystem of a forest. Yet he would like to see a model of the tree roots delivered right into the living room so he could better understand its full complexity. Maybe he wants to see the microscopic insects become real. For example, when watching a documentary about sea animals, the moment when they talk about whales, there could be an interactive whale floating in the living room. When the documentary changes the whale for a dolphin, the AR experience might synchronize and show the audience a dolphin.

The level of interactivity and detail will vary depending on the type of documentary. For a documentary about the Egyptian pyramids, the viewer might wish to pause the video, allowing them time to explore the inside of the pharaoh’s tomb. The value of such AR experiences for education and learning is limitless.

TV for Kids

Frankly, children’s television is almost halfway there. How many times have you seen a child talking to the TV? With their limitless imagination, every fairy toy can have a life and its own personality. Imagine for a second the cartoon characters asking the kids to play with an augmented reality toy. Or see a part of the story happening inside the screen while a character in the living room is replying back. This world of magic fantasy is not that far away.

Augmented Reality applied to children’s programming may be one of the early applications. The imagery is simpler and children are eager for new experiences. Click to enlarge.

Augmented Reality applied to children’s programming may be one of the early applications. The imagery is simpler and children are eager for new experiences. Click to enlarge.

The Power of AR advertising

AR and advertising fit together like peanut butter and jelly. Augmented TV applications could be linked to television advertisements. The technology could allow buyers to see and evaluate purchases with a high degree of realism. When a viewer watches an AR-enabled ad, it could be tagged to a history list, letting them interact with the product live or in the future. This opens a new channel of monetization for broadcasters as the value of interactive advertising is much higher than the classic eyeballs advertising hit.

Shopping is a perfect example of augmented television. Amazon offers an application that permits the consumer to place life-like product images into the buyer's home. Both color and 3-dimensionality is retained while the buyer moves the product around the floor-space. Click to enlarge.

Shopping is a perfect example of augmented television. Amazon offers an application that permits the consumer to place life-like product images into the buyer's home. Both color and 3-dimensionality is retained while the buyer moves the product around the floor-space. Click to enlarge.

AR Has a Bright Future

Until we reach a future where most television content is holographic, broadcasters can expect to have a fruitful present with augmented reality experiences combined with television. This will roll out slowly at the beginning, as this standard becomes a commodity.

José Somolinos, Innovation Strategist, Accedo

José Somolinos, Innovation Strategist, Accedo

You might also like...

KVM & Multiviewer Systems At NAB 2024

We take a look at what to expect in the world of KVM & Multiviewer systems at the 2024 NAB Show. Expect plenty of innovation in KVM over IP and systems that facilitate remote production, distributed teams and cloud integration.

NAB Show 2024 BEIT Sessions Part 2: New Broadcast Technologies

The most tightly focused and fresh technical information for TV engineers at the NAB Show will be analyzed, discussed, and explained during the four days of BEIT sessions. It’s the best opportunity on Earth to learn from and question i…

Standards: Part 6 - About The ISO 14496 – MPEG-4 Standard

This article describes the various parts of the MPEG-4 standard and discusses how it is much more than a video codec. MPEG-4 describes a sophisticated interactive multimedia platform for deployment on digital TV and the Internet.

Chris Brown Discusses The Themes Of The 2024 NAB Show

The Broadcast Bridge sat down with Chris Brown, executive vice president and managing director, NAB Global Connections and Events to discuss this year’s gathering April 13-17 (show floor open April 14-17) and how the industry looks to the show e…

Comms In Hybrid SDI - IP - Cloud Systems - Part 1

We examine the demands placed on hybrid, distributed comms systems and the practical requirements for connectivity, transport and functionality.