Applied Technology: Shotoku Free-d² & Broadcast Graphics

The use of on-screen graphics to add value and content to a broadcast is a longstanding production technique. The integration of interactive graphics into a live broadcast, although a more recent addition, is now a staple of news and sports productions worldwide.

In this live environment, for the interaction of a presenter and their ‘augmented reality’ to at once be convincing and effective, the challenge has always been to accurately and reliably track and capture the positional data of the broadcast camera, so that rendered graphical content can be composited seamlessly onto the video feed. Breakdowns in this process during live playout, or accumulated errors in positional data, quickly become evident to even an untrained viewer. While of course being professionally unacceptable, it would also call into question the value of the technology to help broadcasters tell their story. Additionally a tracking system must be flexible, user friendly and not impede the operation of the camera unnecessarily.

Free-d² is the second generation of ‘absolute’ tracking technology Shotoku Broadcast Systems is developing in partnership with BBC R&D in the UK. Building on the original system with refreshed and technologically advanced hardware, Free-d² calculates the camera’s absolute position, from externally mounted fixed markers and communicates this to a graphics engine in real-time. Being absolute, the system is immune to positional ‘drift’ and can be deployed onto any form of camera support - Tripod, Pedestal, Jib system or indeed, handheld - and has virtually no impact on the operation of the camera chain. 

The spotter camera’s LED shoots upward, reading the reference location disks mounted on the studio ceiling.  The system develops positional calculations including the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders.

The spotter camera’s LED shoots upward, reading the reference location disks mounted on the studio ceiling. The system develops positional calculations including the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders.

The system works by viewing the reflected light from a series of markers mounted at two distinct heights in the studio ceiling space, using an upward facing video camera. The ceiling markers, each with a unique ID, are produced from a highly retro-reflective material. This ensures light from the spotter camera’s LED’s is efficiently reflected directly back toward the camera, eliminating any impact from other bright light sources in the studio.

During installation, a studio calibration file is created containing the coordinates of each marker in extremely accurate X,Y and height positions. During operation, the video feed from the spotter camera is analysed against this calibration file and from it, the precise position of the broadcast camera in the 3D space is calculated with a minimal delay of just 1 to 2 frames. This is combined with zoom and focus data from the lens and is then distributed to the graphics rendering engine via industry standard protocols and timed to the video output.

A key benefit of the Free-d² system is that because the calibration file is for the studio, and not the individual camera, the file is deployed across all current and future additional channels. This file can remain valid for the duration of the installation, which makes the system extremely low maintenance. A new calibration is only required if there are significant changes to a number of the marker positions.

Due to the unique ID of each marker, and their bi-planar installation, positional calculations are able to report accurately and reliably not just the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders. There is also no intervention or operational input required from the broadcast camera operator which keeps integration into workflows very simple. 

A Free-d² equipped studio affords broadcasters the opportunity to enrich programming content with the use of live, real-time graphical content. This could be in a full green screen environment, a format popular in data-intensive, non-regular broadcasts such as election night specials. A full green screen studio with reliable tracking can also be utilised to produce multiple, individually branded shows from a single location with set changes available at the push of a button.

More common recently, and perhaps a tougher test of any camera tracking system is the option to augment a traditional set with graphical content, played out into defined areas of the physical set when relevant, for example, a virtual video screen. This is a technique that has become more common as sports productions, and more recently light entertainment shows, have begun to explore the benefits of the Virtual Studio. For this to add value to a broadcast - and not to detract from it - graphical content must be placed accurately and remain stable during on-air camera moves.

Through the use of modern technologies Shotoku Broadcast’s Free-d² seeks to build on the original system’s reputation as one of the most consistently accurate and reliable tracking systems available, and as such provides the perfect platform for this type of programming. The system allows full and credible engagement with virtual set elements through its accurate positional reporting. Utilising the external, fixed positional markers, positional data will not drift, ensuring dependable, repeatable performance time after time.

Finally, by removing the need for regular recalibration the system minimises maintenance demands and with the spotter camera being operationally independent of the broadcast chain, the tracking system is both unobtrusive and undemanding on the camera operator allowing them to focus fully on making the shots. 

David Shepherd, Sales Manager, Manual & Virtual Studio, Shotoku

David Shepherd, Sales Manager, Manual & Virtual Studio, Shotoku

You might also like...

Ashley Xu - Crossing The Divide Between Creating For Social Media And Traditional Professional Media

Ashley Xu blew up on social media when she started making ‘thirst traps’ (adverts) in her college dorm. Check them out, you may be surprised by just how good they are. It did not take long for major brands to see…

Creating The Look & Feel For Netflix Film ‘Purple Hearts’ With Cinematographer Matt Sakatani Roe

Matt Sakatani Roe shares his experiences shooting the Netflix film ‘Purple Hearts’, staying hands on with the camera, staying close to the characters and leveraging natural light.

Creating The Look & Feel For Netflix Series ‘Mo’ With Cinematographer Timothy M Burton

Timothy Burton shares his insight into how the Netflix series ‘Mo’ tackles a potentially difficult subject with humour and style, with a look and feel that captures the heat of its location and pace of its narrative.

Creating Virtual Production Environments With Production Supervisor Don Bitters

Much of the attention enjoyed by virtual production currently goes to the spectacular stages with LED displays the size of half a dozen cinema screens. The material we put on those displays, though, can come from a number of places,…

Image Capture & Processing At IBC 22 - Pristine Quality, IP & Cloud Native, Multi-Format Workflows

While cloud services and remote production have dominated many conversations at recent trade shows, cameras and image processing remain a key part of any workflow. At the upcoming IBC Show, many companies will show new technology for capturing an image…