Applied Technology: Shotoku Free-d² & Broadcast Graphics

The use of on-screen graphics to add value and content to a broadcast is a longstanding production technique. The integration of interactive graphics into a live broadcast, although a more recent addition, is now a staple of news and sports productions worldwide.

In this live environment, for the interaction of a presenter and their ‘augmented reality’ to at once be convincing and effective, the challenge has always been to accurately and reliably track and capture the positional data of the broadcast camera, so that rendered graphical content can be composited seamlessly onto the video feed. Breakdowns in this process during live playout, or accumulated errors in positional data, quickly become evident to even an untrained viewer. While of course being professionally unacceptable, it would also call into question the value of the technology to help broadcasters tell their story. Additionally a tracking system must be flexible, user friendly and not impede the operation of the camera unnecessarily.

Free-d² is the second generation of ‘absolute’ tracking technology Shotoku Broadcast Systems is developing in partnership with BBC R&D in the UK. Building on the original system with refreshed and technologically advanced hardware, Free-d² calculates the camera’s absolute position, from externally mounted fixed markers and communicates this to a graphics engine in real-time. Being absolute, the system is immune to positional ‘drift’ and can be deployed onto any form of camera support - Tripod, Pedestal, Jib system or indeed, handheld - and has virtually no impact on the operation of the camera chain. 

The spotter camera’s LED shoots upward, reading the reference location disks mounted on the studio ceiling.  The system develops positional calculations including the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders.

The spotter camera’s LED shoots upward, reading the reference location disks mounted on the studio ceiling. The system develops positional calculations including the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders.

The system works by viewing the reflected light from a series of markers mounted at two distinct heights in the studio ceiling space, using an upward facing video camera. The ceiling markers, each with a unique ID, are produced from a highly retro-reflective material. This ensures light from the spotter camera’s LED’s is efficiently reflected directly back toward the camera, eliminating any impact from other bright light sources in the studio.

During installation, a studio calibration file is created containing the coordinates of each marker in extremely accurate X,Y and height positions. During operation, the video feed from the spotter camera is analysed against this calibration file and from it, the precise position of the broadcast camera in the 3D space is calculated with a minimal delay of just 1 to 2 frames. This is combined with zoom and focus data from the lens and is then distributed to the graphics rendering engine via industry standard protocols and timed to the video output.

A key benefit of the Free-d² system is that because the calibration file is for the studio, and not the individual camera, the file is deployed across all current and future additional channels. This file can remain valid for the duration of the installation, which makes the system extremely low maintenance. A new calibration is only required if there are significant changes to a number of the marker positions.

Due to the unique ID of each marker, and their bi-planar installation, positional calculations are able to report accurately and reliably not just the camera’s X,Y and height positions, but also Pan, Tilt and Roll, without the need of supplementary data from accelerometers or encoders. There is also no intervention or operational input required from the broadcast camera operator which keeps integration into workflows very simple. 

A Free-d² equipped studio affords broadcasters the opportunity to enrich programming content with the use of live, real-time graphical content. This could be in a full green screen environment, a format popular in data-intensive, non-regular broadcasts such as election night specials. A full green screen studio with reliable tracking can also be utilised to produce multiple, individually branded shows from a single location with set changes available at the push of a button.

More common recently, and perhaps a tougher test of any camera tracking system is the option to augment a traditional set with graphical content, played out into defined areas of the physical set when relevant, for example, a virtual video screen. This is a technique that has become more common as sports productions, and more recently light entertainment shows, have begun to explore the benefits of the Virtual Studio. For this to add value to a broadcast - and not to detract from it - graphical content must be placed accurately and remain stable during on-air camera moves.

Through the use of modern technologies Shotoku Broadcast’s Free-d² seeks to build on the original system’s reputation as one of the most consistently accurate and reliable tracking systems available, and as such provides the perfect platform for this type of programming. The system allows full and credible engagement with virtual set elements through its accurate positional reporting. Utilising the external, fixed positional markers, positional data will not drift, ensuring dependable, repeatable performance time after time.

Finally, by removing the need for regular recalibration the system minimises maintenance demands and with the spotter camera being operationally independent of the broadcast chain, the tracking system is both unobtrusive and undemanding on the camera operator allowing them to focus fully on making the shots. 

David Shepherd, Sales Manager, Manual & Virtual Studio, Shotoku

David Shepherd, Sales Manager, Manual & Virtual Studio, Shotoku

You might also like...

Cellphone Apps Put Control And Monitoring In The Palm Of Your Hand

Due to the flexibility and virtually unlimited access of the Internet Protocol, manufacturers of broadcast and production equipment have for years provided customers with the remote ability, via an HTML 5 browser interface, to monitor and control hardware devices via a…

AI Assisted Talent Tracking Trims Robotic Shots Without An Operator

As artificial intelligence (AI) continues to invade the video production space, and more specifically the unmanned robotic camera studio, the addition of automated shot correction/facial tracking and recognition software technology is helping to computerize tasks previously performed by a…

Improvements In Efficiencies And Performance Help Lights Shine For Better Video Production

This has been a year in which we all—reporters, producers and station engineers—had to learn the basics of good lighting, particularly the three-point setup pioneered by famed lighting inventor Ross Lowell for in-home studios. However, lighting fixtures and kit…

News Production Technologies Support Seamless Working From Home

With the pandemic’s alarming numbers now decreasing, news anchors have carefully begun reporting from the studio again, albeit in separate parts of the building and socially distanced. However, the IP-enabled technology and remote workflows developed by equipment vendors across t…

Tech + Media Predictions 2021 - Part 2

The industry experienced futureshock head-on in 2020. The impact will take a long time to unwind but it’s already clear that some changes will be profound and not all of them bad. In part 2 we look at what sports and s…