Disaster recovery strategies

We’ve all heard the phrase Disaster Recovery (DR), but what does it actually mean for broadcasters and content owners, and what constitutes a disaster? DR is a broad term that encompasses a range of scenarios, from catastrophic disaster (for instance, the complete destruction of a whole facility), to operational disaster such as a transmission server failing. The ideal strategy for rescuing a situation in the event of a disaster is the seamless continuity of business under all circumstances with no assets being lost.

In the past DR strategies have involved staff picking up boxes of tapes and equipment, jumping in a car, driving to another facility and getting back on air as quickly as possible. These days being off-air for longer than a few seconds, or a minute at most, is a disaster in itself and with content now being delivered globally, broadcasters have even greater responsibility to ensure that channels stay on air. They often also face 99.999 per cent contracts with their channel partners with reference to airtime, and there are rules on compensation for lost or clipped ads. So there are also strong financial drivers for staying on air. If a catastrophe occurs at a facility in one part of the world, a DR strategy needs to be in place that will automatically kick-in from another. With increased availability of wide area bandwidth and dark fibre it’s much easier to share content globally but it is still not cheap.

So how do broadcasters ensure that their DR strategies are secure enough to continue broadcasting in any situation? The DR utopia includes multi-layered safeguards against the unexpected, using automated content replication systems to provide synchronised, mirrored or like-for-like asset duplication, across the same site or at geographically disparate locations.

At its most straightforward, this can be accomplished by duplicating tapes in the main archive and then moving those tapes to remote DR storage. LTFS works well in this environment as any LTFS-capable system can read a tape created by any other, and can identify and retrieve the files stored on it. This means there is no requirement for a second archive system to simply read those files.

At the other end of the functionality scale, a fully automated DR-configured archive can be connected to a remote facility with either a robotic tape or disk storage. In this configuration media assets can be automatically copied across the network and synchronised with the remote site. This model is ideal for broadcasters whose main and DR archives are separated by many hundreds of kilometres.

As we can see, automated site redundancy is an important factor for broadcasters and can be achieved by using rules-based implementations, providing fully-automated data duplication across multiple storage layers and locations. Disaster Recovery systems enable multi-site operations to be mirrored and data synchronised across the globe. If one site becomes inoperative, it can be rebuilt entirely from data that has been replicated to other sites. 

Remote site WAN based SGL FlashNet archive

The more sophisticated archive management systems are able to offer completely customisable rules-based data duplication, through which content can be automatically copied as it is archived across disk and tape layers and, where required, different locations. In single-site scenarios, duplicate tapes can be easily externalised from the storage system, singly or in content-based groups, and removed to safe locations. Once a DR strategy is in place, it’s also important to periodically practise scenarios and test equipment.

SGL has many archives installed around the world where Disaster Recovery workflows are either in use or can be made DR-capable quickly and easily. Its scalable FlashNet architecture provides broadcasters and content owners with a clustered system of multiple servers, or nodes, each in constant communication. Each cluster node has identical software installed, and each is connected via fibre channel into the archive devices - generally disk storage and one or more tape libraries. At the heart of the cluster is a Microsoft SQL database, which is usually installed across two servers running a Microsoft cluster for automatic failover.

Successful DR projects around the world will encourage more multi-site operations to adopt a distributed approach to their content management. By ensuring sufficient time is spent on collaborative design, an efficient DR strategy can be successfully achieved ensuring that a channel can continue transmission regardless of any disaster without losing a single frame.

Quite simply assets equal value, which equals revenue. Can you afford to lose them?

Paul Moran is CTO & joint MD at SGL and Lee Sheppard is director of product management at SGL

You might also like...

Designing IP Broadcast Systems: Integrating Cloud Infrastructure

Connecting on-prem broadcast infrastructures to the public cloud leads to a hybrid system which requires reliable secure high value media exchange and delivery.

Video Quality: Part 1 - Video Quality Faces New Challenges In Generative AI Era

In this first in a new series about Video Quality, we look at how the continuing proliferation of User Generated Content has brought new challenges for video quality assurance, with AI in turn helping address some of them. But new…

Minimizing OTT Churn Rates Through Viewer Engagement

A D2C streaming service requires an understanding of satisfaction with the service – the quality of it, the ease of use, the style of use – which requires the right technology and a focused information-gathering approach.

Production Control Room Tools At NAB 2024

As we approach the 2024 NAB Show we discuss the increasing demands placed on production control rooms and their crew, and the technologies coming to market in this key area of live broadcast production.

Designing IP Broadcast Systems: Where Broadcast Meets IT

Broadcast and IT engineers have historically approached their professions from two different places, but as technology is more reliable, they are moving closer.