Archiving In The Digital Age

In 2007, it was estimated that the planet had stored 275 zettabytes (or 10 to the 1021) of data - the equivalent of a stack of CDs reaching from earth to beyond the moon! And in a single generation, our planet will add more information than in all of recorded history, some 40 trillion gigabytes of digital information. But, that data is delicate and easily lost.

Across the globe, the preservation and importance of archives has risen in prominence due to two important factors. The first is when archives and the archivist are under most scrutiny during moments in history when retrospection is at its most important. National anniversaries, independence celebrations, major political upheavals; across the world archivists are watching a ticking clock toward the moment when the eyes of the nation will be on them to deliver that speech from half a decade earlier or that image of a young child who later would lead a nation to independence.

The other important factor about archives is the continual degradation of the archive media. In an irony of our age, as technology has allowed more information to be stored in greater density, in some aspects, the durability of the medium has declined. Where an inscribed stone tablet from 2000bc Babylon will still be legible today, a 2 1⁄2-inch print from the 1890 will almost certainly be in a state of decay that will render it unrecognizable within a century. 

Many of the 20 or so tape formats developed for audio and video since the 1950’s are now reaching the point where the magnetic encoding is failing as is the physical condition of the cartridges. Even the modern hard disk using electrometric plates, with a storage capacity equivalent to 10 million pages while spinning, will only last a decade at best.

Historically, records were something physical that a human could see and hold. Finding records was often a long and manual process. Image courtesy keptarts.brighton.ac.uk

Historically, records were something physical that a human could see and hold. Finding records was often a long and manual process. Image courtesy keptarts.brighton.ac.uk

The cause of damage

Even climate controlled and highly sterile environments can’t sustain forever the magnetic patterns and chemical compound in film stock that will eventually eat away at the material. The danger is not just ageing. In one of the largest studies of recent times; in 1995 the United Nations Educational, Scientific and Cultural Organization (UNESCO) and International Federation of Library Associations and Institutions (IFLA) gathered information on 6,250 archives and repositories from around the world and found that over 95% had reported significant losses during the previous 50 years. In closer detail; armed conflict, fire, flood, civil disorder and poor storage conditions topped the list of causes of damage.

Yet, it is video that is set to dominate the next century. YouTube, as the world’s most popular video-on-demand website, uploads roughly 4000 hours of video each hour and other popular sites such as DailyMotion and the Chinese site Youku are quickly catching up in terms of viewable material. However, the most valuable archives are still national repositories, often built on less commercial grounds as guardians of history for education and cultural enrichment.

Media degradation

Often working in close co-operation with state broadcasters, over the last 10 years many archives such as British National Archives have added more video content than all of the combined texts collected in the previous century. For every document or video viewed in its reading room in London, a further 200 are distributed via the internet. This is a result of a major digitisation project which started in 2008 that has helped it to offer a better service, to more researchers, historians and filmmakers while crucially reducing costs. Yet the British Archive, like many others in Singapore and Malaysia, has undertaken digitisation and computerisation projects to meet the bigger threat of media degradation.

Before restoration can even begin, the restorer needs to do the groundwork which starts with an investigation of the condition of archive storage such as temperature, humidity and light. The stored metadata attached to each item needs to be understood and catered for in the restoration processes and each tape item needs careful inspection.

Cleaning and preparation

These steps are vital as some restoration processes, although potentially providing the best results, have risks which must be assessed and weighed up by the owner of the content. In some cases, more cumbersome and labour intensive processes need to take place to minimise potential damage. Sometimes tapes may need to be baked in a special oven to remove contaminants with each batch requiring different temperatures, duration and cooling conditions to prepare for digitisation.

The careful cleaning and preparation for playback often needs to take place using custom modified equipment that reduces the strain on fragile stock. In fact, a single reel or tape may go through a 30 man-hour process at the hands of several specialists before it even gets to a digitisation step.

Even well-funded archives rarely do these processes in house as the skills and technical equipment are both costly to obtain and challenging to maintain. Restoration of media is not a subject taught on most university syllabuses - yet the need is there!

As the ticking clock of media degradation intersects with upcoming national anniversaries and increasing requests for access to the archive, archivists are seeking help. Ultimately, there are more unrestored and deteriorating assets in national archives than there are people with the skills to preserve the content – and time is running out!

Spectra Logic T-Finity represents current long-term storage technology

Today's large and long-term storage platforms rely on digital technology. Without the proper technical support, all those records cannot be recovered. While reliable, how long will the media continue to use physical tape and mechanics to store humanity's history?

You might also like...

Standards: Part 11 - Streaming Video & Audio Over IP Networks

Streaming services deliver content to the end-users via an IP network connection. The transport process is similar to broadcasting and shares some of the same technologies but there are some unique caveats.

Designing IP Broadcast Systems: Routing

IP networks are wonderfully flexible, but this flexibility can be the cause of much frustration, especially when broadcasters must decide on a network topology.

Audio For Broadcast: Cloud Based Audio

With several industry leading audio vendors demonstrating milestone product releases based on new technology at the 2024 NAB Show, the evolution of cloud-based audio took a significant step forward. In light of these developments the article below replaces previously published content…

Future Technologies: New Hardware Paradigms

As we continue our series of articles considering technologies of the near future and how they might transform how we think about broadcast, we consider the potential processing paradigm shift offered by GPU based processing.

Standards: Part 10 - Embedding And Multiplexing Streams

Audio visual content is constructed with several different media types. Simplest of all would be a single video and audio stream synchronized together. Additional complexity is commonplace. This requires careful synchronization with accurate timing control.