A Producer’s Guide to Storage – part 2

Part 2 of the Digital Production Partnership’s guidelines for independent producers looking to refresh or create a file-based asset management policy examines where content should be kept and advice for organising a digital file.

Continued from part 1

The danger in asking an expert what kind of technology you should choose for the storage of your file-based material is that it’s rather like asking a financial advisor how you should keep your money: they might actually tell you. Then you’ll be sorry.

So here’s as simple an answer as we can manage:

1. Portable hard drives. Brace yourself for an inconvenient truth: you shouldn’t even think about doing this. Pity, because it’s very probably the way you’re doing it right now. The problem is that hard-drives fail – especially if they are not in constant use.

2. Storage server. This is the non-portable kind of hard drive. Sometimes it’s referred to as spinning disk. It actually consists of a number of hard drives brought together in a server. It’s industrial strength and although it isn’t infallible, it does have built-in safeguards against going wrong. The key thing will be to set this up as a RAID, so that failure of any individual disk or other component won’t corrupt or destroy the contents of the server.

The storage server can be accessed from a desktop machine that sits on the same computer network. Files can be copied backwards and forwards providing the access to store and retrieve media. It may be possible to review clips and programmes directly, depending on the number of people doing this at the same time and the connection speeds within your office. This is a perfectly good form of file-storage, but gets expensive if you have a lot of it.

3. Data Tape. Confusingly this is sometimes referred to as tape! It’s actually very different since it’s designed to keep lots of computer files rather than a single stream of video. This format is tailor-made for long-term storage and can provide a more stable way of keeping content than spinning disk. This is most likely to be an LTO or other proprietary format. It’s relatively cheap and very reliable – as long as you take care to control environmental conditions like temperature, humidity and dust.

The only downside is that you need a way of reading the data tapes.

Larger stores of data tapes tend to be contained inside a cabinet with barcodes to identify them, and a robotic arm to collect and insert them into data tape machines. Usually the material is then moved onto a storage server. So commonly a storage server is used in conjunction with a data tape set-up, with material being moved off the expensive server onto the cheaper data tape store once it is needed less often. It will be necessary to migrate these data tapes to a newer format periodically, as data stored on older data tapes can’t always be restored by newer systems.

So although a data tape in itself may be cheap, the system you need to put around it to make it usable is actually pretty costly to create and maintain.

You can keep and read data tapes without a robotic system. Desktop tape drives are available and can be connected to a computer to allow data tapes to be read and written. It will be necessary to copy files to a local desktop before viewing. In this case, it may be easier to keep a lower quality copy, or a proxy, of media which is needed frequently somewhere more accessible such as a local server. These proxies can be removed when no longer needed, or recreated if a disk drive fails.

In addition to data tapes, there are other removable media storage options, such as optical disks, now available.

How will I find it?

This is really important. It’s no use storing something unless you can find it again when you need it. For a library or an archive you’ll need four key functions:

Search - which enables you to look for content – or elements of that content;

Catalogue - which organises your metadata, or information, about your content;

Physical management - which looks after the storage and moves the files;

Review - which enables you to browse, view and even publish your content.

These elements could all come together in a Media Asset Management (MAM) system, or as a number of different tools all linked together. In either case there are some golden rules to consider:

  1. Give each file a Unique Identifier. This is a number which is distinct from all others in the archive or library.
  2. Decide how you’ll name each file. You’ll need a logical system (File Naming Convention) to make this easy and consistent.
  3. Make sure your unique identifiers and file names enable you to distinguish between, and keep track of, different versions of a file.
  4. Decide what information (metadata) you want to keep about a file.
  5. Decide whether to keep that metadata with the file itself, or in a separate database, or both. If both, decide which you’ll regard as the master set of information.
  6. Work out how best to catalogue (organise) the metadata so it’s most useful.
  7. Figure out how you will search for a file and view it. This will require either a MAM system or separate tools linked together.

Watch out: this summer the DPP will be publishing a more technical guide to storage in the broadcast industry.

Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

Essential Guide:  Practical Broadcast Storage

Ground breaking advances in storage technology are paving the way to empower broadcasters to fully utilize IT storage systems. Taking advantage of state-of-the-art machine learning techniques, IT innovators now deliver storage systems that are more resilient, flexible, and reliable than…

Practical Broadcast Storage - Part 3

Artificial Intelligence (AI) has made its mark on IT and is rapidly advancing into mainstream broadcasting. By employing AI methodologies, specifically machine learning, broadcasters can benefit greatly from the advances in IT infrastructure innovation and advanced storage designs.

Practical Broadcast Storage - Part 2

Broadcast systems are renowned for their high speed and high capacity data demands. Up to recently, they relied on bespoke hardware solutions to deliver the infrastructure required for live real-time uncompressed video. But new advances in IT data storage have…

Practical Broadcast Storage - Part 1

Broadcast and IT technology collaboration is continuing its journey with the cross-over between them becoming ever clearer. Storage is a major growth area with advanced methods achieving greater efficiencies and resilience. And as broadcast repeatedly demands more and more capacity,…

Cost-effective IP Contribution and Distribution

Saving dollars is one of the reasons broadcasters are moving to IP. Network speeds have now reached a level where real-time video and audio distribution is a realistic option. Taking this technology to another level, Rohde and Schwarz demonstrate in…