Continued from part 1

The danger in asking an expert what kind of technology you should choose for the storage of your file-based material is that it’s rather like asking a financial advisor how you should keep your money: they might actually tell you. Then you’ll be sorry.

So here’s as simple an answer as we can manage:

1. Portable hard drives. Brace yourself for an inconvenient truth: you shouldn’t even think about doing this. Pity, because it’s very probably the way you’re doing it right now. The problem is that hard-drives fail – especially if they are not in constant use.

2. Storage server. This is the non-portable kind of hard drive. Sometimes it’s referred to as spinning disk. It actually consists of a number of hard drives brought together in a server. It’s industrial strength and although it isn’t infallible, it does have built-in safeguards against going wrong. The key thing will be to set this up as a RAID, so that failure of any individual disk or other component won’t corrupt or destroy the contents of the server.

The storage server can be accessed from a desktop machine that sits on the same computer network. Files can be copied backwards and forwards providing the access to store and retrieve media. It may be possible to review clips and programmes directly, depending on the number of people doing this at the same time and the connection speeds within your office. This is a perfectly good form of file-storage, but gets expensive if you have a lot of it.

3. Data Tape. Confusingly this is sometimes referred to as tape! It’s actually very different since it’s designed to keep lots of computer files rather than a single stream of video. This format is tailor-made for long-term storage and can provide a more stable way of keeping content than spinning disk. This is most likely to be an LTO or other proprietary format. It’s relatively cheap and very reliable – as long as you take care to control environmental conditions like temperature, humidity and dust.

The only downside is that you need a way of reading the data tapes.

Larger stores of data tapes tend to be contained inside a cabinet with barcodes to identify them, and a robotic arm to collect and insert them into data tape machines. Usually the material is then moved onto a storage server. So commonly a storage server is used in conjunction with a data tape set-up, with material being moved off the expensive server onto the cheaper data tape store once it is needed less often. It will be necessary to migrate these data tapes to a newer format periodically, as data stored on older data tapes can’t always be restored by newer systems.

So although a data tape in itself may be cheap, the system you need to put around it to make it usable is actually pretty costly to create and maintain.

You can keep and read data tapes without a robotic system. Desktop tape drives are available and can be connected to a computer to allow data tapes to be read and written. It will be necessary to copy files to a local desktop before viewing. In this case, it may be easier to keep a lower quality copy, or a proxy, of media which is needed frequently somewhere more accessible such as a local server. These proxies can be removed when no longer needed, or recreated if a disk drive fails.

In addition to data tapes, there are other removable media storage options, such as optical disks, now available.

How will I find it?

This is really important. It’s no use storing something unless you can find it again when you need it. For a library or an archive you’ll need four key functions:

Search - which enables you to look for content – or elements of that content;

Catalogue - which organises your metadata, or information, about your content;

Physical management - which looks after the storage and moves the files;

Review - which enables you to browse, view and even publish your content.

These elements could all come together in a Media Asset Management (MAM) system, or as a number of different tools all linked together. In either case there are some golden rules to consider:

1. Give each file a Unique Identifier. This is a number which is distinct from all others in the archive or library.
2. Decide how you’ll name each file. You’ll need a logical system (File Naming Convention) to make this easy and consistent.
3. Make sure your unique identifiers and file names enable you to distinguish between, and keep track of, different versions of a file.
4. Decide what information (metadata) you want to keep about a file.
5. Decide whether to keep that metadata with the file itself, or in a separate database, or both. If both, decide which you’ll regard as the master set of information.
6. Work out how best to catalogue (organise) the metadata so it’s most useful.
7. Figure out how you will search for a file and view it. This will require either a MAM system or separate tools linked together.

Watch out: this summer the DPP will be publishing a more technical guide to storage in the broadcast industry.