A Producer’s Guide to Storage – part 2

Part 2 of the Digital Production Partnership’s guidelines for independent producers looking to refresh or create a file-based asset management policy examines where content should be kept and advice for organising a digital file.

Continued from part 1

The danger in asking an expert what kind of technology you should choose for the storage of your file-based material is that it’s rather like asking a financial advisor how you should keep your money: they might actually tell you. Then you’ll be sorry.

So here’s as simple an answer as we can manage:

1. Portable hard drives. Brace yourself for an inconvenient truth: you shouldn’t even think about doing this. Pity, because it’s very probably the way you’re doing it right now. The problem is that hard-drives fail – especially if they are not in constant use.

2. Storage server. This is the non-portable kind of hard drive. Sometimes it’s referred to as spinning disk. It actually consists of a number of hard drives brought together in a server. It’s industrial strength and although it isn’t infallible, it does have built-in safeguards against going wrong. The key thing will be to set this up as a RAID, so that failure of any individual disk or other component won’t corrupt or destroy the contents of the server.

The storage server can be accessed from a desktop machine that sits on the same computer network. Files can be copied backwards and forwards providing the access to store and retrieve media. It may be possible to review clips and programmes directly, depending on the number of people doing this at the same time and the connection speeds within your office. This is a perfectly good form of file-storage, but gets expensive if you have a lot of it.

3. Data Tape. Confusingly this is sometimes referred to as tape! It’s actually very different since it’s designed to keep lots of computer files rather than a single stream of video. This format is tailor-made for long-term storage and can provide a more stable way of keeping content than spinning disk. This is most likely to be an LTO or other proprietary format. It’s relatively cheap and very reliable – as long as you take care to control environmental conditions like temperature, humidity and dust.

The only downside is that you need a way of reading the data tapes.

Larger stores of data tapes tend to be contained inside a cabinet with barcodes to identify them, and a robotic arm to collect and insert them into data tape machines. Usually the material is then moved onto a storage server. So commonly a storage server is used in conjunction with a data tape set-up, with material being moved off the expensive server onto the cheaper data tape store once it is needed less often. It will be necessary to migrate these data tapes to a newer format periodically, as data stored on older data tapes can’t always be restored by newer systems.

So although a data tape in itself may be cheap, the system you need to put around it to make it usable is actually pretty costly to create and maintain.

You can keep and read data tapes without a robotic system. Desktop tape drives are available and can be connected to a computer to allow data tapes to be read and written. It will be necessary to copy files to a local desktop before viewing. In this case, it may be easier to keep a lower quality copy, or a proxy, of media which is needed frequently somewhere more accessible such as a local server. These proxies can be removed when no longer needed, or recreated if a disk drive fails.

In addition to data tapes, there are other removable media storage options, such as optical disks, now available.

How will I find it?

This is really important. It’s no use storing something unless you can find it again when you need it. For a library or an archive you’ll need four key functions:

Search - which enables you to look for content – or elements of that content;

Catalogue - which organises your metadata, or information, about your content;

Physical management - which looks after the storage and moves the files;

Review - which enables you to browse, view and even publish your content.

These elements could all come together in a Media Asset Management (MAM) system, or as a number of different tools all linked together. In either case there are some golden rules to consider:

  1. Give each file a Unique Identifier. This is a number which is distinct from all others in the archive or library.
  2. Decide how you’ll name each file. You’ll need a logical system (File Naming Convention) to make this easy and consistent.
  3. Make sure your unique identifiers and file names enable you to distinguish between, and keep track of, different versions of a file.
  4. Decide what information (metadata) you want to keep about a file.
  5. Decide whether to keep that metadata with the file itself, or in a separate database, or both. If both, decide which you’ll regard as the master set of information.
  6. Work out how best to catalogue (organise) the metadata so it’s most useful.
  7. Figure out how you will search for a file and view it. This will require either a MAM system or separate tools linked together.

Watch out: this summer the DPP will be publishing a more technical guide to storage in the broadcast industry.

You might also like...

NAB Show 2024 BEIT Sessions Part 2: New Broadcast Technologies

The most tightly focused and fresh technical information for TV engineers at the NAB Show will be analyzed, discussed, and explained during the four days of BEIT sessions. It’s the best opportunity on Earth to learn from and question i…

Standards: Part 6 - About The ISO 14496 – MPEG-4 Standard

This article describes the various parts of the MPEG-4 standard and discusses how it is much more than a video codec. MPEG-4 describes a sophisticated interactive multimedia platform for deployment on digital TV and the Internet.

Chris Brown Discusses The Themes Of The 2024 NAB Show

The Broadcast Bridge sat down with Chris Brown, executive vice president and managing director, NAB Global Connections and Events to discuss this year’s gathering April 13-17 (show floor open April 14-17) and how the industry looks to the show e…

Essential Guide: Next-Gen 5G Contribution

This Essential Guide explores the technology of 5G and its ongoing roll out. It discusses the technical reasons why 5G has become the new standard in roaming contribution, and explores the potential disruptive impact 5G and MEC could have on…

Audio For Broadcast: Cloud Based Audio

As broadcast production begins to leverage cloud-native production systems, and re-examines how it approaches timing to achieve that potential, audio and its requirement for very low latency remains one of the key challenges.