The Changing Face of Data Storage

In the world of video production, change in data storage comes so fast it is often out of sight, out of mind. For some applications, we have switched in the past few years from hard drives to flash memory. Each new model of gear we buy — whether cameras or editing — accommodates faster formats and generates ever larger files. Where is storage heading?

It wasn’t so long ago that we all used hard drives for our main data storage. Gradually, as costs went down and capacity went up, we migrated to flash memory, or SSD drives. With an SSD as a primary drive, all computer systems — and yes, I mean today’s cameras as well — boot up time is much faster compared to other types of storage.

Whether buying a new camera or recycling to new flash cards, have you noticed that the capacity and speed of storage media goes up? For a long while, the standard SD form factor was used by most smaller cameras, but as the cameras got more miniature, so did the memory. That opened the door to today’s miniSD cards.

As demand for greater amounts of storage looms, what’s next? In a recent white paper, B&H Photo, the large audio-video dealer in New York City, tackled the subject. First, it is important to understand that storage technology does not operate in a vacuum. It follows the speed of general computing and the demands of the system creating the data.

B&H noted that most current computer systems operate on a binary format of “1”s or “0”s. However, in the future, it said, computers will work with this pair of digits, as well as somewhere in between. This area in between the “1” and “0” is known as a qubit. This more refined level of computing is called quantum technology. Quantum computing, essentially, is a way to harness quantum mechanics to process information.

Quantum technology will enable calculations to be computed millions of times faster than today and can probably crack a 128-digit password before the user can even finish typing it out. Development is being fast-tracked to perfect quantum computers. Google already has working quantum machines for research and government use. IBM is making qubits from superconducting metal circuits, but is still encountering high error rates. The technology is in development, but one day quantum computing will get here.

With computing systems this fast, storage will need to be compatible with it. Not only will we need to be able to store incredible amounts of data, but it must be more durable than ever before. Today, most manufacturers rate the reliability of their drives at under a decade or sooner. After that time, it is recommended that users back up their content to a newer drive.

Research on storage is moving in multiple directions. To solve the reliability problem, B&H said research has been done to encode vast amounts of data on a medium that has been available for thousands of years. With a few strands of synthetic DNA, users could store all their data, including major video files.

These DNA-based storage drives have two major advantages: size and reliability. Theoretically, a fraction of an ounce of DNA can fit 300,000 times more data than a one TB hard drive and hold that data for more than 100 years. This storage process won’t be ready for a while, but it will follow the development of quantum technology.

What’s Next in Storage Technology?

For technology that will be available sooner (about a year), 96-layer 3D NAND technology has been recently developed in a partnership with Western Digital and Toshiba. It will provide larger storage capacities in SSDs than the typical 512GB and 1TB now on the market.

This technology will be available in three- and four-bits-per-cell architectures to deliver performance and reliability at an affordable price point, B& H said. With this technology, users should spend the same amount of money and receive twice the amount of storage space when purchasing an SSD drive built next year.

For current buyers, SSD with digital storage space in the double digits has recently come on the market. Most of us think of hard drives having the largest capacity, while SSDs providing the fastest speeds. That, however, has changed.

Seagate has commercial-grade 2.5-inch SSDs available now and offers Nytro 3000 SAS SSDs that deliver up to 15TB of storage capacities and 2,100 MB/s in sequential read speeds through dual 12 Gb/s SAS interfaces. This SAS-outfitted SSD is more than suitable to handle advanced VR rendering, so users can work in detailed 360-degree environments without experiencing stutters.

Hard drives have also improved with helium technology developed a few years ago by HGST, which is now part of Western Digital. With helium technology, cloud storage services, large data centers and multi-bay storage arrays are benefiting from high-capacity, low-operating-cost drives.

Helium technology replaces the air within the hard drive with helium and then seals this gas in the drive’s housing. Since helium is one-seventh the density of air, platters experience less friction, while arms suffer less turbulence within the drive. This reduction in resistance allows helium-based drives to consume less power than air-filled versions.

Helium drives can support an eighth platter to grant systems storage capacities of up to 12TB. If you have a NAS unit, Western Digital’s Gold Datacenter drives are suitable storage options. Another brand that offers these innovative drives is G-Technology designed for the company’s line of NAS servers.

Fujifilm LTO Catridge

Fujifilm LTO Catridge

Another notable storage technology in the market today are LTO magnetic tapes, which are neither hard drives or SSDs. These tapes are well suited to back up the contents stored on a hard drive and then retain that content for up to 30 years. Magnetic tapes are useful for writing data once and restoring it when required, letting users archive large amounts of less-frequently accessed “cold” information. Editor note: For more information on the latest LTO technology, LTO-8 see Archiware And Spectra Logic Voice LTO-8 Support.  Also, LTO-8 Specifications Announced

Higher-end LTO tapes have a native capacity of 6TB and a compressed capacity of up to 15TB. To keep up with the growing capacities of hard drives, Sony and IBM have created magnetic tape technology that can accommodate a high recording density so that future data cartridges can store up to 330TB of content.

These newer data cartridges will feature thinner tapes that sport a nano-grained magnetic layer, created using microscopic magnetic particles. Thinner tapes also allow the cartridge to house longer tapes — up to 3,280 feet in length. In addition to tape development, Sony has made a lubricant, which is placed between the tape’s surface and the magnetic head, to allow large amounts of writing done at high speeds.

Samsung Flash Drive

Samsung Flash Drive

Though storage capacity is now radically greater than a few years ago, there is still a long way to go with today’s technology. With any storage technology now available, ten years is still the maximum time that experts say it should be trusted to house valuable video content. Though the prospect of truly long term storage is on the horizon, it is not here yet.

Until the right storage technology gets here and is proven to work reliably, data should be turned over to new media every decade. That a major cost that must be factored into the cost of all video production.

You might also like...

Project Managing The Creative Elements Of Live Sports Production

Huw Bevan is an Executive Producer, Consultant and Head of Cricket for Sunset+Vine, in London, one of the UK’s leading independent sports production companies that produces a full slate of rugby, soccer and cricket events each year. This…

Designing IP Broadcast Systems: Timing

How adding PTP to asynchronous IP networks provides a synchronization layer that maintains fluidity of motion and distortion free sound in the audio domain.

Standards: Part 4 - Standards For Media Container Files

This article describes the various codecs in common use and their symbiotic relationship to the media container files which are essential when it comes to packaging the resulting content for storage or delivery.

Standards: Appendix E - File Extensions Vs. Container Formats

This list of file container formats and their extensions is not exhaustive but it does describe the important ones whose standards are in everyday use in a broadcasting environment.

System Showcase: Delivering 175 Camera Production For The Repco Supercars ‘Bathurst 1000’

The Bathurst 1000 is a massive production by anybody’s standards, with 175 cameras, 10 OB’s, 250 crew and 31 miles of fiber cable. Here is how the team at Gravity Media Australia pull it off.