The Changing Face of Data Storage

In the world of video production, change in data storage comes so fast it is often out of sight, out of mind. For some applications, we have switched in the past few years from hard drives to flash memory. Each new model of gear we buy — whether cameras or editing — accommodates faster formats and generates ever larger files. Where is storage heading?

It wasn’t so long ago that we all used hard drives for our main data storage. Gradually, as costs went down and capacity went up, we migrated to flash memory, or SSD drives. With an SSD as a primary drive, all computer systems — and yes, I mean today’s cameras as well — boot up time is much faster compared to other types of storage.

Whether buying a new camera or recycling to new flash cards, have you noticed that the capacity and speed of storage media goes up? For a long while, the standard SD form factor was used by most smaller cameras, but as the cameras got more miniature, so did the memory. That opened the door to today’s miniSD cards.

As demand for greater amounts of storage looms, what’s next? In a recent white paper, B&H Photo, the large audio-video dealer in New York City, tackled the subject. First, it is important to understand that storage technology does not operate in a vacuum. It follows the speed of general computing and the demands of the system creating the data.

B&H noted that most current computer systems operate on a binary format of “1”s or “0”s. However, in the future, it said, computers will work with this pair of digits, as well as somewhere in between. This area in between the “1” and “0” is known as a qubit. This more refined level of computing is called quantum technology. Quantum computing, essentially, is a way to harness quantum mechanics to process information.

Quantum technology will enable calculations to be computed millions of times faster than today and can probably crack a 128-digit password before the user can even finish typing it out. Development is being fast-tracked to perfect quantum computers. Google already has working quantum machines for research and government use. IBM is making qubits from superconducting metal circuits, but is still encountering high error rates. The technology is in development, but one day quantum computing will get here.

With computing systems this fast, storage will need to be compatible with it. Not only will we need to be able to store incredible amounts of data, but it must be more durable than ever before. Today, most manufacturers rate the reliability of their drives at under a decade or sooner. After that time, it is recommended that users back up their content to a newer drive.

Research on storage is moving in multiple directions. To solve the reliability problem, B&H said research has been done to encode vast amounts of data on a medium that has been available for thousands of years. With a few strands of synthetic DNA, users could store all their data, including major video files.

These DNA-based storage drives have two major advantages: size and reliability. Theoretically, a fraction of an ounce of DNA can fit 300,000 times more data than a one TB hard drive and hold that data for more than 100 years. This storage process won’t be ready for a while, but it will follow the development of quantum technology.

What’s Next in Storage Technology?

For technology that will be available sooner (about a year), 96-layer 3D NAND technology has been recently developed in a partnership with Western Digital and Toshiba. It will provide larger storage capacities in SSDs than the typical 512GB and 1TB now on the market.

This technology will be available in three- and four-bits-per-cell architectures to deliver performance and reliability at an affordable price point, B& H said. With this technology, users should spend the same amount of money and receive twice the amount of storage space when purchasing an SSD drive built next year.

For current buyers, SSD with digital storage space in the double digits has recently come on the market. Most of us think of hard drives having the largest capacity, while SSDs providing the fastest speeds. That, however, has changed.

Seagate has commercial-grade 2.5-inch SSDs available now and offers Nytro 3000 SAS SSDs that deliver up to 15TB of storage capacities and 2,100 MB/s in sequential read speeds through dual 12 Gb/s SAS interfaces. This SAS-outfitted SSD is more than suitable to handle advanced VR rendering, so users can work in detailed 360-degree environments without experiencing stutters.

Hard drives have also improved with helium technology developed a few years ago by HGST, which is now part of Western Digital. With helium technology, cloud storage services, large data centers and multi-bay storage arrays are benefiting from high-capacity, low-operating-cost drives.

Helium technology replaces the air within the hard drive with helium and then seals this gas in the drive’s housing. Since helium is one-seventh the density of air, platters experience less friction, while arms suffer less turbulence within the drive. This reduction in resistance allows helium-based drives to consume less power than air-filled versions.

Helium drives can support an eighth platter to grant systems storage capacities of up to 12TB. If you have a NAS unit, Western Digital’s Gold Datacenter drives are suitable storage options. Another brand that offers these innovative drives is G-Technology designed for the company’s line of NAS servers.

Fujifilm LTO Catridge

Fujifilm LTO Catridge

Another notable storage technology in the market today are LTO magnetic tapes, which are neither hard drives or SSDs. These tapes are well suited to back up the contents stored on a hard drive and then retain that content for up to 30 years. Magnetic tapes are useful for writing data once and restoring it when required, letting users archive large amounts of less-frequently accessed “cold” information. Editor note: For more information on the latest LTO technology, LTO-8 see Archiware And Spectra Logic Voice LTO-8 Support.  Also, LTO-8 Specifications Announced

Higher-end LTO tapes have a native capacity of 6TB and a compressed capacity of up to 15TB. To keep up with the growing capacities of hard drives, Sony and IBM have created magnetic tape technology that can accommodate a high recording density so that future data cartridges can store up to 330TB of content.

These newer data cartridges will feature thinner tapes that sport a nano-grained magnetic layer, created using microscopic magnetic particles. Thinner tapes also allow the cartridge to house longer tapes — up to 3,280 feet in length. In addition to tape development, Sony has made a lubricant, which is placed between the tape’s surface and the magnetic head, to allow large amounts of writing done at high speeds.

Samsung Flash Drive

Samsung Flash Drive

Though storage capacity is now radically greater than a few years ago, there is still a long way to go with today’s technology. With any storage technology now available, ten years is still the maximum time that experts say it should be trusted to house valuable video content. Though the prospect of truly long term storage is on the horizon, it is not here yet.

Until the right storage technology gets here and is proven to work reliably, data should be turned over to new media every decade. That a major cost that must be factored into the cost of all video production.

Let us know what you think…

Log-in or Register for free to post comments…

You might also like...

Important!  Exercise Your SSDs (Like Any Other Drive)

Spinning disk (HDD) and flash storage (SSD) drives are nearly the same cost these days, so it’s no surprise that broadcasters are turning increasingly to SSDs for long-term storage of our most critical media files. But did you know t…

How to Choose the Fastest Memory Card for Your Application

Today, video and still cameras, tablets and even laptop computers often rely on memory cards for data storage. Each electronic device specifies a unique kind of memory and choosing the right card for the application can be challenging.

Field Report: Canon XC10 Camera

After twenty-five NAB shows I’ve developed a pattern of booths to visit each day. At NAB 2015 walking though a series of expensive C-series cameras in the Canon booth, I encountered a new species of video camera—the XC10. (Figure 1). It …

Avoiding Video Disaster When Using Flash Memory

Increasingly, flash cards are the storage media of choice for video recording. Though mostly reliable, a lot can go wrong with flash memory — most commonly human error in handling the media. An expert explains how to avoid catastrophic problems when u…

Maintaining Memory Cards for Professional Video Production

Flash cards for video production are an essential part of the production workflow. But how many videographers understand these vital storage components and care for them properly? This is a guide to treating video flash memory properly.