The storage of professional video images really began with 2-in tape and quadraplex tape machines. Today’s retiring-age engineers are about the only people who ever relied on the technology. The early versions could be cranky, difficult to optimize and prone to requiring constant maintenance. Even so, both media and machine technology improved and the marriage of tape and recorder/player became the primary storage for video media for several decades.

Today multiple technologies are available to store data. For content creators, a key requirement is the ability to recover, perhaps years later, that content is fundamental.

Survey conclusions

A recent study by the data storage company Spectra Logic examined these four technologies. The results of their investigation of storage platforms revealed five key points.

• Solid-state disk (SSD) will completely own the portable, smartphone, and tablet sector, and will eventually replace all magnetic disk in laptop and desktop storage. Two-and-half-inch 15,000 RPM magnetic disk will be displaced by enterprise flash.
• The flash industry will likely improve its manufacturing costs associated with 3D NAND, thereby allowing it to continue its downward trend of cost per capacity.
• Tape has the easiest path to higher capacity, but that will require continuous drive and media technology investment. The size of the tape market will result in further consolidation, perhaps leaving only one drive and two tape media suppliers.
• Optical disc will remain a niche technology, primarily in the media and entertainment sector, unless the cost of higher capacity discs is reduced.
• No new storage technologies will have significant impact on the storage digital universe through 2026 with the possible exception of solid-state technologies whose characteristics straddle DRAM and flash.

A Look at Trends

Expected technology trends and prices are shown in Figure 1. Each tier represents the speed and frequency of content recall. For instance, the inherent latency with cloud storage makes it a less than optimum solution for editing tasks. Likewise, using SSD for archive is both expensive and unnecessary, as more tuned solutions are available. Content creators may require solutions across all of the tiers.

Solid-State Storage

Today’s primary storage technologies include; SSD, magnetic disk, magnetic tape, optical disc and some niche solutions.

The fastest growing technology in this space is NAND flash. Its durability and speed make it a good solution for both consumer and enterprise segments.

For enterprise applications, solid-state is the storage technology of choice for database applications that require low-latency characteristics. Even so, to support continued development of the technology, manufacturers will need to expand into market areas once dominated by hard disks. Currently, consumer-grade flash drives sell for approximately $0.25 per GB. Consumer-grade magnetic disk drives sell for approximately $0.025 per GB (1/10^th the cost).

The Spectra Logic report shows that solid-state storage will have the highest growth rate of all storage mediums in terms of exabytes shipped. The company suggests that the fabrication technology of 3D NAND will improve and costs will continue decrease.

Magnetic Disk Solutions

Today’s disk drive industry has two major providers (Seagate, HGST) and one smaller provider (Toshiba). The industry is challenged by flash on three fronts:

The 2.5-inch 15,000 RPM enterprise drives are being replaced with enterprise flash. These flash solutions are packaged either as solid-state disks that are drop-in replacements for disk drives or as PCIe-connected drives. The bus-connected storage further eliminates the storage network stack and its latency.

The second challenge to the spinning disk industry comes from the consumer side. Laptops and desktop computers now come fully equipped with SSD storage. While early applications focused primarily on supporting a computer’s OS, new application software can benefit from the speed of an SSD.

A longer-term effect the spinning disk industry faces is in the removal, not displacement, of disk drives in home gaming devices and digital video recorders. This disruption is not due to flash, but rather the availability of higher bandwidth links into the home. The higher bandwidth opens the possibility of gaming from cloud servers, not from local gaming devices, and streaming media on demand rather than from a DVR. *

The following chart shows an overall decline of about 75 million disk drives year-to-year from 2015 to 2016.

Spectra Logic suggests that by 2020, the disk industry will be servicing a singular market composed of large IT shops and cloud providers. In order to preserve that market, their products must maintain current reliability, while at the same time continue to decrease the per-capacity cost. Protection of market share requires a multiple cost differential compared to consumer solid-state disk technologies.

Newer Spinning Desk Technologies

Disk drive makers have introduced three technologies to increase capacity on the standard platter size. First, Helium-filled drives, which means less friction than air-filled drives resulting in lower heat and internal turbulence. This allows the drive to be more efficient and provide higher capacity storage.

Second, Shingled Magnetic Recording (SMR) drives write tracks like shingles on a roof, such that the surface area between traditional disk tracks are eliminated resulting in higher capacity. One vendor is delivering variable capacity drives whereby each drive of a designated capacity arrives at a slightly different capacity point. For example, 6TB drives might have capacities ranging from 6TB to 6.4TB. The manufacturer formats a disk to its full potential instead of downsizing it to a specific capacity.

A third HD storage improvement technology is Heat Assisted Magnetic Recording (HAMR). This process increases the areal density of a disk platter by heating the target area with a laser. This heated area results in reduced coercivity, which allows a lower and more focused charge to “flip” a smaller bit.

The technology launch date for a 16TB HAMR drive has been postponed several times and is predicted to arrive in late 2018. When HAMR drives do become available, they will further challenge RAID controllers as the rebuild times for 20TB, 30TB, and perhaps, 50TB devices, will become prohibitively long.

Tape Solutions

Despite the antiquated “Tape is dead” banter, tape technology is not dead. While the digital tape business for secondary storage has seen year-to-year declines as IT backup moves to disk-based solutions, the need for tape in the long-term archive market continues to grow.

Tape technology is well suited for archive applications as it provides the benefits of low environmental footprint for both floor space and power. It also offers a high level of data integrity over a long period and has a much lower cost per GB of storage than all other storage mediums.

A fundamental shift is underway whereby small tape systems are being replaced by cloud-based storage solutions. At the same time, large cloud providers are adopting tape -- either as the medium of choice for backing up their data farms or for providing an archive tier of storage to their customers.

Linear Tape Open (LTO) technology has been and will continue to be the primary tape technology. The LTO consortium assures interoperability for manufacturers of both LTO tape drives and media. In 2015, the seventh generation of this technology was introduced, providing 6TB native (uncompressed) capacity per cartridge.

An eighth generation of LTO (LTO-8), offering 12TB of storage, is scheduled to ship in October 2017. IBM will soon begin shipping its TS1155 drive, which has a native capacity of 15TB. These tape drives are the first to use TMR technology (Tunneling Magnetoresistance) which should allow the drive capacity to double four more times. Additionally, these drives will be the first generation to offer a native high-speed 10GE RoCE interface. [RDMA over Converged Ethernet (RoCE) is a network protocol that allows remote direct memory access (RDMA) over an Ethernet network.] The Spectra Logic report predicts that IBM will be the sole manufacturer of enterprise-class tape drives and media in the years to come.

Optical Storage

In early 2014, Sony Corporation and Panasonic Corporation announced a new optical disc designed for long-term digital storage. Trademarked “Archival Disc”, it will initially be introduced at a 300GB capacity point and will be write-once. An agreement covers the raw unwritten disk such that vendors previously manufacturing DVDs will have an opportunity to produce Archival Disc media.

However, unlike LTO tape, there is no interchange guarantee between the two drives. In other words, a disc written with one vendor’s drive may or may not be readable with the others. Even if one vendor’s drive is able to read another’s, there may be a penalty of lower performance.

Sony packages 11 discs into its cartridge that is slightly larger in both width and depth than an LTO tape cartridge. The value proposition here is its longevity and its backward read compatibility.

For content creators with long-term (essentially forever) archival requirements, the archive disc will find favor. The size of this particular market segment is small compared with the overall market for archival storage. The ability for this technology to achieve greater market penetration is primarily a function of the cost of the media. Today, the cheapest storage medium is LTO-6 tape with roughly a $25 price for 2.5TB of storage or $.01/GB.

In order to be price equivalent, a 300GB archive disc needs to cost $3.60 (300 GB x $.01/GB). If the media reaches the market at $36 per disc, about the price of today’s 100 GB optical disc, the 10 times differential between this and tape technology will lock archival disc technology into the market niche described above.

The Crystal Ball

An IDC report, commissioned by EMC in 2012, predicted the need to store more than 40 zettabytes (ZB) of digital data in 2020. This caused many in the industry to wonder whether there would be sufficient media to contain it.

A more realistic view might cut that number by 50-percent. For example, the creation of a proposal or slide show might generate dozens of revisions -- some passed through versioning software and some scattered on local disk. Including auto-saved drafts, a copy on the email server, and copies on client machines, there might easily be 100 times the original data that could be archived.

A larger project will create even more easily discarded data. Photos or video clips not chosen can be discarded moved to less-expensive storage. In addition, data stored for longer retention is frequently compressed, further reducing the amount of storage.

In the case of video production, a producer might record ten times or more content than needed for a production. The following post production work could easily result in another 10X to 100X multiplier. Is all that media ever erased? Seldom.

The “long-tail” model will continue to favor current storage, as larger capacity devices are brought online. This means the cost of storing last year’s data becomes less significant. For many companies, all their data from 10 years ago would fit on a single tape today.

Editor’s note:

This article is sourced from the Spectra Logic Digital Data Storage Outlook 2017. A copy of the report can be obtained at the embedded link.

*See The Broadcast Bridge recent articles on these related technologies:

Streaming Quality Improves But Challenges Remain

Apple Selects HEVC For Streaming