Why Creators Need CFexpress

What Is CFexpress?

CFexpress is a new generation storage medium designed from the ground up to provide content creators with the speed, capacity, and versatility they need. It represents a genuine leap forward for mobile storage, featuring a point-to-point connection to a host computer’s data bus (the same bus you plug powerful GPUs into) via exclusive “lanes”. Now supported widely by cameras, adaptors and software, there has never been a more critical storage technology for today’s - and tomorrow’s - media production.

Since SD cards were introduced in 1991, storage capacity has increased and costs have plummeted, both by factors of hundreds of thousands. And yet they look the same.

This sameness or familiarity can lead us to underestimate the rate of change, which can be exponential. 8K video was just an experiment twelve years ago. Today, you can buy 8K monitors and even a 17K cinema camera. The uncompressed data output from a 17K camera at 24 frames per second is a staggering 6.8 gigabytes per second. That’s an entire DVD’s worth of data per second.

Luckily, we don’t need to store uncompressed video at such high resolutions. Modern compression codecs like Apple ProRes and ProRes RAW mean that we can preserve all the visual quality while keeping file sizes manageable. ProRes RAW is widely considered to be the industry standard, and Atomos has played no small part in making it happen, but the seismic shift of DaVinci Resolve supporting ProRes from version 20.2 proved to be the clincher.

Pain Relief

Speed and bandwidth have always been the biggest pain points for videomakers. The resolution of modern cameras is higher than ever. Even though HD is still a common delivery resolution, 4K is preferable for acquisition. As a delivery resolution, it looks fantastic on every kind of screen, and is especially important for cinema, large TVs, and even social media, where laptop screens benefit from the extra resolution and detail. Where available, 6K, 8K, 12K, and even 17K are now entirely feasible, often at higher frame rates than 24, 25, or 30 FPS. This can lead to some eyewatering demands on your mobile or in-camera storage. It would be a shame to have to reduce quality to accommodate slow storage.

Video data is fundamentally different to “traditional” data. It’s huge - often running to hundreds of gigabytes and multiple terabytes, and it’s continuous. Conventional (computer) storage “expects” a shorter, more bursty kind of data, where response time is more important than sustained speed. For video, the opposite is true. Sustained operation is crucial, and while fast “seek” times are important, there won’t be as many individual transactions as with, say, a database.

Bigger, Faster: Storage Has Evolved

Until the advent of digital video, in-camera storage was film or tape. Film still has a certain “something” that keeps it in production, albeit for a niche market. You could say it is the vinyl of filmmaking.

The first digital cameras used tape with a rotating head mechanism - just like domestic video recorders. Tape was cheap, but the cameras were complicated and would never be able to solve the biggest downside of the medium, which was that it wasn’t capable of random (or non-linear) access. Sony made professional cameras with recordable optical drives, which did offer non-linear access, and, crucially, the first glimpse of production with a file system.

Solid-state storage was the obvious next step, but at first, it was eye-wateringly expensive. Panasonic’s P2 range of cameras, launched in 2004, used a stack of memory cards arranged in a RAID format. Even though early cards (with a maximum capacity of 4GB) cost several thousand dollars/euros, the format became the basis for virtually all ENG cameras. The cost soon diminished, as Moore’s law and economies of scale took hold.

Connecting drives to a computer has never been completely straightforward. Luckily, modern technology simplified the process. Before solid state storage, the challenge was to attach a spinning metal plate to a computer which had no concept of materials or rotational physics. The answer was a disk controller interface that translated and mediated the data flow so that the drive became merely a logical device to the computer system. The downside? It was slow. ATA (and Serial ATA, also referred to as “SATA”) made the process more efficient but still couldn’t run at the speed of the computer’s bus and early versions were taxing for the CPU.

Solid State Disks (SSDs) left behind the mechanical nuances, but came with their own issues, now largely smoothed out by newer technology. SATA remains a valid storage interface, but it can’t compete with the speed of a “native” PCIe connection. (SATA SSDs max out at around 600 megabytes per second). That said, SATA is reliable and stable and has formed the basis for attaching drives for over twenty years.

Waiting For The Bus

PCIe is a mass transportation system for data. It’s like the London Underground but with trains moving at the speed of Concorde. Let’s look first at how a data bus works, and then what makes PCIe so fast.

Before PCIe came onto the scene in 2003, a computer bus was a simple affair where every part inside a computer could “talk” to every other part. It did this by claiming an exclusive slice of time on the bus to move its data. These time slices were interwoven so that, from a distance, it appeared as if all the devices were talking at once, but the reality was that while one device had control of the bus, all the other devices remained idle. This technique was called Time Division Multiplex (TDM).

PCIe is a dramatic improvement over a TDM bus. It behaves as if each component (a CPU, GPU or internal memory) has its own exclusive connection or lane to the bus. Nothing else can share a lane. Each connected device has its own superhighway. And that changes everything.

It means that for time-critical data, nothing can interrupt a transfer. It’s perfect for media production because it has a massive bandwidth - for PCIe 5.0 the maximum theoretical throughput is a whopping 4GB per second!

Traditionally, external devices used general-purpose interfaces like USB and Thunderbolt. Both are flexible – and that is their great advantage – but it also limits them. They don’t provide a direct interface to PCIe traffic, but instead, they “tunnel” it, losing some efficiency in the process. How much this matters to you depends on your absolute need for speed. You might not notice it at all.

But CFexpress is designed as a direct PCIe endpoint. It’s essentially like plugging storage directly into your computer’s internal PCIe lanes, providing extremely high throughput without the overhead of protocol conversion. But to take full advantage of that PCIe speed, you need a direct PCIe link.

There are two main ways to achieve this:

• Use a CFexpress reader connected through an M.2-to-CFexpress adaptor that plugs into your motherboard’s PCIe M.2 slot.
• Install a PCIe expansion card that provides a CFexpress slot. This would effectively treat the CFexpress card as an NVMe drive connected directly to the PCIe bus.

These configuration options are ideal for large-scale DIT work and any workflow with massive, sustained data transfers. For most other use cases, recent versions of USB and Thunderbolt still deliver excellent speeds that are likely to be more than sufficient.

Three Types Of CFexpress, But Only One Really Matters To Filmmakers

CFexpress cards come in three flavors: A, B and C. Spoiler alert: You’re most likely to choose type B.

• Type A is the slowest and smallest, supported by a few Sony cameras, but unlikely to be chosen for mainstream use. It uses one PCI lane and with PCIe 4 can achieve sustained speeds of around 600–900 MB/s write, 700–1000 MB/s read depending on the individual card and host.
• Type B is compatible with the largest number of cameras and is also physically compatible with Sony XQD slots, which require Sony to have updated its camera firmware to support CFexpress. It uses two PCIe lanes and with PCIe 4 is capable of sustained speeds of around 1.0–1.7 GB/s write, 1.5–2.0+ GB/s read depending on card and host.
• Type C is the fastest, and similar in appearance to CFast and CompactFlash cards. But it is not used in any cameras available today. Given the already very high speeds of current Type B cards, it’s possible that the need for the additional speed of Type C in consumer products may never materialize or may be several years away.

CFexpress Holds All The Cards

CFexpress is a foundational shift in external, mobile storage technology. It removes every speed limit, every restriction, and every bottleneck from the path to and from an external memory card. It does this by connecting directly to a PCIe bus, “talking” that bus’s language natively.

With no need to translate data to and from intermediate formats (such as USB or Thunderbolt), the connection can run at its full capacity. PCIe’s architecture, which, with CFexpress, is extended all the way to the memory card, dedicates “lanes” to the external storage so that no other transaction within the host computer can interrupt or slow down data transfers. No other memory card format has all these benefits.

CFexpress is the first external memory format that has been designed with filmmakers in mind. That’s why Atomos has designed an entirely new Ninja series around this technology. The new Ninja TX GO and Ninja TX are the most powerful Ninjas we’ve ever made.