Camera Lenses Part 7: Determining Image Quality Requires More Than Just Counting Pixels

In previous articles, John Watkinson argued that the concept of static resolution has been shown not to be a useful metric for moving pictures. In short, pretending that some quality of a still image tells anything about motion is absurd, and leads to flawed ideas like high line or pixel counts that don’t actually achieve anything.

In practically all technologies, early equipment was marginal because, by definition, there were no text books. The Wright Flyer flew, but barely. What the Wright Brothers did was to build a flying machine whose faults didn’t quite prevent it from flying. The faults were not yet understood, else they would have been eradicated.

What happened next should be a role model for the development of other technologies. Because airplanes need a lot of power, they have to be fuel efficient. Because they kill people when things go wrong, they have to be safe. Regulations came into force. Significantly, lots of different designs and techniques were tried out to find the best way to build an airplane for a particular task and what we have today is a variety of flying machines that are efficient in their roles and safer than driving to the airport. Essentially modern airplanes are designed using precise models of flight based totally on a well-researched understanding of the physics of the process. You can buy books that explain it.

Aviation products aren’t aimed at the consumer; there are people in aviation who actually understand the technology. The consumer doesn’t understand technology, nor do politicians or lawyers for that matter. Whenever there is ignorance, something undesirable usually emerges to exploit it; in this case it is marketing. Whilst marketing may have had some altruistic roots, today it means selling more product than you deserve by misleading purchasers. The most powerful tool marketing has is to simplify something complex down to a single parameter and then suggest that more of that is better. In the case of television it started with line count and progressed to pixel count. Horizontal pixel count, of course, because that gives a bigger number. More is better and forget anything else, such as motion portrayal.

Whenever there is ignorance, something undesirable usually emerges to exploit it; in this case it is marketing.

If we compare television technology with aviation, then immediately we see that if there is something wrong with a TV picture nobody dies, so there’s no regulation. There’s no Federal Video Authority to slap a fine on you if you broadcast a fuzzy juddering mess when anything moves.

Compare television with audio. Starting with scratchy cylinders, AM radio and discs, the bandwidth and quality steadily rose until FM radio and the Compact Disc came along and blew everything away. They were stunning successes, but possibly more significantly represented a pinnacle in many respects. Digital Audio Broadcasting (DAB) in Europe sounds worse than FM because of compression. CD has seen off all subsequent audio media, like MiniDisc, that obviously sounded worse and SACD that didn’t sound any better. Internet down-loads have taken their toll, but thirty odd years later it’s still selling. The consumer hasn’t a clue how CD works, but was able to judge the final result was good.

The 405 line TV system that began broadcasting in the UK in 1936 had a picture rate of 50Hz. The 625 line system which followed had a picture rate of 50Hz. When PAL colour TV was introduced, the picture rate was 50Hz. When digital TV came along, the frame rate didn’t change and the quality for a lot of people went down because of compression. When HDTV was introduced, the picture rate remained 50Hz. Guess what it will be for 4K? For U.S. just substitute 60 for 50 and you will get the picture, provided nothing moves.

Will all the buzz about 4K TV sets become mute when people realize the promise of improved image quality was not entirely true?

Will all the buzz about 4K TV sets become mute when people realize the promise of improved image quality was not entirely true?

At what point did the consumer get to choose which frame rate he or she thought was best? The overwhelming majority of the world’s population has only ever seen 24/48Hz movies or 50/60Hz TV, doesn’t know that anything better is possible and has certainly never seen an alternative. Once upon a time, when dinosaurs roamed the earth, there were reasons why TV picture rates should be locked to the local AC supply. None of those reasons are still relevant. The only remaining reasons are inertia and tradition; committee groupthink. Now that the golden days of TV are over and it faces increasing competition from other forms of entertainment, perhaps the time has come to let the consumer choose the frame rate as the movies are finally doing.

Pick a frame rate

But what should the frame rate be? That’s like asking how fast an airplane should fly. It depends what you want it for. The best way of answering is not to dictate some value which could never be universally applicable, but instead to look at the underlying mechanism to show how it is possible to work out the best frame rate for a given task. That process of optimisation is found in all technologies.

In so many cases there is a pair of opposing mechanisms and we have to find some balance between them.

Take airplanes; the drag they suffer is the sum of profile drag and induced drag. The former goes up with speed and the latter goes down. The speed to fly at for the least power is the speed where they do equal damage, so neither of them, by definition, is making things worse. Sailplanes fly at a sweet spot airspeed that puts them in this so-called drag bucket when they want to stay aloft the longest time.

Take piston engines. Run them too slowly and the heat of combustion has more time to be absorbed through the structure so less heat expands the gases. Run them too fast and the friction gets you. Guess what? There’s a sweet spot piston speed where the losses are equal. That’s where the best gas mileage will be found in everything from sub-compacts to super tankers.

Television formats that lock to the local power frequency are like sailplanes or engines going too slowly. 405 lines was the closest we ever got to the sweet spot, or NTSC’ 525 lines in USA. HDTV and 4K both took us further away.

Understand the trade offs

The opposing factors in TV are extremely simple. Movement during the frame exposure time causes loss of resolution. Reducing the number of pixels in the frame causes loss of resolution. For a given motion speed, measured in seconds per picture width, for example, the best picture as seen by a tracking eye will be the one in which the resolution loss due to both effects is the same, for the simple reason that neither is worse than the other.

We can reduce the loss of resolution due to motion by shortening the exposure time, but as we saw in an earlier piece, the result is strobing. The only way we can control motion-induced resolution loss without side effects is to increase the frame rate.

Starting with a fixed pixel rate as a given, the question becomes one of how often to divide it into frames. The more frames we have, the fewer pixels we can have in each frame and vice-versa. Right now we are on the low frame rate side of the sweet spot, but it would be just as dumb to select some fantastically high rate that left only a handful of pixels per frame.

The only complication is that the relationship isn’t linear. Pixels per frame goes as the square of static resolution, whereas motion portrayal is linear with frame rate. That is why the curve of Figure 1 isn’t symmetrical. What this means is that if the number of lines in a format was reduced to 70 percent and the number of pixels in the line was reduced by the same factor, the number of pixels in the frame would be about halved as 70 percent of 70 percent is 49 percent. That would allow the frame rate to be doubled with no change of pixel rate.

This figure assumes a constant pixel rate which is that of a 3840 x 2160 (4K) picture at 60 frames per second. This rate can be divided into various frame rates, shown along the horizontal axis. The effective resolution of an object moving at four seconds per picture width is shown on the vertical axis. 4K/60 loses heavily because it suffers serious motion smear. 720P/540 is slightly beyond the optimum because its negligible smear doesn’t compensate for the reduced line count. Best performance for the bit rate is achieved at 880P/360.

This figure assumes a constant pixel rate which is that of a 3840 x 2160 (4K) picture at 60 frames per second. This rate can be divided into various frame rates, shown along the horizontal axis. The effective resolution of an object moving at four seconds per picture width is shown on the vertical axis. 4K/60 loses heavily because it suffers serious motion smear. 720P/540 is slightly beyond the optimum because its negligible smear doesn’t compensate for the reduced line count. Best performance for the bit rate is achieved at 880P/360.

Looked at another way, with the same pixel rate as 4K/60, 720P could have a frame rate of 540Hz with one third of the static resolution, or 1080P could have a frame rate of 240Hz with one half of the static resolution.

Let’s assume a moderate motion speed of 4 seconds per picture width and a 180 degree shutter. At 60Hz there are 240 frames in 4 seconds, so in one frame period the motion smears by the horizontal pixel count divided by 240 and then by 2 again to allow for the shutter angle. For a 4K picture 1 pixel is smeared to 9 pixels, giving 426 pixel per line performance.

It is clear that for the same bit rate, 1080P/240 gives pictures over twice as sharp as 4K/60.

Suppose we kept the same pixel rate but went for 1080P/240. Now there are 960 frames in 4 seconds, so the smear will be 1920 divided by 960 and further by 2. One pixel is smeared to two pixels, so the resolution is merely halved, giving 960 pixels per line performance. It is clear that for the same bit rate, 1080P/240 gives pictures over twice as sharp as 4K/60.

Figure 1 shows the results of repeating these calculations for different frame rates at constant bit rate. There’s a sweet spot around 360Hz which corresponds to a picture of 885 x 1570 pixels.

The dynamic resolution is about 2.2 times that of 4K/60. The function is quite flat topped, so being to the right of the very top wouldn’t hurt a lot and would cater for faster motion. 

You might also like...

Designing IP Broadcast Systems: Integrating Cloud Infrastructure

Connecting on-prem broadcast infrastructures to the public cloud leads to a hybrid system which requires reliable secure high value media exchange and delivery.

Shooting Apple TV Series ‘Constellation’ With Cinematographer Markus Förderer

We discuss the challenges of shooting the northern lights in the winter dusk and within the confines of a recreated International Space Station with cinematographer Markus Förderer.

Designing IP Broadcast Systems: Where Broadcast Meets IT

Broadcast and IT engineers have historically approached their professions from two different places, but as technology is more reliable, they are moving closer.

Audio At NAB 2024

The 2024 NAB Show will see the big names in audio production embrace and help to drive forward the next generation of software centric distributed production workflows and join the ‘cloud’ revolution. Exciting times for broadcast audio.

SD/HD/UHD & SDR/HDR Video Workflows At NAB 2024

Here is our run down of some of the technology at the 2024 NAB Show that eases the burden of achieving effective workflows that simultaneously support multiple production and delivery video formats.