When evaluating a video or film camera, the professional tester will usually resort to shooting specific charts such as color charts, grayscales, and resolution charts. The footage taken from these charts can be used to evaluate all sorts of things about the imaging capabilities of cameras, and they provide an absolute reference point for being able to compare one camera against another.

The problem, as I see it, is: not all that many people know what the charts are telling them! When a chart is posted, it's only relevant if the person reading it knows what they're looking at, what they're looking for, and what it all means. There's a lot that can be learned from a chart, just like there's a lot of information that can be learned from books, but if the person hasn't been taught how to read, the book won't do them much good. And if they haven't been taught how to read a chart, the chart won't help them either.

We use charts because they're designed by technicians to reveal how the imaging system works, and also will present scenarios that could reveal flaws in the imaging system. Charts are the most useful way of evaluating the performance of a specific camera, and they're especially useful when comparing one camera against another, because the chart is a specific repeatable reference point – anyone with a Chroma Du Monde chart should be able to get the exact same results as anyone else with that same type of Chroma Du Monde chart, whereas when setting up a “still life” to shoot, there's pretty much no way for anyone else to set up the exact same still life you set up, for example.

Not everyone seems appreciative of chart testing (perhaps the understatement of the week). When a chart reveals flaws in one imaging system or another, we will frequently see people make cutesy comments such as “wow, it's a good thing I'm shooting people and not resolution charts!” or “Oh, darn, there goes my plans for making “Resolution Chart: The Movie.” The intent behind such comments is to discredit chart testing as irrelevant, when in fact chart testing is highly relevant. Chart testing shows you what you can expect, and reveals what the camera is truly capable of imaging, and points towards where problem spots might show up.

People are sometimes quick to dismiss what they don't understand (or, worse, sometimes they actively turn a blind eye because they don't want to hear the results!) It's not that hard to understand what the charts are telling you, and once you know how to read and interpret one, you can make decisions about the suitability of the camera you're considering and how it will work for your intended uses.

In some ways, chart testing is akin to testing your drinking water for bacteria, or testing your house for Radon gas or toxic mold. If you were about to take a swig of some luscious cold lake water, and a scientist said “er, you might want to boil that first,” you'd want to know why, right? There could be some nasty germs in there, and you'd want to know about it before you came down with dysentery! Or, with a house: you might see a gorgeous and beautiful house, for a very low price, and think “Great! Let's live there!” and the real estate agent might say “well, you should know, sir, that this house has been tested and shown to be heavily contaminated with mold.” At this point, you have three choices:

A) you can say "well, can it be fixed?"

B) you could say "well, forget it, I don't want the health problems that come from a house with mold"

or
C) you could say "Mold? I don't see any mold. You're just trying to keep me from getting a great deal on a house. What, you want to buy this for yourself, don't you? Mold, hah! Once I paint these walls you wont see any mold at all!"

Now, clearly, one of those responses can lead to dire consequences to your health. So, also, can claiming “I don't care how this camera performs on a chart, because I don't shoot charts, I shoot movies!” have a similarly consequential impact on your footage.

Sometimes it seems like people feel confident in dismissing chart results because they think that lab testing is one thing, but real-world footage is something entirely different. Sorry, but that's just not the case. What happens in the lab will also happen in the real world. There's not some magical “res chart mode” that cameras go into when looking at a chart; it's not like the camera will automatically say “hmm, I seem to be pointed at a res chart, so now I will go into the 'make my footage suck' mode.” And then, when you take it outside, the camera doesn't then go “ah, I see that I am shooting images of people, so now I will make the footage not suck.” What happens in the chart will happen in the real world too. If the chart shows that the camera has problems with thin line detail, then in the real world you may encounter problems when you shoot venetian blinds, picket fences, brick walls, knit shirts, chain-link fences, newspapers, overhead power lines, rippling waves, vinyl siding on houses, striped shirts, door frames and window frames, an interview with someone wearing glasses, a piano keyboard, or anything else that presents thin lines!

Similarly, if the chart shows that your camera introduces color fringing on sharp detail transitions, you can expect that to show up in your real-world footage too, including rainbow moire patterns, speckles and spatters of color throughout, complications with greenscreen keying, or something as insidious as turning a blonde person's hair to somewhat of a redder shade than it actually is.

The charts can also be used to evaluate lenses, for chromatic aberration, for edge sharpness, for overall sharpness, for seeing how different apertures affect the sharpness of the image, etc.

Simply put, there is no “chart mode” when shooting footage. What happens on the chart, will also happen in the real world. If the chart shows aliasing, the camera will have aliasing in the real world. If the chart shows the lens produces soft corners, the lens is going to have soft corners in the real world too. So it's best to acknowledge what happens, learn how to work around it, or learn how to choose which camera will work best for the scenario you have in mind (and also to know what scenarios your chosen camera excels at).

Where do you get charts?

For professional use, I prefer the charts made by DSC Labs. They make the best charts in the world, in a huge variety. My personal chart arsenal includes:

the CamBook 6,

the Chroma Du Monde Billups VF/X 1

the MegaTrumpets 12

and the ToNi

These DSC charts are not inexpensive, ranging from about $400 on up to $1550 or more. They are professional tools designed for professional evaluation. I also use the Stouffer 4110C 41-step wedge for testing dynamic range; this small strip is relatively quite inexpensive, at about $125.

How To Print A Chart

If you don't have a professional chart, there are some freely available downloadable test chart patterns that you can print. DSC Labs has a backfocus star chart you can print, but I think the best collection online is perhaps John Beale's collection of links to some free downloadable charts: http://www.bealecorner.org/red/test-patterns/

Perhaps the most useful of these charts for simple camera and lens evaluation is the PDF version of the ISO 12233. That chart looks like this:

A true, purchased version of the ISO 12233 would be best, of course, as it will be printed at the highest photographic resolution, on photographic paper. You can buy one here, for CHF 124 (124 Swiss Francs)

But, assuming you want to print your own, you can download the one from the links above and print it. What you'd want to do is print a chart at the absolute highest resolution and largest size that you can. I'd recommend using the 12233 and spreading it out across four sheets of paper. Yes it's a hassle to slice the edges and tape it back together, but printing at the larger size will make the chart a lot more accurate and remove the printer's resolution out of the equation. Especially with an inkjet printer, where the fine dots of ink might leak or splatter, that will lead to contamination of the chart and inaccurate results. Even laser printers (which frequently boast 600x600 DPI, or 1200x1200 DPI) don't necessarily deliver adequate resolution on the printed page to give you the best results. Print on photographic paper if you can, as that helps rein in spill and such, and print it at the biggest size that you possibly can. Spread it out among multiple sheets of paper and then cut off the edges and tape the chart together so that you have the biggest, finest-printed chart you can get. You can also download the file and take it to a print shop and have them print it on a larger sheet of paper; many office supply stores have in-house print shops that might do this job for a very reasonable charge.

How Do You Properly Shoot A Resolution Chart?

Once you've printed it out, you should ideally mount it on a firm surface, such as a sheet of foam core. But, if it's for one-time use, you can also just tape it up against a wall. Just make sure that it's completely flat.

Then, for framing up the shot, by all means use a tripod! Frame the camera up perfectly square to the chart; raise the camera on the tripod until it's looking directly level at the chart. You don't want the camera tilted up or down at all, you want the center of the lens to be at the same height as the center of the chart, and you want to have the camera perfectly squared to the chart so that the camera isn't pointing off to the left or to the right at all. The plane of the camera back should be perfectly parallel to the plane of the chart.

Once you've squared up the camera to the chart, you want to illuminate it flatly and evenly. Overhead fluos or daylight can be good for that; it's not so easy to get it flat & even by using tungsten fresnels or other hard lights that will have hotspots. A big softbox would be a good way to light a chart; put it right behind the camera, higher, and tilted down, pointing at the chart, and you should get some nice even flat illumination. This isn't so crucial for a resolution chart but when testing color charts it becomes very important.

Once the chart is flatly lit and the camera is aimed at it, you then want to frame the chart according to the arrows that are all around it. You want the arrowheads to converge exactly on the frame lines. You don't want to see any gaps between the inner arrows and the edge of your frame, and you don't want to see any of the outer arrows. The arrowheads should line up so that they're perfectly pointing at the edges of the frame. Ideally, it should look like this:

If the arrowheads are cut off, then you're framed too close and your resolution numbers will be wrong, they'll be too generous. If the arrowheads are fully visible and don't run right up to the edge of the frame, then you're framed too far away, and your resolution numbers will be wrong, they'll be too low. So try to frame it as absolutely squarely and perfectly as you can. And this isn't easy with an SLR, because every time you change the zoom, the focus changes a little... and if your lens suffers from “breathing,” that means that every time you change the focus, the lens framing will slightly zoom in or out (even on a prime!) So it's a tricky dance to get perfect focus and perfect framing. Focus is more important than framing, so if you can only get the frame "very close" but you can get the focus "perfect" and you're tired of fiddling with it, well, that might be good enough.

To focus on the chart, you can usually use autofocus, but if manually focusing I prefer to open the lens all the way up (f/1.4 or as wide-open as your lens will go) and pull focus so that the chart is absolutely as crisp as possible. Then, I like to stop down a couple of stops, so that the depth of field will grow deeper and ensure that the chart is perfectly crisply focused. If the focus is “off” on your chart, there's really no reason to go shooting it; the capabilities of the camera will only be known if the focus is perfect. Of course, if you're testing lens performance you'll want to test at a variety of apertures, so you may not always be able to stop the lens down again; in cases like that, just do your best.

How Do I Read A Resolution Chart?

When you're reading a chart, you don't go looking for the smallest area where you can make lines out, you look for the point at which the camera quits representing the chart accurately. And this is an absolutely key point – you don't want to just get an image in the camera that looks “good,” you need it to look RIGHT. Accuracy is paramount when evaluating a chart; in fact, it's really when the camera fails to accurately reproduce the chart that we learn the most information. Here is an example of a wedge from the MegaTrumpets 12 chart:

Now, that may look nice and sharp and crisp, and you might even try to interpret that as showing up to 1700 lines of resolution, but – you'd be wrong. Completely wrong. Because the actual chart looks nothing like that! Here's what it's supposed to look like:

Now you can see there should have been straight lines throughout, and you can see just how wrong the camera got it in the prior example. So what we're looking for is the point at which the camera's picture no longer looks like an accurate representation of what the chart actually was.

Here, let's use this chart as an example. This was shot on a camera in 720p mode. I've annotated it with red lettering at certain points so you can follow along as we examine this chart and look for the information it gives us. On this chart, each number you see represents 100 lines of resolution, so when you see “1” that means the line pattern next to that “1” is drawn at 100 lines of resolution; when you see “3” that means the line pattern next to the “3” is drawn with 300 lines of resolution, etc.

Spot A: Look at the diagonal lines that are about 2/3 of the way over on the right, the vertical column of 1/3/5/7/9/10/8/6/4/2. Those should be a patch of diagonal lines. Each block of lines should look basically like this:

(unfortunately my printer cheaped out and rendered a big ol' white line down the middle, so ... ignore that, please... and yeah, that's another reason to go for pro charts!)

Anyway, now that we know what the diagonal blocks should look like, let's look at the big chart and see how it looks. In the “A” column, it looks like blocks “1” and “2” are the only ones that are accurately rendered. “3” is okay-ish, you can clearly see that there's aliasing happening but it's at least still a rendering where the black and white lines are cleanly separated. In the "4" box you can see the aliasing pattern clearly; where the lines are supposed to be going from upper-left to lower-right, there's a cross-pattern being introduced that makes the whole box look like there's a second, much steeper diagonal pattern running through it. That is not an accurate representation. The camera did not successfully represent what the chart looks like in the “4” box. “5” is completely gone, as are 6, 7, 8, 9, and 10. And “8” and “9” are dangerous ones, because they almost look like they're cleanly rendering, don't they? But they're not right at all... if you compare against the original you'll see the difference. In the original file, in box “8”, there are 23 very thin lines, slanting down from left to right. But in this resolution chart, the camera rendered box “8” as 10 thick lines, slanting down from right to left! That's completely unacceptable and entirely wrong! That's why it's important to know what the original looks like, so you know whether your camera has accurately reproduced the original. Someone who didn't know how to read a res chart might look at box “8” and say “hey, that looks cleanly rendered, so there's at least 800 lines of resolution here.” But that would be a very wrong interpretation. Look at how the “8” box should look, and look at how the camera failed in its attempt to recreate it:

So to read the chart properly, we have to go back and look at the first point of failure, and that was on box “4”. The camera almost succeeded in representing box “3,” but failed on “4,” so we know the real resolution being seen here is somewhere around 300, well below 400. If the resolution was over 400 lines, box “4” should have been cleanly represented, and it wasn't. So we know that the vertical resolution of this camera is somewhere between 300 to 400 lines.

Spot B: look at the row of angled mostly-vertical lines along the bottom. The boxes with a slight lean, from upper-right down to lower-left. Here the last box you can see that looks basically "right" is the "4" box. The "5" looks funky, and "6 is clearly fuzzed (can something be "clearly" fuzzed?) In this case, it indicates a true horizontal resolution of somewhere below 500.

Spot C: In the segment of pure horizontal lines (which measure vertical resolution), you should see a uniform pattern of distinct lines. In this chart, it looks like in the horizontal lines (the big vertical column about 1/3 of the way from the left) the camera rendered the “2” block of lines solidly, “3” is pushing it, and by the time we hit “4” there's serious misrepresentation of the chart. It definitely looks like there's some thicker and thinner lines in that patch, and that's not what it's supposed to look like. Again, referencing the original, each patch should be perfectly equally spaced lines, like this:

Based on the clean rendering of the “2”, and the somewhat iffy representation of the “3”, I'd call the vertical resolution in this section about 300 lines.

Spot D: In the pure vertical lines (which is the row of boxes at the bottom of the chart, which measure horizontal resolution) you should again see a clearly delineated series of equal-width lines, alternating between black and white, all of the same thickness. We can see that the “3” section is cleanly rendered, and “4” is acceptable, but “5” starts to introduce some alternatingly thicker lines, and that's not what the chart is supposed to look like. Based on that, it starts to look a little "off" at about 420; and by 550 it's clearly not accurately representing the chart anymore. From this, we could estimate the horiztonal resolution at about 420 lines.

Spot E: Moving on to the resolution trumpets -- these are the wedges in the center and corners of the charts. What you want to see there is a smooth flow of lines with no oddness, as they gradually converge. And you're looking for the point where the camera has not accurately represented what the chart looks like. It's not a point of looking to see where you can distinguish any difference from black to white, you're looking for accurate representation. And based on that standard, I'd say the horizontal line trumpets (in the center of the chart) are showing cleanly up to about 200, with some fuzziness kicking in at about 250, and a failure to accurately represent the lines at about 450. The lines are very stair-steppy starting at about 200, and it's really kind of dodgy thereafter, but the lines are still distinct and technically resolved accurately up until about the 450 point. It's hard to see where the point is where it first fails to accurately represent the chart, but my guess is, once again, about 450. If you look at the horizontal line trumpet in the upper right corner it's a lot easier to see that it can't accurately represent the chart at 450.

Spot F: The vertical trumpets of interest are the ones just left of the center. The camera fails to accurately represent the one on the right (that goes from 600 to 2000) so we'll ignore that and look at the one on the left. The camera resolves from 100 on down to about 450; after that it's a little sketchy as to whether it's successfully resolving or not. It has definitely failed by 550; it's hard to distinguish what's a black line and what's white by that point. I'd say on this chart the point of maximum resolution is no more than 550.

One trick that I learned from Adam Wilt, which is incredibly useful for determining the point at which a camera fails to accurately resolve on a chart, is to very gently, very very mildly wiggle the camera while it's shooting the chart. Resolved detail will stay solid, but fake aliased detail will “move” in the opposite direction. Here's an example of a moving res chart, where you can really see the difference between cleanly resolved detail, and spurious aliasing (the aliasing is at the bottom portion and moves in the wrong direction).

How Else Might A Camera Fail To Represent The Chart Properly?

Finally, let's look at this chart – and notice some really horrible aspects to it. Look at the big block of green in the “5” and “6” block, and the purple. Ugly, yes, but also completely wrong! This chart was printed on my black & white printer – the results should have been entirely black and white. Anywhere that you see color at all, that's false. That was an artifact introduced by aliasing, and a failure of the camera to reproduce the chart accurately. That is simply not an accurate reproduction of the black and white chart. Anywhere you see orange or purple on that chart, that's an artifact coming from the camera's aliasing. And that's a problem, because that means the camera can introduce color artifacting in your images. It would probably be quite challenging to pull a competent chroma key on fine detail from this camera, and it wouldn't be surprising at all to see this purple/green artifacting contaminate a real-world scene. Think of fine detail like this – you might see it in a picket fence, but instead of a white picket fence, it'd be purple and green. Or think of the fine detail of an actress's blonde hair – it's possible you might see green or purple showing up in her hair!

The moral of the story is: if it happens in the charts, it will happen in the real world too. Know how to read and interpret charts and you'll be well on your way to understanding your camera better, and knowing how to deal with the unexpected when it crops up in your “real world” shoots.