Three formats, how best to test - help me Barry and others

capt chuck

capt chuck

Well-known member
I specifically asked Barry because I know he's done a lot of testing... I know plenty of others have too but I haven't ever but am now planning on it.

I've got an FS700 on pre-order and I own a KiPro (the original version). I want to compare the following when it arrives:

- The native codec
- 4:2:2 8 bit sdi into KiPro
- analog component output digitized in the KiPro at 4:2:2 10 bit

As I mentioned I haven't done such tests before so how best should I approach it. Would I need to try all sorts of lighting conditions? That is would it be possible for one configuration to be best in condition A while another outperforms in condition B? I'm guessing yes, and I'm guessing yes for testing at different resolutions as well. Is it important to use a resolution chart (don't currently have one), or would real world subject matter work?

Thanks,
 
To me the toughest test for Codecs is a scene with a lot of detail that is moving and subtle gradients.

Examples are trees with leaves blowing in the wind in sharp focus with trees fairly far away so there is lots of fine detail moving.

Water with small waves all even leaden color.

Arc welding or grinding with lots of sparks flying.

Swish pans.

A very gradual grey gradient. I use a computer monitor that is off and i light it with a hard light from about 6' away and just shoot the resulting gradient, this will show banding very clearly.

Test charts and soft focus shots like talking heads with bg soft focus is very easy for codecs to deal with, so try the more challenging shots I described above.

The ultimate test is rally not charts, it is what you normally shoot that really counts. So just shooting what you normally do is really the ultimate test.
 
I've tested Sony's AVCHD before, and it's quite good, it's on par with Panasonic's AVCCAM implementation. Sometimes each does better than the other, but overall they perform comparably and really very well considering the miniscule bitrate.

For testing codec performance, resolution is one parameter to test, color resolution would be another, and gradient handling would be another, and long-GoP robustness would be yet another.

Testing resolution is best performed with a resolution chart, rather than "real world", because in order to get enough detail to really matter (such as a field of blades of grass and leaves and whatnot) that's not going to be a repeatable scenario, as the wind and the lighting will change. But, if you can test all three recorders simultaneously, then that becomes a non-issue, and that'd be the ideal way to do it. That's how I was able to compare NXCAM against AVCCAM; I used an NXCAM camera recording to its own cards and an external SDI AVCCAM recorder, and matched the frames up based on SDI timecode, and could see how both codecs handled the exact same scenarios in the exact same frames. Because you're talking about three codecs from three different outputs to three different recorders, you could probably do the same (and should, if at all possible). If you can, then yes, real-world will suffice fine.

For color resolution, again a proper chart is best, and the DSC "Wringer" is perfect for that, but testing can be handled with real-world stuff; use the most saturated and pure red, blue, and green you can get. Lay them at cross angles to each other, and film them at various distances. The difference between 4:2:0 onboard and 4:2:2 external should be pretty obvious in such a case. A live greenscreen with a red-haired subject is a pretty good test for codec chroma resolution.

For gradients, that one's easy -- go in a dark room with white walls, shine a bare bulb on one side of the room, and film the wall as the light drops off from bright to dark. That should give you a good smooth even gradient and lots of opportunity for testing gradient banding.

For codec degradation, that's where you're trying to overwhelm the codec to see how it falls apart, the easiest test for that is to shoot a billion tiny leaves and a billion blades of grass all blowing slightly in a soft breeze, set the shutter speed on 1/1000, and rotate the camera while you're filming it or snap-zoom in and out while filming (rotating and zooming defeat most motion-prediction algorithms). This is where you should really see a big jump between the AVCHD and the intraframe ProRes codecs. Another thing that can cause codec degradation is a waterfall or fast-moving river, where there's lots of glints and sparkles all over the water. Or confetti, man, long-GoP codecs *hate* confetti like you get at the end of the talent competition shows, where lights are flashing and confetti is falling and there's smoke in the air... unpredictable events with lots of tiny detail and unpredictable motion give long-GoP codecs fits, and are the type of scenarios where an intraframe codec will prove definitely superior.
 
Thanks olof and Barry for the very helpful information.

I'll be able to test the internal codec and either the sdi or the analog simultaneously, but not the sdi against the analog capture since it's really 1 external recorder in two different modes vs. 2 recorders. So I think I'll shoot for a bit of both the real world tests and the more repeatable so I can be assured I'm comparing apples to apples when comparing the two different input configurations on the kiPro.

My going in assumption (though I'm trying to keep open minded) is that the internal codec will be fine for most all of what I do (mostly event and documentary) and even when I see some artifacts the client may not. I do think it will be valuable to know the limits, however, so I can use another tool when it is called for.
 
I would guess that your observations will match up exactly with your guesses. That's pretty much what we've found with the AF100, that the internal codec is just fine for most uses, but when you need that extra bit of oomph it's good to know what an external recorder can deliver. I think the FS700's internal will also prove to be fine for most uses, and that the need for an external will be primarily for greenscreen or when shooting high motion in very detailed scenarios -- a football game with 100,000 heads in the crowd and a million blades of grass, for example, might be one place where the internal might choke where the external would excel, but then again, as Olof said, shallow DOF makes the job a lot easier on the codec so, if any of the frame was soft-focus, the codec would probably be able to handle it fine. It's only really-wide, super-sharp, highly detailed scenes where the limitations of AVCHD become more noticeable.
 
another (and much more common) place where the codec's fall apart is pulling secondaries, there the compression artifacts can limit what can be done, and the limits are far too easly reached

d
 
I've keyed green screens many times, but I don't have any experience with pulling secondaries so I doubt this would be a test I could execute successfully. It is good to know and may research it a bit to learn more.
 
You must log in or register to reply here.
Back
Top