Ebrahim Saadawi
Carbonite Member
Often we see camera releases, where the product gets complete acceptance or refusal based on the specification list provided alongside the announcement. The camera is evaluated entirely as good or bad purely based on that list. In fact, it's not only on release of new product, it's a practice conducted throughout the product's life, where people compare it to other cameras based on the specification list, even make full reviews and conclusions.
Is it possible to evaluate a product that way? That's the question we're trying to answer. The only way to do so is to look at
1-Whether spec sheet provide enough data
2-The examples in previous and current product quality vs. their spec sheets.
Specification sheets of cameras provide following information
1 -Resolution, stated in a number of 1080p, 4K, etc.
Is that number indicative of the camera's resolution/sharpness? The answer is no. The spec data about resolution merely states the number pixels in the final file. Of course we all know that's not an indicative of final image resolution. Canon DSLRs have a spec. of 1080p, while Canon C300 has a spec resolution on 1080p, yet the difference in resolution is absolutely enormous. The reason for that is, there are many other elements that determine the image resolution that are not mentioned in the spec sheet resolution section. These are,
-how many sensor pixels is that image derived from? Is it derived from the same number of pixels without down-sampling? In this case we know as a Bayer sensor this means the actual resolution is 60% of that number stated in the spec.
Or is it derived from a higher resolution part of a sensor? If it is then the second element that determines resolution is
-The down-scaling method. When the image is derived from a larger number of pixels, how is it actually down-sampled? Is it line-skipping rows of the sensor? is it pixel-binning? is it reading the entire area and down-sampling, and if so, what algorithm is used, Cubic, lanczos, a bad one that results in lowered resolution and aliasing artifacts?
-The lens resolving power: in case of fixed lens cameras, or interchangeable cameras when the user intends to use a specific lens.
Therefore based on that, a resolution of a video cameras cannot be evaluated reading the spec sheet, as it simply does not provide the needed data to do so. The 1080p, 4K, number in the spec sheet is meaningless and does not determine how the resolution of the camera is going to be like.
2- Low-light performance: only stated as minimum and maximum ISO values and not the noise amount or noise pattern at different ISOs, making this section completely useless.
3- Dynamic range: is always in fact not stated at all, but sometimes it is, in a number of stops. The dynamic range number hugely varies among tester and users, and has various definitions, for examples a company can measure the DR of a camera as the number of stops between the points where the camera clips at both ends, yet this does not mean anything to the end user as the quality of the areas near both ends are not stated. The shadow areas can contain detail (not clip) but have significant amount of noise and compression artifacts making a significant decrease in the final DR of the camera to the operator. Plus the highlight roll-off and the shadow roll off are not stated. Making the DR number in the spec sheet nearly a meaningless indication of the final range the camera will be able to output.
4- Colour performance: only stated in bit depth and chroma sub-sampling. In fact, the most important aspect in cameras' colour rendition is the colour science. It's a processing method, a complicated number of code lines that change and shift how each colour seen by the camera is represented in the final image. Each company, and each camera within each company can have a different colour rendition, something not stated in any spec sheets, making it impossible to judge how pleasing or un-pleasing the colours produced by the camera are.
5- Lens: In fixed lens cameras and in lens specifications of a separate lens.
The stated specs are merely the following, the focal length, the aperture values, and the size and weight of the lens. However, nothing about the lens optical performance is stated. A lens performance is a very complicated subject and it's silly one could evaluate it using the focal length and aperture values. These ''only'' state what the zoom range is, and the DOF possible. Nothing else is stated. How the resolution is, how the resolution is at different focal lengths, how the resolution is in the corners vs center, how the out-of-focus rendition looks, how much chromatic aberrations exist, colour fringing, focus breathing, Coma, Contrast, peripheral shading, nothing is mentioned. In fact, even the focal lengths mentioned are not entirely accurate on most lenses, and even the aperture values do not indicate the actual brightness (T value). Therefore a lens simply cannot be evaluated based on its spec list. The Canon 50mm F/1.4 is on specs an identical lens to the zeiss Otus 55mm F/1.4. An example, The 24-240mm F/2.8 lens on the RX10 is only better than the XC10 24-300mm F/3.5-5.6 in aperture value (DOF), yet everything else (about a dozen factors in lenses performance) about the lenses are not known and must be evaluated first, yet people on announcement crowned the 24-240mm F/2.8 the ''better'' lens ultimately, not knowing it could be a much better or a much worse lens. Specs simply don't give enough data, actually in case of lenses, almost none.
6- Codec, is always stated next, as H.264, ProRes, along with the bitrate and bitdepth and chroma subsampling rate.
That is accurate and it does state the compression method used in the camera, but what's important to know is that it does not indicate in anyway how the quality of the image and codec are going to be. That's determined by the implementation of the actual codec in the specific camera, and the compression quality is determined by many other elements not stated in the spec list, such as the actual resolution going into the codec, the downsampling method, the processor performance, the colour dye and colour science within the camera, and the final image enhancing algorithms. Prominent examples are a camera like the Canon C100 which has a 24mbps H.264 4:2:0 8bit codec, yet has a strikingly better compression performance than a 550D with an identical on paper 24mbps 4:2:0 8bit H.264 codec. Therefore the codec numbers on the spec sheet mean very little to almost none in the final quality of the image and even in the final compression performance, as in how hard the image can be pushed in post. The elements that do mean something are simply not stated in specification lists.
7- Frame rates, are always stated next
Yes they do give an accurate representation of the camera's maximum frame rate ability. But just that, all the other more important data about that mode are not stated. The downsides implemented to achieve these higher frame rates are not mentioned, for most cameras they achieve it through skipping lines, which results it horrible aliasing and a significant resolution less at higher frame rates, even on high end cinema cameras. Therefore, when the spec sheet mentions a camera can do 1080p at 25p and 50p, it's true, but whether the 50p mode is usable in quality is an entirely different subject again, not stated in the spec sheet. So it's not the frame rate number is not indicative of the camera's capability as a slow-motion shooter.
8- Written next can be the Audio specs.
What's mentioned is the format, hz number, the type of input/output. However, the most important elements are not stated. The type and quality of the Pre-amps in the camera, the actual sound quality, whether the audio can be controlled manually or not, therefore one simply cannot get any indication of the camera's capability to record sound based on the spec sheet, it's almost a useless section of the sheet.
9- Battery:
Is mentioned in the spec list as the product name, as an LP-E6n or an XD-T62 or a WTF-876 and sometimes it mentions a capacity number. However, the spec list does not mention the power draw of the camera whilst shooting video, it doesn't give any indications whatsoever on how good or bad the battery life in the camera is.
10- There are many very vital elements in cameras that are not mentioned altogether
-Aliasing performance
-Moire performance
-Rolling shutter amount
-Highlight roll-off
-Resolution & quality of crop modes
-LCD & EVF quality and performance under day light
-The menu system speed
-The menu system user interface
-The responsiveness of buttons and dials
-The feel of the camera in hand
-The ergonomics of the handling of the camera
-The heat management
-The durability under various conditions
-The materials used in the built
11- The elements that ARE accurate and mentioned in spec sheets are actually:
-The size of the camera
-The size of the sensor
-The weight
-The media slots
-The LCD size
-The total number of pixels on the sensor
-The price of the camera
That's it.
If you want to review a camera based on that be my guest it's fine,
but it's just a physical impossibility to know whether this camera is ''good'' or ''bad'' when you have absolutely no idea about the 11 elements of image quality it produces, not even one, including resolution.
This was the answer to the question about whether spec sheets provides enough data and the answer is a clear no. And this is actually true regarding all products, from phones to TVs to Cars and air-jets!
Next time there's a new announcement, please think about the meaning of the represented data and whether they can be used to evaluate the camera. A camera is way more than just a sheet from the maker, that sheet only presents about 10% of the quality of the final product. Please, don't buy a camera based on specs, but it based on the quality of the camera's image to you, and the quality of the design, nothing else, and this data can only be obtained by using and testing the camera heavily, or by reading the review of someone you trust who does that.
Let's look at a recent spec-impressive announcement which we all crowned a wonderful camera just based on the list, The Sony A7RII.
We're excited about the 4K resolution, but do we know how much resolution it's going to actually resolve?
in FF or s35 modes?
Do we know whether it has deadly aliasing/moire or not?
do we know if it will have deal-braking rolling shutter?
do we know if it will record more than 10 minutes without over heating?
Do we know how the low-light performance is?
Do we have any information of the colour science?
on the amount of DR?
on the Highlight roll-off?
on the noise pattern and texture?
do we know if the battery is going to last more than half an hour with a 42mp 4K sensor?
Do we know if the 5 axis stabilization will give acceptable results in video mode?
Do we know if it won't have unacceptable audio quality?
Do we know how the still images are going to look?
the compression in RAW and JPEG codecs and whether they're horrible compression methods or not?
Do we know how long it is going to be process and clear the buffer? Do we know the start-up time?
Do we know whether the camera will be broken instantly under rain? Dust? High temperature?
Do we know if the slow-motion will have SD resolution and unacceptable aliasing?
Do we know if the video AF works well?
Do we know if the codec won't have enormous banding, macro-blocking, mosquito noise, edge pixelation issues?
Do we know anything about the camera?
We don't,
yet 99% of the PR readers came out believing firmly it's a great camera, without having any data to support it.
(The A7RII might do extremely well in ALL these areas but the point is made I believe, the previous list applies to any new camera)
Specs only mean the size and weight of the camera. Wait, in fact, the listed size and weight are always misleading as it's stated without a battery, cards, handles, lens, microphone, oh well.
Just please stop reviewing camera on spec-sheets, it's only harming the film-making community and leading people to choose incorrect products.
Is it possible to evaluate a product that way? That's the question we're trying to answer. The only way to do so is to look at
1-Whether spec sheet provide enough data
2-The examples in previous and current product quality vs. their spec sheets.
Specification sheets of cameras provide following information
1 -Resolution, stated in a number of 1080p, 4K, etc.
Is that number indicative of the camera's resolution/sharpness? The answer is no. The spec data about resolution merely states the number pixels in the final file. Of course we all know that's not an indicative of final image resolution. Canon DSLRs have a spec. of 1080p, while Canon C300 has a spec resolution on 1080p, yet the difference in resolution is absolutely enormous. The reason for that is, there are many other elements that determine the image resolution that are not mentioned in the spec sheet resolution section. These are,
-how many sensor pixels is that image derived from? Is it derived from the same number of pixels without down-sampling? In this case we know as a Bayer sensor this means the actual resolution is 60% of that number stated in the spec.
Or is it derived from a higher resolution part of a sensor? If it is then the second element that determines resolution is
-The down-scaling method. When the image is derived from a larger number of pixels, how is it actually down-sampled? Is it line-skipping rows of the sensor? is it pixel-binning? is it reading the entire area and down-sampling, and if so, what algorithm is used, Cubic, lanczos, a bad one that results in lowered resolution and aliasing artifacts?
-The lens resolving power: in case of fixed lens cameras, or interchangeable cameras when the user intends to use a specific lens.
Therefore based on that, a resolution of a video cameras cannot be evaluated reading the spec sheet, as it simply does not provide the needed data to do so. The 1080p, 4K, number in the spec sheet is meaningless and does not determine how the resolution of the camera is going to be like.
2- Low-light performance: only stated as minimum and maximum ISO values and not the noise amount or noise pattern at different ISOs, making this section completely useless.
3- Dynamic range: is always in fact not stated at all, but sometimes it is, in a number of stops. The dynamic range number hugely varies among tester and users, and has various definitions, for examples a company can measure the DR of a camera as the number of stops between the points where the camera clips at both ends, yet this does not mean anything to the end user as the quality of the areas near both ends are not stated. The shadow areas can contain detail (not clip) but have significant amount of noise and compression artifacts making a significant decrease in the final DR of the camera to the operator. Plus the highlight roll-off and the shadow roll off are not stated. Making the DR number in the spec sheet nearly a meaningless indication of the final range the camera will be able to output.
4- Colour performance: only stated in bit depth and chroma sub-sampling. In fact, the most important aspect in cameras' colour rendition is the colour science. It's a processing method, a complicated number of code lines that change and shift how each colour seen by the camera is represented in the final image. Each company, and each camera within each company can have a different colour rendition, something not stated in any spec sheets, making it impossible to judge how pleasing or un-pleasing the colours produced by the camera are.
5- Lens: In fixed lens cameras and in lens specifications of a separate lens.
The stated specs are merely the following, the focal length, the aperture values, and the size and weight of the lens. However, nothing about the lens optical performance is stated. A lens performance is a very complicated subject and it's silly one could evaluate it using the focal length and aperture values. These ''only'' state what the zoom range is, and the DOF possible. Nothing else is stated. How the resolution is, how the resolution is at different focal lengths, how the resolution is in the corners vs center, how the out-of-focus rendition looks, how much chromatic aberrations exist, colour fringing, focus breathing, Coma, Contrast, peripheral shading, nothing is mentioned. In fact, even the focal lengths mentioned are not entirely accurate on most lenses, and even the aperture values do not indicate the actual brightness (T value). Therefore a lens simply cannot be evaluated based on its spec list. The Canon 50mm F/1.4 is on specs an identical lens to the zeiss Otus 55mm F/1.4. An example, The 24-240mm F/2.8 lens on the RX10 is only better than the XC10 24-300mm F/3.5-5.6 in aperture value (DOF), yet everything else (about a dozen factors in lenses performance) about the lenses are not known and must be evaluated first, yet people on announcement crowned the 24-240mm F/2.8 the ''better'' lens ultimately, not knowing it could be a much better or a much worse lens. Specs simply don't give enough data, actually in case of lenses, almost none.
6- Codec, is always stated next, as H.264, ProRes, along with the bitrate and bitdepth and chroma subsampling rate.
That is accurate and it does state the compression method used in the camera, but what's important to know is that it does not indicate in anyway how the quality of the image and codec are going to be. That's determined by the implementation of the actual codec in the specific camera, and the compression quality is determined by many other elements not stated in the spec list, such as the actual resolution going into the codec, the downsampling method, the processor performance, the colour dye and colour science within the camera, and the final image enhancing algorithms. Prominent examples are a camera like the Canon C100 which has a 24mbps H.264 4:2:0 8bit codec, yet has a strikingly better compression performance than a 550D with an identical on paper 24mbps 4:2:0 8bit H.264 codec. Therefore the codec numbers on the spec sheet mean very little to almost none in the final quality of the image and even in the final compression performance, as in how hard the image can be pushed in post. The elements that do mean something are simply not stated in specification lists.
7- Frame rates, are always stated next
Yes they do give an accurate representation of the camera's maximum frame rate ability. But just that, all the other more important data about that mode are not stated. The downsides implemented to achieve these higher frame rates are not mentioned, for most cameras they achieve it through skipping lines, which results it horrible aliasing and a significant resolution less at higher frame rates, even on high end cinema cameras. Therefore, when the spec sheet mentions a camera can do 1080p at 25p and 50p, it's true, but whether the 50p mode is usable in quality is an entirely different subject again, not stated in the spec sheet. So it's not the frame rate number is not indicative of the camera's capability as a slow-motion shooter.
8- Written next can be the Audio specs.
What's mentioned is the format, hz number, the type of input/output. However, the most important elements are not stated. The type and quality of the Pre-amps in the camera, the actual sound quality, whether the audio can be controlled manually or not, therefore one simply cannot get any indication of the camera's capability to record sound based on the spec sheet, it's almost a useless section of the sheet.
9- Battery:
Is mentioned in the spec list as the product name, as an LP-E6n or an XD-T62 or a WTF-876 and sometimes it mentions a capacity number. However, the spec list does not mention the power draw of the camera whilst shooting video, it doesn't give any indications whatsoever on how good or bad the battery life in the camera is.
10- There are many very vital elements in cameras that are not mentioned altogether
-Aliasing performance
-Moire performance
-Rolling shutter amount
-Highlight roll-off
-Resolution & quality of crop modes
-LCD & EVF quality and performance under day light
-The menu system speed
-The menu system user interface
-The responsiveness of buttons and dials
-The feel of the camera in hand
-The ergonomics of the handling of the camera
-The heat management
-The durability under various conditions
-The materials used in the built
11- The elements that ARE accurate and mentioned in spec sheets are actually:
-The size of the camera
-The size of the sensor
-The weight
-The media slots
-The LCD size
-The total number of pixels on the sensor
-The price of the camera
That's it.
If you want to review a camera based on that be my guest it's fine,
but it's just a physical impossibility to know whether this camera is ''good'' or ''bad'' when you have absolutely no idea about the 11 elements of image quality it produces, not even one, including resolution.
This was the answer to the question about whether spec sheets provides enough data and the answer is a clear no. And this is actually true regarding all products, from phones to TVs to Cars and air-jets!
Next time there's a new announcement, please think about the meaning of the represented data and whether they can be used to evaluate the camera. A camera is way more than just a sheet from the maker, that sheet only presents about 10% of the quality of the final product. Please, don't buy a camera based on specs, but it based on the quality of the camera's image to you, and the quality of the design, nothing else, and this data can only be obtained by using and testing the camera heavily, or by reading the review of someone you trust who does that.
Let's look at a recent spec-impressive announcement which we all crowned a wonderful camera just based on the list, The Sony A7RII.
We're excited about the 4K resolution, but do we know how much resolution it's going to actually resolve?
in FF or s35 modes?
Do we know whether it has deadly aliasing/moire or not?
do we know if it will have deal-braking rolling shutter?
do we know if it will record more than 10 minutes without over heating?
Do we know how the low-light performance is?
Do we have any information of the colour science?
on the amount of DR?
on the Highlight roll-off?
on the noise pattern and texture?
do we know if the battery is going to last more than half an hour with a 42mp 4K sensor?
Do we know if the 5 axis stabilization will give acceptable results in video mode?
Do we know if it won't have unacceptable audio quality?
Do we know how the still images are going to look?
the compression in RAW and JPEG codecs and whether they're horrible compression methods or not?
Do we know how long it is going to be process and clear the buffer? Do we know the start-up time?
Do we know whether the camera will be broken instantly under rain? Dust? High temperature?
Do we know if the slow-motion will have SD resolution and unacceptable aliasing?
Do we know if the video AF works well?
Do we know if the codec won't have enormous banding, macro-blocking, mosquito noise, edge pixelation issues?
Do we know anything about the camera?
We don't,
yet 99% of the PR readers came out believing firmly it's a great camera, without having any data to support it.
(The A7RII might do extremely well in ALL these areas but the point is made I believe, the previous list applies to any new camera)
Specs only mean the size and weight of the camera. Wait, in fact, the listed size and weight are always misleading as it's stated without a battery, cards, handles, lens, microphone, oh well.
Just please stop reviewing camera on spec-sheets, it's only harming the film-making community and leading people to choose incorrect products.
