GH2 Are all these new hacks and patches creating a placebo effect?

I think the real issue here is that it is nearly impossible to accurately show the results of what the hack actually does as compared to the original firmware. I spent 6 hours last night trying to get test stills and videos that compare the hack to the new firmware.

I was shooting about 60 clips for each firmware version and that took about 45 minutes per firmware. When I went back to examine the footage I realized that my subject had settled ever so slightly between shots. The camera never moved but the slight change in position of the subject changed some of the shadows.

In addition I realized that my battery powered light had dimmed about 1/3 to 2/3 of a stop during the whole process. That affected the image drastically but I didn’t realize it until after I had taken the shots.

These differences may seem to be insignificant. However, they can skew results drastically. It can cause you to really believe that a setting you just changed has opened up a whole new world for your camera when in actuality it just changed the exposure enough to give you what you wanted to see.

All I am after is what is the truth. Does the hack actually change the color, dynamic range, and exposure? Please note that I am not testing any motion with this. The 3 criteria I listed can all be accurately tested for a static scene. Testing motion is too variable and subjective to accurately come to a conclusion.

I am going to retest again tonight. However, this time I am limiting my tests to only 3 most popular color modes(Cinema, Smooth, and Nostalgic) and 4 ISO values(160, 800, 1600, 3200) for 1080p @ 24 FPS.

I should be able to test those scenarios for each firmware in under 2 minutes. I will use a fixed WB value with a fixed aperture and shutter speed. I will also test in manual focus mode with DR and IR all turned off.

I plan on uploading the original files to Vimeo so that everyone can determine for themselves if the hack affects color, dynamic range, and exposure. Those are all claims that have been made for different hacked firmware versions.

Now even if the test results don’t show a difference in this area it does not mean that the hack is not successful. There are still many other areas that the hack can help. All I want to do is help us determine exactly what areas the hack does help with.

I think a lot has been lost on the fact that I frame only footage is more desirable for editing than I, P, and B frame footage. That definitely does not get mentioned as much as “Hack XXXXX has taken the dynamic range, color, and clarity to a whole new level”.
 
Thank goodness that finally someone is prepared to say that the Emperor Has No Clothes:-) Coming from a scientific background I am just appalled at the total lack objective analysis or comparison regarding the GH2 hack. Where are the comparison videos showing the same scene first with stock firmware & then with a hack? Never mind double-blind just A/B would satisfy me. It's not sufficient to just shoot using the hack & declare it 'Awesome!' nor is the claim that when you 'push it in post' you see a 'huge difference'. Show me the evidence! I am sure that the hacking is a lot of fun & appeals to our obsessive male nerdy character traits but really the GH2 shoots amazing video on a par with Panasonic's own AG-AF101 with the stock firmware so most if not all of this 'Awsomeness' will be there even without the hack. It's funny seeing posts on this forum from people who have just bought a GH2 & are enquiring which hack they should install before they have even received the camera & seen what is capable of without the hack.
 
nigelbb said:
Thank goodness that finally someone is prepared to say that the Emperor Has No Clothes:-) Coming from a scientific background I am just appalled at the total lack objective analysis or comparison regarding the GH2 hack. Where are the comparison videos showing the same scene first with stock firmware & then with a hack? Never mind double-blind just A/B would satisfy me. It's not sufficient to just shoot using the hack & declare it 'Awesome!' nor is the claim that when you 'push it in post' you see a 'huge difference'. Show me the evidence! I am sure that the hacking is a lot of fun & appeals to our obsessive male nerdy character traits but really the GH2 shoots amazing video on a par with Panasonic's own AG-AF101 with the stock firmware so most if not all of this 'Awsomeness' will be there even without the hack. It's funny seeing posts on this forum from people who have just bought a GH2 & are enquiring which hack they should install before they have even received the camera & seen what is capable of without the hack.
Click to expand...

+1 Couldn't have said it better myself. I will give you what you asked for tomorrow.
 
I'm sure that the 176MB patches do have some great quality, and as soon as I can afford the 64GB (I want at least 2), I'll give them a go. For now, Driftwoods SpanMyBitchUp patch offers good quality, Intra, and obviously: spans. For some reason, the bit rate isn't listed on the site, if someone knows, please fill me in.

So, for now I'll stick with that, my Smooth -2 -2 -1 -2, and go out and shoot. If something comes up that's an issue, I'll change things up.
 
nigelbb said:
Thank goodness that finally someone is prepared to say that the Emperor Has No Clothes:-) Coming from a scientific background I am just appalled at the total lack objective analysis or comparison regarding the GH2 hack. Where are the comparison videos showing the same scene first with stock firmware & then with a hack? Never mind double-blind just A/B would satisfy me. It's not sufficient to just shoot using the hack & declare it 'Awesome!' nor is the claim that when you 'push it in post' you see a 'huge difference'. Show me the evidence! I am sure that the hacking is a lot of fun & appeals to our obsessive male nerdy character traits but really the GH2 shoots amazing video on a par with Panasonic's own AG-AF101 with the stock firmware so most if not all of this 'Awsomeness' will be there even without the hack. It's funny seeing posts on this forum from people who have just bought a GH2 & are enquiring which hack they should install before they have even received the camera & seen what is capable of without the hack.
Click to expand...


Without sounding offensive, and I try not to, why not just do it yourself? Especially if you come from a scientific background?

It takes a lot of time to do a real comparison where all variables are the same. Any little bit that's off causes the masses to cry redo, and honestly most of us just do not have the patience to do that.

Then, there's personal preference: some people claim that you can't tell the difference between 1/30 and 1/50th shutter... I find that outright ridiculous, but if people are saying you can't see the difference then how will they see the difference in a hack or in a film mode setting? I personally think stock GH2 footage is pretty unpleasant in any situation where there isn't a ton of light. Others disagreed. I did my own tests on a feature and confirmed that, which is why I didn't use it for anything other than broad daylight shooting. The Hack changed my mind on that, hands down.

Do the tests yourself, draw your own conclusion. If you aren't doing those things then cool, but just because people aren't showing them to satisfy others doesn't make it any less valid, any less true.
 
Last edited:
The problem is that people make claims about the hack that are completely untested. Once one person says it someone else agrees with them and now it is reported as truth. Then you have everyone going out and buying ridiculously expensive memory cards in order to shoot what may or may not be better quality.

Then after awhile everyone starts to get into hacking because they have heard the Hack can do so many marvelous things. However, when they really sit down and test it they can’t determine if there is a significant difference or not.

That is not good because in the end it really gives people a false sense of what the hack does. If everyone really wants to help foster healthy development of the hack then they will follow VKs advice and scientifically test the claims before they make them.
 
I can tell a difference even on a small 720p laptop screen. We don't actually need a big 2K projector or a 50 inch 1080p TV to notice.

I've always found myself with a wandering eye when it comes to visual mediums. When I had VHS tapes, I used to daze off and stare at the fuzzy video artefacts. When we had aerial TV and we had to adjust the reception, I notice the static dots. When I look at internet or DVD footage I find myself always noticing the blockiness, or the squiggly double jagged jpeg lines. When I'm watching a film, I notice the grain patterns.

These hacks definitely are not placebo [maybe up to the point of oh what's the difference between 175 and 220 bitrates?] but there's a difference compared to the stock bitrate. The noise pattern and grain is just so much more nicer looking.

So, without delving into "who cares it's content" (- well duh.) or "it's good to a point" (- well duh too.), I say... why not try and get the best of what you got?
 
The stock Gh2 was fantastic. I bought it in the first few days. Shot quite a bit on it.
Except for the banding on gradients (which was addressed w/ the hack, and did ruin several shots for me), it was great compared to anything else in the dslr world.

Do what is best for you.
 
I don't know why everyone is going on about PB latest camera test... Personally I does not tell me much...

Why set up each camera to look it's worst... we need a test where each camera is set up to be shooting the best possible pic..

Cheers
 
mpgxsvcd said:
I am going to retest again tonight. However, this time I am limiting my tests to only 3 most popular color modes(Cinema, Smooth, and Nostalgic) and 4 ISO values(160, 800, 1600, 3200) for 1080p @ 24 FPS.

I should be able to test those scenarios for each firmware in under 2 minutes. I will use a fixed WB value with a fixed aperture and shutter speed. I will also test in manual focus mode with DR and IR all turned off.

I plan on uploading the original files to Vimeo so that everyone can determine for themselves if the hack affects color, dynamic range, and exposure. Those are all claims that have been made for different hacked firmware versions.
Click to expand...

Looking very forward to the tests. Thanks!
 
For me the hack is somewhere in the middle of extremes and I try to think about it like this. Keep in mind I am talking in the context of photo image clarity and not comparing the two directly...

When artists back in the day wanted to capture an image they had to bust out paint, easel, brush etc and set up a shot. They had to position their subject due to the relation of the sun and sometimes took days due to the light always changing. Then we had the first camera and artificial light where the above restrictions did not apply. Now we don't even need a camera to create pristine moving images (animation). My point is that every evolution in picture technology does not equal picture clarity. The original GH2 was great but the hack just makes it better. Every video you have seen previous to the hack would have been better with the hack had it been applied in that exact scenario. "Better" meaning increased capture data allowing a more pristine out of camera image and better grading ability. Its really just like color data:

4:4:4 > 4:2:2 > 4:2:0

However, with each sample above a 4:4:4@150MB/s will have a "better" image vs 4:4:4@100MB/s. So to my eyes, yes, the hack makes the GH2 better. Can I see it on a daily basis? Absolutely! But it only really stands out when you compare it vs a non-hacked identical image.
 
NOT placebo effect NOR subtle in difference as PROVEN quite obviously by a guy on PERSONAL VIEW (I forget his name) who has posted EXTENSIVE comparisons of different hack setting VS. HDMI output.
RESULTS:
UNhacked suffered very obviously VS. HDMI output.
High Bit-Rate hacks were blatantly closer in appearance to the HDMI image.

These comparisons were GLARING. Not subtle at all - and I'm not even a pro by any means. Really easy to see the difference.
 
Of course there are physical limits to the optics and the sensor that no software hack will ever overcome. But there's a lot you can do with just software.

Noah
 
DPStewart said:
NOT placebo effect NOR subtle in difference as PROVEN quite obviously by a guy on PERSONAL VIEW (I forget his name) who has posted EXTENSIVE comparisons of different hack setting VS. HDMI output.
RESULTS:
UNhacked suffered very obviously VS. HDMI output.
High Bit-Rate hacks were blatantly closer in appearance to the HDMI image.

These comparisons were GLARING. Not subtle at all - and I'm not even a pro by any means. Really easy to see the difference.
Click to expand...

Please provide a link to these "GLARING" results.
 
I guess when I (big focus on I) think of the quality increasing with these hacks I'm looking for the clarity or resolution of the image to increase. My initial post is based on the idea that even tho newer hacks are providing higher bitrates, gops, etc. the videos that catch my eye the most have weirdly been from lower bitrate hacks never really going past 66mb





I'd like to see what you guys see as better footage than the listed videos that have used patches that came after the ones used in the vids. I know this is subjective but I just would like to see anyway.
 
Last edited:
DPStewart said:
These comparisons were GLARING. Not subtle at all - and I'm not even a pro by any means. Really easy to see the difference.
Click to expand...
I totally agree that there could very well be big differences, huge differences, but we will never know, because it has been scientifically proven, time and time again, that once you label them the results will be skewed. You could be absolutely right! I have never seen a blind test where people could really pick these types of differences, which should be easy to do.
Philp B. didn't label his samples, and the results were all over the map. And not only that, people still kept misidentifying the images AFTER he had published the results.
That is, once the ID had been made in a person's mind, they preferred that over the printed answer.
If one were to take these same images, and randomize them without labels in a double blind, few if any could pick which was which.
I recently posted three videos. One was the Olly 45mm, one was a Vivitar 55mm and one was a Minolta 50mm. 200 people looked at the video, but no one picked the Olly.
If I had labelled them, a number of people would have agreed that the Olly looked better.

In camera testing, they have a stock phrase which is pixel peeping. It is generally understood that if you blow an image up to the size of a walrus, that you might see tiny differences. And it is also understood that these differences are not only meaningless, but often within the margin of error for a test, if the light changes or the room temp changes, etc.

So what we have now, I submit, is Video Peeping, or Hack Snacking.
But *I* am guilty. I love playing with the settings. I just don't believe they make a difference unless I can pick it out of a lineup. I have no problem indulging my passion for gear, I just don't draw any conclusions from it.

There is a reason people label video and screenshots when they post them, which is to reinforce a preconceived idea. If you want to know what people think, post a representative sample unlabelled; if you want others to think a certain way, label them. It is as simple as that. No one can claim that labeling each set is easier than not labeling them. And what is even easier in an AB test is to do each test three times. Then you will have six to choose from, and some people will then see differences in a shot taken with the exact same lens and camera five minutes apart.

Another problem is the social aspect. If you post them unlabeled, no one will comment on them (or very few). but if they are labeled, ppl will chime in. And that is why PB's was so much fun: because he is a fairly big cheese, lots of people commented. And only one sharp-eyed commentator got half of the ID's right, which is within a predictable range of statistical variation (in a given sample, you will have some right answers and some wrong answers, tough to get them all wrong.) No one got even seven of ten, for example.

About a year ago, I tested the AF version of the Minolta 50mm against the MF. I made some video, and made some tasting notes about color and sharpness. Then I shot some charts, but I forgot to label the files. And the next day, I had to shoot the whole thing over because I could not tell which was which. Then I played the samples for some friends, and they each had their preferences, which were different from mine. That's not a real blind test, because I could have been giving subconscious signals, flailing with the remote: "Don't you like the $1000 lens better!!??!!!" But the results were all over the map.

I saw a test on TV once where 100 people were given read and white wine in unmarked bottles. And 50 percent picked red, and 50 percent picked white, and 50 percent picked the wrong one. Of course when I saw this I immediately thought it was impossible. I mean, red and white? At least I can tell B&W from color.
 
i guess the only way of really telling a difference, is watching raw footage on a good display option. hope you didn't watch only the vimeo streams.
 
DrDave said:
I totally agree that there could very well be big differences, huge differences, but we will never know, because it has been scientifically proven, time and time again, that once you label them the results will be skewed. You could be absolutely right! I have never seen a blind test where people could really pick these types of differences, which should be easy to do.
Philp B. didn't label his samples, and the results were all over the map. And not only that, people still kept misidentifying the images AFTER he had published the results.
That is, once the ID had been made in a person's mind, they preferred that over the printed answer.
If one were to take these same images, and randomize them without labels in a double blind, few if any could pick which was which.
Click to expand...

This is pretty much correct. Some of us have been forced in the past to take research methodology courses and understand how the placebo effect and double blind testing works. However, this is not to say that there aren't measurable differences between hacked/unhacked or various flavors of hacked. My objection is to people posting footage of their cat or girlfriend and raving about the "measurable performance" of any particular hack. Without even an A-B of a useful scene or chart it all turns into wine tasting (which is not to say that I don't like to drink some wine). I've scoured the net for a couple months looking at footage and tests of various hacks, so far I haven't seen anything that cries out to me as major breakthrough. I've seen some nice pictures and I think its cool that everyone is real happy with their GH2.
 
roei z said:
i guess the only way of really telling a difference, is watching raw footage on a good display option. hope you didn't watch only the vimeo streams.
Click to expand...

That's the best way to tell the difference. That and constantly working with it, knowing what you had before and what you get now.

It's funny, it reminds me of when RED released MX and initially people were saying it didn't male a big diff. But those who worked with it immediately crushed that noise. Then color science and how it changed RED footage. Some people were clamoring quietly that it wasn't doing anything, then RED said to reprocess your old footage.

Again, notions crushed.

We can't go back a reprocess footage with the hack. But for peoPle who've been using it in various situations and then putting it in similar situations again, we see the huge difference.

Like I said, I used the pre hack on my feature and found it unacceptable without a ton of light. The mushy shadows were a big no for me. Especially trying to match it to RED footage. That isn't the case anymore. I don't get why people don't just go out and test themselves instead of telling everyone else that they're seeing things (placebo effect).

Go find your own answers and report back to us. =]
 
You must log in or register to reply here.
Back
Top