GH2 RAW Hardware Hack for GH2 : Info, Progress, Etc.
- Thread starter Dan Hudgins
- Start date
Dan Hudgins
Well-known member
Arm twisting...
Arm twisting...
I deleted this post because one of the URL mentioned seems to have changed, so the link was no longer valid.
Arm twisting...
I deleted this post because one of the URL mentioned seems to have changed, so the link was no longer valid.
Last edited:
The image sensor most likely uses multiple pairs of LVDS for output, probably 8 or 16 pairs (channels) to pump out all that data to the video processor, and you can't just solder to them because they are impedance dependent.
Considering what James is doing, I do believe it would be easier to develop a camera system from scratch like he is doing, than to try and mod the GH2. Panasonic won't give up datasheets on its components. They have no way to know if you are the competition or not.
There are a lot of amazing sensors out there though. I have even seen a 35mm 5k 53fps sensor with global shutter, it has row windowing so you can select the size of the pixel window you want to grab from, smaller window means higher frame rate. Imagine what it could do at 1080p, maybe 120fps? Don't know much about it though since I didn't have access to its datasheet.
I want to develop a camera using it though, lol!
Considering what James is doing, I do believe it would be easier to develop a camera system from scratch like he is doing, than to try and mod the GH2. Panasonic won't give up datasheets on its components. They have no way to know if you are the competition or not.
There are a lot of amazing sensors out there though. I have even seen a 35mm 5k 53fps sensor with global shutter, it has row windowing so you can select the size of the pixel window you want to grab from, smaller window means higher frame rate. Imagine what it could do at 1080p, maybe 120fps? Don't know much about it though since I didn't have access to its datasheet.
I want to develop a camera using it though, lol!
@Dan Hudgin
i think that is better to use usb 3.0 ...
it can do 400MB/s no need for dual GigE vision ...
i spoke with James many times and he told me that his card can go 500MB/s...
107MB/s is just sensor limit ...
http://www.cypress.com/?id=3526
i think that is better to use usb 3.0 ...
it can do 400MB/s no need for dual GigE vision ...
i spoke with James many times and he told me that his card can go 500MB/s...
107MB/s is just sensor limit ...
http://www.cypress.com/?id=3526
Last edited:
Dan Hudgins
Well-known member
It should be possible to tap the digital signals using emitter followers perhaps with gallium arsenide transistors, or broad band RF Op-amps, or some other high input impedance input as the clock rates for multi-readout sensors would be LOWER perhaps 40 to 80MHz. Its possible that the sensor just has one high speed serial output as well, in both cases it should be possable to ease drop on the signals the same way an oscilloscope can be used to look at electronic signals. To dismiss the ability to tap the signal without knowing the details of how the sensor is wired or what methods of circuit can be applied is premature.
James may not fully understand the details of how such analog circuits methods can be applied to tapping data off digital circuits but I could answer some questions if he wanted to talk about it.
Data loggers use signal tapping to log what is going on in active circuits.
I doubt that there are 8 or 16 pairs of channels in the GH2, ARRI ALEXA (tm) only uses 2 and Kodak sensors like used in Acam dII (tm) can use up to 4, but the Kodak sensor is analog output. Its easier to have the digital output one high speed serial output and put the ADC inside the sensor for mass produced small cameras, it reduces the parts count and cost.
The main issue, that is unknown is, is there enough room to fit the needed parts inside the GH2 case to get the signal out without interfering with the lens mount and such. With small cell phone surface mount transistors, the tap pre-amps could be fit on a very small board, and the wires going to the sensor circuits could be short enough to work at the frequencies required.
He told me that he wanted to sell his 2.5K sensor board with the PCIe interface as circuit boards without a case for about $500. I have some footage shot with that set of circuits, but there was a glitch on the bottom of the image, some lines from other frames were showing up, you could not see it other than when the camera or subject was moving side to side. I asked him SEVERAL times if he had fixed the glitch problem and have yet to get an answer, other than he plans on making yet another camera design and to abandon the PCIe card. I thought that if he could get it working without the glitch issue I would maybe get a board set, probably with the other parts needed to shoot a demo video and add it to my Vimeo page since the results would be similar to the KineRAW-S8p (tm). I've yet to get an answer to my questions on this issue.
James told me that he would rather sell his PCIe circuit board set because of the high cost of developing a camera case and making a finished selif contained camera.
Putting a tap on an existing camera solves some issues:
1) The camera firmware already exists and does not need to be developed, and its bug free.
2) The camera case and lens mount already exist and do not need to be fractionated with high precision and at substantial cost in low volume.
3) The cost of purchasing a ready made camera and adding a small circuit board can be much lower than making them, and the sensors selected for such products are purchased in quality and so at lower cost than ones that can be purchased in small orders by a small camera maker.
==
In addition to all that, I don't think you can get the same sensors used in the Canon 5D III with all the needed spec sheets and development circuit information at the same price Canon gets them to make your own camera with, can you?
I asked James if he was interested in this concept and I understood his response as being affirmative, if he did not understand what I was talking about of things its not of interest to him, its not imposable to do and anyone else is welcome to do it, as its been done before in the case of Andromedia (tm) and even if there some additional challenges to the development of the circuit tap micro board, then this generation of young engineers can step up and show their skills with analog RF circuitry unless their electronic eduction was lacking, in which case they can go to the library and find some books on the subject.
==
If James wants to use Kickstarter.com (tm) to develop his own camera case, lens mount, and such. Then he has reversed what he was telling me a few weeks ago.
He asked me to assist him in getting the word out about his PCIe interface development, but if he does not want to sell them then I don't know what that's about.
I have spend quite a bit of time adding support for his data formats in my de-Bayer program, but what for if he does not sell any cameras or boards that capture Bayer data?
Why did he make a BBS forum and then not put its URL on his [URL reference removed] page, or even his email address?
I did what I said I would, I added support for his 2.5K camera to my software, its free and up for download in beta-test now, but where is the camera to go with that. Same with KineRAW-S8p (tm), although they tell me that they plan to work on it some more after the KineRAW-S35 (tm) goes on sale in China. After 4 years of work on my programs it would be nice if something gets on the market from all this development or what was the point?
James said he was going to sell his board set for about $500 but when I sent him examples of the glitch issue, which I brought up over a year ago I think, and brought up again since it would need to be fixed for the tap to be useful as well, he changed the subject and did not answer my questions about it, so maybe he will get something on sale someday, I tried to help and even if naysayers think a tap is not workable based on a lack of facts its always easier to jump to conclusions that to do new things in innovative ways. As I pointed out there are many ways to ease drop on digital signals without serious impedance issues if its done with knowledge and skill, which one assumes such engineering would be done with.
You don't actually need to solder onto the signal lines, you can just glue the insulated sense wires of small size near the conductors on the pins of the sensor to pick up the spike of the up/down transition of the logic signals by capacitive coupling or induction, such a light load of perhaps 1pf is probably not enough to disrupt the camera circuits, but could be enough to amplify and buffer to get a usable signal out.
What is the part number and URL of the part maker for that sensor?
For a Digital Cinema Camera to be useful the dynamic range of the sensor needs to have low FPN, as you know the Canon HDSLR show very high ISO ratings, even with internal temporal noise reduction and FPN subtraction and other signal processing, the FPN of the sensor and its 14bit data need to be clean to get usable results. Sensor made for industrial cameras may not come near what the sensor in a camera like the Canon 5D III does for FPN levels, so its both lower cost to tap such a camera and in the end it may result in better images, and the whole point of recording True RAW in the end it go get better images, and if you can get better images at lower cost then why go the other direction?
Last edited:
I am not saying you can't eaves drop, I am saying it might not be possible to eaves drop on multiple signals and get intelligible data if it is LVDS (which I believe it is) because a change in impedance (by changing its effective length) will change its timing. Every line has to have exactly the same impedance.
That said, how do you know the ARRI Alexa sensor only has 2 outputs? It was a CMOS sensor, meaning digital output, made by Cypress Semiconductors, now owned by ON Semiconductors, all of the sensors they offer use LVDS.
Go take a look at the datasheets for high end CMOS sensors from all the large companies. You will see that they almost exclusively use LVDS for data output, and use SPI for control. Because the amount of data being put out over a single line would be too high. I mean, for 1920x1080x12bit it would be 1.06GBps.
As for your induction idea, I don't think it will work, the signal levels are already miniscule, any inductive pickup will just blend in with the background noise, and if it is LVDS, capactive coupling will mess with the impedance.
As for the sensor, I don't have the link, but it is also made by ON Semiconductor, it is the only 25MP sensor of the bunch, and there are only 11 sensors, so you won't miss it if you go to their website.
That said, how do you know the ARRI Alexa sensor only has 2 outputs? It was a CMOS sensor, meaning digital output, made by Cypress Semiconductors, now owned by ON Semiconductors, all of the sensors they offer use LVDS.
Go take a look at the datasheets for high end CMOS sensors from all the large companies. You will see that they almost exclusively use LVDS for data output, and use SPI for control. Because the amount of data being put out over a single line would be too high. I mean, for 1920x1080x12bit it would be 1.06GBps.
As for your induction idea, I don't think it will work, the signal levels are already miniscule, any inductive pickup will just blend in with the background noise, and if it is LVDS, capactive coupling will mess with the impedance.
As for the sensor, I don't have the link, but it is also made by ON Semiconductor, it is the only 25MP sensor of the bunch, and there are only 11 sensors, so you won't miss it if you go to their website.
Dan Hudgins
Well-known member
I asked an Arri rep and he said that Alexa (tm) has two outputs, so I don't know what it has but I understood that to be the answer.
As for reading the data, there has to be a margin of safety for the logic fan-out and part of that can be used to tap the signal. It better to base claims on actual tests for the circuits to be used, rather than jump to the conclusion that something is imposable, and what is gained by poo-pooing?
Obviously there would be some development to see what strategy would work for the signal tap, to make a blanket claim that its imposable without even knowing anything about the actual circuits is not informed or constructive.
So what do you gain by being un-constructive and un-supportive?
You are trying to save peoples time? Is it not their time to spend in the exploration of a solution?
Its like saying its imposable to go to the moon because you might not succeed, well if you have that attitude then you shurrely would not.
The maximum fps in the HDSLR is lower than in high speed cameras that use many readouts, and so its possible that there is only one readout, the Canon cameras may be using Sony sensors so looking at what data sheets for their sensors may give some insight into the logic drive and fan-out.
Another option if all else fails would be to un-solder the sensor and re-solder it to a small board that is then soldered where the sensor was, that would allow all of the sensor pins to be buffered and so enable any fan-out you want without some of the loading issues (although you might need to adjust the timing of various in-coming signals to match the delay on the outputs), more so in the case of the GH2 maybe as the firmware can be hacked. The lens mount would then be re-Columinated to make up for the sensor being a few mm closer.
There might be issues with the camera being tapped, and some may not work, but some might, so I don't get where the negative attitude comes from, as it could be a big thing if it was figured out how to make it work well. What is it costing you for other people to spend their time on the subject?
Just because something is difficult does not make it imposable or un-profitable, that is how some people get ahead, by doing what others cannot figure out how to or are to timid to try.
As for reading the data, there has to be a margin of safety for the logic fan-out and part of that can be used to tap the signal. It better to base claims on actual tests for the circuits to be used, rather than jump to the conclusion that something is imposable, and what is gained by poo-pooing?
Obviously there would be some development to see what strategy would work for the signal tap, to make a blanket claim that its imposable without even knowing anything about the actual circuits is not informed or constructive.
So what do you gain by being un-constructive and un-supportive?
You are trying to save peoples time? Is it not their time to spend in the exploration of a solution?
Its like saying its imposable to go to the moon because you might not succeed, well if you have that attitude then you shurrely would not.
The maximum fps in the HDSLR is lower than in high speed cameras that use many readouts, and so its possible that there is only one readout, the Canon cameras may be using Sony sensors so looking at what data sheets for their sensors may give some insight into the logic drive and fan-out.
Another option if all else fails would be to un-solder the sensor and re-solder it to a small board that is then soldered where the sensor was, that would allow all of the sensor pins to be buffered and so enable any fan-out you want without some of the loading issues (although you might need to adjust the timing of various in-coming signals to match the delay on the outputs), more so in the case of the GH2 maybe as the firmware can be hacked. The lens mount would then be re-Columinated to make up for the sensor being a few mm closer.
There might be issues with the camera being tapped, and some may not work, but some might, so I don't get where the negative attitude comes from, as it could be a big thing if it was figured out how to make it work well. What is it costing you for other people to spend their time on the subject?
Just because something is difficult does not make it imposable or un-profitable, that is how some people get ahead, by doing what others cannot figure out how to or are to timid to try.
Last edited:
Dan Hudgins
Well-known member
It seems the sensor you are talking about is only 10bit output so not suitable for digital cinema use where the sensor must be underexposed by about two T/ stops.
The sensors in HDSLR have higher bit depth 12 or 14 bits and so are suitable for Digital Cinema Camera exposure levels.
That is an important reason for doing a HDSLR tap, to get the higher bit depth of a state of the art sensor with lower noise levels.
You should understand that a standard gamma correction of 0.45 is applied to sensor cameras, but cameras data bits are output in linear sensor data normally, so you need at least about 10 bits to get 8 to 9 usable bits after gamma correction without histogram gaps. In order to lower the exposure levels 2 to 3 stops so that you have 2 to 3 stops of highlight detail for specular reflections above 90% white card subject brightness, you need a sensor with at least 12bit output, better sensors have 14bit output. Its just not possible to make a high quality digital cinema camera with the lack of highlight room and lack of leeway for avoiding histogram gaps. Such a camera would have limited exposure latitude.
Demo images can be made that look better because when you reduce the images and record True RAW you average the pixels noise and pick up some additional bits, a half size result will have 2 more bits than a full size image. So if you don't use the full size image at a viewing distance where you can see individual pixels you may get usable results, but if the results are compressed full size, the histogram gaps may get locked in by the compression since it interferes with adjacent pixel noise averaging in your eyes.
Is this what you were talking about?
http://www.onsemi.com/PowerSolutions/product.do?id=VITA25K
Quote: [Integrated 10-bit ADCs]
A 14bit ADC sensor can show 16x the tonal range and 4 T/ stops more exposure range, depending on how well it works. Can you show me the link for the sensor spec sheets of the sensor used in the Canon 5D III? Or even the GH2?
http://www.eoshd.com/content/242/25mp-35mm-full-frame-cmos-global-shutter-sensor-introduced
Quote: [nterestingly, Cypress say the sensor also operates in rolling shutter mode, to reduce noise and increase dynamic range. It appears global shutter has not yet been perfected in the image quality stakes, at least for a sensor with 25 megapixels at 53fps.]
Last edited:
Most of what you said doesn't make sense to me from an Electrical engineering standpoint.
For instance "Demo images can be made that look better because when you reduce the images and record True RAW you average the pixels noise and pick up some additional bits, a half size result will have 2 more bits than a full size image."
That doesn't make any sense. You can't pick up additional data. If a sensor outputs at 10bits, than that is it, regardless of what you do to the image, you can't get any more data.
Also "You should understand that a standard gamma correction of 0.45 is applied to sensor cameras, but cameras data bits are output in linear sensor data normally, so you need at least about 10 bits to get 8 to 9 usable bits after gamma correction without histogram gaps."
I don't even understand what that is, "cameras data bits are output in linear sensor data"? That doesn't make sense, it looks like nonsense jargon.
As for the Vita25k sensor, there is nothing in the old EosHD article to signify that the sensor couldn't be used in a cinema like camera.
Finally, You seem to be indicating that 10bits of data isn't usable, but you seem to like the 8bit video you are getting from the GH2. Considering it is only 8 bits, I am wondering if the sensor is only 10bit. As for the datasheets for the GH2, won't happen. The sensor is made in house, so there is no reason for them to release it. Unless we find someone who works there, we won't get any direct info on it.
There are some tear down reports of the GH2, but I don't have the money to pay for them.
Ian-T
Veteran
It’s obvious he’s saying that to get a nice “clean” 8bit image without any (what he calls) “histogram gaps” you need more than a 10bit starting point.
Oh, and by the way…the GH-2 is a 12bit camera (not 10 bit).
As far as him not making sense with this statement
It might not make sense to you but he is correct. When you are downsampling resolution, as long as it is into a codec that will not restrict colour(chroma) info, the downsample will not stay locked at the origional 4:2:0, but will condense the chroma info that it has available. If that is SD 4:4:4, it will take advantage of that space. You aren't gaining anything, you are just condensing what was already there.
Give the brotha a break.
It’s obvious he’s saying that to get a nice “clean” 8bit image without any (what he calls) “histogram gaps” you need more than a 10bit starting point.
Oh, and by the way…the GH-2 is a 12bit camera (not 10 bit).
As far as him not making sense with this statement
It might not make sense to you but he is correct. When you are downsampling resolution, as long as it is into a codec that will not restrict colour(chroma) info, the downsample will not stay locked at the origional 4:2:0, but will condense the chroma info that it has available. If that is SD 4:4:4, it will take advantage of that space. You aren't gaining anything, you are just condensing what was already there.
Give the brotha a break.
I understand what you are saying, but what you are saying and what I get from him are 2 different things. If you take a 12bit image and simply downsample to 10bits, there is no way you will get that extra information back because you have just lost that information. If you want to downsample a 10bit 4:2:0 to a 8bit 4:4:4, you will still only end up with a 8bit 4:2:0 image.
Dan Hudgins
Well-known member
If you load 10bit images into 16bit data format MSB aligned then reduce the image by interpolation to half width and height and it was recorded without histogram gaps to start with, like raw, then the reduction by AREA of 4 gives two more bits,
Say one pixel is a value of 444 and the one next to it has a value of 445 then you reduce the image width by half you then need to store a value of 444.5, if the 10bit data was scaled up to MSB align in 16bits, then there are enough bits to store the half value because of the scaling to align the MSB.
Likewise if you reduce the image in two dimensions you get two more bits worth of tonal data through the 1/4 result size as long as you have MSB aligned the data so that the fractional bits are not truncated.
Like if you have four pixels you get one of four possable results:
n.0
n.25
n.5
n.75
After scaling those values fractional values are not lost.
like if a group of 4 pixels which is averaged to be one pixel, on half size reduction, such as using a 4K sensor to make a 2K image, you get 2 more bits in the resulting data because counting from the top, the 10bit MSB aligned data in 16bits now requires two lower bits to represent the fractions .25, .50, and .75 so you end up with 12 bits that change and the lower four bits in the 16 are still all zeros, unless you are re-sampling the size on a ratio other than 1/4.
Its just math, in binary math it requires two more bits to represent the fractions you get from averaging four pixels down to one pixel.
Obviously if you do not MSB align the data in the 16bits for doing the math first you will lose the bottom to bits because of truncation.
If you reduce an image to 1/4 width and 1/4 height, then you need 4 additional bits to store the fractional values that averaging makes. 1/16 area needs 16x as many tonal values to not lose any of the data fractions.
The reason you want to do this with raw sensor data is so that the pixel values represent the original probability from one frame to another and one pixel to another of the subject brightness after the ADC rounds off the signal to an fixed point number of limited digits.
The more you reduce the image in size through averaging the pixels (not sub-sampling just some of the pixels) the more you increase the accuracy and precision of the tonal representation because you get finer increments closer to the actual value. Likewise with temporal noise reduction between frames.
In order to gain precision the processing of the 10bit data needs to be done with more than 10 bits and the result MSB aligned so that the lower bits generated from the averaging are retained, obviously.
That is why I use 48bpp TIF in my DI system, because even after grading the bulk of the original data and its interpolated intermediate values are retained, although scaled in various ways, only to be seriously truncated for the end use format.
Scaling errors introduce histogram gaps and banding if you don't use a data format with enough bits for the interpolated fractional values in the intermediate steps.
Last edited:
Dan Hudgins
Well-known member
new product or retrfit
new product or retrfit
I talked to James by email and it seems he decided to go with his own free standing project rather than a retrofit. You can email him to ask, but I think that is how things went, its odd to me since he put so much time into developing his PCIe interface and camera software to support that.
Starting from scratch would be much harder than at the level he already was.
I'm sure there would be a lot of people that would like this option, but since then digital Bolex (tm) and Blackmagic 2.5k (tm) have come out into the open,... so?
Also I think I saw somewhere than Canon is thinking of doing raw maybe in the C500?
But those solutions would cost more to the end user than a $500 set of boards and a bit more to add the raw port.
Someone with an Andormedia (tm) wrote to me again and as I understood him he said that some company purchased the Andromedia (tm) technology and then stopped development and basically put it in the 'safe' never to see the light of day again, or something like that? (my paraphrasing the account)
Does anyone know the full story of how Andromedia (tm) was 'taken' off the market (smothered?)? And why?
Back to James, he may still do something but he is not talking about the details in public as far as I know about what exactly that will end up being, but something may happen before the end of the year.
new product or retrfit
I talked to James by email and it seems he decided to go with his own free standing project rather than a retrofit. You can email him to ask, but I think that is how things went, its odd to me since he put so much time into developing his PCIe interface and camera software to support that.
Starting from scratch would be much harder than at the level he already was.
I'm sure there would be a lot of people that would like this option, but since then digital Bolex (tm) and Blackmagic 2.5k (tm) have come out into the open,... so?
Also I think I saw somewhere than Canon is thinking of doing raw maybe in the C500?
But those solutions would cost more to the end user than a $500 set of boards and a bit more to add the raw port.
Someone with an Andormedia (tm) wrote to me again and as I understood him he said that some company purchased the Andromedia (tm) technology and then stopped development and basically put it in the 'safe' never to see the light of day again, or something like that? (my paraphrasing the account)
Does anyone know the full story of how Andromedia (tm) was 'taken' off the market (smothered?)? And why?
Back to James, he may still do something but he is not talking about the details in public as far as I know about what exactly that will end up being, but something may happen before the end of the year.
Last edited:
Dan Hudgins
Well-known member
On the subject above about the number of bits needed, if you have 12bit Bayer data, there are two green images when you add those together you get one of three outcomes n.0, n.5 and n+1.
So you need at least 13bits in the RGB file to represent the color information in the 12bit Bayer data. More so actually since the interpolation uses real numbers can can get into complex ratios.
So its best to MSB align the 12bit sensor data and store the RGB result MSB aligned into 16bit (16bpc 48bpp) TIF or DPX frames to retain the fractional values from the green mixing and the interpolation. If you then reduce the image since additional image tone intermediate values are created from the averaging of adjacent pixels.
That requires True RAW recording so that the dithering noise is retained, if the sensor data is compressed to plateau the tones into blocks then you cannot gain intermediate values from the averaging of adjacent pixels within those uniform blocks.
So you need at least 13bits in the RGB file to represent the color information in the 12bit Bayer data. More so actually since the interpolation uses real numbers can can get into complex ratios.
So its best to MSB align the 12bit sensor data and store the RGB result MSB aligned into 16bit (16bpc 48bpp) TIF or DPX frames to retain the fractional values from the green mixing and the interpolation. If you then reduce the image since additional image tone intermediate values are created from the averaging of adjacent pixels.
That requires True RAW recording so that the dithering noise is retained, if the sensor data is compressed to plateau the tones into blocks then you cannot gain intermediate values from the averaging of adjacent pixels within those uniform blocks.