pulling apart the hvx firmware

C

cyphunk

Member
UPDATE Mar 2010: Deti has gotten further on this. see http://forum.slashcam.de/spa-mit-der-ag-hpx301e-vp409232.html (in German).
UPDATE: Added more information about how to dissect the firmware and notes i took on different components. All the details are being compiled here (very technical details, but also education)
UPDATE: Page 3 contains a run down of targets and various findings from the firmware

SUMMARY: Firmware is compressed using Deflate algorithm. A look at the strings indicates that a SuperH processor is used, SH4 sub-type SH7751 specifically. The system is potentially a Linux 2.4, MontaVista, based? If so, a modified old P2 2GB card and inclusion firewire storage driver glue might present an oportunity for a homebrew based linux system that lets you plug up an external hard drive. This would be for directly recording to the HD but an even easier might be: P2 card in one slot and PCMCIAtoFirewire card in other slot with HD connected to it. You would still record to the P2 but you could then offload the files from the P2 card to hard drive. Would only require inclusion of firewire storage linux drivers?
REQUESTS: Anyone have SuperH machine code references or manuals? Anyone else working on this so we can combine our efforts? Anyone have internal pictures of the boards in the camera. I would like to search for Serial or JTAG pads. Does anyone have P2 cards they would be willing to donate to this project?
TO PANASONIC: I am your friend. Professionals will not want to open up a P2 card to solder wires to plug an external hard drive controller, but those that need cheap but less reliable solutions could. This will increase the lifespan of your product without adverse effect to 3rd party vendor markets.

DETAIL:

I'm now ready to start opening up the HVX200 i have, and the P2 cards, to look for some interesting modifications. To familiarize myself i thought i would start with looking at the firmware. It uses a common compression called Deflate and a decompressed version of the latest firmware can be downloaded here (decompressed using STUNS). Running "strings" on the firmware will start to give you some clues as to the system and internal components on the HVX.

SH IPL+g version 0.9, Copyright (C) 2000 Free Software Foundation, Inc.
Click to expand...
This is the bootloader. "It is an Initial Program Loader(IPL) of many SH-based systems booting Linux kernel. It functions as gdb-stub to provide the way to communicate with GNU gdb debugger. Also, it acts as a BIOS of the system." ref1, ref2



# CONFIG_CPU_SUBTYPE_SH7300 is not set
# CONFIG_CPU_SUBTYPE_SH7707 is not set
# CONFIG_CPU_SUBTYPE_SH7708 is not set
# CONFIG_CPU_SUBTYPE_SH7709 is not set
# CONFIG_CPU_SUBTYPE_SH7750 is not set
CONFIG_CPU_SUBTYPE_SH7751=y
# CONFIG_CPU_SUBTYPE_ST40STB1 is not set
# CONFIG_CPU_SH3 is not set
CONFIG_CPU_SH4=y
CONFIG_CPU_LITTLE_ENDIAN=y
# CONFIG_P2PF_K200 is not set
CONFIG_P2PF_K230=y
# CONFIG_P2PF_K250 is not set
CONFIG_P2PF_HW_VERSION=2
CONFIG_MEMORY_START=08000000
CONFIG_MEMORY_SIZE=04000000
CONFIG_MEMORY_SET=y
Click to expand...
This is the configuration file for a Linux kernel. It shows us that the CPU is a SuperH SH4 subtype SH7751. This family of CPUs are also used with Sega gamesystems, which is exciting because I know a few people that pulled those systems apart.

Linux version 2.4.20_mvl31-ms7751rse01-sh_sh4_le (root@rx7) (gcc version 3.3.1 (MontaVista 3.3.1-7.0.42.0600552 2006-04-30)) #13 Thu Mar 22 19:28:14 JST 2007
Click to expand...
More indication of a linux system.

An entire filesystem/ramdisk appears to be enclosed in the firmware image. My next step will be pulling this out, mounting it (appears to be a cramfs system) and exploring the filesystem. On the hardware end of things I want to pull apart the camera and find a JTAG or serial port. The SH-IPL might support loading a GDB debugger channel. So now I have a reason to really void the warranty :evil:
 
Last edited:
Spec-comm, thanks for the thread. I did read a lot of it and it was generous of your company to answer technical questions but I think purpose I have here is to make opensource revisions of the camera, and general curiosity.

On another note, does anyone have photos of the inside of a P2 and/or the HVX200? Otherwise, I plan to make them.
 
Very interesting development idea. Its ingenuity like this that pushes us towards new ideas and gives end users more options to utilize the tools we invested in.

I like the direction your headed. we shot P2 on 4 - 3700 varicams this past weekend during a sporting event. The poor dump guy was running like a mad man.... Heres a question:

Why not create a card like this that you could send a stream back to video village via SDI so the remote drive could be in a safe place. Then you could ISO each cam AND send your tc / video to the board for switching?

Maybe I dont have a grasp on this but it sounds good.
 
rpmnava said:
Very interesting development idea. Its ingenuity like this Why not create a card like this that you could send a stream back to video village via SDI so the remote drive could be in a safe place. Then you could ISO each cam AND send your tc / video to the board for switching?
Click to expand...

That would be a cool idea but since the HVX doesnt have SDI support it would require, well, I guess adding drivers for a compatible firewire or pcmcia SDI component/bridge. It's a good idea that after I have pulled everything apart and worked on more basic modifications, might be possible. but it is definitely not in the low hanging fruit category.
 
I've now taken apart the firmware and pulled out the different filesystems inside. I started exploring the available drivers and components, looking for targets for reverse engineering. I've documented much of this in a blog post here:
http://deadhacker.com/2009/07/26/targeting-the-panasonic-hvx200-hd-camera/
(warning, very technical, but also educational for anyone wanting to learn how to pull apart embedded hardware)

It is ongoing and I would love feedback here, suggests, help. I will get back to it in a couple days and update here when i have more.
 
Well something different from Hydra.

btw, I've found the update routines and also found a much easier way to decompress and get the proper files out of the firmware:

mv <firmwarefile> <firmwarefile>.tar.gz
tar -zxvf <firmwarefile>.tar.gz

Also, found routines for updating the LCD, LED's and some room for debugging in the form of some supported devices left in for debugging purposes (follow the modules.dep tree). So, we are well on our way to running code. Even if I do not setup a serial interface or other debug console, we could write out debug info to the SD card, as is already done by other programs on the camera.

So, more updated info coming soon. I'm in NYC and DC for the next couple weeks if anyone wants to get together and work on some modifications.
 
This is great. I use CHDK in Canon point-n-shoots with software I wrote (that runs IN the camera) to do tricky time-lapse photography. I'd love to be able to do stuff like this with the HVX.

Plus, the HVX is simply incomplete without DOOM.
 
I use GNU/Linux almost exlusively (my company maintains a distro called Nexradix). As a result, I am very interested in this thread.

I'm mostly interested in the possibility of developing a P2 driver for Linux. Obviously porting the one from the 2.4 kernel that the camera uses to a modern 2.6 kernel might be a challenge, but ANY ability to read P2 cards in Linux (even with an older kernel) would be a really big deal.

I've recently had a breakthrough regarding playing back and editing footage from the HVX/HPX in Linux. I'll be releasing a tutorial in the coming weeks.

Meanwhile, if there is anything I can do to help your discovery process, let me know. I'll take some time this weekend to dig into the firmware image and let you know if I find anything interesting.
 
Any firmware upgrades to the hvx would be great. Could a waveform or focus bar be ported from the newer cameras into the hvx somehow?
 
pmark23 said:
Why can't you read a P2 under Linux? Isn't it just a standard PCMCIA storage device formatted as FAT?
Click to expand...

I wish it was. Sadly, P2 cards only show up as a Universal Mass Storage device when mounted via USB using the camera. The camera itself has the driver, hence my hope that using the firmware we can port the driver to the regular OS.
 
In the low-hanging fruit category...
Software-based image flip for the LCD would be nice.
2.40:1 onscreen guides
Audio-only record mode (two or four channel)
The PAUSE/REC on the display is in a terrible spot... if you could move it to outside the 16x9, that would be great.

How much of a long-shot would 10-bit be? At 720p 24pN there should be enough headroom in terms of bit rate... what about processing? Ok, that's my dreaming for today!
 
Maybe I'm asking for too much, but if possible I'd absolutely LOVE some really high frame rates, especially at standard def, cause I realize more rez=more bandwidth......but I don't know anything about the internals, so ignore me if necessary, lol.

I'm working on a spinning matrix type rig that will use a lower end Casio high frame rate cmos camera(210 or 300fps)....being able to strap in a high frame rate standard def HVX would be AMAZING.

I second an image flip being one of the biggest things missing....if I could use my M2 without an external monitor I'd be one happy camper!
 
Check out CHDK. It doesn't replace the original firmware, but adds to it so there's no chance of screwing anything up and doesn't void the warranty.

CHDK is a firmware enhancement that operates on a number of Canon Cameras. CHDK gets loaded into your camera's memory upon bootup (either manually or automatically). It provides additional functionality beyond that currently provided by the native camera firmware.

CHDK is not a permanent firmware upgrade: you decide how it is loaded (manually or automatically) and you can always easily remove it.​

Some features it adds are:

The current set of extra capabilities fall into six categories:

a. Enhanced ways of recording images - you can capture still pictures in RAW format (as well as JPEG), and for video images you can have increased recording time and length (1 hour or 2 GB), and a greatly increased range of compression options.
b. Additional data displays on the LCD screen - histogram, battery life indicator, depth of field, and many more.
c. Additional photographic settings that are not available on the camera by itself - longer exposure times (up to 65 seconds), faster shutter speeds (1/25,000 sec, and faster in some cases), automatic bracketing of exposure, etc.
d. The ability for the camera to run programs ('scripts', written in a micro-version of the BASIC language) stored on the memory card - these programs allow you to set the camera to perform a sequence of operations under the control of the program. For example, a camera can be programmed to take multiple pictures for focus bracketing, or take a picture when it detects that something in the field of view moves or changes brightness.
e. The ability to take a picture, or start a program on the memory card, by sending a signal into the USB port - you can use the USB cable to take a picture remotely.
f. The ability to do a number of other more useful (and fun) things, such as act as a mini file browser for the memory card, let you play games on the LCD screen, etc. ​

I have CHDK running on my Canon point-n-shoots and it's indispensable. One of them is permanently setup to do long and complicated time-lapse with my own intervalometer I wrote (in Basic!) to use in my client's projects.

I think that this is a good model to follow. There's also lots of information on the CHDK site about how they got their software running alongside Canon's firmware.
 
Great work!

This is exactly the kind of innovation from the computer-centric community I've been looking for on the HVX for years now. P2 cards & the HVX/HPX are exactly the kind of A/V hardware that can finally let hackers take control of their own products, where the A/V industry has succeeded on crippling products well below their true capabilities/customizability.

I thought ProfessorU's suggestions for low-hanging fruit were right on target.

My highest priority, though, would be more along the lines of Linux driver capabilities expressed by other users.

We run multiple cameras in field & studio all over New York and surrounding environs, and both cheaper P2 cards and a hard-drive-on-camera solution are really just poor substitutes to the real ideal: a P2 Store-like device that's more customizable, beefy, user-friendly and capable.

If I could use Linux to build a box that lets me offload to 2 2.5" 7200 RPM SATA drives (2 for redundancy) powered by a Panasonic-compatible camcorder battery, with a small LCD for managing & viewing clips, with USB ports to offload from there to other machines and the ability to serve as an external drive... I'd have the perfect field device that'd allow fundamentally unlimited recording time for a couple hundred bucks, while not making any sacrifices regarding the camera's mobility in the field. And the only thing standing between me & that is Linux compatibility.
 
I thought I'd drag over thoughts from another thread, and propose a target :

1. Getting E-sata drivers into the linux so that a PCMCIA E-sata card can be used in a P2 slot and then attached to a harddrive to store footage.
 
You must log in or register to reply here.
Back
Top