Who would pay Magic Lantern for 50Mbps and 4.2.2?

[Duke M.]

I think that those of us who understand the difference between hardware and software are aware that the Sony PMW-F3 cannot be upgraded to 50mbit 4:2:2 via firmware. Nor should we expect a hardware patch/replacement program for the Sony F3.

Perhaps you'd like to explain that a little more. We know the SxS cards can record at much higher rates than necessary since they do that on other cameras. We also know that the camera is operating at 4.4.4 internally and uncompressed or lightly compressed or it wouldn't output 4.4.4 to the Gemini.

(I highly doubt Sony would ever release such firmware as it might affect another camera line and their business plan seems to be all about maintaining separate lines of cameras at separate price points.)

However, Canon was just as out of touch with their buyers and was exactly the same way about maintaining camera lines but Magic Lantern made cameras like the 550D come alive with features and data rates not even dreamed of by Canon. Finally Canon realized it was an advantage to issue their own firmware. To a lesser extent Red did the same thing, although theirs was by design. They had the hardware and it took them years to write the software to take full advantage of the hardware. (Heck, took them a long time to even become stable.)

Magic Lantern did it the right way. Their only change to the firmware was to add a few lines of code to make the camera look at the card first and if additional software was on the card while booting it loaded that software. This way is a lot more safe since if the new software was no good you just remove it from the card and the camera boots normally. Its very unlikely to brick the camera that way. You can have a separate card as a bootable disk. (Hey something to do with those tiny 8Gig SxS cards which I always found too small for most jobs. LOL)

What Magic Lantern needed was a couple versions of firmware that they could analyze to see how it worked before they started. We are on version 1.40 on F3 firmware so that should be enough to start.

I know I would pay it in a heart beat, frankly I think this is what Sony should offer early S-Log adopters, but I wonder how many people would be willing to pay $1,000 to get 50Mbps (or more) and 4.2.2 internally? If a group of people were willing to do this the ML programmers would likely work on it. Payment upon working software (although they've been good about continuing updates.)

 

[Tata Steva]

I am in...

 

[arturrito]

+1 !

 

[lasikdenver]

Duke -- I like the Samurai, but it sure would a nice option, count me in (Buccaneer F3 project).

 

[Viddovation]

Perhaps you'd like to explain that a little more. We know the SxS cards can record at much higher rates than necessary since they do that on other cameras. We also know that the camera is operating at 4.4.4 internally and uncompressed or lightly compressed or it wouldn't output 4.4.4 to the Gemini.

Duke, you've been in the room when conversations were had about this in the past: http://www.dvxuser.com/V6/archive/index.php/t-232212.html

The Convergent Designs Nanoflash licenses the XDCAM 422HD encoder form Sony, and their people have plainly stated that their device uses two encoder chips to record 4:2:2 material, and that only one chip is required to record 4:2:0 material. The Sony EX1, EX3, and presumably the F3 only have a single encoder chip in their hardware. Opening your EX1 or EX3 with a screwdriver will confirm this, or read the service manual.

Updating firmware changes how microchips utilize available hardware. But it doesn't change the limits of that hardware. In the case of consumer oriented products, there are indeed many examples of companies, including Sony, artificially limiting the products capabilities, by using firmware that does not take full advantage of the hardware.

As for the apparent disconnect between the powerful 4:4:4 uncompressed outputs, and 4:2:2 compression schemes, the fact is these are acheived in different manners, using different hardware. I'm but a novice at circuit boards and electronics design, but you learn early on that general processing units are very inefficient at processing digital signals like audio and video. When the input data always comes in a known "size and shape" with a known set of potentially desired outputs, you can design a special computer chip, called a digital signal processor (DSP), to do the demanding, repetitive math required - quickly. But that efficency comes at the cost of flexibility. Just as in life, you can be a jack of all trades, or a master of one. DSP chips are a master of one.

That said, the image sensor data gets split and sent to different boards. Remember how the camera runs hotter and drains the battery faster when you use the dual HD-sdi outputs? That's because the camera doesn't power the board for those outputs if you're not using them. Seperate board, separate processing chips.

And compression is it's own thing. It's very different to generate an HD-SDI data stream than generate an mpeg2 encoded file. Different chips are used.

I'm not a Sony engineer, and I sure as hell wish they'd shipped the F3 with the 50mbit codec, but I'm not surprised that they haven't added it, considering the existing hardware probably doesn't support it.

Considering that in the wake of the C300 release debacle they did make s-log free to compete, I suspect if they had the capability they would also make 50mbit free in new firmware, so as to prevent the c300 from having a leg up.

 

[Duke M.]

Both my shop teacher and my sergeant (come to think about it my shop teacher was a sergeant at one time too) always used to say "If you assume you make an a$$ out of U and me." (Ass|u|me. Hey, I guess they did drill it into me. Oh, and measure twice and cut once.)

I remember with my Canon 7D it was widely disseminated that the video would always have the same problems it did at the beginning. Low data rates, bad compression, no features, etc. Same with the 550D and the major difference between the 550D and 7D is one DSP instead of two.

Even the Canon reps said it couldn't improve... Until Magic Lantern hacked it and gave us huge improvements step by step. About a year after that was perfected Canon came up with the same features in a firmware update.

I'm not sure it can be done, but I'm not willing to foreclose on the option based on an assumption.

Does anyone out there have access to an F3 repair manual? I remember someone saying the F3 does have two separate circuits. That doesn't mean it can't handle the data rate on one board. Slow mo does record at over 35Mbps. That's documented.

I do recall calculating one time that 24fps at 35Mbps and 4.2.2 isn't more than 30fps at 35Mbps and 4.2.0. Plus engineers usually build in a safety factor. If you go a little over you're still good. Even if its 8 bit I'll still take 50Mbps.

Its not like my warranty from Sony is good anymore. I just wrote an email to one of the ML guys.

 

[Viddovation]

Duke, you pretty neatly avoided all of my points by essentially saying "are you sure?"

I'm not saying conclusively that it's impossible, but if you've even done novice work with integrated circuits, there are plenty of reasons that would come to mind as to why it's improbable.

To speak to my experience, and my lack thereof: I've never released a product, but I have designed and tested a microcontroller circuit (and correlated firmware) for usb control of canon cameras. This was done on the pic32 microprocessor. I've also done technical reading on digital image sensors, image processors, and output encoding. To a pro my understanding is juvenile, but I feel confident in saying that it is far superior to most end users.

It is absolutely true that engineers frequently either intentionally or unintentionally don't provide end users with the full functionality the hardware is capable of. Some of this is practical to avoid overheating and other risks to the product. Frequently, it's because they want to protect the marketshare of more expensive models. And sometimes, engineers use hardware that is capable of more than was requested on the spec sheet, and they only build firmware to match the spec sheet.

For instance, it's common knowledge that the Sony FS100 uses the same image sensor as the Sony F3. However, they use different digital signal processors, and this limits the FS100 from getting the same latitude as the F3. Using the same chip is cost effective though, because they only need to spend R&D money on one sensor chip, and only need to run one assembly line, and it's cheaper to generate a greater volume of chips, per unit.

Does anyone out there have access to an F3 repair manual? I remember someone saying the F3 does have two separate circuits. That doesn't mean it can't handle the data rate on one board. Slow mo does record at over 35Mbps. That's documented.

It would appear you could get a copy of the repair manual from getMANUAL.com

Are you certain that slow mo is over 35Mbps at the *playback* speed, or just at the recording speed? My understanding is that it will be 35Mbps at the playback speed. Keep in mind, even though this demonstrates the chip can process data *faster* than 35Mbps, it does not demonstrate that the chip can *encode* data in any other form than 35Mbps 4:2:0.

I do recall calculating one time that 24fps at 35Mbps and 4.2.2 isn't more than 30fps at 35Mbps and 4.2.0. Plus engineers usually build in a safety factor. If you go a little over you're still good. Even if its 8 bit I'll still take 50Mbps.

Just because the storage size is the same (or close) doesn't mean that the work involved is similar. Just because a semi truck and a sedan can go 80 miles per hour doesn't mean they can carry the same load. The Convergent Designs Nanoflash is 8bit, 4:2:2, and it uses two Sony MPEG2 Torino encoder chips, where the Sony EX1 and EX3 only use one. Don't you think that if Convergent Designs could have made one chip do the same job, they'd do it to cut costs?

Even the Canon reps said it couldn't improve... Until Magic Lantern hacked it and gave us huge improvements step by step. About a year after that was perfected Canon came up with the same features in a firmware update.

I'm not sure it can be done, but I'm not willing to foreclose on the option based on an assumption.

I get you. But, it's worth noting that Magic Lantern software works precisely because Canon enables it. Both Magic Lantern and the Canon Hack Development Kit (CHDK) take advantage of the fact that Canon digital cameras have always had firmware that looked to the storage drive (SD or CF card) for any additional instructions. At boot-up, the Canon camera loads, and like your computer might check for a CD drive to boot from, the Canon camera looks for instructions on the CF or SD card. It's brilliant, really. And Canon is to be applauded for being so open.

Sony and Panasonic have never been so open. The Panasonic GH1 was engineered in an entirely different way, as a replacement set of firmware, sent to the camera as a firmware update. An overwrite. And it had to be reverse engineered, because Panasonic never intended for it to occur.

Reverse engineering firmware updates is difficult, tedious work. My understanding is the Gh1 hack team exploited the fact that a publicly released firmware update wasn't encrypted, or scrambled. But Sony and Panasonic frequently do send the updates encrypted, making it more difficult to reverse engineer.

And fundamentally, the testing risk is low with the Canon method storage media boot method, and high with the firmware overwrite method. Why? Well, when you're only loading a set of instructions on a removeable, rewriteable media, you can "crash" the camera, turn it off, take the card out, and it's 99% likely to boot like nothing happened. When you push a firmware update overwrite, you may "crash" the camera in such a way that it won't boot far enough to accept a new overwrite. Combine that with the fact that we're talking about a $14K camera instead of ones costing $800 to $3,500, and there's plenty of reasons for pause.



Oh. And as to the original question, assuming a $500 or $1000 upgrade was possible to buy... I'd wait until a client asked for it.

 

[PaulReam]

Oh. And as to the original question, assuming a $500 or $1000 upgrade was possible to buy... I'd wait until a client asked for it.

And there's the rub.... clients are never going to ask for it, which is why I wouldn't be interested.

 

[Duke M.]

@Viddovation, I wasn't trying to dis you, but when the canon rep said it couldn't be done and yet it was done I want to know before deciding. You may very well be right in everything you said. On the other hand, we don't have much in the way of facts yet.

@Paul, depends on your market. If you're trying to sell to BBC, Discovery, etc. and think an external recorder is a potential weak link with extra wires, extra switches, etc while moving around then internal 50Mbps and 4.2.2 is highly desirable.

If the ML guy responds I'll let you know.

 

[Viddovation]

@Viddovation, I wasn't trying to dis you, but when the canon rep said it couldn't be done and yet it was done I want to know before deciding. You may very well be right in everything you said. On the other hand, we don't have much in the way of facts yet.

Duke, I never took it as a diss - and I know you're a great guy (got that atomos cable today - Thanks!). I understand your interest in this matter. I'm not sure where you're pulling this information about the Canon rep from - I've not seen actual quotes from Canon about this. But regardless, that's an entirely different company and a wholly different set of hardware.

I would contend that we do indeed have quite a few facts at our disposal, including all the previously made points about circuit design, microcontrollers, mpeg encoder chips, digital signal processing, and cost management in engineering, as well as the C300 competition and the S-Log price change. I would suggest that you merely don't like what those facts imply. To wit, you've not refuted or even attempted to refute any points I've made about hardware design, except with examples about a wholly separate company.

I wish you the best of luck in this pursuit, but you started this thread with a request that I explain why I considered this proposed firmware upgrade to be impossible, and I beleive I've provided ample evidence of why I have that opinion.

 

[FilmDude]

why pay $500 - $1000 to try to hurt your Sony F3 with a hack that "maybe" could get you 50mbps when you can put that same money into an outboard recorder and be done with it?? makes no sense.

Bottom line, F3 CANNOT handle the stream of 50mbps 422 to its cards, there is no encoder chip inside the camera to do so. A "Hack" would only put you at risk for burning out a processor all together and void warranty AND most likely prevent you from future firmware updates. OR future firmware updates could cause your camera to be bricked if the "Hack" doesn't play nice with the codes.

It's a stupid idea to want to hack a $15,000 device. It almost makes sense on these little dslrs that you can buy for around 1K, but not a professional 15k camera. I would never want to be on set with a camera with professional clients with my "hacked" camera that can shoot higher rates, only to have it malfunction, and then worse, no way to fix that with sonys help.

Hack = bad idea, regardless if its done by a company, or just a guy in a basement. It's a nice thought but not a professional way to handle this. That same money spending on a hack you can easily buy an outboard recorder that will retain some resell value, and/or be used with other cameras.

Theres just no sense to it in my book.

 

[Duke M.]

Viddovation,

I thought long and hardlast night about the points you raised. I was trying to remember the comments and results from the 7D and 550D (two DSPs vs. one).

DSP can't handle the data:
The DSP is a dedicated CPU type chip. It processes the signal according to the software instructions. The thing that made the 550D be able to do the higher data rate with only one chip is that the higher data rate isn't the data speed coming in (the chip could already handle that) it was a higher data rate coming out. To do that it was compressing it LESS. As in leaving out certain steps so there were less compression errors because there was less compression.

For the F3 the you have a situation where it can record 60 fps of incoming data and record it on a 24fps timeline. That's 2.5 times (250%) the speed of recording at 24fps.

In comparison, going from 35Mbps to 50Mbps is only a 50% increase. Throw in going from 4.2.0 to 4.2.2 is going to add more data, but file sizes don't grow that much. It wouldn't surprise me if the total of the two was a doubling (200%) of the data rates (but at the same time requireing less processing of the data.)

Therefore, you might not have slow mo at 50Mbps but the DSP probably isn't the limiting factor in achieving 50Mbps for a regular signal.

Can't write the data fast enough:
The same kind of thing is present with the recording of the data. In the old days the ability of the writing device to handle the data was limiting. However, these are solid state and digital. These days the ability to write is limited by the media, not the writing device. Both the SxS cards and class 10 SD-HC cards are fast enough.

Circuits can't handle the data:
However, you may be pefectly right if a hardware circuit is just throwing out a line of color info (in 4.2.0 the zero means no color data recorded in that line) but in every application I've seen the DSP samples the other line and does a version of finding a median color before it tosses the color data.

I should also point out that the Nano, using DSPs now produce files at 220Mbps. That's a 628% increase. They use two chips for 4.2.2, but we only need 200% for 4.2.2.

Its also possible that there is some other hardware issue, but I have my doubts about it. Remember, Sony started selling this camera without having the full software completed (as did Red.) They couldn't afford to build in some roadblock that they might not be able to get around when they were issuing firmware updates. If you're tempted to think the updates only pertained to S-Log they didn't'. They also included writing ability (to SD-HC cards), controls, picture profiles, and lots of other things.

50Mbps isn't within MPEG2 specs:
I could also imagine having to use a plug-in on your NLE to handle some odd flavor of MPEG2. However, the same thing happened with the Nanoflash and there is a plug-in out there that handles virtually every flavor of MPEG2 known to man that you need to use high data rate Nano files.

@Filmdude,
Have you ever had the cable or connection to your external recorder go bad during a shoot? Have you ever had the recorder not trigger when the camera started? Have you ever had your AC not notice the other media was full? I have.

I actually like having the added back up of dual recording so I would probably continue to use an external recorder. 35Mbps is often enough, but there are times when I'd like both video clips to be high quality.

It would give us more safety and more options. No one would be forcing you to modify your camera. Obviously the software would be tested and working before installing it on someone's camera. What we did before was found a couple of cameras that fell off cars and it was more expensive to fix than replace. Use those junkers for test beds.

 

[PaulReam]

@Paul, depends on your market. If you're trying to sell to BBC, Discovery, etc. and think an external recorder is a potential weak link with extra wires, extra switches, etc while moving around then internal 50Mbps and 4.2.2 is highly desirable.

Spot on Duke, that's exactly the situation I'm finding myself in. What's happened is Producers from BBC, Discovery etc now ask specifically for the C300. For me, unless Sony launch a large sensor 50Mb/s camera in the £10,000 price range at IBC next week, they'll have missed this TV market completely. It's a crying shame because the F3 looks to me like a better camera. I do hope Sony didn't cripple the F3 in an effort to protect their own range of products - if so, they shot themselves in the foot!

 

[Viddovation]

DSP can't handle the data:
Your stance is optimistically simplistic. For one, going from 35Mbps 4:2:0 to 50Mbps 4:2:2 isn't just a decrease in compression by 15Mbps - you're also asking the chip to encode in an entirely different colorspace. In fact, you're asking it to process twice as many samples for color reduction. And that's before it does the codec compression.

Say what? Read this: http://www.dvxuser.com/articles/colorspace/
You'll see that to do the 4.2.2 color space means taking twice as many samples as 4.2.0 color space requires. You might interject "but it can send 4.2.2 hd-sdi, so it should be able to reroute." That's almost certainly not true - the path is going to come in from the image sensor DSP, likely debayered in uncompressed 4.4.4.

My guess is the nano uses two chips because there's twice as much work to do the color space sampling, and that it has little to do with the compression data rate.

Regarding writing the data fast enough:
I don't think this is the bottleneck.


Circuits can't handle the data:
I would say "I don't beleive the circuits were designed to handle the data." Generally, safety overhead aside, engineers are charged with delivering a product that meets the requested spec sheet, at the lowest parts cost possible. Building in functionality for features that have not been requested or anticipated in usually bad - it means you left waste on the table. Using cheaper parts, and only the needed parts, means being able to being product to market at the best available price.

Also, please keep in mind that just because a specific DSP model may be capable of running a different set of code, doesn't necessary mean it's possible to alter it's code once installed in the camera. There are multiple reasons for this.
Updating the DSP firmware may be impossible if:
1. The DSP chip is "flash once." Flash once chips are cheaper to mass produce, making them very cost effective for applications that will not change. They are used frequently.
2. If the DSP chip pin out for firmware update is not "wired in." Programmable microchips require a certain pinout for receiving a firmware update. Frequently, they are programmed prior to being installed on the circuit board. They may not use the same pinout on the circuit board.
3. There is no practical way to push an update through the primary CPU to the DSP chip. Even if the pinout exists, it may be very tedious to engineer the firmware push from the primary microcontroller to the DSP. Even if Sony has such a prodcedure, without their documentation, it may be a fools errand. And you could brick a few $14k cameras trying to find out.

50Mbps isn't within MPEG2 specs:
To my knowledge I never made this claim.


Duke, if you have the expertise to take this project on yourself, I wish you the best of luck, but I think there are plenty of reasons why most people with electrical engineering and firmware knowledge are going to pass on this opportunity:
1. Testing & mistakes are potentially expensive.
2. Evidence the existing hardware is unlikely to support 4.2.2
3. Small market share compared to the Canon DSLRs means less glory, and less money.
4. The possibility implied by the PMW-100 and PMW-200 that the next iteration of this camera will ship with this feature standard.

All in all, it's a whole lot of work, plenty of risk, and meager rewards. What I can say about my F3 on Sony firmware is that it's robust. It works, and works well. I for one, cannot afford to take a chance by using a third party firmware that may break during a shoot. I'm with Film Dude on that one.

But I'm sure keen on whatever Sony releases next.

 

[FilmDude]

And you could brick a few $14k cameras trying to find out.

And thats the end all right there. NO ONE is going to work on a hack only to destroy a 14k camera just trying to TEST the limits of the hack.

And to duke

You're talking about cables - they cost at most 100 dollars and you have mutliple of these in your camera bag (or should), recorder not recording due to not triggering is human error and has nothing to do with a dead camera which is what I'm talking about.

Everything you've stated has nothing to do with a BRICKED camera in the field. You can reshoot a take due to error, you can swap out a cable, and you can fire an AC and do it yourself. But You CANNOT bring back a bricked camera. Which shuts down production completely. A dead wire does not kill a set. A missed take does not.

Put it this way, if I'm a producer and I hear you have experimental hack on your camera, and I'm spending $8k for the day of shooting and hiring crew, set, and actors.... well lets just say you're not going to get hired by putting my 8k at risk because you wanna not buy an outborard recorder.

When you update something that wasn't in spec you risk not just your gear, but the entire productions budget and schedule. That sensor overheats and shuts down, you're screwed, and the producer is screwed, and everyone down the chain.

I think its nice that you're optimistic, but the reason why you see it happening in DSLR's is simply because its CHEAP to do. And if you brick a DSLR you're ok... Also when hacking DSLR's they aren't using usable cameras, most of the time they buy "for repair" cameras that are beyond damaged to be able to use and can be bought cheap on ebay. So that 3K Canon was most likely a $500 junk camera from ebay that they felt safe opening up and screwing with. If it failed they lost $500 - NOT 14k

The DSLR world does not translate to this at all. Its different animal, different money, different everything.

The BMD hyperdeck shuttle had to put out 2 different models simply because they needed to put an additional chip inside that could deal with codecs. Sometimes there no hacking something, you simply need a new chip.

 

[Duke M.]

Its funny to see people riled up.

a) I already have an outboard recorder and plan to continue to use it. At this point in time I use the XDCAM footage as a back up. There are times that the outboard recorder has a glitch. The XDCAM footage has saved my ass on those occasions. I just want the XDCAM footage to be a little better, to be sufficient for those certain broadcasters that want more when the outboard recorder does glitch. If it happens that one time a wildebeast is jumping over a river and a crocodile launches itself up to grab it in mid air there is no other choice. I have a shot like that.

b) Experiments would be done on wrecked F3s. We find a couple that fell off a car, etc, out of warranty and its too expensive to fix them.

c) When all you can afford is a 5D I don't think its cheap to brick the 5D. People did it because it works.

I do agree the way the F3 imports firmware may make all the difference in the world. On Canon the ML guys found one Canon flag that they could use to make the camera look on the card first. Sony F3s do not operate that way. On the other hand we have 4examples of firmware to look at.

No, I don't plan on doing it myself. That's why I said I'd ask the ML guys. If they say its not feasible, since they have successfully done this on more than one camera, I'll accept it.

I don't disagree that its "probably not feasible", but that's exactly what the Canon rep told me personally before it was actually done. It made me cynical of pronouncements based on an educated guess, and probably isn't the same as impossible.

I'll just wait and see.

 

[Viddovation]

On Canon the ML guys found one Canon flag that they could use to make the camera look on the card first. Sony F3s do not operate that way. On the other hand we have 4examples of firmware to look at.

No, I don't plan on doing it myself. That's why I said I'd ask the ML guys. If they say its not feasible, since they have successfully done this on more than one camera, I'll accept it.

ML has only been done on Canon DSLRs. I've worked with the USB protocols for the Canon cameras, and I assure you, all the Canon cameras share a great deal of operating system and code structure. This means "porting" code between related cameras is relatively easy, compared to starting from scratch.

Developing firmware for a Sony camera would mean starting from scratch. That's a whole lot of work compared to "porting."

Also... you keep referring to Magic Lantern in reference to getting 50Mbit 4:2:2 video - it's worth pointing out that while the Magic Lantern firmware enables you to change the bit-rate of the h.264 video recorded, it remain in 4.2.0 color space.

From a software design standpoint, that's incredibly significant. It means the dsp on the canon camera accepts a bit-rate selection command. It doesn't demonstrate that the Magic Lantern people did any coding on the codec.

So, in essence, the people you want to ask for help don't actually have any experience doing what you want them to do.

 

[starcentral]

If it could be done, I'm sure anyone would easily pay $1000 to gain higher quality footage.