rendering with Canopus

[stabwound]

Those Edius tutorials are terrific... I'm almost up to speed to take on an actual project.

I enjoy my editing... until it's time to render the video... which takes a while. Until the rendering is done the computer is useless for anything else.

Here's what Edius should do.... implement a rendering system so work can be farmed out to networked computers.... like the way 3D programs do. I've got five other computers in the room sitting idle.

This way I could continue to edit and not miss a beat.

Whaddya thnk, Brandon?

 

[ChuckS]

Distributing a 3D render is very different from distributing 2D compositing or editing. Not to say that it can't be done.

When you render 3D you build a scene disciption file in your favorite 3D application and then you can distribute that (small) file accross multiple CPU's to render frames placing the result on either local or common storage array. But in 2D you may have a terabyte of information sitting on a disk array that requires all the CPU's to access the same data (possibly at the same time), do something to it and either replacing the original (not recommended) or copying the result to a new location so you started with 1TB and ended with 2TB's.

Again it can be done but it requires a very robust network and file system to get any appreciable economies of scale, which is the whole reason to distribute the render in the first place.

 

[stabwound]

But in 2D you may have a terabyte of information sitting on a disk array that requires all the CPU's to access the same data (possibly at the same time), do something to it and either replacing the original (not recommended) or copying the result to a new location so you started with 1TB and ended with 2TB's.

.

What you say is true.

On my present system, I can barely scrub/playback without frame skipping. Another computer accessing the same footage will definitely slow the editing down.

Perhaps it might be possible with really fast raid arrays, one for original footage and another for redered footage.

Also, drive access priority can go to the computer that is editing. For instance, if I were to playback a sequence, the other computers stop accessing the drive and wait their turn until playback is done.

Thanks for the input.

 

[bhiga]

Yup, as Chuck described - in the 3D world, everybody "knows" how to do the same thing. Like a paint-by-numbers where everyone has the same diagram. Thus, you can assign different machines to paint different sections and merge them all together in the end. Since everyone's working off the same diagram, the merge is seamless and correct.

In the video world, the only constant is the video frame itself, and the "remote painters" have no clue what the "diagram" is. In order for them to share the work, they'll all have to access the "diagram" - and since it's video, the "diagram" changes in each frame.

You can do some distributed rendering by "chunking" things and running different sections concurrently. However, since the video data still needs to be moved between the storage and the workers, network connectivity becomes a bottleneck fairly quickly. One solution is to copy the video to local storage first, but there is a balance between the time it takes to copy the video and the time saved by concurrent processing.

Brandon

 

[stabwound]

Mighty interesting, Brandon.

I'm disaapointed the work can't be speeded up with more computers, ofcourse.

But I'm sure some software engineer will think of something...

Thanks for your response.

 

[bhiga]

One solution is blade computing, where multiple CPU blades share a common backbone and therefore can function as if they were a single bonded entity. This means they can all access the same resources over a very high-speed connection.

Actually, Iwill has a motherboard which is sort of a hybrid blade - the DK8HTX uses Infiniband interconnects between units to get the high-speed "joined processing" functionality while the boards themselves sit in individual chassis.

Still, the main problem is data storage access. Since RAM is pretty cheap nowadays, a large cache like a RAM-SAN might work, but the main problem is still the interconnects between the systems and the storage. Perhaps some kind of SATA-based SAN or something might work.

If the remote storage can be pulled as-fast-as or very close to as-fast-as local storage, then the bottleneck moves back to the CPU processing.