HVX200 3D Tracking test... [Archive] - DVXuser.com -- The online community for filmmaking

Danilo Del Tufo

12-31-2005, 11:56 PM

It's incredible! I dunno what's camera tracking!? :shocked:
Can you explain to me how you've done? I know blue screen keyng, but how do you do that! Maybe do you have first draw perspective lines, a sort of mask in evr'y frame as an invisible level in preproduction (as in photoshop, you know?), and then have imported the 3d model in editing software?
I've not still an editing software, I will buy Mac with Final Cut pro, so excuse my ignorance, it's a software in the package that can make this? I remember that I've seen something about it, in Final Cut package, a software that make this amazing things!

Thanks for your knowledge, sharing with us!

Cheers:)

proximity

01-01-2006, 05:10 AM

Thanks to Kaku's footage and Anders conversion to PhotoJPG, I was inspired to take a short bit from one of the clips and do a quick 3D track this evening...

Nice :) What software did you use?

- prox

Darkline

01-01-2006, 05:26 AM

Danilo

3D matchmoving cannot be done from an NLE. The process involves a 3D machmoving package such as Boujou, PFtrack, Syntheyes or Matchmover.

This software allows you to reconstruct a virtual camera that matches the movement of the camera you used to shoot the scene. It does this by tracking points in your footage frame to frame; by calculating which points have moved and by how much over time it can create a 3D camera. Once you have this 3D camera you need to import it into a 3D package like Maya, Lightwave, Cinema 4D, or 3DSMax to create the geometry.

Finally you might choose to composite this in After effects, Combustion or even in your NLE if its a simple shot.

if you want to learn more about it; I wrote a tutorial for the basic process of matchmoving with 'Syntheyes' Camera tracker over at creative cow. Here's the link.

http://forums.creativecow.net/cgi-bin/new_page_wrapper.cgi?forumid=1&page=http://www.creativecow.net/articles/holt_karl/syntheyes/index.html

Anders Holck

01-01-2006, 05:46 AM

That looks great, especially the subtle dropshadow and the 3d motion blur.

Never been into 3D myself, how long did it take you to set this up? (tracking+setup in 3d package)

jung kyu

01-01-2006, 07:42 AM

that was very good...is that mental ray..i don't think it's hdri
how long did it took for rendering??

hope to hear from u

Justin_Kirch

01-01-2006, 08:10 AM

damn, that was pretty freaking cool.

Jim Arthurs

01-01-2006, 09:39 AM

Hi all, thanks for the feedback! This was one of those things that was going to be a very quick test and then turned into a couple hours of work last night... first I just wanted to test the track, maybe put some cylinders or cones out, then what the heck, I started to add an element and then...

I used Icarus, the public beta of a software that became PFtrack later in life. The track took about an hour including garbage matting out the biker and people so they wouldn't influence the track. There were other sections of the shot that could have been used, but this was probably the best area for tracking without other information.

I exported the camera motion to Lightwave, found a stock object off of the content folder and placed it in the scene. I used radiosity to light it, rendering with a plugin called FPrime. I have a nice render farm, but FPrime is so much faster than native LW that I rendered the whole sequence in about 40 minutes on a single dual Xeon. I then did two shadow passes for the ground so that I could comp the whole thing in After Effects and degrade the CG to match the live a bit better (adding artifical "sharpening").

Working with 720p is easy and quick compared to full raster 1080, and what I generally do is render CGI to the true rez of the original codec, so this was 960 by 720 with a 1.3333 pixel aspect ratio. This helps match the live action a bit more.

Match moving is fun, and I've done a bit for my work on various projects. In the old days, you'd line up stuff by hand, so the modern software available is almost magical. I'm getting Syntheyes, and as Karl mentioned it's a pretty swift package at a low cost. (great review Karl!).

How did the HVX perform? I think the lens has a tad less barrel distortion full wide than the DVX... I've always had trouble tracking footage from the DVX without first correcting the barrel distortion before tracking. Any "slop" on this shot could be fixed if it was a paying job.

I truly look forward to getting the HVX and shooting with it.

Regards,

Jim Arthurs

Sean R

01-01-2006, 09:43 AM

damn, that was pretty freaking cool.
Second that.
Is it really that easy now? Did you just plug in the points and it did the rest? Amazing.

Jim Arthurs

01-01-2006, 09:54 AM

Second that.
Is it really that easy now? Did you just plug in the points and it did the rest? Amazing.

It's no exaggeration to say that this track would have taken a team of people several days at ILM back around, say, Jurrassic Park or Twister time frame.

Is it easy? That's harder to qualify. It does take some basic skills in the techniques and more skills to render, light, and comp, but yes, it's easier than trying to do it by hand. :)

I want to do the track again in Syntheyes for some perspective on your question and to get familiar with it. I suspect Syntheyes will do a tighter track with less work.

FYI, Some of the bigger FX houses have virtually every version of commercial matchmover software (and custom tools), and throw a shot at all of them to see which one tracks it the best.

Regards,

Jim Arthurs

Darkline

01-01-2006, 10:42 AM

Hi Jim

One of the things i was looking forward to the most was using 720p for fx work. keying/tracking. Im really pleased to hear that you've had success with it. I have to say that you have a pretty good locked track there - all that from footage which was not shot with tracking in mind.

pretty good if you ask me. Well done.

If you are going to run it through syntheyes, you may well get away with masking out the rider and then running an auto-track. I guess one of the advantages of using 60p is that you dont get the jerky movements which may lead to a poor track with hand-held material.

I may even give it a whirl myself, but i dont have the native footage to view (pc boy here).

And yes, tracking can be a one click process or it can be a whole afternoons work just to get the 3D track correct. The footage is also pretty good as there is a lot of perspective shift in there. Dont let it fool you, tracking can be very hard - Jim just knows what he's doing and spotted how he could tackle the shot :-)

Antoine_Fabi

01-01-2006, 10:49 AM

My God !!!

Jim,

you're very talented !
The tracking, the light, color temp, shades, everything look great.

Can i see some of your work on a WEB site ?

thanks

danny dale

01-01-2006, 12:16 PM

My God !!!

Jim,

you're very talented !
The tracking, the light, color temp, shades, everything look great.

Can i see some of your work on a WEB site ?

thanks

you should check this out...

http://www.saintandmather.com/

danny dale

01-01-2006, 12:22 PM

danny dale

01-01-2006, 12:27 PM

Jim Arthurs

01-01-2006, 02:47 PM

Hi all, again, thanks for the nice comments... some responses...

Karl, I used not the raw files, but the PhotoJPG that Anders made from the originals. Icarus does automated tracking, but it helps to seed it with a few manual track points. And thanks for the additional pointers on tracking... you're so right, it can be easy and it can be hard, it's pretty much subject dependant.

For Antoine and anyone interested, I did a couple updates to my demo reel at my website;

http://www.imageshoppe.com/

Also there's a short film a did a few years ago on-line as well... The website is "ghetto web design '97" without much updating over the years... I hate web work!

Danny, yes, that's some amazing work with "Prey"! Great examples of tracking and compositing. Just goes to show how much can be done on a shoe-string budget with hunk of green fabric hanging behind some talent!

Jim Arthurs

rightfrog

01-01-2006, 02:56 PM

That looks sweet...

Are there any good 3D tracking software packages for the Mac or are they all PC?

Greggl

01-01-2006, 03:28 PM

Syntheyes, PFTrack, Boujou, matchmover and 3D Equalizer all have MAC ports.

Jim Arthurs

01-01-2006, 04:19 PM

Syntheyes, PFTrack, Boujou, matchmover and 3D Equalizer all have MAC ports.

Yes, there's great variety on both platforms... one mention, the price of one seat of Equalizer or Boujou is equal to an HVX with 2 8gig cards! PFTrack has a "lite" version limited to SD resolution, if memory serves for under $500, and the best value is probably Syntheyes with no limitations.

One company I work with has several seats of Boujou, a seat of Equalizer and Syntheyes... they've found with that variety that almost anything is trackable without getting their hands dirty with manual alignment. Their workflow is to try Boujou first, if that fails Equalizer almost always works. As well it should for $10,000 US, IMO.

Jim Arthurs

joren

01-01-2006, 09:57 PM

not to get too far off track, but has anyone worked with Pixel Farm's PFHoe? It only works with standard def quicktime and AVIs, but only costs 85 Dollars (49 GBP). If it works just a little, the price sure is right.

http://www.thepixelfarm.co.uk/products/products.aspx?PID=33

Jim Arthurs

01-01-2006, 10:07 PM

Not off track (pun intended) at all... I forgot about this ultra "lite" version of PFTrack. The "guts" of this code is probably the same as the big brother, and several generations more advanced than Icarus, which was what I used to track Kaku's footage.

Somewhere out there is a review of it I remember seeing... maybe Creative Cow?

You could always do tracks on SD proxies of your HD HVX footage. It wouldn't be as accurate, but depending on your shot could work just fine. On the test I did, if the 3D object was in motion and not rooted to the pavement, you'd never be able to tell some slight slippage...

Jim Arthurs

Sean R

01-01-2006, 10:13 PM

Do you know how the motion trackers actually work? I'm always wondering what software is doing behind the GUI...

Greggl

01-01-2006, 10:16 PM

Their workflow is to try Boujou first, if that fails Equalizer almost always works. As well it should for $10,000 US, IMO.

Its an unbeatable combo though. On the last 'big' gig I did, we only had boujou
and I was forced to run two tracks through Maya Live and essentially hand tracked
em. I've been on the betas for most of the major apps and those two are the only
ones that I'd trust in a crunch... both from a software and a developer standpoint.

Jim Arthurs

01-01-2006, 10:42 PM

and I was forced to run two tracks through Maya Live and essentially hand tracked em.

Ah, but hand tracking puts hair on your chest and gives you manly stories to tell (not to mention carpal tunnel)!

I think everyone, regardless of their graphic careers, should be made to do these things to appreciate how good we now have it;

1.) Hand track a shot where the productions notes have no on-set camera info whatsoever.

2.) Hand rotoscope 200 frames of some slow moving object against a low contrast background.

3.) Do a bluescreen shot the old fashioned way, on film with an optical printer.

4.) dump a 900 frame CGI animation to tape, using a single frame controller to do one-frame-at-a-time insert edits on your buddy's borrowed betaSP deck. Nothing like 900 insert edits on 30 seconds worth of tape to show you mean business.

Jim Arthurs

Greggl

01-01-2006, 10:48 PM

4.) dump a 900 frame CGI animation to tape, using a single frame controller to do one-frame-at-a-time insert edits on your buddy's borrowed betaSP deck. Nothing like 900 insert edits on 30 seconds worth of tape to show you mean business.

Ah.. the good ole days.. I fondly remember that distinctive sound of a rented BVW-75
*grinding* away for hours with hundreds of those inserts :)

2D/3D integration has been my thing for a lot of years now. I think I'm one of
the only guys on the planet that actually likes camera tracking :)

Jim Arthurs

01-01-2006, 11:15 PM

Ah.. the good ole days.. I fondly remember that distinctive sound of a rented BVW-75
*grinding* away for hours with hundreds of those inserts :)

2D/3D integration has been my thing for a lot of years now. I think I'm one of
the only guys on the planet that actually likes camera tracking :)

... and then you discover a shift somewhere around frame 800 or so... and you start all over on a "fresh" area of tape... yep, that's fun. That and sleeping by the deck to make sure it finishes by morning.

I actually like tracking as well, very calming. It's as close to real film "magic" as anything in my mind. And of course, if done well, no one is aware of your work.

Someone else will have to answer Sean's question about the workings under the hood of tracking software. All I know is that it's based on machine vision code work and lots of Trig... two distinct processes in play; One is tracking visual points, either one at a time or many hundreds at once in a brute force automated fashion. The next phase is making sense out of this cloud of points, determining from a few clues very basic things like scale and up/down orientation, etc.

Jim Arthurs

bgundu

01-01-2006, 11:25 PM

I worked at Alias for 6 years and I still would not ever use Maya Live. I think Synth Eyes is the best deal.

Haakon

01-01-2006, 11:54 PM

Jim,

I've never played with tracking before... I suppose I always figured it was one of those things that you'd have to devote yourself to fully in order to become great at it (there's no time to be amazing at everything!), and as such, I don't know/understand everything that goes into the process. That being said, I thought your robot clip was phenomenal... I've watched it probably 50 times now (and I just downloaded the clip a half hour ago!) :thumbsup:

It's amazing to see what can be done these days on a "home PC" - not to mention in such short time. Really, I'm blown away. Your reference to ILM is, I think, more amazing than people probably stop to think about... look how far we've come in such little time... and then just imagine what's right around the corner. If this Red Camera delivers on even half of its promises, it's going to spawn a paradigm shift in filmmaking across the planet. What a great time to be alive.

Anyway, I don't want the unbelievable events of the future to overshadow the topic of today's post and your unbelievable work. I'm completely inspired! :)

Norm Sanders

01-02-2006, 12:37 AM

Wow, Jim, I'm continually impressed by what you can do. Looked near flawless to me ... especially if more time was given to the model to make it more realistic.

Rich Lee

01-02-2006, 12:40 AM

uhg...i sometimes hate camera tracking...but its such a good feeling when you get it locked and everything sticks...

good track man! :thumbsup:

Darkline

01-02-2006, 04:43 AM

I like your short film Jim

What a roto nightmare!!!!

I love to watch what people can do with a small crew and some imagination. Your showreel is very impressive. The short with the running pumpins - is that another one of yours or did you just get comissioned to do some of the effects work?

danm

01-02-2006, 05:11 AM

Great footage Jim.

If people are interested i've done a bit of sniffing around and found this...

Icarus download;

osx - http://files.filefront.com/Icarus_Maczip/;4113431;;/fileinfo.html

windows - http://files.filefront.com/Icarus_209/;3922266;;/fileinfo.html

It appears as though Icarus was sold to (or became) Pixel Farm's pftrack. The links above are for the version that was publicly available but then got removed for download from their official site.

I've never done a 3d track so maybe someone can shed some light on the version on Icarus above.. but for now what the hell, lets all get some funky groovy 3d madness in our films.

danm

01-02-2006, 05:14 AM

heads up.. just found this too;

http://boards.theforce.net/Fan_Films/b10015/17604478/p1/

5 or so video tutorials on Icarus 3D tracking to wet your appetite.

danny dale

01-02-2006, 07:11 AM

heads up.. just found this too;

http://boards.theforce.net/Fan_Films/b10015/17604478/p1/

5 or so video tutorials on Icarus 3D tracking to wet your appetite.

i know the kid (16-17) who did these tutorials... you should check out his site -

http://www.peerlessproductions.com/

he does some pretty amazing stuff.

Jim Arthurs

01-02-2006, 07:57 AM

Your reference to ILM is, I think, more amazing than people probably stop to think about... look how far we've come in such little time... and then just imagine what's right around the corner.

And if we take the time frame back only five years before Jurrasic Park or Twister, the shot simply couldn't be done by the best people in the field.

The only method would be to put out a physical model on a motion control stage, with the camera's video tap feeding a switcher live keyed over the background and endless test passes, and it still would never be perfect.

So, to recap the progress;

-15 years ago: ILM: "Tell the client it's impossible!"
-10 years ago ILM: "We'll put our best team on it, for $$$$ we think we can do it."
-Today: Guy in Basement: "Honey, I'll be up to watch the Dick Clarke New Year's thing in a minute, I'm messing around down here!"

Jim Arthurs

Jim Arthurs

01-02-2006, 07:59 AM

Wow, Jim, I'm continually impressed by what you can do. Looked near flawless to me ... especially if more time was given to the model to make it more realistic.

Thanks, that's a stock model with a single uniform surface. The radiosity lighting gives it the feeling of some shading detail. Again, I was just going to throw up a simple grid but got caught up in the moment...

Jim Arthurs

Jim Arthurs

01-02-2006, 08:02 AM

The short with the running pumpins - is that another one of yours or did you just get comissioned to do some of the effects work?

That was a spec proposal for a Halloween special I wrote called "The Little Pumpkin". It was going to be full CGI, then Stop Motion, and now I'm thinking about minature sets with CGI characters... if it ever gets going again, that is.

Thanks for the kind words!

Jim Arthurs

Antoine_Fabi

01-02-2006, 11:34 AM

Hi Jim,

thanks for the info.

Saw you demo reel. FANTASTIC !
beautifull, tastefull art work !

Greggl

01-02-2006, 11:56 AM

now I'm thinking about minature sets with CGI characters... if it ever gets going again, that is.

If you do go that route, keep in touch! Its expensive and difficult, but its pretty much
my favorite type of work :)

EShy

01-02-2006, 12:08 PM

That tracking looks great
I have 3d/motion tracking as the next thing to get into
there's a hugh community that creates short 3D animations and images and a very big forum on CG society (http://forums.cgsociety.org/) with support for almost any software package (It's like DVXUSER for computer graphics...)

dougspice

01-02-2006, 04:08 PM

Shiloh Arts

01-02-2006, 04:37 PM

i know the kid (16-17) who did these tutorials... you should check out his site -

http://www.peerlessproductions.com/

he does some pretty amazing stuff.

Danny this Kids amazing...17 huh. Well I guess I remember creating dance music at that age. I'll tell you along with the HVX and tracking becoming more available it just makes for more possiblilities.:)

Jim Arthurs

01-02-2006, 11:59 PM

Hey Jim, mind exporting a still of what your track points were? Or did you just clean up the shot and run it on auto? Apologies if you covered this already, I just skimmed the thread and didn't see it. Nice work in any case.

Sure, here's a view of the project... viewport on the left shows the garbage mattes to keep the biker and other folks out of the track, also shows the automated track points. View on right is the cloud of points generated by Icarus that can be exported to your 3D package for reference. Also shows the camera path...

http://ftp.datausa.com/imageshoppe/outgoing/HVX/ICARUS_screenshot.jpg

Jim Arthurs

escozooz

01-03-2006, 12:09 AM

Have you tried PFMatch? Its the embedded Shake version.
http://www.thepixelfarm.co.uk/products/products.aspx?PID=4

I am pretty close to buying it, keeping the tracking in my compositor saves a lot of time. Plus its much cheaper than the stand-alone.

Now I just need an HDRI plug-in and we are set :)

Greggl

01-03-2006, 12:20 AM

Sure, here's a view of the project... viewport on the left shows the garbage mattes to keep the biker and other folks out of the track, also shows the automated track points.

Looks like the auto-track really latched onto a lot of T-Junctions but the features
visible on that frame look nice and solid.

Where is the compression hit on the HVX's image btw? I'm shocked at the amount
of noise in the blue channel for most of the 1080p footage I've worked with. I
generally replace the blue channel with overall luminance if it ends up being
troublesome.

Jim Arthurs

01-03-2006, 04:11 AM

Have you tried PFMatch? Its the embedded Shake version.
http://www.thepixelfarm.co.uk/products/products.aspx?PID=4

No, but I've no doubt it's pretty solid, as Icarus has never failed with a little help, and the FPTrack, etc are all many, many generations past Icarus from the same developers.

In general, I've found 3d tracking (and 2d tracks within the 3d tracker, using all the auto features for pan tilt) more useful than 2d trackers in compositors. They all have their best uses, of course, but 3D tracks can really be useful for the "computer monitor/viewscreen" kind of corner pinning work... the edges can go out of frame with no worries. Of curse if you just need a simple stabilize or track a single feature, no need to go to the 3D trouble...

Fun stuff.

Jim Arthurs

Jim Arthurs

01-03-2006, 04:14 AM

Where is the compression hit on the HVX's image btw? I'm shocked at the amountof noise in the blue channel for most of the 1080p footage I've worked with. I generally replace the blue channel with overall luminance if it ends up being
troublesome.

Funny thing that... the clips I've worked with so far... sometimes the red channel is the noisest and has the most compression! I can't get a bead on it yet.

The Z1 had a very clean blue channel, but that's like saying an ugly dog has a shiny red collar on.

Jim Arthurs

Greggl

01-03-2006, 09:23 AM

Funny thing that... the clips I've worked with so far... sometimes the red channel is the noisest and has the most compression! I can't get a bead on it yet.

SW:Ep2 had a blue channel that was like sandpaper.. 1080p from the first of the
Sony F900s and the bluescreen footage, though fantastically lit, was more difficult
to work with than I would have ever anticipated. Lots and lots of roto work.

VFX is still one area where I prefer film originated footage.

Jim Arthurs

01-03-2006, 09:58 AM

SW:Ep2 had a blue channel that was like sandpaper.. 1080p from the first of the
Sony F900s and the bluescreen footage, though fantastically lit, was more difficult
to work with than I would have ever anticipated. Lots and lots of roto work.

VFX is still one area where I prefer film originated footage.

I heard rumors about that... that's not the kind of PR stuff you hear in, say, Cinefex, where everything is rosey rosey... :)

I don't know how the crew that did Sky Captain survived; original F900 off of tape as well. Lots of soft glow added later can work wonders...

CafeFX guys had major issues on that show, as well on SpyKids 2 & 3 with original HDCam format.. though they said the F950 and SR was a major improvement when they worked on Sin City, as good as any film scan they had seen over the years.

I've seen great film scans and agree, but all the Cineon sequences I've delt with lately have had blue channels that look like the dot pixelation filter from photoshop... mostly high speed night stuff.

Oh, one trick with a bad blue channel; replace with green channel and it works fairly well for all screen colors except saturated yellow and magenta. In fact, doing this can replicate the general look of an "old school" film bluescreen shot. The original Ultimatte was just an electronic version of film's Color Difference Matting technique. Same guy behind it all.

Oh, and the HVX makes good pictures, just to keep on thread.

Jim A.

jung kyu

01-03-2006, 12:23 PM

hi
i did a test on hdr-hc1

http://qlove.co.kr/1/1.mov
it's very simple track test with teapot.
it's not perfect..i need to practice more :(

Shaw

01-03-2006, 02:56 PM

Jim, have you tried replacing the green channel ever? I've tried various approaches in photoshop with the luminance channel but haven't gotten it to look quite right.

Greggl

01-03-2006, 03:13 PM

hi
i did a test on hdr-hc1

http://qlove.co.kr/1/1.mov
it's very simple track test with teapot.
it's not perfect..i need to practice more :(

In shots that lack a lot of parallax, its going to be extremely difficult to get a solid
lock. You've got close to a worst-case scenario going on in that clip. It all depends
on what you're tring to add into the scene however.

Some things to try... you can stabilize the plate, add in your element than add the
handheld back in.. this would be an entirely 2d tracking process. If its a small element
like the teapot, you could track that directly to the element you're locking to, like
the posts with a 3 point track.

If you need a full 3d track for a virtual set extentions, etc.. then most of the auto
trackers are going to fail with their default attempts. You're going to need to make
sure that the software is ONLY exaluating good, solid tracks at various depths and
on different planes. Then you're going to need to weight those tracks to 'bias' the
solution toward the foreground tracks and your good mid-range trackers. Shots like
this are where having 'reference' frames from other angles really come in handy.

jung kyu

01-03-2006, 08:51 PM

thanks...i'll try some diffrent shot..i didn't know about parallax.

Jim Arthurs

01-03-2006, 09:40 PM

Jim, have you tried replacing the green channel ever? I've tried various approaches in photoshop with the luminance channel but haven't gotten it to look quite right.

Yes, but what you want to do is copy the green channel and use it to replace the blue channel. This works well for a range of colors, but you can't get saturated yellows and magenta just doesn't work. Another trick is to just blur the blue channel and half mix it with a copy of the green...

Now, in old school film bluescreen, the "cheap" shots would stop right there, and you can see examples of this with some of the model spaceship photography of the late seventies, early eighties. Keep the ships grey and stay away from yellow/magenta. I recently saw one of the later Superman movies and it was obvious they were just shuffling records around on the elements.

There is a way to return those colors, and that's the basis of Color Difference Matting. You create extra yellow by boosting some densities... good gosh, this is the first time I've thought about this stuff in years. Honestly, until you've had to try a bluescreen shot on film you don't know the sheer pleasure of messing around with sliders and seeing results in front of your eyes.

BTW Jung Kyu, thanks for sharing! I love the feel of the colors of that clip you were working with, and agree fully with Gregg's suggestions. Based on the little range of movement and the small size of the element, 2D tracking is the way to go..

Jim Arthurs

Shaw

01-03-2006, 10:07 PM

Oh certainly. I was just curious about using this methodology with respect to a greenscreen (which I use instead of blue). I wish I knew more about bluescreen effects with film. I'll have to try googling it - see if I can't come up with something. Do you know of any good resources off the top of your head?

Greggl

01-03-2006, 10:59 PM

Do you know of any good resources off the top of your head?

Ron Brinkman's book, The Art and Science of Digital Compositing is a great primer
for the digital process. For chemical techniques its a bit more obscure. My old ASC
manual has a great breakdown of the travelling matte process. The old Kodak
documents relating to their 5295 stock also have some neat info about the
refinements and necessities.

Shaw

01-03-2006, 11:05 PM

Thanks Greggl. I'm quite familiar with the digital process but I've always wondered how it was done chemically.

Jim Arthurs

01-04-2006, 09:06 AM

The absolute best reference on the old school film techniques is Raymond Fielding's"The Technique of Special Effects Cinematogrpahy". Libraries should have it if you don't want to buy, from Focal Press.

As to greenscreen done photo-chemically, well, it wasn't common. Maybe two or three shots that I'm aware off. Some shots in Jaws 3D, if memory serves. The process was bluescreen all the way because the industry understood the steps and the film's emulsion layers were best suited for it.

In fact, for film work, you'd NEVER use a blue painted wall or fabric, the purity of the blue just wasn't high enough. On the movie Willow, ILM used a blue painted cyc for freedom in shooting the actors quickly, and the difficulities in pulling a good matte off of the little fairy characters caused massive problems.

Most film bluescreen work was done with "transmission bluescreen", which is a blue translucent material lit from behind. Passes only 425-450 nanometer blue light. Very pure. I still own a small 4 by 3 foot screen.

Film bluescreen was as much luck, as an art or as a science, and demanded close cooporation with the lab processing all the intermediate b&w material. The slightest change in processing temp, or the age of the proccessing chemicals could spoil a matte or change its density and ruin the shot. Film stock unless it's Estar based will shrink or contract, throwing a matte alignment off. The same shot run through the optical printer on a different day will come out differently. It's totally insane. The big FX houses had their own processing equipment on hand for doing their own comps, it was that critical.

So, when you see a movie from the 70's/80's/90's and the mattes aren't perfect, cut those guys some slack. The optical cameramen were guys at the top of their game, with pressures and finicky tools to work with. When you do see GOOD compositing work, such as The Abyss, or any of the 65mm bluescreen mattes from Boss Films just know how difficult it was to get there.

I've really rambled off topic, but sometimes it's important to know where something started from to know where it's going. Maybe I'll get a thread going somewhere else and we can continue the discussion? I'm sure there's lots of folks with old school experience interested in rambling on...

Regards,

Jim Arthurs

Prodigi

01-04-2006, 03:41 PM

Jim,
My hats off to you! Amazing work!

Taylor Moore

01-04-2006, 05:15 PM

Hey guys really awesome work...
I used to use the 2D/hack 3D tracker in Flame/Inferno; saved my ass or shall I say the director's ass quite a few times....

Shiloh Arts

01-05-2006, 07:17 AM

Jim,

This may sound silly, but what do you mean by Matte (not really sure what that is)? I know call me a newbie:huh: :laugh:

danm

01-05-2006, 03:35 PM

A matte is a user-defined area of exclusion.

I've just ordered my SG35 35mm adapter for my DVX. I'm planning on doing a bit of 3d tracking. Am I going to have a problem tracking footage shot with a shallow depth of field? How does a program like Icarus handle tracking bg data which is out of focus?

Greggl

01-05-2006, 03:59 PM

Its going to be a nightmare. Optically, you've also got a mess on your hands.

For effects shots, you're going to want to minimise your attempt at a shallow
DOF and you're also going to want to shoot a distortion chart to map the
barrel distortion. Removing as much of the optical 'bowing' in the image
before you track, will give you better results.

Ideally, you'll place out tracking markers that can be digitally painted out
after the shoot. The type of marker really depends on the shot and which
tracking software you're using. Some like trianges, some like spheres/circles
and others really 'feed' on crosses.

Whats the shot?

danm

01-05-2006, 04:28 PM

It's a tracking shot of a kid shot from the tideline running along the sand dunes chasing a model glider. The model glider will be the CG element.

He will be in focus and I'll be on a 50mm and 135mm lens.

Any thoughts?

Greggl

01-05-2006, 05:14 PM

Without ground contact, it'll be a lot easier. You've still got some issues that
are going to be hard to deal with. Tracking shots that are perpendicular to
the direction of movement have very little parallax and trackable elements
are going to come on screen, travel across the screen and then go offscreen.
With little parallax, you ususally get bumps in the solution when those
tracks 'turn on' or 'turn off'. This is for two reasons, going offscreen is
the obvious one, but the closer to the edges of the frame they get, the
more they are affected by barrel distortion.

If your bg is mostly sky or seagrass, you'll have very little that is trackable
back there. You never want to track anything thats moving for your
camera solution. So be on the lookout for trackers that latch onto T
junctions where say, two blades of grass cross, and it tracks the junction
where they meet. Waving grasses, etc will also throw your track off.

Your best bet may end up hand tracking the whole thing. Smooth out the
camera jitter with a standard 2d track in you compositing software. Then
either run that through your tracking software or build a simple wireframe
ground plane in your 3d app and animate the camera by hand. In the final
comp, you can put the real world camera jitter back in, but not having it for
the track will make the tracking/3d much easier.

jung kyu

01-05-2006, 08:31 PM

hello
there's something worng with my track...auto track shows good result but when i render from max it shows diffrent result....teapot is still shaking . :(

http://qlove.co.kr/1/333.JPG
http://qlove.co.kr/1/1111.JPG

http://qlove.co.kr/1/22223.JPG

http://qlove.co.kr/1/44444.JPG

http://qlove.co.kr/1/1.avi

Jim Arthurs

01-05-2006, 09:53 PM

Hi Jung!

I can tell one major problem, and that is your camera isn't doing much more than a pan/tilt. This is very difficult for Icarus/PFtrack because it needs to see some dolly or tracking motion to really work well.

Try using setting up the camera to do a 2D track (in Icarus it was originally called Camera Motion/Pan/Tilt/Zoom) and compare the results. There will still likely be some float, and this is caused by yet another issue...

... I *think* you're on a tripod (or a very steady handheld shot) and since the nodal point of the camera isn't exactly on the pan/tilt axis (and understanding a "nodal point" could open up a whole conversation on its own) it truly is a tiny, tiny dolly and track move instead of a pure pan/tilt like the software expects.

This same set up, pointed at a distant landscape would have better results, because the relative size distance between your camera and subject would be much greater than now, both compared to the distance the camera is off nodal. Whew!

What I'd do for the sake of getting a good success on a 3D track is this... hand hold your camera, make sure the image stabilization is off (if it's on for this shot you tried, that's yet another factor in causing a failure) and shoot the scene again, only this time move along the length of your shot, while panning and tilting. Or move toward it from a bit of distance. This will give you MUCH better results!

Give that a try and share back with us!

Jim Arthurs

jung kyu

01-05-2006, 09:56 PM

i just exported to lightwave and it seems to work ok....i'm not lightwave user so i'm still strugling with 3ds max.

http://qlove.co.kr/1/9.mov

Jim Arthurs

01-05-2006, 10:03 PM

i just exported to lightwave and it seems to work ok....i'm not lightwave user so i'm still strugling with 3ds max.

That does seem to line up better... which could indicate that there is some sort of aspect ratio issue with the background image in Max, or some sort of frame-rate difference between Max's frame rate and your clip's frame-rate. I saw a "pop" or two in the motion of the Max clip... sometimes that happens when forcing a clip to a different frame rate.

As to my earlier post... I think PFTrack has advanced quite a bit since it was Icarus... no way you'd have success with a 3D track on something that was basically a 2D pan/tilt...

Jim Arthurs

Edited: Oh, and you need to set your XYZ coordinates up... your ground plane should lie in the same plane as the table top as per this image you sent http://qlove.co.kr/1/1111.JPG so check out the user info on how to do this... it will make things easier to work with in the 3D package...

Jim Arthurs

01-05-2006, 10:13 PM

Its going to be a nightmare. Optically, you've also got a mess on your hands.

Gosh I couldn't agree more. You've got all the optical issues of the HVX lens added to all the optical issues of the 35mm lense (still lenses worse than good cine lenses, but still) and then any alignment issues with the camera and the ground glass. The end result is a distortion solution set way more complicated than it needs to be. I'm not saying it won't work, because I've seen stuff work that shouldn't, and stuff that should work fail, but man...

Gregg, did you ever work with Boyd Shermis? I see you did some tracking on "Gone in 60 seconds"...

Jim Arthurs

jung kyu

01-05-2006, 10:46 PM

i finally did it.. :)

http://qlove.co.kr/1/yes.mov

it was my mistake..i rendered backgound movie file with 3d directly in 3ds max...i thought it would work that way..thanks jim
reminding me about XYZ coordinates

jung kyu

01-06-2006, 12:48 AM

cheack this out...i think this camera is better than HVX.. it records 5mega pix 720p that;s almost film look.

http://www.sanyodigital.com/HD1/features.html (http://www.sanyodigital.com/HD1/features.html)

http://www.sanyodigital.com/interactive_demos.html (http://www.sanyodigital.com/interactive_demos.html)

i might sell my hc1..to buy this..hd1 bit rate is 9mb..it's enough bit rate for 720p and mpeg4 codec is better than mepg2.. sounds too good for $800

Haakon

01-06-2006, 02:18 AM

What does that have to do with the tracking test?

Greggl

01-06-2006, 05:33 AM

Gregg, did you ever work with Boyd Shermis? I see you did some tracking on "Gone in 60 seconds"...

Hehe.. yep. I know Boyd but was book up during Swordfish so only worked with him once.

Ray Smithers

01-08-2006, 02:35 PM

VERY nice work Jim.

I looks like you did not have to "dirty" up the CG very much for a match shot.

I almost cry when I have to crap up great CG when working with SD DV with poor color space.

Am I the first one to say "Can't wait for my HVX-200"?

Thanks for the post Jim.

Ray

Jim Arthurs

01-08-2006, 10:54 PM

I looks like you did not have to "dirty" up the CG very much for a match shot.

Thanks Ray... no, I didn't have to do much more than add some edge enhancement to simulate the detail level that Kaku used and blur it back a bit.

When I match elements to DV 4:1:1, I often change the color space from RGB to YIQ, and literally destroy the chroma sharpness of CG by scaling it to 25% wide and then stretching it back to full screen to fit the luma. That's cruel and unusual punishment for innocent CGI... :)

Jim Arthurs

Anhar Miah

01-09-2006, 07:19 AM

Jim Arthurs

01-09-2006, 10:41 AM

Hey Jim, thats some great talent you got there !

I vote you to start a new Article/thread, basically a "Dummies guide to camera tracking" tutorial right here on DVxuser ! that would be a great help to others here no doubt.

Anhar

(I'm learning 3d max)

P.S when you camera track, how does one tackle the problem of reflection and shadow? and what about HDRI type renders (can you use them in a camera track?)

Thanks Anhar... a topic on tracking is a good idea, and since the tools are becoming very affordable tracking should be part of everyone's CG toolkit.

I think when my HVX ships I'll shoot a couple different styles of background plates and post the footage so everyone can experiment with whatever tracking software you are using.

As to reflection and shadows... if your automatic trackers are failing because of these, it's time to hand track a number of features and use those. Most of the time I find the auto trackers will self cull out track points that don't make sense in the whole of the shot. Any tracks you do by hand are treated as "absolutes" and the rest are suggestions and refinements, if that makes sense.

If you're talking about using CG elements rendered with HDRI lighting tricks, or rendering out CG elements at higher than 8 bit depth, sure, no problem. Tracking is only impacting the camera path, not the actual renders techniques in any way.

Jim Arthurs

Anhar Miah

01-10-2006, 03:49 AM

Cool,

Jim about the shadows and reflections, what I meant to say was, say if you put a shiny CGI object into your footage, how can you "reflect" from the real world ? I don't know if that makes any sense !

Anhar

Jim Arthurs

01-10-2006, 09:02 AM

Oh, gotcha!

A quick and dirty way is to just use a frame of your background plate as a reflection map. You could even paint up a map based off of the colors and tones in Photopshop.

The best way is to create a reflection image on set or location with a digital still camera. This is called a "Light Probe", and there are several ways to do that;

One way is to take a chrome ball or sphere (a cheap kind are "gazing spheres" used in gardens) and back off, zoom in full and fill the frame with it. Crop the image square to fit the sphere and use that as a reflection map. If you take several exposures of the sphere using different shutter speeds you can build up an HDRI image for lighting you 3D scene as well!

I used this technique for reflection maps for my short film "Groom Lake Mistake" which I made back in 2001. In this case I just used the video camera to record the chrome ball (back then I had a Canon XL1).

http://www.imageshoppe.com/groom_lake_mistake.html

This map was also used for lighting the scene for the radiosity effect of natural bounce and shadow.

Here are some links...

http://www.debevec.org/

http://www.debevec.org/ReflectionMapping/

The other way is to take fisheye images with a digital still camera and make a dome image from that. I do this with a Nikon coolpix 995 and the fisheye adaptor.

FYI, this is the same setup John Knoll used to document all the lighting setups and locations on the second Star Wars movie.

Regards,

Jim Arthurs

Isaac_Brody

01-10-2006, 09:08 AM

Jim, thanks for the information. Very useful.

Greggl

01-10-2006, 09:54 AM

FYI, this is the same setup John Knoll used to document all the lighting setups and locations on the second Star Wars movie.

Those reference frames were INVALUABLE during production. They were used
by pretty much every department in some capacity.

When shooting reference and are planning on tracking, a wide angle shot that
includes both the camera setup as well as some detail elements visible in the
footage can be a lifesaver. Just takes a few seconds to snap, but it can save
hours in post when reconstrucing the camera settings.

stabwound

01-11-2006, 12:42 AM

AAieee!

Jim, that was astounding.... that you did it in a couple of hours.

I was expecting a 2D track.... but it's actually 3D, because as we get closer to the robot we see it's side!

I'm under the impression you'd need an actual gizmo ( a wireframe box with ping pong balls at the corners) on the scent (where the robot is) in order to track 3d rotation. Obviously that was not needed.

I've tried 3D tracking with 3DMax, but I have no success so far.

I'll have to look into Icarus... and check out Darkline's tutoria...

Curugon

01-12-2006, 12:30 PM

Terrific work, Jim. I've been using Syntheyes for a while now and I'm very pleased with it (considering how freaking inexpensive it is). Can't wait to motrack some HVX footage, seems like you pulled a really nice solution from that clip.

ericyoung

01-12-2006, 01:38 PM

i know the kid (16-17) who did these tutorials... you should check out his site -

http://www.peerlessproductions.com/

he does some pretty amazing stuff.

Holy cow! Suddenly I feel very old.

The next Spielberg?

Scott Sullivan

01-13-2006, 06:07 AM

Curugon, I hear what you're saying about Syntheyes. I use (version 2006) that with Lightwave and LOVE the results! Needless to say, I'm nowhere near what Jim and some of the others here are getting, but still, it's great.

Speaking of reflections, I just read an article over at Apple's site about a track shot where they had to REMOVE the reflections from a silver dome platter that had the camera crew's reflection in it.

http://www.apple.com/pro/film/andcompany/index2.html

They built a CG room to 'reflect' in the platter, tracked the shot and baddabingbaddaboom, new reflection! They've got a before and after shot in the article.

Great tracking work Jim!

Cheers,
Scott