Cooking with Scanner Darkly

[Matt Grunau]

Hidy ho all.

I am making a short for a friend of mine, and it is about cooking stuffed squid for Christmas. It's going to be an odd comedy type thing, but the reason I am using this footage is because I am lazy and didn't feel like shooting anything different.

I found a way to sort of mimic the look of "A Scanner Darkly", though it is still rough, and I still am refining it.

There are two before and after shots in it, each around 7 seconds, so the whole thing is about 28 seconds. But you should get the idea.

Looking for feedback, though I know I haven't defined the edges well enough yet. Though for only Photoshop and AE, it's coming along nicely. There is no tweakage to the base footage, it is straight out of camera. And it is playing at 29.97 instead of 24.

http://www.paramm.com/cartoon.avi

http://www.paramm.com/cartoon.mov


The .avi is DivX and 7 megs, the .mov is 26 megs.

EDIT: Here are a couple of stills, not part of the footage:


http://www.dvxuser6.com/uploaded/6995/1192245593.jpg


http://www.dvxuser6.com/uploaded/6995/1192245663.jpg

[Luis Caffesse]

Nice job Matt!
It's hard to tell too much from the clips I saw - would have been nice to see a wide shot (like the still you posted) wiht a before and after...but the shots I saw looked really cool.

How are you doing this?
Are you actually having to roto, or are you doing something else?

[Matt Grunau]

Hey Luis. I know you can't see nearly as well from the shots that I put up what is going on, but I didn't want to use video of him talking as I am would want to get his permission first. Later today (Sat) I'll post something else with a few close up shots of a guy talking, shot in 24P.

No, no rotoing, just a highly modified Photoshop action, consisting of two other separate actions used for batch processing of file sequences from AE and then reassembled in AE. Photoshop is doing the main brunt of the work, as would be needed for something like this. AE simply doesn't have the options and filters available for this kind of thing, and if you could do it in AE, your list of track effects would be over 20 or so per track. As it stands, it still takes Photoshop around 8-12 seconds to process each frame.

The problem is I need four or five different and seperate actions depending on the shot. The one now works pretty well for mediums to close ups, but not quite as well as I would like on wides. That's mainly because it is standard def. High def would look much better in the wides, since the frame size is so much larger, and this was an action made for photography. That's one of the reasons I had to alter it so much.

[DeVi| D0do]

Nicely done!

Looks a little too real to me... like maybe it has too much detail and too many colours. Or rather too many shades of the same colour.... I think it should look more vector-y. The stills look a lot better than the video.

Also, I'd try lowering the frame rate quite a bit... Perhaps even to about 12fps. Will probably really help to sell the cartoon look.

[Matt Grunau]

Yeah, it's too smooth. 23.97 may fix that, or I may slam it down to something like you suggested.

And it's not that there are too many colors, just that they are not in defined areas with boarders. That's one of the issues I am having problems with. That calls for even more adjustments and additions to the PS Action, and is proving to be quite the headache. I'll keep on plugging though.

I'll keep you guys posted.

[Christopher Barry]

Matt, very hyper real, a cool look. I would definitely go with what DeVil said. Since it is 30p, try a 300% speed up render, then slow that rendered file down to say 33% using hold frames, not frame blended. If the source was 24p, then try the 12 fps 50% slomo straight up.

Not to deviate from making your own cartoon filter process, in the Creative Cow AE Tutorial Podcast section, there is a three part series on turning Video to Cartoon Look. It definitely looks more posterised (having issues between CU and wides) and includes black line overlay effect. A lot of work involved, and requires AE and Illustrator, together with hacking a script in the Illustrator program, lol. I did my head playing with this stuff a while a go.

I saw death rising from the earth, from the ground itself, in one blue field.

[Anhar Miah]

Matt great work!

would it be possible to do a layer that has just the edges, like embose but with the whites keyed out,

Then if you where to composit this layer ontop of what you have done already, would you then not get the "hand drawn" pen outline?

Tell me what you think.

Excellent work, keep it up.

[Matt Grunau]

Matt great work!

would it be possible to do a layer that has just the edges, like embose but with the whites keyed out,

Then if you where to composit this layer ontop of what you have done already, would you then not get the "hand drawn" pen outline?

Tell me what you think.

Excellent work, keep it up.

Yes you can, but it does not isolate color regions. It only will give you borders, pretty much based on Luminance, as is the nature of the software. In order to do what you suggest, I would have to have Photoshop recognize and effectively outline each color differential. Bear in mind this is still just a test to see if it will work. I'm still tweaking and adding to the Photoshop action, but trying to include that degree of color recognition and seperation would be one hell of a headache.

There may be a work-a-round. If there is, I'll find it, and let you guys know.

[Matt Grunau]

Not to deviate from making your own cartoon filter process, in the Creative Cow AE Tutorial Podcast section, there is a three part series on turning Video to Cartoon Look. It definitely looks more posterised (having issues between CU and wides) and includes black line overlay effect. A lot of work involved, and requires AE and Illustrator, together with hacking a script in the Illustrator program.

I would like to see that tutorial. Though bringing Illustrator into the fray with Auto Tracing, and God knows what else involved would become a three app production.

I'de like to avoid that. It may be that I can't, to get those edges defined, but man, I don't want to involve Illustrator.

[Tom Marshall]

Hey Matt, are you using the Median filter or no?

[Christopher Barry]

I agree Matt, 2 or 3 apps, yuk. I would prefer to make a node based flow in Digital Fusion doing everything in one pass, if I was to again attempt this look.

I would consider re-shooting using 1/250 or at least 1/120 shutter on 24p, then do the post 200% speed up render out from AE and then a 50% slow down (repeat frame) in AE to give you an image sequence for PS. Your sample footage shutter speed is unfortunately very slow. Look at the hand moving in the MS/CU, to remove the foil, it is totally motion blurred, thus it will be difficult to be able to easily generate edge/outline detection from that part of the footage, and a PS action edge detection would likely fail there. Similarly, surroundings of the shoot need colours to be distinct between talent and background, to help you find that edges. For edges, perhaps add to your action chain, at the end, a PS action using:
Brush Strokes >> Accented Edges: Edge Width 2, Edge Brightness 18, Smoothness 5.

As you know, Scanner D was more than a 'push the render button' plugin, however, I do like the look you have created. The more I think about this effect, the more it does my head in.

[Matt Grunau]

I agree Matt, 2 or 3 apps, yuk. I would prefer to make a node based flow in Digital Fusion doing everything in one pass, if I was to again attempt this look.

I would consider re-shooting using 1/250 or at least 1/120 shutter on 24p, then do the post 200% speed up render out from AE and then a 50% slow down (repeat frame) in AE to give you an image sequence for PS. Your sample footage shutter speed is unfortunately very slow. Look at the hand moving in the MS/CU, to remove the foil, it is totally motion blurred, thus it will be difficult to be able to easily generate edge/outline detection from that part of the footage, and a PS action edge detection would likely fail there. Similarly, surroundings of the shoot need colours to be distinct between talent and background, to help you find that edges. For edges, perhaps add to your action chain, at the end, a PS action using:
Brush Strokes >> Accented Edges: Edge Width 2, Edge Brightness 18, Smoothness 5.

As you know, Scanner D was more than a 'push the render button' plugin, however, I do like the look you have created. The more I think about this effect, the more it does my head in.

I agree that the shooting of it is pivotal, but remember this was a test only. As for DF, the node based editing is almost as bad a headache for me as anything. I can't wrap my head around that.

Slamming it into 24 or 12 fps would help, as would setting the shutter to something nice and crisp.

Seperate lighting would be a huge help, but again, I was looking for a way to take footage I had and make it look cartoon-ish. I think I came pretty close, though the 30p cadence is killing it.

I'm sure Scanner Darkly was more than a button push, and I didn't mean to imply that it could be done so. I only meant I found something that was getting near, using programs most of us have access to.

There is no way without advanced scripting and image analyzation of color value that I can (at this time) isolate the regions I need to and give them that cell shaded type border that is so essential to pulling off the look.

But, I am also working with standard DV rez, and someone working with higher would have better lattitude and results.

The main point is that effect is now becoming more accessible to more of us with fairly standard software. Hell, if you have of copy of Photoshop, any NLE will work.

Someone give me a 2 or 4k plate and I can work a hell of a lot more magic.

I don't think I can pull of the look completely without something like Illustrator with it's avalability of fills, strokes, and gradient maps, but I'm gonna try to keep it to just Photoshop and AE.


Thanks for the input, and I will put it all to the test and make something even better.

[Christopher Barry]

Hey Matt, sorry if I came across the wrong way. Hard to express one's self in text, and be concise and articulate. Meant to only present the best parameters I could think of, in a perfect world.

Yes, DF on a timeline/timing basis is not my favourite either. I have to now learn AE, starting soon.

Sorry again, I did not mean to imply you were making a one pass Scanner D plugin.

Shawn, with his RED posted this (http://www.reduser.net/forum/showthread.php?t=5184) thread over at REDuser, so maybe try his 2k H.264 file. That should be a good test, as it has the tree foliage factor, camera motion/pan, children running through (lots of motion blur) and some distinct flowers at the end of the shot.

I think you will end up making about 3-5 presets, all with minor changes to factor in image detail, and some variations of the preset based on shot type and objects in the shot.

Just some thoughts, thinking out loud. cheers man. :beer:

[Larry Rutledge]

Looking good Matt, pretty impressive. I just came across this today, thought you might be interested in it: http://www.digitalanarchy.com/toon/toon_main.html

[Matt Grunau]

Hey Matt, sorry if I came across the wrong way. Hard to express one's self in text, and be concise and articulate. Meant to only present the best parameters I could think of, in a perfect world.

Yes, DF on a timeline/timing basis is not my favourite either. I have to now learn AE, starting soon.

Sorry again, I did not mean to imply you were making a one pass Scanner D plugin.


I think you will end up making about 3-5 presets, all with minor changes to factor in image detail, and some variations of the preset based on shot type and objects in the shot.



No sweat, I didn't think you came off that way. I am in the process of making seperate actions for the different shot types, but the wides are still the hardest, since there simply isnt enough pixels to outline with, and not enough info to analize. I'm going to keep on pluggin though.

Again, I want to keep this Photoshop and AE only so when it is finished I can share it. Most here who do video will have P-Shop and AE, but probably not Illustrator.




Looking good Matt, pretty impressive. I just came across this today, thought you might be interested in it: http://www.digitalanarchy.com/toon/toon_main.html


Thanks Larry. That post is good, but you can see to much posterizing in it. That completely destroys the look, and just seems harsh to my eye. Maybe though, I can find a way after the whole thing is done to use a blend of Posterizing and then Find Edges and then use only the Find Edges results to give the outline stroke.

[Matt Grunau]

Hey Matt, are you using the Median filter or no?

Sorry Tommy, I didn't see this post earlier. No, the Median filter is not part of the Action. Well, not yet. I have been looking at that as a possibility, but am trying to figure out where I could use it. It may help.

[Matt Grunau]

Just an update, using Posterize Time and forcing it to 20 fps makes a hell of a difference. Also, I have been somewhat successful giving boarders to the needed areas with filters like Find Edges, Posterize (which is not used in itself, but used to break up the image and then with Find Edges gives a decent border) on a seperate layer then set to multiply.

I am rendering a new and different clip from a seperate project now shot at 24, and will post it asap. It will be about 15 seconds, and will be close ups and mediums.

[Larry Rutledge]

Thanks Larry. That post is good, but you can see to much posterizing in it. That completely destroys the look, and just seems harsh to my eye. Maybe though, I can find a way after the whole thing is done to use a blend of Posterizing and then Find Edges and then use only the Find Edges results to give the outline stroke.

Yea, I feel ya. Thought maybe it might give some ideas if nothing else. One other thing I just thought of. Did you ever see Norm Sanders SpyFest entry, Without Provocation (http://www.dvxuser.com/V6/showthread.php?t=99258)? He did a similar technique, also going to PS and running action(s) against the frames. Might have some input to help...or not :)

[Matt Grunau]

Yea, I feel ya. Thought maybe it might give some ideas if nothing else. One other thing I just thought of. Did you ever see Norm Sanders SpyFest entry, Without Provocation (http://www.dvxuser.com/V6/showthread.php?t=99258)? He did a similar technique, also going to PS and running action(s) against the frames. Might have some input to help...or not :)

I took a look, and that's pretty similar. But it is a little harsh, the black borders are too thick. But, I'm going to go over that thread and see if I can get some more info.

Can't find a place to download the damn thing.

[Matt Grunau]

New clip.

I refined the Action, and used two close ups. Much better results. Footage was shot 24P, and then Posterized to 20fps. A hell of a difference.

Still, this would work better with high def footage. I need those pixels.


http://www.paramm.com/bob.avi

http://www.paramm.com/bob.mov

4 megs on the DivX avi, 13 on the Quicktime.

[Tom Marshall]

It's interesting with the highlights... maybe a bit distracting though, I'm not sure. That's really the only thing that looks a bit strange to me. Possibly smooth them out somehow?

How was the lighting set up on the shot? Is it a really hard light? (The key light)

[Larry Rutledge]

I took a look, and that's pretty similar. But it is a little harsh, the black borders are too thick. But, I'm going to go over that thread and see if I can get some more info.

Can't find a place to download the damn thing.



We are working on setting up permanent links to festival archives. In the meantime you can access it here:

http://www.dvxfest.com/spyfest/

[Matt Grunau]

It's interesting with the highlights... maybe a bit distracting though, I'm not sure. That's really the only thing that looks a bit strange to me. Possibly smooth them out somehow?

How was the lighting set up on the shot? Is it a really hard light? (The key light)

Key light was a series of spotlights about 20 feet above him (it was a conferance room, with lighting for), back lit with fluorescents. Yes, I just wrote that, and yes, it was a nightmare beyond what any of us should go through.

I actually liked the highlights on his forehead, they came out remarkably defined, seperated, and crisp. Finally.

I also have been thinking that a garbage matte would help tremendously. The background is static, and being able to adjust its properties (and lose the noise) would really help.

Plus, this was put into 20fps. That may make things too stroby for ya, and mayhaps that is bringing some visual issues.. I'm still trying to find a medium point for that.




Larry, thanks for the link, though if you wait the five seconds, you get an error message. It works fine just with right clicking (take that Mac bitches!) so you may want to look into that.

Or not.

[MattinSTL]

I love the look... I'm very impressed with this. I think you're getting very close to something amazing.

[Christopher Barry]

Looking great, Matt. Of course many different looks can be developed to taste. :thumbsup:

I would like to see the 24p dropped to 12p then played out back to 24p, by repeating each frame. Just curious to see how it affects the cartoon feel. Thanks for sharing.

[Matt Grunau]

Looking great, Matt. Of course many different looks can be developed to taste. :thumbsup:

I would like to see the 24p dropped to 12p then played out back to 24p, by repeating each frame. Just curious to see how it affects the cartoon feel. Thanks for sharing.

As asked for:

http://www.paramm.com/1_12.avi

DivX 4 megs

http://www.paramm.com/1_12.mov

38 megs

Doubled frames.




And at 24P not put into a 20 fps comp.

http://www.paramm.com/1_24.avi

DivX 4 megs

http://www.paramm.com/1_24.mov

38 Megs



I put a couple of new adjustments to the Action, and all the clips have been tweaked in AE, Levels and curves. Both lightly used.


Watch, enjoy, give feedback.


I am this close to getting it. The problem is still the wide shots. If someone would donate some HD footage, I would love to put that to the test. Some nice wide angle stuff to test.

M.

[Larry Rutledge]

I am this close to getting it. The problem is still the wide shots. If someone would donate some HD footage, I would love to put that to the test. Some nice wide angle stuff to test.

Why not grab some sample footage from the reduser.net forum?

[Matt Grunau]

Why not grab some sample footage from the reduser.net forum?

I may do that. in the meantime, WATCH THE VIDEOS AND COMMENT!!!!!


:Drogar-BigGrin(DBG)

[DeVi| D0do]

This is looking great Matt!

Still looks a little too real to me... I'd prefer to see more posterizing. Something more like this: http://www.illustratortechniques.com/imitating-a-scanner-darkly.html (the final product I mean, not the process...). That's more of a personal preference though.

Nice work!

[Luis Caffesse]

Matt - ran across this today -
http://www.addictedtodesign.com/blog/?p=19

Might give you some more ideas...though honestly you seem to be doing really well on your own.

[greeches]

ppl on couch look great. What if you put a layer underneath, turn the opacity down on top layer and delay it a frame or two to give it a floaty look? May help in cartoony look?

Would love to hear details on the PS work....

Great stuff!

[Christopher Barry]

As asked for:

http://www.paramm.com/1_12.avi

DivX 4 megs

http://www.paramm.com/1_12.mov

38 megs

Doubled frames.

M.

Thanks for the 24>12>24 version, Matt. Looking really good. Depending on the texture within the content, it can look hyper real, and my mind starts to interprete it as real. Then other more textured footage, it looks more cartoon like. I am mesmerised by the look you created, observing it. :thumbsup:

[Matt Grunau]

I would like to take the time to say thanks to all of you who have commented, given advice, and weighed in with all of the great suggestions.

It has helped my refining process quite a bit.

I have now petty much locked the Action, for better or worse. I feel there is nothing much more I can do to it.

All of you have made this small project of mine into something much larger.

Any suggestions or donations of other video clips to be used are welcome. I'm pretty much tapped with what I have. I've done 60i, 30p, and 24p. The content has not always been the greatest, but that is what I had to work with.


I may give the Action straight out, or I may give you all a link to the base Action I started with and let you have your own way with it. It would help if you looked at each step, understood what is going on and why. That way, you can make changes and new actions to suit each shot, and at whatever rez you shoot in.


Peace out.


M.

[Luis Caffesse]

That's awesome Matt.
Looking forward to it.

I've been playing around with FCP & AE- trying to see how far I can get without going into Photoshop and doing frame by frame processing (well, I guess FCP and AE do it frame by frame as well, but you know what I mean). I've got NOTHING that looks anywhere near as good as your stuff, but if I get a chance I'll post a clip later today just to show what might be possible using only FCP & AE standard filters.

[kai]

Wow Matt, very (very) slick. I gotta see those actions! ;)

[Matt Grunau]

ppl on couch look great. What if you put a layer underneath, turn the opacity down on top layer and delay it a frame or two to give it a floaty look? May help in cartoony look?

Would love to hear details on the PS work....

Great stuff!

Did that, and had better luck just slamming it into a 20 fps posterized comp.

The thing is, it's all about the fps playback. The best results so far are shooting at 24p and playing back at 20p.

otheriwse it is too "smooth", and look video-ish.

I would really like the flexibility of having that be a top layer, but with that route, nothing seems to look right.


Thanks though.

[nicholasraeburn]

Lovely work Matt NOW WHERE THE HELL ARE THOSE ACTIONS:Drogar-Happy(DBG):