And The Winners Are.......

Z B Brox said:
Hey, cool! Thanks a lot.

We didn't do too bad, I'm pleased about that! Given the very mixed reviews we got (One guy claiming he'd give us a perfect 10 if only he could vote, others just utterly uimpressed) I wasn't sure how it'd go. It's funny, though, how most of the films' numbers converge. That is, it seems almost universally if people like the cinemetography they also liked the score and the acting. I wonder if that's just overall "professionalism" helping all the scores, or if people are just very prone to the halo effect.

Thanks again for getting these up, nice to see!
Click to expand...

Very interesting ZB, yeah I seem to notice the same thing as I look around at the spreadsheet...oh well, no scoring system is perfect. How does the Academy do it? people registered under different professions get to vote on the category of their profession? I know the grammy's work somewhat like that. A backup vocalist can't vote for best engineered album, and a mix engineer can't vote for best hiphop collaboaration..something like that..


C.R.
 
Last edited:
Yeah I agree.... it seems like people just voted (on average) one number across the board. Whereas on my scorecard it certainly didn't look like that. Some movies were CLEARLY (more than one or two points) better/worse in the different categories. Strange.
 
I believe the phenomenon is called regression toward the mean. So while most people gave scores with more variation for each film--when the scores are averaged, they tend to flatten out in each category. Maybe I'll include the standard deviation next time. :)
 
Barry--

Ha! Nice idea. Just odd to see how little the scores really say about *why* people felt the way they did about a movie.... And really highlights, as you said, the importance of the verbal reviews over the numbers.

That said, if any of you haven't gone and given feedback, go do it now. ;)
 
I didn't find that at all ZB... I went through and highlighted RED everyplace a film did better, and it creates an interesting visual.... and one I can agree with. Apparently our acting was where we suffered most (LOL) and our screenplay is where we did the best overall.
 
I always figured that you're not scoring against an ideal, you're scoring against the other films. So, if a film has the BEST cinematography, you give that film a 10, it was the best out of all of them. It's sad to see that although everyone on the site is somewhat of an amateur filmmaker they were all judging everyone else so harshly. Am I mistaken in my ideas on how people should have been scoring? I'm confused lol


C.R.
 
Actually, I guess the difference lies in when you score. If you score while you watch, you'll have to score against the Ideal film. But after watching more and more films, I found I had to go back and change other scores. As a movie with better lighting came up, or a better script,I would have to re-adjust my entire value model. Did anyone else score in this fashion?

C.R.
 
Last edited:
Hmm, I suppose with us it's fairly clear our lighting suffered (Damn Jim and his messing up the levels at the last second!), but other than that our scores are all about 0.2 from each other. Just seems odd that there aren't really any movies with a CPU-level storyline, but A New Tomorrow (that's us, go watch, review ;D) level cinematography. S'why I suspect a halo effect--if the cinematography is good, the story seems better, and if it's bad, well... ;)
 
Z B Brox said:
Barry--

Ha! Nice idea. Just odd to see how little the scores really say about *why* people felt the way they did about a movie.... And really highlights, as you said, the importance of the verbal reviews over the numbers.

That said, if any of you haven't gone and given feedback, go do it now. ;)
Click to expand...

Well said.

Because I took a small vacation, and have a computer that is coughing up blood, I only finished watching all the films the first time through on Saturday. I am anxious to start watching through all of them again and posting feedback. You will all hear from me soon - unless my computer dies, in which case some of you will hear from me less soon.

I also would like to say that I am very proud of(/honored at) being included with all the other filmmakers in this festival. I have this feeling of kinship- like we all just graduated high school together. Anyone wanna sign my sci-fest yearbook?

~Ian:love4:
 
EditPhish, freakin' cool banner. I absolutely love it!

I can see with our film, the script was apparently the weakest link, which is interesting as it's what helped pull together all the people that wanted to be a part of it.

So my real question is more along of do people know the difference between story & screenplay? I think some people may have voted a screenplay higher if it hit their funnybone, or they thought it was an original idea, more so than the dialogue, pacing, etc. I'm in no way saying ours should have been higher, but it's interesting to see some of the other scripts & how they scored.

And no offense to Trekky, but how does someone score ACTING in a completely animated short that doesn't even have VO talent ... all subtitles. Yet it scored HIGHER than many other films? I mean, what a slap in the face that is for the talent in the other films to hear that people thought they couldn't even act as well as a video game with no sound?
 
mm... my screenplay score was very weak too... which is what I feel is the strongest part of our film... well... oh well... I am happy I did it! On to Hero-fest!
 
Thanks Envision... hey man, DOGS scored better in acting then us :laugh: ... but since it was our weakest link across the board, it's not surprising... though I gotta say, I kinda thought we were better than the dogs (LOL, no offense Hager) -- then again, maybe not! :laugh:

I think overall it looks to me like the scoring is pretty much what I expected. I did vote on each film comparing it to itself... not to others, but is a very few cases I went back and made some minor adjustments. One thing I did do was watch each film a few times, because on a first watching there's just no way to pay attention to music AND sound AND cinematography AND lighting AND story. On the first run-through, your mind is so focus on the story and acting, it's hard to give enough serious thought to all the other things.

A good example was Hiro's film... his use of color was amazing... but it wasn't until the 3rd and 4th time I watched that I really noticed how much thought he'd put into it, because it was so "invisible" (which is what you want!) that it just played into the emotions of the story and on it's own merits, I wouldn't have noticed it.
 
Oh, and Chris Rojas had asked how people were scoring. Myself, I didn't want to have to watch everything through once, then go back & watch them again based on a curve scale that was set by a particular film.

I just had in my mind what I felt perfection was for something (i.e. cinematography, script, etc.), and scored each section on how far from perfection I felt it fell ... VERY few things did I go back and adjust later.

EditPhish, when you said you scored pretty low across the board in acting, how do you know? Was this based on feedback in the thread, or is there a page in that excel file I missed that shows the individual scores?
 
I re-adjusted a fair bit as my understanding evolved, personally. I think I've definitely still got a school-based grade system in my head, where 7 is "average". Eventually (in part because I tend to be a little stingy with 10s), this really limited the degrees of difference between films. Essentially I was only using 6, 7, 8, and 9. So I had to go back and adjust a bit so I could get a little more differentiation between the films.
 
My scoring really went everywhere from 1's to 10s, and I think all but two films got varying scores for different categories with as much as a 3 or 4 point spread (say for example one film got an 6 for cinematography, but a 3 or 4 for screenplay).
 
Envision said:
I've always seen 5 as average, being as it lands in the middle of 1 and 10.
Click to expand...

Sure, *logically* 5/6 should be average, but after 16 years of schooling I've had it pretty well pounded into me that a 75, or "C" is average, and anything below a 60 is a total failure. School is cruel. ;)
 
It's clear from looking at the ballots that people have very different styles when it comes to scoring. I don't think there's anyway to avoid that, even if we posted intricate instructions. I took a year of grad-level statistics and believe me you don't want to hang your hat on the methodology or the data. :) I think the system is generally good enough to bubble some very good films up to the top, but beyond that, there's a lot of noise in the system. I hope between the comments and the scores you get some idea of where your strengths and weaknesses lie.
 
You must log in or register to reply here.
Back
Top