Keeping an Eye on the Judges (again!)

VintageWhine

25-10-2013

I've done this for a couple of years running, so I thought I'd do it again:

Each judge is scrutinised for the cumulative score they give for each couple, and each of these is compared against the average for all 4 judges - and the result is expressed as a percentage of the average score. This enables you to see at a glance which couples each judge is or isn't favouring.

PHP Code:


			
Craig  Darcey Len    Bruno



100%   104%    97%   100%  Natalie    (average score 34.0)

 96%   102%   102%   102%  Sophie     (average score 32.5)

 96%    96%   106%   102%  Abbey      (average score 31.3)

 89%   102%   109%   102%  Susanna    (average score 30.5)

 91%   105%   100%   105%  Ashley     (average score 29.8)

 94%   101%   104%   101%  Patrick    (average score 28.8)

 93%   101%   109%    98%  Ben        (average score 25.8)

 85%   104%   109%   104%  Rachel     (average score 25.0)

 93%   105%   102%   102%  Deborah    (average score 24.8)

 73%   106%   119%   103%  Mark       (average score 24.5)

 92%   105%   105%   100%  Fiona      (average score 24.0)

 67%   102%   121%   111%  Julien     (average score 20.8)

 66%   118%   118%   100%  Dave       (average score 18.0)



 66%    96%    97%    99%  lowest

100%   118%   121%   105%  highest

 87%   104%   108%   102%  overall average

 93%   102%   104%   102%  average for female celebs

 81%   105%   112%   103%  average for male celebs

From the above, we can see that:

1. Julien and Dave are greatly favoured by Len (Darcy also favouring Dave), but Craig really doesn't think much of them at all. Edit: same goes for Mark.

2. There's a lot of agreement regarding the top 3.

3. Len favours Susanna, Craig doesn't.

4. Len favours the blokes, Craig doesn't.

5. Len is the most generous (but is all over the place), Craig is the least generous, and Bruno is by far the most level-headed judge(

).

Feel free to add to the analysis!

DerChef

25-10-2013

Very Nerdy but interesting reading

I would like to see a breakdown over all the years of SCD ballroom versus Latin.

I reckon overall ballroom has scored much higher on average especially for tail end Charley's and Charlene's

VintageWhine

25-10-2013

Thanks, DerChef. Ballroom vs Latin stats would indeed be interesting - sounds like a lot of work though. Maybe when I'm at a loose end (or two).

By the way, I should have said that the results are based on stats from the brilliant ultimatestrictly.com.

kittenkong42

25-10-2013

Thanks - this is interesting

.

And I agree with the

for Bruno, but I suppose he would balance out quite well as being the midpoint for Craig & Len...

VintageWhine

25-10-2013

Originally Posted by kittenkong42:
“Thanks - this is interesting .

And I agree with the for Bruno, but I suppose he would balance out quite well as being the midpoint for Craig & Len...”

Thanks, for that. And I think you've got it right there, he's fairly consistently the mid-point. Actually, his scoring is more sensible than his behaviour!

fatskia

25-10-2013

Its interesting to look at but drawing conclusions is difficult.

Generally the women will have had more of a previous dance background. They also are led in the dances which is usually more of a benefit than a disadvantage.

The judges are supposed to differentiate between the celebs. When you have a range of 10 to use, you can do that better by using the whole range - Craig is the closest to doing that. Given the small range that Len uses, his variation from the average is even more notable.

There is an inference that the average score is the correct score in comparisons? It may not be.

Taking those things into consideration Craig looks better on scoring (or ranking).

wazzyboy

25-10-2013

I always think that there is broad consensus on placing if not scoring, so that they can as individuals mark someone over a broad-ish range and still place them where most judges if not all agree.

Monaogg

25-10-2013

Originally Posted by fatskia:
“Its interesting to look at but drawing conclusions is difficult.

Generally the women will have had more of a previous dance background. They also are led in the dances which is usually more of a benefit than a disadvantage.

The judges are supposed to differentiate between the celebs. When you have a range of 10 to use, you can do that better by using the whole range - Craig is the closest to doing that. Given the small range that Len uses, his variation from the average is even more notable.

There is an inference that the average score is the correct score in comparisons? It may not be.

Taking those things into consideration Craig looks better on scoring (or ranking).”

To be fair on the women as Ginger Rogers said "I have to do all the same steps but backwards and in heels".

Elan

25-10-2013

Originally Posted by Monaogg:
“To be fair on the women as Ginger Rogers said "I have to do all the same steps but backwards and in heels". ”

That is a brilliant quote thank you!

Thanks for the stats work, it looks like a lot of work but is very interesting.

DiamondBetty

25-10-2013

Love the stats, wonder if Len and Bruno's DWTS scores show the same patterns?

VintageWhine

26-10-2013

Thanks for the observations. Yes, the women are generally more versed in the art of dance, so the generous (but inconsistent) Len will appear to be more favourable to the women than the others - that would probably explain that.

You're correct to say that Craig's variation is due to his use of the full 1-10 scale. Likewise, for the opposite reason, I would expect Len's variation from average score also to increase as you go down the table. However, both judges show a lot of variation within the general trends. Whereas, Bruno seems to show a consistency in staying with the "centre of gravity", scoring-wise. But of course, just by being near the average, he'll naturally have a tighter "grouping" of percentage figures anyway, so it's not totally straightforward to draw conclusions, as you say.

Do note that I'm making no assumptions about the "correctness" of the average values. All I'm doing is showing up trends and inconsistencies in each judge's scoring relative to the others. So, for instance, it's easy to see that Craig is more critical (we knew that anyway!), but, more importantly, you can see when a particular judge has favourites, and not-so-favourites (er... Len?).

To be fair, so far, at least, this series seems somewhat less controversial (from the point of view of judges' bias) than last year, or, particularly, two years ago (Aliona-gate).

Originally Posted by Elan:
“That is a brilliant quote thank you!

Thanks for the stats work, it looks like a lot of work but is very interesting.”

Originally Posted by DiamondBetty:
“Love the stats, wonder if Len and Bruno's DWTS scores show the same patterns?”

Thanks for the thanks!

I must say, I wouldn't know about DWTS, as I've hardly ever watched it, and I'm not sure I can find any results data either. So, I'll probably have to leave that question unanswered.

Kaereste

26-10-2013

Thanks for this - brilliant! I love me a bit of stats!

Supports what I have suspected - Len just doesn't like Natalie.