Craig giving Judy only 57% on average of her scores! 
Keeping an Eye on the Judges Again
2 of 2
Originally Posted by J.R:
“I was confused by Craig on ITT yesterday when he said had he known Judy would be going out he would have given her a 5 but he was convinced she would be staying in so gave her the 4. So which did he think she deserved? I can't see how it made that much difference but that sounded like tactical voting.”
“I was confused by Craig on ITT yesterday when he said had he known Judy would be going out he would have given her a 5 but he was convinced she would be staying in so gave her the 4. So which did he think she deserved? I can't see how it made that much difference but that sounded like tactical voting.”
I think it sounds like the opposite of tactical voting. He genuinely believed she deserved a 4 and he assumed the public would keep her in regardless of what he gave her. But if he had suspected it was going to be her last week he'd have been generous and given her a nice send-off with his best score, going against his better judgement.
Personally, I believe it was her best dance and he should have given her the 5, but that's by the by in this case.
Originally Posted by edy10:
“Thank you for doing this OP
Im convinced that Craig has something agaisnt Simon
; Crazy how he tends to rate Mark higher than him 
when IMO Simon is the best out of the 2.”
“Thank you for doing this OP
Im convinced that Craig has something agaisnt Simon
; Crazy how he tends to rate Mark higher than him when IMO Simon is the best out of the 2.”
I'm so relieved that someone else thinks this. IMO Simon has performed quite a few showstoppers, and Craig in particular always seems rather underwhelmed. I suspect that Craig has an eye on his forthcoming SCD tour, and clearly knows who he wants to sign up!
Originally Posted by Alleycat666:
“One thing I noticed is that the only celeb where anyone other than Craig is less than 100% of the average is Darcey's marking of Frankie. Does she not like her (or does she not like Kevin...).
She definitely seems to have a downer on her.
.”
“One thing I noticed is that the only celeb where anyone other than Craig is less than 100% of the average is Darcey's marking of Frankie. Does she not like her (or does she not like Kevin...).
She definitely seems to have a downer on her.
.”
I think over a few series Darcy has been slightly tougher on the women or generous to the men. Arlene might have done that too. I'm guessing they have a different perspective on what the female celebs should be doing and what the male celebs would be like as a partner.
Originally Posted by davegold:
“I think over a few series Darcy has been slightly tougher on the women or generous to the men. Arlene might have done that too. I'm guessing they have a different perspective on what the female celebs should be doing and what the male celebs would be like as a partner.”
“I think over a few series Darcy has been slightly tougher on the women or generous to the men. Arlene might have done that too. I'm guessing they have a different perspective on what the female celebs should be doing and what the male celebs would be like as a partner.”
Voting Ovaries?
Originally Posted by Veri:
“That Craig uses a greater range of marks does not mean he contributes less to ties.”
“That Craig uses a greater range of marks does not mean he contributes less to ties.”
Not necessarily, but everything else being equal, it does make it less likely that he contributes to ties.
Quote:
“For example, in Blackpool, when there were three ties, one of the reasons Simon and Pixie were tied was that Craig gave them different marks (8, 9) that balanced the different marks Darcey gave them (10, 9).
The same thing happened with Frankie and Caroline's tie, except that it was Len who gave the (10, 9).
So what about the 3rd tie, between Jake and Mark? Craig was in there too, because all of the judges gave both 9.”
“For example, in Blackpool, when there were three ties, one of the reasons Simon and Pixie were tied was that Craig gave them different marks (8, 9) that balanced the different marks Darcey gave them (10, 9).
The same thing happened with Frankie and Caroline's tie, except that it was Len who gave the (10, 9).
So what about the 3rd tie, between Jake and Mark? Craig was in there too, because all of the judges gave both 9.”
One example. Let's take series 11, same judges - over the complete series.
Code:
week couples ties No C No D No L No B 1 15 10 9 10 12 14 2 15 2 6 7 4 6 3 14 8 5 9 6 8 4 13 10 8 7 8 8 5 12 8 9 8 8 8 6 11 5 7 6 6 5 7 10 4 3 5 2 4 8 9 6 8 4 6 6 9 8 4 4 6 4 4 10 7 2 5 2 2 2 11 6 2 2 4 2 2 12 d1 5 0 4 2 0 0 12 d2 5 0 0 2 2 2 13 d1 4 2 3 2 2 2 13 d2 4 0 2 0 0 0 13 d3 3 0 2 0 2 0 avg 3.94 4.81 4.63 4.13 4.44
The above shows, week by week, dance by dance, how many of the added scores per couple were tied. There's an average 3.94 tied scores per dance. We can see the influence of any single judge's voting pattern on the number of ties by ignoring their votes in turn. Overall, 4 judges produce fewer ties than 3 judges, that's what you might expect since the range of total scores possible is extended. But some judges are more influential than others. Ignoring Craig's votes gives an average 4.81 ties, Darcy 4.63, Len 4.13 and Bruno 4.44.
If we (theoretically) had three judges, there would have been around an extra tied score per week if Craig was not there. That's nearly an extra point every week to the bottom couple on the leader board. Craig's voting pattern contributed more to reducing th number of tied scores than any of the other judges. In contrast Len's voting pattern contributed to increasing the number of ties more than any other judge. Ironic since Len complains loudest if any particular couple is "unjustly" in the dance off, yet he is arithmetically more reponsible for the situation than anyone else.
Actually, the effect is more pronounced in the later rounds. If we take only the rounds from week 5 when three low scoring couples have already gone, the average tied scores per week is 2.75. Ignoring Craig would give 4.08 ties, Darcey 3.42, Len 3.00 and Bruno 2.92.
Quote:
“There used to be a similar view that Craig's marks made more difference to leader-board positions than the marks from the other judges, but a closer look showed that wasn't necessarily so.”
“There used to be a similar view that Craig's marks made more difference to leader-board positions than the marks from the other judges, but a closer look showed that wasn't necessarily so.”
Not necessarily so, no. But the evidence in series 11 is that, on average, Craig's marks did more to reduce ties than other judges and actually did make more difference than any other judge to leaderboard positions.
Originally Posted by holly berry:
“Great work - it suggests some interesting trends.
What about Donny? Was he really as bad as he is remembered or was the Donny 10 unrepresentative of his overall scoring profile?”
“Great work - it suggests some interesting trends.
What about Donny? Was he really as bad as he is remembered or was the Donny 10 unrepresentative of his overall scoring profile?”
Thanks Holly. Donny was mostly overmarking - except for Steve, where he undermarked. Here's the relevant page:
http://www.ultimatestrictly.com/series-12-week-3/
Originally Posted by via_487:
“Thanks for doing this OP.
I wonder if Craig marks Simon lower because he thinks he is really very, very good and wants him to improve even more, rather than out of favouritism for Mark.
I like to think that anyway.
Out of all the judges, I think that Craig is the most reliable overall, but even he can get swayed (whether by the moment or the need to balance out the voting figures, who knows?).”
“Thanks for doing this OP.
I wonder if Craig marks Simon lower because he thinks he is really very, very good and wants him to improve even more, rather than out of favouritism for Mark.
I like to think that anyway.
Out of all the judges, I think that Craig is the most reliable overall, but even he can get swayed (whether by the moment or the need to balance out the voting figures, who knows?).”
You're welcome! I've been doing this since series 9 (the year of Alionagate) and certainly Len hasn't been the most impressive in terms of consistency. As has been pointed out by others, Craig and Lisa stuck out like a sore thumb, and I think Bruno has often looked (just about) the most consistent judge. Apart from the odd anomaly though, Craig is pretty good, and at least he makes full use of the range of paddles.
Originally Posted by Veri:
“I don't think those figures do imply bias. A difference of opinion doesn't have to mean bias. Indeed, it could be the other judges who are biased and the judge who's score is furthest from the average was the only unbiased one.
But it's also not clear how you're using the figures to find "bias". What, for example, means that Craig is "down on Simon" but not on Steve or Judy? Craig's score for Simon was 81% of the average, but it was also 81% for Steve, and 57% for Judy.”
“I don't think those figures do imply bias. A difference of opinion doesn't have to mean bias. Indeed, it could be the other judges who are biased and the judge who's score is furthest from the average was the only unbiased one.
But it's also not clear how you're using the figures to find "bias". What, for example, means that Craig is "down on Simon" but not on Steve or Judy? Craig's score for Simon was 81% of the average, but it was also 81% for Steve, and 57% for Judy.”
Obviously, the only way of assessing any one judge is to compare him/her with as many other judges as possible - in this case the other three. And a difference of opinion in judging is a bias - it doesn't necessarily imply anything underhand.
As for looking for bias, you will notice Craig's figures show a downward trend, caused by his use of the lower-scoring paddles. If you take this into account, you'll see that Simon's score of 81% is somewhat lower than the trend would otherwise imply.
Originally Posted by davegold:
“I think over a few series Darcy has been slightly tougher on the women or generous to the men. Arlene might have done that too. I'm guessing they have a different perspective on what the female celebs should be doing and what the male celebs would be like as a partner.”
“I think over a few series Darcy has been slightly tougher on the women or generous to the men. Arlene might have done that too. I'm guessing they have a different perspective on what the female celebs should be doing and what the male celebs would be like as a partner.”
Yes - I've been doing this for a few years now, and Darcey does indeed show a preference for the younger men. I also did it retrospectively for some of the earlier series too, but I can't remember about Arlene. If there's a way of searching this forum for my old threads (can anyone help?), I'll be able to see what results I got.
Originally Posted by Alleycat666:
“One thing I noticed is that the only celeb where anyone other than Craig is less than 100% of the average is Darcey's marking of Frankie. Does she not like her (or does she not like Kevin...).
She definitely seems to have a downer on her.
Other than that, pretty much what you think is happening, but nice to see it layed out before us.
Good work VintageWhine.”
“One thing I noticed is that the only celeb where anyone other than Craig is less than 100% of the average is Darcey's marking of Frankie. Does she not like her (or does she not like Kevin...).
She definitely seems to have a downer on her.
Other than that, pretty much what you think is happening, but nice to see it layed out before us.
Good work VintageWhine.”
Thanks for that - we aim to please!
I do think that Darcey is more strict when it comes to the girls - it's been the same story in previous years.
Originally Posted by alan_tracy:
“
...But the evidence in series 11 is that, on average, Craig's marks did more to reduce ties than other judges and actually did make more difference than any other judge to leaderboard positions.”
“
...But the evidence in series 11 is that, on average, Craig's marks did more to reduce ties than other judges and actually did make more difference than any other judge to leaderboard positions.”
Wow, that piece of analysis was very impressive. Thanks for the effort!
2 of 2
