The impact of the running order on the Eurovision results (with some maths)

Afternoon all,

So I've been reading up far too many academic papers over Christmas and New Year around the running order changes, serial jury evaluations, sequential order bias... (yeah, it was that or Miranda, I still think I won).

In short, I can translate that to English and have some nice articles coming up in the next few months. To start with, I;ve posted thoughts, references, and statistics on the running order's impact and contribution to the final score.

ESC Insight: How much influence does the running order really have?

For those who want the answer, the running order contributes to 5% of the final score.

Comments

  • MissMidgardMissMidgard Posts: 5,756
    Forum Member
    Interesting stuff, great piece.

    Someone with more knowledge than me can probably answer this: is the solution to this problem not to simply open the phone lines at the beginning of the contest? So that earlier songs won't be 'forgotten' by the viewers?
  • [Deleted User][Deleted User] Posts: 220
    Forum Member
    Interesting stuff, great piece.

    Someone with more knowledge than me can probably answer this: is the solution to this problem not to simply open the phone lines at the beginning of the contest? So that earlier songs won't be 'forgotten' by the viewers?

    Short answer is no, see Wandi Bruine de Bruin 2004, "Unwanted Serial Position Effects In Jury Evaluation". There was no statistical difference between end of sequence and step by step judging from juries.

    (Of course in the real world the diaspora would vote when lines opened, no matter the song)
  • steeleuro_wolfsteeleuro_wolf Posts: 13,336
    Forum Member
    ✭✭
    Excellent piece Ewen, I'm a bit of a statistics nerd myself so it's interesting to look at the contest from that point of view. Looking forward to the follow-up articles.
  • SaturnSaturn Posts: 18,971
    Forum Member
    ✭✭
    I think the running order has a lot more impact than 5%

    I think you'd get a more realistic result if you compared semi final results to final results for qualifiers, in this way you can compare 2 performances for the same song rather than the previous reports which just said songs near the end are likely to get more points.

    I'd use these as prime examples....

    Netherlands 2004 - 23/23 146 points, 7/24 11 points.
    Latvia 2005 - 5/25 - 85 points, 23/24 - 153 points.

    There are lots of other smaller examples, Armenia and Greece in 2008 going from being ahead of Russia to way behind after getting poorer draws.
  • rosetta12rosetta12 Posts: 4,010
    Forum Member
    ✭✭✭
    Saturn wrote: »
    I think the running order has a lot more impact than 5%

    I think you'd get a more realistic result if you compared semi final results to final results for qualifiers, in this way you can compare 2 performances for the same song rather than the previous reports which just said songs near the end are likely to get more points.

    I'd use these as prime examples....

    Netherlands 2004 - 23/23 146 points, 7/24 11 points.
    Latvia 2005 - 5/25 - 85 points, 23/24 - 153 points.

    There are lots of other smaller examples, Armenia and Greece in 2008 going from being ahead of Russia to way behind after getting poorer draws.
    But in these cases the differences are probably also partly due to voting countries being different in semis and final, and voters within the countries too, as semis usually have a smaller audience than the grand final.

    One example would also be us in 2011 - 3rd in semi 1 with 103 points, but 21st with 57 points in the final, when performing 1/25... But, like I said, that's likely due to other factors than just the running order.
  • SaturnSaturn Posts: 18,971
    Forum Member
    ✭✭
    rosetta12 wrote: »
    But in these cases the differences are probably also partly due to voting countries being different in semis and final, and voters within the countries too, as semis usually have a smaller audience than the grand final.

    Not in those years when there was only one semi final ;)

    I wouldn't say the voters are actually that different. 1,000 people is considered a good sample on which to base a national election result after all.
  • rosetta12rosetta12 Posts: 4,010
    Forum Member
    ✭✭✭
    Saturn wrote: »
    Not in those years when there was only one semi final ;)
    Yeah, it would apply better since 2008 ;), but I don't actually remember if all countries were allowed to vote in the semi before that..?
    Saturn wrote: »
    I wouldn't say the voters are actually that different. 1,000 people is considered a good sample on which to base a national election result after all.
    Don't know about election results ;), but I have sometimes compared especially our semi results and final results, by points & country, and there have been quite big differences. Anyway I do think it does affect points (votes) if e.g. 10 000 people in a country are watching a semi and 1 million the final...
  • SaturnSaturn Posts: 18,971
    Forum Member
    ✭✭
    rosetta12 wrote: »
    Yeah, it would apply better since 2008 ;), but I don't actually remember if all countries were allowed to vote in the semi before that..?


    Don't know about election results ;), but I have sometimes compared especially our semi results and final results, by points & country, and there have been quite big differences. Anyway I do think it does affect points (votes) if e.g. 10 000 people in a country are watching a semi and 1 million the final...

    All countries could vote in the semis between 2004-2007 but not all of them actually did.

    It's harder to compare the results now because some of it is kept secret and the jury results seem to vary a lot between the semis and the final - Lithuania 2011 for instance.

    Portugal 2008 is also a good example of a song doing very well and then much less well with a poorer draw.

    I'm not sure about Finland but I know in the UK we tend to give high points to the same countries in the semi as we do in the final. The only odd one I can remember is poor Charlotte getting decent points and Ani Lorak nothing only for that to be reversed in the final.
  • ollie1004ollie1004 Posts: 754
    Forum Member
    ✭✭
    You've been reading far too many academic papers over Christmas and New Year and "For those who want the answer, the running order contributes to 5% of the final score."

    I'd kindly suggest that for all your reading of academic papers, I may have been expecting a little more than a single line in a spoiler tag! :/
  • [Deleted User][Deleted User] Posts: 220
    Forum Member
    ollie1004 wrote: »
    You've been reading far too many academic papers over Christmas and New Year and "For those who want the answer, the running order contributes to 5% of the final score."

    I'd kindly suggest that for all your reading of academic papers, I may have been expecting a little more than a single line in a spoiler tag! :/
    Well I thought a cut n paste of the 1750 words into the tag would have been excessive
  • danprodanpro Posts: 127
    Forum Member
    Is there anyway to view more detail on the statistical analysis for us statistics nerds?

    The actual effect of position is not really important.
    It is our perception of the effect that makes it interesting and exciting.
    If we think position is relevent then a random draw is fun to debate but ultiimately feels fair.
    If the draw is pre-determined, we will feel manipulated and there will be suspicion of "favourites" even if there are none.
  • phill363phill363 Posts: 24,311
    Forum Member
    ✭✭✭
    I do agree that there shouldn't be a fixed running order but for some songs a late draw can kill off its chances, I would love to see where Ukraine would have came if it was drawn first in 2012.
  • [Deleted User][Deleted User] Posts: 220
    Forum Member
    danpro wrote: »
    Is there anyway to view more detail on the statistical analysis for us statistics nerds?

    The actual effect of position is not really important.
    It is our perception of the effect that makes it interesting and exciting.
    If we think position is relevent then a random draw is fun to debate but ultiimately feels fair.
    If the draw is pre-determined, we will feel manipulated and there will be suspicion of "favourites" even if there are none.

    Cough... article has the full name, title, and year of the papers used. Google is your friend.
  • SaturnSaturn Posts: 18,971
    Forum Member
    ✭✭
    phill363 wrote: »
    I do agree that there shouldn't be a fixed running order but for some songs a late draw can kill off its chances, I would love to see where Ukraine would have came if it was drawn first in 2012.

    It would have done even worse. That song had next to no public support. 24 televote points in a semi with friends like Lithuania, Estonia, Belarus, Portugal and Georgia voting is abysmal.
Sign In or Register to comment.