I was looking at BARB's site earlier on and I noticed that in your average week, E4's top rated shows get around 500,000 viewers.

Now am I the only one this sounds a bit odd to? 500,000 people is about the population of two medium-sized british cities, about the same as 250-500 large secondary schools, about a quarter of the viewing figures that Hollyoaks supposedly gets.

Surely that many can't be watching E4, the channel of friends repeats and spin-offs of channel 4 shows?

Not sure how the episodes go but don't things like Hollyoaks get a first viewing on E4, and looking at the latest week, they're getting about a third of the C4 audience for that. Sounds about right to me?

K

BARB results are quite accurate.

And 500,000 viewers is only 1% of the population, so the number is not outlandish

The margin of error on BARB figures is less than 3%.
So if BARB reports 500,000 viewers, then the actual figure is somewhere between 485,000 and 515,000.

Given that E4 is a free to air channel on DTT, 500,000 seems more than likely.

I think the BARB-quoted margin of error (+/-1.4% IIRC) may be based on the total viewing population (ie. about 20m during peak periods), so might be +/- 280,000.

However, I've never seen that explicitly stated anywhere...

K

this is the link i've posted in the past that explains how it works.

basically, the margin of error = 1 divided by the square root of the number of people in the sample.

so i'm not sure the total viewing audience comes into it Kenny - or at least that had always been my understanding anyway.

using that formula, the margin of error would be about 1.4% based on a sample size of 5,000.

Iain

The problem comes with the small audience sizes. If the error is +/1.4% of the predicted audience for a programme then, for example, Animal Planet's top programme gets about 50,000 viewers. 1.4% of that implies an accuracy of +/- 700 viewers. Given that each BARB viewer represents 5000 viewers, 700 seems a bit too precise!

Hence my wondering about what the accuracy figure means in those circumstances...

K

the low audiences thing does throw up some funny things - but that's my understanding of the maths.

Iain

Yes, when i worked in TV, we would sometimes see these huge audiences in the middle of the night in the Ulster region. Literally Coronation St audiences for things like the teletext that came on at 2am. This is beacause Ulster is so small and a few people asleep in front of the telly could distort the Ulster total.
It would have negligible affect on the Network total however.

I agree with the basic stats for predicting, say, the total audience share for the day but I've never seen a statement (by a qualified statistician, rather than an enthusiastic amateur like me!) about how the margin of error should be applied in these circumstances.

K

It is just univerally accepted mathematics

Yes, and I agree with them, but for low audiences of individual programmes, it doesn't make sense to just apply the overall margin of error to their stats. Something else must be involved...

K

on the one hand i'd be inclined to agree, but on the other i wonder if that's really any different from someone else just not getting that it can be accurate at all with just 5,000 in the sample.

but ultimately it just does, because that's how the maths works.

if it's a universally accepted mathematical formula, need it suddenly just stop working at a given point? dunno.

Iain

Well, the basic formula for margin of error being dependent on sample size rather than population size is based on the assumption that the population is infinite, if I remember my A-level maths correctly. 20million is close enough to infinity for our purposes, but, where the population starts to get "small", something else comes into play but I can't remember ever being taught what it was!

K

AC Nielsen in Dublin miscalculated the figures for minor channels over a period of time in 2007. When it emerged, some channels threatened to sue sue them for lost revenue. So the commercial stakes are high for companies providing ratings information. The biggest risk a company like Barb has, is that its statistical sampling is not representative of the population as a whoile.

Over 1m people tune into Big Brother Live on E4 on launch night.

As we are moving into a digital age more and more people are getting digital TV so its available to more people so its not surprising that E4 is getting those sorts of figures.

I don't believe that BARB is accurate and I don't think it can ever be proven either way. Its still interesting to look at viewing figures.

As a confirmed sceptic in these viewing ratings...

Do they vary the 20m viewing audience depending on time of day? It is just about plausible that there really are this number watching the box in the peak evening period, but it must be much lower (and biassed to women) during the day and very much lower over night.

Viewing figures always amuse me. They might say that 15m people watched Coronation St (or whatever) on Christmas Day but nobody I know will ever admit to it... Those people who volunteer to have boxes fitted are presumably confirmed addicts anyway, they would get a totally different picture here - 15 minutes of BBC News (24) a day, the occasional snooker frame, and nothing else at all.

And on the flip side there are always programmes where it appears almost everyone you know watched it yet it gets small figures.

Yes, if you look at, say, last night's numbers for BBC1:

Eastenders - 8.9m (39%) => 22.8m total between 7:30-8:00
Holby City - 5.7m (23%) => 24.8m between 8:00-9:00
Waking The Dead - 5.9m (23%) => 25.6m from 9:00-10:00
BBC News at Ten - 5.7m (29%) => 19.6m from 10:00-10:30

K

Kenny - out of curiosity, i emailed that Robert Niles bloke to pick his brains :

I've quoted your explanation of how sample sizes and margins of
error work in discussions about tv viewing figures in the UK. ie how can
they be so accurate if there are only 5,000 people in the sample.

Can you shed any light on if this holds absolutely true in the case of
very small audience figures?

ie, if a programme is quoted as having an audience of just 50,000, is that
really accurate to within +/- 700, given that each individual member of
the sample represents 5,000 viewers?'

this was his reply :

'Yep.

Remember, we're not starting from 50,000 and working down to a sample that still represents the whole. We start at zero and work up to the point where we don't need to add any more people to make the sample significantly more representative.

In other words, the sample size is a function of the desired margin of error, not the population being sampled.'

if that helps!

Iain

Thanks for following this up and I'm not sure it helps me to understand what's really happening in this case. In most other statistical situations (e.g political polling), a small number of outcomes is being predicted and those results lie within the bulk of the normal distribution. In the case of the audience numbers for small channels and progs, they live at the edges of the normal curve (ie beyond the 95% confidence limits) and I think things start to go awry.

What is interesting is that Neilsen in the US (when they can be bothered to quote it at all) quote their margin of error as a number of viewers (+- 0.3pts, where 1pt=1.1million viewers) rather than a percentage of the programme's audience. See http://publiceditor.blogs.nytimes.co...lsen-footnote/

But I need to think about it some more...

K

I've thought about it a bit more and came up with this...

BARBs panel is demographically balanced rather than being purely random. This is good for sampling the big 'cross-demographic' items (like the terrestrial channels and their programmes). However, imagine a programme that is specifically targetted at a small demographic (at say a specific 1% of the audience). Now, is the sample size 5000 or is it the "demographic 1%" (ie only 50)? If it's 50, then the margin of error becomes +-14% of that audience figure, so for an audience of 50,000, the MOE is +-7000 (which, to me, makes more sense).

If that principle is correct then I think all the Sky subscription-only channels and their programmes should be thought of as not being based on a sample size of 5000, but of the proportion of the panel that has access to them. So, if only 10% of Sky subscribers have National Geographic, then the BARB panel probably only has 250 members that have access to it and so its MOE should be +-6%.

K

i can just about follow that Kenny - although i'd still have thought the sample size was the sample size. (as it were.)

if a programmes is aimed at a particular 1%, then isn't it likely that a high proportion of that 1% would watch? in which case if you only use that 1% of the total sample size to generate you're estimated audience and margin of error, isn't that going to distort things?

Iain

Your maths is flawed.
If 10% of the population has access to Nat Geo, and 10% of the sample also has Nat Geo, then the data will be just as representative as for a universally available channel.

The audience share, yes, but it's the margin of error I'm intersted in...

K