Originally Posted by tinka2:
“A lot depends on how good their sampling system is as to how good a measure 5000 sets will give you. And one has to concede that a research organisation is going to know their onions on that score. But I would have thought that as the number of variables (channels) and population size has increased, then statistically a greater sample size would be necessary for the findings to remain as accurate as they were when there were only 3 or 4 channels to choose from.”
What's interesting about the subject is that, the larger the population that you're trying to estimate, the better your estimates become! Sounds daft but if you think about it for a second, all becomes clear...
How good is any prediction based on a sample of 5100 homes going to be about what YOU personally watch? Terrible! But it will do better at predicting what your street watches. And better still at what your town watches, and so on.
(To see this used in a SF context, read "Foundation"!)
Statistically, a random sample of 5100 homes would come up with answers that are accurate to about 1.5%. This assumes
1 no so-called 'bias' (or systematic error) in the sample you have (e.g. that you haven't asked a disproportionate number of white males between 16 and 34)
2 that the behaviour of the population follows what is known as a 'normal' (or Bell-curve) distribution. In simple terms, more people behave in an average manner than an extreme manner.
In fact, the BARB panel are specifically chosen to be representative rather than random (so the accuracy is probably a bit better than that quoted, but you can't prove that mathematically).
Anyway given those assumptions, the accuracy figure comes naturally from the maths. (Note that this is a mathematical process, the 'lies, damned lies and statistics' effect comes from when people start to misuse the numbers later on!)
The comment about the number of channels is more valid for the 'hardly watched' channels/programmes. If there's only a small number of sampled people watching something (e.g. 1), a small change in that number (e.g. 2!) gets scaled up to 'Hey our ratings have doubled!.
K