|
June 2011
Chef Pierre is preparing a batch of soup for the attendees of the American Sampling Association's annual conference. More than 400 people are expected. He pulls out
a giant pot and begins. After creating his masterpiece, he sets aside his soup spoons and fills his sampling bucket with soup.
Then he chugs two gallons to make sure he has the spices in his huge batch of soup just right. His stomach is bulging by the time he's taken his last gulp, so he's relieved that the soup tastes great.
What?! Who in the world would fill a bucket to taste test a recipe? When tasting soup, no matter the size of the batch, you simply have to stir it thoroughly and dip in a spoon at random to get a good idea of how it tastes. There's no need to taste more just because the batch is larger.
That's pretty much how it works with sampling for a survey. We're often asked, "I have an audience of 150,000.
How many people do I need to survey in order for the results to be statistically significant?"
Similar to the sample of Pierre's soup, from a statistical perspective, there is virtually no correlation between the size of the population (pot of soup) and the recommended sample size (spoon vs. bucket). When using a randomized sampling technique, 600 responses will represent a population of ten thousand with practically the same precision that it will represent a population of ten million. Results to both would have a maximum sampling error of about
+4
percentage points at the 95% confidence level.
 It's only when the number of survey returns/completed interviews gets within 10%
of the population do we start to see a "benefit" (a lower maximum sampling error
figure). That is, 600 responses that are captured from a group of 6,000 or fewer
members will yield a slightly lower sampling error, but only by two-tenths of a percentage point.
The fact that the size of the population has virtually no impact on the sample size runs counter to common sense, but we see it in practice virtually any time we look at a political poll. Whether the question is who we intend to vote for President, or how we feel about a particular issue, the number of completed interviews/returned questionnaires often ranges from 600 to 1,000, even though the goal of the project is to understand the opinions and plans of those likely to vote, a group of over 100 million.
The key is to ensure that the people asked to participate in the survey were selected at random. Not just Republicans. Or just males. Or just from the South. Choosing at random or on an nth name basis is the key to successful sampling.
Choosing the right size sample for your survey can save you money. There's no need to pay to process and analyze additional responses that have little to no impact on maximum sampling error. Keep Chef Pierre's bloated belly in mind and you'll see
that you can prevent similar budget bloat in your survey.
Learn more about Maximum Sampling Error.
|
