We’re often asked, “what’s the best sample size?” Seems simple enough, but it isn’t. Larger is better, because the more responses you have, the lower your margin of error (MOE). However, there’s something to be said for balancing the need for precise information with budget constraints.
For many applications, an acceptable MOE is five percent. At this level of precision, the survey is likely to be both economical and useful.
In addition to considering your acceptable MOE, we need to know who the data is going to represent before determining sample size. If the data just needs to represent the population (such as a publication’s circulation), a smaller sample size will fit the bill. If, however, it is desirable to represent certain groups within the population (such as readers with less experience, different job functions, etc.) then there’s more to mull over.
An appropriate analogy for determining sample size is ordering a pizza. One of the first considerations made is how many people will be eating. Once you know that, you’ll have a better idea of whether a small, medium or large pizza will serve your diners best. Similarly, determining a sample size begins with considering how many groups of people you’d like the resulting data to represent, so you can determine whether a small, medium or large sample size will be best. The following example illustrates some of the many factors that impact the sample size decision.
Looking at Data in Aggregate: Small
If you’re dining alone, a small pizza is usually enough because you’re not splitting it among others. Likewise, a smaller sample size keeps an eye on the budget, yet is sufficient to analyze the data for the circulation. That is, you’re not splitting the responses up into different groups. At the aggregate level, the results achieve a MOE small enough to satisfactorily project to the circulation as a whole.
Splitting Responses into Two Groups: Medium
If you’re dining with another person, you would consider a medium pizza to make sure there would be enough for both of you, because splitting the small pizza in two would leave both diners hungry after the last bite is had. The same holds true if you’re trying to project the results over two different groups. Dividing a small number of responses into two groups results in a MOE larger than would be practical for decision-making. Upgrading to a medium sample size increases the responses and lowers the MOE at the segmented level.
Splitting Responses into Four Groups: Large
Now if you’ve got a crew of four to feed, you’re looking at a large or extra-large pizza to make sure no one has to grab a sandwich on the way home. Likewise, if you’re projecting a survey’s results among a number of groups, you’ll want a large enough sample size to make sure there will be enough responses from each group to yield a MOE on which you’d be confident basing decisions.