Sampling Error's Effect on Research Results
Maximum Sampling Error is the ± figure you see associated with survey results ("Results are subject to a maximum sampling error (MSE) of ± 4% at the 95% confidence level"). That means the chances are 95 in 100 that the results you get from the survey are within 4 percentage points—higher or lower—of the true percentage for the entire population.
Let's say you conducted a survey asking which feature respondents liked best. 54% said feature A and 44% said feature B. It would appear to be a clear mandate for feature A, but a look at how the number of responses impacts data precision. Depending on how many responses were included in the analysis, the story isn't so clear.
Maximum Sampling Error is primarily based on the number of responses the survey yields: the more responses your results are based on, the lower the error. Unfortunately, as you can see from the graph below, the relationship is exponential rather than linear. For example, in order to cut the MSE in half, you need to quadruple the number of responses.
