|
September 2011
 Before Your Do it Yourself Survey,
Take a Turn at Our
Game of Chance
We've started to wonder if the availability of low or no cost online survey sites have lulled many non-researchers into believing that they are, in fact, researchers. Think about it. Owning a digital camera doesn't turn a novice into an expert photographer; nor does owning a stethoscope turn someone who failed biology into a doctor. The point is, having the ability to set up and send online surveys doesn't transform someone into a researcher.
Instead, the ability to conduct surveys online without the guidance of a
researcher can be compared to a game of chance. Without expertise, sometimes you
don't know what you missed until it pops up.
The DIY Game of Chance illustrates how the unexpected frequently pops up when you're doing something with which you're unfamiliar.
So before you decide to do your next survey yourself, first consider our game of
chance above, and then make sure you know:
- How to develop a sampling plan: Stratified or random? What groups to include? Exclude? Once you know your sampling frame, do you have the capability to select it in the way that you envision? For example, can you execute a random selection?
- Sample size implications: Depending on what you want to know, how much is enough? How much is too much? Why does it matter?
- Why response rate is important, how to estimate it, and how you have control over it.
- How the following are relevant to your project: cross-tabulations, non-response bias, and mixed-mode. When necessary, can you execute them?
- What questions to ask. Who's guiding question writing--someone who
has been in your department for a couple of years, or someone who builds on the experience of decades of question writing?
- The ways in which the survey instrument demands both art and science: its order, question wording, and response options.
- Whether your service offers the ability to collect and process data in the ways that best fit your survey's goals. Can it be broken out by key segments? Will it be cleaned and checked for logic so that you have the most accurate data possible? Do you know how to accomplish these research tasks?
- Ways to analyze the resulting data and pick up on the subtle insights that will maximize your efforts and give you the information you need to meet your information goals.
If you're not on top of each and every one of the items listed above, you would benefit from working with a researcher. Some may retort, "Any data is better than no data, so we'll do it ourselves anyway." However, that's not always true. The risk of basing important decisions on flawed data or losing credibility in the eyes of the market is just too great to ignore.
Instead of playing the game of chance, we would recommend the survey be delayed until
you have budget to do it right. Keep in mind that professional research companies can often make suggestions that will provide you with expertly conducted surveys within your budget constraints.
|