Planning a survey requires many steps and decisions along the way: “How many people do I need to survey? How am I going to distribute the survey?” And, while people often figure out what questions they want to ask, many overlook the importance of expert questionnaire design. It’s natural to believe that anyone can create a survey. Heck, many of us have been asking questions since we were two years old.
However, when you’re trying to ask questions with the purpose of reliably representing a large group of people and, perhaps, even making high-stakes decisions based on the answers, it’s important that your questions avoid bias, are not open to personal interpretation, and gather the information you think they do. In short, ask clear questions that gather straightforward answers instead of riddles that require mental gymnastics.
Questionnaire development is more of a science than an art, and reliance on people with research experience can make the difference between a lackluster or downright disastrous effort and one that turns you into a hero. Here are just a few examples of how attention to the questionnaire can impact the overall project.
Ready to Go:
Recently a customer provided us with a ready-to-go survey, which we used as a starting point for the final version. Of the 15 questions in the the survey we received, the majority required edits for respondent usability or for clarity. Data collected from the ready-to-go survey would have been misleading—effectively meaning nothing. Furthermore, if the survey sponsors didn’t realize how unreliable the data was, they would have been better off making decisions by flipping a coin. At least then they would have known the gamble they were taking, instead of erroneously relying on baseless data to help inform their decisions.
No One Likes to Do the Math!
Since you’re asking your respondent to do you a favor, it’s important that the questions you ask are easy to understand, and your instructions are easy to follow. You want to leave your respondents feeling positive about their experiences. In a past survey, respondents were asked to provide percentages for 11 categories, which, when added together would equal 100%. It’s no surprise that almost 1 in 5 respondents skipped that question. Missing out on the answers of that many people increases the potential for error associated with the results, weakening data reliability.
Make it Easy and Intuitive:
You also want to make it easy for your respondents to understand and carry out your instructions. Since they can save space and group similar questions together, a table or matrix format is often used in questionnaire design. Unfortunately, this format may not contribute to accurate data collection because these questions tend to get skipped or be completed incorrectly. For one survey, almost a quarter of respondents simply didn’t comply with the instructions when presented with a matrix question. On the other hand, when each question was asked separately in another survey, the non-response was much lower.
During the early planning of your survey, consider how the questionnaire will be developed and keep in mind the importance of strategic question writing. Furthermore, if you feel you have all the other details figured out and just need help with questionnaire development a survey research firm, such as Readex, should be able to assist with that step alone.