Make Sure the Proposal Responds to Your Objectives
The availability of convenient and inexpensive surveying methods enables almost anyone to conduct a survey. Consequently, statistics can be found everywhere trying to prove this or discredit that. Because of the abundance of survey data, their relevance runs the gamut from merely trivial to crucial during decision making. In addition, the reaction to statistics that support polarizing subjects tends to be extreme. People in support of the subject strongly believe the reported data, while those who are against the subject cast a skeptical, even cynical eye over the information. As a result, there are a number of reasons someone would want to objectively gauge the quality of statistics they see in articles, on websites, and in a company’s advertising.
When faced with a statistic that is important to you, there are several considerations you can make to judge whether the survey the data came from was conducted well—no matter where you stand on the issue. The Summer 2010 issue of the Readex Review covered these considerations. Three of the considerations included in that list are the sponsor of the survey, the survey’s objective, and who conducted the survey.
Sponsor and Objective
The sponsor of the survey and the objective go hand in hand. Learning who sponsored the survey gives you an idea of possible objectives and alerts you to whether there may be biases resulting from the organization’s mission or core beliefs. When you know the sponsor, you can decide whether you should look further into how the survey was conducted and make sure that accepted research practices were followed.
Who Conducted the Survey
While knowing the sponsor and objective are significant details regarding a survey, it’s more important to review who conducted the survey in order to gauge the potential for errors made during the survey process. Generally three types of organizations will conduct a survey: a professional market research organization that has been hired by the sponsor, the research department of the sponsoring company, or another department within the sponsoring company.
Professional market research organizations
Professional market research companies understand and follow accepted research practices. These organizations won’t risk their reputation in order to satisfy a single client. Surveys conducted by an unbiased, third-party researcher have an added dimension of credibility. If you’ve never heard of the research company, check out their website to get a better idea of the company and its qualifications. Furthermore, look for professional research association memberships, such as the Market Research Association or the Council of American Survey Research Organizations. Members of these organizations have made a commitment to employ accepted research practices.
The sponsor’s market research department
In-house market research departments are likely run by professionals that know the ins and outs of conducting a survey. An additional review of the sampling plan (who was invited to respond to the survey) and question wording can help you determine whether any bias could have been introduced into the survey. Two reminders: the survey only represents the group of people from which the sample was pulled, and the way a question is worded can lead respondents to answer in a specific way.
Another department within the sponsoring company
Surveys conducted by another department within the sponsoring company, and not a research group, do have the potential to be conducted flawlessly. Unfortunately, without sufficient training that addresses how to avoid the ways a survey can go wrong, many people aren’t aware of all the details that require consideration. As a result, a close look at the sampling plan and question wording will help you gauge how to judge the statistic. Again, the survey only represents the group of people from which the sample was pulled, and the way a question is worded can lead respondents to answer in a specific way.
When you consider a survey’s sponsor, its objectives, and who conducted it, you’ll be able to make an educated judgment of the relevance and quality of data. You’ll then be able to cite statistics more confidently in your presentations and conversations.