Make Sure the Proposal Responds to Your Objectives
The availability of convenient and inexpensive surveying methods enables almost anyone to conduct a survey. Because of this, statistics are everywhere trying to prove this or discredit that. Because of the abundance of survey data, their relevance runs the gamut from trivial to crucial during decision-making.
Besides, the reaction to statistics that support polarizing subjects tends to be extreme. People in support of the subject strongly believe the reported data. While those who are against the subject cast a skeptical, even cynical eye over the information. As a result, there are many reasons someone would want to gauge the quality of statistics they see in articles, on websites, and in a company’s advertising.
When faced with a statistic that is important to you, there are several considerations you can make to judge whether the survey the data came from was conducted well.
Sponsor and Objective
The sponsor of the survey and the objective go hand in hand. Learning who sponsored the survey gives you an idea of possible objectives. It also alerts you to if there may be biases resulting from the organization’s mission or core beliefs. When you know the sponsor, you can decide whether you should look further into how the survey was conducted and make sure that accepted research practices were followed.
Who Conducted the Survey
While knowing the sponsor and objective are significant details of a survey, it’s more important to review who conducted the survey to gauge the potential for errors made during the survey process. Generally, three types of organizations will conduct a survey: a professional market research organization that has been hired by the sponsor, the research department of the sponsoring company, or another department within the sponsoring company.
Professional market research organizations
Professional market research companies understand and follow accepted research practices. These organizations won’t risk their reputation to please a single client. Surveys conducted by an unbiased, third-party researcher have an added dimension of credibility. If you’ve never heard of the research company, check out their website to get a better idea of the company and its qualifications. Furthermore, look for professional research association memberships, such as the Market Research Association or the Council of American Survey Research Organizations. Members of these organizations have made a commitment to use accepted research practices.
The sponsor’s market research department
In-house market research departments are likely run by professionals that know the ins and outs of conducting a survey. An additional review of the sampling plan (who was invited to respond to the survey) and question-wording can help you determine whether any bias could have been introduced into the survey.
1. The survey only represents the group of people from which the sample was pulled.
2. The way a question is worded can lead respondents to answer in a specific way.
Another department within the sponsoring company
Surveys conducted by another department within the sponsoring company, and not a research group, do have the potential to be conducted flawlessly. Unfortunately, without enough training that addresses how to avoid the ways a survey can go wrong, many people aren’t aware of all the details that need consideration. As a result, a close look at the sampling plan and question-wording will help you gauge how to judge the statistic. Again, the survey only represents the group of people from which the sample was pulled, and the way a question is worded can lead respondents to answer in a specific way.
When you consider a survey’s sponsor, its objectives, and who conducted it, you’ll be able to make an educated judgment of the relevance and quality of data. You’ll then be able to cite statistics more confidently in your presentations and conversations.