Fill in the Holes in Your Story—Total Audience Surveys with Multiple Lists
You may end up with holes in your story if you neglect to answer "Who?" while planning your survey.
Since surveys unlock the answers to important questions, nothing brings more energy to a survey project than brainstorming the questions you want to ask. It’s exciting to think of all the information you could soon have at your fingertips to help you drive decisions or make sales.
On the other hand, it’s been our experience that nothing dumps cold water on a survey effort like discussing a sampling plan. The truth is, though, it’s just as important to give extensive thought to who should be the source of new-found knowledge. The typical and off-the-cuff remark of "we have a list" is no longer enough once you consider the complexity of today’s media mix. Consider the following plans:
We’ll just email surveys to the print subscribers for whom we have emails. OK, but then your results will only represent that subset of print subscribers, and no one else. Not enewsletter subscribers, not those who get your digital edition, not web registrants, etc.
We’ll do a mailed survey to our print subscribers. OK, those results will represent all print subscribers, but do you want to know about the preferences and profile of your enewsletter subscribers or digital edition readers, etc.?
Note that surveys can be conducted using only a print subscriber list, and if that is the only group you need to represent, then read no further. More likely, though, you want to learn about a variety of channels and limiting the focus to the print audience will only provide you with a small part of the story.
Your brand may find itself under pressure to allocate the right mix of resources across your channels. Discovering the balance is difficult, yet crucial for success. Before making important decisions, it’s essential to understand the dynamics behind your total audience instead of trying to put the story together like a jigsaw puzzle that’s missing a few, but vital, pieces.
On top of your own pressures, your advertisers need to decide their marketing expenditures. Without data about your total audience, they’ll make assumptions—and not necessarily the correct or most profitable assumptions for you.
So, think back to the "who do you want the data to represent?" question. Ideally, the total audience, right?
You want to:
- Get a sense for how all the properties work together.
- Show exactly how readers are using each channel.
- Understand reader expectations of each property, including frequency.
- Confirm or refute some of the assumptions you’ve made about each channel (for example, have younger readers truly turned their back on print?).
Unfortunately, gathering information for a "Total Audience Survey" isn’t as simple as just shooting out questions to a single list of people. Instead, serious consideration needs to be given to the following issues during the survey’s initial planning stages…not after the data has been collected.
Which channels will be included? It’s likely that you have one list of print subscribers, a separate list of enewsletter subscribers, etc. Multiple lists do add complexity to the sampling process, but incorporating all the channels into the survey will result in data that represents them all.
Once the data is collected, the data must be weighted to make sure the level of overlap and duplication is understood and quantified. See the box below to learn why this is so important.
Because of its complexity, a survey across channels is not for do-it-yourselfers. When you’re working with a research company, let them take the lead and advise you of the best way to proceed when it comes to incorporating all your lists into the sampling plan and determining the right way to weight the data.
In order to achieve a Total Audience Survey, a number of considerations need to be made in the early stages of the survey project to avoid having to piece together what’s really going on. The additional time spent during planning will yield a clear and accurate story that represents your entire audience.
Here’s an example that illustrates the importance of data weighting.
Our goal is to learn the total audience’s preferences for format of information.
A survey is sent to samples obtained from two lists:
50,000 enewsletter subscribers &
10,000 print magazine subscribers.
We receive:
200 responses from the enewsletter subscribers &
200 from print magazine subscribers.
10% (20 responses) of enewsletter subscribers prefer a printed format over an emailed format.
90% (180 responses) of print subscribers prefer a printed format over an emailed format.
If we just combine the data, we end up believing that 50% (200 of 400) of the audience prefers a printed format.
Unfortunately, our calculation neglects the underlying populations (there are 5 enewsletter subscribers for every print subscriber) as well as the potential for duplication (enewsletter subscribers can also be print subscribers).
Once we weight the data considering population counts and duplication, the results no longer over-emphasize the print subscribers’ responses and accurately represent the true audience mix.
