Pre-Post surveys look into how things change after some type of event takes place.
The event, sometimes called a treatment, can be almost anything, although we usually talk about Pre-Post research in terms of the effect of advertising. However, Pre-Post could be applied to a new employee benefit program, a new PR campaign, or a patient awareness study.
The point is this: You want to measure how things change, whatever those “things” are, be they awareness, usage, or attitudes after a treatment.
There is a challenge to Pre-Post research, but, that makes the study set up unique.
If it’s not done right you risk having murky data. In some cases, we have to live with the ambiguity, but it’s best not to if possible. The challenge relates to the sample frames. One frame is ideally a control group. A control group is a stand-alone group that is not exposed to the advertising, new program or whatever our treatment is. The other frame represents those who are exposed. Let’s call this our normal sample. If there is no control, we won’t have the best possible situation to calculate the change.
We’ve seen situations where some firms are unwilling to set up the control for fear of missing out on a sale. In other circumstances, there is no reasonable way to isolate a control from, say, an advertising or PR campaign. In those cases, while not ideal, you can still run a Pre-Post but with the warning that the change measured will be contaminated.
Getting started, the Pre-study must be done before the treatment is applied.
Let’s say it’s an ad campaign for a new product. In other words, develop the baseline (Pre) measures before the advertising is released. Then at some point in the future, there will be a Post study of the same groups after the advertising is applied. Presumably, those in the group who received the advertising will report a higher level of awareness. However, it may also be that those in the control, those not exposed to the ad campaign, also report a change in awareness. This change can be attributed to “noise.” Perhaps control members heard about the product by word of mouth advertising or misreported awareness? In any event, there invariable is some of this noise and the more exact value of change can be determined by comparing what happened in the control versus the normal sample.
Rate of change in control: +2%.
Rate of change in normal: +10%
Final rate of change: +8%
If you are considering a Pre-Post study, keep in mind one of the most important elements of the project is the sample frame design. Use a control if possible to weed out noise and have a better picture of the change your program makes.