Research procedure
Ask outcomes before media recall
- Stop ads; call voters in random order
- Ask candidate evaluations, then policy recall
- Ask media recall and platform use last
Illustrative summary of the source’s telephone questionnaire order. Interviewers do not know treatment assignment. Calls may be live or automated; outcome questions precede online exposure questions to avoid priming. No effects are shown.
See source →How
When to use
Researchers need to measure the impact of online ad campaigns on voter attitudes without alerting voters to the connection between the poll and the ads. Asking about ad recall or internet media usage before candidate evaluations primes respondents and distorts outcome data. Placing candidate recognition and impression questions prior to media recall items captures unbiased voter evaluations.
Measure candidate evaluations through telephone surveys that ask outcome questions before inquiring about media exposure.
- Halt all campaign advertising during the telephone polling period.
- Select telephone numbers of registered voters from the voter list in random order.
- Call voters via live interviewers or automated interactive voice response systems without disclosing treatment assignment status.
- Ask key dependent variable questions first, including candidate name recognition, positive or negative candidate impressions, and voting intention.
- Ask specific candidate policy issue recall questions following candidate evaluation items.
- Ask media recall and platform usage questions at the end of the survey to avoid priming respondents.
Why it works
This ordering avoids priming bias and yields valid causal estimates of candidate recognition and favorability.
Example
In Study 1, polling firm AMM Political Strategies completed live telephone interviews with 2,984 registered voters over three days following a Facebook ad campaign, placing name recognition and candidate impression questions before online ad recall items.
About the study
Participants
2,984 registered voters aged 30 to 75 across 34 towns in Study 1, and 3,557 matched registered voters aged 18 to 64 across 8 counties in Study 2.
Study Design
Post-campaign telephone outcome polling (live interviews in Study 1, automated IVR calls in Study 2) administered immediately after a weeklong Facebook ad campaign, with interviewers blinded to treatment assignment.
Results
In Study 1, ad exposure yielded no significant effect on candidate name recognition (-1.1 percentage points, 95% CI [-0.041, 0.018]) or positive candidate impressions (0.8 percentage points, 95% CI [-0.017, 0.034]).