Research procedure
Randomise shared demographic clusters
- Group voters by targetable demographics
- Block similar clusters; randomise within blocks
- Deliver assigned ads, then telephone survey
Illustrative summary of the source’s public-voter-list design. Study 1 grouped 32,029 voters in 34 towns into 1,220 clusters. Each of 244 five-cluster blocks allocated two clusters to control and three to Facebook ads; no outcomes are shown.
See source →How
When to use
Researchers need to evaluate offline impacts of online advertising without access to proprietary user identification tools. Online ad platforms permit demographic targeting based on attributes like age, gender, and location. Merging these targeting parameters with public voter records allows campaigns and researchers to assign population clusters to ad exposure and measure offline outcomes.
Assign target population clusters to online advertisement treatments using public demographic attributes that match ad platform targeting criteria.
- Obtain a public voter registration list containing voter age, gender, and location.
- Group voters into unique clusters based on shared demographic attributes available in both the voter file and the ad platform.
- Block clusters of similar sizes into groups to maintain a constant treatment-to-control ratio.
- Randomly assign clusters within each block to receive specific online advertising treatments or control conditions.
- Upload targeted advertisements to the online platform configured to deliver ads only to the demographic attributes of treatment clusters.
- Survey registered voters from the target list via telephone after treatment delivery to measure offline outcomes.
Why it works
Clustered random assignment on shared demographic traits permits valid causal inference across target populations at low cost.
Example
In a 2012 state legislative election study, researchers assigned 32,029 voters across 34 district towns into 1,220 clusters defined by age range, gender, and town. Within 244 blocks of five clusters, two clusters were assigned to control and three to Facebook ad treatments delivered over one week.
About the study
Participants
32,029 registered voters aged 30 to 75 across 34 rural, predominantly white towns in a state legislative district, grouped into 1,220 clusters.
Study Design
Clustered randomized field experiment blocking clusters into 244 groups of five by size, age, town, and gender, with outcome evaluation via a post-campaign telephone survey of 2,984 voters.
Results
Exposure to Facebook ads produced a non-significant -1.1 percentage point change in candidate name recognition (95% CI [-0.041, 0.018]) and a 0.8 percentage point change in positive candidate impression (95% CI [-0.017, 0.034]).