Weighting under-represented demographic strata increases their influence in statistical estimates to match actual platform demographics.

Download image · 1200 × 630 PNG

Research steps

Weight estimates to ad-viewer demographics

  1. Compare age and governorate distributions
  2. Combine demographic and assignment weights
  3. Fit weighted models; compare unweighted estimates

Illustrative summary of the source-described post-stratification of 3,910 Egyptian endline respondents. The reference population is female Facebook users who saw recruitment ads. Weights combine demographic ratios and inverse assignment probabilities, not a national-population guarantee.

See source →

How

When to use

Online study samples recruited through social media ads often suffer from demographic skews, such as over-representing younger or urban participants. Post-stratification weighting adjusts survey data to align sample distributions with broader platform populations. This method ensures that experimental findings remain representative of target digital demographics.

Apply post-stratification sampling weights to survey dataset estimations to match target platform demographic distributions.

  1. Obtain baseline demographic analytics of the overall population reached by promotional digital advertisements.
  2. Calculate demographic joint distributions across age brackets and geographic governorates for both advertisement impressions and survey respondents.
  3. Construct individual sampling weights equal to the product of inverse treatment assignment probability and demographic post-stratification ratios.
  4. Estimate treatment impacts using weighted generalized least squares regressions incorporating the constructed weights.
  5. Compare weighted treatment estimates against unweighted baseline models to evaluate generalizability across platform users.

Why it works

Rebalancing sample proportions corrects for participant self-selection bias without requiring additional field data collection.

Example

In Egypt, researchers applied post-stratification weights to 3,910 experimental respondents to match the governorate-age distribution of Egyptian female Facebook ad viewers, demonstrating that treatment effects remained stable across weighted models.

About the study

Participants

3,910 female Facebook users in Egypt in the weighted endline sample across governorates and age groups.

Study Design

Weighted generalized least squares (WGLS) regression reanalyzing RCT experimental outcomes using weights combining inverse probability of treatment assignment with post-stratification weights matching Facebook advertisement demographic analytics.

Results

Post-stratified reweighting confirmed main experimental findings with minimal changes in effect sizes, including a 1.024 standard deviation increase in video recall and a 0.211 standard deviation increase in resource knowledge.

Source

See source →