Hierarchical Bayesian Treatment Effect Targeting
RCTClinical Trial
Evaluating policy interventions across diverse population subgroups often suffers from small sample sizes within specific demographic cells, leading to noisy and unreliable subgroup treatment effect estimates [14, 15]. Simple cell-by-cell estimation produces high variance, while completely ignoring subgroup heterogeneity fails to deliver personalized policy targeting [14, 15].
Picture this
Think of estimating school performance across several small classrooms in a district [14, 16]. Instead of judging a single classroom solely on its small test sample or assuming all classrooms in the district are identical, a supervisor calculates a weighted average [14, 17]. The weighting depends on how different classrooms are from one another overall and how many students were tested in that specific room [14, 17]. If a classroom has very few students, its score is pulled closer to the district average; if it has many students, its score relies mostly on its own data [15, 17].
What the evidence says
The optimal targeted policy yielded an estimated 1.7 percentage point increase in employment relative to control (a 35% gain), whereas the optimal non-targeted policy yielded only a 0.6 percentage point gain [21].
- Who was studied
- N = 3,770 Syrian refugees and Jordanian jobseekers partitioned across 16 demographic strata defined by nationality, gender, education level, and work experience [7-9, 18].
- How
- Hierarchical Bayesian model with Beta-Bernoulli data generating process and Markov Chain Monte Carlo sampling (1,000 burn-in iterations and 10,000 draws) to update posterior distributions of success parameters across strata [7, 16, 19, 20].
What to do
Estimate stratum-specific treatment effect distributions using a hierarchical Bayesian model to adaptively update posterior assignment probabilities based on pooled cross-stratum variance and stratum sample sizes [14, 16, 17].
From the source
"At each time period t, the treatment effect of each treatment d in each stratum x is estimated as a weighted average of the observed success rate for d in x and the observed success rates for d across all other strata." [14]
An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan