aikyam school

Welfare Contrasts in Policy Choice

Evaluating experimental interventions requires distinguishing between adaptive learning gains realized during trial execution, total potential welfare gains from an optimal targeted policy, and welfare gains from an optimal non-targeted policy compared to baseline controls.

Picture this

Think of evaluating a new school lunch menu against the old default menu. Welfare contrast 1 checks if adapting choices during the school year fed kids better than flipping a coin every day. Contrast 2 measures how much health improves if every single student gets their exact personalized ideal meal instead of the old default. Contrast 3 measures how much health improves if every student gets the single best overall group meal instead of the old default.

What the evidence says

Contrast $\Delta_1 = 0.002$ (95% CS: [0.000, 0.004]), showing negligible in-sample adaptive gain due to muted 6-week treatment effects. Contrast $\Delta_2 = 0.017$ (95% CS: [0.001, 0.034]), demonstrating a 35% gain in employment for the optimal targeted policy over control. Contrast $\Delta_3 = 0.006$ (95% CS: [-0.015, 0.027]), showing half the gain for an un-targeted policy.

Who
N = 3,770 Syrian refugees and Jordanian jobseekers in urban Jordan.
How
Bayesian hierarchical posterior expectations evaluating three formal welfare contrasts ($\Delta_1$: adaptive vs equal random assignment; $\Delta_2$: optimal targeted policy vs control; $\Delta_3$: optimal non-targeted policy vs control) on 6-week wage employment.

What to do

Calculate formal welfare contrasts ($\Delta_1, \Delta_2, \Delta_3$) comparing observed adaptive allocations, optimal targeted policies, and non-targeted policies against baseline controls to isolate targeting gains from adaptive design gains.

From the source

"We estimate that the optimal targeted policy has a treatment effect on six-week employment that is one percentage point larger than the optimal non-targeted policy..."

An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan

Tagged

Nearby findings