Statistical Surrogacy in Sequential Policy Trials
Observational StudyClinical Trial
Adaptive experimental algorithms rely on short-run outcome signals to update treatment assignment probabilities quickly, but primary policy goals often involve long-term welfare outcomes [14, 15]. If short-term proxy metrics fail to reflect long-term effectiveness, adaptive algorithms can allocate participants toward treatments that lack long-run benefit [14, 16].
Picture this
Imagine a school testing three teaching methods to improve student graduation rates ten years later. Waiting a decade to update the teaching strategy makes adaptive learning impossible. Instead, the school uses end-of-month reading test scores as a quick intermediate check. If a high reading score guaranteed a higher graduation rate regardless of the teaching method used, reading scores would act as a perfect statistical surrogate, allowing the school to optimize teaching methods in real time [17, 18].
What the evidence says
The surrogacy condition held statistically for 2-month employment predicting 4-month employment (Wald test p = 0.575 for Syrians, p = 0.527 for Jordanians) [20]. Targeting 2-month employment in counterfactual simulations doubled the employment gains compared to a standard RCT (13.4% vs 12.4%), whereas targeting noisy 6-week employment yielded zero adaptive gains [20, 21].
- Who was studied
- N = 3,770 jobseekers (1,633 Syrians and 2,035 Jordanians interviewed in rapid follow-up) [19].
- How
- Empirical testing of the surrogacy condition using linear regressions of 4-month employment on treatment dummies, baseline covariates, and short-term surrogates via Wald tests [17, 20].
What to do
Validate short-run outcomes against the surrogacy condition before deploying them as reward metrics in adaptive assignment algorithms to ensure alignment with long-term policy welfare [14, 17].
From the source
"This condition requires that the long-term outcome should be independent of the treatment assignment conditional on observable characteristics X and short-run outcomes [17]."
An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan