In Simple Words
Imagine a school testing three teaching methods to improve student graduation rates ten years later. Waiting a decade to update the teaching strategy makes adaptive learning impossible. Instead, the school uses end-of-month reading test scores as a quick intermediate check. If a high reading score guaranteed a higher graduation rate regardless of the teaching method used, reading scores would act as a perfect statistical surrogate, allowing the school to optimize teaching methods in real time.
Research question
Adaptive experimental algorithms rely on short-run outcome signals to update treatment assignment probabilities quickly, but primary policy goals often involve long-term welfare outcomes. If short-term proxy metrics fail to reflect long-term effectiveness, adaptive algorithms can allocate participants toward treatments that lack long-run benefit.
Participants
N = 3,770 jobseekers (1,633 Syrians and 2,035 Jordanians interviewed in rapid follow-up).
Study Design
Empirical testing of the surrogacy condition using linear regressions of 4-month employment on treatment dummies, baseline covariates, and short-term surrogates via Wald tests.
Results
The surrogacy condition held statistically for 2-month employment predicting 4-month employment (Wald test p = 0.575 for Syrians, p = 0.527 for Jordanians). Targeting 2-month employment in counterfactual simulations doubled the employment gains compared to a standard RCT (13.4% vs 12.4%), whereas targeting noisy 6-week employment yielded zero adaptive gains.
Takeaway
Validate short-run outcomes against the surrogacy condition before deploying them as reward metrics in adaptive assignment algorithms to ensure alignment with long-term policy welfare.
Source and citation
See source →Read alongside the research
Plain-English glossary
Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.
- Follow-up
Collecting more information about participants later in a study to see what has happened to them. The term can also mean the length of time researchers keep track of them.
ExampleResearchers interview the same workers three months after a programme ends. That later interview is a follow-up.
GET-IT glossary · Follow-up ↗Read more about follow-up →- Baseline
The starting measurements in a study. Researchers use these to compare what participants were like at the start with what happens later.
ExampleBefore a training programme starts, researchers measure workers’ earnings. That first measurement is the baseline.
GET-IT glossary · Baseline characteristics ↗Read more about baseline →- Randomized controlled trial
A study that uses chance, like drawing lots, to decide who receives each option being compared. Researchers compare the groups’ outcomes to estimate the effect of those options.
ExampleA lottery assigns volunteers to a training programme or a comparison group. Researchers then compare their outcomes.
GET-IT glossary · Randomized study ↗Read more about randomized controlled trial →