Statistical Surrogate Outcomes in Adaptive Trials

Picture this

Imagine training a racehorse using its running speed over the first 10 meters as the metric for success. If starting speed does not predict who wins the full 1,000-meter race, selecting horses based on that initial split selects the wrong champions. Using a statistical surrogate ensures that short-term intermediate milestones reliably align with winning the overall long-term race.

What the evidence says

The surrogacy condition held for 2-month employment outcomes across subgroups. Counterfactual simulations demonstrated that adaptively targeting 2-month employment outcomes using the Tempered Thompson Algorithm achieved an average employment rate of 13.4%, doubling the employment gain produced by a conventional static RCT (12.4%).

Who was studied

Evaluated across N = 3,770 study participants (1,663 Syrian refugees and 2,107 Jordanians) in Jordan.

How

Empirical validation of the surrogacy condition using Wald tests regressing 4-month employment on treatment indicators conditional on 6-week and 2-month intermediate employment outcomes, followed by counterfactual simulations.

What to do

Conduct formal Wald tests on intermediate trial data to verify that short-term proxies satisfy conditional independence with primary long-term endpoints before using them as target metrics in adaptive algorithms.

From the source

"Instead of only measuring and targeting long-term outcomes, the designer might therefore wish to find a set of short-run proxies, i.e., 'statistical surrogates', for long-term welfare... an adaptive targeted field experiment would therefore be designed in order to target these statistical surrogates."

An_Adaptive_Targeted_Field_Experiment_Job_Search_Assistance_for.pdf

Tags

• surrogate outcomes
• adaptive experiments
• reward signals