Multiple Comparison with the Best (MCB) for Statistical Treatment Selection
Decision-makers evaluating multiple treatment choices often face statistical uncertainty when selecting a single optimal intervention. Single-point estimates can create false precision when multiple options have statistically indistinguishable potential outcomes.
Picture this
Imagine a weather forecasting app rating three different routes to work. Rather than declaring one route definitively faster when all three are within a two-minute difference, the app highlights all three routes in green as equally good choices, while painting clearly congested routes in red.
What the evidence says
MCB routines dynamically categorized interventions based on individual covariate precision; the set of significantly best programmes varied from a single clear optimal choice for some jobseekers to all available programmes for jobseekers with high variance in predicted potential outcomes.
- Who
- Historical estimation sample of N = 460,442 Swiss jobseekers aged 25 to 55 (2001–2003 administrative records); field predictions delivered for N = 18,713 jobseekers across 21 regional employment offices.
- How
- Implementation of Multiple Comparison with the Best (MCB) routines (Horrace and Schmidt, 2000; Frölich, 2008) to partition 6–8 active labour market programme categories into statistically significantly best, intermediate, and worst sets.
What to do
Partition multidimensional algorithmic output into statistical confidence tiers using Multiple Comparison with the Best routines before displaying recommendations to human decision-makers.
From the source
"The statistical precision of the estimates was also conveyed to the caseworker in that the set of all programmes was divided into three groups: The significantly best programmes, the intermediate programmes, and the worst programmes... estimated by Multiple Comparison with the Best (MCB) routines."
Targeting_Labour_Market_Programmes_Results_from_a_Randomized_Experiment.pdf