In Simple Words
Imagine asking local basketball coaches to predict how many games a new training routine will win before the season starts. Because the coaches are excited about the program, they predict the team will win 20 games, whereas actual match results show the team only wins 1 game. Eliciting predictions before the season starts allows managers to measure how far off expert intuition is from actual performance and make better decisions in the future.
Staff forecasts for six-week cash-arm employment
- Local Amman staff
- Median forecast 20% employed
- Senior New York staff
- Median forecast 7% employed
Observed forecasts by 16 local and four senior staff before trial results. These are predictions, not achieved employment rates.
See source →Research question
Policy interventions and development programs are often selected based on subjective expert opinions, but experts may hold overly optimistic or inaccurate beliefs regarding treatment effectiveness, leading to suboptimal allocation of humanitarian funds.
Participants
N = 20 implementation experts (16 local employment staff in Amman, Jordan, and 4 senior international management staff in New York).
Study Design
Online incentivized belief elicitation exercise measuring predicted 6-week wage employment rates across control and three treatment arms prior to field trial completion.
Results
Local field staff exhibited substantial overoptimism, predicting median 6-week employment rates of 20% for cash, 10% for information, and 9% for nudge (compared to actual control/treatment rates under 5%). Senior international staff had more accurate median forecasts (7% for cash, 5% for info, 4% for nudge). Both groups correctly anticipated that cash would be the most effective intervention.
Takeaway
Elicit incentivized baseline outcome predictions from implementation staff prior to intervention trials to quantify professional expectation biases and calibrate Bayesian priors.
Source and citation
See source →Read alongside the research
Plain-English glossary
Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.
- Median
The middle number after arranging the values from smallest to largest. If there are two middle numbers, use their average.
ExampleFor the values 2, 4 and 12, the middle value is 4. The median is 4.
NIST · Measures of location ↗Read more about median →- Baseline
The starting measurements in a study. Researchers use these to compare what participants were like at the start with what happens later.
ExampleBefore a training programme starts, researchers measure workers’ earnings. That first measurement is the baseline.
GET-IT glossary · Baseline characteristics ↗Read more about baseline →