In Simple Words
Imagine testing whether wearing a new type of safety goggles reduces eye injuries in a factory. If workers randomly offered goggles choose to wear them only half the time, while unoffered workers can never get them, researchers can use the initial random offer as a lever to isolate the pure effect of actually wearing the goggles.
Swiss jobseekers: estimated download effect
- Prediction not downloaded
- Comparison condition
- Prediction downloaded
- No clear overall change in programme matching
2005 trial, excluding Geneva. Instrumental-variable estimates; matching measured within 90 days. An uncertain effect is not proof of zero.
See source →Research question
When evaluating digital decision-support tools under full discretion, frontline staff frequently choose not to inspect algorithmic predictions, creating endogenous selection bias. Standard intent-to-treat estimates dilute actual software effects, while direct comparisons of users versus non-users suffer from self-selection confounding.
Participants
N = 14,977 treatment jobseekers and N = 16,566 control jobseekers across 21 Swiss employment offices.
Study Design
Two-Stage Least Squares (2SLS) Instrumental Variables regression using caseworker randomization status (Z) as an instrument for prediction download behavior (D) under one-sided non-compliance (Imbens & Angrist, 1994).
Results
IV estimates proved that downloading predictions had zero statistically significant impact on short-term caseworker compliance (estimates ranged from -0.05 to +0.03 across specifications, all statistically insignificant).
Takeaway
Instrument actual software usage with randomized assignment status to estimate local average treatment effects when software adoption is voluntary and non-compliance is one-sided.
Source and citation
See source →Read alongside the research
Plain-English glossary
Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.
- Statistically significant
A label used when a result passes a chosen statistical threshold, often p < 0.05. It does not show how large or useful an effect is. A result that is not statistically significant does not prove there is no effect.
ExampleA study reports p = 0.03 against a 0.05 threshold. It passes that test, but the effect could still be too small to matter.
American Statistical Association · Statement on p-values ↗Read more about statistically significant →