Experimental Self-Selection Verification and External Validity
Randomized trials requiring voluntary participant opt-in risk selection bias if participating firms systematically differ in unobserved capabilities or motivation from non-participating industry peers.
Picture this
Think of a free health clinic offering checkups to local residents. To prove that clinic attendees represent the whole town rather than just the sickest or richest residents, doctors compare the age, income, and health history of attendees against census data for the entire town.
What the evidence says
Project firms showed no statistically significant differences from nonproject firms in pre-intervention assets ($12.8M vs $13.9M, p = 0.841), employees (204 vs 221, p = 0.552), total borrowings ($4.9M vs $5.5M, p = 0.756), or baseline BVR management scores (2.52 vs 2.55, p = 0.859).
- Who
- 17 participating project firms compared against 49 nonproject contact firms and 96 ground-surveyed industry comparison textile firms near Mumbai, India.
- How
- Cross-sectional observational comparison testing pre-intervention differences in mean assets, employee counts, debt borrowings, age, and baseline management scores using t-tests.
What to do
Compare volunteer program participants against non-participating industry populations across observable baseline metrics to test for self-selection bias and verify external validity.
From the source
"We found the 17 project firms were not significantly different in terms of preintervention observables from the 96 nonproject firms that responded to this survey."
541 Management in India QJE.pdf