Non-Applicant Selection Bias in Active Labor Market Evaluations
Standard randomized evaluations of job training programs estimate treatment impacts by comparing accepted applicants with rejected applicants, systematically ignoring eligible non-applicants who were deterred during recruitment.
Picture this
Evaluating a program using only people who signed up is like judging the popularity of a movie by surveying people inside the theater, while completely ignoring everyone who looked at the movie poster outside and walked away.
What the evidence says
Control application rate was 43% in Experiment 1 and job fair attendance was 5.9% in Experiment 2, establishing that recruitment messaging alters who enters the sample pool before program training impact evaluation occurs.
- Who
- N = 1,460 youth in Experiment 1 and N = 768 youth in Experiment 2 in Cairo, Egypt.
- How
- Randomized field trial analyzing recruitment-stage eligibility and application drop-off before post-application randomization.
What to do
Measure recruitment-stage eligibility and non-applicant drop-off when evaluating active labor market policies rather than restricting randomization to existing applicant pools.
From the source
"There is a third group to consider, however: those who were eligible but never applied due to the method of recruitment. Estimates of the treatment effects in such a case may not be valid for this third group, and various types of stigma may explain why they are not in the sample."
Stigma and Take-Up of Labor Market Assistance- Evidence from Three Experiments
Tagged
Nearby findings
- General Equilibrium Program Evaluation Gap in Scaled InterventionsEconomics (general)
- Monotonic Selection Bounds AdjustmentEconomics (general)
- Administrative Social Security Data Linkage for Earnings MeasurementEconomics (general)
- Agnostic Machine Learning Heterogeneity Detection in Field TrialsEconomics (general)