Recruitment-Induced Sample Selection in Evaluation
RCTClinical Trial
Standard randomized evaluations compare participants randomly assigned to treatment versus control after application, ignoring eligible non-applicants deterred by recruitment messaging. This creates recruitment-induced sample selection bias, making evaluation findings invalid for the broader non-applicant population.
Picture this
Imagine testing a workout program on people who signed up after seeing a poster at a local gym. If the poster wording accidentally deterred beginner athletes from entering the room, your trial only measures how well the workout helps fit people. Claiming the workout program will produce the exact same results for out-of-shape people who stayed home is an invalid generalization.
What the evidence says
Varying recruitment messaging altered the baseline applicant population along major socioeconomic dimensions (rich +62%, employed +28%), demonstrating that randomized trial estimates reflect treatment effects on a selected subset rather than the broader eligible population.
- Who was studied
- N = 1,460 jobseekers (Experiment 1) and N = 768 jobseekers (Experiment 2) in Cairo, Egypt.
- How
- Methodological assessment analyzing baseline compositional shifts between treatment and control applicant pools generated under distinct recruitment pitch conditions.
What to do
Standardize and audit exact partner recruitment pitch scripts during randomized trials to account for selection bias across trial arms.
From the source
"There is a third group to consider, however: those who were eligible but never applied due to the method of recruitment. Estimates of the treatment effects in such a case may not be valid for this third group..."
Stigma and Take-Up of Labor Market Assistance: Evidence from Two Field Experiments