Research procedure
Compare peer predictions with scores
- Ask referrers privately for expected scores: 0 to 4 puzzles
- Test candidates independently without disclosing predictions
- Compare predictions and scores by referrer ability
Illustrative summary of the source-described prediction procedure. Analysis used Heckman selection models for 505 referrers, including 280 high-ability and 225 low-ability participants; no results are shown.
See source →How
When to use
Elicit direct predictions from existing workers regarding the anticipated task performance of their recommended referrals. This assesses whether peer screening failures stem from lack of private information or from unaligned referral incentives. Use this method when evaluating peer knowledge and information accuracy in recruitment programs.
Administer an interim questionnaire asking referring workers to estimate their referred candidates' exact task completion score before candidate testing begins.
- Escort referring workers to a separate interview room while candidates fill out intake forms.
- Ask referring workers to state the expected number of puzzles their candidate will solve correctly without making mistakes.
- Record expected correct puzzle scores on a discrete scale from 0 to 4 correct puzzles.
- Administer the cognitive puzzle test to candidates independently without disclosing referrer expectations.
- Regress actual candidate task performance on referrer predictions across high-ability and low-ability referrer subsamples using Heckman selection models.
Why it works
Comparing predictions to actual test scores reveals whether low-performing workers lack screening ability or simply choose not to recruit skilled peers.
Example
Field staff surveyed 505 referring participants in Kolkata, India, asking how many puzzles their referral would solve correctly. High-ability participants accurately predicted referral scores with a coefficient of 0.190, whereas low-ability participants showed a non-significant coefficient of 0.025.
About the study
Participants
505 male labor market participants (280 high-ability, 225 low-ability) in Kolkata, India.
Study Design
Interim survey elicitation of discrete predicted puzzle scores analyzed via Heckman selection models against actual referral test scores.
Results
High-ability participants predicted referral performance accurately (0.190 coefficient, p < 0.05), whereas low-ability participants showed no statistically significant predictive power (0.025 coefficient, p > 0.10).