Research procedure
Record beliefs about future performance
- Explain grades and probability weights by video
- Collect beliefs for objective and principal ratings
- Score accuracy against year-end ratings; pay rewards
Illustrative summary of the source-described belief exercise. Teachers assign probabilities across appraisal tiers for both metrics. The accuracy reward is up to 500 rupees; no future rating is assumed.
See source →How
When to use
School leaders need to know whether teachers can accurately assess their own teaching ability and future performance rankings. Self-reported belief surveys often suffer from overconfidence bias when choices carry no real monetary consequences. This method uses quadratic scoring rules and monetary incentives to elicit truthful probability distributions of employees' performance expectations.
Elicit worker beliefs about future performance metrics using incentivized probability distributions linked to financial rewards.
- Show employees an instructional video explaining internal performance grades and probability weighting rules.
- Present employees with familiar letter-grade appraisal categories corresponding to within-school performance percentiles.
- Prompt employees to assign subjective probability likelihoods across all performance appraisal tiers from top to bottom.
- Calculate financial rewards for prediction accuracy using an incentivized scoring rule that pays out up to 500 rupees based on actual end-of-year ratings.
- Collect belief measurements across both objective test score growth percentile metrics and subjective supervisor ratings.
Why it works
Tying survey payouts to accuracy removes hypothetical bias and reveals true underlying worker expectations.
Example
Researchers measured performance beliefs among 1,284 private school teachers across Pakistan during baseline surveys in October 2017. Teachers assigned subjective probabilities across letter grades, earning up to 500 rupees based on prediction accuracy. At baseline, un-informed teachers showed flat predictions that did not correlate with actual value-added percentiles, revealing widespread miscalibration before receiving historical feedback.
About the study
Participants
1,284 private school teachers across nearly 300 private schools in Pakistan participating in baseline survey choice exercises; sample teachers averaged 35 years of age and 81 percent were female.
Study Design
Incentivized survey belief elicitation exercise conducted during baseline testing in October 2017 using quadratic-style accuracy payouts tied to actual end-of-year teacher value-added and principal ratings.
Results
At baseline, teachers' predicted value-added percentile showed a flat relationship with actual value-added (slope coefficient near 0), but providing historical value-added feedback increased positive sorting into performance pay by 50 percent.