Managerial Evaluation Accuracy Gate

In Simple Words

Think of a driver using a GPS navigation system; if the satellite signal is accurate, the driver follows the turns with confidence, but if the GPS is glitchy and places the car in the middle of an ocean, the driver ignores the navigation prompts entirely. Similarly, if teachers believe their principal cannot accurately evaluate classroom effort, they ignore the subjective raise system and do not increase their effort.

Research question

Subjective performance incentives rely entirely on managerial observation quality; when employees perceive managers as inaccurate assessors of effort, subjective incentive power collapses.

Participants

N = 440,595 student-subject-level observations across 255 school clusters in Pakistan.

Study Design

Instrumental variable RCT interacting subjective contract treatment with teacher-reported manager rating inaccuracy (measured on a 5-step appraisal scale).

Results

For managers in the most inaccurate quintile (rating error > 2 steps out of 5), subjective performance pay had no statistically significant effect on student test scores (interaction coefficient = -0.211 SD, p < 0.05).

Takeaway

Assess evaluator accuracy using peer-appraisal cross-checks before deploying discretionary subjective incentive schemes, ensuring managerial evaluation error remains low.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Randomized controlled trial

A study that uses chance, like drawing lots, to decide who receives each option being compared. Researchers compare the groups’ outcomes to estimate the effect of those options.

ExampleA lottery assigns volunteers to a training programme or a comparison group. Researchers then compare their outcomes.

GET-IT glossary · Randomized study ↗Read more about randomized controlled trial →

Statistically significant

A label used when a result passes a chosen statistical threshold, often p < 0.05. It does not show how large or useful an effect is. A result that is not statistically significant does not prove there is no effect.

ExampleA study reports p = 0.03 against a 0.05 threshold. It passes that test, but the effect could still be too small to matter.

American Statistical Association · Statement on p-values ↗Read more about statistically significant →

Related findings

• Evaluation-Induced Gender Bias →Economics (general)

  Imagine a judge who gives honest, balanced feedback during non-competitive practice rounds, but subconsciously gives higher scores to familiar faces when actual prize money is on the line. Adding high financial stakes to human judgment opens the door for unconscious demographic preferences to sway final pay decisions.

• Administrative Social Security Data Linkage for Earnings Measurement →Economics (general)

  Instead of asking individuals to remember and report exact yearly income numbers from a decade ago, researchers obtain signed consent to link survey profiles directly to official government tax records. This creates an unalterable, objective earnings ledger covering years before and after a financial event.

• Agnostic Machine Learning Heterogeneity Detection in Field Trials →Economics (general)

  Evaluating a program using only average results is like measuring the average temperature of a room with one frozen wall and one burning wall and concluding the room is comfortable. Machine learning acts like a thermal camera, splitting the room into sections to reveal the extreme cold and extreme heat hiding behind the average.

• Asymmetric Wealth Effect of Subsidized Index Contracts →Economics (general)

  Imagine buying a lottery ticket that costs $1, but pays out $1,000 if a major storm hits. Paying $1 during normal weather is so cheap that it does not change weekly spending or force extra work hours to make up for the cost.