Evaluation-Induced Gender Bias

In Simple Words

Imagine a judge who gives honest, balanced feedback during non-competitive practice rounds, but subconsciously gives higher scores to familiar faces when actual prize money is on the line. Adding high financial stakes to human judgment opens the door for unconscious demographic preferences to sway final pay decisions.

Research question

Subjective evaluation systems can introduce systematic demographic bias when financial stakes are introduced into performance appraisals. Evaluators who display no observable bias in non-monetary feedback may favor specific employee demographic groups when evaluations directly dictate wage increases.

Participants

N = 6,080 teachers (80% female) and 189 managers (61% female) across 234 private schools in Pakistan.

Study Design

Randomized controlled trial (RCT) combined with vignette-based survey experiments and administrative appraisal analysis controlling for teacher actions and student outcomes.

Results

When subjective evaluations directly determined financial raises, male teachers received 10% higher performance ratings controlling for teacher actions and student outcomes; when evaluations carried no financial stake, no evaluation difference by gender existed.

Takeaway

Pair high-stakes subjective rating systems with high-frequency managerial monitoring and explicit evaluation rubrics to eliminate demographic evaluation disparities.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Randomized controlled trial

A study that uses chance, like drawing lots, to decide who receives each option being compared. Researchers compare the groups’ outcomes to estimate the effect of those options.

ExampleA lottery assigns volunteers to a training programme or a comparison group. Researchers then compare their outcomes.

GET-IT glossary · Randomized study ↗Read more about randomized controlled trial →

Related findings

• Managerial Evaluation Accuracy Gate →Economics (general)

  Think of a driver using a GPS navigation system; if the satellite signal is accurate, the driver follows the turns with confidence, but if the GPS is glitchy and places the car in the middle of an ocean, the driver ignores the navigation prompts entirely. Similarly, if teachers believe their principal cannot accurately evaluate classroom effort, they ignore the subjective raise system and do not increase their effort.

• Administrative Social Security Data Linkage for Earnings Measurement →Economics (general)

  Instead of asking individuals to remember and report exact yearly income numbers from a decade ago, researchers obtain signed consent to link survey profiles directly to official government tax records. This creates an unalterable, objective earnings ledger covering years before and after a financial event.

• Agnostic Machine Learning Heterogeneity Detection in Field Trials →Economics (general)

  Evaluating a program using only average results is like measuring the average temperature of a room with one frozen wall and one burning wall and concluding the room is comfortable. Machine learning acts like a thermal camera, splitting the room into sections to reveal the extreme cold and extreme heat hiding behind the average.

• Asymmetric Wealth Effect of Subsidized Index Contracts →Economics (general)

  Imagine buying a lottery ticket that costs $1, but pays out $1,000 if a major storm hits. Paying $1 during normal weather is so cheap that it does not change weekly spending or force extra work hours to make up for the cost.