Monotonic Selection Bounds Adjustment

In Simple Words

Think of a running coach whose new training program keeps slower runners from quitting the team. If team speed is measured without adjusting for the fact that slower runners now participate in time trials, overall average team speed appears lower than it actually is. Bounds adjustment recalculates score distributions to reveal the true performance gain.

Research question

Standard test score comparisons between voucher winners and losers suffer from selection bias when vouchers increase the proportion of lower-ability marginal students who stay in school and take standardized tests.

Participants

N = 1,176 human secondary school voucher applicants (256 ICFES test takers analyzed) in Bogotá, Colombia.

Study Design

Upper-bound selection bias adjustment under monotonicity assumptions comparing raw test score differences to adjusted potential score distributions.

Results

Unconditional reading test score differences of 2.06 points increased to an upper bound estimate of 4.36 points (0.33 to 0.66 standard deviations), and math test score differences increased from 0.77 to an upper bound of 2.52 points among vocational voucher winners.

Takeaway

Apply monotonicity-based selection bounds when evaluating educational interventions that alter standardized test participation rates among low-performing students.

Related findings

• Non-Applicant Selection Bias in Active Labor Market Evaluations →Economics (general)

  Evaluating a program using only people who signed up is like judging the popularity of a movie by surveying people inside the theater, while completely ignoring everyone who looked at the movie poster outside and walked away.

• Administrative Social Security Data Linkage for Earnings Measurement →Economics (general)

  Instead of asking individuals to remember and report exact yearly income numbers from a decade ago, researchers obtain signed consent to link survey profiles directly to official government tax records. This creates an unalterable, objective earnings ledger covering years before and after a financial event.

• Agnostic Machine Learning Heterogeneity Detection in Field Trials →Economics (general)

  Evaluating a program using only average results is like measuring the average temperature of a room with one frozen wall and one burning wall and concluding the room is comfortable. Machine learning acts like a thermal camera, splitting the room into sections to reveal the extreme cold and extreme heat hiding behind the average.

• Asymmetric Wealth Effect of Subsidized Index Contracts →Economics (general)

  Imagine buying a lottery ticket that costs $1, but pays out $1,000 if a major storm hits. Paying $1 during normal weather is so cheap that it does not change weekly spending or force extra work hours to make up for the cost.