Weather-Shock Selection Correction

In Simple Words

Think of evaluating who attends an optional weekend training session. If bad weather makes traveling difficult, only the most dedicated employees show up. Using rainfall as a decision variable allows analysts to mathematically correct for travel reluctance without distorting actual indoor test performance.

Research question

In labor market hiring experiments and field trials, non-random participant attrition—where individuals fail to return with referrals—biases estimates of how financial incentives affect candidate quality.

Participants

N = 561 original labor market participants tracked across 3-day recruitment windows in Kolkata, India.

Study Design

Heckman two-step selection model utilizing the number of rainy days (0 to 3) during the recruitment window as an exclusion restriction instrument.

Results

One additional day of rainfall during the 3-day window reduced participant return probability by 20.7 percentage points (p < 0.01, chi-squared > 12.6), providing a valid instrument that eliminated non-response selection bias without affecting indoor cognitive task scores.

Takeaway

Utilize exogenous environmental friction variables (such as local precipitation) as exclusion restrictions in selection models to adjust for non-random participant attrition.

Read alongside the research

Plain-English glossary

Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.

Percentage points

The difference between two percentages.

ExampleA rise from 40% to 50% is 10 percentage points. Relative to the starting value, that is a 25% increase.

Office for National Statistics · Percentages ↗Read more about percentage points →

Related findings

• Administrative Social Security Data Linkage for Earnings Measurement →Economics (general)

  Instead of asking individuals to remember and report exact yearly income numbers from a decade ago, researchers obtain signed consent to link survey profiles directly to official government tax records. This creates an unalterable, objective earnings ledger covering years before and after a financial event.

• Agnostic Machine Learning Heterogeneity Detection in Field Trials →Economics (general)

  Evaluating a program using only average results is like measuring the average temperature of a room with one frozen wall and one burning wall and concluding the room is comfortable. Machine learning acts like a thermal camera, splitting the room into sections to reveal the extreme cold and extreme heat hiding behind the average.

• Asymmetric Wealth Effect of Subsidized Index Contracts →Economics (general)

  Imagine buying a lottery ticket that costs $1, but pays out $1,000 if a major storm hits. Paying $1 during normal weather is so cheap that it does not change weekly spending or force extra work hours to make up for the cost.

• Audit-Protected Performance Measurement →Economics (general)

  Imagine a classroom exam where students know that random answer sheets will be double-checked by independent outside inspectors; unpredictable audits prevent students from cheating or making fake arrangements with graders.