Matching treatment individuals with comparison peers who possess similar propensity scores eliminates selection bias from observable household and locality characteristics.

Download image · 1200 × 630 PNG

Research procedure

Match long-term comparison respondents

  1. Pool baseline data; estimate logit scores
  2. Trim below treatment’s first percentile
  3. Check balance; match five neighbours

Illustrative summary of the source-described non-experimental method: five nearest-neighbour matching with bias adjustment on common support. Compare pooled early and late treatment respondents with comparison respondents using 2010 outcomes. Matching on observed characteristics does not establish balance on unobserved factors.

See source →

How

When to use

Evaluators face challenges measuring 10-year absolute program impacts when randomized phase-in designs leave no long-term experimental control group. Non-experimental comparison groups selected at baseline may differ systematically in observable characteristics from program beneficiaries. Propensity score nearest-neighbor matching constructs a balanced counterfactual group to estimate long-term absolute program effects.

Estimate absolute 10-year program effects by matching program beneficiaries to non-experimental comparison respondents using individual propensity scores and nearest-neighbor matching.

  1. Evaluators pool baseline data from program census and comparison household surveys across target and comparison localities.
  2. Econometricians estimate individual propensity scores using a logit specification that includes marginality index, mother's education, family size, household electricity, and distance to municipal capital.
  3. Econometricians restrict the estimation sample to the common support by trimming observations below the first percentile of the treatment distribution.
  4. Econometricians verify baseline covariate balance across propensity score blocks using five nearest-neighbor bias-adjusted matching.
  5. Econometricians estimate average treatment effects on the treated by comparing pooled 2010 outcomes between early and late treatment respondents and matched comparison group respondents.

Why it works

Trimming off-support observations ensures that treatment and control comparisons occur only where covariate overlap is sufficient.

Example

In Nicaragua, evaluators estimated the 10-year absolute impacts of the Red de Protección Social CCT program by matching 888 early and late treatment girls aged 9 to 12 against 202 comparison girls from neighboring non-program localities. Five nearest-neighbor bias-adjusted matching on common support revealed absolute gains of 1.14 completed school grades and a 0.26 standard deviation increase in learning achievement test scores ten years after baseline.

About the study

Participants

1,090 young women aged 19 to 22 (888 pooled treatment, 202 comparison) across 42 treatment localities and 21 comparison localities in central and northern Nicaragua.

Study Design

Non-experimental propensity score five nearest-neighbor bias-adjusted matching trimmed at the 1st percentile of common support, with analytic standard errors clustered at the locality level.

Results

Ten years after baseline, propensity score matching showed absolute program impacts of 1.137 additional school grades completed and a 0.257 standard deviation increase in learning test scores compared to the unserved control group.

Source

See source →