Research procedure
Match long-term comparison respondents
- Pool baseline data; estimate logit scores
- Trim below treatment’s first percentile
- Check balance; match five neighbours
Illustrative summary of the source-described non-experimental method: five nearest-neighbour matching with bias adjustment on common support. Compare pooled early and late treatment respondents with comparison respondents using 2010 outcomes. Matching on observed characteristics does not establish balance on unobserved factors.
See source →How
When to use
Evaluators face challenges measuring 10-year absolute program impacts when randomized phase-in designs leave no long-term experimental control group. Non-experimental comparison groups selected at baseline may differ systematically in observable characteristics from program beneficiaries. Propensity score nearest-neighbor matching constructs a balanced counterfactual group to estimate long-term absolute program effects.
Estimate absolute 10-year program effects by matching program beneficiaries to non-experimental comparison respondents using individual propensity scores and nearest-neighbor matching.
- Evaluators pool baseline data from program census and comparison household surveys across target and comparison localities.
- Econometricians estimate individual propensity scores using a logit specification that includes marginality index, mother's education, family size, household electricity, and distance to municipal capital.
- Econometricians restrict the estimation sample to the common support by trimming observations below the first percentile of the treatment distribution.
- Econometricians verify baseline covariate balance across propensity score blocks using five nearest-neighbor bias-adjusted matching.
- Econometricians estimate average treatment effects on the treated by comparing pooled 2010 outcomes between early and late treatment respondents and matched comparison group respondents.
Why it works
Trimming off-support observations ensures that treatment and control comparisons occur only where covariate overlap is sufficient.
Example
In Nicaragua, evaluators estimated the 10-year absolute impacts of the Red de Protección Social CCT program by matching 888 early and late treatment girls aged 9 to 12 against 202 comparison girls from neighboring non-program localities. Five nearest-neighbor bias-adjusted matching on common support revealed absolute gains of 1.14 completed school grades and a 0.26 standard deviation increase in learning achievement test scores ten years after baseline.
About the study
Participants
1,090 young women aged 19 to 22 (888 pooled treatment, 202 comparison) across 42 treatment localities and 21 comparison localities in central and northern Nicaragua.
Study Design
Non-experimental propensity score five nearest-neighbor bias-adjusted matching trimmed at the 1st percentile of common support, with analytic standard errors clustered at the locality level.
Results
Ten years after baseline, propensity score matching showed absolute program impacts of 1.137 additional school grades completed and a 0.257 standard deviation increase in learning test scores compared to the unserved control group.