Taking differences across time removes time-invariant unobserved differences between municipalities.

Download image · 1200 × 630 PNG

Research analysis

Compare changes across municipalities

  1. Link 1995 and 2005 cohorts by residence five years earlier
  2. Compare changes in six programme and six nearby municipalities
  3. Estimate double differences; test placebo trends

Illustrative summary of the source-described non-experimental census analysis of 17,061 rural girls aged 9 to 12. Robust standard errors and household-head placebo tests assess uncertainty and assumptions; no randomized comparison is implied.

See source →

How

When to use

Evaluators face challenges measuring absolute long-term impacts when social protection programs scale up across all target communities and eliminate experimental control groups. Double difference estimation using repeated national population censuses provides a non-experimental counterfactual. This method enables researchers to assess overall program effects while controlling for domestic migration.

Estimate non-experimental absolute program impacts by comparing pre-program and post-program cohort outcomes from repeated national population censuses between program and comparison municipalities.

  1. Evaluators extract individual demographic, educational, and fertility records for target age cohorts from national population censuses conducted before program launch and after program completion.
  2. Evaluators assign each individual to their municipality of residence five years prior to each census date to adjust for domestic migration.
  3. Evaluators define treatment indicators for individuals residing in municipalities where social protection programs achieved near-universal coverage.
  4. Evaluators select comparison cohorts from adjacent rural municipalities sharing similar baseline poverty and marginality levels.
  5. Evaluators calculate the double difference estimator by subtracting changes over time in comparison municipalities from changes in treatment municipalities.
  6. Evaluators estimate robust standard errors and test the common trends assumption using unaffected placebo groups such as household heads.

Why it works

Reconstructing residency five years prior prevents migration-driven selection bias from distorting impact estimates.

Example

In Nicaragua, researchers evaluated the 5-year absolute impact of the Red de Protección Social CCT program using national censuses from 1995 and 2005. Comparing 17,061 rural girls aged 9 to 12 in program municipalities against adjacent comparison municipalities revealed an absolute increase of 0.54 completed grades, an 11 percentage point increase in fourth-grade completion, and a 2 percentage point reduction in early pregnancy.

About the study

Participants

17,061 rural girls aged 9 to 12 across six program municipalities and six comparison municipalities in central and northern Nicaragua.

Study Design

Non-experimental double difference specification comparing individual-level 1995 pre-program and 2005 post-program census records between treatment and comparison municipalities based on residency five years prior.

Results

Five years of program coverage increased primary school grades completed by 0.536 grades, increased fourth-grade completion by 10.5 percentage points, increased literacy by 6.6 percentage points, and reduced early pregnancy by 2.3 percentage points.

Source

See source →