Research design
Compare early and delayed reform rollout
296 sub-districts randomly assigned in Andhra Pradesh
- Early rollout
- Implement the Smartcard reform in treatment areas.
- Delayed rollout
- Continue existing procedures during the comparison period.
Illustrative summary of the source-described design. Independent baseline and follow-up data cover 8,000 households in eight districts over two years. Compare service delivery, leakage and preferences. Randomisation is by sub-district, not by individual household.
See source →How
When to use
When governments roll out major administrative reforms facing local political backlash or implementation challenges. Conducting a randomized phased rollout enables policymakers to rigorously measure reform impacts on corruption, efficiency, and citizen satisfaction. This provides objective empirical evidence to protect effective policy interventions from premature cancellation and guide broader national scaling.
Randomize the phased geographical rollout of administrative reforms across sub-districts to evaluate policy impact against comparison areas.
- Government leadership partners with independent research institutions to design a randomized phased evaluation of planned public service reforms.
- Evaluators cluster target administrative regions into distinct sub-districts to serve as randomization units.
- Researchers randomly assign sub-districts to an early rollout treatment group or a delayed rollout comparison group.
- Program administrators launch the reform in treatment sub-districts while comparison sub-districts continue operating under status quo procedures.
- Survey teams collect independent baseline and follow-up data from households and administrative records across both treatment and comparison groups.
- Research teams analyze differences in service delivery, leakages, and citizen preferences to evaluate impact and present empirical findings to government decision-makers.
Why it works
Objective empirical evidence prevents political rhetoric or local bureaucratic opposition from shutting down successful state capacity reforms.
Example
In Andhra Pradesh, government officials partnered with researchers to evaluate the Smartcard reform across 296 sub-districts randomly divided into early treatment and comparison groups. When local officials nearly stopped the reform in 2013 due to negative feedback, leaders reviewed the evaluation's positive findings on leakage reduction and citizen satisfaction, preserving the program and informing India's national Aadhaar policy.
About the study
Participants
8,000 households in 8 districts of Andhra Pradesh, covering a rural population of nineteen million people.
Study Design
Cluster-randomized controlled trial across 296 sub-districts randomly assigned to early rollout (treatment) or delayed rollout (comparison) over two years.
Results
The evaluation demonstrated a 41 percent decrease in NREGS leakages and a 47 percent decrease in SSP leakages, reversing a 2013 decision to cancel the program and subsequently informing the national Aadhaar Act of 2016.