Research procedure
Randomise offices within baseline groups
- Wave 1: stratify 112 offices; randomise scorecards
- Wave 2: stratify 199 new offices; randomise
- Randomise add-ons; estimate office-clustered effects
Illustrative summary of the source-described two-wave trial. Wave 1 strata use baseline performance and application volume; Wave 2 uses volume. Add-ons are randomised within scorecard offices. Analysis uses inverse-frequency weighting and office-level standard errors.
See source →How
When to use
Public sector interventions often risk spillover effects across individual officials within the same administrative unit. In addition, administrative data and performance levels vary substantially across offices, threatening statistical power. This method allows researchers to evaluate organizational reforms across public sector agencies while controlling for baseline office workload and performance.
Randomize public sector administrative units into treatment and control groups using stratified cluster assignment across implementation waves.
- Define the subdistrict land office as the cluster unit of randomization to prevent spillover across local bureaucrats.
- Group eligible land offices into randomization strata based on baseline administrative performance and application volumes.
- Randomize offices within each stratum in Wave 1 to receive monthly performance scorecards or serve as control offices.
- Expand the study sample in Wave 2 by stratifying newly connected offices based on baseline application volume before randomization.
- Randomize sub-treatments, such as shared peer performance lists, within the existing treatment group using identical stratification procedures.
- Estimate treatment effects using cluster-level standard errors and inverse-frequency weighting across land offices.
Why it works
Stratifying on baseline workload and application delay metrics balances key administrative characteristics across treatment arms, improving statistical precision.
Example
In Bangladesh, a study evaluated performance scorecards across 311 subdistrict land offices serving 97 million people by stratifying 112 Wave 1 offices and 199 Wave 2 offices on baseline application processing metrics.
About the study
Participants
311 subdistrict land offices in Bangladesh serving approximately 97 million people, evaluating 1,034,688 administrative applications and survey data from 2,869 applicants.
Study Design
Stratified cluster-randomized trial conducted over 16 months across two implementation waves, using inverse cluster weighting and standard errors clustered at the land office level.
Results
Achieving balance across treatment arms (no statistically significant pre-treatment differences across 27 balance tests), the design enabled detecting a 6 percentage point increase in on-time processing and a 13% reduction in processing duration.