Non-Response Bounding and Weighting Robustness
RCTReview
Field experiments tracking job search outcomes over extended follow-up periods frequently suffer from attrition, creating risk that non-random dropout rates between control and treatment groups bias instrumental variable estimates.
Picture this
Think of a survey where some people drop out before answering questions. To test whether missing responses distort the final score, researchers test the worst-case scenario by assuming everyone who dropped out got the lowest possible score, and then assuming they all got the highest possible score. If the overall pattern holds true under both extreme boundaries, the study's conclusions are rock solid.
What the evidence says
Post-intervention wage returns remain positive and statistically significant under weighted adjustments (IV estimate = $4.20/day, p < 0.05; IHS daily wage = 0.605, p < 0.05) and conservative bounding scenarios, confirming that survey non-response does not account for the observed labor market returns.
- Who was studied
- N = 268 male job seekers at baseline; N = 227 successfully re-interviewed at 9-month follow-up (84.7% response rate) in Lilongwe, Malawi.
- How
- Sensitivity testing using Fitzgerald-Gottschalk-Moffitt inverse probability weighting and Horowitz-Manski conservative minimum-maximum bounds to test treatment effect robustness across differential attrition rates (18.9% attrition in the 0% group vs 7.1% in the 75% group).
What to do
Calculate conservative Manski bounds and inverse probability weights on follow-up data to verify that differential survey attrition does not drive experimental findings.
From the source
"To address concerns related to the nontrivial level of attrition, we conduct two bounding exercises... In both cases, we find broadly consistent results."
b23e842f-67cd-47d7-9b15-b6b6c8e557a2-Employment Exposure- Employment and Wage Effects in Urban Malawi.pdf