Compositional Selection Shift Under Zero Average Treatment Effects
Evaluations that rely exclusively on average treatment effects risk misinterpreting ineffective interventions when messaging significantly alters applicant demographics while leaving overall take-up unchanged.
Picture this
Changing a recruitment message acts like adjusting a mesh filter on a sifter. Even if the total volume of sand passing through the sifter remains identical, the size and type of grains that make it through change completely.
What the evidence says
Professional stigma framing shifted applicant composition significantly, increasing the share of rich applicants by 62% (p < 0.01) and currently working applicants by 28% relative to control, despite near-zero net application growth.
- Who
- N = 611 to 1,470 street-recruited unemployed/underemployed youth in Cairo, Egypt [19, 20].
- How
- Machine learning (Lasso regression and split-sample validation) estimating individual treatment effects and evaluating baseline demographic shifts among applicants [17, 21].
What to do
Audit recruitment interventions using Lasso-based index metrics and machine learning heterogeneity tests rather than evaluating success solely on aggregate participation volume.
From the source
"Even if a particular treatment intervention has no average effect on take-up relative to control, it can alter who participates if the effects are heterogeneous."
Stigma and Take-Up of Labor Market Assistance- Evidence from Three Experiments
Tagged
Nearby findings
- Agnostic Machine Learning Heterogeneity Detection in Field TrialsEconomics (general)
- Administrative Social Security Data Linkage for Earnings MeasurementEconomics (general)
- Asymmetric Wealth Effect of Subsidized Index ContractsEconomics (general)
- Audit-Protected Performance MeasurementEconomics (general)