Machine learning algorithms detect non-linear interactions across high-dimensional covariates without restrictive functional form assumptions.

Download image · 1200 × 630 PNG

Research procedure

Test variation in programme effects

  1. Split data for training and inference
  2. Train predictors; test on main sample
  3. Compare predicted effect quartiles

Illustrative summary of the source’s machine-learning analysis. Repeat sample splitting 100 times using baseline features. Keep training separate from inference and aggregate across splits. The source finds variation during the programme, but no statistically significant post-programme heterogeneity.

See source →

How

When to use

Evaluators of social protection programs need to assess whether treatment impacts vary across beneficiary sub-groups without making ad-hoc assumptions about functional forms. Standard regression interaction models struggle with high-dimensional baseline covariates and potential false discoveries. Flexible machine learning estimation detects non-parametric conditional average treatment effects to evaluate policy targeting and welfare optimization.

Estimate conditional average treatment effects using generic machine learning algorithms combined with sample splitting and statistical inference protocols.

  1. Assemble a high-dimensional set of baseline participant features including demographics, education, household assets, prior earnings, and behavioral preferences.
  2. Partition experimental data into an auxiliary sample for model training and a main sample for statistical inference, repeating the data-splitting process 100 times.
  3. Train machine learning algorithms such as Generalized Random Forest, Elastic Net, and Boosted Trees on the auxiliary sample to construct proxy predictors of baseline outcomes and treatment effect heterogeneity.
  4. Estimate the average treatment effect and the heterogeneity loading parameter on the main sample to formally test for the presence of treatment effect variation.
  5. Project outcomes across predicted treatment effect quartiles to calculate Group Average Treatment Effects and analyze baseline characteristics of high-impact versus low-impact subgroups.

Why it works

Combining proxy predictors with repeated sample splitting and median aggregation generates valid confidence intervals while guarding against over-fitting and false discoveries.

Example

In Côte d'Ivoire, researchers applied Generalized Random Forest algorithms across 2,877 urban workfare applicants to evaluate heterogeneity in program impacts on monthly earnings. The method detected statistically significant heterogeneity during program implementation with a loading parameter of 1.289, revealing that monthly earnings gains in the top quartile of 32,849 CFA francs were 2.4 times larger than in the bottom quartile of 13,822 CFA francs. In contrast, the analysis detected no statistically significant heterogeneity in post-program earnings with a loading parameter of 0.021.

About the study

Participants

2,877 urban youth aged 18 to 30 years across 16 localities in Côte d'Ivoire for midline analysis, and 3,865 youth for endline analysis.

Study Design

Methodological application of the Chernozhukov et al. generic machine learning framework using 100 random stratified sample splits across 21 baseline covariates, comparing Elastic Net, Random Forest, Boosted Trees, R-Learner, and Generalized Random Forest algorithms.

Results

Midline heterogeneity parameter was 1.289 (p-value less than 0.001), with Group Average Treatment Effects ranging from 13,822 CFA francs in the bottom quartile to 32,849 CFA francs in the top quartile. Endline heterogeneity parameter was 0.021 (p-value equal to 1.000), demonstrating no systematic post-program treatment effect variation across baseline characteristics.

Source

See source →