Policy-Relevant Treatment Effect vs. Microeconometric Treatment Effect
Expert TheoryReview
Standard microeconometric randomized controlled trial evaluations measure the relative performance difference between participants and nonparticipants within a single treated market. This produces biased policy recommendations because full program implementation alters overall baseline market equilibrium conditions.
Picture this
Imagine evaluating a new toll-free express lane on a highway by comparing the speed of drivers who received special fast-pass tickets versus drivers in regular lanes on the exact same road. The fast-pass drivers look significantly faster because they bypass local traffic. However, if every single driver on the highway is given a fast-pass ticket, the express lane becomes just as jammed as the regular road, completely erasing the relative speed advantage observed in the initial trial.
What the evidence says
A microeconometric RCT evaluation calculates a 5.4 percentage point monthly job-finding advantage (0.245 vs 0.191 matching rate) at 30% treatment intensity. In contrast, the macro policy-relevant treatment effect under full 100% rollout yields only a 0.3 percentage point increase (0.208 vs 0.205 matching rate), showing micro trials overstate true policy efficacy by 18-fold.
- Who was studied
- N = 40,403 Danish unemployment benefit applicants across 14 counties (3,751 in experiment counties Storstrøm and South Jutland).
- How
- Comparison of the microeconometric treatment effect moment $E[m_1 | \tau=0.3] - E[m_0 | \tau=0.3]$ against the general equilibrium policy-relevant treatment effect $E[m_1 | \tau=1] - E[m_0 | \tau=0]$ using indirect inference simulations.
What to do
Evaluate large-scale active labor market policies by calculating the macro policy-relevant treatment effect comparing full-rollout equilibrium to zero-rollout equilibrium rather than relying on partial trial treatment-control differentials.
From the source
"The evaluation of the randomized experiment estimates a treatment effect on the matching rates equal to 0.245 - 0.191 = 0.054... However, as we mentioned before the policy relevant treatment effect is E[m1|τ=1] - E[m0|τ=0], which is 0.208 - 0.205 = 0.003."
816b5187-9076-4043-aba7-184043b2e524-Estimating Equilibrium Effects of Job Search Assistance.pdf