Instrumental Variables Estimation for Experimental Non-Compliance
RCTClinical Trial
When randomized assignments in field experiments suffer from non-compliance (such as students transferring across classrooms or schools), estimating the causal effect of actual exposure using standard comparisons introduces non-random sorting bias.
Picture this
Imagine assigning individuals to a specific fitness regimen, but some participants trade routines with friends. Instead of evaluating the regimen based on what people actually chose to do—which is biased by personal preference—one evaluates the initial assignment ticket as a push lever and scales the observed outcome up by the proportion of participants who actually followed their assigned ticket.
What the evidence says
Randomly assigned teacher effectiveness translated into actual teacher effectiveness at a rate of 0.293 (first stage) due to non-compliance. The IV coefficient of actual teacher effectiveness on student test achievement was 0.955 (p < 0.01, SE = 0.123), proving that a 1-unit predicted difference caused a statistically equivalent ~1-unit actual achievement gain.
- Who was studied
- N = 31,439 randomized students (27,265 with state test scores; 1,181 randomized teachers) across grades 4–8 in 6 urban school districts (Charlotte-Mecklenburg, Dallas, Denver, Hillsborough, Memphis, and New York).
- How
- Two-stage Instrumental Variables (IV) / Limited Information Maximum Likelihood (LIML) estimation using assigned teacher effectiveness as an instrument for actual teacher effectiveness.
What to do
Apply initial randomized assignment rosters as an instrumental variable in a two-stage LIML model to isolate unbiased causal treatment effects when experimental compliance is incomplete.
From the source
"In the end, many students ended up with an 'actual' teacher who was different from their 'assigned teacher.' Nevertheless, we were able to exploit the random assignment to infer the effect of the 'actual' teacher, using a two-step process known as 'instrumental variables' or 'IV estimation.'"
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment