Attrition Independence from Assigned Treatment Effectiveness
RCTClinical Trial
In educational randomized controlled trials, if student drop-out rates or missing test score data correlate systematically with the effectiveness of assigned instructors, the remaining sample introduces attrition bias into causal effect estimates.
Picture this
In a drug study, if patients assigned to less effective doctors quit the trial far more frequently than those assigned to top doctors, comparing the patients who stayed creates a skewed picture. Verifying that missing data rates are equal regardless of doctor rating confirms the experiment remains fair.
What the evidence says
Assigned teacher effectiveness had no statistically significant impact on student state test data availability (-0.012, SE = 0.033), 4th–8th grade MET project outcome availability (-0.070, SE = 0.143), or high school MET project outcome availability (-0.077, SE = 0.144), establishing that sample attrition was independent of treatment assignment.
- Who was studied
- N = 31,439 randomized students in grades 4–8 and N = 8,657 randomized high school students across 6 urban school districts.
- How
- Linear regression of outcome data availability indicator variables on assigned teacher effectiveness with randomization block fixed effects.
What to do
Regress outcome data availability indicators on assigned treatment ratings to test for attrition bias in multi-site randomized field trials.
From the source
"There was no statistically significant relationship between the assigned teacher's measured effectiveness and the availability of the subsequent state achievement measures."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment