Student Sorting Bias and Baseline Score Adjustment
RCTClinical Trial
Non-random assignment of students to classrooms causes systematic baseline achievement differences across instructors, biasing unadjusted outcome measures by conflating student background with instructor quality.
Picture this
If one coach receives elite varsity athletes while another coach receives untrained beginners, directly comparing their end-of-season win totals misattributes student talent to coaching quality. Adjusting for incoming skill levels resets the starting line so both coaches are evaluated solely on the improvement generated during the season.
What the evidence says
Prior to randomization, the within-school standard deviation of teacher mean baseline math scores was 0.382 to 0.520. Evaluating instructors using unadjusted end-of-year student test scores yielded a severely biased predictive coefficient of 0.228 (SE = 0.042), significantly overstating true instructor impact.
- Who was studied
- 1,181 randomized MET project teachers, 3,802 non-randomized MET school teachers, and 17,153 non-MET school teachers across 6 urban school districts.
- How
- Within-school baseline achievement covariance analysis across academic years and comparative IV estimation of value-added models with versus without baseline test controls.
What to do
Control for prior baseline student achievement test scores in evaluation models to eliminate selection bias caused by classroom sorting.
From the source
"In other words, if we used end-of-year scores to assess teachers' effectiveness and failed to adjust for students' prior achievement, then we would be overstating the differences between teachers."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment