Persistent Student Baseline Sorting Covariance
RCTClinical Trial
Failing to separate multi-year, persistent assignment of high- or low-achieving students to specific instructors from single-year random classroom assignment fluctuations leads to inaccurate estimates of teacher evaluation model bias.
Picture this
If a card dealer accidentally gives a player a winning hand once, it is random variation; if that same player receives top cards every single game year after year, it reflects structural sorting. Measuring year-to-year score alignment isolates permanent sorting patterns from temporary luck.
What the evidence says
The signal standard deviation of persistent within-school student baseline sorting was 0.297 in math and 0.300 in ELA for randomized MET teachers, compared to 0.437 in math and 0.440 in ELA for non-randomized MET school teachers, proving that structural student sorting was substantial prior to experimental intervention.
- Who was studied
- N = 1,181 randomized teachers, 3,802 non-randomized school teachers, and 17,153 non-study school teachers across 6 urban school districts.
- How
- Cross-year within-school covariance estimation of teachers' mean baseline student test scores between 2008–09 and 2009–10.
What to do
Calculate multi-year baseline score covariance across instructor rosters to measure persistent student sorting before implementing unadjusted evaluation metrics.
From the source
"We measure persistent sorting using the covariance in a teacher's students' baseline test scores over time. If a teacher gets a group of high-scoring students one year but not the next, that may or may not have occurred by chance that year, but it is not persistent sorting."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment