In Simple Words
Evaluating a marathon coach based purely on how fast their runners cross the finish line—without knowing if those runners started as elite athletes or complete beginners—evaluates runner selection rather than coaching quality.
Research question
Evaluators who rely on unadjusted end-of-year student achievement scores to evaluate instructors fail to separate baseline student skill levels from actual teaching quality, creating severe predictive bias.
Participants
N = 27,255 randomized students in grades 4–8 across 619 randomization blocks in 6 urban school districts.
Study Design
LIML IV estimation comparing unadjusted end-of-year student score ratings from prior years against post-randomization student test achievement.
Results
Unadjusted end-of-year score ratings yielded a predictive coefficient of 0.228 (p < 0.01, SE = 0.042), significantly overstating true teacher effectiveness differences compared to baseline-adjusted growth models (coefficient = 0.955).
Takeaway
Replace raw unadjusted status test scores with prior achievement-adjusted growth models when constructing instructor evaluation frameworks.
Source and citation
See source →Read alongside the research
Plain-English glossary
Original explanations, informed by the references below. Examples are illustrations, not data from the studies above. The research text stays unchanged.
- Baseline
The starting measurements in a study. Researchers use these to compare what participants were like at the start with what happens later.
ExampleBefore a training programme starts, researchers measure workers’ earnings. That first measurement is the baseline.
GET-IT glossary · Baseline characteristics ↗Read more about baseline →