In Simple Words
Having an expert watch a surgeon's technique using a detailed checklist evaluates their procedural skill, but that checklist score must be tested against patient health outcomes to confirm that higher observation scores genuinely lead to better patient recovery.
Research question
Relying on structured administrator and peer classroom observations to evaluate teachers raises questions about whether rubric-based practice scores causally predict future student test score achievement.
Participants
N = 27,255 randomized students in grades 4–8 across 619 randomization blocks in 6 urban school districts.
Study Design
LIML IV estimation isolating the standalone predictive validity of prior-year Charlotte Danielson Framework for Teaching (FFT) classroom observation scores on student achievement following random roster assignment.
Results
FFT classroom observation scores used as a standalone predictor yielded an IV coefficient of 0.807 (p < 0.01, SE = 0.293); while causally valid and not statistically different from 1, standalone observations exhibited roughly double the estimation error of test-based value-added measures (SE = 0.149).
Takeaway
Incorporate structured classroom observation rubrics into multi-measure composite evaluation systems alongside test-based value-added metrics to maintain qualitative feedback while reducing overall predictive variance.