Alternative Classroom Observation Rubric Sensitivity and Generalizability
Observational StudyReview
Standardizing teacher evaluation systems around a single observational rubric (such as Charlotte Danielson's Framework for Teaching) leaves open the question of whether alternative observation instruments provide comparable predictive power or offer subject-specific diagnostic advantages.
Picture this
Evaluating a driver using a general safety checklist measures basic road habits, but using a specialized racing rubric or passenger comfort survey evaluates distinct nuances. Choosing between a general checklist and a specialized tool changes which specific instruction skills are highlighted, even if overall performance ratings correlate.
What the evidence says
While FFT was selected as the primary rubric to approximate general district implementation, alternate observation tools demonstrated comparable predictive validity when adjusted for baseline student characteristics. However, specialized subject tools (such as MQI in math and PLATO in ELA) captured distinct domain-specific instructional practices.
- Who was studied
- N = 1,181 participating MET project teachers across 6 urban school districts.
- How
- Comparative correlation and predictive validity estimation across multiple observational protocols scored from 4 recorded classroom videos per teacher: Framework for Teaching (FFT), Classroom Assessment Scoring System (CLASS), Protocol for Language Arts Teacher Observations (PLATO), Mathematical Quality of Instruction (MQI), and UTeach Teacher Observation Protocol (UTOP).
What to do
Evaluate subject-specific observational rubrics (such as MQI and PLATO) alongside general frameworks (such as FFT) when designing diagnostic feedback and evaluation systems.
From the source
"As described in Kane and Staiger (2012), the project collected teacher scores on a number of other instruments as well: the Classroom Assessment Scoring System (CLASS), the Protocol for Language Arts Teacher Observations (PLATO), the Mathematical Quality of Instruction (MQI), and the UTeach Teacher Observation Protocol (UTOP). However, since one of our goals was to approximate what a given district or state could do with its data, we used only one instrument, Framework for Teaching, in this analysis."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment