Supplemental Conceptual Assessment Transfer
RCTClinical Trial
Evaluations based primarily on state standardized test gains risk rewarding teaching methods tailored to multiple-choice formats rather than true conceptual understanding and higher-order skill transfer.
Picture this
Teaching someone to memorize answers for a written driving test does not guarantee they can handle complex real-world traffic scenarios. Testing students on novel open-ended tasks measures whether an instructor built deep understanding or merely drilled specific test formats.
What the evidence says
Instructors predicted to raise state test scores by 1.0 standard deviation increased student performance on supplemental conceptual assessments by 0.661 standard deviations (p < 0.01, SE = 0.153), indicating significant cross-format skill transfer at approximately two-thirds the state test effect size.
- Who was studied
- N = 16,642 randomized students across 553 randomization blocks in grades 4–8 across 6 urban school districts.
- How
- LIML IV estimation evaluating teacher impact on alternative conceptual math (Balanced Assessment in Mathematics) and open-ended literacy (SAT9 open-ended reading) tests following random classroom assignment.
What to do
Administer conceptual and open-ended supplemental assessments to verify that value-added achievement gains reflect transferable higher-order problem-solving skills rather than narrow test-preparation effects.
From the source
"When a teacher was predicted to improve student achievement on state tests by one standard deviation, his or her students' performance on the supplemental assessments increased by .7 standard deviations on average."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment