Demographic Covariate Inclusion in Value-Added Modeling
RCTClinical Trial
Controlling for student demographic traits (such as race, ethnicity, and free/reduced-price lunch status) in teacher evaluation models is controversial because it may account for out-of-school disadvantages or unfairly lower achievement expectations for specific student populations.
Picture this
Imagine adjusting a runner's marathon time based on the weight of their shoes. If shoe weight represents an external obstacle outside the runner's control, adjusting scores levels the playing field; however, if the adjustment masks underlying training differences, it distorts the true measurement of running ability.
What the evidence says
The incremental component removed by controlling for student demographic characteristics yielded an IV coefficient of -0.025 (SE = 0.521), demonstrating that removing demographic variables does not significantly alter post-randomization outcome predictions, though large standard errors leave the policy trade-off statistically imprecise.
- Who was studied
- N = 27,255 randomized students in grades 4–8 across 619 randomization blocks in 6 urban school districts.
- How
- LIML IV regression decomposing teacher value-added variance to isolate whether the incremental component removed by student demographic controls predicts student test achievement following random assignment.
What to do
Prioritize student baseline achievement controls as the primary bias-reduction mechanism while carefully evaluating district policy trade-offs before adding or prohibiting student demographic covariates.
From the source
"Although the coefficients on $\hat{\tau}_{j}^{0}$ and $\hat{\tau}_{j}^{1}-\hat{\tau}_{j}^{0}$ are both statistically different from zero and not statistically different from one, the component associated with student demographics, $\hat{\tau}_{j}^{2}-\hat{\tau}_{j}^{1}$, is not statistically different from zero."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment