Joint Hypothesis Balance Testing in Pooled Randomization Blocks
RCTClinical Trial
In multi-site field experiments combining hundreds of separate randomization blocks, evaluating individual baseline variables sequentially fails to provide a comprehensive statistical test of overall experimental balance.
Picture this
Checking whether teams in a multi-league tournament were assigned fairly by testing every player characteristic simultaneously across all divisions at once, ensuring no hidden advantages exist in any league.
What the evidence says
Joint hypothesis tests produced p-values of 0.163 for the full randomized sample, 0.282 for the state test sample, and 0.106 for the supplemental outcome sample, confirming that randomized assignment created statistically balanced groups.
- Who was studied
- N = 31,439 randomized students (27,265 with state test scores) across 619 randomization blocks in 6 urban school districts.
- How
- Seemingly Unrelated Regressions (SURE) joint hypothesis testing regressing assigned treatment effectiveness on a vector of baseline student characteristics (prior scores, demographics, special education, ELL status) with randomization block fixed effects.
What to do
Execute a SURE joint hypothesis test across all baseline covariates simultaneously to verify experimental randomization balance in blocked multi-site trials.
From the source
"The joint hypothesis test was estimated using the 'seemingly unrelated regressions' or SURE model proposed by Zellner (1962)."
Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment