Kling-Liebman-Katz Summary Index Standardization
RCTClinical Trial
Evaluating multi-faceted educational interventions across numerous individual survey items increases the probability of false positive statistical conclusions due to multiple hypothesis testing errors.
Picture this
Instead of judging a student's entire school performance by looking at twenty separate homework assignments, evaluators convert every assignment score to a standardized scale and calculate a single grade point average, giving one clear overall metric without getting lost in individual noise.
What the evidence says
Successfully controlled family-wise error rates while identifying statistically significant aggregate treatment effects of 0.38 SD in knowledge (p < 0.01) and 0.17 SD in attitudes (p < 0.01) six months post-intervention.
- Who was studied
- N = 4,599 human ninth-grade students across 138 classrooms in 21 Colombian cities.
- How
- Methodological aggregation normalizing outcome variables to baseline mean 0 and standard deviation 1, creating unweighted summary index averages following Kling, Liebman, and Katz (2007).
What to do
Aggregate related survey items into standardized summary indices using baseline parameters to prevent over-rejection of true null hypotheses during evaluation.
From the source
"Testing multiple outcomes using (1) for each measure independently increases the probability of rejecting a true null hypothesis... Hence, the analysis follows Kling, Liebman, and Katz (2007) and defines a summary measure Y* as the unweighted average of all standardized outcomes in a family."
Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia