Omnibus Super-Family Index Inference
When an intervention influences dozens of secondary indicators across varied domains (such as psychological well-being, financial behavior, and social networks), testing individual variables independently creates severe multi-testing noise and masks broad domain-level impacts.
Picture this
Imagine evaluating a patient's overall recovery by aggregating individual blood tests, blood pressure, and heart rate into a single health index score—testing one combined score prevents false alarms from individual daily fluctuations and gives a clear picture of overall health.
What the evidence says
Constructing standardized family indices confirmed that secondary domains (e.g., total financial expenditure $p = 0.797$, life satisfaction $p = 0.901$) showed no spurious treatment effects, maintaining statistical rigor across multi-domain evaluations.
- Who
- **N = 2,841 job-seekers** evaluated across 7 secondary outcome families in Addis Ababa, Ethiopia.
- How
- Summary index aggregation across outcome families (Job Quality, Financial Outcomes, Expectations, Mobility, Education, Wellbeing, Networks) grouped into a "super-family" tested via Benjamini et al. (2006) sharpened $q$-value procedures.
What to do
Aggregate related secondary survey metrics into standardized domain indices and control False Discovery Rates across the resulting super-family of indices.
From the source
"In addition to investigating each outcome in a family separately, we use a standard 'omnibus' approach: we construct an index for each family and test whether the index is affected by our treatments... we report both p values and false discovery rate q-values by treating each index as a separate member of a 'super-family' of indices."
Anonymity or Distance? Job Search and Labour Market Exclusion in a Growing African City