Randomization Inference Permutation Testing
RCTClinical Trial
Standard asymptotic t-tests in randomized field experiments can produce overconfident significance claims when outcome distributions exhibit heavy skewness, clustering, or potential outlier distortion.
Picture this
Think of shuffling a deck of cards thousands of times to see how often a rare winning hand appears purely by random chance. By randomly swapping treatment labels across participants 10,000 times, researchers construct an exact benchmark distribution to verify if the real experimental result was a true effect or a statistical fluke.
What the evidence says
Permutation testing yielded exact p-values of p = 0.024 for permanent hires of registered jobseekers and p = 0.037 for theoretical workdays created, confirming significant treatment effects independently of asymptotic distributional assumptions.
- Who was studied
- N = 7,438 establishments across 129 local public employment agencies in France.
- How
- Randomization inference student test using 10,000 permutation runs within assignment strata (Young, 2018) alongside Mann-Whitney ranksum tests.
What to do
Validate field experiment statistical significance by running 10,000-iteration randomization permutation tests alongside standard asymptotic t-tests.
From the source
"Following Young (2018), we implement randomization inference for the usual student test, using 10,000 permutations tests. This allows to obtain a consistent estimate of the exact p-value of our test."
Are_Active_Labor_Market_Policies_Directed_at_Firms_Effective_Evidence.pdf