Randomization Tests

10.1 Permutation Test

python
Run Inline
import numpy as np
# Two-sample permutation test
group_a = np.array([2, 3, 4, 5])
group_b = np.array([6, 7, 8, 9])
observed_diff = np.mean(group_b) - np.mean(group_a)
all_data = np.concatenate([group_a, group_b])
n_perm = 10000
count = 0
for _ in range(n_perm):
    np.random.shuffle(all_data)
    a = all_data[:4]
    b = all_data[4:]
    if np.mean(b) - np.mean(a) >= observed_diff:
        count += 1
p_value = count / n_perm
print(f"p-value: {p_value}")

10.2 Advantages

• No distributional assumptions
• Exact for any sample size (not asymptotic)
• Works for any test statistic Join Discord PreviousHigh-Dimensional ComputingNextSmoothing & Splines