Bootstrap Methods

2.1 Intuition: The Plug-in Principle

🔑 Key Insight: The bootstrap works because the empirical distribution function converges to the true distribution.

python
Run Inline
import numpy as np
# original sample
data = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
n = len(data)
# bootstrap estimate of mean's standard error
B = 1000
boot_means = np.zeros(B)
for i in range(B):
    sample = np.random.choice(data, size=n, replace=True)
    boot_means[i] = np.mean(sample)
se = np.std(boot_means)
ci = np.percentile(boot_means, [2.5, 97.5])
print(f"SE: {se:.3f}, 95% CI: {ci}")

2.2 Bootstrap Confidence Intervals

2.3 Bootstrap for Regression

python
Run Inline
# Bootstrap for regression coefficients
from sklearn.linear_model import LinearRegression
X = np.random.randn(100, 2)
y = X[:, 0] * 2 + X[:, 1] * 3 + np.random.randn(100)
B = 500
coefs = np.zeros((B, 2))
for i in range(B):
    idx = np.random.choice(100, 100, replace=True)
    model = LinearRegression().fit(X[idx], y[idx])
    coefs[i] = model.coef_
# Standard errors from bootstrap
se_boot = np.std(coefs, axis=0)