Importance Sampling and Variance Reduction
105 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Importance Sampling and Variance Reduction ## Importance Sampling Estimate $E_f[h(X)]$ by sampling from $g(x)$ instead of $f(x)$: $$ \hat{\theta} = \frac{1}{n} \sum_{i=1}^n h(X_i) \frac{f(X_i)}{g(X_i)} $$ ## Control Variates If $E[U] = \mu_U$ known, use $Y_{CV} = Y - \beta(U - \mu_U)$. Optimal $\beta = \text{Cov}(...
Importance Sampling and Variance Reduction
Importance Sampling
Estimate by sampling from instead of :
Control Variates
If known, use .
Optimal .
pythonimport numpy as np # Control variate: use known mean of U u = np.random.uniform(0, 1, 1000) y = np.exp(u) # want to estimate integral of e^x u_mean = 0.5 beta = np.cov(y, u)[0,1] / np.var(u) y_cv = y - beta * (u - u_mean) print(f"Var(Y): {np.var(y):.4f}, Var(Y_CV): {np.var(y_cv):.4f}")