Neural Sync Active

Monte Carlo Methods

Registry Synced

Monte Carlo Methods

306 words

2 min read

Reading compass

Now · 🎯 Learning Objectives

Monte Carlo Methods

🎯 Learning Objectives

Understand the law of large numbers and its role in Monte Carlo
Estimate integrals using Monte Carlo simulation
Compute confidence intervals for Monte Carlo estimates
Implement variance reduction techniques

1.1 Intuition: Estimating by Random Sampling

Monte Carlo methods use random sampling to estimate quantities that are difficult to compute analytically. The idea: generate many random samples, compute the statistic, and average.

🔑 Key Insight: The law of large numbers guarantees that Monte Carlo estimates converge to the true value as sample size increases.

1.2 Monte Carlo Integration

Estimate

\theta = \int_a^b f(x) dx

\hat{\theta} = \frac{b-a}{n} \sum_{i=1}^n f(U_i)

Where

U_i \sim \text{Uniform}(a,b)


python
import numpy as np
# estimate integral of sin(x) from 0 to pi
n = 10000
x = np.random.uniform(0, np.pi, n)
estimate = np.pi * np.mean(np.sin(x))
print(f"MC estimate: {estimate:.4f}, true: 2.0")

1.3 Variance Reduction

Antithetic Variates

Use negatively correlated pairs:

U

and

1-U


python
u = np.random.uniform(0, 1, n//2)
estimates = (f(u) + f(1-u)) / 2

Control Variates

Leverage known expectation of a correlated variable.

1.4 Confidence Intervals

\hat{\theta} \pm z_{\alpha/2} \frac{s}{\sqrt{n}}

Where

s

is the sample standard deviation of the Monte Carlo estimates.

✅ Practice Questions

Q1: Estimate

\int_0^1 e^x dx

using MC with 10000 samples.

Solution


python
import numpy as np
x = np.random.uniform(0, 1, 10000)
estimate = np.mean(np.exp(x))  # since (b-a) = 1
print(estimate)  # approx 1.718 (true: e-1 = 1.71828)

Q2: How does the MC error decrease with

n

Solution

MC error $\propto \sigma / \sqrt{n}$ . To halve the error, quadruple $n$ (this is the curse of MC — slow convergence). Join Discord NextImportance Sampling