Quiz 2

Week 5.2: Law of Large Numbers & Central Limit Theorem

604 words
3 min read
Python Week 1: the first filter for runtime behavior
Visual companion
Python
Type and operator map

Python Week 1: the first filter for runtime behavior

Open galleryRelated surface
View
Revision summary

What this note is really saying

Short form

# Week 5.2: Law of Large Numbers & Central Limit Theorem > **Prerequisites:** Expectation ([Week 3.1: Expected Value](/notes/01-foundation-bsma1004-stats-2-week03-08-expectation)), Variance ([Week 3.2: Variance & Standard Deviation](/notes/01-foundation-bsma1004-stats-2-week03-09-variance)) > **Cross-links:** BSMA30...

Week 5.2: Law of Large Numbers & Central Limit Theorem

Prerequisites: Expectation (Week 3.1: Expected Value), Variance (Week 3.2: Variance & Standard Deviation) Cross-links: BSMA3012 (Linear Stat Models) — asymptotic properties Core question: What happens to sample averages as the sample size grows?

1. Intuition: The "Stabilisation" and "Normalisation" of Averages

Take repeated measurements. Two things happen as n→∞n \to \infty:
  1. LLN: The sample average Xˉn\bar{X}_n converges to the population mean μ\mu. Uncertainty vanishes.
  2. CLT: The distribution of Xˉn\bar{X}_n (when standardised) approaches N(0,1)\mathcal{N}(0,1), regardless of the parent distribution. The LLN says averages become accurate; the CLT says they become normal.

2. Law of Large Numbers (LLN)

Theorem (Weak Law of Large Numbers) Let X1,X2,…X_1, X_2, \dots be i.i.d. with E[Xi]=μ\mathbb{E}[X_i] = \mu and Var⁡(Xi)=σ2<∞\operatorname{Var}(X_i) = \sigma^2 < \infty. Then for any ϵ>0\epsilon > 0:
>lim⁡n→∞P(∣Xˉn−μ∣≥ϵ)=0.>> \lim_{n \to \infty} \mathbb{P}\left( \left| \bar{X}_n - \mu \right| \geq \epsilon \right) = 0. >
In words: Xˉn→pμ\bar{X}_n \xrightarrow{p} \mu. Proof sketch (using Chebyshev):
P(∣Xˉn−μ∣≥ϵ)≤Var⁡(Xˉn)ϵ2=σ2/nϵ2→0.\mathbb{P}(|\bar{X}_n - \mu| \geq \epsilon) \leq \frac{\operatorname{Var}(\bar{X}_n)}{\epsilon^2} = \frac{\sigma^2/n}{\epsilon^2} \to 0.
Intuition: The variance of the sample mean shrinks as 1/n1/n, so the mean concentrates around μ\mu.

3. Central Limit Theorem (CLT)

Theorem (Central Limit Theorem) Let X1,…,XnX_1, \dots, X_n be i.i.d. with E[Xi]=μ\mathbb{E}[X_i] = \mu and Var⁡(Xi)=σ2<∞\operatorname{Var}(X_i) = \sigma^2 < \infty. Then as n→∞n \to \infty:
>Xˉn−μσ/n→dN(0,1).>> \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} \mathcal{N}(0, 1). >
Equivalently:
Xˉn→dN(μ,σ2n).\bar{X}_n \xrightarrow{d} \mathcal{N}\left(\mu, \frac{\sigma^2}{n}\right).
The CLT is the single most important theorem in statistics — it justifies normal approximations for sample means, hypothesis tests, and confidence intervals.

4. Visual Intuition

(Diagram)

5. Worked Examples

Example 1: Sum of Dice

Roll a fair die 100 times. What is P(350≤sum≤400)\mathbb{P}(350 \leq \text{sum} \leq 400)? Xi∼Uniform{1,…,6}X_i \sim \text{Uniform}\{1,\dots,6\}, μ=3.5\mu = 3.5, σ2=35/12≈2.917\sigma^2 = 35/12 \approx 2.917. Sum Sn=∑XiS_n = \sum X_i, E[Sn]=350\mathbb{E}[S_n] = 350, Var⁡(Sn)=100×2.917=291.7\operatorname{Var}(S_n) = 100 \times 2.917 = 291.7, SD=17.08\text{SD} = 17.08. By CLT:
P(350≤Sn≤400)≈P(350−35017.08≤Z≤400−35017.08)=Φ(2.93)−Φ(0)=0.9983−0.5=0.4983.\mathbb{P}(350 \leq S_n \leq 400) \approx \mathbb{P}\left( \frac{350-350}{17.08} \leq Z \leq \frac{400-350}{17.08} \right) = \Phi(2.93) - \Phi(0) = 0.9983 - 0.5 = 0.4983.

Example 2: Bernoulli Proportion

Toss a coin 1000 times. What is P(0.48≤p^≤0.52)\mathbb{P}(0.48 \leq \hat{p} \leq 0.52)? Xi∼Bernoulli(0.5)X_i \sim \text{Bernoulli}(0.5), μ=0.5\mu = 0.5, σ2=0.25\sigma^2 = 0.25. p^=Xˉ\hat{p} = \bar{X}, E[p^]=0.5\mathbb{E}[\hat{p}] = 0.5, SE=0.25/1000=0.0158\text{SE} = \sqrt{0.25/1000} = 0.0158.
P(0.48≤p^≤0.52)≈Φ(0.52−0.50.0158)−Φ(0.48−0.50.0158)=Φ(1.26)−Φ(−1.26)≈0.8962−0.1038=0.7924.\mathbb{P}(0.48 \leq \hat{p} \leq 0.52) \approx \Phi\left(\frac{0.52-0.5}{0.0158}\right) - \Phi\left(\frac{0.48-0.5}{0.0158}\right) = \Phi(1.26) - \Phi(-1.26) \approx 0.8962 - 0.1038 = 0.7924.

6. Practice Questions

Q1 (Easy)

XiX_i i.i.d. with μ=10\mu = 10, σ=3\sigma = 3. Using CLT, what is P(Xˉ36>11)\mathbb{P}(\bar{X}_{36} > 11)?
Full Solution
SE=3/36=0.5\text{SE} = 3/\sqrt{36} = 0.5. Z=(11−10)/0.5=2Z = (11-10)/0.5 = 2. P(Z>2)=0.0228\mathbb{P}(Z > 2) = 0.0228.

Q2 (Medium)

Light bulbs have mean life 1000h, SD 200h. A box of 64 bulbs: bound P(Xˉ<950)\mathbb{P}(\bar{X} < 950) using CLT.
Full Solution
SE=200/64=25\text{SE} = 200/\sqrt{64} = 25. Z=(950−1000)/25=−2Z = (950-1000)/25 = -2. P(Z<−2)=0.0228\mathbb{P}(Z < -2) = 0.0228.

Q3 (Hard)

How large must nn be so that P(∣Xˉn−μ∣<0.1σ)≥0.95\mathbb{P}(|\bar{X}_n - \mu| < 0.1\sigma) \geq 0.95?
Full Solution
We need P(∣Xˉn−μ∣/(σ/n)<0.1n)≥0.95\mathbb{P}(|\bar{X}_n - \mu| / (\sigma/\sqrt{n}) < 0.1\sqrt{n}) \geq 0.95.
By CLT, 0.1n≥1.96  ⟹  n≥19.6  ⟹  n≥384.160.1\sqrt{n} \geq 1.96 \implies \sqrt{n} \geq 19.6 \implies n \geq 384.16, so n≥385n \geq 385.
Next topic: Week 5.3: Bivariate Normal Distribution — The bivariate normal distribution. Join Discord PreviousWeek 5.1: Joint Continuous DistributionsNextWeek 5.3: Bivariate Normal Distribution
Document outline

Keep your place and jump directly to a heading.

Table of Contents
System Normal // Awaiting Context

Intelligence Hub

Navigate the knowledge graph to generate context. The Hub adapts dynamically to surface backlinks, related notes, and metadata insights.

Open library →Launch Assessment Engine