Neural Sync Active

Week 6.2: Sampling Distributions

Registry Synced

Week 6.2: Sampling Distributions

581 words

3 min read

Reading compass

Now · 1. Intuition: The Distribution of a Statistic

Week 6.2: Sampling Distributions

Prerequisites: CLT (Week 5.2: Law of Large Numbers & Central Limit Theorem), Normal Distribution (Week 4.2: Common Continuous Distributions) Cross-links: BSMA3012 (Linear Stat Models) — inference Core question: What distributions describe sample statistics before we observe data?

1. Intuition: The Distribution of a Statistic

A statistic is any function of the sample (e.g.,

\bar{X}

S^2

, sample median). Its sampling distribution is the probability distribution of that statistic over repeated sampling.

Key idea: Before we draw the sample, $\bar{X}$ is a random variable with its own distribution. After we draw, it's a number.

2. Sampling Distribution of the Sample Mean

X_1,\dots,X_n \sim \text{i.i.d. } (\mu, \sigma^2)

Exact (normal data): If $X_i \sim \mathcal{N}(\mu, \sigma^2)$ , then $\bar{X} \sim \mathcal{N}(\mu, \sigma^2/n)$ .
Approximate (CLT): For large $n$ , $\bar{X} \approx \mathcal{N}(\mu, \sigma^2/n)$ . Standard error: $\text{SE}(\bar{X}) = \sigma / \sqrt{n}$ .

3. Sampling Distribution of the Sample Variance

Definition The sample variance is $S^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2$ . For $X_i \sim \mathcal{N}(\mu, \sigma^2)$ i.i.d.:

\frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1},

where

\chi^2_{n-1}

is the chi-squared distribution with

n-1

degrees of freedom.

4. The Chi-Squared Distribution

Definition ( $\chi^2_k$ ) If $Z_1,\dots,Z_k \sim \mathcal{N}(0,1)$ i.i.d., then $\sum_{i=1}^k Z_i^2 \sim \chi^2_k$ . Properties:

$\mathbb{E}[\chi^2_k] = k$
$\operatorname{Var}(\chi^2_k) = 2k$
As $k \to \infty$ , $\chi^2_k \approx \mathcal{N}(k, 2k)$ PDF:

f(x) = \frac{1}{2^{k/2} \Gamma(k/2)} x^{k/2-1} e^{-x/2}, \quad x > 0.

5. The t-Distribution

Definition ( $t_k$ ) If $Z \sim \mathcal{N}(0,1)$ and $V \sim \chi^2_k$ are independent, then:

> T = \frac{Z}{\sqrt{V/k}} \sim t_k. >

Properties:

Symmetric around 0, heavier tails than normal
As $k \to \infty$ , $t_k \to \mathcal{N}(0,1)$
$\operatorname{Var}(T) = k/(k-2)$ for $k > 2$ Critical use: When $\sigma$ is unknown, replace with $S$ :

\frac{\bar{X} - \mu}{S/\sqrt{n}} \sim t_{n-1}.

6. The F-Distribution

Definition ( $F_{d_1,d_2}$ ) If $V_1 \sim \chi^2_{d_1}$ and $V_2 \sim \chi^2_{d_2}$ are independent:

> F = \frac{V_1/d_1}{V_2/d_2} \sim F_{d_1,d_2}. >

Use: Comparing two variances:

S_1^2/S_2^2 \sim F_{n_1-1, n_2-1}

under equal variances.

7. Summary Table

Statistic	Distribution	When
$\frac{\bar{X} - \mu}{\sigma/\sqrt{n}}$	$\mathcal{N}(0,1)$	Known $\sigma$ , normal data or large $n$
$\frac{\bar{X} - \mu}{S/\sqrt{n}}$	$t_{n-1}$	Unknown $\sigma$ , normal data
$\frac{(n-1)S^2}{\sigma^2}$	$\chi^2_{n-1}$	Normal data
$\frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2}$	$F_{n_1-1,n_2-1}$	Two normal samples

8. Practice Questions

Q1 (Easy)

X_i \sim \mathcal{N}(10, 4)

i.i.d. for

n=25

, what is the distribution of

\bar{X}

Full Solution

$\bar{X} \sim \mathcal{N}(10, 4/25) = \mathcal{N}(10, 0.16)$ .

Q2 (Medium)

For

n=20

normal samples,

\bar{x}=15

s=3

. Find a distribution for

\frac{\bar{X}-\mu}{s/\sqrt{n}}

Full Solution

$\frac{\bar{X} - \mu}{S/\sqrt{n}} \sim t_{19}$ (t-distribution with 19 df).

Q3 (Hard)

S^2

is the sample variance from

n=10

normal samples, find

\mathbb{E}[S^2]

and

\operatorname{Var}(S^2)

Full Solution

$\frac{9S^2}{\sigma^2} \sim \chi^2_9$ .

$\mathbb{E}[9S^2/\sigma^2] = 9 \implies \mathbb{E}[S^2] = \sigma^2$ (unbiased). ✓

$\operatorname{Var}(9S^2/\sigma^2) = 18 \implies 81 \cdot \operatorname{Var}(S^2)/\sigma^4 = 18 \implies \operatorname{Var}(S^2) = \frac{18\sigma^4}{81} = \frac{2\sigma^4}{9}$ .

Next topic: Week 7.1: Point Estimation — Method of Moments (MME) — Method of Moments Estimation. Join Discord PreviousWeek 6.1: Moment Generating FunctionsNextWeek 7.1: Point Estimation — Method of Moments (MME)