Week 3.2: Variance & Standard Deviation

552 words

3 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Week 3.2: Variance & Standard Deviation > **Prerequisites:** Expected Value ([Week 3.1: Expected Value](/notes/01-foundation-bsma1004-stats-2-week03-08-expectation)) > **Cross-links:** BSMA3012 (Linear Stat Models) — residual variance > **Core question:** How spread out is a random variable around its mean? * * *...

Week 3.2: Variance & Standard Deviation

Prerequisites: Expected Value (Week 3.1: Expected Value) Cross-links: BSMA3012 (Linear Stat Models) — residual variance Core question: How spread out is a random variable around its mean?

1. Intuition: Beyond the Average

Expected value tells us the centre, but two RVs can have the same mean yet behave very differently:

$X \equiv 10$ (constant) → mean $10$ , no spread
$Y \in \{9,11\}$ equally → mean $10$ , small spread
$Z \in \{0,20\}$ equally → mean $10$ , large spread We need a measure of dispersion or spread. The variance quantifies how far values typically fall from the mean.

2. Formal Definition

Definition (Variance & Standard Deviation) The variance of $X$ is:

> \operatorname{Var}(X) = \mathbb{E}\left[(X - \mu)^2\right], \quad \text{where } \mu = \mathbb{E}[X]. >

The standard deviation is $\operatorname{SD}(X) = \sqrt{\operatorname{Var}(X)}$ . Alternative formula (easier for computation):

\operatorname{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2.

Proof:

\begin{aligned} \operatorname{Var}(X) &= \mathbb{E}[(X-\mu)^2] = \mathbb{E}[X^2 - 2\mu X + \mu^2] \\ &= \mathbb{E}[X^2] - 2\mu\mathbb{E}[X] + \mu^2 = \mathbb{E}[X^2] - 2\mu^2 + \mu^2 = \mathbb{E}[X^2] - \mu^2. \end{aligned}

3. Properties

$\operatorname{Var}(c) = 0$ for a constant $c$
$\operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X)$ (scaling changes variance, shifting does not)
$\operatorname{SD}(aX + b) = |a| \operatorname{SD}(X)$
If $X$ and $Y$ are independent: $\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$

4. Variance of Common Distributions

Distribution	PMF	$\mathbb{E}[X]$	$\operatorname{Var}(X)$
Bernoulli( $p$ )	$p^x(1-p)^{1-x}$	$p$	$p(1-p)$
Binomial( $n,p$ )	$\binom{n}{x}p^x(1-p)^{n-x}$	$np$	$np(1-p)$
Geometric( $p$ )	$(1-p)^{x-1}p$	$1/p$	$(1-p)/p^2$
Poisson( $\lambda$ )	$e^{-\lambda}\lambda^x/x!$	$\lambda$	$\lambda$
Uniform $\{1,\dots,n\}$	$1/n$	$(n+1)/2$	$(n^2-1)/12$

5. Standardised Random Variables

Definition $X$ is standardised if $\mathbb{E}[X] = 0$ and $\operatorname{Var}(X) = 1$ . For any $X$ with finite mean and variance:

Z = \frac{X - \mu}{\sigma}

is standardised. This Z-score expresses values in "standard deviation units".

6. Practice Questions

Q1 (Easy)

X

has PMF:

f_X(-2)=0.1

f_X(0)=0.6

f_X(2)=0.3

. Find

\operatorname{Var}(X)

Full Solution

$\mu = (-2)(0.1) + 0(0.6) + 2(0.3) = -0.2 + 0 + 0.6 = 0.4$ .

$\mathbb{E}[X^2] = 4(0.1) + 0(0.6) + 4(0.3) = 0.4 + 0 + 1.2 = 1.6$ .

$\operatorname{Var}(X) = 1.6 - (0.4)^2 = 1.6 - 0.16 = 1.44$ .

Q2 (Medium)

X \sim \text{Binomial}(100, 0.3)

. Find

\mathbb{E}[X]

and

\operatorname{Var}(X)

Full Solution

$\mathbb{E}[X] = np = 100 \times 0.3 = 30$ . $\operatorname{Var}(X) = np(1-p) = 100 \times 0.3 \times 0.7 = 21$ . $\operatorname{SD}(X) = \sqrt{21} \approx 4.58$ .

Q3 (Hard)

X \sim \text{Poisson}(5)

. Find

\mathbb{E}[X^2]

and

\operatorname{Var}(X)

Full Solution

For Poisson( $\lambda$ ): $\mu = \lambda$ , $\operatorname{Var}(X) = \lambda = 5$ .

$\operatorname{Var}(X) = \mathbb{E}[X^2] - \mu^2 \implies \mathbb{E}[X^2] = \operatorname{Var}(X) + \mu^2 = 5 + 25 = 30$ .

Next topic: Week 3.3: Covariance & Correlation — Covariance and correlation coefficient. Join Discord PreviousWeek 3.1: Expected ValueNextWeek 3.3: Covariance & Correlation

What this note is really saying

Week 3.2: Variance & Standard Deviation

1. Intuition: Beyond the Average

2. Formal Definition

3. Properties

4. Variance of Common Distributions

5. Standardised Random Variables

6. Practice Questions

Q1 (Easy)

Q2 (Medium)

Q3 (Hard)

Intelligence Hub