Week 3.1: Expected Value

756 words

4 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Week 3.1: Expected Value > **Prerequisites:** Joint PMF ([Week 1.1: Joint Probability Mass Function (Joint PMF)](/notes/01-foundation-bsma1004-stats-2-week01-01-joint-pmf)), Functions of RVs ([Week 2.1: Functions of Random Variables](/notes/01-foundation-bsma1004-stats-2-week02-05-functions-of-rvs)) > **Cross-link...

Week 3.1: Expected Value

Prerequisites: Joint PMF (Week 1.1: Joint Probability Mass Function (Joint PMF)), Functions of RVs (Week 2.1: Functions of Random Variables) Cross-links: BSMA1002 (Stats 1) — expectation, BSMA3012 (Linear Stat Models) — moments Core question: What is the "average" or "central" value of a random variable?

1. Intuition: The Balancing Point

The expected value

\mathbb{E}[X]

is the probability-weighted average of all possible values of

X

. It represents the "centre of mass" of the PMF.

Physical analogy: If you place weights equal to $f_X(x)$ at positions $x$ on a see-saw, the expected value is the balance point. Key facts:

$\mathbb{E}[X]$ may not be a possible value of $X$ (e.g., $3.5$ for a die).
$\mathbb{E}[X]$ has the same units as $X$ .
Not all random variables have finite expectation.

2. Formal Definition

Definition (Expected value) For a discrete random variable $X$ with range $T_X$ and PMF $f_X$ :

> \mathbb{E}[X] = \sum_{t \in T_X} t \cdot f_X(t), >

provided the sum converges absolutely. Examples:

Distribution	PMF	$\mathbb{E}[X]$
Bernoulli( $p$ )	$p^x(1-p)^{1-x}$ , $x\in\{0,1\}$	$p$
Binomial( $n,p$ )	$\binom{n}{x}p^x(1-p)^{n-x}$	$np$
Geometric( $p$ )	$(1-p)^{x-1}p$ , $x=1,2,\dots$	$1/p$
Poisson( $\lambda$ )	$e^{-\lambda}\lambda^x/x!$ , $x=0,1,\dots$	$\lambda$
Uniform $\{a,\dots,b\}$	$1/(b-a+1)$	$(a+b)/2$

3. Expected Value of a Function: The LOTUS

The Law of the Unconscious Statistician (LOTUS) is a crucial result:

Theorem (LOTUS) For any function $g$ and random variable $X$ with PMF $f_X$ :

> \mathbb{E}[g(X)] = \sum_{t \in T_X} g(t) \cdot f_X(t). >

For $g(X_1,\dots,X_n)$ with joint PMF $f_{X_1\dots X_n}$ :

> \mathbb{E}[g(X_1,\dots,X_n)] = \sum_{t_1,\dots,t_n} g(t_1,\dots,t_n) \cdot f_{X_1\dots X_n}(t_1,\dots,t_n). >

Why it matters: You don't need to find the PMF of

g(X)

first — just apply

g

to each

x

and use the original PMF.

Example

X \sim \text{Uniform}\{-2,-1,0,1,2\}

g(X) = X^2

\mathbb{E}[X^2] = (-2)^2\cdot\frac{1}{5} + (-1)^2\cdot\frac{1}{5} + 0^2\cdot\frac{1}{5} + 1^2\cdot\frac{1}{5} + 2^2\cdot\frac{1}{5} = \frac{4+1+0+1+4}{5} = 2.

4. Linearity of Expectation

Theorem (Linearity) For random variables $X,Y$ and constants $a,b$ :

> \mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y]. >

This holds whether or not $X$ and $Y$ are independent — one of the most powerful properties in probability. Proof sketch:

\begin{aligned} \mathbb{E}[aX + bY] &= \sum_{x,y} (ax + by) f_{XY}(x,y) \\ &= a\sum_{x,y} x f_{XY}(x,y) + b\sum_{x,y} y f_{XY}(x,y) \\ &= a\mathbb{E}[X] + b\mathbb{E}[Y]. \end{aligned}

Example: Expected value of Binomial via linearity

Y \sim \text{Binomial}(n,p)

. Write

Y = \sum_{i=1}^n X_i

where

X_i \sim \text{Bernoulli}(p)

i.i.d.

\mathbb{E}[Y] = \sum_{i=1}^n \mathbb{E}[X_i] = \sum_{i=1}^n p = np.

This is much simpler than summing

k\binom{n}{k}p^k(1-p)^{n-k}

directly!

5. Expected Value of Independent Products

If $X$ and $Y$ are independent:

> \mathbb{E}[XY] = \mathbb{E}[X] \cdot \mathbb{E}[Y]. >

Proof: Using independence

f_{XY}(x,y) = f_X(x)f_Y(y)

\mathbb{E}[XY] = \sum_{x,y} xy f_X(x) f_Y(y) = \left(\sum_x x f_X(x)\right)\left(\sum_y y f_Y(y)\right) = \mathbb{E}[X]\mathbb{E}[Y].

6. Moments

Definition (Moments) The $k$ -th raw moment of $X$ is $\mathbb{E}[X^k]$ . The $k$ -th central moment is $\mathbb{E}[(X - \mu)^k]$ , where $\mu = \mathbb{E}[X]$ .

$\mathbb{E}[X]$ = first raw moment (mean)
$\mathbb{E}[X^2]$ = second raw moment
$\operatorname{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2$ = second central moment

7. Practice Questions

Q1 (Easy)

X

takes values

\{-2,0,2,4\}

with probabilities

\{0.2, 0.3, 0.4, 0.1\}

. Find

\mathbb{E}[X]

Full Solution

$\mathbb{E}[X] = (-2)(0.2) + 0(0.3) + 2(0.4) + 4(0.1) = -0.4 + 0 + 0.8 + 0.4 = 0.8$ .

Q2 (Medium)

X \sim \text{Poisson}(4)

. Find

\mathbb{E}[X^2]

Full Solution

For Poisson( $\lambda$ ): $\mathbb{E}[X] = \lambda$ , $\operatorname{Var}(X) = \lambda$ .

$\operatorname{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 \implies \mathbb{E}[X^2] = \lambda + \lambda^2 = 4 + 16 = 20$ .

Q3 (Hard — LOTUS)

X \sim \text{Uniform}\{1,2,3,4,5,6\}

. Find

\mathbb{E}[X^3]

Full Solution

> \mathbb{E}[X^3] = \frac{1}{6}(1^3 + 2^3 + 3^3 + 4^3 + 5^3 + 6^3) = \frac{1}{6}(1 + 8 + 27 + 64 + 125 + 216) = \frac{441}{6} = 73.5. >

Next topic: Week 3.2: Variance & Standard Deviation — Variance and standard deviation. Join Discord PreviousWeek 2.3: Independence of Random VariablesNextWeek 3.2: Variance & Standard Deviation

What this note is really saying

Week 3.1: Expected Value

1. Intuition: The Balancing Point

2. Formal Definition

3. Expected Value of a Function: The LOTUS

Example

4. Linearity of Expectation

Example: Expected value of Binomial via linearity

5. Expected Value of Independent Products

6. Moments

7. Practice Questions

Q1 (Easy)

Q2 (Medium)

Q3 (Hard — LOTUS)

Intelligence Hub