Week 6.1: Moment Generating Functions

Prerequisites: Expectation (Week 3.1: Expected Value), Independence (Week 2.3: Independence of Random Variables) Cross-links: BSMA1001 (Maths 1) — series expansions Core question: Is there a single function that encodes all moments of a distribution?

1. Intuition: A "Generating" Function

2. Definition

Definition (MGF) For a random variable $X$, the MGF is:

defined for $t$ in a neighbourhood of $0$ where the expectation exists. For discrete: $M_X(t) = \sum_x e^{tx} f_X(x)$ For continuous: $M_X(t) = \int_{-\infty}^\infty e^{tx} f_X(x) \, dx$

3. Key Properties

1. $M_X(0) = 1$
2. $M_X^{(k)}(0) = \mathbb{E}[X^k]$ — the $k$-th derivative at 0 gives the $k$-th moment
3. $M_{aX+b}(t) = e^{bt} M_X(at)$
4. If $X$ and $Y$ are independent: $M_{X+Y}(t) = M_X(t) \cdot M_Y(t)$
5. If $M_X(t) = M_Y(t)$ in a neighbourhood of 0, then $X$ and $Y$ have the same distribution

4. MGFs of Common Distributions

5. Using MGFs: Sum of Independent Normals

6. Practice Questions

Q1 (Easy)

Full Solution

$M_X(t) = \exp(3(e^t-1))$.

$M_X'(t) = \exp(3(e^t-1)) \cdot 3e^t$, so $\mathbb{E}[X] = M_X'(0) = \exp(0) \cdot 3 = 3$. ✓

Q2 (Medium)

Full Solution

$M_X(t) = \frac{2}{2-t}$ for $t < 2$.

$M_X'(t) = \frac{2}{(2-t)^2}$, $M_X''(t) = \frac{4}{(2-t)^3}$.

$\mathbb{E}[X] = M_X'(0) = 2/4 = 1/2$. $\mathbb{E}[X^2] = M_X''(0) = 4/8 = 1/2$.

Next topic: Week 6.2: Sampling Distributions — Sampling distributions. Join Discord PreviousWeek 5.3: Bivariate Normal DistributionNextWeek 6.2: Sampling Distributions