Neural Sync Active

Week 2.1: Functions of Random Variables

Registry Synced

Week 2.1: Functions of Random Variables

1178 words

6 min read

Reading compass

Now · 1. Intuition: The "Push-Forward" Problem

Week 2.1: Functions of Random Variables

Prerequisites: Joint PMF (Week 1.1: Joint Probability Mass Function (Joint PMF)), Conditional Distributions (Week 1.3: Conditional Distributions) Cross-links: BSMA1002 (Stats 1) — transformations of RVs Core question: Given the distribution of $X$ , what is the distribution of $Y = g(X)$ ?

1. Intuition: The "Push-Forward" Problem

We know the probability of every value of

X

. If we apply a function

g

, the probability "flows" from

X

Y = g(X)

. The question: how does the PMF of

Y

relate to the PMF of

X

One-to-one functions: Each $X$ value maps to a unique $Y$ value (e.g., $Y = X + 5$ , $Y = 2^X$ ). Probability transfers directly.
Many-to-one functions: Multiple $X$ values map to the same $Y$ value (e.g., $Y = X^2$ , $Y = (X-5)^2$ ). We must add the probabilities of all $X$ values that lead to the same $Y$ .

2. General Method: The Table Method

Algorithm (PMF of $Y = g(X)$ )

List all possible values of $X$ with their probabilities.

Compute $y = g(x)$ for each $x$ .

Group by $y$ : for each distinct $y$ , sum the probabilities of all $x$ that map to it.

The result is $f_Y(y)$ . Formula:

f_Y(y) = \mathbb{P}(g(X) = y) = \sum_{x: g(x) = y} f_X(x).

3. One-to-One Functions

A function is one-to-one (injective) if

g(x_1) = g(x_2) \implies x_1 = x_2

. Monotonic functions (strictly increasing or decreasing) are one-to-one.

Key result: If $g$ is one-to-one and $Y = g(X)$ , then

> f_Y(g(x)) = f_X(x) \quad \text{or equivalently} \quad f_Y(y) = f_X(g^{-1}(y)). >

The PMF shape "shifts" but probability mass is preserved exactly.

Example: Uniform $\{0,\dots,10\}$ , $Y = X - 5$

$x$	$f_X(x)$	$y = x-5$
0	1/11	-5
1	1/11	-4
2	1/11	-3
3	1/11	-2
4	1/11	-1
5	1/11	0
6	1/11	1
7	1/11	2
8	1/11	3
9	1/11	4
10	1/11	5

Y \sim \text{Uniform}\{-5, -4, \dots, 5\}

, each with probability

1/11

Example: Binomial(10, 0.5), $Y = 2^X$

Each

x

maps uniquely to

2^x

. So

f_Y(2^x) = f_X(x)

. The PMF of

Y

looks "stretched" — values like

1, 2, 4, 8, \dots, 1024

each have the same probability as the corresponding

x

4. Many-to-One Functions

When multiple

x

values map to the same

y

, we add probabilities.

Example: $X \sim \text{Uniform}\{0,\dots,10\}$ , $Y = (X-5)^2$

$x$	$f_X(x)$	$y = (x-5)^2$
0	1/11	25
1	1/11	16
2	1/11	9
3	1/11	4
4	1/11	1
5	1/11	0
6	1/11	1
7	1/11	4
8	1/11	9
9	1/11	16
10	1/11	25

Grouping by

y

$y$	$f_Y(y)$	Calculation
0	1/11	Only $x=5$
1	2/11	$x=4,6$
4	2/11	$x=3,7$
9	2/11	$x=2,8$
16	2/11	$x=1,9$
25	2/11	$x=0,10$

Example: Clipped Function

Let

X \sim \text{Uniform}\{-5, -4, \dots, 5\}

and

g(x) = \begin{cases} x, & x > 0 \\ 0, & x \leq 0. \end{cases}

Find

f_Y(y)

. Solution: Values

x \leq 0

(

-5,-4,\dots,0

) all map to

y=0

. Each has probability

1/11

, so:

f_Y(0) = \frac{6}{11} \quad (\text{for } x=-5,-4,-3,-2,-1,0).

Values

x > 0

(

1,2,3,4,5

) map to themselves:

f_Y(1) = f_Y(2) = f_Y(3) = f_Y(4) = f_Y(5) = \frac{1}{11}.

Check:

6/11 + 5(1/11) = 1

. ✓

5. Functions of Multiple RVs

For

Z = g(X_1, \dots, X_n)

with joint PMF

f_{X_1\dots X_n}

> f_Z(z) = \mathbb{P}(g(X_1,\dots,X_n) = z) = \sum_{(x_1,\dots,x_n): g(x_1,\dots,x_n) = z} f_{X_1\dots X_n}(x_1,\dots,x_n). >

The table method works for small problems. For larger ones, we use contours (sets of

(x,y)

with

g(x,y) = z

Example: Sum of Two Dice

X,Y \sim \text{Uniform}\{1,\dots,6\}

independent,

Z = X + Y

. Range:

Z \in \{2,3,\dots,12\}

. Counting contour lines:

$z$	Pairs $(x,y)$	Count	$f_Z(z)$
2	(1,1)	1	1/36
3	(1,2),(2,1)	2	2/36
4	(1,3),(2,2),(3,1)	3	3/36
5	(1,4),(2,3),(3,2),(4,1)	4	4/36
6	(1,5),(2,4),(3,3),(4,2),(5,1)	5	5/36
7	(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)	6	6/36
8	(2,6),(3,5),(4,4),(5,3),(6,2)	5	5/36
9	(3,6),(4,5),(5,4),(6,3)	4	4/36
10	(4,6),(5,5),(6,4)	3	3/36
11	(5,6),(6,5)	2	2/36
12	(6,6)	1	1/36

Example: Max of Two Dice

X,Y \sim \text{Uniform}\{1,\dots,6\}

independent,

W = \max(X,Y)

W = w

when one die shows

w

and the other shows

\leq w

, excluding the case where both are

< w

f_W(w) = \frac{2w-1}{36}.

$w$	1	2	3	4	5	6
$f_W(w)$	1/36	3/36	5/36	7/36	9/36	11/36

6. Mermaid: Decision Flow for Finding $f_{g(X)}$

(Diagram)

7. Practice Questions

Q1 (Easy)

X \sim \text{Uniform}\{1,2,3,4\}

Y = 3X - 2

. Find

f_Y

Full Solution

One-to-one (linear). $Y$ takes values $1,4,7,10$ , each with probability $1/4$ .

$f_Y(1) = f_Y(4) = f_Y(7) = f_Y(10) = 1/4$ .

Q2 (Medium)

X \sim \text{Uniform}\{-3,-2,-1,0,1,2,3\}

Y = |X|

. Find

f_Y

Full Solution

Many-to-one. Mapping:

$Y=0$ : $X=0$ (1 value) → $f_Y(0) = 1/7$ $Y=1$ : $X=-1,1$ (2 values) → $f_Y(1) = 2/7$ $Y=2$ : $X=-2,2$ (2 values) → $f_Y(2) = 2/7$ $Y=3$ : $X=-3,3$ (2 values) → $f_Y(3) = 2/7$

Check: $1/7 + 2/7 + 2/7 + 2/7 = 1$ . ✓

Q3 (Hard)

Let

X \sim \text{Binomial}(5, 0.4)

. Find the PMF of

Y = X^2

Full Solution

$f_X(x) = \binom{5}{x} (0.4)^x (0.6)^{5-x}$ for $x=0,1,2,3,4,5$ .

Compute:

$x=0 \to y=0$ : $f_X(0) = (0.6)^5 = 0.07776$

$x=1 \to y=1$ : $f_X(1) = 5(0.4)(0.6)^4 = 0.2592$

$x=2 \to y=4$ : $f_X(2) = 10(0.4)^2(0.6)^3 = 0.3456$

$x=3 \to y=9$ : $f_X(3) = 10(0.4)^3(0.6)^2 = 0.2304$

$x=4 \to y=16$ : $f_X(4) = 5(0.4)^4(0.6) = 0.0768$

$x=5 \to y=25$ : $f_X(5) = (0.4)^5 = 0.01024$

This is one-to-one on $\{0,1,2,3,4,5\}$ , so $f_Y(y) = f_X(\sqrt{y})$ for $y \in \{0,1,4,9,16,25\}$ .

Q4 (Exam-style — Max)

Two fair dice are rolled. Find the PMF of

W = \max(X,Y)

using the contour method.

Full Solution

$W = w$ when $(X,Y)$ is in the set $\{(x,y): \max(x,y)=w\}$ . This means: either $X=w$ and $Y \leq w$ , or $Y=w$ and $X \leq w$ , but subtract the overlap $(w,w)$ counted twice.

Count of favourable outcomes: $w$ (for $X=w, Y=1..w$ ) $+ w$ (for $Y=w, X=1..w$ ) $- 1$ (double-count $(w,w)$ ) $= 2w-1$ .

So $f_W(w) = \frac{2w-1}{36}$ for $w=1,\dots,6$ .

$w$	1	2	3	4	5	6
$f_W(w)$	1/36	3/36	5/36	7/36	9/36	11/36

Next topic: Week 2.2: Sum (Convolution), Min & Max of Random Variables — Deep dive into convolution, min, and max of independent RVs. Join Discord PreviousWeek 1.4: Multiple Random Variables (n > 2)NextWeek 2.2: Sum (Convolution), Min & Max of Random Variables