Neural Sync Active

Week 1.3: Conditional Distributions

Registry Synced

Week 1.3: Conditional Distributions

1213 words

6 min read

Reading compass

Now · 1. Intuition: Updating Beliefs

Week 1.3: Conditional Distributions

Prerequisites: Joint PMF (Week 1.1: Joint Probability Mass Function (Joint PMF)), Marginal Distributions (Week 1.2: Marginal Distributions) Cross-links: BSMA1002 (Stats 1) — conditional probability, Bayes' theorem Core question: How does knowing the value of one random variable change the distribution of another?

1. Intuition: Updating Beliefs

In Statistics 1, you learned conditional probability:

\mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}

. Conditional distributions extend this idea to random variables. When we observe

X = x

, our uncertainty about $Y$ updates — the conditional distribution

Y \mid X = x

replaces the unconditional (marginal) distribution of

Y

. This is the foundation of prediction and Bayesian inference.

Example: In the IPL over, before seeing wickets we expect ~6-7 runs. But if we learn $Y = 2$ (two wickets fallen), our expectation drops to ~3 runs.

2. Conditional Distribution Given an Event

First, we review the simpler case: conditioning on an event

A

Definition (Conditional PMF given an event) Let $X$ be discrete with range $T_X$ , and $A$ an event with $\mathbb{P}(A) > 0$ . The conditional PMF of $X$ given $A$ is

> f_{X \mid A}(t) = \mathbb{P}(X = t \mid A) = \frac{\mathbb{P}((X = t) \cap A)}{\mathbb{P}(A)}, \quad t \in T_X. >

Key properties:

$f_{X \mid A}$ is a valid PMF: $\sum_{t \in T_X} f_{X \mid A}(t) = 1$
The range of $(X \mid A)$ may differ from $T_X$ (some values may become impossible)

3. Conditional Distribution of $Y$ Given $X = x$

Definition (Conditional PMF) Let $X, Y$ have joint PMF $f_{XY}$ . For any $x \in T_X$ with $f_X(x) > 0$ , the conditional PMF of $Y$ given $X = x$ is

> f_{Y \mid X = x}(y) = \mathbb{P}(Y = y \mid X = x) = \frac{f_{XY}(x, y)}{f_X(x)}, \quad y \in T_Y. >

Similarly, the conditional PMF of

X

given

Y = y

> f_{X \mid Y = y}(x) = \frac{f_{XY}(x, y)}{f_Y(y)}, \quad f_Y(y) > 0. >

Implications:

The joint PMF factorises as $f_{XY}(x,y) = f_X(x) \cdot f_{Y \mid X = x}(y)$
If $X$ and $Y$ are independent, $f_{Y \mid X = x}(y) = f_Y(y)$ — conditioning does nothing

4. Worked Examples

Example 1: Fair Coin Tossed Twice

From the joint PMF table:

$X_2 \backslash X_1$	0	1
0	1/4	1/4
1	1/4	1/4

Compute

f_{X_2 \mid X_1 = 0}(y)

f_{X_2 \mid X_1 = 0}(0) = \frac{f_{X_1X_2}(0,0)}{f_{X_1}(0)} = \frac{1/4}{1/2} = \frac{1}{2}, \quad f_{X_2 \mid X_1 = 0}(1) = \frac{1/4}{1/2} = \frac{1}{2}.

This equals the marginal

f_{X_2}

— as expected since

X_1

and

X_2

are independent.

Example 2: Random 2-Digit Number

From the joint PMF table, compute

f_{Y \mid X = 0}(y)

f_X(0) = 1/10

, and only

Y=0

and

Y=2

have non-zero joint probability with

X=0

f_{Y \mid X=0}(0) = \frac{1/20}{1/10} = \frac{1}{2}, \quad f_{Y \mid X=0}(2) = \frac{1/20}{1/10} = \frac{1}{2}, \quad f_{Y \mid X=0}(1) = f_{Y \mid X=0}(3) = 0.

So if we see the units digit is 0, the remainder modulo 4 is equally likely 0 or 2 — it cannot be 1 or 3.

Example 3: Die Roll + Coin Tosses (Hierarchical)

X \sim \text{Uniform}\{1,\dots,6\}

(Y \mid X=t) \sim \text{Binomial}(t, 1/2)

. The conditional PMF is given in the problem statement itself:

f_{Y \mid X = t}(y) = \binom{t}{y} \left(\frac{1}{2}\right)^t, \quad y = 0,1,\dots,t.

But what about the reverse conditional:

f_{X \mid Y = y}(x)

? Using Bayes' theorem:

f_{X \mid Y = y}(x) = \frac{f_{Y \mid X = x}(y) \cdot f_X(x)}{f_Y(y)} = \frac{\binom{x}{y}(1/2)^x \cdot (1/6)}{f_Y(y)}.

For example, if we observed

Y = 3

heads, what's the probability the die showed

X = 4

? First compute

f_Y(3)

\begin{aligned} f_Y(3) &= \sum_{t=1}^6 f_{XY}(t,3) = \sum_{t=3}^6 \frac{1}{6} \binom{t}{3} \left(\frac{1}{2}\right)^t \\ &= \frac{1}{6} \left[ \binom{3}{3}\frac{1}{8} + \binom{4}{3}\frac{1}{16} + \binom{5}{3}\frac{1}{32} + \binom{6}{3}\frac{1}{64} \right] \\ &= \frac{1}{6} \left[ \frac{1}{8} + \frac{4}{16} + \frac{10}{32} + \frac{20}{64} \right] \\ &= \frac{1}{6} \left[ \frac{8}{64} + \frac{16}{64} + \frac{20}{64} + \frac{20}{64} \right] = \frac{1}{6} \cdot \frac{64}{64} = \frac{1}{6}. \end{aligned}

Now:

f_{X \mid Y=3}(4) = \frac{\binom{4}{3}(1/2)^4 \cdot (1/6)}{1/6} = \binom{4}{3} \cdot \frac{1}{16} = \frac{4}{16} = \frac{1}{4}.

5. Conditional Distributions for Multiple RVs

For

n

random variables, we can condition on any subset. The key idea: factorise the joint PMF as a product of conditionals:

f_{X_1\dots X_n}(t_1,\dots,t_n) = f_{X_1 \mid X_2 = t_2, \dots, X_n = t_n}(t_1) \cdot f_{X_2 \mid X_3 = t_3, \dots, X_n = t_n}(t_2) \cdots f_{X_n}(t_n).

This chain rule of probability can be applied in any order. Example: For

X_1, X_2, X_3, X_4

\begin{aligned} f_{X_1X_2X_3X_4} &= f_{X_1 \mid X_2, X_3, X_4} \cdot f_{X_2 \mid X_3, X_4} \cdot f_{X_3 \mid X_4} \cdot f_{X_4} \\ &= f_{X_4 \mid X_3, X_2, X_1} \cdot f_{X_3 \mid X_2, X_1} \cdot f_{X_2 \mid X_1} \cdot f_{X_1} \quad (\text{same thing, different order}) \end{aligned}

6. Mermaid Flowchart: Choosing the Right Distribution

(Diagram)

7. Formula Summary

Concept	Formula	When to Use
Conditional PMF	$f_{Y \mid X=x}(y) = \frac{f_{XY}(x,y)}{f_X(x)}$	Need distribution of $Y$ after observing $X=x$
Factorisation	$f_{XY}(x,y) = f_X(x) \cdot f_{Y \mid X=x}(y)$	Building joint from conditionals
Chain rule (n RVs)	$f = f_{X_1 \mid X_2,\dots,X_n} \cdot f_{X_2 \mid X_3,\dots,X_n} \cdots f_{X_n}$	Factorising high-dimensional joints
Bayes for RVs	$f_{X \mid Y=y}(x) = \frac{f_{Y \mid X=x}(y) f_X(x)}{f_Y(y)}$	Reversing the conditioning

8. Practice Questions

Q1 (Easy)

From the joint PMF below, find

f_{Y \mid X=1}(y)

$Y \backslash X$	0	1
0	0.3	0.2
1	0.1	0.4

Full Solution

$f_X(1) = 0.2 + 0.4 = 0.6$

> f_{Y \mid X=1}(0) = \frac{0.2}{0.6} = \frac{1}{3}, \quad f_{Y \mid X=1}(1) = \frac{0.4}{0.6} = \frac{2}{3}. >

Check: $1/3 + 2/3 = 1$ . ✓

Q2 (Medium)

For the same table, find

f_{X \mid Y=0}(x)

. Compare with

f_X(x)

Full Solution

$f_Y(0) = 0.3 + 0.2 = 0.5$

> f_{X \mid Y=0}(0) = \frac{0.3}{0.5} = 0.6, \quad f_{X \mid Y=0}(1) = \frac{0.2}{0.5} = 0.4. >

Marginal: $f_X(0) = 0.4$ , $f_X(1) = 0.6$ . These are different from the conditionals, confirming $X$ and $Y$ are dependent.

Q3 (Hard — IPL)

For the IPL example, find

f_{Y \mid X=10}(y)

Full Solution

First compute $f_X(10)$ :

> \begin{aligned} f_X(10) &= f_{XY}(10,0) + f_{XY}(10,1) + f_{XY}(10,2) \\ &= \frac{13}{16}\cdot\frac{1}{7} + \frac{1}{8}\cdot\frac{1}{7} + \frac{1}{16}\cdot\frac{1}{7} = \frac{16}{112} = \frac{1}{7}. \end{aligned} >

Now:

> f_{Y \mid X=10}(0) = \frac{13/112}{1/7} = \frac{13}{16}, \quad f_{Y \mid X=10}(1) = \frac{1/56}{1/7} = \frac{1}{8}, \quad f_{Y \mid X=10}(2) = \frac{1/112}{1/7} = \frac{1}{16}. >

So $Y \mid X=10$ has the same distribution as the marginal of $Y$ — interestingly, $X=10$ gives no extra info about $Y$ in this model!

Next topic: Week 1.4: Multiple Random Variables (n > 2) — Extending to more than two random variables. Join Discord PreviousWeek 1.2: Marginal DistributionsNextWeek 1.4: Multiple Random Variables (n > 2)