Week 1.1: Joint Probability Mass Function (Joint PMF)

2240 words

11 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Week 1.1: Joint Probability Mass Function (Joint PMF) > **Prerequisites:** BSMA1002 (Stats 1) — Probability, PMF, random variables > **Cross-links:** BSMA3012 (Linear Stat Models) — multivariate analysis > **Core question:** When two or more random variables live in the same experiment, how do we describe their _j...

Week 1.1: Joint Probability Mass Function (Joint PMF)

Prerequisites: BSMA1002 (Stats 1) — Probability, PMF, random variables Cross-links: BSMA3012 (Linear Stat Models) — multivariate analysis Core question: When two or more random variables live in the same experiment, how do we describe their joint behaviour?

1. Intuition: Why Joint Distributions?

In Stats 1, we studied random variables one at a time. But real experiments produce multiple measurements simultaneously:

A cricket over: runs scored ( $X$ ) and wickets lost ( $Y$ )
A patient: height ( $X$ ) and weight ( $Y$ )
A stock: today's return ( $X_1$ ), yesterday's return ( $X_2$ ) The value of one variable can influence the other. A wicket tends to reduce runs. Taller people tend to weigh more. Stocks cluster.

Key idea: The joint distribution of $X$ and $Y$ captures the full probabilistic relationship — not just how each behaves alone, but how they co-vary.

2. Formal Definition

2.1 Joint PMF (Two Discrete RVs)

Suppose

X

and

Y

are discrete random variables defined on the same probability space, with ranges

T_X

and

T_Y

respectively.

Definition (Joint PMF) The joint probability mass function of $X$ and $Y$ , denoted $f_{XY}$ , is

> f_{XY}(t_1, t_2) = \mathbb{P}(X = t_1 \text{ and } Y = t_2), \quad t_1 \in T_X, \; t_2 \in T_Y. >

Properties:

$0 \leq f_{XY}(t_1, t_2) \leq 1$
$\displaystyle\sum_{t_1 \in T_X}\sum_{t_2 \in T_Y} f_{XY}(t_1, t_2) = 1$
The joint PMF completely determines all probabilistic relationships between $X$ and $Y$ .

Notation: We write $\mathbb{P}(X = t_1, Y = t_2)$ as shorthand for $\mathbb{P}(X = t_1 \text{ and } Y = t_2)$ .

3. Visualising Joint PMF: The Contingency Table

The joint PMF is most commonly displayed as a two-way table (also called a contingency table). Rows correspond to values of

Y

, columns to values of

X

$X \backslash Y$	$y_1$	$y_2$	$\dots$	$y_m$
$x_1$	$p_{11}$	$p_{12}$	$\dots$	$p_{1m}$
$x_2$	$p_{21}$	$p_{22}$	$\dots$	$p_{2m}$
$\vdots$	$\vdots$	$\vdots$	$\ddots$	$\vdots$
$x_n$	$p_{n1}$	$p_{n2}$	$\dots$	$p_{nm}$

where

p_{ij} = \mathbb{P}(X = x_i, Y = y_j)

and

\sum_{i,j} p_{ij} = 1

4. Worked Examples

Example 1: Toss a Fair Coin Twice

Let

X_1 = 1

if first toss is heads (0 otherwise),

X_2 = 1

if second toss is heads (0 otherwise). Since the coin is fair and tosses are independent, each of the 4 outcomes has probability

1/4

. Joint PMF table:

$X_2 \backslash X_1$	0	1
0	$1/4$	$1/4$
1	$1/4$	$1/4$

Verification: Sum of all entries

= 1/4 + 1/4 + 1/4 + 1/4 = 1

. ✓

Example 2: Random 2-Digit Number (00–99)

Select a number uniformly from

\{00, 01, \dots, 99\}

. Define:

$X$ = digit in the units place (0–9)
$Y$ = remainder when the number is divided by 4 (0–3)

Question: What is $\mathbb{P}(X = 0, Y = 0)$ ? Solution: We need numbers ending in 0 that are divisible by 4. Numbers ending in 0 that are multiples of 4: $\{00, 20, 40, 60, 80\}$ — 5 numbers.

f_{XY}(0, 0) = \frac{5}{100} = \frac{1}{20}

Full joint PMF table:

$Y \backslash X$	0	1	2	3	4	5	6	7	8	9
0	1/20	0	1/20	0	1/20	0	1/20	0	1/20	0
1	0	1/20	0	1/20	0	1/20	0	1/20	0	1/20
2	1/20	0	1/20	0	1/20	0	1/20	0	1/20	0
3	0	1/20	0	1/20	0	1/20	0	1/20	0	1/20

Observations:

$X$ and $Y$ are not independent — knowing $X$ tells us something about $Y$ (e.g., if $X$ is odd, $Y$ must be odd too).
Both marginals are uniform: $X \sim \text{Uniform}\{0,\dots,9\}$ , $Y \sim \text{Uniform}\{0,1,2,3\}$ .

Example 3: Die Roll + Coin Tosses (Hierarchical Experiment)

Throw a fair die. Let $X$ be the number shown. Toss a fair coin $X$ times. Let $Y$ be the number of heads. Step 1 — Marginal of $X$ : $X \sim \text{Uniform}\{1, 2, 3, 4, 5, 6\}$ , so $f_X(t) = \frac{1}{6}$ . Step 2 — Conditional of $Y$ given $X = t$ : Given $X = t$ , we toss a coin $t$ times, so

(Y \mid X = t) \sim \text{Binomial}\left(t, \frac{1}{2}\right)

f_{Y \mid X = t}(y) = \binom{t}{y} \left(\frac{1}{2}\right)^t, \quad y = 0, 1, \dots, t.

Step 3 — Joint PMF: Using the factorization

f_{XY}(t, y) = f_X(t) \cdot f_{Y \mid X = t}(y)

f_{XY}(t, y) = \frac{1}{6} \cdot \binom{t}{y} \left(\frac{1}{2}\right)^t, \quad t = 1,\dots,6,\; y = 0,\dots,t.

For instance:

f_{XY}(3, 1) = \frac{1}{6} \cdot \binom{3}{1} \left(\frac{1}{2}\right)^3 = \frac{1}{6} \cdot 3 \cdot \frac{1}{8} = \frac{3}{48} = \frac{1}{16}.

5. Joint PMF for More Than Two RVs

The definition extends naturally to

n

random variables

X_1, X_2, \dots, X_n

Definition (Joint PMF for $n$ RVs)

> f_{X_1 \dots X_n}(t_1, \dots, t_n) = \mathbb{P}(X_1 = t_1, \dots, X_n = t_n), \quad t_i \in T_{X_i}. >

Example — Toss a fair coin thrice:

$X_1$	$X_2$	$X_3$	$f_{X_1X_2X_3}$
0	0	0	1/8
0	0	1	1/8
0	1	0	1/8
0	1	1	1/8
1	0	0	1/8
1	0	1	1/8
1	1	0	1/8
1	1	1	1/8

6. Formula Summary

Concept	Formula	Example
Joint PMF (2 RVs)	$\mathbb{P}(X = x, Y = y) = f_{XY}(x, y)$	Coin toss: $f_{X_1X_2}(0,0) = 1/4$
Total probability	$\sum_x \sum_y f_{XY}(x,y) = 1$	$\sum_{i=0}^1\sum_{j=0}^1 1/4 = 1$
Factorization	$f_{XY}(x,y) = f_X(x) \cdot f_{Y \mid X = x}(y)$	Die+coin: $1/6 \cdot \binom{t}{y}(1/2)^t$
Joint PMF ( $n$ RVs)	$f_{X_1\dots X_n}(t_1,\dots,t_n) = \mathbb{P}(\bigcap_i X_i = t_i)$	Triple coin: each outcome $=1/8$

7. Practice Questions

Q1 (Easy)

A fair coin is tossed 3 times. Let

X

= number of heads in first 2 tosses,

Y

= number of heads in last 2 tosses. Find the joint PMF of

X

and

Y

Strategy Hint

List all 8 outcomes. Count heads in positions (1,2) for $X$ , positions (2,3) for $Y$ . Full Solution

Sample space: $\{HHH, HHT, HTH, HTT, THH, THT, TTH, TTT\}$ , each with probability $1/8$ .

Compute $(X, Y)$ for each:

Outcome	$X$	$Y$
HHH	2	2
HHT	2	1
HTH	1	1
HTT	1	0
THH	1	2
THT	1	1
TTH	0	1
TTT	0	0

Now count joint probabilities:

$X \backslash Y$	0	1	2
0	1/8	1/8	0
1	1/8	2/8	1/8
2	0	1/8	1/8

Check: sum $= 1/8 + 1/8 + 1/8 + 2/8 + 1/8 + 1/8 + 1/8 = 8/8 = 1$ . ✓

Q2 (Medium)

A random 3-digit number from 000 to 999 is selected uniformly. Define

X

= first digit (hundreds place),

Y

= number modulo 2,

Z

= last digit (units place). Find

\mathbb{P}(X = 0, Y = 0, Z = 0)

Strategy Hint

$X=0$ means the number is in 000–099. $Y=0$ means even. $Z=0$ means ends in 0. Count numbers in 000–099 that are even and end in 0. Full Solution

Numbers in 000–099 are $\{00, 01, \dots, 99\}$ (100 numbers). Among these, those ending in 0 and even: $\{00, 10, 20, 30, 40, 50, 60, 70, 80, 90\}$ — 10 numbers.

> \mathbb{P}(X=0, Y=0, Z=0) = \frac{10}{1000} = \frac{1}{100}. >

Alternatively, compute via independence: $X$ and $Z$ are independent uniform $\{0,\dots,9\}$ , $Y$ depends on $Z$ only. $f_{XYZ}(0,0,0) = f_X(0) \cdot f_{Z}(0) \cdot \mathbb{P}(Y=0 \mid Z=0) = (1/10)(1/10)(1) = 1/100$ .

Q3 (Hard — IPL Data)

In an IPL powerplay over, let

X

= number of runs,

Y

= number of wickets. Suppose:

Y \sim \left\{ \frac{13}{16}, \frac{1}{8}, \frac{1}{16} \right\} \text{ with values } \{0, 1, 2\}

(X \mid Y=0) \sim \text{Uniform}\{6,7,8,9,10,11,12\}

(X \mid Y=1) \sim \text{Uniform}\{2,3,4,5,6,7,8\}

(X \mid Y=2) \sim \text{Uniform}\{0,1,2,3,4,5,6\}

Find

\mathbb{P}(X = 6, Y = 0)

and

\mathbb{P}(X = 6)

Strategy Hint

Use factorization $f_{XY}(x,y) = f_Y(y) \cdot f_{X \mid Y=y}(x)$ . For $\mathbb{P}(X=6)$ , sum over all $y$ . Full Solution

Part 1: $\mathbb{P}(X=6, Y=0)$

> f_{XY}(6,0) = f_Y(0) \cdot f_{X \mid Y=0}(6) = \frac{13}{16} \cdot \frac{1}{7} = \frac{13}{112}. >

Part 2: $\mathbb{P}(X=6)$

> \begin{aligned} \mathbb{P}(X=6) &= f_{XY}(6,0) + f_{XY}(6,1) + f_{XY}(6,2) \\ &= \frac{13}{16}\cdot\frac{1}{7} + \frac{1}{8}\cdot\frac{1}{7} + \frac{1}{16}\cdot\frac{1}{7} \\ &= \frac{13}{112} + \frac{1}{56} + \frac{1}{112} \\ &= \frac{13 + 2 + 1}{112} = \frac{16}{112} = \frac{1}{7}. \end{aligned} >

So $\mathbb{P}(X=6) = 1/7$ .

Q4 (Easy)

For the random 2-digit number example (Example 2), verify that

\sum_{x=0}^9 \sum_{y=0}^3 f_{XY}(x,y) = 1

Full Solution

From the table: each of the 40 cells has probability $1/20$ or $0$ . The non-zero cells are exactly those where $x$ mod 2 = $y$ mod 2 (both even or both odd). There are 20 such cells (5 even $x$ values $\times$ 2 even $y$ values + 5 odd $x$ values $\times$ 2 odd $y$ values). Each has $1/20$ .

Sum $= 20 \times 1/20 = 1$ . ✓

Q5 (Medium)

For the triple coin toss, find the joint PMF of

X = X_1 + X_2

(sum of first two) and

Y = X_2 + X_3

(sum of last two).

Strategy Hint

List all 8 outcomes, compute $(X,Y)$ for each, then count frequencies. Full Solution

Outcomes: HHH→(2,2), HHT→(2,1), HTH→(1,1), HTT→(1,0), THH→(1,2), THT→(1,1), TTH→(0,1), TTT→(0,0).

Joint PMF table:

$X \backslash Y$	0	1	2
0	1/8	1/8	0
1	1/8	2/8	1/8
2	0	1/8	1/8

Sum $= 8/8 = 1$ . ✓

Q6 (Hard)

Let

X \sim \text{Uniform}\{1,2,3\}

and independently

Y \sim \text{Uniform}\{1,2\}

. Find the joint PMF of

X

and

Y

Full Solution

Since $X$ and $Y$ are independent: $f_{XY}(x,y) = f_X(x) \cdot f_Y(y)$ .

$f_X(x) = 1/3$ for $x = 1,2,3$ . $f_Y(y) = 1/2$ for $y = 1,2$ .

So $f_{XY}(x,y) = 1/3 \cdot 1/2 = 1/6$ for all $x \in \{1,2,3\}, y \in \{1,2\}$ .

$Y \backslash X$	1	2	3
1	1/6	1/6	1/6
2	1/6	1/6	1/6

Check: $3 \times 2 \times 1/6 = 1$ . ✓

Q7 (Exam-style)

A fair die is rolled twice. Let

X

= maximum of the two rolls,

Y

= minimum of the two rolls. Find

\mathbb{P}(X = 5, Y = 3)

Strategy Hint

The ordered pairs that give max=5, min=3 are (3,5) and (5,3). Count the unordered outcomes carefully. Full Solution

There are $6 \times 6 = 36$ equally likely ordered outcomes.

$X=5, Y=3$ occurs when the two rolls are 3 and 5. The ordered pairs are $(3,5)$ and $(5,3)$ .

> \mathbb{P}(X=5, Y=3) = \frac{2}{36} = \frac{1}{18}. >

Q8 (Medium)

For the 2-digit number example, compute

\mathbb{P}(X \leq 4, Y = 1)

Full Solution

$Y=1$ means the number modulo 4 is 1. $X \leq 4$ means units digit is 0,1,2,3,4.

From the joint PMF table, for $Y=1$ :

$X=0$ : 0

$X=1$ : 1/20

$X=2$ : 0

$X=3$ : 1/20

$X=4$ : 0

Sum $= 1/20 + 1/20 = 1/10$ .

Q9 (Hard — Three RVs)

Let

X_1, X_2, X_3

be i.i.d. Bernoulli(

p

). Find the joint PMF and verify it sums to 1.

Full Solution

Since $X_i$ are independent and identically distributed Bernoulli( $p$ ):

> \mathbb{P}(X_i = 0) = 1-p, \quad \mathbb{P}(X_i = 1) = p. >

Joint PMF (using independence):

> f_{X_1X_2X_3}(t_1, t_2, t_3) = p^{\sum t_i} (1-p)^{3 - \sum t_i}. >

Sum over all 8 outcomes:

> \sum_{(t_1,t_2,t_3) \in \{0,1\}^3} p^{\sum t_i} (1-p)^{3-\sum t_i} = (p + (1-p))^3 = 1^3 = 1. >

where the identity $\sum_{k=0}^3 \binom{3}{k} p^k (1-p)^{3-k} = (p+(1-p))^3$ is used.

Q10 (Exam-style)

For the IPL example (Q3), find

\mathbb{P}(Y=0 \mid X=8)

Strategy Hint

Use Bayes' theorem: $\mathbb{P}(Y=0 \mid X=8) = f_{XY}(8,0) / f_X(8)$ . Compute $f_X(8)$ by summing over $Y$ . Full Solution

First compute $f_X(8)$ :

> \begin{aligned} f_X(8) &= f_{XY}(8,0) + f_{XY}(8,1) + f_{XY}(8,2) \\ &= \frac{13}{16}\cdot\frac{1}{7} + \frac{1}{8}\cdot\frac{1}{7} + \frac{1}{16}\cdot\frac{1}{7} \\ &= \frac{13}{112} + \frac{2}{112} + \frac{1}{112} = \frac{16}{112} = \frac{1}{7}. \end{aligned} >

Now:

> \mathbb{P}(Y=0 \mid X=8) = \frac{f_{XY}(8,0)}{f_X(8)} = \frac{13/112}{1/7} = \frac{13}{112} \cdot \frac{7}{1} = \frac{13}{16}. >

Next topic: Week 1.2: Marginal Distributions — Summing over rows and columns to get marginal PMFs. Join Discord NextWeek 1.2: Marginal Distributions

What this note is really saying

Week 1.1: Joint Probability Mass Function (Joint PMF)

1. Intuition: Why Joint Distributions?

2. Formal Definition

2.1 Joint PMF (Two Discrete RVs)

3. Visualising Joint PMF: The Contingency Table

4. Worked Examples

Example 1: Toss a Fair Coin Twice

Example 2: Random 2-Digit Number (00–99)

Example 3: Die Roll + Coin Tosses (Hierarchical Experiment)

5. Joint PMF for More Than Two RVs

6. Formula Summary

7. Practice Questions

Q1 (Easy)

Q2 (Medium)

Q3 (Hard — IPL Data)

Q4 (Easy)

Q5 (Medium)

Q6 (Hard)

Q7 (Exam-style)

Q8 (Medium)

Q9 (Hard — Three RVs)

Q10 (Exam-style)

Intelligence Hub