Quiz 3

Week 2.2: Sum (Convolution), Min & Max of Random Variables

1035 words
5 min read
Python Week 1: the first filter for runtime behavior
Visual companion
Python
Type and operator map

Python Week 1: the first filter for runtime behavior

Open galleryRelated surface
View
Revision summary

What this note is really saying

Short form

# Week 2.2: Sum (Convolution), Min & Max of Random Variables > **Prerequisites:** Independence ([Week 2.3: Independence of Random Variables](/notes/01-foundation-bsma1004-stats-2-week02-07-independence)), Functions of RVs ([Week 2.1: Functions of Random Variables](/notes/01-foundation-bsma1004-stats-2-week02-05-func...

Week 2.2: Sum (Convolution), Min & Max of Random Variables

Prerequisites: Independence (Week 2.3: Independence of Random Variables), Functions of RVs (Week 2.1: Functions of Random Variables) Cross-links: BSMA3012 (Linear Stat Models) — order statistics Core question: What are the distributions of X+YX+Y, min⁡(X,Y)\min(X,Y), and max⁡(X,Y)\max(X,Y)?

1. Intuition

Three operations on random variables appear constantly in statistics:
  • Sum X+YX+Y: The total of independent measurements. Its distribution is found via convolution — the PMF of a sum is the convolution of the individual PMFs.
  • Maximum max⁡(X1,…,Xn)\max(X_1,\dots,X_n): The "best" outcome. Used in extreme value theory, reliability.
  • Minimum min⁡(X1,…,Xn)\min(X_1,\dots,X_n): The "weakest link". Used in survival analysis, system reliability.

2. Sum of Two Independent RVs: Convolution

2.1 General Formula

Let XX and YY be independent discrete RVs with PMFs fXf_X and fYf_Y. For Z=X+YZ = X + Y:
>fZ(z)=∑xfX(x)⋅fY(z−x)=∑yfX(z−y)⋅fY(y).>> f_Z(z) = \sum_{x} f_X(x) \cdot f_Y(z - x) = \sum_{y} f_X(z - y) \cdot f_Y(y). >
This operation is called the convolution of fXf_X and fYf_Y. Proof: Using the joint PMF and independence:
fZ(z)=P(X+Y=z)=∑xP(X=x,Y=z−x)=∑xfX(x)fY(z−x).\begin{aligned} f_Z(z) &= \mathbb{P}(X+Y = z) = \sum_{x} \mathbb{P}(X = x, Y = z - x) \\ &= \sum_{x} f_X(x) f_Y(z - x). \end{aligned}

2.2 Example: Sum of Two Independent Uniforms {0,1,2,3}\{0,1,2,3\}

X∼Uniform{0,1,2,3}X \sim \text{Uniform}\{0,1,2,3\}, Y∼Uniform{0,1,2,3}Y \sim \text{Uniform}\{0,1,2,3\}, independent. Z=X+YZ = X + Y, range Z∈{0,1,…,6}Z \in \{0,1,\dots,6\}. Using convolution:
fZ(z)=∑x=0314⋅fY(z−x).f_Z(z) = \sum_{x=0}^3 \frac{1}{4} \cdot f_Y(z-x).
zzTermsfZ(z)f_Z(z)
0x=0,y=0x=0,y=01/16
1(0,1)+(1,0)(0,1)+(1,0)2/16
2(0,2)+(1,1)+(2,0)(0,2)+(1,1)+(2,0)3/16
3(0,3)+(1,2)+(2,1)+(3,0)(0,3)+(1,2)+(2,1)+(3,0)4/16
4(1,3)+(2,2)+(3,1)(1,3)+(2,2)+(3,1)3/16
5(2,3)+(3,2)(2,3)+(3,2)2/16
6(3,3)(3,3)1/16

2.3 General Formula: Sum of Two i.i.d. Uniform{1,…,n}\{1,\dots,n\}

Let X,Y∼Uniform{1,…,n}X,Y \sim \text{Uniform}\{1,\dots,n\} i.i.d., W=X+YW = X+Y, range {2,…,2n}\{2,\dots,2n\}:
P(W=w)={w−1n2,2≤w≤n+1,2n−w+1n2,n+2≤w≤2n.\mathbb{P}(W = w) = \begin{cases} \frac{w-1}{n^2}, & 2 \leq w \leq n+1, \\[4pt] \frac{2n - w + 1}{n^2}, & n+2 \leq w \leq 2n. \end{cases}
This creates a triangular distribution — peaking at w=n+1w = n+1.

2.4 Key Convolution Results

Distribution 1Distribution 2Sum Distribution
Binomial( mm , pp )Binomial( nn , pp )Binomial( m+nm+n , pp )
Poisson( λ1\lambda_1 )Poisson( λ2\lambda_2 )Poisson( λ1+λ2\lambda_1+\lambda_2 )
Geometric( pp )Geometric( pp )Negative Binomial( 22 , pp )
Normal( μ1,σ12\mu_1,\sigma_1^2 )Normal( μ2,σ22\mu_2,\sigma_2^2 )Normal( μ1+μ2,σ12+σ22\mu_1+\mu_2,\sigma_1^2+\sigma_2^2 )
Example: X∼Poisson(3)X \sim \text{Poisson}(3), Y∼Poisson(5)Y \sim \text{Poisson}(5) independent → X+Y∼Poisson(8)X+Y \sim \text{Poisson}(8).

3. Distribution of the Maximum

3.1 General Approach Using CDF

For Z=max⁡(X,Y)Z = \max(X,Y):
FZ(z)=P(max⁡(X,Y)≤z)=P(X≤z,Y≤z)=FXY(z,z).\begin{aligned} F_Z(z) &= \mathbb{P}(\max(X,Y) \leq z) = \mathbb{P}(X \leq z, Y \leq z) \\ &= F_{XY}(z, z). \end{aligned}
If XX and YY are independent:
FZ(z)=FX(z)⋅FY(z).F_Z(z) = F_X(z) \cdot F_Y(z).
The PMF (for discrete) or PDF (for continuous) is then:
fZ(z)=FZ(z)−FZ(z−).f_Z(z) = F_Z(z) - F_Z(z^-).

3.2 Example: Max of Two Dice

X,Y∼Uniform{1,…,6}X,Y \sim \text{Uniform}\{1,\dots,6\} i.i.d. For w=1,…,6w = 1,\dots,6:
FZ(w)=P(X≤w)2=(w6)2fZ(w)=FZ(w)−FZ(w−1)=w236−(w−1)236=2w−136.\begin{aligned} F_Z(w) &= \mathbb{P}(X \leq w)^2 = \left(\frac{w}{6}\right)^2 \\ f_Z(w) &= F_Z(w) - F_Z(w-1) = \frac{w^2}{36} - \frac{(w-1)^2}{36} = \frac{2w-1}{36}. \end{aligned}

3.3 General: Max of nn i.i.d. RVs

For X1,…,XnX_1,\dots,X_n i.i.d. with CDF FF:
Fmax⁡(t)=[F(t)]n,fmax⁡(t)=n[F(t)]n−1f(t).F_{\max}(t) = [F(t)]^n, \quad f_{\max}(t) = n [F(t)]^{n-1} f(t).

4. Distribution of the Minimum

4.1 General Approach

For W=min⁡(X,Y)W = \min(X,Y):
FW(w)=P(min⁡(X,Y)≤w)=1−P(min⁡(X,Y)>w)=1−P(X>w,Y>w).\begin{aligned} F_W(w) &= \mathbb{P}(\min(X,Y) \leq w) = 1 - \mathbb{P}(\min(X,Y) > w) \\ &= 1 - \mathbb{P}(X > w, Y > w). \end{aligned}
If XX and YY are independent:
FW(w)=1−[1−FX(w)]⋅[1−FY(w)].F_W(w) = 1 - [1 - F_X(w)] \cdot [1 - F_Y(w)].

4.2 Example: Min of Two Dice

X,Y∼Uniform{1,…,6}X,Y \sim \text{Uniform}\{1,\dots,6\} i.i.d.:
FW(w)=1−(1−w6)2fW(w)=FW(w)−FW(w−1).\begin{aligned} F_W(w) &= 1 - \left(1 - \frac{w}{6}\right)^2 \\ f_W(w) &= F_W(w) - F_W(w-1). \end{aligned}
For w=1w=1: FW(1)=1−(5/6)2=11/36F_W(1) = 1 - (5/6)^2 = 11/36, so fW(1)=11/36f_W(1) = 11/36 (since FW(0)=0F_W(0)=0). For w=2w=2: FW(2)=1−(4/6)2=20/36F_W(2) = 1 - (4/6)^2 = 20/36, so fW(2)=20/36−11/36=9/36f_W(2) = 20/36 - 11/36 = 9/36. For w=3w=3: FW(3)=1−(3/6)2=27/36F_W(3) = 1 - (3/6)^2 = 27/36, so fW(3)=7/36f_W(3) = 7/36. For w=4w=4: FW(4)=1−(2/6)2=32/36F_W(4) = 1 - (2/6)^2 = 32/36, so fW(4)=5/36f_W(4) = 5/36. For w=5w=5: FW(5)=1−(1/6)2=35/36F_W(5) = 1 - (1/6)^2 = 35/36, so fW(5)=3/36f_W(5) = 3/36. For w=6w=6: FW(6)=1F_W(6) = 1, so fW(6)=1−35/36=1/36f_W(6) = 1 - 35/36 = 1/36.

4.3 General: Min of nn i.i.d. RVs

For X1,…,XnX_1,\dots,X_n i.i.d. with CDF FF:
Fmin⁡(t)=1−[1−F(t)]n,fmin⁡(t)=n[1−F(t)]n−1f(t).F_{\min}(t) = 1 - [1 - F(t)]^n, \quad f_{\min}(t) = n [1 - F(t)]^{n-1} f(t).

5. Min of Two Independent Geometrics

Let X∼Geometric(p)X \sim \text{Geometric}(p) and Y∼Geometric(p)Y \sim \text{Geometric}(p) be independent. For Geometric, FX(k)=1−(1−p)kF_X(k) = 1 - (1-p)^k for k=1,2,…k = 1,2,\dots.
Fmin⁡(k)=1−[1−FX(k)]2=1−[(1−p)k]2=1−(1−p)2kfmin⁡(k)=Fmin⁡(k)−Fmin⁡(k−1)=(1−p)2(k−1)−(1−p)2k=(1−p)2(k−1)[1−(1−p)2]=(1−p)2(k−1)(2p−p2).\begin{aligned} F_{\min}(k) &= 1 - [1 - F_X(k)]^2 = 1 - [(1-p)^k]^2 = 1 - (1-p)^{2k} \\ f_{\min}(k) &= F_{\min}(k) - F_{\min}(k-1) = (1-p)^{2(k-1)} - (1-p)^{2k} \\ &= (1-p)^{2(k-1)}[1 - (1-p)^2] = (1-p)^{2(k-1)} (2p - p^2). \end{aligned}
So min⁡(X,Y)∼Geometric(1−(1−p)2)=Geometric(2p−p2)\min(X,Y) \sim \text{Geometric}(1 - (1-p)^2) = \text{Geometric}(2p - p^2).

6. Mermaid: Sum vs Min vs Max

(Diagram)

7. Practice Questions

Q1 (Easy)

X∼Poisson(2)X \sim \text{Poisson}(2), Y∼Poisson(3)Y \sim \text{Poisson}(3) independent. Find P(X+Y=4)\mathbb{P}(X+Y = 4).
Full Solution
X+Y∼Poisson(5)X+Y \sim \text{Poisson}(5). P(X+Y=4)=e−5⋅544!=e−5⋅62524≈0.1755\mathbb{P}(X+Y=4) = e^{-5} \cdot \frac{5^4}{4!} = e^{-5} \cdot \frac{625}{24} \approx 0.1755.

Q2 (Medium)

Two fair dice are rolled. Find P(max⁡(X,Y)≥5)\mathbb{P}(\max(X,Y) \geq 5).
Full Solution
P(max⁡≥5)=1−P(max⁡≤4)=1−P(X≤4)2=1−(4/6)2=1−16/36=20/36=5/9\mathbb{P}(\max \geq 5) = 1 - \mathbb{P}(\max \leq 4) = 1 - \mathbb{P}(X \leq 4)^2 = 1 - (4/6)^2 = 1 - 16/36 = 20/36 = 5/9.

Q3 (Hard)

X∼Geometric(0.2)X \sim \text{Geometric}(0.2), Y∼Geometric(0.3)Y \sim \text{Geometric}(0.3) independent. Find P(min⁡(X,Y)>5)\mathbb{P}(\min(X,Y) > 5).
Full Solution
P(min⁡>5)=P(X>5,Y>5)=P(X>5)⋅P(Y>5)\mathbb{P}(\min > 5) = \mathbb{P}(X > 5, Y > 5) = \mathbb{P}(X > 5) \cdot \mathbb{P}(Y > 5).
For Geometric: P(X>k)=(1−p)k\mathbb{P}(X > k) = (1-p)^k.
P(X>5)=(0.8)5=0.32768\mathbb{P}(X > 5) = (0.8)^5 = 0.32768, P(Y>5)=(0.7)5=0.16807\mathbb{P}(Y > 5) = (0.7)^5 = 0.16807.
P(min⁡>5)=0.32768×0.16807≈0.0551\mathbb{P}(\min > 5) = 0.32768 \times 0.16807 \approx 0.0551.
Next topic: Week 2.3: Independence of Random Variables — Independence of random variables, i.i.d. sequences. Join Discord PreviousWeek 2.1: Functions of Random VariablesNextWeek 2.3: Independence of Random Variables
Document outline

Keep your place and jump directly to a heading.

Table of contents

No note selected

Open a chapter to see its outline, backlinks, and sources here.

Open libraryCheckpoint desk