Week 5.1: Joint Continuous Distributions

598 words

3 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Week 5.1: Joint Continuous Distributions > **Prerequisites:** Continuous RVs ([Week 4.1: Continuous Random Variables & PDFs](/notes/01-foundation-bsma1004-stats-2-week04-12-continuous-rvs-pdf)), Joint Discrete ([Week 1.1: Joint Probability Mass Function (Joint PMF)](/notes/01-foundation-bsma1004-stats-2-week01-01-...

Week 5.1: Joint Continuous Distributions

Prerequisites: Continuous RVs (Week 4.1: Continuous Random Variables & PDFs), Joint Discrete (Week 1.1: Joint Probability Mass Function (Joint PMF)) Cross-links: BSMA3012 (Linear Stat Models) — multivariate normal Core question: How do we describe multiple continuous random variables together?

1. Intuition: From Double Sums to Double Integrals

For discrete RVs, joint PMF

f_{XY}(x,y)

sums to 1. For continuous RVs, we have a joint PDF

f_{XY}(x,y)

that integrates to 1:

\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{XY}(x,y) \, dx \, dy = 1.

Probabilities are volumes under this surface.

2. Formal Definitions

Definition (Joint PDF) $X$ and $Y$ are jointly continuous with PDF $f_{XY}(x,y)$ if:

$f_{XY}(x,y) \geq 0$

$\iint_{\mathbb{R}^2} f_{XY}(x,y) \, dx \, dy = 1$

$\mathbb{P}((X,Y) \in A) = \iint_A f_{XY}(x,y) \, dx \, dy$ Marginal PDFs:

f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y) \, dy, \quad f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x,y) \, dx.

Conditional PDF:

f_{Y \mid X = x}(y) = \frac{f_{XY}(x,y)}{f_X(x)}, \quad f_X(x) > 0.

Independence:

f_{XY}(x,y) = f_X(x) \cdot f_Y(y) \quad \text{for all } x,y.

3. Worked Example

Let

f_{XY}(x,y) = c(x+y)

for

0 \leq x \leq 1

0 \leq y \leq 1

, zero elsewhere. Step 1 — Find $c$ :

1 = \int_0^1 \int_0^1 c(x+y) \, dx \, dy = c \int_0^1 \left[ \frac{x^2}{2} + xy \right]_{x=0}^1 dy = c \int_0^1 \left( \frac{1}{2} + y \right) dy = c \left[ \frac{y}{2} + \frac{y^2}{2} \right]_0^1 = c(1) = c.

c = 1

. Step 2 — Marginal $f_X(x)$ :

f_X(x) = \int_0^1 (x+y) \, dy = \left[ xy + \frac{y^2}{2} \right]_0^1 = x + \frac{1}{2}, \quad 0 \leq x \leq 1.

Step 3 — $\mathbb{P}(X+Y > 1)$ : Region:

0 \leq x \leq 1

0 \leq y \leq 1

x+y > 1

\begin{aligned} \mathbb{P}(X+Y > 1) &= \int_0^1 \int_{1-x}^1 (x+y) \, dy \, dx \\ &= \int_0^1 \left[ xy + \frac{y^2}{2} \right]_{y=1-x}^1 dx \\ &= \int_0^1 \left( x + \frac{1}{2} - x(1-x) - \frac{(1-x)^2}{2} \right) dx \\ &= \int_0^1 \left( \frac{1}{2} + x - x^2 - \frac{1}{2}(1 - 2x + x^2) \right) dx \\ &= \int_0^1 \left( x^2 \right) dx = \frac{1}{3}. \end{aligned}

4. Bivariate Normal Distribution

$(X,Y) \sim \mathcal{N}(\mu_X, \mu_Y, \sigma_X^2, \sigma_Y^2, \rho)$ has PDF:

> f_{XY}(x,y) = \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} \exp\left( -\frac{1}{2(1-\rho^2)} \left[ \frac{(x-\mu_X)^2}{\sigma_X^2} - 2\rho\frac{(x-\mu_X)(y-\mu_Y)}{\sigma_X\sigma_Y} + \frac{(y-\mu_Y)^2}{\sigma_Y^2} \right] \right). >

Properties:

Marginals are normal: $X \sim \mathcal{N}(\mu_X, \sigma_X^2)$ , $Y \sim \mathcal{N}(\mu_Y, \sigma_Y^2)$
$\rho$ is the correlation coefficient
If $\rho = 0$ , $X$ and $Y$ are independent (unique to normal)
Conditional distributions are also normal

5. Practice Questions

Q1 (Easy)

f_{XY}(x,y) = 2

for

0 \leq x \leq y \leq 1

. Find

f_X(x)

Full Solution

$f_X(x) = \int_x^1 2 \, dy = 2(1-x)$ , $0 \leq x \leq 1$ .

Q2 (Medium)

For the joint PDF

f_{XY}(x,y) = 6x

for

0 \leq x \leq y \leq 1

, find

\mathbb{P}(Y > 0.5)

Full Solution

> \mathbb{P}(Y > 0.5) = \int_{0.5}^1 \int_0^y 6x \, dx \, dy = \int_{0.5}^1 [3x^2]_0^y \, dy = \int_{0.5}^1 3y^2 \, dy = [y^3]_{0.5}^1 = 1 - 0.125 = 0.875. >

Next topic: Week 5.2: Law of Large Numbers & Central Limit Theorem — Law of Large Numbers and Central Limit Theorem. Join Discord PreviousWeek 4.3: Expectations for Continuous Random VariablesNextWeek 5.2: Law of Large Numbers & Central Limit Theorem

What this note is really saying

Week 5.1: Joint Continuous Distributions

1. Intuition: From Double Sums to Double Integrals

2. Formal Definitions

3. Worked Example

4. Bivariate Normal Distribution

5. Practice Questions

Q1 (Easy)

Q2 (Medium)

Intelligence Hub