Discrete Random Variables — PMF, CDF, Expectation & Variance (Layered)

2099 words

10 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Discrete Random Variables — PMF, CDF, Expectation & Variance > **Layered Deep-Dive:** This chapter reconstructs Discrete Random Variables from first principles. We map abstract sample spaces $\Omega$ onto real numbers $\mathbb{R}$, master the Probability Mass Function (PMF) and the right-continuous step nature of...

Discrete Random Variables — PMF, CDF, Expectation & Variance

Layered Deep-Dive: This chapter reconstructs Discrete Random Variables from first principles. We map abstract sample spaces $\Omega$ onto real numbers $\mathbb{R}$ , master the Probability Mass Function (PMF) and the right-continuous step nature of the Cumulative Distribution Function (CDF), derive interval probabilities without boundary errors, and calculate Expectation $E[X]$ and Variance $Var(X)$ for Quiz 3.

1. Orientation & Intuition: From Events to Numbers

In elementary probability, we describe random experiments using descriptive sample spaces:

Tossing three coins yields outcomes like $\Omega = \{HHH, HHT, HTH, HTT, THH, THT, TTH, TTT\}$ .
Describing events like "at least two heads" requires grouping text tuples: $E = \{HHH, HHT, HTH, THH\}$ .

While intuitive, working directly with sets of strings becomes unwieldy when modeling complex systems like stock price shifts, defect counts, or queue wait times.

We need a mathematical mechanism to translate qualitative outcomes into numerical values:

\text{Outcome } \omega \in \Omega \xrightarrow{\quad X \quad} \text{Number } x \in \mathbb{R}

This mapping function is a Random Variable (RV). For three coin tosses, let

X

denote the "number of heads":

$X(TTT) = 0$
$X(HTT) = X(THT) = X(TTH) = 1$
$X(HHT) = X(HTH) = X(THH) = 2$
$X(HHH) = 3$

Now, instead of tracking 8 outcome strings, we can perform arithmetic, calculate averages, compute spread, and model distributions on the set of numbers

\{0, 1, 2, 3\}

2. Terminology, Notation & Mathematical Definitions

Concept / Symbol	Mathematical Definition	Key Meaning & Properties
Random Variable $X$	$X: \Omega \to \mathbb{R}$	A real-valued function defined over the sample space $\Omega$ .
Support $S_X$	$\{x \in \mathbb{R} : P(X = x) > 0\}$	The set of all possible numerical values $X$ can take with non-zero probability.
Discrete RV	$S_X$ is countable	Takes either a finite set of values or a countably infinite set $\{x_1, x_2, \dots\}$ .
PMF $p_X(x)$	$P(X = x)$	Probability Mass Function: probability concentrated at exact point $x$ .
CDF $F_X(x)$	$P(X \le x) = \sum_{t \le x} p_X(t)$	Cumulative Distribution Function: accumulated probability up to $x$ .
Expectation $E[X]$	$\sum_{x \in S_X} x \cdot p_X(x)$	The probability-weighted average (mean / center of mass) of $X$ .
Variance $Var(X)$	$E[(X - \mu)^2] = E[X^2] - (E[X])^2$	The expected squared deviation from the mean; measures dispersion.
Std Deviation $\sigma_X$	$\sqrt{Var(X)}$	Spread expressed in the original units of measurement.

3. The Probability Mass Function (PMF)

The Probability Mass Function (PMF) of a discrete random variable

X

assigns a probability to each possible discrete value.

3.1 Axiomatic Requirements of a Valid PMF

Any function

p(x)

is a valid PMF if and only if it satisfies two conditions:

Non-negativity: $p(x) \ge 0 \quad \text{for all } x \in S_X$
Total Probability Normalization: $\sum_{x \in S_X} p(x) = 1$

3.2 Finding Unknown Normalization Constants

A recurring IITM exam question gives a PMF with an unknown constant

c

(e.g.,

p(x) = c x^2

for

x \in \{1, 2, 3\}

). To find

c

, enforce the normalization axiom:

\sum_{x=1}^3 c x^2 = c(1^2 + 2^2 + 3^2) = c(1 + 4 + 9) = 14c = 1 \implies c = \frac{1}{14}

4. The Cumulative Distribution Function (CDF)

The Cumulative Distribution Function (CDF), denoted by

F(x)

, describes the cumulative probability accumulated up to and including

x

F(x) = P(X \le x) = \sum_{t \le x} p(t)

4.1 Fundamental Properties of $F(x)$

Bounded Range: $0 \le F(x) \le 1$ for all $x \in \mathbb{R}$ .
Asymptotic Limits: $\lim_{x \to -\infty} F(x) = 0 \quad \text{and} \quad \lim_{x \to +\infty} F(x) = 1$
Monotonically Non-Decreasing: If $x_1 < x_2$ , then $F(x_1) \le F(x_2)$ .
Right-Continuous: For any point $a$ : $\lim_{x \to a^+} F(x) = F(a)$
Step Discontinuity Theorem: For a discrete RV, $F(x)$ is a piecewise-constant step function. The height of the jump at any value $k$ equals the probability mass at that value: $P(X = k) = F(k) - \lim_{x \to k^-} F(x) = F(k) - F(k^-)$


F(x) CDF Step Graph:
1.0 |                _________
0.7 |        _______|
0.3 |  _____|
0.0 |__|
    +---+-------+----+--------
        1       2    3        x
Jump at x=2 is: F(2) - F(2^-) = 0.7 - 0.3 = 0.4 = P(X=2)

5. Converting Between Intervals and CDF Values

Because discrete random variables place probability mass in concentrated point spikes, the presence or absence of the equality sign strictly alters the probability!

Desired Probability	Discrete CDF Formula	Explanation
$P(X \le b)$	$F(b)$	By direct definition of CDF.
$P(X < b)$	$F(b^-) = \lim_{x \to b^-} F(x)$	Excludes the point mass at $b$ .
$P(X > a)$	$1 - P(X \le a) = 1 - F(a)$	Complement rule.
$P(X \ge a)$	$1 - P(X < a) = 1 - F(a^-)$	Complement of strictly less than $a$ .
$P(a < X \le b)$	$F(b) - F(a)$	Standard cumulative interval formula.
$P(a \le X \le b)$	$F(b) - F(a^-) = F(b) - F(a) + P(X = a)$	Includes both boundaries $a$ and $b$ .
$P(a < X < b)$	$F(b^-) - F(a)$	Excludes both boundaries $a$ and $b$ .

In continuous probability,

P(X < b) = P(X \le b)

because point probabilities are zero. In discrete probability, $P(X < b) \neq P(X \le b)$ ! For integer-valued RVs,

P(X < 5) = P(X \le 4) = F(4)

6. Expectation, Variance & Moments

6.1 Expected Value (Mean $\mu$ )

The expected value is the probability-weighted average of all values

X

can take:

E[X] = \mu = \sum_{x \in S_X} x \cdot p(x)

Properties of Expectation:

Linearity: For any constants $a, b \in \mathbb{R}$ : $E[aX + b] = aE[X] + b$
Expectation of a Function $g(X)$ (LOTUS): $E[g(X)] = \sum_{x \in S_X} g(x) \cdot p(x)$

6.2 Variance and Standard Deviation

Variance measures the expected squared distance between the random variable and its mean:

Var(X) = \sigma^2 = E[(X - \mu)^2] = \sum_{x \in S_X} (x - \mu)^2 \cdot p(x)

Computational Shortcut Formula:

Var(X) = E[X^2] - (E[X])^2

Where

E[X^2] = \sum x^2 \cdot p(x)

is the second raw moment.

Properties of Variance:

Non-negativity: $Var(X) \ge 0$ .
Constants have zero variance: $Var(c) = 0$ .
Adding a constant does not change spread: $Var(X + b) = Var(X)$ .
Multiplying by a constant scales variance by the square: $Var(aX + b) = a^2 Var(X)$
Standard Deviation: $\sigma(aX + b) = |a| \sigma(X)$

7. Worked Exam Walkthroughs

Walkthrough 1: PMF Normalization, Expectation, and Variance

Problem: A discrete random variable

X

has the PMF:

p(x) = \begin{cases} k(2x - 1) & \text{for } x \in \{1, 2, 3, 4\} \\ 0 & \text{otherwise} \end{cases}

Find:

The value of the constant $k$ .
The expected value $E[X]$ .
The variance $Var(X)$ .

Solution:

Part 1: Find $k$ Evaluate

p(x)

for each

x \in \{1, 2, 3, 4\}

$p(1) = k(2(1) - 1) = 1k$
$p(2) = k(2(2) - 1) = 3k$
$p(3) = k(2(3) - 1) = 5k$
$p(4) = k(2(4) - 1) = 7k$

Sum of probabilities:

\sum_{x=1}^4 p(x) = k(1 + 3 + 5 + 7) = 16k = 1 \implies k = \frac{1}{16}

Thus, the exact probabilities are:

p(1) = \frac{1}{16}, \quad p(2) = \frac{3}{16}, \quad p(3) = \frac{5}{16}, \quad p(4) = \frac{7}{16}

Part 2: Find $E[X]$

E[X] = \sum x \cdot p(x) = 1\left(\frac{1}{16}\right) + 2\left(\frac{3}{16}\right) + 3\left(\frac{5}{16}\right) + 4\left(\frac{7}{16}\right)

E[X] = \frac{1 + 6 + 15 + 28}{16} = \frac{50}{16} = \frac{25}{8} = 3.125

Part 3: Find $Var(X)$ First, compute

E[X^2]

E[X^2] = \sum x^2 \cdot p(x) = 1^2\left(\frac{1}{16}\right) + 2^2\left(\frac{3}{16}\right) + 3^2\left(\frac{5}{16}\right) + 4^2\left(\frac{7}{16}\right)

E[X^2] = \frac{1(1) + 4(3) + 9(5) + 16(7)}{16} = \frac{1 + 12 + 45 + 112}{16} = \frac{170}{16} = \frac{85}{8} = 10.625

Now apply the variance formula:

Var(X) = E[X^2] - (E[X])^2 = \frac{85}{8} - \left(\frac{25}{8}\right)^2 = \frac{85}{8} - \frac{625}{64}

Var(X) = \frac{680 - 625}{64} = \frac{55}{64} \approx 0.859

\boxed{k = \frac{1}{16}, \quad E[X] = \frac{25}{8} = 3.125, \quad Var(X) = \frac{55}{64} \approx 0.859}

Walkthrough 2: Deconstructing a Step CDF

Problem: The CDF of an integer-valued random variable

X

is given by:

F(x) = \begin{cases} 0 & x < 0 \\ 0.15 & 0 \le x < 2 \\ 0.55 & 2 \le x < 5 \\ 0.85 & 5 \le x < 8 \\ 1.00 & x \ge 8 \end{cases}

Compute:

$P(X = 2)$
$P(X > 2)$
$P(2 \le X < 8)$

Solution:

$P(X = 2)$ : By the jump discontinuity formula: $P(X = 2) = F(2) - F(2^-) = 0.55 - 0.15 = 0.40$
$P(X > 2)$ : By the complement rule: $P(X > 2) = 1 - P(X \le 2) = 1 - F(2) = 1 - 0.55 = 0.45$
$P(2 \le X < 8)$ : Break down the discrete points in the range: $\{2, 5\}$ : $P(X = 2) + P(X = 5) = 0.40 + (F(5) - F(5^-)) = 0.40 + (0.85 - 0.55) = 0.40 + 0.30 = 0.70$ Alternatively, using CDF limits: $F(8^-) - F(2^-) = 0.85 - 0.15 = 0.70$

\boxed{P(X = 2) = 0.40, \quad P(X > 2) = 0.45, \quad P(2 \le X < 8) = 0.70}

8. Common Traps & Pitfalls in IITM Stats 1 Exams

Confusing $P(X < k)$ with $P(X \le k)$ : In discrete RVs taking integers, $P(X < 4) = P(X \le 3) = F(3)$ , whereas $P(X \le 4) = F(4)$ . If $P(X = 4) > 0$ , these two values are strictly different.
Variance Scaling Sign Error: Remember that $Var(-3X + 5) = (-3)^2 Var(X) = 9 Var(X)$ . A negative coefficient inside variance is squared and becomes positive.
Subtracting Probabilities in the Wrong Order: Always remember that $P(a < X \le b) = F(b) - F(a)$ . If you write $F(a) - F(b)$ , you get a negative number, which violates probability axioms.

9. Practice Quiz Questions

Question 1 (MCQ)

Var(X) = 4

, what is

Var(5 - 2X)

(A) $-8$
(B) $8$
(C) $16$
(D) $21$

Answer: (C) Detailed Explanation: Using the variance property

Var(aX + b) = a^2 Var(X)

: Here

a = -2

and

b = 5

Var(5 - 2X) = (-2)^2 Var(X) = 4 \times 4 = 16

Question 2 (MSQ)

Which of the following functions

F(x)

represent valid Cumulative Distribution Functions for some random variable

X

(A) $F(x) = 0$ for $x < 0$ , $F(x) = 0.5$ for $0 \le x < 1$ , $F(x) = 1$ for $x \ge 1$ .
(B) $F(x) = 1 - e^{-x}$ for $x \in \mathbb{R}$ .
(C) $F(x) = 0$ for $x < 0$ , $F(x) = x/2$ for $0 \le x < 2$ , $F(x) = 1$ for $x \ge 2$ .
(D) $F(x) = 0.2$ for $x < 1$ , $F(x) = 0.8$ for $1 \le x < 2$ , $F(x) = 1.2$ for $x \ge 2$ .

Explanation:

(A) Valid discrete CDF: ranges from 0 to 1, right-continuous, non-decreasing.
(B) Invalid: for $x < 0$ , $1 - e^{-x}$ is negative (e.g. at $x=-1$ , $1-e \approx -1.718 < 0$ ).
(C) Valid continuous CDF: non-decreasing, limits are 0 and 1.
(D) Invalid: probabilities cannot exceed 1 ( $F(x)$ hits 1.2).

10. Active Recall Flashcards

How do you find the probability mass $P(X = c)$ from a CDF $F(x)$ ? Answer: Measure the height of the jump at $c$ : $P(X = c) = F(c) - \lim_{x \to c^-} F(x)$ .
What are the two conditions that every PMF $p(x)$ must satisfy? Answer: (1) $p(x) \ge 0$ for all $x$ , and (2) $\sum_{x \in S_X} p(x) = 1$ .
What is the effect of a linear transformation $Y = aX + b$ on the expectation and variance of $X$ ? Answer: $E[Y] = aE[X] + b$ , and $Var(Y) = a^2 Var(X)$ .
For an integer-valued random variable, what is $P(X \ge 3)$ in terms of the CDF $F(x)$ ? Answer: $P(X \ge 3) = 1 - P(X < 3) = 1 - P(X \le 2) = 1 - F(2)$ .