Quiz 2

Discrete Random Variables — PMF, CDF, and Properties

2530 words
13 min read
Python Week 1: the first filter for runtime behavior
Visual companion
Python
Type and operator map

Python Week 1: the first filter for runtime behavior

Open galleryRelated surface
View
Revision summary

What this note is really saying

Short form

# Discrete Random Variables — PMF, CDF, and Properties ## 🎯 Learning Objectives After completing this topic, you will be able to: - Define **random variable** and distinguish **discrete** from **continuous** - Construct and interpret a **probability mass function (PMF)** - Construct and interpret a **cumulative dis...

Discrete Random Variables — PMF, CDF, and Properties

🎯 Learning Objectives

After completing this topic, you will be able to:
  • Define random variable and distinguish discrete from continuous
  • Construct and interpret a probability mass function (PMF)
  • Construct and interpret a cumulative distribution function (CDF)
  • Compute expected value E(X)E(X) and variance Var(X)Var(X) for discrete random variables
  • Apply these concepts to real-world problems

📋 Prerequisites

📖 Core Content

16.1 Intuition: From Events to Numbers

So far, we've worked with events ("roll an even number"). Random variables transform outcomes into numbers, making them easier to work with mathematically. Definition: A random variable (RV) is a function that assigns a numerical value to each outcome of a random experiment.
  • Discrete random variable: Takes on a countable number of distinct values (e.g., number of heads in 3 coin flips)
  • Continuous random variable: Takes on any value in an interval (e.g., height of a person)
Everyday analogy: Think of a random variable as a "measurement" on a random process. Before the experiment, we don't know what number we'll get. Afterward, we have a specific value. 🔑 Key Insight: Random variables turn probability into a mathematical system where we can compute averages, variances, and make predictions.

16.2 Examples of Random Variables

ExperimentRandom VariableTypePossible Values
Flip 3 coinsX = # of headsDiscrete{0, 1, 2, 3}
Roll a dieX = number shownDiscrete{1, 2, 3, 4, 5, 6}
Measure heightX = height (cm)Continuous(0, ∞)
Count customersX = # of customers/hourDiscrete{0, 1, 2, ...}

16.3 Probability Mass Function (PMF)

16.3.1 Definition

The probability mass function p(x)p(x) gives the probability that a discrete random variable X equals a specific value x:
p(x)=P(X=x)p(x) = P(X = x)
Properties:
  1. 0≤p(x)≤10 \leq p(x) \leq 1 for all x
  2. ∑all xp(x)=1\sum_{\text{all }x} p(x) = 1 (probabilities sum to 1)

16.3.2 Example: Sum of Two Dice

Let X = sum when rolling two fair dice.
x23456789101112
p(x)1/362/363/364/365/366/365/364/363/362/361/36
Check: sum = 1+2+3+4+5+6+5+4+3+2+1=361+2+3+4+5+6+5+4+3+2+1 = 36 ✓

16.4 Cumulative Distribution Function (CDF)

16.4.1 Definition

The cumulative distribution function F(x)F(x) gives the probability that X is less than or equal to a value:
F(x)=P(X≤x)=∑t≤xp(t)F(x) = P(X \leq x) = \sum_{t \leq x} p(t)
Properties:
  1. F(x)F(x) is non-decreasing (as x increases, F(x)F(x) never decreases)
  2. lim⁡x→−∞F(x)=0\lim_{x \to -\infty} F(x) = 0
  3. lim⁡x→+∞F(x)=1\lim_{x \to +\infty} F(x) = 1
  4. P(a<X≤b)=F(b)−F(a)P(a < X \leq b) = F(b) - F(a)

16.4.2 Example: Sum of Two Dice

x23456789101112
F(x)1/363/366/3610/3615/3621/3626/3630/3633/3635/361
F(5)=P(X≤5)=1/36+2/36+3/36+4/36=10/36F(5) = P(X \leq 5) = 1/36 + 2/36 + 3/36 + 4/36 = 10/36

16.5 Expected Value of a Discrete RV

16.5.1 Intuition

The expected value E(X)E(X) is the long-run average of the random variable — what you'd expect to get on average over many repetitions.

16.5.2 Definition

E(X)=μX=∑all xx⋅P(X=x)=∑x⋅p(x)E(X) = \mu_X = \sum_{\text{all }x} x \cdot P(X = x) = \sum x \cdot p(x)

16.5.3 Example

Roll a fair die: X=number shownX = \text{number shown}
E(X)=1(16)+2(16)+3(16)+4(16)+5(16)+6(16)=3.5E(X) = 1(\frac{1}{6}) + 2(\frac{1}{6}) + 3(\frac{1}{6}) + 4(\frac{1}{6}) + 5(\frac{1}{6}) + 6(\frac{1}{6}) = 3.5
Interpretation: If you roll a fair die many times, the average will be about 3.5.

16.6 Variance of a Discrete RV

16.6.1 Definition

Var(X)=σX2=E[(X−μ)2]=∑(x−μ)2⋅p(x)Var(X) = \sigma^2_X = E[(X - \mu)^2] = \sum (x - \mu)^2 \cdot p(x)
Computational formula:
Var(X)=E(X2)−[E(X)]2Var(X) = E(X^2) - [E(X)]^2
Where E(X2)=∑x2⋅p(x)E(X^2) = \sum x^2 \cdot p(x)

16.6.2 Example: Roll a Fair Die

Step 1: μ=3.5\mu = 3.5 Step 2: E(X2)=12(16)+22(16)+...+62(16)E(X^2) = 1^2(\frac{1}{6}) + 2^2(\frac{1}{6}) + ... + 6^2(\frac{1}{6}) =(1+4+9+16+25+36)/6=91/6≈15.167= (1+4+9+16+25+36)/6 = 91/6 \approx 15.167 Step 3: Var(X)=15.167−3.52=15.167−12.25=2.917Var(X) = 15.167 - 3.5^2 = 15.167 - 12.25 = 2.917 Step 4: σ=2.917≈1.708\sigma = \sqrt{2.917} \approx 1.708

16.7 Properties of Expected Value and Variance

PropertyFormula
Expected value of constantE(c)=cE(c) = c
Linearity of expectationE(aX+b)=aE(X)+bE(aX + b) = aE(X) + b
Variance of constantVar(c)=0Var(c) = 0
Variance linear transformationVar(aX+b)=a2Var(X)Var(aX + b) = a^2 Var(X)
Standard deviationσX=Var(X)\sigma_X = \sqrt{Var(X)}

16.8 Worked Examples

Example 1: Constructing a PMF (Easy)

Scenario: A bag has 3 red and 5 blue chips. Draw 2 without replacement. Let X = number of red chips drawn. Find the PMF of X. Solution: Possible values: X = 0, 1, 2 P(X=0)=P(both blue)=C(5,2)C(8,2)=1028=514P(X = 0) = P(\text{both blue}) = \frac{C(5,2)}{C(8,2)} = \frac{10}{28} = \frac{5}{14} P(X=2)=P(both red)=C(3,2)C(8,2)=328P(X = 2) = P(\text{both red}) = \frac{C(3,2)}{C(8,2)} = \frac{3}{28} P(X=1)=1−P(X=0)−P(X=2)=1−514−328=28−10−328=1528P(X = 1) = 1 - P(X=0) - P(X=2) = 1 - \frac{5}{14} - \frac{3}{28} = \frac{28 - 10 - 3}{28} = \frac{15}{28} PMF:
p(0)=514,p(1)=1528,p(2)=328p(0) = \frac{5}{14},\quad p(1) = \frac{15}{28},\quad p(2) = \frac{3}{28}
Check: 1028+1528+328=2828=1\frac{10}{28} + \frac{15}{28} + \frac{3}{28} = \frac{28}{28} = 1 ✓

Example 2: Expected Value in a Game (Medium)

Scenario: A game costs ₹10 to play. You roll a die:
  • If even: you win ₹5
  • If odd: you win nothing
  • If the number is 6: you win ₹20 (including the even win) Find the expected net gain. Solution: Let X = net gain (winnings - cost).
OutcomeNet Gain XP(X = x)
1 (odd)0 - 10 = -101/6
3 (odd)0 - 10 = -101/6
5 (odd)0 - 10 = -101/6
2,4 (even, not 6)5 - 10 = -52/6
6 (even, special)20 - 10 = 101/6
E(X)=(−10)(3/6)+(−5)(2/6)+(10)(1/6)=−30/6−10/6+10/6=−30/6=−5E(X) = (-10)(3/6) + (-5)(2/6) + (10)(1/6) = -30/6 - 10/6 + 10/6 = -30/6 = -5 So the expected net gain is -₹5 per game. Not a good game to play!

Example 3: PMF and CDF (Harder)

Scenario: Let X = number of heads in 3 coin flips. a) Find the PMF b) Find the CDF c) Find P(1≤X≤2)P(1 \leq X \leq 2) Solution: a) PMF: X can be 0, 1, 2, 3. P(X=0)=(1/2)3=1/8P(X=0) = (1/2)^3 = 1/8 P(X=3)=(1/2)3=1/8P(X=3) = (1/2)^3 = 1/8 P(X=1)=3×(1/2)3=3/8P(X=1) = 3 \times (1/2)^3 = 3/8 (THH, HTH, HHT) P(X=2)=3×(1/2)3=3/8P(X=2) = 3 \times (1/2)^3 = 3/8
x0123
p(x)1/83/83/81/8
b) CDF: F(0)=1/8F(0) = 1/8 F(1)=1/8+3/8=4/8=1/2F(1) = 1/8 + 3/8 = 4/8 = 1/2 F(2)=1/8+3/8+3/8=7/8F(2) = 1/8 + 3/8 + 3/8 = 7/8 F(3)=1F(3) = 1 c) P(1≤X≤2)P(1 \leq X \leq 2): Using PMF: p(1)+p(2)=3/8+3/8=3/4p(1) + p(2) = 3/8 + 3/8 = 3/4 Using CDF: F(2)−F(0)=7/8−1/8=6/8=3/4F(2) - F(0) = 7/8 - 1/8 = 6/8 = 3/4 ✓

16.9 Edge Cases & Gotchas

Expected Value May Not Be a Possible Value

E(die roll)=3.5E(\text{die roll}) = 3.5, but you can never roll 3.5. The expected value is a long-run average, not the "most likely" value.

Variance Is Always Non-Negative

Var(X)≥0Var(X) \geq 0 always. Var(X)=0Var(X) = 0 only if X is constant (no randomness).

PMF vs PDF

Don't confuse PMF (for discrete) with PDF (for continuous). A PMF gives P(X=x)P(X = x), which can be >0. A PDF gives density, and P(X=x)=0P(X = x) = 0 for continuous variables.

16.10 Why This Matters

Random variables are the foundation of all statistical modeling:
  • Week 10: Formal properties of expectation and variance
  • Weeks 11-12: Specific distributions (Binomial, Poisson, Normal, etc.)
  • BSMA1004 (Stats 2): Sampling distributions are random variables
  • BSCS2004 (ML Foundations): Loss functions and risk are expectations

📐 Key Formulas / Concepts

ConceptFormulaNotes
PMFp(x)=P(X=x)p(x) = P(X = x)∑p(x)=1\sum p(x) = 1
CDFF(x)=P(X≤x)F(x) = P(X \leq x)Non-decreasing, ranges [0,1]
Expected ValueE(X)=∑x⋅p(x)E(X) = \sum x \cdot p(x)Long-run average
Expected Value of FunctionE(g(X))=∑g(x)⋅p(x)E(g(X)) = \sum g(x) \cdot p(x)For any function g
VarianceVar(X)=∑(x−μ)2p(x)Var(X) = \sum (x-\mu)^2 p(x)Expected squared deviation
Computational VarianceVar(X)=E(X2)−[E(X)]2Var(X) = E(X^2) - [E(X)]^2Easier calculation
LinearityE(aX+b)=aE(X)+bE(aX+b) = aE(X)+bExpectation is linear
Variance scalingVar(aX+b)=a2Var(X)Var(aX+b) = a^2 Var(X)Variance scales by a2a^2

⚠️ Common Pitfalls

Pitfall 1: Confusing E(X2)E(X^2) with [E(X)]2[E(X)]^2

The mistake: Thinking E(X2)=[E(X)]2E(X^2) = [E(X)]^2. Why it happens: Notation is subtle. Example: X = {1, 2, 3} equally likely. E(X)=2E(X) = 2, [E(X)]2=4[E(X)]^2 = 4 E(X2)=(12+22+32)/3=14/3≈4.67E(X^2) = (1^2+2^2+3^2)/3 = 14/3 \approx 4.67 They're different! Their difference IS the variance.

Pitfall 2: Thinking Expected Value Is the Most Likely Value

The mistake: Saying "the expected number of heads in 3 flips is 1.5" and expecting to see 1.5 heads. Correction: Expected value is a long-run average, not a prediction for a single trial.

Pitfall 3: Forgetting That PMF Probabilities Must Sum to 1

The mistake: Writing a PMF where probabilities don't sum to 1. How to check: Always verify ∑p(x)=1\sum p(x) = 1 as a sanity check.

📝 Practice Questions

Q1: PMF Construction
</strong>
A bag has 4 red, 3 green chips. Draw 2 without replacement. X = # of red chips. Find PMF.
<details> <strong>Solution</strong>
P(X=0)=C(3,2)C(7,2)=321=17P(X=0) = \frac{C(3,2)}{C(7,2)} = \frac{3}{21} = \frac{1}{7}
P(X=2)=C(4,2)C(7,2)=621=27P(X=2) = \frac{C(4,2)}{C(7,2)} = \frac{6}{21} = \frac{2}{7}
P(X=1)=1−17−27=47P(X=1) = 1 - \frac{1}{7} - \frac{2}{7} = \frac{4}{7}
p(0)=17, p(1)=47, p(2)=27\boxed{p(0)=\frac{1}{7},\ p(1)=\frac{4}{7},\ p(2)=\frac{2}{7}}
</details> > **Q2: Expected Value** > > </strong> > > X has PMF: $p(1)=0.2$, $p(2)=0.3$, $p(3)=0.4$, $p(4)=0.1$. Find $E(X)$. > > <details> <strong>Solution</strong> > > $E(X) = 1(0.2) + 2(0.3) + 3(0.4) + 4(0.1) = 0.2 + 0.6 + 1.2 + 0.4 = 2.4$ > > $\boxed{E(X) = 2.4}$ </details> > **Q3: Variance** > > </strong> > > Using Q2's distribution, find $Var(X)$. > > <details> <strong>Solution</strong> > > $E(X^2) = 1^2(0.2) + 2^2(0.3) + 3^2(0.4) + 4^2(0.1) = 0.2 + 1.2 + 3.6 + 1.6 = 6.6$ > > $Var(X) = 6.6 - 2.4^2 = 6.6 - 5.76 = 0.84$ > > $\boxed{Var(X) = 0.84}$ </details> > **Q4: CDF** > > </strong> > > X has PMF: $p(0)=0.1$, $p(1)=0.3$, $p(2)=0.2$, $p(3)=0.1$, $p(4)=0.3$. > > Find $F(2)$ and $P(1 \leq X \leq 3)$. > > <details> <strong>Solution</strong> > > $F(2) = p(0) + p(1) + p(2) = 0.1 + 0.3 + 0.2 = 0.6$ > > $P(1 \leq X \leq 3) = p(1) + p(2) + p(3) = 0.3 + 0.2 + 0.1 = 0.6$ > > Or: $F(3) - F(0) = (0.6+0.1) - 0.1 = 0.6$ ✓ > > $\boxed{F(2)=0.6,\ P(1\leq X\leq3)=0.6}$ </details> > **Q5: Linear Transformation** > > </strong> > > If $E(X) = 5$ and $Var(X) = 4$, find $E(3X + 2)$ and $Var(3X + 2)$. > > <details> <strong>Solution</strong> > > $E(3X + 2) = 3E(X) + 2 = 3(5) + 2 = 17$ > > $Var(3X + 2) = 3^2 Var(X) = 9 \times 4 = 36$ > > $\boxed{E = 17,\ Var = 36}$ </details> > **Q6: Application — Insurance** > > </strong> > > An insurance policy costs ₹500. With probability 0.001, the policy pays ₹200,000. With probability 0.01, it pays ₹50,000. Otherwise, it pays nothing. Find the expected value to the insurance company (their profit). > > <details> <strong>Solution</strong> > > Let X = company's profit. > >
EventProfit (X)P(X)
No claim5001 - 0.001 - 0.01 = 0.989
Small claim500 - 50,000 = -49,5000.01
Large claim500 - 200,000 = -199,5000.001
E(X)=500(0.989)+(−49,500)(0.01)+(−199,500)(0.001)E(X) = 500(0.989) + (-49,500)(0.01) + (-199,500)(0.001) =494.5−495−199.5=−200= 494.5 - 495 - 199.5 = -200
The company expects to lose ₹200 per policy! They should raise the premium.
E(X)=−200\boxed{E(X) = -200}
</details> > **Q7: Fair Game** > > </strong> > > A game costs c rupees to play. You draw a card: if it's a heart, you win ₹20; if it's an ace (but not hearts), you win ₹10; otherwise, nothing. Find c so the game is fair ($E(\text{net gain}) = 0$). > > <details> <strong>Solution</strong> > > Deck: 52 cards. > > - Heart: 13 cards (incl. Ace of hearts). Win ₹20. > - Non-heart ace: 3 cards (Ace of spades, clubs, diamonds). Win ₹10. > - Other: 52 - 13 - 3 = 36 cards. Win ₹0. > > $E(\text{winnings}) = 20(\frac{13}{52}) + 10(\frac{3}{52}) + 0(\frac{36}{52}) = \frac{260}{52} + \frac{30}{52} = \frac{290}{52} \approx 5.577$ > > To make net gain = 0: $c = E(\text{winnings}) \approx 5.58$ > > So the game should cost about ₹5.58 to play. > > $\boxed{c \approx 5.58}$ </details> > **Q8: PMF Properties** > > Which of the following could be a valid PMF? > > a) $p(1)=0.3$, $p(2)=0.5$, $p(3)=0.2$ b) $p(1)=0.4$, $p(2)=0.4$, $p(3)=0.4$ c) $p(1)=0.5$, $p(2)=0.3$, $p(3)=0.3$ > > <details> <strong>Solution</strong> > > a) Sum = 0.3 + 0.5 + 0.2 = 1.0, each between 0 and 1 → **Valid** ✓ > > b) Sum = 0.4 + 0.4 + 0.4 = 1.2 ≠ 1 → **Invalid** > > c) Sum = 0.5 + 0.3 + 0.3 = 1.1 ≠ 1 → **Invalid** > > Only (a) is a valid PMF. </details> > **Q9: Law of the Unconscious Statistician** > > X has PMF: $p(1)=0.2$, $p(2)=0.5$, $p(3)=0.3$. Find $E(X^2 + 2X)$. > > <details> <strong>Solution</strong> > > **Method 1:** $E(X^2 + 2X) = E(X^2) + 2E(X)$ (linearity) > > $E(X) = 1(0.2) + 2(0.5) + 3(0.3) = 0.2 + 1.0 + 0.9 = 2.1$ $E(X^2) = 1(0.2) + 4(0.5) + 9(0.3) = 0.2 + 2.0 + 2.7 = 4.9$ > > $E(X^2 + 2X) = 4.9 + 2(2.1) = 4.9 + 4.2 = 9.1$ > > **Method 2:** Direct computation $E(X^2 + 2X) = \sum (x^2 + 2x)p(x)$ $= (1+2)(0.2) + (4+4)(0.5) + (9+6)(0.3)$ $= 3(0.2) + 8(0.5) + 15(0.3) = 0.6 + 4.0 + 4.5 = 9.1$ ✓ > > $\boxed{9.1}$ </details> > **Q10: Expected Value in Decision Making** > > A store must decide how many cakes to bake. Each cake costs ₹50 to make and sells for ₹150. Unsold cakes are donated (no salvage value). Daily demand has PMF: > >
Demand0123
p(d)0.10.30.40.2
How many cakes should they bake to maximize expected profit?
<details> <strong>Solution</strong>
Bake 1 cake:
  • If demand ≥ 1 (prob 0.9): profit = 150 - 50 = 100
  • If demand = 0 (prob 0.1): profit = -50
  • E=100(0.9)+(−50)(0.1)=90−5=85E = 100(0.9) + (-50)(0.1) = 90 - 5 = 85
Bake 2 cakes:
  • Demand 0: profit = -100 (0.1)
  • Demand 1: profit = 150 - 100 = 50 (0.3)
  • Demand ≥ 2: profit = 300 - 100 = 200 (0.6)
  • E=−100(0.1)+50(0.3)+200(0.6)=−10+15+120=125E = -100(0.1) + 50(0.3) + 200(0.6) = -10 + 15 + 120 = 125
Bake 3 cakes:
  • Demand 0: profit = -150 (0.1)
  • Demand 1: profit = 150 - 150 = 0 (0.3)
  • Demand 2: profit = 300 - 150 = 150 (0.4)
  • Demand 3: profit = 450 - 150 = 300 (0.2)
  • E=−150(0.1)+0(0.3)+150(0.4)+300(0.2)=−15+0+60+60=105E = -150(0.1) + 0(0.3) + 150(0.4) + 300(0.2) = -15 + 0 + 60 + 60 = 105
Best: Bake 2 cakes for expected profit of ₹125.
Bake 2 cakes, expected profit ₹125\boxed{\text{Bake 2 cakes, expected profit } ₹125}
</details> * * * ## 🔗 Cross-References - **Next topic:** [Continuous Random Variables](/notes/01-foundation-bsma1002-stats-1-week09-16-continuous-random-variables) — the continuous analogue - **Week 10:** Expectation and Variance (deeper dive) - **Week 11 (Distributions):** Bernoulli, Binomial, Poisson, Geometric - **Week 12 (Distributions):** Uniform, Exponential, Normal - **BSMA1004 (Stats 2):** Sampling distributions, central limit theorem [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**Bayes' Theorem**](/notes/01-foundation-bsma1002-stats-1-week08-14-bayes-theorem)[Next**Continuous Random Variables**](/notes/01-foundation-bsma1002-stats-1-week09-16-continuous-random-variables)
Document outline

Keep your place and jump directly to a heading.

Table of Contents
System Normal // Awaiting Context

Intelligence Hub

Navigate the knowledge graph to generate context. The Hub adapts dynamically to surface backlinks, related notes, and metadata insights.

Open library →Launch Assessment Engine