Quiz 2

Week 3.4: Markov & Chebyshev Inequalities

693 words
3 min read
Python Week 1: the first filter for runtime behavior
Visual companion
Python
Type and operator map

Python Week 1: the first filter for runtime behavior

Open galleryRelated surface
View
Revision summary

What this note is really saying

Short form

# Week 3.4: Markov & Chebyshev Inequalities > **Prerequisites:** Expected Value ([Week 3.1: Expected Value](/notes/01-foundation-bsma1004-stats-2-week03-08-expectation)), Variance ([Week 3.2: Variance & Standard Deviation](/notes/01-foundation-bsma1004-stats-2-week03-09-variance)) > **Cross-links:** BSMA3012 (Linear...

Week 3.4: Markov & Chebyshev Inequalities

Prerequisites: Expected Value (Week 3.1: Expected Value), Variance (Week 3.2: Variance & Standard Deviation) Cross-links: BSMA3012 (Linear Stat Models) — concentration bounds Core question: What can we say about tail probabilities using only the mean (and variance)?

1. Intuition: Bounds Without Full Distribution

Often we don't know the full distribution of XX, but we know its mean μ\mu (and possibly variance σ2\sigma^2). Can we bound probabilities like P(X≥c)\mathbb{P}(X \geq c) without knowing the whole PMF/PDF? Yes! Two simple inequalities provide universal bounds:
  • Markov: Uses only the mean (for non-negative RVs)
  • Chebyshev: Uses mean and variance (for any RV) These are worst-case bounds — they hold for any distribution with the given moments. The actual probability is often much smaller.

2. Markov's Inequality

Theorem (Markov's Inequality) Let XX be a non-negative random variable (X≥0X \geq 0) with finite mean μ=E[X]\mu = \mathbb{E}[X]. For any c>0c > 0:
>P(X≥c)≤μc.>> \mathbb{P}(X \geq c) \leq \frac{\mu}{c}. >
Proof:
μ=E[X]=∑tt⋅fX(t)≥∑t≥ct⋅fX(t)(dropping non-negative terms with t<c)≥∑t≥cc⋅fX(t)=c⋅P(X≥c).\begin{aligned} \mu &= \mathbb{E}[X] = \sum_{t} t \cdot f_X(t) \\ &\geq \sum_{t \geq c} t \cdot f_X(t) \quad \text{(dropping non-negative terms with $t<c$)} \\ &\geq \sum_{t \geq c} c \cdot f_X(t) = c \cdot \mathbb{P}(X \geq c). \end{aligned}

Example

If the average exam score is 50/100, what fraction of students scored ≥80\geq 80?
P(X≥80)≤5080=0.625.\mathbb{P}(X \geq 80) \leq \frac{50}{80} = 0.625.
So at most 62.5% could have scored 80 or above. (In reality, likely much less.)

3. Chebyshev's Inequality

Theorem (Chebyshev's Inequality) Let XX have mean μ\mu and finite variance σ2\sigma^2. For any k>0k > 0:
>P(∣X−μ∣≥kσ)≤1k2.>> \mathbb{P}(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}. >
Equivalent forms:
  • P(∣X−μ∣≥c)≤σ2c2\mathbb{P}(|X - \mu| \geq c) \leq \frac{\sigma^2}{c^2}
  • P(μ−kσ<X<μ+kσ)≥1−1k2\mathbb{P}(\mu - k\sigma < X < \mu + k\sigma) \geq 1 - \frac{1}{k^2} Proof: Apply Markov to (X−μ)2(X-\mu)^2:
P((X−μ)2≥k2σ2)≤E[(X−μ)2]k2σ2=σ2k2σ2=1k2.\mathbb{P}((X-\mu)^2 \geq k^2\sigma^2) \leq \frac{\mathbb{E}[(X-\mu)^2]}{k^2\sigma^2} = \frac{\sigma^2}{k^2\sigma^2} = \frac{1}{k^2}.

Example

Light bulbs have mean life 800 hours, standard deviation 50 hours. Bound the probability a bulb lasts outside (700,900)(700, 900) hours. c=100c = 100, k=c/σ=100/50=2k = c/\sigma = 100/50 = 2.
P(∣X−800∣≥100)≤122=0.25.\mathbb{P}(|X - 800| \geq 100) \leq \frac{1}{2^2} = 0.25.
So at least 75% of bulbs last between 700 and 900 hours.

4. Comparison: Actual vs Chebyshev

| Distribution | P(∥X−μ∥≥2σ)\mathbb{P}(\|X-\mu\| \geq 2\sigma) | Chebyshev bound | |:-------------|:---------------------------------:|:---------------:| | Normal | ≈0.045\approx 0.045 | ≤0.25\leq 0.25 | | Binomial(10, 0.5) | ≈0.021\approx 0.021 | ≤0.25\leq 0.25 | | Uniform | 00 (if range <4σ< 4\sigma) | ≤0.25\leq 0.25 | Chebyshev is conservative — it bounds from above, often loosely.

5. Mermaid: Decision Flow

(Diagram)

6. Practice Questions

Q1 (Easy)

A factory produces items with average weight 100g. Give an upper bound on the probability that an item weighs ≥500\geq 500g.
Full Solution
Markov (weight is non-negative): P(X≥500)≤100500=0.2\mathbb{P}(X \geq 500) \leq \frac{100}{500} = 0.2. So at most 20% can weigh ≥500\geq 500g.

Q2 (Medium)

XX has mean 5 and variance 4. Bound P(∣X−5∣≥4)\mathbb{P}(|X-5| \geq 4).
Full Solution
σ=2\sigma = 2, k=4/2=2k = 4/2 = 2. Chebyshev: P(∣X−5∣≥4)≤1/22=0.25\mathbb{P}(|X-5| \geq 4) \leq 1/2^2 = 0.25.

Q3 (Hard)

Suppose E[X]=10\mathbb{E}[X] = 10 and P(X≥20)≤0.1\mathbb{P}(X \geq 20) \leq 0.1. What is the smallest possible E[X]\mathbb{E}[X]? Use Markov.
Full Solution
Wait — this is the reverse problem. Markov says P(X≥20)≤μ/20\mathbb{P}(X \geq 20) \leq \mu/20.
Given P(X≥20)≤0.1\mathbb{P}(X \geq 20) \leq 0.1, we have μ/20≤0.1  ⟹  μ≤2\mu/20 \leq 0.1 \implies \mu \leq 2.
But μ=10\mu = 10 is given, so P(X≥20)≤10/20=0.5\mathbb{P}(X \geq 20) \leq 10/20 = 0.5. The bound 0.10.1 is tighter than Markov allows — this would require μ≤2\mu \leq 2, contradicting μ=10\mu = 10. So the situation is impossible if Markov holds with μ=10\mu = 10.
Next topic: Week 4.1: Continuous Random Variables & PDFs — Continuous random variables and probability density functions. Join Discord PreviousWeek 3.3: Covariance & CorrelationNextWeek 4.1: Continuous Random Variables & PDFs
Document outline

Keep your place and jump directly to a heading.

Table of Contents
System Normal // Awaiting Context

Intelligence Hub

Navigate the knowledge graph to generate context. The Hub adapts dynamically to surface backlinks, related notes, and metadata insights.

Open library →Launch Assessment Engine