Neural Sync Active

Conditional Probability and Independence

Registry Synced

Conditional Probability and Independence

2057 words

10 min read

Reading compass

Now · 🎯 Learning Objectives

Conditional Probability and Independence

🎯 Learning Objectives

After completing this topic, you will be able to:

Compute conditional probability $P(A|B)$ for any two events
Understand when and why conditional probability differs from joint probability
Determine whether events are independent using the formal definition
Apply the multiplication rule for dependent events
Use probability trees to visualize conditional probabilities

📋 Prerequisites

Probability Intro (11-probability-intro) — sample spaces, events, basic probability
Probability Rules (12-probability-rules) — addition and multiplication rules
Basic algebra

📖 Core Content

14.1 Intuition: How New Information Changes Probability

Conditional probability answers the question: "Given that we know something happened, how does that change the probability of something else?" Example: The probability of having lung cancer (about 0.1%) is very different from the probability of having lung cancer given that you smoke (about 15%). The information "smokes" dramatically changes the probability.

Everyday analogy: Before a cricket match, you estimate India's chance of winning at 60%. But then you learn the pitch is a spin-friendly one (favoring India), and you update your estimate to 75%. That's conditional probability — updating based on new information. 🔑 Key Insight: Almost all real-world probability is conditional. When you check the weather forecast, you're looking at $P(\text{rain} | \text{current weather conditions})$ .

14.2 Formal Definition

P(A|B) = \frac{P(A \cap B)}{P(B)}

Read as: "The probability of A given B equals the probability of both A and B happening, divided by the probability of B." Why divide by $P(B)$ ? When we condition on B, we're restricting our attention to cases where B happened. Out of those cases, how many also have A? The probability is the proportion of B that is also A. Conditions:

$P(B) > 0$ . If $P(B) = 0$ , $P(A|B)$ is undefined (you can't condition on an impossible event).

14.3 Intuitive Interpretation

(Diagram)

P(A|B)

= (grey area) / (entire B circle) When we condition on B, B becomes the "new sample space." We're only looking at the B circle, and asking what fraction of it is also in A.

14.4 The Multiplication Rule (General Form)

Rearranging the conditional probability formula:

P(A \cap B) = P(A) \times P(B|A) = P(B) \times P(A|B)

Example: Draw 2 cards without replacement. Probability both are hearts?

P(\text{H}_2 | \text{H}_1) = \frac{12}{51}

P(\text{H}_1 \cap \text{H}_2) = \frac{13}{52} \times \frac{12}{51} = \frac{156}{2652} = \frac{1}{17}

14.5 Independence Revisited

Definition: Events A and B are independent if and only if:

P(A|B) = P(A)

(This is equivalent to

P(B|A) = P(B)

and

P(A \cap B) = P(A)P(B)

.) Intuition: If A and B are independent, knowing B happened gives you no information about A. The conditional probability is the same as the unconditional probability.

Disjoint vs. Independent

CRITICAL: Disjoint events are NEVER independent (unless one has probability 0). Proof: If A and B are disjoint and $P(A) > 0$ , $P(B) > 0$ , then $P(A|B) = 0 \neq P(A)$ . Knowing B happened tells you A definitely didn't happen — that's dependence.

14.6 Conditional Probability from Contingency Tables

Example:

	Coffee	Tea	Total
Male	40	30	70
Female	35	45	80
Total	75	75	150

P(\text{Coffee} | \text{Male}) = \frac{P(\text{Coffee} \cap \text{Male})}{P(\text{Male})} = \frac{40/150}{70/150} = \frac{40}{70} = \frac{4}{7}

Shortcut: Just look at the Male row:

40/70 = 4/7

14.7 Law of Total Probability

The law of total probability allows us to compute

P(A)

by conditioning on a partition of the sample space. If

B_1, B_2, ..., B_k

form a partition (mutually exclusive and exhaustive):

P(A) = P(A|B_1)P(B_1) + P(A|B_2)P(B_2) + \cdots + P(A|B_k)P(B_k)

Example: 60% of items come from Machine 1 (5% defective), 40% from Machine 2 (8% defective).

P(\text{defective}) = P(D|M_1)P(M_1) + P(D|M_2)P(M_2)

= 0.05 \times 0.60 + 0.08 \times 0.40 = 0.030 + 0.032 = 0.062

14.8 Worked Examples

Example 1: Basic Conditional Probability (Easy)

Scenario: Roll a fair die. Given that the result is even, what's the probability it's a 6? Solution:

P(\text{even}) = 3/6 = 1/2

P(6 \cap \text{even}) = 1/6

P(6|\text{even}) = \frac{1/6}{1/2} = \frac{1}{3}

Intuition: Among the even numbers {2,4,6}, one of three is a 6 → 1/3.

Example 2: From Contingency Table (Medium)

	Owns a Car	No Car	Total
Lives in City	30	20	50
Lives in Suburb	60	15	75
Total	90	35	125

Find: a)

P(\text{Car} | \text{City})

P(\text{Suburb} | \text{No Car})

Solution: a)

P(\text{Car} | \text{City}) = 30/50 = 3/5 = 0.6

P(\text{Suburb} | \text{No Car}) = 15/35 = 3/7 \approx 0.429

Example 3: Testing Independence (Harder)

Scenario: Using the table above, are "Lives in City" and "Owns a Car" independent? Solution:

P(\text{City} \cap \text{Car}) = 30/125 = 0.24

P(\text{City}) = 50/125 = 0.40

P(\text{Car}) = 90/125 = 0.72

Check:

P(\text{City}) \times P(\text{Car}) = 0.40 \times 0.72 = 0.288

Since

0.24 \neq 0.288

, they are not independent. The conditional view:

P(\text{Car} | \text{City}) = 0.60 \neq P(\text{Car}) = 0.72

. City dwellers are less likely to own cars.

14.9 Edge Cases & Gotchas

Conditioning on a Zero-Probability Event

P(A|B)

is undefined when

P(B) = 0

. In continuous distributions, the conditional probability is defined differently (using PDFs).

Symmetry of Conditional Probability

P(A|B)

is generally NOT equal to

P(B|A)

. This is a common and dangerous confusion (see Bayes' theorem). Example:

P(\text{rain} | \text{clouds})

is high (most rainy days are cloudy). But

P(\text{clouds} | \text{rain})

= 1 (if it's raining, it must be cloudy). These are very different.

14.10 Why This Matters

Conditional probability is the foundation of:

Bayes' theorem (next topic): Updating beliefs with evidence
Statistical inference: All conclusions are conditional on the data
Machine learning: Naive Bayes classifier, Bayesian networks
Medical testing: Positive predictive value = $P(\text{disease} | \text{positive test})$

📐 Key Formulas / Concepts

Concept	Formula	Interpretation
Conditional Probability	$P(A	B) = \frac{P(A \cap B)}{P(B)}$
Multiplication Rule	$P(A \cap B) = P(A) \times P(B	A)$
Independence	$P(A	B) = P(A)$
Law of Total Probability	$P(A) = \sum P(A	B_i)P(B_i)$

⚠️ Common Pitfalls

Pitfall 1: Confusing $P(A|B)$ with $P(B|A)$

The mistake: Thinking

P(\text{disease} | \text{positive test}) = P(\text{positive test} | \text{disease})

. Why it happens: The notation looks symmetric; people conflate the two. Example:

P(\text{positive} | \text{disease}) = 0.95

(sensitivity), but

P(\text{disease} | \text{positive})

could be very small if the disease is rare.

Pitfall 2: Assuming Independence Without Checking

The mistake: Multiplying probabilities

P(A)P(B)

to get

P(A \cap B)

without checking if events are independent. Why it happens: It's the simplest formula. Correction: Always ask: "Does A affect B?" If drawing without replacement, events are dependent.

Pitfall 3: Forgetting That Conditional Probabilities Are Probabilities

The mistake: Thinking

P(A|B)

can be > 1. Why it happens: The formula involves division, so students worry about overflow. Correction:

P(A|B)

is a probability — it must satisfy

0 \leq P(A|B) \leq 1

. Since

P(A \cap B) \leq P(B)

, the ratio is always ≤ 1.

📝 Practice Questions

Q1: Basic Conditional Probability

Two fair dice are rolled. Given that the sum is 7, what's the probability that one die shows a 6?
<details> Solution
Step 1: Outcomes with sum 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) → 6 outcomes Step 2: Among these, those with a 6: (1,6), (6,1) → 2 outcomes Step 3: $P = 2/6 = 1/3$

$\boxed{1/3}$

</details> > **Q2: From Table** > > > >

	Smartphone	No Smartphone	Total
Age < 30	45	15	60
Age ≥ 30	30	35	65
Total	75	50	125

Find $P(\text{Smartphone} | \text{Age < 30})$ .
<details> Solution
$P(\text{Smartphone} | <30) = \frac{45}{60} = \frac{3}{4} = 0.75$

$\boxed{0.75}$

</details> > **Q3: Multiplication Rule** > > > > A bag has 5 red and 7 blue chips. Draw 2 without replacement. Probability both are the same color? > > <details> Solution > > $P(\text{both red}) = \frac{5}{12} \times \frac{4}{11} = \frac{20}{132}$ > > $P(\text{both blue}) = \frac{7}{12} \times \frac{6}{11} = \frac{42}{132}$ > > $P(\text{same color}) = \frac{20}{132} + \frac{42}{132} = \frac{62}{132} = \frac{31}{66}$ > > $\boxed{31/66 \approx 0.470}$ </details> > **Q4: Independence Check** > > > > $P(A) = 0.3$, $P(B) = 0.6$, $P(A \cap B) = 0.18$. Are A and B independent? > > <details> Solution > > $P(A) \times P(B) = 0.3 \times 0.6 = 0.18$ $P(A \cap B) = 0.18$ > > Since $0.18 = 0.18$, they **are independent**. > > Alternatively: $P(A|B) = 0.18/0.6 = 0.3 = P(A)$ ✅ </details> > **Q5: Law of Total Probability** > > > > A company has 3 divisions. Division A has 20% of employees (5% managers), Division B has 50% (10% managers), Division C has 30% (3% managers). What percent of all employees are managers? > > <details> Solution > > $P(\text{manager}) = P(M|A)P(A) + P(M|B)P(B) + P(M|C)P(C)$ $= 0.05(0.20) + 0.10(0.50) + 0.03(0.30)$ $= 0.01 + 0.05 + 0.009 = 0.069$ > > So 6.9% of all employees are managers. > > $\boxed{0.069 \text{ or } 6.9\%}$ </details> > **Q6: Two-Stage Drawing** > > > > A bag has 4 red, 3 blue, 5 green marbles. Draw 2 without replacement. $P(\text{second is blue})$? > > <details> Solution > > Using the law of total probability: > > $P(\text{second blue}) = P(\text{second blue} | \text{first red})P(\text{first red}) + P(\text{second blue} | \text{first blue})P(\text{first blue}) + P(\text{second blue} | \text{first green})P(\text{first green})$ > > $P(\text{second blue}) = \frac{3}{11} \times \frac{4}{12} + \frac{2}{11} \times \frac{3}{12} + \frac{3}{11} \times \frac{5}{12}$ > > $= \frac{12}{132} + \frac{6}{132} + \frac{15}{132} = \frac{33}{132} = \frac{1}{4}$ > > **Notice:** $P(\text{second blue}) = 3/12 = 1/4 = P(\text{first blue})$! The marginal probability of drawing a blue on the second draw is the same as the first. This is always true for random draws without replacement. > > $\boxed{1/4}$ </details> > **Q7: Conditional from Joint** > > > > $P(A) = 0.4$, $P(B) = 0.5$, $P(A \cup B) = 0.7$. Find $P(A|B)$. > > <details> Solution > > **Step 1:** Find $P(A \cap B)$: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ $0.7 = 0.4 + 0.5 - P(A \cap B)$ $P(A \cap B) = 0.9 - 0.7 = 0.2$ > > **Step 2:** $P(A|B) = \frac{0.2}{0.5} = 0.4$ > > $\boxed{0.4}$ </details> > **Q8: Application — Weather** > > > > In a certain city: $P(\text{rain}) = 0.3$, $P(\text{cloudy}) = 0.5$, $P(\text{rain} \cap \text{cloudy}) = 0.25$. > > a) Find $P(\text{rain} | \text{cloudy})$. b) Find $P(\text{cloudy} | \text{rain})$. c) Are rain and cloudy independent? > > <details> Solution > > a) $P(\text{rain} | \text{cloudy}) = \frac{0.25}{0.5} = 0.5$ > > b) $P(\text{cloudy} | \text{rain}) = \frac{0.25}{0.3} \approx 0.833$ > > c) Check independence: $P(\text{rain}) \times P(\text{cloudy}) = 0.3 \times 0.5 = 0.15$ $P(\text{rain} \cap \text{cloudy}) = 0.25$ Since $0.15 \neq 0.25$, they're **dependent** (as expected). > > $\boxed{0.5,\ 0.833,\ \text{Dependent}}$ </details> > **Q9: Sensitivity/Specificity** > > > > A test has 90% sensitivity and 85% specificity. Disease prevalence is 2%. Find: > > a) $P(\text{positive} | \text{disease})$ b) $P(\text{negative} | \text{no disease})$ c) $P(\text{disease} | \text{positive})$ > > <details> Solution > > a) Sensitivity = $P(\text{positive} | \text{disease}) = 0.90$ > > b) Specificity = $P(\text{negative} | \text{no disease}) = 0.85$ > > c) $P(\text{positive}) = P(\text{pos} | \text{disease})P(\text{disease}) + P(\text{pos} | \text{no disease})P(\text{no disease})$ $= 0.90(0.02) + 0.15(0.98) = 0.018 + 0.147 = 0.165$ > > $P(\text{disease} | \text{positive}) = \frac{0.018}{0.165} \approx 0.109 = 10.9\%$ > > $\boxed{0.90,\ 0.85,\ 0.109}$ </details> > **Q10: Sequential Probability** > > > > A box has 3 red and 7 blue chips. Draw chips with replacement until you get a red one. Find the probability that you need exactly 3 draws. > > <details> Solution > > For "exactly 3 draws," the first 2 must be blue and the 3rd must be red. > > $P(\text{blue}) = 7/10$, $P(\text{red}) = 3/10$ > > Since draws are with replacement (independent): > > $P(\text{first 2 blue, 3rd red}) = \frac{7}{10} \times \frac{7}{10} \times \frac{3}{10} = \frac{147}{1000} = 0.147$ > > This is a geometric distribution problem (Week 11). > > $\boxed{0.147}$ </details> * * * ## 🔗 Cross-References - **Next topic:** [Bayes' Theorem](/notes/01-foundation-bsma1002-stats-1-week08-14-bayes-theorem) — the reverse conditional probability - **Previous:** [Probability Rules](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules) — multiplication rule foundation - **Week 11 (Geometric Distribution):** "Waiting time" problems use conditional probability - **BSMA1004 (Stats 2):** Bayesian inference - **BSMA1001-maths-1:** Partition of a set (law of total probability) [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**Probability Rules**](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules)[Next**Bayes' Theorem**](/notes/01-foundation-bsma1002-stats-1-week08-14-bayes-theorem)