Probability — Basic Concepts and Axioms

🎯 Learning Objectives

• Define random experiments, sample spaces, and events
• Understand the three axioms of probability (Kolmogorov's axioms)
• Compute probabilities of equally likely outcomes
• Distinguish between simple events and compound events
• Apply the classical, relative frequency, and subjective interpretations of probability

📋 Prerequisites

• Counting Fundamentals (08-counting-fundamentals) — counting outcomes
• Permutations & Combinations (09-permutations, 10-combinations) — needed for complex counting
• Basic set theory from BSMA1001-maths-1 (union, intersection, complement)

---

📖 Core Content

12.1 Intuition: What Is Probability?

• 0 = impossible (sun rises in the west)
• 0.5 = equally likely to happen or not (coin flip)
• 1 = certain (sun rises in the east)

Everyday analogy: When a weather forecaster says "80% chance of rain," they mean that in 80 out of 100 similar weather situations, it rained. It doesn't mean rain is "almost certain" — it means there's an 80% probability. 🔑 Key Insight: Probability is NOT about certainty — it's about quantifying uncertainty in a principled way.

12.2 Three Interpretations of Probability

12.3 Key Terminology

12.4 The Three Axioms of Probability

Axiom 1 (Non-negativity)

$$P(E) \geq 0 \text{ for any event } E$$

Axiom 2 (Certainty)

$$P(S) = 1$$

Axiom 3 (Additivity for Disjoint Events)

$$P(E_1 \cup E_2 \cup \cdots) = P(E_1) + P(E_2) + \cdots$$

12.5 Basic Probability Rules

Complement Rule

$$P(\bar{E}) = 1 - P(E)$$

Range of Probability

$$0 \leq P(E) \leq 1$$

Probability of Equally Likely Outcomes

$$P(E) = \frac{|E|}{|S|} = \frac{\text{number of outcomes in E}}{\text{number of outcomes in S}}$$

12.6 Relationships Between Events

12.7 Worked Examples

Example 1: Classical Probability (Easy)

Example 2: Deck of Cards (Medium)

$$P = 12/52 = 3/13 \approx 0.231$$ $$P = 1/52 \approx 0.0192$$

Example 3: Using Counting (Harder)

$$C(5,1) \times C(3,1) \times C(2,1) = 5 \times 3 \times 2 = 30$$

12.8 Edge Cases & Gotchas

The Sample Space Must Be Exhaustive

Equally Likely Assumption

Probability 0 and Probability 1

• $P(E) = 0$ doesn't necessarily mean E is impossible — it could be a probability-zero event like picking a specific point from a continuous range.
• $P(E) = 1$ doesn't necessarily mean E is certain — it could be "almost sure" (e.g., picking a non-rational number from the real line).

12.9 Why This Matters

• Inferential statistics: Hypothesis tests and confidence intervals
• Machine learning: Probabilistic models (Naive Bayes, Bayesian networks)
• Decision theory: Expected value, risk analysis
• Everyday life: Understanding weather forecasts, medical test results, and risk

---

📐 Key Formulas / Concepts

---

⚠️ Common Pitfalls

Pitfall 1: Assuming All Outcomes Are Equally Likely

Pitfall 2: Confusing "And" with "Or"

Pitfall 3: Thinking $P(\bar{A}) = \frac{1}{P(A)}$

---

📝 Practice Questions

Q1: Sample Space

</strong>

A coin is flipped twice. List the sample space.

<details> <strong>Solution</strong>

$S = \{HH, HT, TH, TT\}$

There are $2^2 = 4$ equally likely outcomes.

</details> > **Q2: Basic Probability** > > </strong> > > A fair die is rolled. Find the probability of getting a number greater than 4. > > <details> <strong>Solution</strong> > > $S = \{1,2,3,4,5,6\}$, $|S| = 6$ $E = \{5,6\}$, $|E| = 2$ $P = 2/6 = 1/3$ > > $\boxed{1/3}$ </details> > **Q3: Complement** > > </strong> > > The probability of rain tomorrow is 0.35. What's the probability it does NOT rain? > > <details> <strong>Solution</strong> > > $P(\text{no rain}) = 1 - 0.35 = 0.65$ > > $\boxed{0.65}$ </details> > **Q4: Union of Events** > > </strong> > > $P(A) = 0.4$, $P(B) = 0.3$, $P(A \cap B) = 0.1$. Find $P(A \cup B)$. > > <details> <strong>Solution</strong> > > $P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.4 + 0.3 - 0.1 = 0.6$ > > $\boxed{0.6}$ </details> > **Q5: Deck of Cards** > > </strong> > > Draw one card from a standard deck. Probability it's either a heart or a king? > > <details> <strong>Solution</strong> > > $P(\text{heart}) = 13/52$, $P(\text{king}) = 4/52$, $P(\text{heart} \cap \text{king}) = 1/52$ (king of hearts) > > $P(\text{heart} \cup \text{king}) = \frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}$ > > $\boxed{4/13 \approx 0.3077}$ </details> > **Q6: Rolling Two Dice** > > </strong> > > Two fair dice are rolled. Probability the sum is 7? > > <details> <strong>Solution</strong> > > Total outcomes: $6 \times 6 = 36$ > > Outcomes with sum 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) → 6 outcomes > > $P = 6/36 = 1/6$ > > $\boxed{1/6}$ </details> > **Q7: Using Counting** > > </strong> > > A bag has 8 red and 4 blue marbles. Draw 2 without replacement. Probability both are red? > > <details> <strong>Solution</strong> > > Total ways: $C(12,2) = 66$ Ways to get 2 red: $C(8,2) = 28$ > > $P(\text{both red}) = 28/66 = 14/33 \approx 0.424$ > > $\boxed{14/33}$ </details> > **Q8: Mutually Exclusive** > > </strong> > > Events A and B are mutually exclusive with $P(A) = 0.2$ and $P(B) = 0.5$. Find $P(A \cup B)$. > > <details> <strong>Solution</strong> > > Since A and B are mutually exclusive, $P(A \cap B) = 0$. > > $P(A \cup B) = P(A) + P(B) = 0.2 + 0.5 = 0.7$ > > $\boxed{0.7}$ </details> > **Q9: At Least One** > > Roll a fair die 3 times. Probability of getting at least one 6? > > <details> <strong>Solution</strong> > > $P(\text{at least one 6}) = 1 - P(\text{no 6s})$ > > $P(\text{no 6 on one roll}) = 5/6$ > > $P(\text{no 6 in 3 rolls}) = (5/6)^3 = 125/216$ > > $P(\text{at least one 6}) = 1 - 125/216 = 91/216 \approx 0.421$ > > $\boxed{91/216}$ </details> > **Q10: Application** > > </strong> > > A medical test detects a disease with 95% accuracy (if you have the disease, it's positive 95% of the time). The disease affects 2% of the population. If a random person tests positive, does it mean they almost certainly have the disease? (We'll answer this fully in Bayes' theorem — for now, guess and then compute the "positive predictive value" if the false positive rate is 5%.) > > <details> <strong>Solution</strong> > > This preview of Bayes' theorem: > > Out of 10,000 people: > > - 200 have the disease (2%) > - 200 × 0.95 = 190 test positive (true positives) > - 200 × 0.05 = 10 test negative (false negatives) > - 9,800 don't have the disease (98%) > - 9,800 × 0.05 = 490 test positive (false positives) > - 9,800 × 0.95 = 9,310 test negative (true negatives) > > Total positive tests: 190 + 490 = 680 > > Probability of having the disease given a positive test: 190/680 \\approx 0.279 \\text{ (only 27.9%!)} > > **The test is "95% accurate" but a positive result means less than 30% chance of having the disease!** This is because the disease is rare, so false positives outnumber true positives. This is why understanding conditional probability (next topic) is critical. </details> * * * ## 🔗 Cross-References - **Next topic:** [Probability Rules](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules) — the addition rule, multiplication rule, and probability trees - **Week 8 (Conditional Probability & Bayes):** Building on these foundations - **Weeks 9-12 (Random Variables & Distributions):** Probability as the foundation - **BSMA1001-maths-1:** Set operations (union, intersection, complement) - **BSMA1004 (Stats 2):** Hypothesis testing and p-values [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**Combinations**](/notes/01-foundation-bsma1002-stats-1-week06-10-combinations)[Next**Probability Rules**](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules)