Neural Sync Active

Probability — Basic Concepts and Axioms

Registry Synced

Probability — Basic Concepts and Axioms

1940 words

10 min read

Reading compass

Now · 🎯 Learning Objectives

Probability — Basic Concepts and Axioms

🎯 Learning Objectives

After completing this topic, you will be able to:

Define random experiments, sample spaces, and events
Understand the three axioms of probability (Kolmogorov's axioms)
Compute probabilities of equally likely outcomes
Distinguish between simple events and compound events
Apply the classical, relative frequency, and subjective interpretations of probability

📋 Prerequisites

Counting Fundamentals (08-counting-fundamentals) — counting outcomes
Permutations & Combinations (09-permutations, 10-combinations) — needed for complex counting
Basic set theory from BSMA1001-maths-1 (union, intersection, complement)

📖 Core Content

12.1 Intuition: What Is Probability?

Probability quantifies uncertainty — it's a number between 0 and 1 that tells us how likely something is to happen.

0 = impossible (sun rises in the west)
0.5 = equally likely to happen or not (coin flip)
1 = certain (sun rises in the east)

Everyday analogy: When a weather forecaster says "80% chance of rain," they mean that in 80 out of 100 similar weather situations, it rained. It doesn't mean rain is "almost certain" — it means there's an 80% probability. 🔑 Key Insight: Probability is NOT about certainty — it's about quantifying uncertainty in a principled way.

12.2 Three Interpretations of Probability

Interpretation	Definition	Example
Classical (equally likely)	$P(E) = \frac{\text{favorable outcomes}}{\text{total outcomes}}$	Fair coin: $P(H) = 1/2$
Relative frequency (empirical)	$P(E) \approx \frac{\text{times E occurred}}{\text{total trials}}$	Out of 1000 flips, 510 heads → $P(H) \approx 0.51$
Subjective (personal)	Degree of belief	"I think there's a 70% chance I'll pass this exam"

12.3 Key Terminology

Term	Definition	Example
Random Experiment	A process whose outcome is uncertain	Rolling a die
Sample Space (S)	Set of all possible outcomes	$S = \{1, 2, 3, 4, 5, 6\}$
Event (E)	A subset of the sample space	$E = \{\text{even number}\} = \{2, 4, 6\}$
Simple Event	An event with exactly one outcome	$\{3\}$
Compound Event	An event with multiple outcomes	$\{2, 4, 6\}$
Equally Likely	Each outcome has the same probability	Fair die: each face has $1/6$

12.4 The Three Axioms of Probability

Axiom 1 (Non-negativity)

P(E) \geq 0 \text{ for any event } E

Probability can't be negative.

Axiom 2 (Certainty)

P(S) = 1

The probability of the entire sample space is 1 (something must happen).

Axiom 3 (Additivity for Disjoint Events)

If events

E_1, E_2, ...

are mutually exclusive (disjoint — they can't happen simultaneously):

P(E_1 \cup E_2 \cup \cdots) = P(E_1) + P(E_2) + \cdots

12.5 Basic Probability Rules

From the axioms, we derive these rules:

Complement Rule

P(\bar{E}) = 1 - P(E)

Range of Probability

0 \leq P(E) \leq 1

Probability of Equally Likely Outcomes

P(E) = \frac{|E|}{|S|} = \frac{\text{number of outcomes in E}}{\text{number of outcomes in S}}

12.6 Relationships Between Events

(Diagram)

Term	Symbol	Definition	Example (Die)
Union	$A \cup B$	Outcomes in A OR B (or both)	$\{2,4,6\} \cup \{4,5,6\} = \{2,4,5,6\}$
Intersection	$A \cap B$	Outcomes in BOTH A and B	$\{2,4,6\} \cap \{4,5,6\} = \{4,6\}$
Complement	$\bar{A}$ or $A^c$	Outcomes NOT in A	$\bar{\{2,4,6\}} = \{1,3,5\}$
Mutually Exclusive	$A \cap B = \varnothing$	A and B can't happen together	$\{1\} \cap \{2\} = \varnothing$

12.7 Worked Examples

Example 1: Classical Probability (Easy)

Scenario: A fair six-sided die is rolled. What's the probability of getting an odd number? Solution: Step 1: Sample space:

S = \{1, 2, 3, 4, 5, 6\}

|S| = 6

Step 2: Event (odd):

E = \{1, 3, 5\}

|E| = 3

Step 3:

P(\text{odd}) = 3/6 = 1/2

Example 2: Deck of Cards (Medium)

Scenario: A card is drawn from a standard 52-card deck. Find the probability it's: a) A heart b) A face card (J, Q, K) c) The ace of spades Solution: a) Heart: 13 hearts / 52 cards =

13/52 = 1/4

b) Face card: 3 face cards × 4 suits = 12 face cards

P = 12/52 = 3/13 \approx 0.231

c) Ace of spades: Only 1 such card

P = 1/52 \approx 0.0192

Example 3: Using Counting (Harder)

Scenario: A bag contains 5 red, 3 blue, and 2 green marbles. You draw 3 without replacement. Find the probability of getting one of each color. Solution: Step 1: Total marbles = 5 + 3 + 2 = 10 Step 2: Total ways to draw 3 from 10:

C(10, 3) = 120

Step 3: Ways to get one of each color:

C(5,1) \times C(3,1) \times C(2,1) = 5 \times 3 \times 2 = 30

Step 4: Probability:

30/120 = 1/4 = 0.25

12.8 Edge Cases & Gotchas

The Sample Space Must Be Exhaustive

Every possible outcome must be in the sample space. If you're rolling a die,

S = \{1,2,3,4,5,6\}

, not

\{1,2,3,4,5\}

Equally Likely Assumption

The formula

P(E) = |E|/|S|

only works when all outcomes are equally likely. If a die is loaded (weighted), the probability of each face is not

1/6

Probability 0 and Probability 1

$P(E) = 0$ doesn't necessarily mean E is impossible — it could be a probability-zero event like picking a specific point from a continuous range.
$P(E) = 1$ doesn't necessarily mean E is certain — it could be "almost sure" (e.g., picking a non-rational number from the real line).

12.9 Why This Matters

Probability is the mathematical foundation for:

Inferential statistics: Hypothesis tests and confidence intervals
Machine learning: Probabilistic models (Naive Bayes, Bayesian networks)
Decision theory: Expected value, risk analysis
Everyday life: Understanding weather forecasts, medical test results, and risk

📐 Key Formulas / Concepts

Concept	Formula	Notes
Probability (classical)	$P(E) = \frac{	E
Complement Rule	$P(\bar{E}) = 1 - P(E)$	"At least one" problems
Additivity (disjoint)	$P(A \cup B) = P(A) + P(B)$	For mutually exclusive events
Range	$0 \leq P(E) \leq 1$	Always
Certainty	$P(S) = 1$	Something must happen

⚠️ Common Pitfalls

Pitfall 1: Assuming All Outcomes Are Equally Likely

The mistake: Saying

P(\text{rain tomorrow}) = 1/2

because either it rains or it doesn't. Why it happens: Two possible outcomes → 50-50 — but they're not equally likely! Correction: The classical definition requires equally likely outcomes. Weather outcomes are not equally likely. Use relative frequency or subjective probability instead.

Pitfall 2: Confusing "And" with "Or"

The mistake: Computing

P(A \text{ or } B)

by adding when events overlap. Why it happens: "Or" sounds like addition in everyday language. Correction: For overlapping events,

P(A \cup B) = P(A) + P(B) - P(A \cap B)

. Just adding double-counts the overlap.

Pitfall 3: Thinking $P(\bar{A}) = \frac{1}{P(A)}$

The mistake: Assuming complement means reciprocal. Why it happens: Confusing complement (1 - P) with something else. Correct:

P(\bar{A}) = 1 - P(A)

. If

P(A) = 0.3

, then

P(\bar{A}) = 0.7

, not

1/0.3

📝 Practice Questions

Q1: Sample Space

A coin is flipped twice. List the sample space.
<details> Solution
$S = \{HH, HT, TH, TT\}$

There are $2^2 = 4$ equally likely outcomes.

</details> > **Q2: Basic Probability** > > > > A fair die is rolled. Find the probability of getting a number greater than 4. > > <details> Solution > > $S = \{1,2,3,4,5,6\}$, $|S| = 6$ $E = \{5,6\}$, $|E| = 2$ $P = 2/6 = 1/3$ > > $\boxed{1/3}$ </details> > **Q3: Complement** > > > > The probability of rain tomorrow is 0.35. What's the probability it does NOT rain? > > <details> Solution > > $P(\text{no rain}) = 1 - 0.35 = 0.65$ > > $\boxed{0.65}$ </details> > **Q4: Union of Events** > > > > $P(A) = 0.4$, $P(B) = 0.3$, $P(A \cap B) = 0.1$. Find $P(A \cup B)$. > > <details> Solution > > $P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.4 + 0.3 - 0.1 = 0.6$ > > $\boxed{0.6}$ </details> > **Q5: Deck of Cards** > > > > Draw one card from a standard deck. Probability it's either a heart or a king? > > <details> Solution > > $P(\text{heart}) = 13/52$, $P(\text{king}) = 4/52$, $P(\text{heart} \cap \text{king}) = 1/52$ (king of hearts) > > $P(\text{heart} \cup \text{king}) = \frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}$ > > $\boxed{4/13 \approx 0.3077}$ </details> > **Q6: Rolling Two Dice** > > > > Two fair dice are rolled. Probability the sum is 7? > > <details> Solution > > Total outcomes: $6 \times 6 = 36$ > > Outcomes with sum 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) → 6 outcomes > > $P = 6/36 = 1/6$ > > $\boxed{1/6}$ </details> > **Q7: Using Counting** > > > > A bag has 8 red and 4 blue marbles. Draw 2 without replacement. Probability both are red? > > <details> Solution > > Total ways: $C(12,2) = 66$ Ways to get 2 red: $C(8,2) = 28$ > > $P(\text{both red}) = 28/66 = 14/33 \approx 0.424$ > > $\boxed{14/33}$ </details> > **Q8: Mutually Exclusive** > > > > Events A and B are mutually exclusive with $P(A) = 0.2$ and $P(B) = 0.5$. Find $P(A \cup B)$. > > <details> Solution > > Since A and B are mutually exclusive, $P(A \cap B) = 0$. > > $P(A \cup B) = P(A) + P(B) = 0.2 + 0.5 = 0.7$ > > $\boxed{0.7}$ </details> > **Q9: At Least One** > > Roll a fair die 3 times. Probability of getting at least one 6? > > <details> Solution > > $P(\text{at least one 6}) = 1 - P(\text{no 6s})$ > > $P(\text{no 6 on one roll}) = 5/6$ > > $P(\text{no 6 in 3 rolls}) = (5/6)^3 = 125/216$ > > $P(\text{at least one 6}) = 1 - 125/216 = 91/216 \approx 0.421$ > > $\boxed{91/216}$ </details> > **Q10: Application** > > > > A medical test detects a disease with 95% accuracy (if you have the disease, it's positive 95% of the time). The disease affects 2% of the population. If a random person tests positive, does it mean they almost certainly have the disease? (We'll answer this fully in Bayes' theorem — for now, guess and then compute the "positive predictive value" if the false positive rate is 5%.) > > <details> Solution > > This preview of Bayes' theorem: > > Out of 10,000 people: > > - 200 have the disease (2%) > - 200 × 0.95 = 190 test positive (true positives) > - 200 × 0.05 = 10 test negative (false negatives) > - 9,800 don't have the disease (98%) > - 9,800 × 0.05 = 490 test positive (false positives) > - 9,800 × 0.95 = 9,310 test negative (true negatives) > > Total positive tests: 190 + 490 = 680 > > Probability of having the disease given a positive test: 190/680 \\approx 0.279 \\text{ (only 27.9%!)} > > **The test is "95% accurate" but a positive result means less than 30% chance of having the disease!** This is because the disease is rare, so false positives outnumber true positives. This is why understanding conditional probability (next topic) is critical. </details> * * * ## 🔗 Cross-References - **Next topic:** [Probability Rules](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules) — the addition rule, multiplication rule, and probability trees - **Week 8 (Conditional Probability & Bayes):** Building on these foundations - **Weeks 9-12 (Random Variables & Distributions):** Probability as the foundation - **BSMA1001-maths-1:** Set operations (union, intersection, complement) - **BSMA1004 (Stats 2):** Hypothesis testing and p-values [Join Discord](https://discord.gg/gE2m4Qrdqv) [Previous**Combinations**](/notes/01-foundation-bsma1002-stats-1-week06-10-combinations)[Next**Probability Rules**](/notes/01-foundation-bsma1002-stats-1-week07-12-probability-rules)