Neural Sync Active

Week 7: Probability Foundations & Properties

Registry Synced

Week 7: Probability Foundations & Properties

3779 words

19 min read

2026-09-12

Reading compass

Now · TL;DR

Statistics for Data Science I — Week 7: Probability Foundations & Properties

IITM’s Week 7 covers probability, basic definitions of probability, events, and properties of probability.

The syllabus gives the topic headings rather than every sub-result, so the detailed notes below are a standard expansion of those listed concepts.

TL;DR

Probability formalizes uncertainty.

The basic objects are:

\boxed{ \text{Random experiment} \rightarrow \text{Sample space} \rightarrow \text{Events} \rightarrow \text{Probabilities} }

Core rules:

\boxed{0\leq P(A)\leq1}

\boxed{P(S)=1}

\boxed{P(\varnothing)=0}

\boxed{P(A^c)=1-P(A)}

and:

\boxed{ P(A\cup B) = P(A)+P(B)-P(A\cap B) }

For mutually exclusive events:

\boxed{ P(A\cup B)=P(A)+P(B) }

1. Random experiment

A random experiment is a process whose exact outcome is uncertain before it occurs.

Examples:

toss a coin
roll a die
draw a card
record whether a customer buys a product
observe tomorrow's rainfall

The experiment may have known possible outcomes even though we do not know which one will occur.

2. Outcome

An outcome is one possible result of an experiment.

For a die roll:

1,2,3,4,5,6

are individual outcomes.

3. Sample space

The sample space is the set of all possible outcomes.

Usually denoted:

S

or:

\Omega.

For one die:

\boxed{ S=\{1,2,3,4,5,6\} }

4. Coin toss sample space

For one coin toss:

S=\{H,T\}.

For two coin tosses:

\boxed{ S=\{HH,HT,TH,TT\} }

Notice:

HT\neq TH.

The order of tosses matters.

5. Event

An event is a subset of the sample space.

Formally:

\boxed{ A\subseteq S }

Example:

Roll a die.

Let:

A=\{\text{even number}\}.

Then:

A=\{2,4,6\}.

6. Elementary event

An event containing exactly one outcome is an elementary or simple event.

Example:

A=\{4\}.

It represents:

the die shows exactly 4.

7. Compound event

An event containing more than one outcome is a compound event.

Example:

A=\{2,4,6\}.

This represents:

an even number occurs.

8. Certain event

The entire sample space represents an event that must happen:

\boxed{S}

Therefore:

\boxed{P(S)=1}

9. Impossible event

The empty set:

\varnothing

represents an event that cannot occur.

Therefore:

\boxed{ P(\varnothing)=0 }

10. Probability as a number

For an event

A

\boxed{ 0\leq P(A)\leq1 }

Interpretation:

P(A)=0

means impossible in a basic finite model.

P(A)=1

means certain.

Values nearer

1

indicate greater likelihood.

11. Percentage interpretation

Probability may also be written as a percentage.

For example:

P(A)=0.25

means:

25\%.

But probability calculations are usually cleaner in fractions or decimals.

12. Classical probability

If all outcomes in a finite sample space are equally likely:

\boxed{ P(A) = \frac{|A|}{|S|} }

where:

$|A|$ = number of favorable outcomes
$|S|$ = number of possible outcomes

This is exactly why Weeks 5–6 counting matter.

13. Example: die

Sample space:

S=\{1,2,3,4,5,6\}.

Event:

A=\{\text{even}\} = \{2,4,6\}.

Therefore:

P(A)=\frac36=\frac12.

14. Example: two coins

For two fair coin tosses:

S=\{HH,HT,TH,TT\}.

Let:

A=\{\text{exactly one head}\}.

Then:

A=\{HT,TH\}.

Therefore:

P(A)=\frac24=\frac12.

15. Equal likelihood matters

The formula:

P(A)=\frac{|A|}{|S|}

only works directly if outcomes are equally likely.

Suppose a biased coin has:

P(H)=0.8, \qquad P(T)=0.2.

There are two outcomes, but:

P(H)\neq\frac12.

So simply counting outcomes is insufficient.

16. Probability distribution over outcomes

For a finite sample space:

S=\{\omega_1,\omega_2,\dots,\omega_n\},

we may assign probabilities:

P(\omega_i).

They must satisfy:

P(\omega_i)\geq0

and:

\boxed{ \sum_{i=1}^nP(\omega_i)=1 }

17. Probability of an event from outcome probabilities

If event:

A=\{\omega_1,\omega_3,\omega_5\},

then:

\boxed{ P(A) = P(\omega_1)+P(\omega_3)+P(\omega_5) }

when those elementary outcomes are mutually exclusive.

18. Set-theory connection

Probability is deeply built on set theory.

Recall from Maths 1:

union
intersection
complement
subset
empty set

Probability translates these directly into event language.

This is why Week 1 Maths becomes useful again.

19. Union of events

For events:

A

and:

B,

the union:

A\cup B

means:

A occurs or B occurs or both occur.

Important:

In probability, "or" is normally inclusive OR.

20. Example of union

Roll a die.

Let:

A=\{\text{even}\} = \{2,4,6\}.

Let:

B=\{\text{greater than 3}\} = \{4,5,6\}.

Then:

A\cup B = \{2,4,5,6\}.

21. Intersection of events

A\cap B

means:

A and B both occur.

Using the previous example:

A=\{2,4,6\}

B=\{4,5,6\}.

Then:

\boxed{ A\cap B=\{4,6\} }

22. Complement

The complement:

A^c

means:

A does not occur.

If:

S=\{1,2,3,4,5,6\}

and:

A=\{2,4,6\},

then:

A^c=\{1,3,5\}.

23. Complement probability rule

Because exactly one of:

A

or:

A^c

must happen:

P(A)+P(A^c)=1.

Therefore:

\boxed{ P(A^c)=1-P(A) }

24. Why complement is powerful

Suppose you need:

probability of at least one success.

Direct counting may require many cases.

Instead:

\boxed{ P(\text{at least one}) = 1-P(\text{none}) }

This is one of the most useful probability strategies.

25. Example: at least one head

Toss two fair coins.

Probability of at least one head:

Instead of counting:

HH,HT,TH,

use complement.

No heads means:

TT.

So:

P(\text{none})=\frac14.

Thus:

P(\text{at least one head}) = 1-\frac14 = \boxed{\frac34}.

26. Mutually exclusive events

Two events are mutually exclusive or disjoint if they cannot occur simultaneously.

Formally:

\boxed{ A\cap B=\varnothing }

Example from one die roll:

A=\{\text{even}\}

B=\{\text{odd}\}.

They cannot both happen on one roll.

27. Not mutually exclusive

Suppose:

A=\{\text{even}\} = \{2,4,6\}

and:

B=\{\text{greater than 3}\} = \{4,5,6\}.

Their intersection:

A\cap B=\{4,6\}

is not empty.

So they are not mutually exclusive.

28. Addition rule for mutually exclusive events

If:

A\cap B=\varnothing,

then:

\boxed{ P(A\cup B)=P(A)+P(B) }

because there is no overlap to double-count.

29. Example

Roll a die.

Let:

A=\{1\}

B=\{6\}.

They are mutually exclusive.

Therefore:

P(A\cup B) = \frac16+\frac16 = \boxed{\frac13}.

30. General addition rule

If events may overlap:

\boxed{ P(A\cup B) = P(A)+P(B)-P(A\cap B) }

This is the probability version of inclusion-exclusion.

31. Why subtract the intersection?

When we calculate:

P(A)+P(B),

outcomes in:

A\cap B

are counted twice.

Subtract:

P(A\cap B)

once.

32. Example

Suppose:

P(A)=0.6

P(B)=0.5

P(A\cap B)=0.3.

Then:

P(A\cup B) = 0.6+0.5-0.3

= \boxed{0.8}.

33. Useful rearrangement

From:

P(A\cup B) = P(A)+P(B)-P(A\cap B),

we can solve:

\boxed{ P(A\cap B) = P(A)+P(B)-P(A\cup B) }

Useful when the intersection is unknown.

34. Example

Suppose:

P(A)=0.7,

P(B)=0.5,

and:

P(A\cup B)=0.9.

Then:

P(A\cap B) = 0.7+0.5-0.9

= \boxed{0.3}.

35. Event difference

A-B

or:

A\setminus B

means:

A occurs but B does not.

Set-theoretically:

\boxed{ A-B=A\cap B^c }

36. Probability of A but not B

Since:

A=(A\cap B)\cup(A\cap B^c)

with disjoint pieces:

P(A) = P(A\cap B)+P(A\cap B^c).

Therefore:

\boxed{ P(A\cap B^c) = P(A)-P(A\cap B) }

37. Example

Suppose:

P(A)=0.6

and:

P(A\cap B)=0.2.

Then:

P(A\text{ but not }B) = 0.6-0.2 = \boxed{0.4}.

38. De Morgan's laws

From set theory:

\boxed{ (A\cup B)^c = A^c\cap B^c }

and:

\boxed{ (A\cap B)^c = A^c\cup B^c }

These become very useful in probability.

39. Probability interpretation of De Morgan

"Neither A nor B":

(A\cup B)^c = A^c\cap B^c.

"Not both":

(A\cap B)^c = A^c\cup B^c.

Language translation matters a lot.

40. At least one

For two events:

\boxed{ P(A\cup B) }

means at least one occurs.

Its complement:

\boxed{ P(A^c\cap B^c) }

means neither occurs.

Thus:

P(A\cup B) = 1-P(A^c\cap B^c).

41. Exactly one of A and B

Exactly one means:

(A\cap B^c)\cup(A^c\cap B).

The two parts are disjoint.

Therefore:

P(\text{exactly one}) = P(A\cap B^c)+P(A^c\cap B).

42. Exactly one formula

We can rewrite:

P(A\cap B^c)=P(A)-P(A\cap B)

and:

P(A^c\cap B)=P(B)-P(A\cap B).

Therefore:

\boxed{ P(\text{exactly one}) = P(A)+P(B)-2P(A\cap B) }

43. Neither A nor B

\boxed{ P(\text{neither}) = 1-P(A\cup B) }

Using the addition rule:

\boxed{ P(\text{neither}) = 1-P(A)-P(B)+P(A\cap B) }

44. Three-event union

The inclusion-exclusion rule extends to three events:

P(A\cup B\cup C)

= P(A)+P(B)+P(C)

-P(A\cap B)-P(A\cap C)-P(B\cap C)

+P(A\cap B\cap C).

So:

\boxed{ P(A\cup B\cup C) = \sum P(\text{single}) -\sum P(\text{pair intersections}) +P(\text{triple intersection}) }

45. Why the triple intersection is added back

An outcome lying in all three events is initially counted:

3

times.

Then pairwise subtraction removes it:

3

times.

So it becomes counted:

0

times.

We add it once to obtain the correct count:

1.

46. Probability axioms

A standard probability model rests on three fundamental axioms.

Non-negativity

\boxed{ P(A)\geq0 }

Normalization

\boxed{ P(S)=1 }

Additivity for disjoint events

If:

A\cap B=\varnothing,

then:

\boxed{ P(A\cup B)=P(A)+P(B) }

Many other probability rules follow from these.

47. Why $P(\varnothing)=0$

Since:

S\cup\varnothing=S

and they are disjoint:

P(S) = P(S)+P(\varnothing).

Subtract:

P(S)

from both sides:

\boxed{ P(\varnothing)=0 }

48. Monotonicity

If:

A\subseteq B,

then:

\boxed{ P(A)\leq P(B) }

Why?

Because

B

contains every outcome in

A

plus possibly more.

49. Example

Let:

A=\{\text{roll 6}\}

and:

B=\{\text{roll even}\}.

Since:

A\subseteq B,

we expect:

P(A)\leq P(B).

Indeed:

\frac16\leq\frac12.

50. Bounds on intersections

For two events:

P(A\cap B)

cannot exceed either individual probability.

Therefore:

\boxed{ P(A\cap B) \leq \min(P(A),P(B)) }

51. Lower bound on intersection

From:

P(A\cup B)\leq1,

we get:

P(A)+P(B)-P(A\cap B)\leq1.

Therefore:

P(A\cap B)\geq P(A)+P(B)-1.

Since probability cannot be negative:

\boxed{ P(A\cap B) \geq \max(0,P(A)+P(B)-1) }

This is a useful consistency check.

52. Bounds on union

Clearly:

P(A\cup B) \geq \max(P(A),P(B)).

And:

P(A\cup B) \leq P(A)+P(B),

with maximum probability capped at

1

So:

\boxed{ \max(P(A),P(B)) \leq P(A\cup B) \leq \min(1,P(A)+P(B)) }

53. Uniform finite sample spaces

If:

|S|=n

and all outcomes are equally likely, then each elementary outcome has probability:

\boxed{ \frac1n }

If event

A

contains

k

outcomes:

\boxed{ P(A)=\frac{k}{n} }

54. Example using combinations

Suppose 2 people are chosen at random from 5 people.

Total possible unordered pairs:

\binom52=10.

Suppose A and B are two specific people.

Probability they are chosen together:

Favorable selections:

1.

Thus:

P(\text{A and B}) = \boxed{\frac1{10}}.

This is the direct bridge from Week 6 to Week 7.

55. Example: card selection

A standard deck has:

52

cards.

There are:

13

hearts.

If one card is drawn uniformly:

P(\text{heart}) = \frac{13}{52} = \boxed{\frac14}.

56. Example: face card

There are:

Jack
Queen
King

in each of 4 suits.

So:

3\times4=12

face cards.

Therefore:

P(\text{face card}) = \frac{12}{52} = \boxed{\frac3{13}}.

57. Example: heart or king

Let:

A=\{\text{heart}\}

and:

B=\{\text{king}\}.

Then:

P(A)=\frac{13}{52}

P(B)=\frac4{52}.

But the king of hearts belongs to both:

P(A\cap B)=\frac1{52}.

Thus:

P(A\cup B) = \frac{13+4-1}{52} = \frac{16}{52} = \boxed{\frac4{13}}.

Common mistake: adding overlapping probabilities

Wrong:

P(A\cup B)=P(A)+P(B)

for arbitrary events.

Correct:

\boxed{ P(A\cup B) = P(A)+P(B)-P(A\cap B) }

Only drop the intersection when events are mutually exclusive.

Common mistake: mutually exclusive = independent

These are completely different concepts.

Mutually exclusive

Cannot occur together:

A\cap B=\varnothing.

Independent

Knowing one occurred does not change the probability of the other.

Independence is officially part of Week 8, so keep the distinction ready.

If two nonzero-probability events are mutually exclusive, they are actually not independent.

60. Example of why

Suppose:

A=\{\text{roll 1}\}

and:

B=\{\text{roll 2}\}.

A

happens, then

B

becomes impossible.

So knowing A occurred strongly affects B.

Hence mutual exclusivity does not mean independence.

Common mistake: "or" means exactly one

In probability:

A\cup B

usually means:

A or B or both.

If the question means exactly one, the intersection must be excluded.

Common mistake: complement of "A and B"

The complement of:

A\cap B

is not:

A^c\cap B^c.

Correct:

\boxed{ (A\cap B)^c = A^c\cup B^c }

This means:

at least one of A or B fails.

Common mistake: complement of "A or B"

(A\cup B)^c

means neither happens:

\boxed{ A^c\cap B^c }

Common mistake: probability greater than 1

A result like:

1.2

cannot be a probability.

Immediately recheck the calculation.

Valid range:

\boxed{ 0\leq P(A)\leq1 }

Common mistake: negative probability

Similarly:

P(A)<0

is invalid in ordinary probability.

If algebra gives a negative result, some supplied values or calculation are inconsistent.

Common mistake: counting unequally likely outcomes equally

Suppose one event representation contains three labels and another one label.

That does not automatically mean probability:

\frac34

unless the elementary outcomes are equally likely.

Always verify the model.

Common mistake: incorrect sample space

Suppose two coins are tossed.

Wrong:

S=\{0H,1H,2H\}

if you're trying to treat these three as equally likely outcomes.

They are not equally likely:

P(0H)=\frac14

P(1H)=\frac12

P(2H)=\frac14.

The more fundamental equally likely sample space is:

\{HH,HT,TH,TT\}.

68. Granularity matters

A sample space can sometimes be represented at different levels.

For two coins:

Detailed

\{HH,HT,TH,TT\}

Number of heads

\{0,1,2\}

Both can be useful.

But only the first has equal probabilities under fair independent tosses.

So do not automatically assume all elements of any written sample space are equally likely.

69. Venn-diagram mental model

Think of the sample space as a rectangle:


text
+---------------------------+
|             S             |
|       ______              |
|      /  A   \____         |
|     /       / B  \        |
|     \      /     /        |
|      \____/_____/         |
|                           |
+---------------------------+

Areas conceptually represent probabilities.

Then:

overlap = $A\cap B$
combined region = $A\cup B$
outside A = $A^c$

This is useful for translating wording.

70. Probability-from-table example

Suppose 100 students are classified:

	Pass	Fail	Total
Group A	40	10	50
Group B	30	20	50
Total	70	30	100

Choose one student uniformly.

Probability student passed:

\frac{70}{100}=0.7.

Probability student is Group A:

\frac{50}{100}=0.5.

Probability student is Group A and passed:

\frac{40}{100}=0.4.

71. Union from a contingency table

Using the same table:

Let:

A=\{\text{Group A}\}

P=\{\text{Pass}\}.

Then:

P(A\cup P) = P(A)+P(P)-P(A\cap P).

So:

=0.5+0.7-0.4

=\boxed{0.8}.

72. Empirical probability

Probability can also be estimated from observed relative frequency.

Suppose an event occurs:

240

times in:

1000

trials.

Estimated probability:

\frac{240}{1000}=0.24.

As trials accumulate, relative frequency may stabilize around an underlying probability.

This connects descriptive statistics to probability.

73. Theoretical vs empirical probability

Theoretical

Derived from a probability model.

Example fair die:

P(6)=\frac16.

Empirical

Estimated from actual observations.

Example:

\frac{\text{number of sixes observed}} {\text{number of rolls}}.

They need not match exactly in a finite experiment.

74. Long-run frequency intuition

If a fair coin is tossed only 4 times, you might observe:

HHHH.

That does not prove:

P(H)=1.

Probability is not a guarantee about small samples.

Instead, it describes long-run uncertainty.

75. Probability language translation

Wording	Mathematical event
A or B	$A\cup B$
A and B	$A\cap B$
not A	$A^c$
neither A nor B	$A^c\cap B^c$
at least one	union / complement of none
exactly one	$(A\cap B^c)\cup(A^c\cap B)$
A but not B	$A\cap B^c$
both	$A\cap B$

This table is worth learning.

76. Full worked example

Suppose:

P(A)=0.45

P(B)=0.50

and:

P(A\cap B)=0.20.

A or B

P(A\cup B) = 0.45+0.50-0.20 = \boxed{0.75}.

Neither

1-0.75 = \boxed{0.25}.

A but not B

0.45-0.20 = \boxed{0.25}.

B but not A

0.50-0.20 = \boxed{0.30}.

Exactly one

0.25+0.30 = \boxed{0.55}.

Check:

0.20+0.25+0.30+0.25=1.

Everything partitions the sample space correctly.

77. Partition intuition

Events:

A\cap B

A\cap B^c

A^c\cap B

A^c\cap B^c

form four mutually exclusive regions covering the entire sample space.

Therefore their probabilities add to:

1.

This becomes very useful next week.

78. Counting + probability example

Suppose a committee of 3 is chosen uniformly from 8 people.

Total committees:

\binom83=56.

Suppose Alice must be on the committee for event

A

Fix Alice and choose 2 of remaining 7:

\binom72=21.

Thus:

P(A) = \frac{21}{56} = \boxed{\frac38}.

79. Why this simplifies

There is also an intuitive answer.

Three of the eight committee positions are occupied by three randomly selected people.

By symmetry, Alice has probability:

\boxed{\frac38}

of being included.

Counting confirms it.

80. At least one special item example

Suppose 3 objects are chosen from 10:

4 special
6 ordinary

Find probability of at least one special object.

Total:

\binom{10}{3}.

Complement: all 3 ordinary:

\binom63.

Therefore:

P(\text{at least one special}) = 1- \frac{\binom63}{\binom{10}{3}}.

Since:

\binom63=20

and:

\binom{10}{3}=120,

P = 1-\frac16 = \boxed{\frac56}.

This combines Weeks 6 and 7 cleanly.

81. Core formula sheet

Probability range

\boxed{ 0\leq P(A)\leq1 }

Certain event

\boxed{ P(S)=1 }

Impossible event

\boxed{ P(\varnothing)=0 }

Complement

\boxed{ P(A^c)=1-P(A) }

Addition rule

\boxed{ P(A\cup B) = P(A)+P(B)-P(A\cap B) }

Mutually exclusive events

If:

A\cap B=\varnothing,

then:

\boxed{ P(A\cup B)=P(A)+P(B) }

Equally likely finite outcomes

\boxed{ P(A)=\frac{|A|}{|S|} }

82. Week 7 conceptual map


text
Random experiment
      ↓
Sample space S
      ↓
Events A, B, ...
      ↓
Set operations
├── union        A ∪ B
├── intersection A ∩ B
└── complement   Aᶜ
      ↓
Probability measure
├── 0 ≤ P(A) ≤ 1
├── P(S) = 1
├── P(Aᶜ) = 1-P(A)
└── P(A∪B)
    = P(A)+P(B)-P(A∩B)

83. Weeks 5–7 as one story


text
Week 5
How many outcomes exist?
        ↓
Week 6
How many selections/arrangements exist?
        ↓
Week 7
How likely are events?

In equally likely models:

\boxed{ \text{Probability} = \frac{\text{favorable count}} {\text{total count}} }

So combinatorics is not a separate random topic. It feeds directly into probability.

84. What you should actually master

By the end of Stats 1 Week 7, you should be able to:

Define a random experiment.
Construct a sample space.
Define an event as a subset of the sample space.
Distinguish simple and compound events.
Use: $0\leq P(A)\leq1.$
Know: $P(S)=1,\qquad P(\varnothing)=0.$
Compute classical probability for equally likely outcomes.
Use complements: $P(A^c)=1-P(A).$
Translate "or" into union.
Translate "and" into intersection.
Recognize mutually exclusive events.
Apply:

P(A\cup B)=P(A)+P(B)-P(A\cap B).

Solve "at least one" using complements.
Solve "neither," "exactly one," and "A but not B."
Use combinatorics inside probability problems.
Avoid assuming arbitrary sample-space outcomes are equally likely.
Distinguish mutually exclusive events from the idea of independence.

If those are comfortable, Stats 1 Week 7 is under control.

Quick Recall & Flashcards

Test your active recall before proceeding to practice problems. Format:

Question :: Answer.

Week 7 Diagnostic: Two fair 6-sided dice are rolled. What is the probability that the sum of the faces equals 7? :: Answer: A — Total outcomes = 36. Outcomes with sum 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) -> 6 outcomes. Probability = 6/36 = 1/6.
Week 7 Diagnostic: If events A and B are mutually exclusive with P(A) = 0.3 and P(B) = 0.4, what is P(A or B)? :: Answer: 0.7 — For mutually exclusive events, P(A or B) = P(A) + P(B) = 0.3 + 0.4 = 0.7.

Practice & Self-Check

Attempt these diagnostic questions to verify your operational understanding.

Question 1

Two fair 6-sided dice are rolled. What is the probability that the sum of the faces equals 7?

MCQ

3 Unit Assessment

A. 1/6

B. 7/36

C. 5/36

D. 1/12

Question 2

If events A and B are mutually exclusive with P(A) = 0.3 and P(B) = 0.4, what is P(A or B)?

NAT

3 Unit Assessment