Week 10.1: Introduction to Hypothesis Testing

Prerequisites: CLT (Week 5.2: Law of Large Numbers & Central Limit Theorem), Confidence Intervals (Week 8.2: Confidence Intervals) Cross-links: BSMA3012 (Linear Stat Models) — significance testing Core question: How do we decide if a claim about a parameter is supported by data?

1. Intuition: The Scientific Method in Statistics

1. Start with a null hypothesis ($H_0$) — the "default" or "status quo".
2. Collect data.
3. If the data is unlikely under $H_0$, reject $H_0$ in favour of the alternative ($H_1$).
4. Otherwise, fail to reject $H_0$ (not the same as "accepting" $H_0$).

Example: "Innocent until proven guilty." $H_0$: innocent. Evidence must be strong enough to convict (reject $H_0$).

2. Components of a Hypothesis Test

• Two-sided: $\theta \neq \theta_0$
• One-sided (upper): $\theta > \theta_0$
• One-sided (lower): $\theta < \theta_0$ 3. Test statistic: A function of the data whose distribution under $H_0$ is known. 4. Rejection region: Values of the test statistic that lead to rejecting $H_0$. 5. Decision: Reject $H_0$ if test statistic falls in rejection region; otherwise fail to reject.

3. Errors in Hypothesis Testing

• $\alpha = \mathbb{P}(\text{Type I error}) = \mathbb{P}(\text{Reject } H_0 \mid H_0 \text{ true})$ — significance level
• $\beta = \mathbb{P}(\text{Type II error}) = \mathbb{P}(\text{Fail to reject } H_0 \mid H_0 \text{ false})$
• Power $= 1 - \beta = \mathbb{P}(\text{Reject } H_0 \mid H_0 \text{ false})$

4. Mermaid: Decision Flowchart

5. The Test Statistic Approach

1. State $H_0$ and $H_1$.
2. Choose $\alpha$ (commonly $0.05$ or $0.01$).
3. Determine the appropriate test statistic.
4. Find the critical value(s) from the reference distribution.
5. Compute the test statistic from the data.
6. Make a decision.

6. Power of a Test

• The true parameter value (effect size)
• Sample size $n$
• Significance level $\alpha$
• The test used

Goal: Design tests with high power. Low power means we might miss real effects.

7. Practice Questions

Q1 (Easy)

Full Solution

$\mathbb{P}(\text{Type I error}) = \alpha = 0.05$ by definition.

Q2 (Medium)

Full Solution

Power $= 1 - \beta = 1 - 0.2 = 0.8$.

Q3 (Hard)

Full Solution

Failing to reject $H_0$ means the evidence was insufficient to conclude otherwise. It does not prove $H_0$ true — just that we couldn't detect a difference. This is like a "not guilty" verdict (lack of evidence for conviction), not "innocent".

Next topic: Week 10.2: Z-test & p-value — Z-test and p-value approach. Join Discord PreviousBayesian EstimationNextWeek 10.2: Z-test & p-value