Introduction to Linear Regression

🎯 Learning Objectives

• Formulate the simple linear regression model
• Estimate parameters using ordinary least squares (OLS)
• Interpret regression coefficients
• Understand assumptions underpinning the model

1.1 Intuition: Finding the Line of Best Fit

🔑 Key Insight: Regression is about explaining variation in an outcome variable $y$ using one or more predictor variables $x$.

1.2 The Simple Linear Regression Model

• $y_i$ = response (dependent) variable
• $x_i$ = predictor (independent) variable
• $\beta_0$ = intercept (expected $y$ when $x = 0$)
• $\beta_1$ = slope (expected change in $y$ per unit change in $x$)
• $\varepsilon_i$ = random error term, $\varepsilon_i \sim N(0, \sigma^2)$

1.3 Least Squares Estimation

1.4 Interpretation

• $\hat{\beta}_1$ = "a one-unit increase in $x$ is associated with a $\hat{\beta}_1$ change in $y$"
• $\hat{\beta}_0$ = "when $x = 0$, the predicted $y$ is $\hat{\beta}_0$" (may not be meaningful if $x=0$ is outside data range)

✅ Practice Questions

Solution

$\bar{x} = 2$, $\bar{y} = 11/3 \approx 3.67$ $S_{xx} = (1-2)^2 + (2-2)^2 + (3-2)^2 = 1 + 0 + 1 = 2$ $S_{xy} = (1-2)(2-3.67) + (2-2)(4-3.67) + (3-2)(5-3.67) = (-1)(-1.67) + 0 + (1)(1.33) = 1.67 + 1.33 = 3$ $\hat{\beta}_1 = 3/2 = 1.5$ $\hat{\beta}_0 = 3.67 - 1.5(2) = 0.67$ $\hat{y} = 0.67 + 1.5x$ Join Discord NextRegression Applications