Multiple Linear Regression

213 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Multiple Linear Regression ## 2.1 Model Formulation $$ y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \cdots + \beta_k x_{ik} + \varepsilon_i $$ In matrix form: $\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon}$ Where: $$ \mathbf{y} = \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix}, \quad...

Multiple Linear Regression

2.1 Model Formulation

y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \cdots + \beta_k x_{ik} + \varepsilon_i

In matrix form:

\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon}

Where:

\mathbf{y} = \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix}, \quad \mathbf{X} = \begin{pmatrix} 1 & x_{11} & \cdots & x_{1k} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{n1} & \cdots & x_{nk} \end{pmatrix}, \quad \boldsymbol{\beta} = \begin{pmatrix} \beta_0 \\ \vdots \\ \beta_k \end{pmatrix}

2.2 Normal Equations

\mathbf{X}^T\mathbf{X}\hat{\boldsymbol{\beta}} = \mathbf{X}^T\mathbf{y}

Solution:

\hat{\boldsymbol{\beta}} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}

(when

\mathbf{X}^T\mathbf{X}

is invertible)

2.3 Interpretation of Coefficients

Each coefficient

\hat{\beta}_j

represents the expected change in

y

for a one-unit change in

x_j

, holding all other variables constant.

2.4 ANOVA Decomposition

SS_{total} = SS_{reg} + SS_{res}

R^2 = \frac{SS_{reg}}{SS_{total}}

= proportion of variance explained by the model.

✅ Practice Questions

Q1: For a model with

y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \varepsilon

, how do you interpret

\hat{\beta}_1

Solution

$\hat{\beta}_1$ is the expected change in $y$ for a one-unit increase in $x_1$ , holding $x_2$ constant. It's a partial regression coefficient controlling for the effect of $x_2$ . Join Discord PreviousRegression ApplicationsNextGoodness of Fit

What this note is really saying

Multiple Linear Regression

2.1 Model Formulation

2.2 Normal Equations

2.3 Interpretation of Coefficients

2.4 ANOVA Decomposition

✅ Practice Questions

Intelligence Hub