Multicollinearity
79 words
1 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Multicollinearity ## Detection **VIF (Variance Inflation Factor):** $VIF_j = \frac{1}{1 - R_j^2}$ where $R_j^2$ is from regressing $x_j$ on all other predictors. - VIF $> 5$: Moderate multicollinearity - VIF $> 10$: Severe multicollinearity (needs attention) **Condition number:** $\kappa = \sqrt{\lambda_{\max}/\la...
Multicollinearity
Detection
VIF (Variance Inflation Factor): where is from regressing on all other predictors.
- VIF : Moderate multicollinearity
- VIF : Severe multicollinearity (needs attention) Condition number: . indicates serious multicollinearity.
Remedies
| Method | Description |
|---|---|
| Remove variables | Drop one of the correlated predictors |
| PCA | Replace correlated predictors with principal components |
| Ridge regression | Add penalty |
| Partial least squares | Find latent components that predict both X and y |