Variable Selection

76 words

1 min read

Visual companion

Python

Type and operator map

Python Week 1: the first filter for runtime behavior

Open gallery Related surface

View

Revision summary

What this note is really saying

Short form

# Variable Selection ## 7.1 Selection Criteria Criterion Formula Lower is $C_p$ (Mallows) $C_p = \frac{SS_{res}(p)}{ \hat{\sigma}^2} - n + 2p$ Better (close to $p$ ) AIC $-2\ln(L) + 2p$ Better BIC $-2\ln(L) + p\ln(n)$ Better (penalizes complexity more) Adjusted $R^2$ $1 - \frac{SS_{res}/(n-p)}{SS_{tot}/(n-1)}$ Highe...

Variable Selection

7.1 Selection Criteria

Criterion	Formula	Lower is
$C_p$ (Mallows)	$C_p = \frac{SS_{res}(p)}{ \hat{\sigma}^2} - n + 2p$	Better (close to $p$ )
AIC	$-2\ln(L) + 2p$	Better
BIC	$-2\ln(L) + p\ln(n)$	Better (penalizes complexity more)
Adjusted $R^2$	$1 - \frac{SS_{res}/(n-p)}{SS_{tot}/(n-1)}$	Higher is better

7.2 Selection Methods

Forward selection: Start empty, add variables one by one
Backward elimination: Start full, remove least significant
Stepwise: Both forward and backward Join Discord PreviousTransformationsNextGeneralized Linear Models