Cross-Validation: K-Fold, Stratified, Group, and Time Series Splits

🎯 Learning Objectives

• Implement K-fold cross-validation for reliable model evaluation
• Use stratified CV for imbalanced datasets
• Apply GroupKFold when data has natural groupings
• Handle temporal dependencies with TimeSeriesSplit

📖 Core Content

1.1 Why Cross-Validation?

1.2 CV Strategies

python
Run Inline
# runnable
from sklearn.model_selection import (
    KFold, StratifiedKFold, GroupKFold, TimeSeriesSplit,
    cross_val_score
)
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
import numpy as np
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
# Standard K-Fold (5 folds)
kf = KFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(RandomForestClassifier(), X, y, cv=kf, scoring='accuracy')
print(f"K-Fold: {scores.mean():.4f} ± {scores.std():.4f}")
# Stratified K-Fold (preserves class proportions)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores_s = cross_val_score(RandomForestClassifier(), X, y, cv=skf, scoring='accuracy')
# Time Series Split (no shuffling, sequential expanding window)
tscv = TimeSeriesSplit(n_splits=5)
# For time series: train on past, test on future

1.3 Choosing the Right CV

1.4 Why This Matters

2. 📝 Practice Questions

Q1: You build a model to predict student performance. Your dataset has 500 students with 5 assignments each (2500 rows). Which CV method should you use?

GroupKFold with student ID as the group.

Using regular K-Fold would place different assignments from the same student in training AND test sets. Since assignments from the same student are correlated (a good student does well on all assignments), this inflates performance estimates. The model would effectively "see" the student in training and predict their other assignments trivially.

With GroupKFold, all 5 assignments from a student stay together in either train or test — never split across both. This gives a realistic estimate of how the model performs on unseen students. Join Discord PreviousEvaluation MetricsNextEnsemble Methods