Learning Objectives

• Interpret model predictions
• Calculate feature importance
• Use SHAP and LIME

python
Run Inline
import shap
import matplotlib.pyplot as plt
# Train model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
# Feature importance (built-in)
importances = pd.DataFrame({
    'feature': X.columns,
    'importance': model.feature_importances_
}).sort_values('importance', ascending=False)
print(importances.head(10))
# SHAP explanation (for specific prediction)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
# Summary plot
shap.summary_plot(shap_values[1], X_test)
plt.tight_layout()
plt.show()

Q1: Why is model interpretability important?

Trust, debugging, bias detection, regulatory compliance (GDPR), feature engineering guidance, stakeholder communication. Q2: What is SHAP?

SHapley Additive exPlanations. Game-theoretic approach assigning each feature a contribution to prediction. Consistent, local (per prediction) and global (overall). Q3: What is the difference between global and local interpretation?

Global: overall model behavior (feature importance, partial dependence). Local: why a specific prediction was made (SHAP values, LIME). Q4: How are feature importances calculated for tree models?

Total reduction in impurity (Gini/entropy) weighted by samples reaching node, averaged across all trees. Features used higher in tree have more importance. Q5: What is a partial dependence plot?

Shows marginal effect of one/two features on model predictions. Helps understand relationship (linear, monotonic, complex). Check if model behavior matches domain knowledge. Q6: Implementation tip:

python
Run Inline
# PDP example
from sklearn.inspection import PartialDependenceDisplay
PartialDependenceDisplay.from_estimator(model, X_train, ['feature1', 'feature2'])
plt.show()