Model Deployment & Serving

🎯 Learning Objectives

• Deploy ML models as REST APIs using FastAPI
• Implement batch inference pipelines
• Set up A/B testing infrastructure for model comparison
• Monitor deployed models for drift and performance

📖 Core Content

4.1 Deployment Strategies

4.2 FastAPI Model Serving

python
Run Inline
# runnable
# from fastapi import FastAPI
# from pydantic import BaseModel
# import joblib
# import numpy as np
#
# app = FastAPI()
# model = joblib.load('model.pkl')
#
# class PredictionRequest(BaseModel):
#     features: list[float]
#
# @app.post("/predict")
# async def predict(request: PredictionRequest):
#     features = np.array(request.features).reshape(1, -1)
#     prediction = model.predict(features)[0]
#     probability = model.predict_proba(features)[0].tolist()
#     return {
#         "prediction": int(prediction),
#         "probability": probability
#     }
#
# if __name__ == "__main__":
#     import uvicorn
#     uvicorn.run(app, host="0.0.0.0", port=8000)

4.3 A/B Testing Models in Production

4.4 Model Monitoring

• Latency: p50, p95, p99 response times
• Traffic: Requests per second
• Errors: 4xx/5xx rates
• Prediction distribution: Drift detection
• Feature values: Data drift detection
• Business metric: Conversion rate, revenue per prediction

📝 Practice Questions

Q1: What's the difference between batch and real-time deployment?

Batch: Processes all data at scheduled intervals (nightly). Lower cost, good for non-time-sensitive (recommendations). Real-time: Serves predictions on demand (sub-second). Higher cost, good for time-sensitive (fraud). Many systems use both: batch for daily recommendations, real-time for click-through predictions. Q2: How do you handle model versioning in production?

Each model version gets a unique ID (or Git hash). Deploy with shadow mode (serve both versions, only return A's response). Track performance metrics per version. Use blue-green deployment: maintain two identical environments, switch traffic atomically. The Model Registry (MLflow) tracks which version is deployed where. Q3: What is model drift and how do you detect it?

Model performance degrades over time due to data or concept drift. Detection: (1) monitor prediction distribution (KS statistic), (2) track accuracy when ground truth arrives (delayed feedback), (3) alert when key feature distributions shift. Retrain when drift exceeds threshold. Some models need daily retraining (e-commerce), others monthly (credit scoring). Join Discord PreviousEDA & VisualizationNextHyperparameter Tuning