Retraining and Versioning

Retraining is rebuilding the model on updated data that includes the most recent closed deals. Versioning is the practice of tracking which model produced which scores, so you can compare performance across model versions and roll back if a new model performs worse than the previous one. Together, retraining and versioning are the lifecycle management practices that keep the propensity model accurate and trustworthy.

1. Rolling Training Window Retrain on the most recent 12-18 months of closed deals. Too short a window (6 months) and the model overreacts to recent anomalies. Too long a window (3 years) and the model includes patterns from a business reality that no longer exists. The rolling window balances recency with stability.
2. Shadow Scoring Before deploying a new model version, run it in shadow mode — it scores deals but the scores are not visible to users. Compare the new model's predictions against the current model's predictions and actual outcomes. Deploy only if the new model outperforms or matches the current one.
3. Version Registry Maintain a registry of every model version: training data range, features used, performance metrics (AUC, calibration error), deployment date, and retirement date. The registry is the audit trail that answers "what model produced this score?" and "how did this model perform over its lifetime?"