Model Validation and Iteration

An intent scoring model is a prediction engine, and predictions must be validated against outcomes. Without validation, you are making targeting decisions based on a model that might be wrong — and you would never know. Model validation compares the model's predictions (this account is in-market) against actual outcomes (did the account convert?) and uses the gap to improve the model.

1. Conversion Rate by Score Band Every quarter, calculate the conversion rate at each score tier. Tier 1 should convert at the highest rate, tier 2 at a moderate rate, and tier 3 at the lowest rate. If tier 2 converts higher than tier 1, the weighting is wrong. If all tiers convert at similar rates, the model is not differentiating — it needs more predictive signals or better weighting.
2. Signal Contribution Analysis Analyze which signals contributed most to accounts that actually converted. Did first-party behavioral data predict better than third-party intent? Were hiring signals more predictive than technology signals? The contribution analysis tells you where to invest in better data and where the model is relying on weak signals.
3. A/B Model Testing When making significant changes to the model — adding new signal sources, changing weights, or adjusting decay functions — run the new model alongside the old one for 90 days. Compare conversion rates between accounts prioritized by each model. The better-performing model becomes the new production model.