DS-301h · Module 1

Statistical Anomaly Detection

3 min read

Statistical detection is the foundation. The principle: establish what "normal" looks like, then flag what deviates. Z-score detection: calculate the mean and standard deviation of the metric. Any value more than two or three standard deviations from the mean is an anomaly. Simple, fast, and effective for stationary data. Seasonal decomposition: separate the metric into trend, seasonality, and residual. Apply z-score detection to the residual — the component that remains after accounting for expected patterns. This catches anomalies that would be hidden by seasonal fluctuations. Moving average bands: calculate the rolling average and standard deviation. Values outside the bands are anomalies. This adapts to trend changes better than the global z-score.

  1. Start with Z-Score For any metric without strong seasonality, z-score detection with a three-sigma threshold catches 99.7% of normal variation. Anything beyond three sigma warrants investigation.
  2. Add Seasonal Decomposition For metrics with weekly or monthly patterns, decompose first. Revenue that drops 30% on Saturday is not an anomaly — it is the weekend. Decomposition prevents seasonal false positives.
  3. Apply Dynamic Baselines For metrics with trend changes, use a rolling window (30-90 days) for the baseline instead of the full history. The baseline adapts to gradual changes and catches only the sudden deviations.