Behavioral Anomaly Detection

Agent anomalies are not just errors. An error produces an error log. An anomaly is a behavioral deviation that does not trigger an error — the agent completes the task successfully but does it differently than expected. It takes twice as long. It produces output that is valid but stylistically different. It makes a decision that is technically correct but inconsistent with its persona. These anomalies are invisible to error monitoring and health checks. They require behavioral baselines and deviation detection.

The behavioral baseline captures the agent's normal operating patterns over a 30-day calibration period: average task duration by task type, typical output length by output type, decision confidence distribution, persona consistency scores, and interaction patterns with other agents. After calibration, the anomaly detector compares current behavior against the baseline and flags deviations that exceed configured thresholds — typically two standard deviations from the mean.

1. 1. Calibrate the Baseline Run the agent in production for 30 days while collecting behavioral metrics: task duration, output length, quality scores, decision patterns. This data becomes the behavioral baseline against which anomalies are detected.
2. 2. Set Detection Thresholds Start at two standard deviations from the baseline mean. This catches significant deviations while minimizing false positives. Tighten to 1.5 standard deviations for high-stakes agents. Loosen to 2.5 for experimental agents.
3. 3. Investigate Before Acting An anomaly is a signal, not a verdict. Investigate before correcting. The anomaly might indicate a genuine problem, a change in input patterns, or an improvement the agent discovered on its own. Kill the improvement and you lose value.