Metrics That Matter

I track 14 metrics across the Ryan Consulting agent team. But only four of them matter for operational health. Throughput — how many tasks does the team complete per day? Quality — what percentage of deliverables pass the review gate on the first attempt? Cost — what is the average token spend per completed task? And coordination efficiency — what percentage of dispatched tasks complete successfully without re-dispatch, escalation, or human intervention?

These four metrics form a health quadrant. A team with high throughput but low quality is churning out junk. A team with high quality but low throughput is over-engineering. A team with low cost but low coordination efficiency is cheap but unreliable. The goal is all four in acceptable ranges simultaneously. When one metric improves at the expense of another, you have an optimization problem, not a success.

Coordination efficiency is the metric I report to two decimal places because it is the one metric that captures team health as a whole. It is calculated as: (tasks completed successfully on first dispatch) / (total tasks dispatched) * 100. A CE of 94.7% means that out of every 100 tasks I dispatch, 94.7 complete without re-dispatch, escalation, or failure. The remaining 5.3 required intervention — a re-dispatch with clearer instructions, a cross-functional escalation, or a human decision.

When CE drops, something in the system changed. A new agent with an unclear role definition produces ambiguous output. An API provider changed their response format and a module is not handling it. A prompt template was updated and introduced a regression. CE is the canary in the coal mine. Track it daily. Investigate any drop of more than 2 points.