EI-301g · Module 3

Benchmarking Your Intelligence Practice

3 min read

Benchmarking compares your intelligence practice's performance against internal historical performance and, where possible, against external standards. Internal benchmarks: is your calibration improving quarter over quarter? Is your signal-to-briefing cycle time decreasing? Is your recommendation action rate increasing? External benchmarks: how does your prediction accuracy compare to published forecasters in your domain? How does your source coverage compare to recognized intelligence frameworks? Benchmarking provides the context that raw metrics lack.

Do This

  • Track internal benchmarks quarterly — the trend line is more important than any single quarter's number
  • Compare against your own historical performance first — self-improvement is the most relevant and controllable benchmark
  • Publish your benchmark results to stakeholders — demonstrated improvement justifies continued investment in the intelligence function

Avoid This

  • Benchmark against practices with fundamentally different scope or resources — a 20-person intelligence team's output is not a fair benchmark for a single analyst
  • Use benchmarks to justify the current state without improvement targets — "we are better than average" is not a target; "we will improve calibration by 10% this year" is
  • Abandon metrics that show poor performance — the whole point of measurement is to identify and address weaknesses

The most powerful internal benchmark is the Year-over-Year Improvement Rate. Calculate your key metrics (accuracy, calibration, cycle time, action rate) at the end of each year. Compare to the prior year. A positive improvement rate on all four metrics demonstrates that the retrospective process is working. A stagnant or declining metric on any dimension identifies where the improvement process needs attention. The Year-over-Year rate is the single number that tells you whether your intelligence practice is getting better.