EI-301g · Module 2

Source Effectiveness Review

3 min read

The quarterly retrospective includes a review of source performance: which sources produced the most actionable signals, which sources produced noise that consumed time without yielding intelligence, and which sources failed to detect signals that materialized. Source effectiveness is measured on three dimensions: signal yield (percentage of signals from this source that made it into briefings), accuracy (percentage of signals from this source that were subsequently verified), and exclusivity (how many of this source's signals were not available from any other source in your network).

Do This

  • Rank sources by signal yield per unit of monitoring time invested — this reveals which sources are worth the effort
  • Identify exclusive sources — sources that provide signals no other source catches are the most strategically valuable and should be protected
  • Flag zero-yield sources for the quarter — a source that produced no actionable signals in 90 days may need to be replaced or repositioned in your rotation

Avoid This

  • Evaluate sources only by volume — a source that produces 50 signals, 2 of which are actionable, is less effective than one that produces 5 signals, 4 of which are actionable
  • Drop low-yield sources without checking coverage impact — the source may be the only coverage for a critical ecosystem segment
  • Keep high-cost, low-yield sources out of organizational inertia — every source should justify its cost with demonstrable signal value

The most valuable finding from the source effectiveness review is often the identification of signals that were missed — ecosystem events that materialized without prior detection. For each missed signal, ask: which source should have caught it? Was the source in our network but we missed the signal in processing? Was the source not in our network, revealing a coverage gap? Each missed signal is a specific diagnosis that produces a specific improvement.