DR-301i · Module 3

Pipeline Capacity Planning

3 min read

Pipeline capacity is measured along three dimensions: source throughput (how many sources the collector can process per cycle), analysis throughput (how many findings the analyzer can process per hour), and synthesis throughput (how many intelligence products the synthesizer can produce per delivery cycle). Each dimension has a saturation point where adding more input degrades output quality — the normalizer falls behind, the analysis queue grows, or synthesis quality drops because the AI is processing more findings than it can integrate coherently.

Do This

  • Measure throughput at each pipeline stage independently — the bottleneck may not be where you expect it
  • Monitor queue depths as a leading indicator of capacity problems — growing queues precede degradation
  • Scale horizontally by adding pipeline instances for different domains rather than one pipeline for everything
  • Plan capacity for peak load, not average load — event-triggered collection creates spikes

Avoid This

  • Assume the pipeline will handle double the current load without testing — it will not
  • Scale by making single components faster without addressing the bottleneck — the pipeline is as fast as its slowest component
  • Ignore queue depths until they cause failures — by then, you have hours of backlog to clear