DS-301e · Module 1

Matching Latency to Decision Cadence

3 min read

Real-time is not a speed — it is a requirement. A fraud detection dashboard needs sub-second latency because the decision (block the transaction) has a sub-second window. A revenue dashboard needs daily latency because the decision (adjust the forecast) happens weekly. An HR dashboard needs weekly latency because the decisions (hiring, staffing) happen monthly. Over-engineering latency wastes infrastructure cost. Under-engineering it misses decision windows. The design question is not "how fast can we make it?" It is "how fast does the decision require?" Map every dashboard to the decision it supports. The decision cadence determines the latency requirement.

  1. Identify the Decision What decision does this dashboard support? Pipeline adjustment, campaign optimization, incident response, resource allocation. The decision determines everything else.
  2. Determine the Decision Cadence How often is this decision made? Real-time (seconds), intraday (hours), daily, weekly, monthly. The cadence is the maximum acceptable data latency.
  3. Design the Pipeline Match the data pipeline architecture to the latency requirement. Streaming for real-time. Micro-batch for intraday. Batch for daily and beyond. Each architecture has different cost, complexity, and reliability characteristics.