DS-301i · Module 1

Recommendation Ranking

3 min read

A decision engine that recommends ten actions is a decision engine that recommends nothing. The human cannot process ten simultaneous recommendations. Ranking reduces the recommendations to the top one to three, ordered by expected impact. The ranking function considers: urgency (how time-sensitive is the action?), impact (what is the expected outcome in dollar terms?), effort (how much time does the action require?), and confidence (how sure is the model that this action will produce the expected outcome?). The composite score: (impact × confidence × urgency) ÷ effort. This produces an ROI-per-hour metric that ranks recommendations by their expected return on the decision-maker's time.

Do This

  • Rank recommendations by expected ROI per hour of effort — this maximizes the decision-maker's time value
  • Present the top one to three recommendations — more than three creates decision paralysis
  • Show the rationale for each recommendation — "this account's churn probability is 72% based on declining usage"

Avoid This

  • Present all possible actions and let the user sort through them — curation is the engine's job
  • Rank by model confidence alone without considering impact — a high-confidence recommendation for a low-value action wastes time
  • Hide the rationale — decision-makers who do not understand why an action is recommended do not trust it