DR-301d · Module 2

Reliability Degradation Detection

3 min read

Source reliability is not permanent. A publication that was accurate for three years can degrade when its editorial leadership changes, its funding model shifts, or its domain expertise becomes outdated. Degradation detection monitors reliability trends over time and flags sources whose accuracy is declining before the decline becomes catastrophic. The mechanism is a rolling window: instead of calculating lifetime hit rate, calculate hit rate over the most recent 90 days. When the rolling rate drops below the lifetime rate by more than 15 percentage points, the source is flagged for re-evaluation.

Do This

  • Monitor rolling hit rates alongside lifetime rates — recent performance matters more than historical average
  • Flag degradation automatically when the rolling rate diverges from the lifetime rate
  • Investigate flagged sources — is the degradation domain-specific or across all claims?
  • Downgrade sources that show sustained degradation — do not wait for a catastrophic failure

Avoid This

  • Rely on lifetime hit rates alone — they mask recent degradation under years of good performance
  • Wait for a source to publish something dramatically wrong before questioning its reliability
  • Assume degradation is temporary without investigating the cause
  • Maintain a degraded source at its original tier because of historical loyalty