Query Caching & Invalidation

AI conversations generate repetitive data reads. A model working through a customer analysis might read the same customer record five times as it refines its response. Without caching, each read executes a database query — consuming a pool connection, adding latency, and creating unnecessary load on the database. A query cache at the MCP server level stores the result of the first read and serves subsequent reads from memory until the cache entry expires or is invalidated.

Cache key design determines correctness. The simplest key is the full resource URI — db://customers/abc-123 maps to one cache entry. But template resources with pagination add complexity: db://orders?page=1&limit=50 and db://orders?page=2&limit=50 are different cache entries. The cache must preserve parameter ordering or normalize the key (sort query parameters alphabetically) to avoid duplicate entries for the same data. Cache keys should also include the MCP user identity if access control returns different data for different users.

interface CacheEntry<T> {
  data: T;
  cachedAt: number;
  ttlMs: number;
}

class QueryCache {
  private store = new Map<string, CacheEntry<string>>();
  private maxEntries = 1000;

  /** Normalize URI to consistent cache key */
  private normalizeKey(uri: string): string {
    const url = new URL(uri);
    url.searchParams.sort(); // Consistent param ordering
    return url.toString();
  }

  get(uri: string): string | null {
    const key = this.normalizeKey(uri);
    const entry = this.store.get(key);
    if (!entry) return null;
    if (Date.now() - entry.cachedAt > entry.ttlMs) {
      this.store.delete(key);
      return null;
    }
    return entry.data;
  }

  set(uri: string, data: string, ttlMs: number): void {
    // LRU eviction when at capacity
    if (this.store.size >= this.maxEntries) {
      const oldest = this.store.keys().next().value;
      if (oldest) this.store.delete(oldest);
    }
    const key = this.normalizeKey(uri);
    this.store.set(key, { data, cachedAt: Date.now(), ttlMs });
  }

  /** Invalidate by exact URI or prefix */
  invalidate(uriOrPrefix: string): number {
    let count = 0;
    for (const key of this.store.keys()) {
      if (key === uriOrPrefix || key.startsWith(uriOrPrefix)) {
        this.store.delete(key);
        count++;
      }
    }
    return count;
  }
}

const cache = new QueryCache();

// TTL guidelines by data volatility
// Static reference data: 3600s (1 hour)
// Transactional data:     60s  (1 minute)
// Real-time metrics:       0s  (no cache)