CDX-301d · Module 3

Warm Pools & Production Tuning

3 min read

A warm pool is a set of pre-booted, pre-configured microVMs held in a suspended state, ready to accept tasks immediately. Instead of booting a new VM for each task (cold start: 15-60 seconds), the system resumes a suspended VM from the pool (warm start: 1-2 seconds). The pool maintains a configurable number of VMs per configuration — each with the correct base image, dependencies, and repository state. When a task completes and the VM is destroyed, the pool provisions a replacement in the background to maintain the target count.

Warm pool sizing is a cost-availability tradeoff. Each warm VM in the pool consumes resources (memory, snapshot storage) even when idle. A pool of 10 VMs means 10 VMs worth of memory reserved at all times. Too small a pool means tasks queue behind cold starts during burst periods. Too large a pool wastes budget on idle VMs. The optimal size depends on your task arrival rate: measure the peak concurrent tasks over a week, add 20% headroom, and set that as your pool size. Scale down during off-hours with scheduled pool size adjustments.

# Warm pool configuration

pool:
  name: "main-pool"
  image: "custom-base:v3"
  repository: "github.com/org/repo"
  branch: "main"
  target_size: 5              # VMs ready at all times
  max_size: 20                # Burst capacity
  idle_timeout: 30m           # Destroy idle VMs after 30 min
  refresh_interval: 6h        # Rebuild snapshots every 6 hours

# Schedule-based scaling
schedule:
  weekday_business:           # Mon-Fri 8am-6pm
    target_size: 10
  weekday_off_hours:          # Mon-Fri 6pm-8am
    target_size: 3
  weekend:
    target_size: 1

# Pool metrics
- Cold start rate:  % of tasks that hit a cold start
- Queue depth:      Tasks waiting for a VM
- Utilization:      Active VMs / pool size
- Cost per idle VM: $/hour for reserved resources

Do This

  • Size warm pools based on measured peak concurrent task counts, not guesses
  • Use schedule-based scaling to reduce pool size during off-hours and weekends
  • Set refresh intervals that match your dependency update cadence
  • Monitor cold start rate — if it exceeds 20%, increase pool size or refresh more frequently

Avoid This

  • Set a large warm pool and forget it — idle VMs cost money around the clock
  • Skip refresh intervals — stale snapshots cause subtle dependency version mismatches
  • Use warm pools for one-off projects with irregular task submission patterns
  • Ignore queue depth metrics — tasks waiting for VMs are developers waiting for results