FieldManual

Operator field manual: playbooks, runbooks, checklists, and postmortems.

Queue Backlog Burn

Severity: high Last verified: 2026-03-01
queueworkers message-delayretry-storm worker-scalepoison-isolation

Procedure

  1. Measure backlog age and retry amplification.
  2. Quarantine poison messages to dead-letter immediately.
  3. Scale workers based on oldest message age target.

Post-checks

  • Oldest message age under 60 seconds
  • Retry ratio under 1.2x