Failure Modes & Mitigations

Consumer lag / stream backlog

  • Cause: traffic spikes or downstream slowness.
  • Mitigations: autoscale consumers, prioritized messages, rate-limiting at ingest.

Partial ordering violations

  • Cause: routing errors or rebalancing.
  • Mitigations: partition by deterministic key, idempotent processors, graceful rebalancing.

Database contention

  • Mitigations: optimistic concurrency, retries with backoff, specialized single-writer components.