Eventual Failure

Problem

A distributed system contains: servers networks disks databases cloud services external APIs The question isn't: “Can something fail?” Given enough time, something will fail.

Solution — Design for Failure

Treat failure as a normal operating condition. Use the mechanisms we've already derived: timeouts retries idempotency redundancy queues health checks backups observability graceful degradation Architecture then asks not merely “How does this work?”, but: “What happens when each assumption stops being true?”
Core idea: resilient systems are designed around the expectation that components will eventually fail.