Availability Patterns

"Available" means a user's request gets served even while something underneath has failed. These three patterns are how a system keeps serving traffic through individual failures instead of going down with them.

TL;DR — in 30 seconds:

• A health check is a cheap endpoint (e.g. /healthz) that reports whether an instance can currently serve traffic — the mechanism every other availability pattern depends on to know what's actually healthy.
• A load balancer spreads requests across multiple healthy instances, using health checks to stop sending traffic to one that's failing, and keeps serving from the rest.
• Multi-AZ (multiple Availability Zones — physically separate data centers within a cloud region) means a whole data center going down doesn't take the service down with it.
• None of this makes any single instance more reliable — it accepts that instances fail and arranges for the system as a whole to keep serving anyway.

1. Health checks: the signal everything else depends on

flowchart LR
    LB["Load balancer"] -->|"GET /healthz<br/>every N seconds"| I1["Instance A"]
    LB -->|"GET /healthz"| I2["Instance B"]
    LB -->|"GET /healthz"| I3["Instance C (failing)"]
    LB -->|"routes user traffic"| I1
    LB -->|"routes user traffic"| I2
    I3 -.->|"removed from rotation"| LB

• A health check endpoint answers one question: can this instance serve a request right now? — not "is the process running" (a hung process can still answer 200 OK on a bad check, or a healthy process can be starved of a dependency it needs).
• A liveness check (is the process alive at all) and a readiness check (is it ready to take traffic — e.g. has it finished startup, can it reach its database) answer different questions; conflating them means an instance that's alive but not ready still gets traffic sent to it.
• Every pattern below — load balancing, autoscaling, orchestration — depends on an honest health check to know which instances are actually usable.

Common gotchas

• A health check that always returns 200 OK. Fix: it must actually verify the thing that matters (can it reach its database, is a critical dependency up) — a check that can't fail can't protect anything.
• Confusing liveness with readiness. Fix: an instance can be alive (don't restart it) but not ready (don't send it traffic yet) — e.g. during startup or while reconnecting to a dependency.

2. Load balancing: spreading traffic across healthy instances

Task	What the load balancer does
Normal operation	Distributes requests across all instances reporting healthy
An instance fails its health check	Stops routing new requests to it until it reports healthy again
Traffic grows	Works with autoscaling (see page 3) to spread load across newly added instances

• The load balancer is itself a potential single point of failure, so production setups run it as a managed, highly-available service (e.g. a cloud load balancer) rather than a single box.

3. Multi-AZ: surviving a whole data center failure

flowchart TB
    subgraph Region
        subgraph "AZ 1"
            A1["Instance"]
        end
        subgraph "AZ 2"
            A2["Instance"]
        end
        subgraph "AZ 3"
            A3["Instance"]
        end
    end
    LB["Load balancer"] --> A1
    LB --> A2
    LB --> A3

• An Availability Zone (AZ) is one or more physically separate data centers within a cloud region, each with independent power, cooling, and networking — one AZ failing (power outage, network incident) doesn't take down another.
• Running instances across multiple AZs, behind the same load balancer, means the loss of one entire AZ still leaves the service running from the others.
• This is a deployment-topology decision, not a code change — it's where instances run, combined with the load balancer already knowing to route around unhealthy ones.

Common gotchas

• Running "highly available" instances all in one AZ. Fix: multiple instances in a single AZ survive one instance failing, not the AZ itself failing — spread across AZs for real availability.
• Assuming multi-AZ means multi-region. Fix: AZs are within one region (low-latency, often free inter-AZ traffic in some setups); surviving a whole region outage is a separate, bigger commitment (multi-region), not covered by multi-AZ alone.

Check yourself — a service runs three instances, all in the same Availability Zone, behind a load balancer. What failure does this survive, and what does it not survive?

It survives a single instance failing — the load balancer routes around it using health checks. It does NOT survive the AZ itself failing (e.g. a data-center-level power or network incident), since all three instances go down together. Surviving that requires spreading instances across multiple AZs.

Done when: you can explain what a health check actually verifies, how a load balancer uses it, and why multi-AZ protects against a failure that multiple-instances-in-one-AZ does not.