Mechanical Sympathy¶

Advanced (low-level performance & concurrency) · prereq None strictly required — foundational CS/performance material. Pairs naturally with 91 (Aeron), which assumes this content (single-writer principle, cache-friendly layouts, avoiding false sharing) — recommended before or alongside it. · ~1 day

Every module so far has treated performance as infrastructure's problem to fix — more pods, a bigger instance, a cache in front of the database. Mechanical sympathy is what's underneath those fixes: the memory-hierarchy and concurrency reasoning that explains why a cache-friendly scan can beat a smarter algorithm, why two threads touching the same memory need a lock or a single owner, and why two unrelated variables sharing a cache line can silently throttle each other. This module is the foundation module 91 (Aeron) assumes — its single-writer principle, cache-friendly layouts, and false-sharing avoidance stop being trivia and become concrete design decisions there.

TL;DR — in 30 seconds:

Only two general-purpose fixes exist for shared mutable state: a lock (coordination cost that climbs under contention) or single-writer/message-passing (no lock needed, by construction).
A cache miss out to RAM costs roughly 500 instructions' worth of time — which is why a cache-friendly O(n) scan regularly beats a cache-hostile O(log n) lookup in practice.
False sharing throttles two logically-independent variables that share a cache line (fix: pad or align); fences and atomics aren't free either — single-writer often lets you skip both.

Learning objectives¶

By the end, the junior can:

Explain what a data race is and the two fundamental fixes — mutual exclusion (lock) vs. message passing / single-writer — and when to reach for each.
State the memory hierarchy from registers to RAM with approximate latencies, and explain why one main-memory cache miss costs roughly 500 instructions of work.
Apply locality (temporal, spatial, sequential/striding) to predict whether a data structure or access pattern will be cache-friendly, and choose contiguous layouts over pointer-chasing ones accordingly.
Diagnose false sharing and explain how cache coherence (MESI) turns logically-independent writes into a real bottleneck — and how to fix it.
Reason about memory ordering: why reordering happens, what a fence/barrier actually buys you, and why the single-writer principle often lets you avoid both locks and fences.

Do it with Claude

Pair, Do and Prove run in Claude Code, with the onboarding-tutor skill. Click a button to copy the exact prompt, then paste it into Claude Code in this repo.

1. Learn · acquire the concept¶

No external course — read this brief, then skim the linked docs.

The mental model has five parts. Read them top to bottom — each one sets up the next.

Concurrency fundamentals¶

A data race is two-plus threads touching the same memory concurrently with no ordering between them, at least one access a write.

There are exactly two general-purpose fixes:

mutual exclusion (a lock around the write); or
message passing — funnel all mutation through a single owning thread, the single-writer pattern.

Runtime race detectors (ThreadSanitizer-style tooling) instrument accesses to catch what code review misses.

Memory hierarchy & latency¶

The memory hierarchy runs Registers → L1 → L2 → L3 → RAM (→ disk), each level roughly 3–5× slower than the one above it (registers effectively free):

Level	Approx latency	Note
Registers	~1 ns	effectively free
L1	~1 ns
L2	~4 ns
L3	~15 ns
RAM	~65–100 ns	one full miss ≈ 500 instructions' worth of time

flowchart TB
  R["Registers · ~1 ns · effectively free"] --> L1["L1 · ~1 ns"]
  L1 --> L2["L2 · ~4 ns"]
  L2 --> L3["L3 · ~15 ns"]
  L3 --> RAM["RAM · ~65–100 ns · one full miss ≈ 500 instructions"]
  RAM --> DISK["Disk · next level down"]

CPU speed has outpaced memory-latency improvements for decades — bandwidth kept climbing, latency barely moved — so the entire cache hierarchy exists to hide that gap.

That single miss — on the order of 500 instructions' worth of time — is why a cache-friendly algorithm regularly beats a lower-Big-O but cache-hostile one in practice.

Cache behavior & access patterns¶

Memory moves in 64-byte cache lines, not single values, so three kinds of locality all pay off:

spatial locality — nearby data;
sequential/striding locality — predictable steps the hardware prefetcher can preload;
temporal locality — reuse.

A random walk across a large heap uses roughly 1 of every 8 words in each fetched line, defeats the prefetcher, and thrashes the TLB. That's why linear access can beat random access by ~30× on otherwise identical work — and why contiguous layouts (arrays, open-addressing) beat pointer-chasing ones (linked lists, chained hash maps).

Cache coherence (MESI: Modified / Exclusive / Shared / Invalid) keeps every core's view of shared memory consistent, at the cost of inter-core snoop traffic.

stateDiagram-v2
  [*] --> Invalid
  Invalid --> Exclusive: this core reads,<br/>no one else has it
  Invalid --> Shared: this core reads,<br/>others already hold it
  Exclusive --> Modified: this core writes
  Shared --> Modified: this core writes<br/>(invalidates other copies)
  Modified --> Shared: another core reads<br/>(this core's copy written back)
  Shared --> Invalid: another core writes<br/>(snoop invalidates this copy)
  Modified --> Invalid: another core writes<br/>(snoop invalidates this copy)

False sharing is the classic trap: two cores writing logically independent variables that happen to land on the same 64-byte cache line still ping-pong that line back and forth — fixed by padding/aligning so contended fields sit on separate lines.

flowchart TB
  LINE["One 64-byte cache line<br/>holds var A and var B — logically independent<br/>ping-pongs between the two cores"]
  C0["Core 0<br/>writes var A"] -->|invalidate + re-fetch| LINE
  C1["Core 1<br/>writes var B"] -->|invalidate + re-fetch| LINE

Reads scale well (every core can hold a shared read-only copy); shared writes are what's expensive — which is why the single-writer principle removes the coordination cost outright rather than optimizing it.

flowchart TB
  subgraph CONTEND["Shared write via lock / CAS — coordination cost, worsens under contention"]
    T1["Thread 1"] --> LK["lock / CAS<br/>on one shared var"]
    T2["Thread 2"] --> LK
    T3["Thread 3"] --> LK
    LK --> SV["shared state"]
  end
  subgraph SINGLE["Single-writer — no coordination needed"]
    P1["Thread 1"] -->|message| OW["Owner thread<br/>(only writer)"]
    P2["Thread 2"] -->|message| OW
    P3["Thread 3"] -->|message| OW
    OW --> OS["its own state"]
  end

Both locks and CAS degrade sharply under contention — a locked counter benchmark went from ~300 ms to ~118,000 ms with just one more contending thread. On multi-socket boxes, NUMA adds a further remote-memory penalty (~20 ns per hop), so keep data local to the core using it.

Memory ordering, atomics & fences¶

CPUs and compilers reorder reads/writes for speed (store buffers, out-of-order execution), so another core can observe your operations in a different order than you wrote them.

Atomics (CAS, atomic add) are indivisible and safe under interleaving;
fences/barriers explicitly constrain reordering;
acquire/release semantics order a handoff between threads (acquire stops later ops moving earlier, release stops earlier ops moving later).

None of this is free — even single-threaded, adding a barrier took the same benchmark counter from ~300 ms to ~4,700 ms.

The single-writer principle earns its keep here too: if you're never racing to overwrite what another core just wrote, you often don't need fences at all.

Read next: Martin Thompson's Mechanical Sympathy blog — the source of the single-writer principle and most of the benchmark numbers in this module's deck. Skim "Latency Numbers Every Programmer Should Know" for the canonical hierarchy figures. Retain deck (beat 5): SE Onboarding::90 Mechanical Sympathy.

Done when: you can state, from memory (brief closed) — (1) what makes something a data race and the two general-purpose fixes, (2) the memory-hierarchy latency ladder and why a RAM miss dominates cache-friendly performance, (3) which locality pattern makes an access cache-friendly and why pointer-chasing defeats it, (4) what false sharing is and its fix, and (5) why atomics/fences aren't free even with zero contention.

2. Pair · passive → active¶

Drive the onboarding-tutor:

"Explain why message-passing/single-writer avoids the bugs locking does, and what it costs you in return."
"Walk me through why a cache miss out to RAM is so expensive — connect it to why Big-O isn't the whole performance story."
"Quiz me on temporal vs. spatial vs. sequential locality, one concrete data-structure example for each."
"Narrate false sharing like you're describing the diagram out loud: two threads, two independent variables, one cache line — what actually happens on each write?"
"Walk me through why a fence/barrier isn't free, and when the single-writer principle lets me skip them entirely."

Done when: you can explain, unprompted, why "the fastest concurrency primitive is the one you don't need" (single-writer) recurs across every theme in this module.

3. Do · produce an artifact¶

Exercise — reason about a shared counter (on paper, or benchmark it if you have a runtime handy):

Sketch two threads incrementing a shared counter three ways: (a) unsynchronized, (b) behind a lock, (c) single-writer via message passing. For each, mark exactly where the data race would occur — or explain why it can't.
Take a struct with two int64 fields written by two different threads. Draw its memory layout, mark the 64-byte cache-line boundary, and state whether this is a false-sharing setup — then show the padding fix.
Using the latency ladder (L1 ~1 ns … RAM ~65–100 ns), estimate the relative wall-clock cost of walking a 10,000-node linked list scattered across the heap vs. an equivalent array — state your hit/miss assumption for each.

Optional (deeper): if you have Java/Rust/Go handy, actually benchmark a contended lock vs. an uncontended single-writer counter and see if your numbers land near the deck's ~300 ms vs. ~118,000 ms.

Done when: your three counter sketches correctly identify where the race is (or isn't), your false-sharing diagram shows the correct padding fix, and your latency estimate uses the right order-of-magnitude numbers.

Common gotchas

Reaching for a lock by default. Fix: check whether the state has exactly one natural owner first — single-writer avoids the coordination cost outright rather than optimizing it, and only fits poorly when several threads genuinely need to read-modify-write the same value right now.
Judging performance by Big-O alone. Fix: weigh cache-friendliness too — a cache-hostile O(log n) pointer-chasing lookup regularly loses to a cache-friendly O(n) contiguous scan, because each cache miss costs on the order of 500 instructions.
Padding every struct field "just in case" of false sharing. Fix: only pad the handful of hot, independently-written fields you've actually identified as sharing a line — padding everything wastes memory on fields that were never contended.
Adding fences or atomics defensively, everywhere. Fix: a fence isn't free even with zero contention (one benchmark went from ~300 ms to ~4,700 ms from a single barrier) — reach for single-writer or a narrow acquire/release handoff instead of a blanket fence.
Trusting that a program "worked in testing" proves there's no data race. Fix: a race is a property of the program, not of one run — use a race detector (ThreadSanitizer-style tooling), not repeated manual testing, to actually catch it.

4. Prove · understanding gate¶

Mandatory. Pass bar in the Socratic gate.

Data races & fixes: What exactly makes something a data race, and what are the two fundamental ways to eliminate one? When would you reach for message-passing/single-writer over a lock?
Latency & locality: Why does a single main-memory cache miss cost roughly 500 instructions of work, and what does that imply about choosing a cache-friendly O(n) approach over a cache-hostile O(log n) one?
False sharing: Two threads write two logically-independent variables and throughput craters. Diagnose why, describe the fix, and explain why this isn't a data race in the correctness sense.
Ordering cost: Why aren't atomics and fences free, even with zero contention? When does the single-writer principle let you avoid needing them at all?
Provenance: For your false-sharing diagram or your linked-list-vs-array latency estimate, which numbers or cache-line boundaries did the tutor supply? Take one and show how you sanity-checked it against the latency ladder or the 64-byte cache-line rule — "the agent said ~30×" is not verification.

Pass bar: you can defend each fix with the mechanism underneath it — why coherence causes false sharing, why a miss costs ~500 instructions, why ordering has a real cost — not just name the fix — and say how you checked any figure the tutor gave you. "Pad the struct", "add a lock", or "the agent's number looked right" without the why = back to Pair.

5. Retain · fight forgetting¶

Study the module deck SE Onboarding::90 Mechanical Sympathy — its theme subdecks let you drill one area at a time:

1 - Concurrency Fundamentals · 2 - Memory Hierarchy & Latency · 3 - Cache Behavior & Access Patterns · 4 - Multicore: Coherence, False Sharing, Contention · 5 - Memory Ordering, Atomics & Fences

The deck syncs to your Anki automatically on pull (anki-sync). Done when you can recall, from the deck: the two fixes for a data race, the memory-hierarchy latency ladder, why locality drives cache-friendliness, what false sharing is and its fix, and why memory ordering isn't free.

Check yourself — why does single-writer avoid a lock's contention cost instead of just being a faster lock?

Because there's structurally only one writer, there's nothing to coordinate at all — no thread waits on another and no queue builds up, unlike a lock whose cost climbs as more threads contend for it.

Check yourself — two threads write two logically independent counters that happen to land on the same cache line. Is this a data race?

No — it's false sharing, a performance bug, not a correctness bug. There's no unordered read/write on the same logical variable; MESI just can't tell the two offsets apart, so it invalidates the whole line on every write. The fix is padding/alignment, not a lock.

Check yourself — why does a cache-friendly O(n) array scan sometimes beat a cache-hostile O(log n) lookup?

Because each cache miss out to RAM costs on the order of 500 instructions' worth of time — a pointer-chasing structure pays that miss on nearly every hop, while a contiguous array's sequential access lets the prefetcher and cache lines absorb most of the cost, so the real constant factor can swamp the asymptotic advantage.

Check yourself — why does adding a fence cost something even in a single-threaded benchmark with zero contention?

Because a fence blocks the store buffer and out-of-order execution's normal reordering at that point regardless of whether anything was ever racing to observe it — one benchmark went from ~300 ms to ~4,700 ms from a single barrier, with no other thread even touching the data.

Key takeaways¶

Only two general-purpose fixes exist for a data race: a lock (mutual exclusion) or single-writer/ message-passing — and a lock's coordination cost climbs sharply under contention, while single-writer has nothing to coordinate at all.
A single cache miss out to RAM costs roughly 500 instructions' worth of time, which is why Big-O isn't the whole performance story — a cache-friendly O(n) scan regularly beats a cache-hostile O(log n) lookup.
Locality predicts cache-friendliness: spatial, sequential/striding, and temporal locality all pay off, while pointer-chasing structures defeat the hardware prefetcher and thrash the TLB.
False sharing is a performance bug, not a correctness bug — two logically-independent variables on the same 64-byte cache line still ping-pong under MESI coherence; the fix is padding/alignment, not a lock.
Atomics and fences aren't free even with zero contention — one benchmark went from ~300 ms to ~4,700 ms from a single barrier — so reach for them narrowly (acquire/release handoffs) rather than defensively.
The single-writer principle removes both lock contention and ordering cost at the source rather than optimizing either after the fact — "the fastest concurrency primitive is the one you don't need."

Go deeper — curated resources¶

Hand-picked external material to take this topic further — the best books, courses, talks, and writing. Cost is tagged Free, Free online (full text/course free on the web), or Paid.

Blog · Mechanical Sympathy — Martin Thompson. The blog this module is named after — the source of the single-writer principle and much of the deck's benchmark data. Free.
Talk · Mythbusting Modern Hardware to Gain 'Mechanical Sympathy' — Martin Thompson, GOTO 2012. The same author walking through how modern CPUs and memory really behave — a video companion to this module. Free.
Book · Systems Performance (2nd ed.) — Brendan Gregg. The performance bible — methodology and tooling from application down to hardware. Paid.
Repo · LMAX Disruptor — LMAX Exchange. The lock-free ring-buffer library that popularized mechanical sympathy; read the code and its linked design docs. Free.

Gate log (mentor fills on pass)¶

Date passed:
Passed by: / checked by:
Weak spots noticed (feeds system improvement):