Synchronisation in C

What the language guarantees, and what it does not¶

Since C11 the language has a memory model, and it is worth being exact about what it provides. It defines a data race — two conflicting accesses to the same object, at least one a write, neither atomic, with no happens-before between them — and makes it undefined behaviour. That is the whole guarantee. It does not say a racy read returns a stale value, or a torn one; it says the program has no meaning at all, which is why a race can manifest as a deleted null check three functions away.

Everything else on this page is a library. <threads.h> is a conditional feature that a conforming implementation may omit, and several do, which is why production code still calls POSIX threads directly. So the practical stack is: the C11 memory model underneath, _Atomic and <stdatomic.h> as the only synchronisation the language itself defines, and pthreads for everything with a queue behind it. The examples here use pthreads for that reason.

Back to top

Condition variables: the loop is not optional¶

A condition variable is not a signal, a flag, or a queue. It is a parking place, and it holds no state of its own. The state lives in your program, guarded by a mutex; the condition variable only lets a thread stop consuming CPU until it is worth re-reading that state. Every property that surprises people follows from those two sentences.

/* The shape every correct use of a condition variable has. */
pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t  cond = PTHREAD_COND_INITIALIZER;
int ready = 0;                       /* the predicate's state, guarded by lock */

void *waiter(void *unused)
{
    pthread_mutex_lock(&lock);
    while (!ready)                   /* while, never if -- see below */
        pthread_cond_wait(&cond, &lock);
    consume();                       /* runs with the mutex held */
    pthread_mutex_unlock(&lock);
    return NULL;
}

void *signaller(void *unused)
{
    pthread_mutex_lock(&lock);
    ready = 1;                       /* publish the state ... */
    pthread_cond_signal(&cond);      /* ... then announce it */
    pthread_mutex_unlock(&lock);
    return NULL;
}

pthread_cond_wait does three things atomically enough to matter: it releases the mutex, parks the thread, and — on return — reacquires the mutex before handing control back. The release and the park are a unit, which is what closes the window in which a signal could arrive after you decided to wait but before you were waiting.

The while is not defensive style. Four separate mechanisms can put a woken thread in front of a false predicate, and only re-testing covers all four.

The four ways a woken waiter can find its predicate false. Only the first is commonly named; the second is the one that bites in production, because it needs no exotic behaviour from the implementation at all. Source: POSIX.1-2024 pthread_cond_wait rationale.

Two further details separate working code from code that works on your machine.

Signal inside or outside the mutex. Both are correct. Signalling with the mutex held is simpler to reason about and is what the examples here do; signalling after unlocking can avoid a woken thread immediately blocking on a mutex the signaller still owns. The optimisation is real but small, and getting it wrong — publishing the state after signalling rather than before — is a lost wakeup. Publish, then announce.

pthread_cond_timedwait defaults to the wrong clock. Its absolute deadline is measured against CLOCK_REALTIME, which an NTP step or an operator can move. A five-second timeout can become a five-hour one. Fix it once, at initialisation:

pthread_condattr_t attr;
pthread_condattr_init(&attr);
pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);   /* immune to clock steps */
pthread_cond_init(&cond, &attr);
pthread_condattr_destroy(&attr);

struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);           /* must match the attr */
deadline.tv_sec += 5;

int rc = 0;                                          /* declared, and zeroed */
pthread_mutex_lock(&lock);
while (!ready && rc == 0)                            /* still a loop */
    rc = pthread_cond_timedwait(&cond, &lock, &deadline);
if (rc == ETIMEDOUT)
    handle_timeout();                  /* the deadline passed: ready is false */
pthread_mutex_unlock(&lock);

Back to top

The primitives, and what separates them¶

The useful axis is not speed. It is ownership — whether the primitive knows which thread holds it — and what a waiter does with its core. Ownership is what lets a mutex support priority inheritance, robustness and deadlock detection, and it is exactly what a semaphore gives up in exchange for being postable by anyone, including a signal handler.

The primitives a portable C program has. Costs are medians from data/sync/bench.c on the machine under Reproducing; the ownership and waiting columns are properties of the specification. Source: POSIX.1-2024 and ISO/IEC 9899:2024 §7.17.

Two consequences of that table are worth stating outright, because the folklore says otherwise.

A mutex does not enter the kernel to lock. On Linux it is a futex: the uncontended path is a compare-and-swap on a word in user space, and the syscall happens only when a thread actually has to sleep. That is why 7.9 ns is the measured uncontended cost here — the same order as a single atomic increment, and cheaper than the 9.5 ns compare-and-swap loop people replace it with.

A semaphore is not a slower mutex. It has no owner, so it cannot be inherited, cannot be made robust, and cannot tell a debugger who is holding it. In exchange, it can be posted by a thread that never waited, and by a signal handler — sem_post is on the short list of async-signal-safe functions, which is the reason it still appears in code that has to wake a worker from a handler.

Back to top

What each one costs, measured¶

All of these do the same work: add one to a shared counter. What changes is how the addition is made safe. The uncontended numbers are stable to within 2% across runs; the contended ones vary by up to a factor of two, which is why both are given as a range.

Cost per increment, median of three runs of five repeats each. Each ratio names what it is measured against, because the two blocks compare against different baselines: the unsynchronised increment above, the same primitive at one thread below. Source: data/sync/bench.c.

Three findings come out of this, and each one contradicts something commonly repeated.

On x86-64, memory_order_seq_cst is free for read-modify-write. The relaxed and sequentially consistent increments are within noise of each other (5.49 against 5.59 ns), because both compile to the same lock xadd and that instruction is already a full barrier on this architecture. Memory order is not primarily a runtime cost here — it is a constraint on the compiler's reordering, and a real runtime cost on weakly ordered machines like AArch64, where the same choice selects between ldaddal and a plain ldadd. Writing relaxed for speed on x86 and calling it an optimisation is measuring nothing and risking the port.

The spinlock inverts under contention. Uncontended it costs 8.4 ns, within 8% of the mutex. With four threads it costs 106.7 ns, 1.58× the mutex, because a spinning waiter is not idle — it is competing for the very core the lock holder needs to finish and release. The rule of thumb “spinlocks are for short critical sections” is stated from the wrong end: what matters is whether the holder can be preempted while holding. In a kernel with preemption disabled, it cannot, and spinning wins. In a user-space thread on a shared machine, it can, and spinning is a way to pay a whole scheduling quantum for a lock that was free.

Under contention every lock converges on the cache line. The mutex slows by 8.6× and the atomic by 3.9×, and neither is executing more instructions than before. What changed is that one 64-byte line is now moving between four cores' caches, and every operation waits for it. That is the same mechanism as the next section — except there, nothing is even shared.

Back to top

False sharing: the contention you did not write¶

Coherence works on cache lines, not on objects. A line is 64 bytes on every x86-64 and on most AArch64 parts, and a core that writes any byte of a line must own that line exclusively — which means taking it away from every other core that holds it. Two threads writing two different variables in the same line therefore contend as hard as two threads writing the same variable. The program has no bug a reviewer can point at; the layout does.

14.9×, from padding alone. Larger effects appear elsewhere on this page — parking and waking a thread costs 162× a spin hand-off — but those are the price of a different design. This one is the price of a structure definition: same calls, same algorithm, same number of instructions retired.

/* The bug. Nothing here is shared between threads -- and it is 13x slower. */
struct { atomic_long hits; } counter[4];        /* 4 x 8 bytes = one line */

/* The fix. One line each, so a write by thread 0 does not invalidate the
   line thread 1 is writing. _Alignas is C11; alignof(max_align_t) is not
   enough, the constant that matters is the cache line, not the type. */
struct {
    _Alignas(64) atomic_long hits;
} counter[4];

/* Same idea inside one structure: keep what one thread writes away from what
   another reads. A hot write field next to a hot read field is the same bug
   with the padding on the other side. */
typedef struct {
    _Alignas(64) atomic_size_t head;            /* producer writes this */
    _Alignas(64) atomic_size_t tail;            /* consumer writes this */
    _Alignas(64) void *slot[1024];
} ring_t;

The padded case measures 1.43 ns per operation — below the 1.68 ns single-threaded baseline — because four threads are now genuinely running in parallel and the figure counts wall-clock across all their operations. That is what scaling looks like when nothing is shared, and it is the reason the packed number is so damaging: it converts a perfectly parallel workload into a serial one without changing a line of logic.

Three practical notes. Find it, don't guess: perf c2c record and perf c2c report on Linux attribute cache-line contention to specific lines and offsets, which is the only way to distinguish this from ordinary memory pressure. Don't pad everything: 64 bytes per counter is a real cost in cache footprint, and a structure padded field-by-field can push the working set out of L1, which is the same problem in the other direction. Pad what is written concurrently. The line size is a property of the machine, not the language: C23 has no equivalent of C++17's hardware_destructive_interference_size, so 64 is a constant you write down, ideally fetched from sysconf(_SC_LEVEL1_DCACHE_LINESIZE) at build time and asserted with static_assert.

Back to top

Atomics and a lock-free queue¶

An atomic operation is not a small lock. It is one indivisible read-modify-write on a single object, plus an ordering constraint on everything around it — and it is the ordering, not the indivisibility, that people get wrong. memory_order_relaxed guarantees the operation itself is atomic and nothing about what other loads and stores may be moved across it, by either the compiler or the processor.

The pairing that matters is release/acquire. A store-release on an object makes every write the storing thread performed before it visible to any thread that performs a load-acquire on that same object and reads the stored value. That is the entire mechanism behind the queue below: one plain store to a slot, published by one release, consumed by one acquire.

/* A lock-free SPSC ring. Correct only for exactly one producer and one
   consumer -- that restriction is what makes it this simple. */
#define CAP 1024                    /* power of two, so & replaces % */
#define MASK (CAP - 1)

typedef struct {
    _Alignas(64) atomic_size_t head;        /* written by the producer only */
    _Alignas(64) atomic_size_t tail;        /* written by the consumer only */
    _Alignas(64) void *slot[CAP];
} spsc_t;

void spsc_push(spsc_t *q, void *v)
{
    size_t h = atomic_load_explicit(&q->head, memory_order_relaxed);  /* mine */
    while (h - atomic_load_explicit(&q->tail, memory_order_acquire) == CAP)
        cpu_relax();                        /* full */
    q->slot[h & MASK] = v;                  /* plain store: not yet visible */
    atomic_store_explicit(&q->head, h + 1, memory_order_release);     /* publish */
}

void *spsc_pop(spsc_t *q)
{
    size_t t = atomic_load_explicit(&q->tail, memory_order_relaxed);  /* mine */
    while (t == atomic_load_explicit(&q->head, memory_order_acquire))
        cpu_relax();                        /* empty */
    void *v = q->slot[t & MASK];            /* safe: the acquire ordered it */
    atomic_store_explicit(&q->tail, t + 1, memory_order_release);     /* free */
    return v;
}

Single-producer, single-consumer is the case where lock-free is genuinely easy, and it is easy for a reason worth naming: each index has exactly one writer, so no index ever needs a compare-and-swap, and the ABA problem — a value that changed to something else and back while you were not looking, which a bare CAS cannot detect — cannot arise. Add a second producer and both of those protections vanish at once. Multi-producer queues need CAS loops, and then either hazard pointers, epochs, or RCU to answer the question a garbage-collected language never has to ask: when is it safe to free a node another thread might still be reading?

Two hand-off measurements. The queue is the honest comparison — same work, two designs. The round trip isolates the wake-up itself, which is what the queue is paying for. Source: data/sync/bench.c, median of 3 runs; the queue figures range 25.5–34.9 and 215.4–301.3 ns.

The queue result — 10.3× — is real but should not be read as “lock-free is faster”. What it shows is that waking a sleeping thread costs microseconds, and a design that does it once per item pays that per item. A mutex queue that batches — signalling once per batch rather than once per element — closes most of the gap without any atomics. The lock-free version wins here because it never sleeps at all, which is also its cost: two cores are pinned for the duration whether or not there is work.

Measured with GCC 13.3 on x86-64, default flags: a 16-byte atomic store compiles to jmp __atomic_store_16@PLT, and atomic_is_lock_free returns 0 for it even when -mcx16 is given. Check the macro at compile time and the function at run time; do not assume. Source: data/sync/ and ISO/IEC 9899:2024 §7.17.1, §7.17.5.

Back to top

Reader-writer locks and where they stop paying¶

The premise is that readers can share, so a read-dominated workload should scale. The premise is true and the conclusion often is not, because acquiring a read lock is itself a write: the reader count has to be incremented, atomically, in a line every reader is touching. That cost is paid whether the read section is one nanosecond or one millisecond.

Four reader threads, no writers at all — the case most favourable to a reader-writer lock. The short section reads one element of a hot 16-element array; the long one performs 256 reads over those same 16 elements, so the data stays in L1 and the difference measured is the length of the critical section rather than memory latency. The crossover between the two rows is the finding: the same lock is 1.6× worse and 3.4× better on the same data, and only the length of the critical section changed. Source: data/sync/bench.c.

So the rule is not “use an rwlock when reads dominate”. It is use one when reads dominate and each read holds the lock long enough to amortise an atomic increment — which, on this machine, means somewhere above a hundred nanoseconds of actual reading. Below that, a plain mutex is both faster and simpler, and the honest alternative for very short reads is not a lock at all: it is a seqlock, an RCU-style scheme, or a per-thread copy.

The second problem is fairness. POSIX leaves reader/writer preference unspecified, so a writer can starve under continuous read load on a conforming implementation. Where that matters, the fix is a hand-written lock that blocks arriving readers once a writer is queued:

/* Writer-preferred rwlock. The standard pthread_rwlock_t does not specify
   whether readers or writers are favoured, and a reader-favouring
   implementation can starve a writer indefinitely under continuous read load.
   Blocking new readers as soon as a writer is waiting is what fixes that -- at
   the cost of some read throughput, which is the trade being made. */
typedef struct {
    pthread_mutex_t lock;
    pthread_cond_t  read_ok, write_ok;
    int active_readers, active_writers, waiting_writers;
} wprw_t;

void wprw_rdlock(wprw_t *rw)
{
    pthread_mutex_lock(&rw->lock);
    /* the waiting_writers term is the whole point: without it, readers
       arriving faster than writers can ever mean the writer never runs */
    while (rw->active_writers > 0 || rw->waiting_writers > 0)
        pthread_cond_wait(&rw->read_ok, &rw->lock);
    rw->active_readers++;
    pthread_mutex_unlock(&rw->lock);
}

void wprw_wrunlock(wprw_t *rw)
{
    pthread_mutex_lock(&rw->lock);
    rw->active_writers = 0;
    if (rw->waiting_writers > 0)
        pthread_cond_signal(&rw->write_ok);   /* one writer: only one can run */
    else
        pthread_cond_broadcast(&rw->read_ok); /* all readers: they can share */
    pthread_mutex_unlock(&rw->lock);
}

Two alternatives are worth knowing when even that is too slow. A seqlock lets readers run with no writes at all — they read an even sequence number, read the data, and re-read the sequence; if it changed, they retry — which makes reads nearly free and writes slightly more expensive, at the cost of readers that must be side-effect-free and retry-safe. RCU goes further: readers are literally free, and writers publish a new version and wait for a grace period before freeing the old one. Both trade a harder correctness argument for removing the reader's write, which is the thing the measurement above says you are actually paying for.

Back to top

Deadlock, and the three ways out¶

Deadlock needs four conditions simultaneously — Coffman's, from 1971 — and preventing it means making any one of them impossible. Naming them is not academic: it tells you which techniques are alternatives and which are the same technique described twice.

The four Coffman conditions. Every deadlock prevention technique in practice attacks the third or the fourth. Source: Coffman, Elphick and Shoshani, System Deadlocks, ACM Computing Surveys, 1971.

/* Technique 1: a total order. Any consistent order works; the address of the
   lock is one that needs no registry and no naming scheme.

   Two traps sit in the obvious version of this function, and both are in the
   two lines that choose the order.

   Relational comparison of pointers into *different* objects is not defined
   by C -- only == and != are -- so `from < to` is exactly the construct the
   article warns about under Pointers. Converting to uintptr_t first is the
   portable way to get a total order out of addresses.

   And a transfer from an account to itself passes both pointers equal, so a
   naive version locks the same non-recursive mutex twice and deadlocks
   against itself instantly. It is the degenerate case every ordering scheme
   has, and it must be handled before the ordering, not inside it. */
void transfer(account_t *from, account_t *to, long amount)
{
    if (from == to)
        return;                        /* self-transfer: nothing to move, and
                                          locking twice would deadlock */

    uintptr_t a = (uintptr_t)from, b = (uintptr_t)to;   /* defined ordering */
    account_t *first  = a < b ? from : to;
    account_t *second = a < b ? to : from;

    pthread_mutex_lock(&first->lock);
    pthread_mutex_lock(&second->lock);
    from->balance -= amount;
    to->balance   += amount;
    pthread_mutex_unlock(&second->lock);
    pthread_mutex_unlock(&first->lock);
}

/* Technique 2: back off rather than block. Note the randomised delay -- a
   fixed one lets two threads retry in lockstep forever, trading a deadlock
   for a livelock, which is harder to see because the process stays busy. */
void lock_both(pthread_mutex_t *a, pthread_mutex_t *b, unsigned *seed)
{
    for (;;) {
        pthread_mutex_lock(a);
        if (pthread_mutex_trylock(b) == 0)
            return;                                 /* holding both */
        pthread_mutex_unlock(a);                    /* release, do not wait */
        usleep(rand_r(seed) % 200 + 50);            /* randomised back-off */
    }
}

/* Technique 3: a deadline, on a clock that cannot jump. CLOCK_REALTIME can be
   stepped by NTP or an operator; clocklock takes the clock as an argument. */
struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += 2;
if (pthread_mutex_clocklock(&lock, CLOCK_MONOTONIC, &deadline) == ETIMEDOUT)
    return recover();                               /* a path, not a hang */

The comments in the first function are not padding: the two lines that establish the order are where both of its bugs live. Ordering by from < to compares pointers into different objects, which C does not define — see Pointers, arrays, and provenance — and a transfer to the same account locks one non-recursive mutex twice and deadlocks against itself before any second thread is involved. Both are invisible in testing until the day they are not.

Ordering is the only one of the three that prevents deadlock rather than surviving it. Back-off and timeouts both admit the cycle can form and give the program a way out, which means every caller needs a failure path, and a back-off loop with a fixed delay converts a deadlock into a livelock — two threads politely retrying in lockstep, CPU pinned, making no progress, with no blocked thread for a debugger to point at.

Neither ordering nor back-off is checkable by reading code at scale, which is why the tooling matters more than the technique. ThreadSanitizer finds both classes, and the second demonstration is the one worth internalising:

$ cc -O2 -std=c11 -pthread -o race data/sync/race.c
$ ./race race
counter = 20000 (expected 20000)          # correct, three runs out of three

$ cc -O1 -g -std=c11 -pthread -fsanitize=thread -o race data/sync/race.c
$ ./race race
WARNING: ThreadSanitizer: data race (pid=3449)
  Read of size 8 at 0x5629dc5f40a8 by thread T2:
    #0 racer data/sync/race.c:24
  Previous write of size 8 at 0x5629dc5f40a8 by thread T1:
    #0 racer data/sync/race.c:24
  Location is global 'counter' of size 8 at 0x5629dc5f40a8
SUMMARY: ThreadSanitizer: data race data/sync/race.c:24 in racer
counter = 20000 (expected 20000)

The unsynchronised program printed the right answer on every run. It would pass a test suite indefinitely. The sanitizer reports it anyway, because it is not watching for a wrong answer — it is tracking happens-before edges and noticing two accesses with none between them.

$ ./race deadlock                          # the two threads never overlap:
both orders completed                      # each is joined before the next starts

$ ./race deadlock                          # same program, under TSan
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=3454)
  Cycle in lock order graph: M0 (0x563421370080) => M1 (0x563421370040) => M0

  Mutex M1 acquired here while holding mutex M0 in thread T1:
    #0 pthread_mutex_lock
    #1 ab_order data/sync/race.c:33

  Mutex M0 acquired here while holding mutex M1 in thread T2:
    #0 pthread_mutex_lock
    #1 ba_order data/sync/race.c:43

Here the two threads are joined serially: at no point do both run at once, so this execution cannot deadlock and no amount of stress testing would ever hang it. TSan reports the inversion regardless, because it maintains a lock-order graph across the whole run and finds the cycle in it. That is the argument for running the sanitizers over a normal test suite rather than trying to reproduce a hang: they find the ordering, not the outcome.

The cost is real — roughly 5–15× slowdown and several times the memory — so this is a test-time build, alongside the release-time hardening listed on the reference page. helgrind under Valgrind finds the same classes without recompiling, more slowly. PTHREAD_MUTEX_ERRORCHECK catches self-deadlock and unlocking a mutex you do not own, cheaply enough to leave enabled in debug builds.

Back to top

What the textbook examples leave out¶

Every example so far, here and everywhere else, assumes threads that do not die, do not get cancelled, do not get signals, and share an address space. Production code violates all four assumptions, and the resulting failures are the ones that survive review, because the code that has them looks exactly like the code in the book.

The clearest case is a mutex shared between processes. It is the fastest IPC there is — two processes agreeing through mmap'd memory with no kernel object and no syscall on the fast path — and it introduces a failure mode that cannot occur inside one process: the owner can die.

$ cc -O2 -std=c11 -pthread -o robust data/sync/robust.c && ./robust

plain PTHREAD_PROCESS_SHARED mutex
  parent: timed out after 2s -- the lock is held by a dead process,
        and nothing will ever release it. Without the timeout this
        call blocks forever.

same mutex with PTHREAD_MUTEX_ROBUST
  parent: EOWNERDEAD -- the kernel handed the lock over and told us the
        previous owner died. State may be torn: counter = -1
        repaired, pthread_mutex_consistent() called, counter = 0
  parent: lock reused after recovery, counter = 1

The first half of that transcript is what a plain shared mutex does when its holder dies: nothing, forever. The timeout is only there so the demonstration terminates — a normal pthread_mutex_lock would still be blocked. The second half is the same program with one attribute added. The kernel hands the lock to the next waiter with EOWNERDEAD, which is not an error to log and ignore: it is a statement that the shared state may be half-written — visible here as counter = -1, exactly where the dead process was interrupted — and an obligation to repair it and call pthread_mutex_consistent. Skip that call and the mutex becomes permanently ENOTRECOVERABLE for every process, which is the design working: an unrepaired invariant fails loudly rather than silently.

/* A mutex two processes can share, that survives one of them dying. */
pthread_mutexattr_t attr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);  /* in mmap'd memory */
pthread_mutexattr_setrobust(&attr, PTHREAD_MUTEX_ROBUST);     /* survive the owner */
pthread_mutex_init(&shared->lock, &attr);
pthread_mutexattr_destroy(&attr);

/* Every lock site now has a third outcome to handle. */
int rc = pthread_mutex_lock(&shared->lock);
if (rc == EOWNERDEAD) {
    repair_invariants(shared);           /* the previous owner died mid-update */
    pthread_mutex_consistent(&shared->lock);   /* or the mutex stays unusable */
} else if (rc == ENOTRECOVERABLE) {
    /* someone got EOWNERDEAD and did not call consistent(): permanently dead */
    return -1;
} else if (rc != 0) {
    return -1;
}

The seven cases that separate a working example from a working program. Every row is a documented POSIX behaviour rather than an implementation quirk. Source: POSIX.1-2024 pthread_mutexattr_setrobust, setprotocol, pthread_cancel, signal-safety(7), and pthread_atfork.

Back to top

Choosing¶

The selection this page's measurements support. Costs are from the machine under Reproducing and will differ on yours; the orderings are more portable than the numbers, except where noted as architecture-specific. Source: data/sync/bench.c.

The ordering of that table is deliberate: the first row is the answer far more often than the rest of the page implies. Every alternative below it buys throughput by giving up something — an owner, a sleeping waiter, a simple correctness argument — and the measurements here exist to say what the purchase costs, not to suggest it is usually worth making.

Back to top

Reproducing this page¶

Every number here comes from three sources that ship with the site, so they can be checked rather than believed:

The three programs behind this page. Each is a single translation unit with no dependencies beyond libc and pthreads.

A correction that moved the numbers. The first version of this page was measured with a harness that re-initialised the mutex queue's mutex and condition variables inside the timed region — itself undefined behaviour, and charged to the queue — and with the packed counter array aligned only to 8 bytes, so the four counters it calls “one cache line” were not reliably in one. Both are fixed in the file that ships here, and every figure above is from the corrected harness. The conclusions did not change; the magnitudes did.

The machine. A four-vCPU x86-64 Linux container, 64-byte cache lines, GCC 13.3.0, glibc 2.39. Each case runs five times inside one process and reports the median; the whole binary was run three times and this page reports the median of those medians. Threads are pinned to distinct CPUs where the container permits it.

What the variance says. Uncontended measurements are stable to within 2% across runs. Contended ones vary by up to 1.4× between runs on a shared machine, which is why every contended figure is quoted with its range and why the page leans on ratios measured within a single run rather than on absolute nanoseconds. The ratios are stable. Taking each run's own ratio — which is not the same statistic as the 14.9× quoted above, that being the ratio of the medians — false sharing came out at 15.2×, 10.5× and 15.8×, the lock-free queue at 11.8×, 6.2× and 10.3×, and the reader-writer crossover reproduced in every one — the short-read case 1.36×, 1.54× and 1.62× worse than a mutex, the long-read case 0.28×, 0.40× and 0.41× of it. Every conclusion this page draws survives the worst run.

What would change the numbers. More cores make every contended figure worse, not better, because more caches contend for the same line. A weakly ordered machine — AArch64, RISC-V — would separate relaxed from seq_cst, which are identical here. A dedicated machine would tighten the contended ranges without moving the medians much. None of those would change the orderings this page draws conclusions from.

Back to top

Sources¶