Repository navigation
Conversation
The general-purpose queue pays for two mutexes and a node allocation per push because a library cannot know how many threads will use it. Where exactly one thread pushes and one pops, none of that is needed. spsc_ring is an unbounded wait-free ring of blocks, keeping the generic queue's semantics exactly: FIFO, push never blocks or fails, try_pop never blocks, no capacity. Two queue-global counters cross between the sides, each on its own cache line and each written by one side only, so the consumer performs no shared store per element at all. Slots are aligned to a cache line, because 64-byte elements straddling two lines cost a factor of forty, measured. The shim gains a topology tag and a union of the two implementations. That tag is the whole substitution mechanism: rewriting one constructor argument changes the object's representation, so no type surgery is needed. Building with -DWTC_CHECK turns each static verdict into a claim the program checks: the ring records its first producer and consumer and aborts the moment a second appears. The suite includes a test that must fail, because a suite that cannot fail proves nothing.
The analysis used to count spawn sites, so eight spawns in a loop read as one thread and a worker pool came out SPSC. It now counts thread instances by call string: a thread is a spawn instance under a chain of call sites, and a function instance carries its site and parent, so a parameter resolves as the operand of the call that produced it. Every unresolved case maps to unknown, never to a smaller count, because a wrong SPSC verdict does not run slower, it races. Lifting and lowering now read what the compiler actually emits rather than hand-written IR. A returned non-trivial object arrives through a hidden pointer argument, so the queue is not operand zero; lowering has to restore that argument attribute, and has to strip our own attributes, or cir-translate rejects the module. Instances are pruned for functions that cannot reach a queue operation, without which the cap filled with standard-library functions and any program including <thread> gave up. WTCSpecialize acts on the verdict by rewriting the constant the shim's constructor receives. tools/wtc-cc drives the whole path from C++ to a running executable, and the tests cover the paths where the passes must refuse, not only those where they must fire.
The ring had to be measured before any pass learned to emit it. If a hand-written single-producer ring is not meaningfully faster than the general-purpose queue on a real workload, no amount of compiler work makes it so, and it is better to find that out early. Six workloads cover throughput, the latency distribution, a three-stage pipeline, the multi-producer case the compiler must refuse, and the ring's own shape. Fourteen baselines are the libraries a developer would otherwise reach for, plus the hand-written implementation, which is the ceiling: the compiler cannot beat code a human wrote for this exact case, only reach it without the human. campaign.sh refuses to measure in an unfit environment, discards a warm-up, interleaves variants across repetitions so drift hits all of them equally, and takes the host's measurement lock per run. The summary carries two drift checks, because the obvious one compares single runs of a noisy variant, flags every leg, and means nothing on its own.
The gate passes on all four criteria: 3.07x throughput at 64-byte elements, p99 latency 0.16x and p99.9 0.08x, a three-stage pipeline 2.8x end to end, the compiler's binary byte-identical to the hand-tagged one, and the fallback costing nothing where the verdict is not SPSC. The finding that decides what to do next is not the speedup. The same ring with elements stored inline runs forty-one times faster at 64 bytes than behind the shared_ptr the shim's interface hands back: the synchronisation this ring removes was the smaller of the queue's two costs, and the larger is an allocation per element.
The memory-layout-by-owner work needs a ground truth before it needs an analysis: every hand-written cache-line annotation is a place where an expert decided that different threads touch these fields. These tools fetch 17 codebases and count those places from the source text, so the scan reaches code the pipeline cannot ingest yet, Linux included. corpus-fetch.sh clones the codebases, pinned where a release matters (MySQL 8.0.39, SQLite 3.50.2) and upstream llvm-project for llvm and compiler-rt, so the whole corpus comes from one script on any host. corpus-census.py matches the mechanisms (alignas and aligned attributes to a line, the kernel's cacheline macros and groups, DPDK's guards, oneTBB's alignas(max_nfs_size), project macros, explicit padding) on code only: comments, preprocessor lines, string literals and build and test trees are set aside first, because each of them was found to inflate the count when read. The brace state is saved at `#if` and restored at `#else`, parentheses after a record keyword count as an attribute's until two names have been seen, and a padding field is a byte type: each of these rules closes a miscount a reader found in the output. corpus-classify.py says whether a site separates fields of one record or whole objects, and for fields whether it marks one member or a group, since those need different transformations. A count of spellings is not a count of sites until its precision and its misses are measured. corpus-sample.py draws a seeded random sample of sites with their context for a reader to label and scores the labelled file with a Wilson interval; corpus-recall.py lists the lines that mention cache lines in code and are not census sites, for the same labelling. Both labelled samples live beside the document that cites them. corpus-contention.py reads a false-sharing report as a graph per object, to say whether reordering could separate its fields at all.
Every analysis here needs the same answer before it can say anything: which dynamic threads execute a given operation. The queue analysis needed it to count producers and consumers, and the layout analysis needs it to tell whether two fields are written by different threads. It was buried in the queue pass, and a second copy of the subtlest code in the project is not a thing to have. The client now supplies the pruning predicate; that is the only change in shape. The walk keeps its rules: a spawn in a loop stands for many threads, an instance carries the call site that created it so a parameter resolves to the caller's argument, and recursion is handed back to the client. What it adds is what real code needed: - Threads started with pthread_create, thrd_create and std::thread's _M_start_thread are spawns. Without them a C program has threads but no known start, and the start-up phase never ends. - A spawn's entry is resolved through the parameters that carried it, as in memcached's create_worker(func, arg), and a function whose address only flows into spawn entries is not a thread of its own. - Reachability and "can this spawn" are fixpoints over the call graph; memoised recursion cached "no" while a cycle was being explored. - The start-up phase ends at the first visible spawn, including one in a global constructor, which ends it for every constructor. - The instance cap is 250,000 and reported: amalgamated memcached needs 132,997, and the old cap of 10,000 was exceeded silently.
Two fields of one object, written by different threads, landing in one cache line: nothing in the program says so and nothing reports it, and the line travels between cores for the rest of the run. What that costs is measured per program, not assumed. Dynamic tools see only what one run touched and report an address, not a field; the walk already knows which threads touch which field, and CIR carries member offsets. The pass only reports, so it changes no byte. Its verdicts: - definite: certainly different threads, repeated accesses on both sides, outside the start-up phase; - possible: contention the walk cannot rule out, such as one instance that stands for many threads touching both fields; - cold: a resolved claim that the pair costs nothing, for instance one side runs once. Unknown is never printed as cold. Two threads on the same field is true sharing and is not reported. A stack slot is a separate object per invocation, so it is definite only in a function that runs once. Objects aligned below a line may start at several offsets, and each pair says in how many of them the two fields share one. Every remark names both accesses, and the pass prints a summary with the accesses it could not resolve, which is the honest measure of its reach. Past the walk's instance cap it gives no verdicts. Each rule has a test that fails without it, most of them taken from a memcached case that exposed the rule. wtc-scan.sh runs the pass over translation units and collects the findings as CSV; lit learns .c tests and %clang for the C integration tests.
A global's alignment is an attribute of the operation, not of its type, so raising it rebuilds no type and moves no offset; the linker takes the maximum and nothing inside the object shifts. That makes a global the one object whose placement a pass after CIRGen can change safely. The pass raises a global to a line of its own when a thread repeatedly writes it and a different thread repeatedly touches a partner global. It keeps every partner, not the first one found, and raises a global only if some partner can also be raised, since aligning one end alone does not separate the pair. It refuses what it must not touch, and says why in a remark: constants and globals already on a line; declarations, where align 64 would be a promise about an object another module defined with alignment 8; weak and link-once copies the linker may drop; globals in a named section, whose padding would break the __start_/__stop_ walks; thread-local objects, which have nothing to be separated from; and a global whose record type is incomplete, since asking the data layout for its size asserts. A refused global is not a partner that could be separated either. The padding cost is counted from the ABI alignment, 8 for an i64 with no attribute, and dry-run reports without changing anything. It is kept as a measurement more than as an optimisation. On amalgamated memcached it selects 60 of the 141 mutable globals for 3,489 bytes, where aligning all 141 costs 7,890 and gives the same guarantee, so for globals this is close to a build flag. The test align_by_owner_runs.c compiles, transforms, links and runs a program and asks it where its counters landed.
Both layout passes recognised a memory access as a cir.load or a cir.store. An atomic fetch-add is neither: it is one operation that reads and writes its address, and it is how lock-free code bumps a counter. So the textbook pair -- two counters in one line, two threads, one fetch-add each -- produced no verdict and no summary line, in either pass. Exchange, compare-exchange, test-and-set, clear, bit-field stores and cir.copy were invisible the same way. One function now lists every address an operation reads or writes, with what is read or written there, and both passes use it. A read-modify-write counts as a write, since it is the write that moves the line; a bit-field store rewrites its whole storage unit; a copy is a read of its source and a write of its destination.
A review ran the analysis over small programs with known answers and over amalgamated memcached, and found the verdicts wrong in both directions and the walk losing threads. Each finding is a rule with a test that fails without it. Threads. A helper that parks the worker's address in a local and spawns from the local was never expanded, so the worker ran on no thread; the reachability pass now resolves a spawn's entry as the walk does. A worker's address handed to a spawn counted as taken, which turned every indirect call into a possible spawn and ended main's start-up phase at its first function pointer; the root-finding rule is one function for both. The instance cap bounded spawned and called instances but not roots; remarks are held until the walk ends, printed once per operation, and not at all when the pass gives up. Over-claims. A thread-local global got the same object for every thread, so its own fields were a pair. A worker spawned from one arm of an if or the other was two threads; threads the walk cannot show both exist make a pair possible. A stack slot of a function called twice was one object. A goto made every access in its function repeated; the rate is unknown there. A store to a whole record and a store to one of its members were two fields of one line; overlapping bytes are true sharing. Under-claims. A recursive function ran "once", so the pair it touched was cold; a function on a call-graph cycle runs many times. A thread with an access the walk could not place still made a pair cold; it may be writing that very field through the pointer it could not follow, so the pair is possible, the remark says why, and the summary counts the pairs that would be cold but for unplaced accesses -- on memcached all 349 of them, since every thread there has such an access. Tests. Three rule tests wrote their fields once and fell out before the rule they named; their writes are in loops, and separate-lines runs again with its padding stripped. Five checked a verdict and not the summary; they check the summary. The alignment tests check the output IR as well as the remark. Figures. The pass's header and its integration test cited a 6.4x cost for two counters in one line; the figure came from an unreserved probe with no record of its output, and is gone. The header no longer calls the output "ground truth" for the layout pass: on memcached every reported pair was serialised by a lock or written too rarely to matter.
corpus.md is the record of the memory-layout-by-owner work: what people pad by hand across 17 codebases, what the analysis reports on memcached, and whether any of it survives a reading of the source. As counted at this commit, the census finds 1,659 sites in 110,016 files, 46% of them separating fields of one record and 54% whole objects, with 90% of the field annotations on a single member. The precision and recall of that count were not yet measured; later commits remeasure the census and correct the figures. Earlier counts were wrong in ways two reviews found; a table lists every number that moved and why, and the retracted text stays, marked, rather than being deleted. On amalgamated memcached the analysis at this commit reports 155 definite and 503 possible pairs. Every object they sit on was read against the source, and none describes traffic that separating fields would remove: each pair is decided by how often one side runs, which a static analysis cannot tell. Two memtier legs on a second host test that reading: applying the verdict by hand moves the point estimates by under 1%, with intervals reaching from -3.2% to +2.0%, and compiling through ClangIR alone costs 1.3% to 3.4% by point estimate. A later commit rewrites the measurement text to say no more than the data show. The document says what this does not show: arrays and heap objects the analysis does not report, gains below about 2%, and pairs that are never concurrent, since verdicts have no notion of a join.
Three scripts under tools/ printed their prompts and notes in another language, with comments to match; everything that reaches the repository is written in English, so they are translated. Their behaviour is unchanged, and each still parses. The prompt in protect-main.sh accepted a localised yes as well; it accepts y and yes. docs/README.md described what vision.md leaves out by naming a role. The committed text names no person and no role, so it says "funding roles" instead.
Two readings of the corpus document, the second of the rewrite the first produced, checked every claim against the sources, the data and the code, and the document now says what they support. The memcached review stands, for its five objects and for the 349 pairs that were "cold" and are possible, since all sit on the same objects. The reasons are corrected where they were wrong: the @stats argument rested on two fields that never share a line and on rates nobody measured, accepting_conns is written under the lock, the logger with a watch client is a workload neither leg ran, a shared lock was credited with deciding the pairs in one paragraph and denied it in the next, and memcached joins its threads only on a graceful shutdown while `lru_crawler disable` stops the crawler without a join. The measurements are described as run: all of memcached's threads on four CPUs, no warm-up, leg 1's arm order unbalanced with a 1.6% drift by position, the thresholds written with the results rather than before them, the host recorded from lscpu; the site counts under STATS_LOCK, the thread count, the p99 figure and the leg-2 positions are each corrected. "Less than 1%" becomes point estimates within 0.7% and intervals from -3.2% to +2.0%; the round-trip gate is judged on the point estimates and said to be, since no interval resolves it. The census carries its precision -- 74% [65%, 82%] on 100 sites after an independent second reader changed six labels, which carry a note -- and its misses, a tenth to a half of what it finds per codebase, with both labelled samples committed; test files are set aside and the classifier bug that put MySQL's sites on the wrong side is corrected, which moves the field share to 51%, and the text below the line that still argued from objects being the larger half is annotated. leveldb's "none" was a missing platform define. Stale passages are annotated rather than deleted. The glossary gains the verdicts and the walk's terms, the docs index gains gate.md, audit.md gains the layout passes' closed defects and open limits, and README and vision.md say that the gate was measured and what the second direction found. Two compiled Python caches that had been committed with the tools are untracked and ignored.
A review of the previous round's fixes, on small programs with known answers, found eighteen ways the verdicts were still wrong. Each is a rule now, with a test that fails on the code before it. The rate of an access was read off its own function, so a goto in a caller, or in the main that spawned the thread, still turned every single store below it into a repeated one. A site's multiplicity is kept conservative, as the queue analysis needs, and the instance records when that Many is a guess; the diagnosis treats such an instance as neither once nor repeated. A loop is a loop whatever else the function does. The overlap rule judged a pair on the widest access anyone made at an offset, so a start-up snapshot of a whole record hid every pair of its members; overlap and line geometry are judged on the two accesses that make the verdict. A once-writer the walk could vouch for masked another whose thread has unplaced accesses; one such writer makes the pair unknown. Unplaced accesses count only when they repeat, or may: one touch through an unknown pointer cannot make a side hot. Two spawns in two cases of a switch that break, or one in an arm that returns and one after the if, are not two certain threads either; two arms inside a loop are. A function that spawns itself through a helper is a chain of threads, and a recursive call is walked once more rather than skipped, so what it reaches is seen. A stack slot is one object when the function that owns it runs once, whatever the threads that touch it stand for. An access through a cast of an object's own address, a memset or a memcpy, and an access to an array element are accesses: the first two are placed, the third is counted until elements are read. The pass clears its state between modules. The alignment pass follows the same rate and exclusion rules and refuses comdat globals and aliases.
Nine more defects, from a review of the previous commit on programs with known answers; each is a rule with a test that fails before it. A memset with a length the pass could not read asked the data layout for the size of void and aborted; the size is the pointee of the program's own pointer, or a byte. Unrolling every recurrence once per path made the walk exponential on a group of mutually recursive functions, with the cap reached at six; each function's recurrence is unrolled once in the whole walk. The relevance filter took every function with a local scalar as relevant; only a record counts. A start-up memset of a whole struct by the thread that later writes a member hid that member's pair, since one size per thread covered both; repeated and once-only accesses keep their own sizes. A goto in main made the end of its start-up phase "known", so init's writes paired with a worker's as definite; a guess about start-up rates the access unknown. A local array whose address never leaves its invocation was counted as an access the walk could not place, which turned a cold pair possible; the escape check itself was answering yes on every exit, so no slot had ever been private. A whole-record access that came first at an offset named the field after the record; the most specific name wins. The remark about a once-writer with unplaced accesses pointed at another thread's store and at another pair's geometry. An alias and its aliasee were two roots for one storage.
traceToLocalRoot followed every cir.cast, so a pointer made from an integer -- the address of an object cast to an integer and back, or that integer plus an offset -- was placed on whatever object the integer had been made from. The walk cannot vouch for what an integer is between two casts. It now follows a bitcast and an array's decay to a pointer to its first element, which still name the object, and stops at anything else; the access is then one it could not place, and counted as such.
A field access through a heap pointer was an access the analysis could not place, and on memcached that was its whole worker pool, one struct per thread from one calloc. The call that allocated an object is now its identity, the way a global or a stack slot is: malloc, calloc, aligned_alloc, memalign, posix_memalign and operator new; realloc is not one, since its result replaces an older object. The walk reaches the call through a stack slot written once, through a parameter resolved at the spawn or call site, and through a pointer-typed global the module stores exactly once and only ever loads -- memcached's `threads = calloc(...)` -- and refuses a global written twice. An allocation in a loop, or in a function that runs more than once, stands for many objects, and a pair on it is possible at most, as on a stack slot of such a function. Where the object starts is what the allocator promises: the call's own alignment operand, else the platform's guarantee for malloc, 16 bytes and an option. An element step with a constant index -- arr[2].f, s.counts[1] -- now adds that many elements to the offset, before the first member or between members, and names the element in the field's path; any other index leaves the access unplaced. The queue analysis keeps treating a heap queue as unknown until it can tell one allocation from many.
The next review found the start-up phase ending too early and the one unrolling of a recurrence going to the wrong caller. A call to a function that can spawn ended the caller's start-up phase at the call, so what the callee did before its own spawn -- memcached's memcached_thread_init, which fills in the workers before starting them -- counted as running beside the threads it had not yet started. The call is before the spawn, and the callee's instance reads its own phase from its own operations. A function with a goto that cannot start a thread runs wholly before any spawn, and keeps its start-up phase; a goto beside a spawn is still the guess it was. A recurrence was unrolled once per function in the whole walk, and the walk being breadth-first, a start-up call took it, so a worker's recursive writes to another object were never seen. Once per function, thread and phase for a recursion through calls; once per function for a recursion through a spawn, which starts a new thread each time. A store that holds a slot's address both as its destination and, through another derived value, as the value stored was judged once and the slot passed as private. A once-only read of a whole record widened the same thread's once-only write of a member. An alias declared with another type than its aliasee was dropped as not an object.
A worker pool is usually an array of per-thread structs, one element handed to each thread at a spawn inside a loop: `pthread_create(..., &args[i])`. The walk resolved the parameter through the spawn site to `&args[i]`, met the element step and gave up, so every access through such a handle was one it could not place. Read the step when its index is the induction variable of the loop the spawn sits in (a slot stored only before the loop and in its step region): every thread the spawn starts gets a different element. A place is then the root plus that loop, offsets are within one element, and the report labels it "(one element per thread)". Within an element the thread standing for many is one thread, so two of its fields are no pair. Across the boundary the tail of one element and the head of the next belong to two threads; a hot write on one and a hot touch on the other is definite, judged at every start an element may have, gcd(alignment, element size, line) apart. A call that hands back one allocation -- every return of its body traces to the same allocator call, through further such wrappers -- is that allocation; the call to the wrapper, not the allocator inside it, is what runs once or many times, and its alignment is the allocator's. Phoenix linear_regression built with -DNDEBUG now reports 17 definite pairs across elements of its 64-byte per-thread results struct, the case the dynamic detectors use as their canonical example; every access the workers make to that array is placed, while main's own accesses to it, in the spawn loop and after the join, are not yet. Built with assertions, its CHECK_ERROR macro evaluates pthread_create twice, the second time only when the first failed; the analysis, which does not read conditions, sees a second thread on each element and reports the fields of one element as shared between the two. memcached is unchanged at 155 definite, 852 possible, 0 cold, with 4,860 accesses unplaced (was 4,868).
An element step whose index is neither a constant nor the spawn loop's induction variable -- `red[val]++`, `slabclass[id].sl_curr`, main's `tid_args[i].points` written in the spawn loop -- named an element the analysis could not read, and the whole access was counted as one it could not place. The array is known all the same. Such an access is now placed on the array, at its offset within some element, under a place of its own: "(element not known)". Two threads there may be on one element or on two, so a pair is possible and never definite, and the same field pairs with itself, since in neighbouring elements it sits on one line; neighbouring is judged with the element size, and the report says so when the two fields differ. The place is never cold, and a thread with a repeated access at an unknown element of an array blocks cold on that array, as an access it could not place at all does, because it may be touching the very field the verdict says it touches once. An array decayed to a pointer with no step after it is its element 0, so two per-thread counters in one int array, counts[0] and counts[1], are now a definite pair; they produced no output at all before. Two tests used an unknown-index access on another array as their "unplaced" device and now see it placed where it belongs; their pairs on the first object are cold, which still shows what each test is for. memcached: unplaced accesses 4,860 -> 4,322, 531 at elements the analysis cannot tell apart; 1,207 possible pairs (was 852), the 355 new ones all under "element not known". Phoenix linear_regression: unplaced 20 -> 11, verdicts unchanged.
Three readings of the five Part 2 commits -- the walk, the diagnosis pass, and the claims -- found eleven defects each. The ones that could make a verdict false: A thread spawned in a loop stands for many threads, and every access of it was "repeated"; on an element each thread owns alone that made a once-only write of the element a sustained pair with the neighbour's. What repeats on such an element is what repeats inside the thread (multWithinThread), so a worker that writes its result once makes no pair, and a once-write across the boundary is cold. "The two threads that own them" was said of any loop whose variable indexed the element: a loop stepping by two, one spawning under an if, one inside another loop, one in a function that runs more than once, one whose element address was computed before it. Each of those hands an element to no thread, or to several. A thread owns its element alone only when the loop steps by one, spawns on every iteration, is the only repetition around the spawn, the stride sits inside it and the spawning instance runs once; otherwise the place is relative to an element but no pair is claimed across elements, and two of its fields are possible, not nothing. A wrapper that returns its allocation and also stores it in a global gave the object two roots, and a hot write under one name let the pair under the other be cold. The root is the allocator call behind the wrapper; whether it runs once is read from the call graph, so an init() that touches no field, and so has no instance, is no longer "may run more than once". A pointer global with a global_view in another global's initializer, or with an alias, was single-stored while being written through the other name; any symbol use but a cir.get_global now spoils it. An entry handed to the spawn through an integer round trip was neither resolved nor a root, and its thread was lost; it is a root. A recurrence the walk visits and does not expand reaches objects the expanded one did not, so its thread blocks cold. Places are now keyed by root and kind -- absolute, relative to a thread's element, or at an element nobody can name -- rather than by the spawn loop: two spawn loops over one array meet on the element, and main's `args[0].sum` after the join meets the per-thread accesses at the offset modulo the element size. A constant index at the spawn (`&args[1]`) is so many bytes in. A whole-object access through a pointer the walk resolves (memset of a heap object, `*me` of a per-thread counter) is an access; it was dropped before placement. cir.get_element at the spawn is read like a stride. Element starts are stepped by gcd(alignment, element size) for element-relative places too. Nested unknown steps use the smallest element. A pair on an allocation that may run more than once is said, as possible, rather than dropped. One operation is counted once in the summary. The remark no longer promises that padding each element to a line removes a pair: padding to a multiple of a line and aligning the array does. memcached: 156 definite (slabclass[0].slabs against slab_list, now that `p = &slabclass[0]` resolves), 1,335 possible, 4,377 unplaced, 678 at elements the analysis cannot tell apart, 137,526 instances. Phoenix linear_regression is unchanged at 17 definite across elements.
corpus.md gains a section on what the analysis places since October -- heap objects by their allocating call, array elements by a constant index or by the spawn loop's own, one element per thread, elements it cannot tell apart -- with the verdict rules across elements, the Phoenix results (linear_regression's 17 definite pairs across elements, matched by hand against the struct layout; the other programs' counts and why histogram's and word_count's known cases are not findings yet), the memcached recount (156 definite, 1,335 possible, 483 of them at elements not told apart and unread) and the 33 defects the audit of the round found. The status, the tables and the limits are brought to the new figures: 4,377 accesses through pointers the walk cannot follow, where it said 5,106 on arrays and heap objects it could not see at all. audit.md records the fourth review and what the passes still cannot name; glossary.md defines place, allocation site, one element per thread and element not known, and widens definite, possible and cold to the element cases; vision.md no longer lists arrays and heap objects among what the analysis cannot see. The census figures are untouched: the rerun with the corrected tools waits for the corpus host.
The summary counted the accesses the analysis could not place as one number, and what to place next was a guess. The walk now records why it gave up on a handle -- a pointer loaded from a field of another object, from a global written more than once or named elsewhere, from a stack slot written more than once; a parameter of a function nobody in the module is seen to call; the result of a call that is not an allocation; an integer cast to a pointer; anything else -- and the summary line ends with the account. On memcached the 4,377 unplaced accesses divide into 983 through a pointer loaded from a field, 1,578 from a slot written more than once, 1,119 through a parameter of a root (the libevent callbacks' `void *arg` above all), 311 from a call that is not an allocation, 26 from a global written twice, and 360 otherwise. The slots and the fields are what a "one of" reading of a pointer's stores could reach; the root parameters are out of a module's sight.
A pointer loaded from a stack slot, a pointer global or a field written in several places was left unplaced: on memcached that was 1,521 accesses through slots, 984 through fields and 26 through globals, more than half of what the analysis could not identify. Every store into such a place names a value, and when each of those values resolves, the loaded pointer is one of the places they name. The walk now lists the stores into a slot (in the same instance, under the slot's own name or a bitcast of it, when no other use of the address but a load, a store or a bitcast exists), into a pointer global (nothing else naming it) and into a (record, field) pair anywhere in the module, resolves each stored value where its store runs -- in every instance of the storing function, or without a call string when it has none -- and hands back the set, each place marked oneOf. A store of a null is no second value: `T *p = NULL; p = &x;` is the commonest shape of a pointer local and leaves the slot single-assigned. The load stays unplaced, with the cause it had, when one source cannot be placed; when the record is written some other way than through the field -- assigned or copied whole, a global with an initialiser, seen through a view of another type (`*(S **)((char *)&c)`), a union member written beside the one read, its address handed to a function without a body or leaving as `void *` through a local or a forwarding function; on a cycle (`n->next = m->next`); or past 8 sources or 64 instances of a storing function. A set of one is that one place, not a set: two stores of the same object, one on each arm of an if, name it as surely as one. A candidate resolved in the storing thread's instance keeps no element of that thread: the loading thread's element is unknown, and the set never owns one. Chains of locals assigned on two arms are memoised per slot and instance, where they were resolved once per path. The diagnosis records an access through a set on every member, kept apart from the accesses that name the object, with sizes of its own, so a 64-byte copy through a set cannot widen the thread's own 8-byte write: such an access makes a pair possible and never definite, and a repeated one blocks a cold verdict on that object the way an unplaced access of the same thread does. The reason printed for a pair judges the writer by its role, and the set's reason comes first. The walk runs to completion before any access is placed, since the instances a field's stores run in exist only then. The summary counts the accesses so placed and says how many sources were too many; an access whose set failed on one source is counted under that source's reason -- the frontier the walk stopped at -- rather than under the load. Two independent readings of the first version of this, one of the walk and one of the diagnosis, found fifteen defects; the ones that changed a verdict are the poisoning rules, the per-thread element and the set's own sizes above, each closed with a test that fails on the code before it. On memcached the verdicts stand at 159 definite and 1,363 possible (from 156 and 1,335), with three definite pairs on extstore's engine object, calloc'd in extstore_init, to be read against the source like the others; 4,265 accesses are unplaced (from 4,377), 51 placed as one of several objects. Counted at their sources: 1,183 through a field whose list says nothing, 14 through a global, 324 through a slot whose address goes elsewhere, 1,223 through a parameter of a root nobody is seen to call, 932 through the result of a call that is not an allocation, 62 past the caps, 527 otherwise. Phoenix is unchanged; streamcluster's possible pairs fall from 36 to 22. Three tests used a twice-written global as a device for "unplaced"; they keep their claims through a store of a pointer made from an integer, and heap-two-sites-one-global now pins the set it was written to deny.
The corpus document gains the round that places a pointer loaded from a slot, a global or a field written in several places as one of the places the stores name, with its memcached figures (159 definite, 1,363 possible, 4,265 unplaced, counted at the sources the walk stopped at), the reading of the three definite pairs it added on extstore's engine (lock-serialised or rare, like the pairs on the globals), the Phoenix and PARSEC objects named by the dynamic-detector papers set against what the analysis names (four of the five it can reach), and the audit of the round, two readings and fifteen defects, all closed. The census is recounted on a corpus fetched again on the second host: 1,546 sites, which mixes the tool fixes with the corpus having moved and is said to. The audit record and the glossary follow.
A program on DPDK's EAL has no pthread_create and no malloc: its workers start with rte_eal_remote_launch(f, arg, lcore) and its objects come from rte_malloc and its zeroing, counting and per-socket forms, or from the constructors of its data structures. Without those names the analysis saw no thread at all in app/test/test_ring_perf.c and placed nothing on the heap, which left the oracle against DPDK's own cache-line annotations nothing to score. rte_eal_remote_launch takes the entry first and its argument next, the shim's own convention, so only the name is new. The rte_malloc family carries its alignment in an operand, read as aligned_alloc's is, with 0 meaning the EAL's default, which is left to the heap-align option. rte_ring_create, rte_ring_create_elem and rte_stack_create hand back a fresh object that exists nowhere else; counting them as allocations is a reading of the API, stated here, not a fact of the module.
A record written as a whole -- assigned, copied, initialised -- marks
every field's store list as saying nothing, since the write puts values
there that no store lists. `c = {0}` and `c = { .n = 8 }` are such
writes in form only: the front end emits them as a copy out of a private
constant global, and that constant holds zeros, integers and nulls, no
address of anything. The same goes for a global record whose initialiser
names no object, and for a store of such a constant. Each is now read
like a memset: the list stands, and a pointer field set on several paths
after the zeroing is one of the places those paths name.
Found on DPDK's ring test, where `struct ring_params ring_params = {0}`
cost every field of the record that carries the ring: 91 of the test's
unplaced accesses came through that one zeroing, 71 of them the ring's
own heads and tails. A constant that does hold an address, `c = { &u }`,
still marks the record written whole, and a copy out of a non-constant
global still does, since what it holds is not known.
A call is an object when every return of the function behind it hands back one allocation: Phoenix's checked xcalloc. A constructor that picks between allocators -- DPDK's test_ring_create calls rte_ring_create or rte_ring_create_elem by the element size -- has two, and its result was "the result of a call that is not an allocation", unplaced. Each return of such a function stores into the front end's `__retval` slot and loads it back, so the sites are read through that slot's stores as well as through a direct return, and the result is one of the sites: the one-of machinery of the field and slot listings, with the same cap and the same rule that a set is never definite and never owns an element. On the ring test the 71 accesses through the ring's handle move from "call result" to "one of several objects"; on memcached nothing changes, since its wrappers each have one site.
The walk read a member access only where it stood: `d->head` in the routine that moves a ring's head was placed on `d`, and `d` came from the caller as `&r->prod`, a member's address, which no rule took apart. The access was "otherwise", and with it every head and tail of DPDK's ring in the routines that move and update them, and every access of memcached's extstore IO threads, which take `&e->io_threads[i]` as their argument and read the engine from a field of it. A member's address is so many bytes into the record it is a member of, by the data layout, and the walk goes on from the record's address; a union's members all start where the union does. On memcached the extstore engine gains 69 definite pairs and the crawler's module 18, all read: lock-serialised counters, fields set once per page beside fields read per request, and start-up writes in a loop that runs after the worker threads exist; 162 fewer accesses are unplaced (4,103).
The summary counts accesses by the source the walk stopped at, which says what kind of thing is unplaced and nothing about which. Reading a count back to the code meant guessing. The pass now takes list-accesses=true and emits a remark per field access: the field, its offset, the object it was placed on -- as one of several, at an element not known, one element per thread -- the thread and function, the rate and the direction; or the cause it was not placed for. The notes on a possible pair name both offsets as well, so pairs can be sorted by region without the field names, which unions make ambiguous. The object naming moves into a helper the report and the listing share. The listing is what found the three gaps closed before this: the zeroed record, the two-site constructor and the member's address.
An operation's fallback_impl was trusted until lowering rebuilt the call: a symbol that names nothing, a function annotated for another operation, or one taking a different number of arguments surfaced only as a lowered call that failed to verify or to link. Each operation now checks, as a symbol use, that fallback_impl names a cir.func carrying the shim's annotation for the same operation (or a substitute's, wtc_impl_<name>_<operation>) and taking the arguments lowering passes. A push or pop also checks that source_queue_type points to the object the queue type names, since lowering casts the handle back to it. Every operation now declares that it reads and writes memory, which is what the call it came from did; nothing finer is claimed yet.
The verdict on each push and pop said what kind of queue it reached, not which one, so a pass that has to see every operation of one queue -- to prove that its consumer only borrows what it pops, or to check a declared topology -- had nothing to group them by. The ownership analysis now numbers each queue root in discovery order, writes the number on the root as wtc.queue_id, and gives an operation that can only act on that root the same number as its queue_id. An operation that may act on several queues names none, so that no later pass mistakes it for an operation on just one.
The operations now verify against the calls they lower to, but say nothing about concurrency: whether a substitution is correct depends on what the program was entitled to rely on, and nothing writes that down. This proposes, for discussion, a minimal contract per operation kind -- a push happens before the pop that returns its item, linearisable per queue, FIFO per producer, no ordering against other objects, no progress guarantee -- with room for an ordering attribute later. Ported from the stacked pull request that introduced it, with the substitution it names changed to the constructor tag this branch uses.
A pass after specialisation that needs to know what an operation will execute -- whether it takes a mutex, whether it allocates, whether its popped value lives in the ring -- had to find the queue's constructor again and read its tag. The specialisation pass now does that once: it reads back the tag each numbered queue's one constructor passes, after its own rewriting, and writes it on every operation of that queue as #wtc.impl. Anything it cannot be sure of -- no constructor, several, a tag that is not a constant, an operation that may act on several queues -- is recorded as the general-purpose implementation. With rewrite=false the pass only records, which is how wtc-cc now builds an unmodified program, so the attribute is there in every build. The contract decisions/0003 proposed is taken as its option 2 and written into the operations' descriptions.
The runtime is what a substitution puts into a program, and a defect in it races rather than slows down, yet CI only compiled the shim. A new job compiles every runtime header on its own and runs the runtime's tests three ways: plain, under ThreadSanitizer, and with the contract checks compiled in. The last two include the test that two producers are caught, so a checker that stopped firing fails the job rather than passing it. The runner's address randomisation is narrowed first, because the sanitizer of LLVM 18 cannot map its shadow memory on a 6.x kernel's default.
The gate found that the queue's cost is not its synchronisation but the shared_ptr the shim allocates for every element: the same ring with the element held inline runs ten to forty times faster. Handing an inline element to a consumer without a copy needs a pop that leaves it where it is, and that needs the ring to control each element's lifetime. Slots become raw storage. An element is constructed in place by push and destroyed by the pop that takes it, so a new block no longer constructs a slot object per entry and the element type needs no default constructor. try_pop_borrow returns the next element's address instead of moving it out; the element lives until the consumer's next pop of either kind, or the destructor. The lent slot is always in the consumer's current block, which it has not announced as left, so the producer cannot reuse it. With WTC_CHECK a released slot is overwritten with 0xDB, so a pointer kept past the next pop reads garbage rather than a plausible element. Tests cover an element type without a default constructor, every element destroyed exactly once however it left the ring, a lent element surviving pushes and a producer that moves on to new blocks, and a two-thread stress run that borrows every element, plain and under ThreadSanitizer.
The specialised queue still allocated a shared_ptr for every element, which the gate measured as the larger of its two costs. A third tag, spsc_inline, holds the elements in the ring itself: push copies the element into its slot, and try_pop hands it out in place through a shared_ptr that owns nothing -- the aliasing constructor over an empty owner -- so neither side allocates. The pointer is valid until the consumer's next try_pop, which is what the compiler will have to prove before it writes this tag. The element is destroyed in place at the next pop rather than when the last shared_ptr to it goes, so only a trivially destructible type is held inline; any other type runs as spsc, whatever the tag says. With WTC_CHECK every element handed out shares a per-queue guard instead, and a try_pop, or the destructor, that finds it still referenced aborts naming the queue. The benchmarks gain the shim with this tag and the bare ring popped by borrowing, and the pipeline workload a hand-written spsc_inline flavour, so the prize can be measured before any analysis claims it.
Holding a queue's elements inline in the ring and handing each out in place is only correct if the program cannot tell: the element is destroyed at the consumer's next pop, and the shared_ptr it receives owns nothing. A new pass, --wtc-borrow-analysis, proves that for every pop of a queue already proved spsc, from the stack slot the compiler builds the popped shared_ptr in. The slot may be used only by the pop, its lifetime markers and members of shared_ptr that neither copy nor share it: operator bool, get, *, ->, comparison with nullptr and the destructor, recognised by their demangled names so that libstdc++ and libc++ spellings both count. A pointer into the element may be read and written through, stepped into members and elements, compared, kept in a local slot whose address goes nowhere, pushed on to another queue, which copies it, or passed to a function with a body that does no more with it. Nothing in the slot's scope may pop the same queue, directly or through a call. A queue passes only if all its pops do; the root then carries wtc.borrowed and each pop `borrowed`, and otherwise a remark names the first use in the way. Both queues of the compiler-visible pipeline benchmark pass at -O0 and at -O2. Tests cover five consumer shapes that pass at both levels, eight that must not (a copy, a stored pointer, a second pop, an opaque call, use_count, a returned pointer, a call that pops, a member function that keeps `this`), libc++'s names, and a queue that is not spsc.
The borrow proof had no consumer: a queue whose every pop only borrows was still tagged plain spsc, and its elements still allocated. The specialisation pass now writes spsc_inline for a queue proved spsc and borrowed, provided its element type may be held inline. The shim says which types may, by calling an empty function annotated wtc_queue_inline_ok from its constructor exactly when the type is trivially destructible and copyable; it is a call rather than an argument of the constructor's annotation because ClangIR does not carry annotation arguments yet. A borrowed queue of any other type stays plain spsc, with a remark saying why. The force-inline option, and wtc-cc's --force-inline, tag a queue regardless, for the negative control. wtc-cc now runs the borrow analysis in every build. On the pipeline benchmark both queues are borrowed, and the compiler's build is byte for byte the build whose source asks for spsc_inline itself, which the flavour script now checks instead of the spsc build.
A program that passes spsc or spsc_inline to a queue's constructor makes a claim, and the compiler took it on trust: the specialisation pass left a declared tag alone, and a second producer surfaced at run time as corruption, or under WTC_CHECK as an abort. The ownership analysis now checks the claim against the threads it found. When the claim is certainly broken -- two threads the module accounts for on one side, or one thread certainly standing for several -- the build fails with an error at the constructor and a note at each thread's first access, naming the function the thread starts in and where it is spawned. When the verdict is not spsc but nothing is certain, for instance because a producer is handed to code the module cannot see, a warning says why the claim cannot be checked; when the verdict agrees, a remark says it is verified. Finding a queue's constructors and reading its tag move into Utils, shared by the analysis and the specialisation pass.
The dialect knew which implementation each queue operation runs, but nothing said what that implementation guarantees, and a program had no way to state that a function must not block. Each wtc operation now answers, through ProgressOpInterface, whether its algorithm can wait for another thread and whether it may call the allocator, from one table keyed by the implementation recorded on it (decisions/0004). A function marked WTC_LOCK_FREE (shim/progress.hpp) promises that nothing it reaches blocks, and --wtc-check-progress, which wtc-cc runs in every build, holds it to that: a wtc operation that blocks or allocates, a known blocking call -- the allocator, the threading library's waits, input and output, sleeping, throwing -- and any call it cannot see into fail the build, with the call chain from the function to the operation as notes. The shared_ptr of a borrowed pop of an inline queue owns nothing, so its members are not followed; that is what lets a consumer of such a queue pass. The check is a necessary condition for lock-freedom, not a proof: a loop spinning on an atomic is not judged. examples/progress_consumer.cpp shows the four outcomes one source can have: it passes as the compiler builds it, and fails with the general implementation, when it keeps the element, or when it logs.
A producer that pushes in a loop pays a release store per element, and under load the consumer's cached count goes stale as often, pulling the published counter's line across. The ring now separates the two halves of push: push_deferred writes the element and counts it, publish makes everything counted so far visible with one release store, and push is the two in sequence. Elements published together become visible together and in order, which is what one-by-one pushes would have shown a consumer that happened to look away; a block inserted on the way is covered by the same store, since the consumer reads a block's next pointer only after acquiring a count beyond it. A unit test checks that deferred elements are invisible until publish and then arrive in order across block boundaries, and a stress run publishes once per 64 pushes, plain and under ThreadSanitizer.
A producer that pushes in a loop pays the queue's synchronisation once per element: a mutex acquisition on the general-purpose queue, a release store on the rings. The shim gains the batched form of those pushes: batch_begin, push_deferred for each element, batch_end. The elements become visible together and in order at batch_end, or every WTC_BATCH_K (64) elements, and the cost is paid once per run. On the general-purpose queue the nodes and elements are allocated outside the lock and spliced in under one acquisition, giving the list the same pushes made one by one would have built; on the rings each run is one release store. The batch is raw storage that batch_begin brings to life, so the compiler can set it aside without running a constructor. The three members are marked used, so they exist in every module that uses the queue and the compiler can call them. The dialect gains queue_batch_begin, queue_push_deferred and queue_batch_end, lifted from those calls and lowered back to them, verified like the other operations, and placed in the progress table. All three are the producer's side for the ownership analysis: a run written by hand is a producer like any other, and a test fails if the verdict ignores it. wtc-cc no longer prints the whole operation after each diagnostic, which repeated a queue's type on every line.
A loop that pushes to a queue on every iteration pays the queue's synchronisation on every iteration, and no library can do better: it sees one push at a time. The compiler sees the loop. A new pass, --wtc-batch-pushes, which wtc-cc runs right after lifting, rewrites such a loop into one batched run: batch_begin before it, a deferred push in place of the push, batch_end after it, with the run's storage in a slot at the top of the function. Elements then reach consumers later -- every WTC_BATCH_K pushes and at the end -- but each still arrives once, in its producer's order, and after the push that made it (decisions/0003). The delay is the program's choice to make, so only a function marked WTC_BATCH (shim/progress.hpp) is touched, and in it only a loop where nothing can be waiting for an element not yet published and nothing can leave without passing batch_end: one push, made on every iteration, to a queue the loop does not change; no other queue or mutex operation, no atomic or volatile access, fence or inline assembly, in the loop or in what it calls; no return, goto or continue; no call that may throw, that goes through a pointer, or that enters the threading library. Each refusal is a remark naming the reason and where it is. A run lost to an allocation failure in the middle is accepted, and said so. Tests cover three loop shapes that are batched and nine that are not, and the batched producer runs correctly with the compiler's choice of implementation, the general one, and a declared spsc.
Batching is a trade, throughput against how long an element waits, and neither half had a workload. W7 is a producer pushing in a plain loop and a consumer checking order and, for 64-byte elements, the delay of every 64th element, with the queue a global the compiler can reason about. build_w7_flavours.sh builds it twelve ways through wtc-cc -- general-purpose, declared spsc and compiler-chosen inline, each with and without the WTC_BATCH mark, for 8- and 64-byte elements -- and checks that each build carries the implementation and the batching its name says. campaign.sh gains the w7 leg that runs them interleaved. The pipeline workload's header now lists its fourth build, hand-inline.
The summaries of the gate's legs print "smt on", read from the machine- wide smt/control, while the legs ran in reservations that take each reserved core's sibling offline one by one, which that switch does not show. A reader of the results could not tell which was true. The environment check now looks at what matters: whether every reserved core's other hardware thread is offline or itself reserved, and fails the leg when one is online outside the set. meta.json records that answer and the load average beside the switch. campaign.sh also gains w1mvp, the throughput leg reduced to the five variants the MVP quotes, so that it fits a short quiet window.
Three legs on the lab server under the gate's protocol: a reservation of three cores whose SMT siblings were offline, governor at performance, turbo off, five interleaved repetitions of fresh processes. w1mvp is the throughput of the five queue variants the MVP quotes, w3wtc the compiler-visible pipeline in its four builds, closed and paced at two million records a second, and w7 the producer loop with and without batching for the three implementations at 8 and 64 bytes. Every run of a leg produced the same checksum. The leg's own check reported three, because it compared whole lines and each line also names the implementation the run carried; it now compares the values. The inline rings are bimodal in w1mvp, which five repetitions cannot settle, and the result directory says so rather than quoting a median.
…e it The README described the gate's one slice; the dialect now carries a contract and a progress table, and the compiler borrows, inlines, batches and checks two kinds of promise. The status table says so, the glossary gains the terms, and docs/mvp.md walks through each of the four things a library cannot do, with the command that shows it and the remarks or errors it prints, and then what the measurements say: on the pipeline benchmark the compiler's inline choice runs at 8.4 times the general-purpose queue and 2.4 times the spsc tag, and batching gains 1.9 times on the general-purpose queue and 3.3 times on the inline ring, at a delay it states. Where five repetitions could not settle a bimodal ring, it gives the range instead of a ratio. docs/gate.md notes the re-measured first row, 2.64 times against 3.07, and that "smt on" in its summaries was the machine-wide switch while the reserved cores' siblings were offline.
This was referenced Oct 19, 2026
ZweZeya
approved these changes
Oct 19, 2026
Collaborator
Author
|
After this merge, If you have git fetch origin
git switch main
git reset --hard origin/mainA local branch started from the old |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three lines of work that had been developed off-line, rebased onto
main, one self-contained commit per change. Every commit builds and passescheck-wtc; every test a commit adds fails on the code before it.What it contains
The gate (4 commits). The SPSC ring and the topology tag in the shim; the queue ownership analysis by thread instances under call strings, with lifting and lowering that read what the compiler actually emits; the benchmarks and the measurement protocol;
docs/gate.mdwith the results.Memory layout by owner (31 commits). The thread-instance walk extracted and shared; a false-sharing diagnosis with
definite,possibleandcoldverdicts over globals, heap objects by allocation site, per-thread array elements and "one of several" places, each round closed by an independent review whose findings are fixed with tests; the alignment-by-owner pass; a census of hand padding in 17 codebases; checks against DPDK's own cache-line annotations.docs/corpus.mdrecords what it finds and what it cannot see.The MVP (17 commits plus measurements). What the dialect now lets the compiler do that a library cannot:
--wtc-borrow-analysisproves that every pop of an spsc queue only borrows its element; the queue is then taggedspsc_inline, which holds elements in the ring and allocates nothing per element.--wtc-batch-pushesturns a loop of pushes in aWTC_BATCHfunction into one batched run, published every 64 elements, where nothing in the loop can wait for an unpublished element.WTC_LOCK_FREEfunction that can reach a blocking operation fails the build with the call chain.Underneath: each operation is verified against the call it lowers to, carries the contract of
decisions/0003, and answers throughProgressOpInterfacewhat it guarantees under the implementation chosen for it (decisions/0004). The runtime ring holds elements in raw storage, lends the last one popped, and publishes per element or per run; CI runs it under ThreadSanitizer and with its contract checks.docs/mvp.mdwalks through each with the command that shows it.Relation to the open stack #2 to #9
That stack and this branch reimplemented the same first slice in parallel. This branch keeps its own analysis and substitution and ports what the stack had that it did not: the operation verifiers and
decisions/0003(both under their original authorship), thequeue_ididea, raw slot storage and the ThreadSanitizer job. The stack is closed with a note on each PR saying where its content went.Measurements
Lab server, 19 October 2026, under the gate's protocol (three reserved cores with siblings offline, governor fixed, turbo off, five interleaved repetitions); results in
benchmarks/results/2026-10-19-*, read indocs/mvp.md.The compiler's pipeline build is byte for byte the hand-tagged inline build. The gate's first row re-measured at 2.64× (was 3.07×). Batching costs delay, which
docs/mvp.mdstates per implementation.Not done
Queues reached through a struct member or the heap are still
unknownto the queue analysis; the analysis assumes one translation unit; the progress check is a necessary condition for lock-freedom, not a proof.