Substitute spsc_queue for a global queue proven SPSC, and measure it

README.md

@@ -20,7 +20,7 @@ what exists is one vertical slice and it is not finished:
 | Lifting CIR calls into the dialect | works on hand-written IR; the C++ path is not yet exercised end to end |
 | Queue ownership analysis: SPSC / MPSC / SPMC / MPMC | prints a verdict; sound for direct calls, UNKNOWN wherever a function's address escapes; thread lifetime and escaping queues not modelled |
 | Lowering back to CIR | faithful round trip; crashes on operations that carry no source-type attribute |
-| Specialised implementations | none yet |
+| Specialised implementations | an unbounded SPSC queue, substituted for a global `wtc::queue<T>` proven SPSC; 3.6x throughput on one producer and one consumer (`benchmarks/`) |
 
 `docs/audit.md` lists the known defects in full. Do not build anything on top of
 the analysis verdict until they are closed.

benchmarks/README.md

@@ -14,4 +14,36 @@ for, and the hand-written specialised implementation from `runtime/`. The
 second one is the ceiling: the compiler cannot beat code a human wrote for
 this exact case, it can only reach it without the human.
 
-Nothing here yet.
+## What is here
+
+- `queue_bench.cpp`: one producer and one consumer move 5M ints through a
+  global `wtc::queue<int>`. The consumer checks the count, the sum and the
+  order, and the run fails if any is wrong. Nothing in it names the
+  substitute: the shim makes `wtc::runtime::spsc_queue<int>` available.
+- `spsc_direct.cpp`: the same workload written by hand against
+  `runtime/spsc_queue.hpp`, with its allocation-free pop. This is the ceiling.
+- `run.sh`: builds `queue_bench.cpp` twice from the same CIR, once as
+  written and once with `--select-queue-impl` in the pipeline, plus
+  `spsc_direct`, through the pinned toolchain down to a native binary, and
+  reports the median of `RUNS` runs of each. Set `LLVM_BUILD` to the pinned
+  toolchain (default `/opt/llvm`, as in the image) and `WTC_BUILD` to this
+  project's build tree.
+
+Only throughput is measured so far, not the tail, and runs are not pinned.
+
+## Measured
+
+At the pin (`1ebf89d8c5e6`), 2026-10-18, on an Apple M1 Pro (8 cores,
+16 GB), macOS, threads not pinned, five runs each, median:
+
+| variant | items/s | vs baseline |
+| --- | ---: | ---: |
+| baseline: `wtc::queue<int>` | 2.07M | 1.0x |
+| optimized: substituted by `--select-queue-impl` | 7.44M | 3.6x |
+| direct: `spsc_queue` by hand | 43.1M | 20.9x |
+
+The gap between optimized and direct is almost all one allocation per item.
+The substitute has to keep `wtc::queue<T>::try_pop`'s interface, which
+returns a `std::shared_ptr<T>`, so every pop builds one; the hand-written
+consumer uses `try_pop_optional` and allocates nothing. Closing it means
+changing what the dialect's pop returns, not the queue.

benchmarks/queue_bench.cpp

@@ -0,0 +1,66 @@
+// One producer, one consumer, kNumItems ints through a global
+// wtc::queue<int>. Built twice from this one source by run.sh: once with the
+// queue as written, once after --select-queue-impl has substituted
+// wtc::runtime::spsc_queue<int> for it. Nothing here names the substitute;
+// the shim makes it available.
+//
+// The consumer checks what it receives: a substitution that drops,
+// duplicates or reorders an item fails the run instead of timing it.
+
+#include "queue.hpp"
+#include "thread.hpp"
+
+#include <chrono>
+#include <cstdio>
+
+constexpr long kNumItems = 5'000'000;
+
+wtc::queue<int> g_queue;
+
+long g_received = 0;
+long g_sum = 0;
+long g_out_of_order = 0;
+
+void producer() {
+    for (long i = 0; i < kNumItems; ++i) {
+        int value = static_cast<int>(i);
+        g_queue.push(value);
+    }
+}
+
+void consumer() {
+    long expected = 0;
+    while (g_received < kNumItems) {
+        auto value = g_queue.try_pop();
+        if (value) {
+            if (*value != expected)
+                ++g_out_of_order;
+            g_sum += *value;
+            ++expected;
+            ++g_received;
+        }
+    }
+}
+
+int main() {
+    auto start = std::chrono::steady_clock::now();
+
+    auto producer_thread = wtc::spawn(producer);
+    auto consumer_thread = wtc::spawn(consumer);
+    producer_thread.join();
+    consumer_thread.join();
+
+    auto end = std::chrono::steady_clock::now();
+    double seconds = std::chrono::duration<double>(end - start).count();
+
+    long expected_sum = kNumItems * (kNumItems - 1) / 2;
+    if (g_received != kNumItems || g_sum != expected_sum || g_out_of_order != 0) {
+        std::printf("FAIL received=%ld sum=%ld (expected %ld) out_of_order=%ld\n",
+                    g_received, g_sum, expected_sum, g_out_of_order);
+        return 1;
+    }
+
+    std::printf("items=%ld elapsed_ms=%.3f items_per_sec=%.0f\n",
+                kNumItems, seconds * 1000.0, static_cast<double>(kNumItems) / seconds);
+    return 0;
+}

benchmarks/run.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+# Builds the queue benchmark two ways from the same source -- baseline
+# (wtc::queue<int>, lifted and lowered back unchanged) and optimized (the
+# same pipeline plus --select-queue-impl) -- and spsc_direct, the
+# hand-written ceiling. Runs each RUNS times and reports the median.
+#
+#   LLVM_BUILD  the pinned toolchain, built or installed; default /opt/llvm,
+#               where the toolchain image puts it
+#   WTC_BUILD   this project's build tree; default <repo>/build
+#   RUNS        runs per variant; default 5
+set -euo pipefail
+
+here="$(cd "$(dirname "$0")" && pwd)"
+repo="$(dirname "$here")"
+LLVM_BUILD="${LLVM_BUILD:-/opt/llvm}"
+WTC_BUILD="${WTC_BUILD:-$repo/build}"
+RUNS="${RUNS:-5}"
+out="$WTC_BUILD/benchmarks"
+mkdir -p "$out"
+
+for tool in clang cir-opt mlir-translate llc; do
+  if [ ! -x "$LLVM_BUILD/bin/$tool" ]; then
+    echo "no $tool in $LLVM_BUILD/bin; set LLVM_BUILD to the pinned toolchain" >&2
+    exit 1
+  fi
+done
+wtc_opt="$WTC_BUILD/bin/wtc-opt"
+[ -x "$wtc_opt" ] || { echo "no wtc-opt in $WTC_BUILD/bin; set WTC_BUILD" >&2; exit 1; }
+
+# A clang built from source does not know where macOS keeps the SDK, and the
+# final link needs the system's C++ runtime either way.
+sysroot=()
+link_cxx=(c++)
+if [ "$(uname)" = Darwin ]; then
+  sysroot=(-isysroot "$(xcrun --show-sdk-path)")
+  link_cxx=(xcrun clang++)
+fi
+
+emit_cir() { # <source> <out.cir>
+  "$LLVM_BUILD/bin/clang" -x c++ -std=c++17 -O1 -I "$repo/shim" "${sysroot[@]}" \
+    -fclangir -emit-cir "$1" -o "$2"
+}
+
+# cir-to-llvm leaves CIR types behind in attributes, even for plain clang
+# output, so mlir-translate has to be told not to choke on them.
+link() { # <in.cir> <name>
+  "$LLVM_BUILD/bin/cir-opt" "$1" --cir-to-llvm -o "$out/$2.llvm.mlir"
+  "$LLVM_BUILD/bin/mlir-translate" --allow-unregistered-dialect --mlir-to-llvmir \
+    "$out/$2.llvm.mlir" -o "$out/$2.ll"
+  "$LLVM_BUILD/bin/llc" -O2 -filetype=obj "$out/$2.ll" -o "$out/$2.o"
+  "${link_cxx[@]}" "$out/$2.o" -o "$out/$2" -lpthread
+}
+
+emit_cir "$here/queue_bench.cpp" "$out/queue_bench.cir"
+
+"$wtc_opt" "$out/queue_bench.cir" --mlir-print-op-on-diagnostic=false \
+  --lift-cir-to-wtc --queue-ownership-analysis --lower-wtc-to-cir \
+  --reconcile-unrealized-casts -o "$out/queue_bench_baseline.cir" > /dev/null
+link "$out/queue_bench_baseline.cir" queue_bench_baseline
+
+"$wtc_opt" "$out/queue_bench.cir" --mlir-print-op-on-diagnostic=false \
+  --lift-cir-to-wtc --queue-ownership-analysis --select-queue-impl --lower-wtc-to-cir \
+  --reconcile-unrealized-casts -o "$out/queue_bench_optimized.cir" > /dev/null
+# Linkage keywords differ by target (ELF globals are dso_local), so match
+# only the name and the type.
+grep -qE 'cir\.global .*@g_queue = #cir\.zero : !rec_wtc3A3Aruntime' "$out/queue_bench_optimized.cir" \
+  || { echo "--select-queue-impl did not substitute g_queue" >&2; exit 1; }
+link "$out/queue_bench_optimized.cir" queue_bench_optimized
+
+emit_cir "$here/spsc_direct.cpp" "$out/spsc_direct.cir"
+link "$out/spsc_direct.cir" spsc_direct
+
+for bin in queue_bench_baseline queue_bench_optimized spsc_direct; do
+  rates=()
+  for _ in $(seq "$RUNS"); do
+    line="$("$out/$bin")"
+    rates+=("$(sed -n 's/.*items_per_sec=\([0-9]*\).*/\1/p' <<<"$line")")
+  done
+  median="$(printf '%s\n' "${rates[@]}" | sort -n | sed -n "$(( (RUNS + 1) / 2 ))p")"
+  echo "$bin: median $median items/s over $RUNS runs (${rates[*]})"
+done

benchmarks/spsc_direct.cpp

@@ -0,0 +1,61 @@
+// The same workload as queue_bench.cpp, written by hand against the
+// substitute with its allocation-free pop. This is the ceiling: the
+// compiler cannot beat it, only reach it without the human.
+
+#include "../runtime/spsc_queue.hpp"
+#include "thread.hpp"
+
+#include <chrono>
+#include <cstdio>
+
+constexpr long kNumItems = 5'000'000;
+
+wtc::runtime::spsc_queue<int> g_queue;
+
+long g_received = 0;
+long g_sum = 0;
+long g_out_of_order = 0;
+
+void producer() {
+    for (long i = 0; i < kNumItems; ++i) {
+        int value = static_cast<int>(i);
+        g_queue.push(value);
+    }
+}
+
+void consumer() {
+    long expected = 0;
+    while (g_received < kNumItems) {
+        auto value = g_queue.try_pop_optional();
+        if (value) {
+            if (*value != expected)
+                ++g_out_of_order;
+            g_sum += *value;
+            ++expected;
+            ++g_received;
+        }
+    }
+}
+
+int main() {
+    auto start = std::chrono::steady_clock::now();
+
+    auto producer_thread = wtc::spawn(producer);
+    auto consumer_thread = wtc::spawn(consumer);
+    producer_thread.join();
+    consumer_thread.join();
+
+    auto end = std::chrono::steady_clock::now();
+    double seconds = std::chrono::duration<double>(end - start).count();
+
+    long expected_sum = kNumItems * (kNumItems - 1) / 2;
+    if (g_received != kNumItems || g_sum != expected_sum || g_out_of_order != 0) {
+        std::printf("FAIL received=%ld sum=%ld (expected %ld) out_of_order=%ld\n",
+                    g_received, g_sum, expected_sum, g_out_of_order);
+        return 1;
+    }
+
+    std::printf("items=%ld elapsed_ms=%.3f items_per_sec=%.0f\n",
+                kNumItems, seconds * 1000.0, static_cast<double>(kNumItems) / seconds);
+    return 0;
+}

docs/audit.md

@@ -191,6 +191,32 @@ this, and treat a call with no result as a case to handle rather than one to
 assume away. None of that is attempted in this document; it is the work, not
 the diagnosis.
 
+## The first specialisation: SPSC queues — October 2026
+
+`--select-queue-impl` substitutes `runtime/spsc_queue.hpp` for a global
+`wtc::queue<T>` whose verdict is SPSC. Built from 253a2bc, with these
+changes measured or found on the way:
+
+- **A shared destructor was retargeted for every queue.** Two globals of the
+  same type share a destructor, and the earlier version rewrote every
+  reference to it, so substituting one queue registered the substitute's
+  destructor for the other too. Retargeting now follows the substituted
+  global's own `__cxa_atexit` registration
+  (`test/Transforms/SelectQueueImplementation/global-spsc.mlir`).
+- **Any other use of the global was left with the wrong type.** The global's
+  type changes, so a use the pass does not retarget would read the new
+  storage as the old type. Such a global is now not substituted, and the
+  remark names the use (`refusals.mlir`).
+- **The substitute was bounded and dropped items when full.** It is now
+  unbounded, like the queue it replaces.
+- **The program had to name the substitute.** The shim now makes it
+  available, through an anchor the pass erases.
+- **Alignment.** The data layout of a CIR record type does not see `alignas`;
+  `cir.record_layouts` does, and is what the pass uses.
+
+Only globals are substituted. A queue on the stack or reached only through
+parameters keeps its implementation.
+
 ## Infrastructure
 
 Fixed before the move into this repository:

mlir/include/WTC/Transforms/Passes.h

@@ -6,6 +6,7 @@ namespace mlir::wtc {
     void registerLiftCIRToWTCPass();
     void registerLowerWTCToCIRPass();
     void registerQueueOwnershipAnalysisPass();
+    void registerSelectQueueImplementationPass();
     void analyseQueues(ModuleOp module);
 }
 

mlir/include/WTC/Transforms/Utils.h

@@ -19,6 +19,12 @@ namespace mlir::wtc {
 
     bool isWTCAnnotatedCall(cir::CallOp op, StringRef annotation);
 
+    bool funcHasAnnotation(cir::FuncOp funcOp, StringRef annotation);
+
+    // The element type of a queue whose push is pushFunc: the pointee of the
+    // reference it takes the value by.
+    Type deriveQueueElementTypeFromPush(cir::FuncOp pushFunc);
+
     SemanticOpKind getSemanticKind(cir::FuncOp funcOp);
 
     // The argument through which funcOp returns its value indirectly, if it

mlir/lib/WTC/Transforms/CMakeLists.txt

@@ -2,6 +2,7 @@ add_mlir_library(MLIRWTCTransforms
   LiftCIRToWTC.cpp
   LowerWTCToCIR.cpp
   QueueOwnershipAnalysis.cpp
+  SelectQueueImplementation.cpp
   Utils.cpp
   LINK_LIBS PUBLIC
   MLIRWTC
@@ -11,6 +12,7 @@ add_mlir_library(MLIRWTCTransforms
   # wtc-opt linked MLIRCIR itself; a second consumer would not be so lucky.
   MLIRCIR
   MLIRCIRInterfaces
+  MLIRDataLayoutInterfaces
 
   MLIRIR
   MLIRPass