Decouple thread fusion from the Read/Write Operation authoring contract

[johnnynunez]

What

Memory Operations no longer implement template <uint ELEMS_PER_THREAD> exec(...) overloads. Every Operation authors only its scalar exec(), and the vectorized multi-element machinery lives in exactly one place: ThreadFusionAdapter in core/execution_model/thread_fusion.h synthesizes the wide load/store and per-element packing/unpacking.

Thread-fusion capability is now a declarative opt-in on the Operation, defaulting to disabled:

• contiguous_data(params) — for Operations whose exec() is a plain per-thread access into a pitch-linear RawPtr (PerThreadRead/PerThreadWrite/TensorRead/TensorWrite: one-line identity accessor each).
• forwarded_access(thread, params) -> ForwardedAccess<Operation> — for wrapper Operations that only remap the thread and/or select per-plane data and delegate the access to a wrapped Operation (BatchRead/BatchWrite, CircularBatchRead/Write, CircularTensorRead/Write). Their scalar exec() is now written in terms of the same hook, removing the duplicated if constexpr (THREAD_FUSION) dispatch blocks.

ReadOperation/WriteOperation parents and the DECLARE_READ/WRITE_PARENT macros no longer take an enum TF parameter nor define THREAD_FUSION; capability is derived via isThreadFusionCapable<Op> (hook detection, recursing through forwarded targets). ELEMS_PER_THREAD now appears only in thread_fusion.h.

Why

• Authoring a Read/Write Operation previously required understanding and plumbing thread-fusion machinery (a TF template argument, THREAD_FUSION propagation, templated exec signatures using ThreadFusionType) even for ops that never vectorize. Six ops opted out with TF::DISABLED purely to avoid that machinery.
• The wide load/store logic was duplicated across ~4 files and ~46 ELEMS_PER_THREAD mentions; a single adapter removes the duplication and makes the vectorized path testable and reviewable in one place.
• The macro-generated dispatchers are enable_if-constrained on the operation-data type, so they stay SFINAE-friendly for detection traits while their bodies remain lazily checked — Operations may still implement a non-constexpr scalar exec() (e.g. Fp8TokenDequantRead).

Behavior

• Public API unchanged: executeOperations<...>, TransformDPP<PA, TF::ENABLED/DISABLED> and all existing call sites compile and behave identically. TF is still silently disabled for ReadBack chains, fused reads, and non-capable endpoints.
• Codegen preserved: SASS of the thread-fusion benchmark (sm_121) has an identical global load/store instruction mix before/after (e.g. 48x LDG.E.128, 73x STG.E.128); no kernel grew, 119/151 kernels got 8–32 instructions smaller (mostly the non-divisible remainder variants). Benchmark TF-vs-normal geomean speedups are unchanged (1.067→1.059, 1.024→1.025, 1.010→1.011).
• isThreadDivisible now passes the IOps directly to num_elems_x instead of re-constructing an OperationData from .params. This is behavior-identical and additionally makes TF::ENABLED compile for BatchWrite (array ParamsType), so TF now works for batch read→write pipelines.
• Authoring-contract break (main is the API-may-break branch): out-of-tree Operations instantiating ReadOperation<..., TF::X, Child> / WriteOperation<..., TF::X, Child> must drop the TF argument (loud compile error, one-token fix). All in-repo instantiations updated.

Tests

• New tests/thread_fusion/test_thread_fusion_exec.h: executes each pipeline with TF::DISABLED and TF::ENABLED on both backends and compares outputs — contiguous ops (uchar/uchar2/uchar3/float/float4), non-divisible widths (remainder path), type-changing pipelines, Tensor ops, and the batch/circular wrappers. Comments document the pre-existing constraint that Tensor rows are unpadded, so TF on Tensors needs divisible widths.
• Full suite passes on CPU and CUDA backends; benchmark_thread_fusion passes on both backends (its compareAndCheck validates TF vs non-TF equality across all supported element types).

Docs

• .github/skills/fkl-implementing-operations/SKILL.md: new section describing the two hooks and stating that ELEMS_PER_THREAD overloads must never be hand-written.
• CLAUDE.md: thread-fusion bullet updated to the new mechanism.

🤖 Generated with Claude Code