benchdnn: matmul: paneled (periodic) fast native reference (--mode-modifier=A)

[karturov]

1. Problem

benchdnn GPU matmul correctness runs can be dominated by the CPU "golden" reference. For configs with no fast CPU primitive (e.g. f8/f4/mx, some weight-decompression cases) benchdnn falls back to the scalar native reference, which is O(MB·M·N·K). On large layers this reference alone takes minutes per case, making broad sweeps impractical.

Currently, in oneDNN functional testing, we have to trim large shapes significantly (see #4355). While it works ok, two potential issues remain:

1. There is a chance that trimmed shapes result in other strategies were selected, hence a testing gap
2. We need to repeat this trimming every time we add a new set of models to testing

2. Solution

Fill the matmul inputs so the result is periodic along M and N (K kept full). Both the GPU primitive and the reference then produce a periodic output, so the native reference only needs to compute one representative panel_m × panel_n panel and broadcast it. Reference cost drops from O(MB·M·N·K) to ~O(panel_m·panel_n·K); the GPU primitive still runs the full shape. Panel width ≥128 and aligned to dst scale/zero-point groups so no block/group quantization parameter straddles a panel boundary.

Opt-in via --mode-modifier=A (env var DNNL_BENCHDNN_MATMUL_PANEL_FILL=1 kept as a deprecated alias). It does not disable --fast-ref: paneling engages only on the native-reference path. When a fast CPU primitive reference exists it runs the full shape vectorized and is generally faster, so paneling is skipped there to avoid regressions.

Periodic inputs can mask M/N address-arithmetic bugs, so this is opt-in and not a CI replacement for the default reference.

3. Implementation

• utils/bench_mode.{hpp,cpp}, utils/parser.cpp: new mode_modifier_t::ref_periodic_fill (A).
• matmul/matmul.cpp: apply_paneled_fill periodizes src/wei/bias, scales/zero-points, sum and binary/prelu post-op operands on the M/N panel grid; gated on the native path (!prim_ref).
• matmul/ref_matmul.cpp: compute_ref_matmul_periodic computes the representative panel and broadcasts dst values and dynamic/derived dst scales (mx, dynamic_fp).
• Supported: all dtypes, per-tensor/-channel/grouped scales & zero-points, dynamic dst scales, bias, sum, eltwise, binary/prelu post-ops, precomputed-reductions, runtime dims, batch. Falls back to the full reference for sparse/grouped, dropout, and stochastic rounding.
• Docs: doc/driver_matmul.md, doc/knobs_common.md.

4. Validation

Validated on Intel Arc B580 (Battlemage), OpenCL GPU runtime.

• Functional PASS across dtypes (f32/f64/bf16/f16/int8/f8/f4), quantization (per-tensor/-channel/grouped scales & zero-points, dynamic dynamic_fp dst scales with exact DST_SCALES comparison), bias, sum, eltwise, binary/prelu post-ops, precomputed-reductions, runtime dims, batched.
• Coverage analysis of a full matmul nightly (15,269 cases): ~100% of large layers (≥1G MACs) are handled; the only uncovered large cases are sparse and dropout, which are non-periodic by construction and correctly fall back.
• Fast-ref-aware gating verified: with a fast CPU reference available the modifier is a no-op (prim reference used); paneling engages only on the native path.

5. Performance (Intel Arc B580, layers from real LLM models, naturally native path, OFF vs ON)

These are matmul layers taken from real LLM models. Their configs (weight-decompression and mx/f4 scaling) have no fast CPU primitive reference, so benchdnn naturally uses the scalar native reference — exactly the slow case this feature targets. No flags force this path; --mode-modifier=A is the only difference between OFF and ON. All PASS.

Layer (M×N×K)	dtype / attrs	Wall OFF	Wall ON	Wall speedup	compute_ref OFF → ON
4096×11008×4096	s8:u4:f16 woq (grouped wei scales+zp) + swish + binary_mul	187.9 s	5.6 s	33×	182.8 s → 0.29 s (~630×)
1009×14336×4096	f16:u4:f16 woq (grouped wei scales+zp, fpmath=f16)	48.2 s	5.1 s	9.5×	43.5 s → 0.12 s (~360×)
14558×3072×3072	f4_e2m1 + mx (e8m0) src/wei scales + bias	196.2 s	3.7 s	53×	192.9 s → 0.29 s (~665×)

The native reference itself speeds up ~360–665×. Since the data is periodic, the input fill was optimized to compute the reference once and broadcast it: fill_data's device reorder is skipped when paneling is active (the single post-periodize reorder supplies the device data), and periodization copies whole contiguous lines with memcpy in parallel. The remaining ON wall time is now the single unavoidable f32→device reorder (the GPU still runs the full shape).

[rjoursler]

There is a chance that trimmed shapes result in other strategies were selected, hence a testing gap

Rather than trying to optimize the current work, it may be simpler and faster to directly testing our strategies. For a given strategy, we really don't need to run a particularly large problem to validate all the code paths are correct (roughly 10-20 sizes smaller than 1Kx1Kx1K likely gets 100% functional validation). We could enable fairly direct strategy choice via something like #5461, by extending the configuration dispatcher to support something like entry=<index>. Exhaustive testing then just becomes running a small suite a problems against each entry until we hit the end of the supported entries.