[WIP] fix(rocjitsu): model CDNA wait-counter capacity in race detector

[newling]

Summary

• Treat CDNA's all-ones legacy wait operands as no-ops: lgkmcnt(15) and
  vmcnt(63) do not complete pending events.
• Model the issue backpressure that prevents the finite LGKMCNT and VMCNT
  counters from overflowing.
• Track completion-order classes separately from broad memory-event types, so
  partial waits and write-after-write checks use only ordering that hardware
  guarantees.
• Dispatch only the legacy memory-wait fields explicitly encoded by
  s_waitcnt, s_waitcnt_vmcnt, and s_waitcnt_lgkmcnt.
• Add gfx950 HIP coverage at both sides of the counter-capacity boundary and
  gfx1151 HIP coverage for the standalone legacy wait forms.

Problem

CDNA's LGKMCNT is four bits and VMCNT is six bits:

counter	maximum outstanding	no-wait operand	largest effective explicit wait
LGKMCNT	15	15	14
VMCNT	63	63	62

The 15-entry LGKMCNT capacity applies to the complete counter, not just LDS.
On pre-GFX12 hardware LGKMCNT accounts for LDS, GDS, scalar-memory, and message
events. Those event classes share capacity but are not all ordered relative to
one another. Fifteen later LDS operations are sufficient to prove that an
older LDS operation completed; fifteen arbitrary LGKM-counted operations are
not sufficient to identify which operation completed.

The race detector previously kept an unbounded event queue and passed every
decoded wait value through total - limit retirement logic. That produced two
opposite errors:

1. An all-ones wait could appear to complete old events once the synthetic
   queue exceeded the hardware counter's capacity, even though that operand is
   architecturally “do not wait.”
2. Without a wait, events remained pending beyond the point at which hardware
   must have completed them to make room for another issue.

The detector also used MemoryEventType as a proxy for completion order. That
could suppress a real WAW race between ordinary VMEM and generic FLAT loads,
or choose an arbitrary scalar-memory result after a partial LGKM wait.

This became visible in production-generated gfx950 prefixes. In one case an
LDS result had 123 younger local-DS operations before its consumer; in another
the result had 17 younger local-DS operations. Both must have completed before
their consumers, even though the intervening lgkmcnt(15) itself performs no
wait.

Architecture and compiler cross-checks

• The CDNA3/CDNA4 machine-readable ISA defines only the combined S_WAITCNT
  instruction. Its operand is four LGKM bits, six VM bits, and three export
  bits; all ones means “do not wait” for each field.
• The standalone S_WAITCNT_VMCNT and S_WAITCNT_LGKMCNT instructions occur
  in the RDNA1--RDNA3.5 specifications. LLVM's MC tests explicitly reject them
  on gfx9 and gfx12.
• gfx12 replaces them with split instructions such as S_WAIT_LOADCNT,
  S_WAIT_DSCNT, and S_WAIT_KMCNT.
• LLVM's pre-GFX12 wait-event table assigns SMEM, LDS, GDS, and message events
  to LGKMCNT. Its ordering logic treats scalar memory and mixed event types as
  potentially out of order, while pre-VSCNT VMEM loads and stores share one
  ordered counter stream.
• LLVM's waitcnt-overflow.mir regression test and
  SIInsertWaitcnts.cpp::determineWaitForScore cap overflow-age dependencies
  at limit - 1: lgkmcnt(14) and vmcnt(62) on gfx9.

Relevant upstream references:

• https://github.com/llvm/llvm-project/blob/main/llvm/test/CodeGen/AMDGPU/waitcnt-overflow.mir
• https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/AMDGPU/SIInsertWaitcnts.cpp
• https://github.com/llvm/llvm-project/blob/main/llvm/test/MC/AMDGPU/gfx9_unsupported.s
• https://github.com/llvm/llvm-project/blob/main/llvm/test/MC/AMDGPU/gfx11_asm_sopk.s
• https://github.com/llvm/llvm-project/blob/main/llvm/test/MC/AMDGPU/gfx12_unsupported.s

Implementation

Each event now carries an explicit completion-order class:

• VMEM for CDNA non-FLAT VMEM operations, including stores and direct-to-LDS;
• LDS for native local-DS operations; and
• UNORDERED for scalar memory, generic FLAT, GDS, and cases without a usable
  ordering guarantee.

Before an ordered memory instruction executes, the detector checks whether
its class already occupies the counter's capacity. If so, it marks the oldest
event in that class wave-complete before the current instruction's operand and
memory checks. This matches the hardware stall that makes room for the new
issue.

For a nonzero partial wait, only the oldest prefix that is provably complete
within an ordered class is retired. A zero wait still drains every event on
the selected counter. All-ones operands retire nothing. Mixed-class pressure
is deliberately conservative: it may leave an event pending when hardware has
made some progress, but it never guesses which event completed.

gfx950 encodes these fields in the combined s_waitcnt instruction. The
standalone s_waitcnt_vmcnt and s_waitcnt_lgkmcnt forms are retained for the
already-supported gfx1151 path; they are GFX10/GFX11 instructions, not gfx1250
instructions. gfx1250 uses split instructions such as s_wait_loadcnt and
s_wait_dscnt, which remain outside this PR's race-detector scope.

Evidence

Native MI355X probes distinguish an explicit no-wait from capacity-forced
progress:

sequence	checked	stale
one LDS read, lgkmcnt(15)	26,214,400	26,213,424
producer + 15 younger LDS reads, no wait	26,214,400	0
producer + 10 younger VMEM loads, vmcnt(63)	1,310,720	1,310,720
producer + 63 younger VMEM loads, no wait	1,310,720	0
producer + 20 younger VMEM stores, vmcnt(63)	1,310,720	1,310,720
producer + 63 younger VMEM stores, no wait	1,310,720	0

The new end-to-end tests demonstrate that the all-ones operands leave a
below-capacity dependency pending (runtime addresses omitted):

RACE kernel=vmcnt_no_wait_race_kernel symbol=_Z25vmcnt_no_wait_race_kernelPKfPf dispatch=4 type=VGPR access=read reg=2 wave=0 lane=0 wg=0,0,0 conflict=unknown
Race on VGPR v2 [workgroup (0, 0, 0), wave 0, lane 0]
  ==> global_load_dword v2, v[2:3]
      s_waitcnt vmcnt(63) expcnt(7) lgkmcnt(15)
  ==> global_store_dword v[0:1], v2
END_RACE

RACE kernel=lgkmcnt_no_wait_race_kernel symbol=_Z27lgkmcnt_no_wait_race_kernelPi dispatch=3 type=VGPR access=read reg=1 wave=0 lane=0 wg=0,0,0 conflict=unknown
Race on VGPR v1 [workgroup (0, 0, 0), wave 0, lane 0]
  ==> ds_read_b32 v1, v0
      s_waitcnt vmcnt(63) expcnt(7) lgkmcnt(15)
  ==> global_store_dword v0, v1, s[0:1]
END_RACE

With the complete capacity model, four bounded production-prefix replays are
clean:

bug2-baseline exit=0 races=0 completed=1
bug2-fixed    exit=0 races=0 completed=1
bug3-baseline exit=0 races=0 completed=1
bug3-fixed    exit=0 races=0 completed=1

The former bug-2 and bug-3 race reports were therefore detector-model
artifacts, not evidence that hipBLASLt needed the proposed extra waits.

The HIP boundary tests exercise the exact issue counts:

gfx950: producer + 14 younger LDS operations  -> race
gfx950: producer + 15 younger LDS operations  -> clean
gfx950: producer + 62 younger VMEM operations -> race
gfx950: producer + 63 younger VMEM operations -> clean

Four gfx1151 HIP tests independently verify that each standalone legacy wait
drains its own counter and does not drain the other counter.

Testing

• 110/110 RaceDetector.* and RaceDetectorPlugin.* tests passed.
• The same 110-test selection passed 100 consecutive repetitions (11,000
  test executions).
• 52/52 gfx950 and gfx1151 race-plugin end-to-end tests passed, including
  ten new no-wait, capacity-boundary, and standalone-wait cases.
• All four production-prefix replays above completed with zero race reports.
• The complete unit binary ran 4,223 tests: 4,220 passed, two were skipped, and
  the one failure was an unrelated DBT offset expectation that reproduces on
  the pre-change build.
• Formatting hooks and git diff --check passed for every changed file.

Remaining work before marking ready

• Decide how to land or transplant this commit after its unmerged
  sanitizer/Waitcheck prerequisite branch.
• Confirm in review that CDNA's documented same-type ordering is the intended
  basis for grouping all native local-DS operations and all non-FLAT VMEM
  operations. LLVM uses the same distinction, and native MI355X probes cover
  LDS reads plus VMEM loads/stores, but this is the central architectural
  premise.
• Let normal CI exercise the repository's other supported configurations.
• Consider a later precision enhancement for mixed-class counter saturation;
  this implementation intentionally keeps such events pending rather than
  guessing which one hardware completed.

Issue Tracking

Related: ROCM-28430
Related: #9577