[WIP] fix(rocjitsu): model finite CDNA wait counters in Waitcheck - #10926
Draft
newling wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
completion-order class have issued that hardware must have completed it.
instruction's operands.
conservative across straight-line analysis and CFG merges.
including stores and direct-to-LDS visibility.
Problem
On CDNA, LGKMCNT is four bits and VMCNT is six bits. Their all-ones operands
are no-wait values:
Hardware prevents these counters from overflowing by stalling an instruction
whose issue would exceed the limit. Waitcheck already treated the all-ones
operands as no-ops, but retained an unbounded dependency history. It could
therefore report a producer as pending after hardware had necessarily
completed it.
One production-generated gfx950 example was reported as:
lgkmcnt(15)does not synchronize the producer. Nevertheless, an older LDSoperation cannot remain outstanding after 15 younger ordered LDS operations
have issued, much less 123. A second production case had 17 younger LDS reads
before its first MFMA use and is safe for the same reason.
The 15-entry LGKMCNT limit belongs to the complete counter, which also accounts
for GDS, scalar-memory, and message events. Sharing a counter is not the same
as sharing a completion order. Fifteen later LDS operations are sufficient to
prove completion of an older LDS operation; fifteen arbitrary LGKM-counted
operations are not sufficient to identify which event completed.
This deliberately does not change hipBLASLt's
MaxLgkmcntfrom 15 to 14.Fifteen is the correct capacity/no-wait encoding. Clamping it globally to 14
inserts explicit stalls even where issue backpressure has already made the
dependency safe.
Implementation
Each pending Waitcheck event now carries a separate ordered-counter domain:
Ldsfor native local-DS operations;Vmemfor CDNA non-FLAT VMEM operations, including stores anddirect-to-LDS; and
Nonefor scalar memory, generic FLAT, GDS, and cases without a usableordering guarantee.
Waitcheck tracks the minimum number of later operations in the same ordered
class along every feasible path. Once that lower bound reaches 15 for LDS or
63 for VMEM, the event is retired. Backpressure is applied before dependency
checks, so the instruction that would overflow the counter may safely consume
the oldest completed result. CFG merges retain the minimum proven ordered age.
Mixed-class counter pressure remains deliberately conservative. It can cause
hardware progress, but Waitcheck does not guess whether an LDS, scalar-memory,
GDS, or generic-FLAT event was the one that completed.
Architecture and compiler cross-checks
in
S_WAITCNT; all ones is documented as “do not wait.”LGKMCNT, and folds stores into VMCNT on targets without VSCNT.
while pre-VSCNT VMEM loads/stores form one ordered stream.
waitcnt-overflow.mirregression test andSIInsertWaitcnts.cpp::determineWaitForScoreconservatively cap anoverflow-age dependency at
limit - 1, producinglgkmcnt(14)andvmcnt(62)for gfx9.Relevant upstream references:
Evidence
Native MI355X probes distinguish no-wait operands from capacity-forced
progress:
lgkmcnt(15)vmcnt(63)vmcnt(63)At the LDS boundary, the old analyzer reported:
The corrected analyzer reports zero diagnostics after 15 younger LDS reads.
The 14-younger-read control still reports the dependency at
.text+0xd0.Full scans of the frozen rocm-libraries
fa8f393bcorpus produced:No kernel gained diagnostics and no new producer/consumer instruction pair
appeared. The gfx950 runs contain only explained refinements at existing sites:
some conservative
vmcnt(0)recommendations became precise partial counts,and some multi-register loads acquired a corrected overlapping-register label.
The gfx942 run had no new diagnostic identity and no changed requirement.
Testing
423/423WaitcheckTest.*tests passed.(
42,300test executions).git diff --checkpassed for every changed file.Remaining work before marking ready
sanitizer/Waitcheck prerequisite branch.
this patch removes impossible reports but does not classify every survivor.
CCOBinputs directly. CurrentTensileCreateLibraryemits them by default, so this audit had to unbundlethem before scanning.
Issue Tracking
Related: ROCM-28430