WIP: perf(rocjitsu): streamline bulk vector memory transfers - #10923
Draft
newling wants to merge 2 commits into
Draft
WIP: perf(rocjitsu): streamline bulk vector memory transfers#10923newling wants to merge 2 commits into
newling wants to merge 2 commits into
Conversation
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
perf(rocjitsu): streamline bulk vector memory transfers
Overview
rocJITsu's functional global-memory path performs substantial host-side work
for wide vector loads and stores. This is especially visible while loading a
large code object: the ROCclr copy kernel repeatedly moves wide lane payloads,
and complete destination cache lines are read before being entirely
overwritten.
This change removes three sources of redundant work:
into lane-major storage, instead of repeating ownership lookup and plugin
dispatch for every register/lane pair;
after validating ownership for the complete range, instead of making one
virtual write call per register/lane pair; and
without first fetching bytes that the store will replace.
These are generic functional-emulation paths. The implementation does not
recognize specific kernels, bypass instruction execution, or suppress plugin
observation. The complete-line path retains dirty-eviction publication and the
existing write-through behavior.
The generated flat-store path uses the wave-owned register-access boundary
introduced by #10030. The VGPR-region snapshot follows the same general pattern
as the scalar MFMA operand snapshot in #10706, while the cache work complements
the page-local MTYPE batching in #10583. PR #10898 optimizes the separate f16
and bf16 MFMA path and was present in both arms of the performance comparison
below.
Performance
The primary workload was one f16 1024x1024x1024 GEMM in functional emulation
with one simulator thread:
Four balanced parent/candidate pairs measured the complete process without an
execution plugin:
Two balanced pairs with the throughput plugin enabled showed where the saving
comes from:
The module-copy dispatch executed exactly 2,287,712 wave instructions in both
revisions. Its global-instruction handler interval fell from approximately
0.736 seconds to 0.250 seconds, a 2.95x improvement. A separate 25 MiB
hipMemsetprobe improved from approximately 0.416 seconds to 0.331 seconds,or 1.26x.
The comparison used exact revision
af74731ccas its parent so that #10898'sMFMA optimization was identical in both arms. The implementation was then
ported to current
developat2d311cd6f, including adaptation to #10030'swave-owned register-access API.
Validation
rocjitsu_tests,rocjitsu_shared, and the throughputplugin
git diff --checknorm_error=0Issue Tracking
Related: #9577