[rocjitsu] Round FMA mix BF16 results to nearest even

[sjain-stanford]

Provenance and context

Closes #10806.

The issue was reported by @Yu-Zhewen against rocJITsu 08ae38600aa96bc29fde7701ef2d17240d4f1945 with HIP 7.16.26315 and the gfx1250_mi455x.json configuration. The posted 256-element harness showed that v_fma_mixlo_bf16 disagreed with round-to-nearest-even for 132 inputs and matched plain FP32-to-BF16 truncation for all 256 inputs. The FP32-result form, v_fma_mix_f32_bf16, remained correct. A follow-up established that v_fma_mixhi_bf16 was affected identically while the FP16-result form was not.

I independently reproduced the result on rocJITsu 97ab4d1f880b1814f736de11453b525bed21a250 with ROCm 7.16.26332, then confirmed that the relevant implementation was unchanged at the current origin/develop base, d562788ea0a24ae91847df3073579dffc36ccd11. This affects ordinary compiler output: mixed BF16/FP32 expressions such as the multiply chain in BF16 RMSNorm can lower to v_fma_mix_f32_bf16 followed by v_fma_mixlo_bf16, causing roughly half of stored BF16 results to be low by one BF16 ULP.

Root cause

The fused arithmetic and input conversion are correct. gen_mad_mix_bf16() in the authoritative packed-instruction generator computes the FP32 result with std::fma(a, b, c), but the BF16-result branch narrows it with util::f32_to_bf16(result).

f32_to_bf16() is a deliberately truncating bit pack: it returns the upper 16 bits of the FP32 representation. Generation propagated that call into all four affected CDNA5 execution paths: ordinary and DPP forms of both v_fma_mixlo_bf16 and v_fma_mixhi_bf16. The existing util::f32_to_bf16_rne() helper provides the required round-to-nearest-even conversion, including retained-LSB tie handling and NaN preservation.

The fix is intentionally made at this result-pack site instead of changing f32_to_bf16() globally, because other callers may rely on truncation. It also intentionally uses f32_to_bf16_rne() rather than the MODE-aware helper: packed BF16 arithmetic is not MODE-aware in rocJITsu, consistent with the existing packed BF16 overflow behavior.

Implemented fix

The BF16-result branch of gen_mad_mix_bf16() now emits util::f32_to_bf16_rne(result), and the checked-in CDNA5 VOP3P execution source was regenerated. This corrects both destination halves and both ordinary and DPP execution paths without changing the FP32-result path or unrelated BF16 conversions.

The generator regression test now requires the RNE helper for both mixlo and mixhi and explicitly rejects the truncating helper. A CDNA5 execution regression exercises both opcodes through ordinary and identity-DPP encodings, verifies preservation of the untouched destination half, and covers:

• exact representable input;
• values immediately below, at, and immediately above a halfway boundary;
• halfway cases with even and odd retained LSBs;
• a negative halfway case;
• a subnormal halfway case;
• a finite value that rounds to infinity.

Validation

The original HIP reproducer was rebuilt with the same ROCm 7.16 toolchain and run unchanged through the fixed rocJITsu interpreter:

v_fma_mixlo_bf16: 0 of 256 wrong
124 of 256 equal plain truncation
v_fma_mix_f32_bf16: 0 of 256 wrong

Before the fix, the same run produced 132 of 256 wrong and 256 of 256 equal plain truncation. The post-fix result matches the expected FFM behavior, while the FP32-result control remains unchanged.

Additional local validation:

• full amdisa regeneration via emulation/rocjitsu/scripts/generate-amdisa.sh;
• clean Release build of rocjitsu_tests, rocjitsu_bin, and rocjitsu_shared;
• all 25 tests in test_packed_codegen.py passed;
• 14 focused CDNA5 execution and VOP3P FMA-mix SIMD tests passed;
• all pre-commit hooks passed;
• git diff --check passed.

A broader local sweep ran 114 tests and passed 113. The sole failure was Gfx1250ExecutionTest.FusedOperationsHonorF16F64ModeControls in the existing FP64 rounding-mode path; this change is confined to BF16 result packing, and all directly affected and neighboring FMA-mix tests pass.

Co-authored-by: GPT-5.6 Sol codex@openai.com

🤖 Generated with Codex

[therock-pr-bot]

✅ All Checks Passed — Ready for Review

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[therock-pr-bot]

🎉 All checks passed! This PR is ready for review.

[Copilot]

Pull request overview

This PR fixes incorrect BF16 result packing for CDNA5 v_fma_mixlo_bf16 / v_fma_mixhi_bf16 execution by switching the final FP32→BF16 narrowing step from truncation to round-to-nearest-even (RNE), aligning rocJITsu’s behavior with expected hardware semantics and preventing 1-ULP-low BF16 results in common mixed BF16/FP32 compiler output patterns.

Changes:

• Update the packed-instruction generator to use util::f32_to_bf16_rne(result) when emitting BF16-result FMA_MIX variants.
• Regenerate CDNA5 VOP3P execution code to apply RNE for both mixlo and mixhi, including modifier/DPP paths.
• Add/strengthen regression coverage in both the generator tests and CDNA5 execution tests for tie-to-even and boundary cases.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File	Description
emulation/rocjitsu/tests/cdna5_execution_test.cpp	Adds a targeted CDNA5 execution regression test covering RNE behavior for mixlo/mixhi across ordinary and identity-DPP encodings, including halfway/tie cases and overflow-to-inf.
emulation/rocjitsu/lib/rocjitsu/src/rocjitsu/isa/arch/amdgpu/generated/cdna5/vop3p_exec.cpp	Applies util::f32_to_bf16_rne for BF16-result packing in VFmaMixloBf16Vop3p and VFmaMixhiBf16Vop3p (normal + modifier/DPP paths).
emulation/rocjitsu/lib/python/amdisa/tests/test_packed_codegen.py	Updates generator regression expectations to require the RNE helper for both mixlo and mixhi BF16-result variants and reject truncation.
emulation/rocjitsu/lib/python/amdisa/codegen/execute/packed.py	Changes gen_mad_mix_bf16() BF16-result emission to use util::f32_to_bf16_rne(result) instead of util::f32_to_bf16(result).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.