[AMD][MI35X] 0612 DSV4

[1am9trash]

Successful run:
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27406714195
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27406714195

Change:

• Bump image to lmsysorg/sglang-rocm:v0.5.13-rocm720-mi35x-20260612.
• Fix the intermediate_pad setting in the MoE computation in sglang PR#27858. This avoids the unnecessary overhead of computing useless padding.
• Correct the chunk prefill setting size under tp8/dp8 config.

---

Note

Low Risk
Benchmark-only image pin and serving-flag tweak for one AMD DSv4 config; no application or auth changes.

Overview
Updates the dsv4-fp4-mi355x-sglang benchmark to SGLang ROCm v0.5.13 (lmsysorg/sglang-rocm:v0.5.13-rocm720-mi35x-20260612), which picks up upstream MoE intermediate_pad fixes (sglang PR#27858) so padding work is not wasted in MoE compute.

In dsv4_fp4_mi355x_sglang.sh, --chunked-prefill-size is no longer fixed at 8192 when DP attention is on: it stays 8192 for TP-only paths and becomes 8192 × TP (e.g. 65536 at TP8) for the TP8/DP8 sweep. perf-changelog.yaml records the image bump and chunked-prefill correction for this config key.

^{Reviewed by Cursor Bugbot for commit dbf706b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27406714195
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27406714195

[1am9trash]

@functionstackx could you please review this?

[functionstackx]

yes. lgtm once passes and then i will do /reuse on it https://github.com/SemiAnalysisAI/InferenceX/actions/runs/27475314602/job/81213395277?pr=1715

[functionstackx]

hi @1am9trash it seems like conc

[functionstackx]

hi @1am9trash it seems like conc512 is failing, can u take a look? (will cancel the rest of the conc for now to avoid clogging up the CI queue since conc512 failed already)

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27475314602
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27475314602

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27475314602
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27475314602

[1am9trash]

Hi, @functionstackx
I checked the server logs for the failed task (conc=512) and found that it never actually started running successfully because it hit an Out of Memory (OOM) issue. This has never happened in our previous tests, even with higher concurrency levels.

Callback: Queue 0x796394800000 Aborting with error : HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. 
This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.
Code: 0x1008 Available Free mem : 0 MB

My assumption is that there were other workloads on the server competing for resources at the time, and therefore the failure is unrelated to the v4 testing changes introduced in this PR. I reran the task (conc=512), and it completed successfully without encountering the issue.
Reran: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/27475314602/job/81251602351?pr=1715

Thanks.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27475314602
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27475314602

[functionstackx]

i am merging it rn, ty for these changes!

[functionstackx]

thanks for the contribution @1am9trash any chance yall can optimize the MTP shapes too & DI too (escpially DI with MTP composed together would be amazing). mi355 agg MTP doesnt seem close to SOL.

[functionstackx]

/reuse-sweep-run

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27519237142
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27519237142