[AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / Qwen3.5-FP8 MI355X SGLang disagg 性能调优 更新Docker image#2094
Conversation
…pool, concurrency sweep - Bump image from sglang-rocm:v0.5.11-rocm700 to sglang:v0.5.14-rocm720 - Add mia1-p01-g09 to disagg SLURM exclude list (broken pyxis) - Expand mi355x-disagg runner pool from 3 to 9 nodes - Set 1K1K and 8K1K concurrency to [ 8, 16, 32, 64, 128 ] Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
ChangLiu0709
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
|
Claude finished @ChangLiu0709's task in 1m 52s —— View job PR Review
SummaryOne 🟡 warning, no blocking issues. The changelog entry's Everything else checks out:
|
Co-authored-by: Cursor <cursoragent@cursor.com>
ChangLiu0709
force-pushed
the
chang/qwen3.5-mi355-di-perf-tuning-rebased
branch
from
9c4b981 to
f557d68
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Fix disagg SLURM exclude list: add mia1-p01-g09 (broken pyxis) to submit.sh — was missing from multi-node path, causing prefill worker 503 failures." | ||
| - "Expand mi355x-disagg runner pool from 3 to 9 nodes." | ||
| - "Concurrency sweep: 1K1K and 8K1K both [ 8, 16, 32, 64, 128 ]." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092 |
There was a problem hiding this comment.
🟡 WARNING: pr-link points to #2092, which is the CLOSED pre-rebase predecessor of this PR — the changes actually land via this PR (#2094).
Why it matters: The perf changelog is the traceability record for config changes; linking to a closed, unmerged PR breaks that trail.
Fix:
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092 | |
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2094 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28795705489 |
| - "Fix disagg SLURM exclude list: add mia1-p01-g09 (broken pyxis) to submit.sh — was missing from multi-node path, causing prefill worker 503 failures." | ||
| - "Expand mi355x-disagg runner pool from 3 to 9 nodes." | ||
| - "Concurrency sweep: 1K1K and 8K1K both [ 8, 16, 32, 64, 128 ]." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092 |
There was a problem hiding this comment.
🟡 The new perf-changelog entry sets pr-link: .../pull/2092, but this PR is #2094 — the entry's bullets (image bump v0.5.11→v0.5.14, mia1-p01-g09 exclude, mi355x-disagg pool expansion, conc-list trim) exactly describe this PR's diff. Neighboring entries follow the convention of linking to their authoring PR (e.g. line 4558 → #1994). Bump the pr-link to #2094 so future perf-regression traces land on the right review context.
Extended reasoning...
What the bug is. The new entry in perf-changelog.yaml (line 4567) declares pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092, but this PR is #2094. The bullets inside the same entry describe exactly this PR's four-part diff: (a) image bump lmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511 → lmsysorg/sglang:v0.5.14-rocm720-mi35x (matches configs/amd-master.yaml), (b) mia1-p01-g09 added to SLURM_EXCLUDE_NODES in benchmarks/multi_node/amd_utils/submit.sh, (c) mi355x-disagg runner pool grown from 3 to 9 nodes in configs/runners.yaml, and (d) 1K1K/8K1K conc-list trimmed to [8, 16, 32, 64, 128]. So the entry is unambiguously documenting this PR, not a predecessor.\n\nWhy it's #2094 not #2092. The PR metadata title is verbatim [AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / ..., which matches HEAD commit db4487a ([AMD] Qwen3.5-FP8 MI355X SGLang disagg: image bump, node fix, runner pool, concurrency sweep). Recent git log shows the surrounding merged PRs — #2006, #2007, #1994, etc. — but no #2092. The neighboring changelog entries follow a strict convention: line 4558 → #1994 (matches commit b7ffc28 Update Minimax M3 B300 FP4 vllm (#1994)), line 4552 → #2006, line 4545 → #2007, and line 4538 → #2010. Every one references its own authoring PR, so #2092 for this entry is a copy-paste/typo — most plausibly a 4↔2 digit swap.\n\nImpact. No runtime behavior is affected — this is metadata used by the perf-changelog tooling to link a regression back to review context. Per the changelog's own convention (and the AGENTS.md pr-link contract), when someone traces a future Qwen3.5-FP8 MI355X SGLang disagg regression back to this entry, the hyperlink is what gives them the code-review + discussion history. Following the link to a non-existent / unrelated #2092 breaks that traceability. The description bullets themselves are detailed enough that a reader still has full context, which is why this is a nit, not a blocker.\n\nStep-by-step proof.\n1. Open perf-changelog.yaml line 4567: pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092.\n2. Read the entry's bullets (lines 4560–4566): image bump, mia1-p01-g09 exclude, runner pool expansion, conc-list trim.\n3. Diff each bullet against the PR:\n - configs/amd-master.yaml line 353: image was v0.5.11-rocm700, now v0.5.14-rocm720 ✔\n - benchmarks/multi_node/amd_utils/submit.sh line 159: adds mia1-p01-g09 to SLURM_EXCLUDE_NODES ✔\n - configs/runners.yaml line 126: adds mi355x-amds_{12,14,16,17,18,19} to mi355x-disagg (3 → 9 nodes) ✔\n - configs/amd-master.yaml lines 368 & 399: conc-list shrunk to [8, 16, 32, 64, 128] ✔\n4. Read PR metadata: number = 2094, title matches db4487a.\n5. Grep neighboring changelog entries for the pr-link convention: lines 4538/4545/4552/4558 point to #2010/#2007/#2006/#1994 — each matches its authoring commit's (#XXXX) suffix. Conclusion: the pr-link is supposed to be the authoring PR, and #2094 is that PR.\n\nFix. Change pull/2092 → pull/2094 on perf-changelog.yaml:4567. One-character edit; no runtime or CI impact.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28795797050 |
1 participant
Changes
qwen3.5-fp8-mi355x-sglang-disaggimage fromlmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511tolmsysorg/sglang:v0.5.14-rocm720-mi35x(ROCm 7.0 → 7.2, sglang v0.5.11 → v0.5.14).[ 8, 16, 32, 64, 128 ].Authors
@ChangLiu0709
@chunfangamd
中文说明
Qwen3.5-FP8 MI355X SGLang 分离式推理性能调优：
lmsysorg/sglang:v0.5.14-rocm720-mi35x（ROCm 7.0 → 7.2，sglang v0.5.11 → v0.5.14）。[ 8, 16, 32, 64, 128 ]。Made with Cursor