[AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / Qwen3.5-FP8 MI355X SGLang disagg 性能调优 更新Docker image

[ChangLiu0709]

Changes

• Docker image bump: update qwen3.5-fp8-mi355x-sglang-disagg image from lmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511 to lmsysorg/sglang:v0.5.14-rocm720-mi35x (ROCm 7.0 → 7.2, sglang v0.5.11 → v0.5.14).
• Concurrency sweep: concurrency [ 1, 2, 4, 8, 16, 32, 64, 128 ]

Authors

@ChangLiu0709
@chunfangamd

中文说明

Qwen3.5-FP8 MI355X SGLang 分离式推理性能调优：

• 升级 Docker 镜像至 lmsysorg/sglang:v0.5.14-rocm720-mi35x（ROCm 7.0 → 7.2，sglang v0.5.11 → v0.5.14）。
• 并发扫描：并发 [ 1, 2, 4, 8, 16, 32, 64, 128 ]。

Made with Cursor

[claude]

🔴 This PR adds a new top-level key qwen3.5-fp8-mi355x-sglang-mtp: at lines 425-443, but that key already exists earlier in the file (line 285) and was modified by this same PR. PyYAML's safe_load (used by utils/matrix_logic/validation.py:553) silently keeps only the last value for duplicate mapping keys, so this new block wins — reverting the mtp recipe to lmsysorg/sglang:v0.5.10.post1-rocm700-mi35x with a single tp:4, ep:1, conc-end:64 sweep and discarding every earlier edit (image bump to v0.5.12-rocm720-mi35x-20260517 and new TP=8/EP=8, TP=2/EP=2 rows). This block appears to be a rebase/copy-paste artifact — the PR's stated scope is the -sglang-disagg variant, not -mtp. Fix: delete the block, or rename to a unique key if it was intentional.

Extended reasoning...

What the bug is

The PR adds a second top-level mapping key with the same name (qwen3.5-fp8-mi355x-sglang-mtp:) to .github/configs/amd-master.yaml. grep -n '^qwen3.5-fp8-mi355x-sglang-mtp:' .github/configs/amd-master.yaml returns two lines: 285 (the original, which this PR edits) and 425 (the new block added by this PR, sitting immediately after the -sglang-disagg entry).

Why the earlier edits become dead code

utils/matrix_logic/validation.py:553 loads the file with plain yaml.safe_load. PyYAML silently keeps only the last value for duplicate mapping keys — no error is raised. The loader's duplicate-key check (validation.py:536) only runs across files, not within a single file, so nothing flags this.

I confirmed the effective config by loading the file with PyYAML:

image: lmsysorg/sglang:v0.5.10.post1-rocm700-mi35x
search-space: [{tp: 4, ep: 1, conc-start: 4, conc-end: 64, spec-decoding: mtp}]  # both 1K1K and 8K1K

i.e. the block at line 425 wins. All of this PR's edits to the block at line 285 — image bump to v0.5.12-rocm720-mi35x-20260517, new TP=8/EP=1, TP=8/EP=8, TP=2/EP=2 search-space rows on the 1K1K and 8K1K scenarios — are silently discarded.

Impact

The mtp sweep runs on the older v0.5.10.post1-rocm700-mi35x image (ROCm 7.0, sglang v0.5.10.post1) with a single tp:4, ep:1 sweep at conc 4-64, instead of the intended multi-shape sweep on the newer v0.5.12-rocm720-mi35x-20260517 image. This is a silent regression on a sibling recipe that this PR did not intend to touch — the PR's stated scope is the -sglang-disagg variant.

Step-by-step proof

1. grep -n '^qwen3\.5-fp8-mi355x-sglang-mtp:' .github/configs/amd-master.yaml → 285:qwen3.5-fp8-mi355x-sglang-mtp: and 425:qwen3.5-fp8-mi355x-sglang-mtp:.
2. The diff shows the block at line 425 is entirely new (+qwen3.5-fp8-mi355x-sglang-mtp: hunk immediately after the -sglang-disagg block).
3. python3 -c "import yaml; d = yaml.safe_load(open('.github/configs/amd-master.yaml')); e = d['qwen3.5-fp8-mi355x-sglang-mtp']; print(e['image']); print(e['scenarios']['fixed-seq-len'][0]['search-space'])" prints lmsysorg/sglang:v0.5.10.post1-rocm700-mi35x and [{tp: 4, ep: 1, conc-start: 4, conc-end: 64, spec-decoding: mtp}] — the second block's contents, confirming the first block is shadowed.
4. utils/matrix_logic/validation.py:553 uses yaml.safe_load; the duplicate-key check at :536 only fires across files, not within one file, so no error is raised.

Fix

Delete the block at lines 425-443 (looks like a rebase / copy-paste artifact — this PR's scope is the disagg variant), or, if the new sweep was actually intended, rename it to a unique key such as qwen3.5-fp8-mi355x-sglang-mtp-legacy or similar.

[claude]

🟡 The perf-changelog entry states "1K1K commented out for this tuning round", but commit 7bca252 (in this same PR) uncommented the 1K1K scenario, so it is fully active in amd-master.yaml with the same conc-list as 8K1K. Suggest updating to something like: "1K1K + 8K1K concurrency sweep: [ 1, 2, 4, 8, 16, 32, 64, 128 ]".

Extended reasoning...

What the bug is\n\nThe perf-changelog entry added in this PR (perf-changelog.yaml lines 4327-4334) contains this description bullet:\n\n> "8K1K concurrency sweep: [ 1, 2, 4, 8, 16, 32, 64, 128 ]; 1K1K commented out for this tuning round."\n\nThat claim is stale. The 1K1K scenario for qwen3.5-fp8-mi355x-sglang-disagg (isl: 1024, osl: 1024) is fully active in .github/configs/amd-master.yaml (lines ~371-401) with conc-list: [ 1, 2, 4, 8, 16, 32, 64, 128 ] — identical to the 8K1K row.\n\nWhy it looks this way\n\nGit history shows how this happened:\n- Commit 52d7cca ("Add perf-changelog entry for qwen3.5 disagg tuning PR #2092") added the changelog entry with the 1K1K-commented-out claim, which was accurate at the time relative to an earlier commit that had disabled 1K1K.\n- Commit 7bca252 ("Uncomment 1K1K scenario and update concurrency for full sweep") — applied later on this branch — re-enabled the 1K1K scenario but did not touch perf-changelog.yaml.\n\nSo the code and changelog drifted out of sync inside the same PR.\n\nImpact\n\nDocumentation-only. Nothing at runtime is affected — both 1K1K and 8K1K execute exactly as configured. The inaccuracy only affects future readers trying to reconstruct what the tuning round covered from the changelog.\n\nHow to fix\n\nUpdate the fourth bullet in the new perf-changelog entry to reflect the actual sweep, e.g.:\n\nyaml\n- "1K1K and 8K1K concurrency sweep: [ 1, 2, 4, 8, 16, 32, 64, 128 ]."\n\n\nStep-by-step proof\n\n1. Open .github/configs/amd-master.yaml and locate qwen3.5-fp8-mi355x-sglang-disagg (line 370).\n2. The first scenario block (lines 375-401) is isl: 1024, osl: 1024 (the 1K1K scenario) with conc-list: [ 1, 2, 4, 8, 16, 32, 64, 128 ] — no commented-out block, no # markers around it.\n3. The second scenario block (lines 403+) is isl: 8192, osl: 1024 (the 8K1K scenario) with the same conc-list.\n4. Both scenarios are equally active; nothing in the file matches the changelog's "1K1K commented out" description.\n5. git log --oneline on this branch shows 52d7cca (add perf-changelog) preceded 7bca252 (uncomment 1K1K), confirming the changelog text was frozen before the 1K1K re-enable landed.