[Klaud Cold] Remove dsv4-fp8-mi355x-sglang recipe#1501
Conversation
The v0.5.12-rocm720-mi35x-20260517 image rejects bf16 weights in fp8 slots (ValueError: Downcasting not allowed at deepseek_v4.py:1544 → parameter.py:73 copy_with_check) — DSV4-Pro-FP8 has bf16 shared-expert/ embedding layers and the old rocm/sgl-dev:deepseek-v4-mi35x image tolerated this, the generic v0.5.12 ROCm image doesn't. Bump PR #1470 root-caused via the failing sweep; the recipe isn't viable on any tag in the current sglang ROCm MI355X lineage until upstream relaxes the downcast check or ships a converter. Closing that PR; if the recipe is needed again later it can be re-added with whichever image works at that point. Also removes the orphan launch script benchmarks/single_node/ dsv4_fp8_mi355x.sh (not referenced by any remaining recipe). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
There was a problem hiding this comment.
LGTM — clean removal of a permanently-failing recipe and its orphan launch script.
Extended reasoning...
Overview
This PR removes the dsv4-fp8-mi355x-sglang recipe block from .github/configs/amd-master.yaml and deletes its launch script benchmarks/single_node/dsv4_fp8_mi355x.sh. The motivation is sound and clearly explained: the upstream sglang ROCm MI355X image added a strict downcast check that rejects DSV4-Pro-FP8's bf16 layers, and the prior custom image is no longer being maintained, so this recipe was permanently failing in cron sweeps.
Security risks
None. This is a configuration removal — no auth, crypto, permissions, or input handling involved.
Level of scrutiny
Low. Pure deletion of a failing benchmark recipe and its orphan script. The change is mechanical, the blast radius is contained to one cron-sweep configuration, and the only externally-visible effect is one fewer (broken) job in the sweep.
Other factors
I verified the test plan claims: outside the deleted lines, the only remaining dsv4-fp8-mi355x references are in perf-changelog.yaml (historical log entries — appropriate to leave) and a comment referencing dsv4_fp8_mi355x_vllm.sh, which is a separate vllm script that still exists on disk. No orphan references to the deleted artifacts remain. Other dsv4-mi355x recipes (fp4-sglang, fp4-atom, fp8-vllm, fp8-atom) are untouched as advertised. No bugs reported by the bug hunting system.
…xups
Resolutions:
- perf-changelog.yaml: took main verbatim.
- runners/launch_b300-nv.sh: took main (drops --nodelist pin entirely;
supersedes our narrower 017-019 fix).
- benchmarks/single_node/fixed_seq_len/dsv4_fp8_mi355x{,_vllm}.sh:
accepted main's deletes (orphan recipes removed in #1374, #1501).
- .github/configs/amd-master.yaml: took main as the base, then re-applied
our agentic-only additions on top:
* qwen3.5-fp8-mi355x-sglang-agentic-hicache (new entry)
* dsv4-fp4-mi355x-vllm-agentic (new entry)
* dsv4-fp4-mi355x-sglang-agentic (new entry)
* kimik2.5-fp4-mi355x-vllm-agentic (cpu -> lmcache)
Dropped our comment-path edit for dsv4_fp8_mi355x_vllm.sh since main
deleted that entry.
Fixed_seq_len reorg fixups for files added on main during our branch's
lifetime:
- git mv 14 stranded scripts from benchmarks/single_node/*.sh into
benchmarks/single_node/fixed_seq_len/ (dsr1_fp4_b200_mtp,
dsr1_fp4_mi355x_mtp, dsr1_fp8_h200_mtp, dsr1_fp8_mi325x_mtp,
dsr1_fp8_mi355x_mtp, dsv4_fp4_mi355x_vllm, glm5_fp8_h200_mtp,
glm5_fp8_mi325x, glm5_fp8_mi325x_mtp, qwen3.5_bf16_mi325x_mtp,
qwen3.5_fp4_mi355x_mtp, qwen3.5_fp8_h100, qwen3.5_fp8_h100_mtp,
qwen3.5_fp8_mi325x_mtp). Patched their source paths from
../benchmark_lib.sh to ../../benchmark_lib.sh.
- runners/launch_mi355x-amds.sh: multinode-non-disagg BENCHMARK_SUBDIR
bumped from `single_node` to `single_node/fixed_seq_len`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Summary
Removes the
dsv4-fp8-mi355x-sglangrecipe (and its orphan launch scriptbenchmarks/single_node/dsv4_fp8_mi355x.sh). The current sglang ROCm MI355X image lineage cannot serve the DSV4-Pro-FP8 checkpoint without errors.Why
Bump PR #1470 tried to advance the image from
rocm/sgl-dev:deepseek-v4-mi35x→lmsysorg/sglang-rocm:v0.5.12-rocm720-mi35x-20260517. Every sweep job on the new image failed at server boot:DSV4-Pro-FP8 has some bf16 layers (shared-expert / embedding); the old
rocm/sgl-dev:deepseek-v4-mi35xwas a custom build that tolerated this, the generic v0.5.12 ROCm MI355X release added a strict downcast check that we can't satisfy without an upstream change.Path forward: if we re-introduce dsv4-fp8 on mi355x later, we'll either need (a) a re-bake of the custom image, (b) an upstream sglang change relaxing the downcast check, or (c) a DSV4 checkpoint that's pure fp8 with no bf16 layers. None of those are immediate, so leaving the recipe in place creates a permanently-FAILED config in every cron sweep — better to remove it cleanly.
Diff
.github/configs/amd-master.yaml: dropdsv4-fp8-mi355x-sglangblock (lines 1709-1727)benchmarks/single_node/dsv4_fp8_mi355x.sh: deleted (no remaining recipe references it)Other dsv4-mi355x recipes (
dsv4-fp4-mi355x-sglang,dsv4-fp4-mi355x-atom,dsv4-fp8-mi355x-vllm,dsv4-fp8-mi355x-atom) are untouched.Test plan
python3 -c "import yaml; yaml.safe_load(open('.github/configs/amd-master.yaml'))").dsv4-fp8-mi355x-sglangordsv4_fp8_mi355x.sh.🤖 Generated with Claude Code