[AMD] MiniMax-M3 FP4 MI355X vLLM STP: close gap vs ATOM (INT4 all-reduce + index-sharing), TP4 sweep / MiniMax-M3 FP4 MI355X vLLM STP：缩小与 ATOM 的性能差距（INT4 all-reduce + 跨层索引共享），TP4 扫描#2102
Conversation
…, TP4 sweep Revive #1969 recipe tuning (INT4 quick-reduce, fp8 KV, index_topk_freq) with the TP4-only conc 1-512 search space from #2003. 中文：恢复 #1969 的 MiniMax-M3 FP4 MI355X vLLM STP 优化（INT4 all-reduce、fp8 KV、index_topk_freq），并将扫描范围收窄为 #2003 的 TP4 单节点 conc 1-512（1k1k/8k1k）。 Co-authored-by: Cursor <cursoragent@cursor.com>
中文：changelog pr-link 暂用 XXX 占位符，待 PR 创建后更新。 Co-authored-by: Cursor <cursoragent@cursor.com>
Pin nightly-69715823 (2026-07-06 Docker Hub nightly). 中文：将 minimaxm3-fp4-mi355x-vllm 镜像升级至最新 vLLM ROCm nightly（nightly-69715823，2026-07-06）。 Co-authored-by: Cursor <cursoragent@cursor.com>
中文：在 amd-master.yaml 中将 minimaxm3-fp4-mi355x-vllm 镜像固定为 nightly-69715823。 Co-authored-by: Cursor <cursoragent@cursor.com>
Fangzhou-Ai
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
中文：将 changelog pr-link 更新为 #2102。 Co-authored-by: Cursor <cursoragent@cursor.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28816590882 |
|
hi @Fangzhou-Ai as @cquil11 & @Oseltamivir discussed with @chunfangamd , this PR has --hf-overrides and model arch changes which is not something we can merge. @chunfangamd has the full details of the reasoning |
@functionstackx Got it! Just talked with @chunfangamd . I will close this PR and re-open one with those env settings only, they can improve the high-concurrency performance. |
2 participants
Summary
Revives the closed #1969 recipe tuning for
minimaxm3-fp4-mi355x-vllm(single-node STP) and narrows the sweep to the TP4-only search space from merged #2003:VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4+..._CAST_BF16_TO_FP16=0+..._QUANTIZATION_MIN_SIZE_KB=256. Reduces the all-reduce cost (the biggest decode kernel).--kv-cache-dtype fp8).--hf-overrides index_topk_freq=4) — requires [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing vllm-project/vllm#47269 (merged); image bumped to latestnightly-69715823.minimaxm3-fp8-mi355x-vllm([AMD] MiniMax-M3 MXFP8 MI355X vLLM: nightly + AITER-on TP4 + emulatin linear / MiniMax-M3 MXFP8 MI355X vLLM：升级 nightly + 启用 AITER TP4 + emulation linear #2003).Supersedes the closed #1969, which was marked superseded but had no linked replacement for FP4 STP.
Measured (local, amd/MiniMax-M3-MXFP4, MI355X gfx950, TP4, 8k1k)
GSM8K limit=100 = 0.95 (above ATOM's 0.9363).
Test plan
full-sweep-fail-fastsweep onminimaxm3-fp4-mi355x-vllm(TP4, 1k1k + 8k1k)中文说明
恢复已关闭的 #1969 对
minimaxm3-fp4-mi355x-vllm单节点 STP 的 recipe 调优，并将扫描范围收窄为已合并 #2003 的 TP4-only 配置：--kv-cache-dtype fp8）。--hf-overrides index_topk_freq=4）— 镜像升级至最新nightly-69715823（含 [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing vllm-project/vllm#47269）。minimaxm3-fp8-mi355x-vllm（[AMD] MiniMax-M3 MXFP8 MI355X vLLM: nightly + AITER-on TP4 + emulatin linear / MiniMax-M3 MXFP8 MI355X vLLM：升级 nightly + 启用 AITER TP4 + emulation linear #2003）一致。本地测试（MI355X TP4, 8k1k）并发 64 时 TPOT 与 ATOM 基本持平；GSM8K 精度 0.95，高于 ATOM 的 0.9363。
Made with Cursor