Update DSv4 B300 vllm#1221
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
Claude finished @wzhao18's task in 1m 59s —— View job Review of PR #1221
Summary: The
No blocking issues found. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| Fields.SPEC_DECODING.value: spec_decoding, | ||
| Fields.EXP_NAME.value: f"{model_code}_{seq_len_str}", | ||
| Fields.DISAGG.value: disagg, | ||
| Fields.RUN_EVAL.value: False, # Default, may be overridden by mark_eval_entries |
There was a problem hiding this comment.
📊 Line Count Report
- Total Lines: 966
- Base Lines: 963
- Change: 📈 +3 lines
…isAI/InferenceX into nv/b300-vllm-mega-moe
|
Ready for review. Once approved, will fix Sweep: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/25118794641/job/73613407458?pr=1221 |
|
@claude please update pr description |
|
Claude finished @kedarpotdar-nv's task in 1m 43s —— View job Updating PR Description
PR description has been updated with a structured summary covering the image bump, Mega MoE backend gating, search-space tuning changes, and perf changelog entry. |
4 participants
Summary
Integrate Mega MoE backend and update vLLM image for the B300 DeepSeek-V4-Pro FP4 config (
dsv4-fp4-b300-vllm).Changes
Image Update
vllm/vllm-openai:deepseekv4-cu130→vllm/vllm-openai:v0.20.0-cu130Mega MoE Backend (Benchmark Script)
--moe-backend deep_gemm_mega_moeonDP_ATTENTION=trueinbenchmarks/single_node/dsv4_fp4_b300_vllm.sh, matching the B200 sibling script pattern from PR dsv4-fp4-b200-vllm: bump to vllm v0.20.0, deep_gemm_mega_moe MoE #1204BENCHMARK_MAX_MODEL_LEN=4096override for 1k1k sequence lengthsSearch-Space Tuning (Master Config)
tp:8(non-DP-attn) concurrency range:conc-end: 128→conc-end: 4for both 1k1k and 8k1ktp:4 ep:4 dp-attn8k1k concurrency range:conc-end: 1024→conc-end: 512Perf Changelog
dsv4-fp4-b300-vllmentry documenting the image bump and Mega MoE enablementSweep Results