Add Nemotron Lightning 3.5 NVFP4 recipe and QAD example - #2146
Conversation
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a Nemotron 3.5 Lightning NVFP4 recipe and QAD launcher pipeline. Updates Nemotron Nano quantization settings, preserves checkpoint path overrides, and disables default MTP quantization. ChangesNemotron quantization and QAD workflows
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant HuggingFaceCheckpoint
participant ImportTask
participant QuantizationTask
participant QADDistillation
participant ExportTask
HuggingFaceCheckpoint->>ImportTask: BF16 checkpoint
ImportTask->>QuantizationTask: MCore checkpoint
QuantizationTask->>QADDistillation: NVFP4 quantized student
QADDistillation->>ExportTask: 400-iteration distilled checkpoint
ExportTask->>ExportTask: Write NVFP4 checkpoint
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/megatron_lm_qad.yaml`:
- Line 53: Update task_3’s QUANT_CFG to MAMBA_MOE_NVFP4_AGGRESSIVE_CFG so
export.sh reconstructs the model with the same preset used by task_1; only
retain the conservative value if export.sh explicitly ignores QUANT_CFG in favor
of checkpoint metadata.
In
`@tools/launcher/examples/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/megatron_lm_qad.yaml`:
- Line 53: Update the MLM_MODEL_CFG values used by all three Megatron-LM tasks
to nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, including the definitions
near the existing Nano model entry and the two additional occurrences, so
import, quantization, distillation, and export consistently use the Lightning
model ID.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0c9d7f54-3b70-4596-ac4a-0144d837f910
📒 Files selected for processing (3)
modelopt_recipes/huggingface/models/nvidia/Nemotron-3.5-Lightning-30B-A3B/lightning_w4a16_nvfp4_4o6.yamltools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/megatron_lm_qad.yamltools/launcher/examples/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/megatron_lm_qad.yaml
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2146 +/- ##
==========================================
- Coverage 78.73% 78.15% -0.58%
==========================================
Files 522 522
Lines 60356 60395 +39
==========================================
- Hits 47522 47204 -318
- Misses 12834 13191 +357
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
| enable: false | ||
| - quantizer_name: '*router*' | ||
| enable: false | ||
| - quantizer_name: 'mtp.*' |
There was a problem hiding this comment.
@claude will this rule be applied in the end after all rules from 406 recipe yaml is applied? For example, if we first disable mtp.* but then in our recipe, we enable *output_layer*weight_quantizer, will this re-enable mtp.*output_layer*weight_quantizer?
There was a problem hiding this comment.
Claude finished @kevalmorabia97's task in 3m 18s —— View job
Answer: no, the unit is spliced in place — not applied last
- Read the changed files
- Trace where
default_disabled_quantizersis composed into a recipe'squant_cfg - Confirm rule application order / last-match-wins semantics
- Answer the
mtp.*vs*output_layer*weight_quantizerquestion
1. Ordering is purely list order, and $import splices at its position — it is not appended at the end.
modelopt/torch/quantization/conversion.py:260-264:
entries are applied in list order; later entries override earlier ones for any quantizer they match
and the composition guidance in modelopt/torch/quantization/config.py:87-97 says the default-disabled unit is meant to be appended (step 3), i.e. it only "wins" if its $import is the last item in quant_cfg. That holds in e.g. configs/ptq/presets/model/mamba_moe_nvfp4_aggressive.yaml, but not universally — huggingface/models/nvidia/Nemotron-3-Nano-4B/ptq/nvfp4_w4a16.yaml:32 imports it early and dozens of per-layer entries follow it, which do override it.
2. Your hypothetical: yes, it would re-enable. With mtp.* disabled first and *output_layer*weight_quantizer (with cfg) enabled later, the later entry wins for mtp.…output_layer….weight_quantizer — fnmatch's * crosses ., so a leading/trailing wildcard reaches into mtp.. And because the recipe entry carries a cfg, it's a full replacement with implicit enable: true, not just a toggle.
3. The bigger issue for this PR: lightning_w4a16_nvfp4_4o6.yaml never imports default_disabled_quantizers. Its imports: are only configs/numerics/nvfp4_four_over_six and configs/numerics/fp8; it builds its list from its own {'*': enable: false} deny-all plus explicit enables. And tools/launcher/.../NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/megatron_lm_qad.yaml:54 points QUANT_CFG straight at that recipe, not at a preset. So the mtp.* line added to the unit has no effect on the Lightning flow — MTP is still picked up by the recipe's own *mixer.experts*weight_quantizer, *mlp.experts*weight_quantizer, and *output_layer*weight_quantizer patterns, which match MCore names under mtp..
So @kevalmorabia97's original suggestion is still needed: add the disable as the last entry of the recipe's own quant_cfg (after the *output_layer*weight_quantizer entry at line 73). Keeping the unit entry as well is fine — it covers the preset-based paths (MAMBA_MOE_*_CFG) where the import sits at the end.
4. Nit on the pattern: mtp.* is anchored at the start of the quantizer name, unlike its sibling entries in the same unit and unlike the existing convention elsewhere — '*mtp*' in huggingface/qwen3_5/ptq/w4a16_nvfp4-fp8_attn-kv_fp8_cast.quant_cfg.yaml:87, huggingface/qwen3_5_moe/ptq/nvfp4_experts_mse-fp8_rest-kv_fp8.yaml:59, and "*mtp*" in configs/auto_quantize/units/base_cost_excluded_layers.yaml:24. Quantizer names come from named_modules() on whatever root is passed to mtq.quantize, so mtp.* matches a bare MCore GPTModel (mtp.layers.0.…) but misses HF-side names (model.mtp.…) and any wrapped root (module.mtp.…). '*mtp*' would be consistent and robust.
No code changed — this was a question. · branch jennifchen/lightning_nvfp4
There was a problem hiding this comment.
i think usually this default_disabled yaml is used in the beginning of a quant config .. to disable mtp.output_layerweight_quantizer I can explicitly add that in my lightning recipe
cjluo-nv
left a comment
cjluo-nv
left a comment
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.
Answer to the ordering question first (default_disabled_quantizers vs. the 4/6 recipe):
quant_cfgis a flat ordered list and the last matching entry wins (docs/source/guides/_quant_cfg.rst→ "Ordering and Precedence"; enable-only entries flip state without resetting attributes). There is no implicit "defaults applied at the end" step —default_disabled_quantizersonly takes effect where a preset/recipe$imports it, at exactly that position in the list.- In the presets it is imported after the enable rules (
configs/ptq/presets/model/mamba_moe_nvfp4_aggressive.yaml:base_disable_all→w4a4_nvfp4_nvfp4→default_disabled_quantizers→mamba_moe_disabled_quantizers), so the newmtp.*disable does win there. That's the path the Nano QAD switch toMAMBA_MOE_NVFP4_AGGRESSIVE_CFGuses, so the new entry is effective for that pipeline. - Yes to your specific worry: if a recipe re-enables after the import (the
qwen3_5recipes do exactly this for*lm_head*weight_quantizer), the later enable wins, and because the enable patterns are leading-wildcard (*output_layer*weight_quantizer) they also matchmtp.…output_layer.weight_quantizer— MTP would be re-enabled. Exclusions must come last, or the enable patterns must be scoped. - Important for this PR:
lightning_w4a16_nvfp4_4o6.yamlnever importsdefault_disabled_quantizers, so the newmtp.*entry has no effect on the new Lightning recipe at all, even though the PR body presents the two changes as related. The recipe's own*mlp.experts*weight_quantizer/*mixer.experts*weight_quantizer/*output_layer*weight_quantizerrules will match MTP-internal modules if the Lightning MCore model carries an MTP block. - Also note
mtp.*is anchored (no leading*), unlike the*mtp*used in the qwen3_5 recipes: it only matches quantizer names that literally start withmtp., so under any wrapper/root prefix (module.,model.,decoder.) it silently matches nothing.
Other findings (details inline): the new launcher pipeline points MLM_MODEL_CFG at the Nano model config and sets MLM_MODEL_SAVE, which quantize.sh unconditionally overrides — as written, task_2 loads a checkpoint path that task_1 never writes; the Nano CONSERVATIVE → AGGRESSIVE switch is an unexplained numerics change (Mamba in_proj/out_proj go from BF16 to NVFP4 W4A4, whereas the published Super/Ultra mirrors deliberately keep them at FP8) bundled into a Lightning PR; recipe file placement/naming deviates from the <checkpoint>/ptq/<scheme>.yaml convention; PR "Testing" section is empty and there is no CHANGELOG entry.
| - --skip-generate | ||
| - --export-default-te-spec | ||
| environment: | ||
| - MLM_MODEL_CFG: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 |
There was a problem hiding this comment.
Bot comment.
MLM_MODEL_CFG is the Nano model config (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) in tasks 1, 2 and 3 of a Lightning pipeline, while every checkpoint path points at Lightning. Every other launcher example keeps MLM_MODEL_CFG equal to the model directory (see NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml). Looks like a copy-paste leftover — building a Lightning MCore checkpoint under the Nano arch config will either fail to load or silently mis-shape. If it is intentional (Megatron-LM has no Lightning cfg yet), please add a comment saying so and fix the path derivation noted below.
There was a problem hiding this comment.
this is intentional, they are the same architecture
| - MLM_MODEL_CFG: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | ||
| - QUANT_CFG: huggingface/models/nvidia/Nemotron-3.5-Lightning-30B-A3B/lightning_w4a16_nvfp4_4o6 | ||
| - MLM_MODEL_CKPT: /cicd/megatron-lm-bf16/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16-MCore | ||
| - MLM_MODEL_SAVE: /cicd/megatron-lm/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 |
There was a problem hiding this comment.
Bot comment.
MLM_MODEL_SAVE set here is a no-op: common/megatron_lm/quantize/quantize.sh does export MLM_MODEL_SAVE="/cicd/megatron-lm/${MLM_MODEL_CFG}" unconditionally. Combined with MLM_MODEL_CFG being the Nano config (line 53), the quantized student is written to /cicd/megatron-lm/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16, while task_2 loads MLM_MODEL_CKPT: /cicd/megatron-lm/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 — a path nothing ever writes. As written this pipeline (with allow_to_fail: false) should fail at the QAD step; the Nano example avoids this by leaving the save path at the MLM_MODEL_CFG default. Please fix MLM_MODEL_CFG (and drop the redundant MLM_MODEL_SAVE, or make quantize.sh respect it).
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
| cfg: | ||
| num_bits: e4m3 | ||
| # MCore lm_head (output_layer) W4A16 4/6 weight-only. | ||
| - quantizer_name: 'output_layer.weight_quantizer' |
There was a problem hiding this comment.
@kevalmorabia97 removed the * wild cards from output_layer so it doesn't capture MTP layers
There was a problem hiding this comment.
What if we use *language_model.*output_layer.weight_quantizer like naming in all rules? So only language_model's submodules are picked up?
There was a problem hiding this comment.
Its not just applicable to output_layer. Other layers in mtp also get quantized. For example
mtp.layers.0.mtp_model_layer.layers.0.self_attention.core_attention.k_bmm_quantizer TensorQuantizer((4, 3) bit fake per-tensor amax=1.52e+01 calibrator=MaxCalibrator quant)
mtp.layers.0.mtp_model_layer.layers.0.self_attention.core_attention.v_bmm_quantizer TensorQuantizer((4, 3) bit fake per-tensor amax=7.00e+00 calibrator=MaxCalibrator quant)
...
mtp.layers.0.mtp_model_layer.layers.1.mlp.experts.linear_fc1.weight_quantizer.0 StaticBlockScaleQuantizer((2, 1) bit fake block_sizes={-1: 16, 'type': 'static', 'scale_bits': (4, 3), 'four_over_six': 1}, amax=[0.0142, 0.1809](311808) calibrator=MseCalibrator quant)
mtp.layers.0.mtp_model_layer.layers.1.mlp.experts.linear_fc1.weight_quantizer.1 StaticBlockScaleQuantizer((2, 1) bit fake block_sizes={-1: 16, 'type': 'static', 'scale_bits': (4, 3), 'four_over_six': 1}, amax=[0.0129, 0.1934](311808) calibrator=MseCalibrator quant)
mtp.layers.0.mtp_model_layer.layers.1.mlp.experts.linear_fc1.weight_quantizer.2 StaticBlockScaleQuantizer((2, 1) bit fake block_sizes={-1: 16, 'type': 'static', 'scale_bits': (4, 3), 'four_over_six': 1}, amax=[0.0122, 0.1685](311808) calibrator=MseCalibrator quant)
There was a problem hiding this comment.
Actually *language_model*.output_layer.weight_quantizer wont work. From .quant_summary.txt, I see language model entries as decoder.layers.* so perhaps we need to use *decoder.*output_layer.weight_quantizer so nothing except LLM decoder is activated by the rules
There was a problem hiding this comment.
output_layer.weight_quantizer works and excludes MTP layers
There was a problem hiding this comment.
The LM head is not under decoder.* in the MCore quantizer namespace. The PTQ summary reports it exactly as output_layer.weight_quantizer; decoder.layers.* applies to the internal model layers. Therefore *decoder.*output_layer.weight_quantizer would not match the LM head at all. I changed the rule to the exact name output_layer.weight_quantizer, and verified in PTQ that it enables the root LM head without enabling any MTP-qualified output layer.
-- codex
There was a problem hiding this comment.
I just ran MBridge PTQ with your recipe and still see same mtp layers quantized as I posted above.
Also MTP doesnt actually have output_layer submodule
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
|
…them Calibration tolerates MTP: process_mtp_loss early-returns when labels is None (multi_token_prediction.py) and megatron_prefill passes none, so mtp_num_layers=0 was never required. Dropping the heads is what forced distill to rebuild the student without them and export to recover the shape from run_config.yaml, so all three compensations go with it. MTP heads are kept and simply not quantized -- 'mtp.*' is disabled in default_disabled_quantizers and in the model recipes (#2146). Verified on Nemotron-3.5-Lightning with main + #2112 + #2113 + this branch: PTQ mtp_num_layers=1, 6660 quantizers, all 278 mtp.* quantizers disabled export "Successfully loaded 270 MTP tensors", 18487 keys QAD 900 quantizers, trained past iteration 50 with no KeyError: "mtp.layers.0.enorm.weight from model not in state dict" Signed-off-by: James Shen <yueshen@nvidia.com>
kevalmorabia97
added
the
cherry-pick-done
#2159 #2112 (#2179) ## Cherry-picked PRs - #1975 - #2076 - #2071 - #2093 - #2084 - #2115 - #2133 - #2146 - #2064 - #2159 - #2112 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added PTQ recipes for Nemotron model families and NVFP4 mixer-MLP quantization. * Added Qwen3-VL multimodal speculative-decoding support, including video inputs. * Added compatibility with multiple vLLM KV-cache layouts and newer Transformers versions. * Added a Nemotron 3.5 Lightning quantization-aware distillation workflow. * **Bug Fixes** * Improved Hugging Face, QLoRA, and PEFT checkpoint exports. * Improved distributed job shutdown when a process encounters an error. * Improved pruning validation and candidate selection for MoE models. * **Documentation** * Updated supported-model lists and recipe paths. * Removed Phi-3 Vision and Phi-4 Multimodal quantization support. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Slawomir Kierat <skierat@nvidia.com> Signed-off-by: h-guo18 <67671475+h-guo18@users.noreply.github.com> Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Signed-off-by: James Shen <yueshen@nvidia.com> Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com> Signed-off-by: Kinjal Patel <kinjalpravin@nvidia.com> Signed-off-by: Shiyang Chen <shiychen@nvidia.com> Signed-off-by: Chenjie Luo <chenjiel@nvidia.com> Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com> Signed-off-by: Suguna Velury <178320438+sugunav14@users.noreply.github.com> Co-authored-by: skierat <skierat@nvidia.com> Co-authored-by: h-guo18 <67671475+h-guo18@users.noreply.github.com> Co-authored-by: yueshen2016 <39203804+yueshen2016@users.noreply.github.com> Co-authored-by: Zhiyu <zhiyuc@nvidia.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: kinjalpatel27 <31936134+kinjalpatel27@users.noreply.github.com> Co-authored-by: sychen52 <41452870+sychen52@users.noreply.github.com> Co-authored-by: Chenjie Luo <108829653+cjluo-nv@users.noreply.github.com> Co-authored-by: Wei-Ming Chen <17592131+meenchen@users.noreply.github.com> Co-authored-by: Jenny Chen <jennifchen@nvidia.com> Co-authored-by: sugunav14 <178320438+sugunav14@users.noreply.github.com>
…otron-3.5-Lightning-30B-A3B (#2142) ## What does this PR do? Adds `mbridge_qad.yaml`, a launcher example running NVFP4 quantization-aware distillation for **Nemotron-3.5-Lightning-30B-A3B** through the Megatron-Bridge scripts in `examples/megatron_bridge/`, alongside the existing `mbridge_prune.yaml` / `mbridge_quantize.yaml`. `megatron_lm_qad.yaml` (#2146) runs the same recipe and the same data through Megatron-LM. This is the Megatron-Bridge counterpart: the same `huggingface/models/nvidia/Nemotron-3.5-Lightning-30B-A3B-BF16/ptq/w4a16_nvfp4_4o6` recipe and the same `nvidia/Nemotron-Post-Training-Dataset-v2` chat data, with the training hyperparameters from our public-data QAD run. Four tasks: tokenize the training data, PTQ the student, distill it against the frozen BF16 teacher, export to unified HF. ### Why the extra tokenize task Megatron-LM's finetune path reads an HF parquet shard directly. Megatron-Bridge trains from pre-tokenized data, so `distill.py` consumes Megatron `.bin`/`.idx` via `--data_paths`. The chat split is therefore tokenized once with `modelopt.torch.utils.plugins.megatron_preprocess_data`. `--hf_streaming` avoids the Arrow cast errors this dataset's nested tool-call fields trigger in non-streaming mode, and `--append_eod` is omitted because chat rows already terminate each conversation via the chat template. ### Details - Training topology 8 nodes x 4 GPUs, TP=1 PP=1 CP=4 EP=16 -> DP=8; `gbs` 64 at `mbs` 1 is 8 gradient-accumulation microbatches. 200 iters x 64 x 32768 = 419M training tokens. - PTQ runs TP=EP=PP=1 across 4 ranks (pure DP), so each rank calibrates on its own shard. `--calib_dataset_name` is left unset, selecting the default public `cnn_nemotron_v2_mix` (cnn_dailymail + Nemotron-Post-Training-Dataset-v2). - Export uses TP=1 (the HF writer does not gather TP shards) and PP=4, splitting 52 layers 13/stage. - Pins `nvcr.io/nvidia/nemo:26.06` like the other `mbridge_*` examples. ## Dependencies Based on `main`; the PTQ recipe ships in #2146 (merged). No other PR required. Nemotron-3.5-Lightning has `tie_word_embeddings: false`, so a correct quantized `lm_head` in the exported checkpoint also depends on #2112. ## Testing The PTQ -> export -> QAD flow and these hyperparameters were run end to end on Nemotron-3.5-Lightning (`main` + #2112 + #2113): - PTQ completed, 6660 quantizers, MTP heads retained (`mtp_num_layers: 1`) with all 278 `mtp.*` quantizers disabled by the recipe. - Export produced a unified-HF checkpoint (18487 keys, including 270 MTP tensors). - QAD trained with 900 quantizers through a validation pass at iteration 50. The YAML itself is validated against the launcher's conventions (`ntasks_per_node == gpus_per_node` on Slurm, single-line `inline`, no `args` alongside `inline`, all `<<global_vars.X>>` resolve, output prefix matches `megatron_preprocess_data`'s naming) and by the repo's `validate launcher YAML references` pre-commit hook. Topology arithmetic checked: EP divides world/(TP*PP), `gbs` divisible by DP*mbs. ## Before your PR is "*Ready for review*" - Is this change backward compatible?: ✅ (new example file only) - Did you write any new necessary tests?: N/A - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: N/A 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an example workflow for NVFP4 quantization-aware distillation of NVIDIA Nemotron 3.5 Lightning 30B-A3B. * Supports dataset tokenization, post-training quantization, teacher-student distillation, and export of a unified Hugging Face checkpoint. * Includes configurable model, dataset, and checkpoint paths, distributed execution settings, and support for local or Slurm-based workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: James Shen <yueshen@nvidia.com>
5 participants
What does this PR do?
Type of change: New example
Add Nemotron Lightning 3.5 NVFP4 recipe and QAD example
Also exclude MTP in default disabled quantizers
Usage
# uv run launch.py --yaml examples/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/megatron_lm_qad.yaml --yesTesting
Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded
trust_remote_code=True,torch.load(..., weights_only=False),pickle, etc.).CONTRIBUTING.md: ✅ / ❌ / N/AAdditional Information
Summary by CodeRabbit
New Features
Bug Fixes
Configuration