[Pfannkuchensack]

Summary

Wan 2.2 GGUF transformers that store non-block tensors as BF16 fail on the first denoise step with RuntimeError: mat1 and mat2 must have the same dtype, but got Byte and BFloat16. One example is darksidewalker/DaSiWa-WAN2.2-I2V v11 Q4_K_M, which stores time_embedding, text_embedding, time_projection and head as BF16.

gguf_sd_loader wraps BF16 like a quantized type, so the tensor reports its uint8 storage dtype. Wan's GGUF loader had its own unwrap helper, which only unpacked F32/F16 and left BF16 packed. diffusers' WanTimeTextImageEmbedding casts the timestep to next(time_embedder.parameters()).dtype, which here is uint8. A release that quantizes the time embedder fails the same way.

The Wan GGUF transformer loader now uses the shared unpack_ggml_at_load, like every other GGUF transformer loader. It unpacks:

• unquantized tensors, reading BF16 as a byte view;
• every non-Linear parameter;
• the model's declared skip modules: patch_embedding, condition_embedder and norms.

Quantized block Linears stay packed. The private helper is removed.

Related Issues / Discussions

User report on the v7 beta, DaSiWa Wan 2.2 I2V GGUF.

QA Instructions

• pytest tests/backend/model_manager/load/test_wan_loader.py tests/backend/model_manager/load/test_wan_checkpoint_loader.py tests/backend/quantization/gguf: 142 passed. These ran CPU-only.

• The new test, test_gguf_loader_unpacks_bf16_and_quantized_condition_embedder_weights, uses a tiny real WanTransformer3DModel. It feeds in the following tensor types:

  • BF16 and Q8_0 condition-embedder weights;
  • an F16 patch_embedding and scale_shift_table;
  • F32 norms;
  • a Q8_0 block Linear.

  It checks that the BF16 bytes are reinterpreted exactly, the Q8_0 weights match gguf-py's dequantization, and the block Linear stays packed. It then runs a full forward pass. Against main's loader it fails because the BF16 time-embedder weight is still packed.

• Real file, RTX 4090: DaSiWa v11 I2V A14B high and low Q4_K_M pair, 832×480, 17 frames, 6 steps.

  • On main the first run fails with mat1 and mat2 must have the same dtype, but got Byte and BFloat16.
  • On this branch, two videos complete in 59 s and 50 s.

• On CPU with the same file, condition_embedder gives:

  • on main: a GGMLTensor reporting torch.uint8, then the error above;
  • on this branch: a bf16 parameter and a finite output.

Review

No material findings remain open.

Memory, worked out from the file headers:

• On the DaSiWa file, resident size does not change. The six BF16 tensors are 443 MiB packed or unpacked.
• The roughly 1.5 GB per-forward dequantization transient of the BF16 time_proj is gone.
• The loader now reserves RAM through make_room like the other GGUF loaders. That is about 0.3 GiB above the file size; the extra is the in-flight term.
• Files that ship the condition embedder in F16/F32, such as the QuantStack releases, already unpacked it, so for them only the reservation changes.
• A release that quantizes the condition-embedder Linears now holds them unpacked. For 14B that is about +0.3 GB per expert. This follows the skip patterns diffusers declares for the model, and those files did not load before.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

[joshistoast]

wan small fix for tensors, wan giant leap for video gen