MagnumQuant and FWHT rotation

01 Outliers waste a 4-bit grid

Group quantization stores, per group of weights, a scale and a zero point, and each weight as a small integer (16 levels at 4 bits). One large weight stretches the group's range, and every other weight in the group gets squeezed into a few of the 16 levels.

Multiply the group by an orthogonal matrix that mixes every element with every other one and the outlier's energy spreads across the group. The range shrinks, and the 16 levels are spent where the weights are.

02 The rotation: FWHT-256

Each 256-weight group is multiplied by a fixed rotation built from a Walsh–Hadamard transform between two ±1 sign diagonals:

R = D2 · H · D1 / 16          (H: 256×256 Hadamard, D1/D2: fixed ±1 signs, 1/√256 = 1/16)

offline:     w′ = R w   →  quantize w′
inference:   x′ = R x       (the same R, not its inverse)
             y  = (w′)ᵀ x′ = wᵀ Rᵀ R x = wᵀ x      because R is orthogonal

• Cheap. The Hadamard part is a butterfly: 8 stages of adds and subtracts for 256 elements, no multiplies until the 1/16 scale.
• Free-ish at inference. The activation rotation is fused into RMSNorm (fused_rmsnorm_mq_rotate), so it rides on a pass the model already makes.
• Deterministic. The sign tables come from fixed seeds (42 and 1042) shared by the quantizer and the runtime.
• Exact in math, not in floats. The cancellation is exact in real arithmetic; in fp32 it adds rounding drift that is typically below the quantization error.

Rule of thumb from hipfire's docs: MQ for Qwen 3.5+, HFQ for everything else. The rotation was calibrated against the Qwen 3.5/3.6 hybrid weight distributions; on Llama-style dense models the math still cancels but there is usually no quality lift, only the extra rotation cost.

03 The V2 group layout

MQ V2 keeps an 8-byte header per 256 weights but spends it on two halves: an fp16 scale and an fp16 zero for weights 0–127, and another pair for 128–255. The payload follows at byte 8.

  byte  0      2      4      6      8                                   136
        ┌──────┬──────┬──────┬──────┬─────────────────────────────────────┐
        │ sc₀  │ zp₀  │ sc₁  │ zp₁  │ 256 × 4-bit codes (128 bytes)       │
        └──────┴──────┴──────┴──────┴─────────────────────────────────────┘
          fp16 pair, weights 0–127    fp16 pair, weights 128–255

MQ2 V2 loads and decodes but is quality-rejected as a product body; MQ3–MQ6 V2 are measured candidates on the product ladder. MQ4 (V2) is the default for Qwen 3.5+ safetensors.

04 Symmetric grids and int4 WMMA

An affine group decodes a code q as sc·q + zp. If the zero point is pinned to zp = −8·sc, that becomes sc·(q − 8): a signed 4-bit integer times a scale. Signed int4 is exactly what RDNA's integer WMMA consumes (v_wmma_i32_16x16x16_iu4 on gfx11, v_wmma_i32_16x16x32_iu4 on gfx12), so with activations also quantized to 4 bits (A4), the expert GEMM becomes an integer matrix multiply with one scale applied at the end, instead of dequantizing every weight to fp16 first.

0.4.1's default Flash-Next pull (qwen3.8:flash-next, file qwen3.8-flash-next-gptq3.mq4) is a symmetric GPTQ requant: its routed experts are symmetric MQ4 V2, solved with Hessian GPTQ, and its trunk projections are MQ6 V2. At load, hipfire checks every routed expert header on the device for the symmetric grid; a layer takes the IU4 route only if every header passes. On Strix Halo and the R9700, prefill chunks of 512 rows or more on passing layers then run deterministic expert grouping, A4 activations and grouped IU4 GEMMs built with PeaceMaker. Asymmetric artifacts fail the check and keep the F16 WMMA route; decode and MTP never take it.

05 What it buys you

06 Limits and switches

07 Sources

• docs/QUANTIZATION.md: weight families, V2 header layout, “FWHT rotation (the M in MQ)”.
• PRIOR-ART.md § 4: MagnumQuant origin and canonical files.
• docs/quant-formats/mq-v2-family.md.
• CHANGELOG, v0.4.1 (GPTQ3 default pull, symmetric IU4 MoE route) and the v0.4.1 release notes (numbers, opt-outs).