MQ V2 family

Status: qt=47 MQ6G256V2, qt=48 MQ5G256V2, qt=49 MQ3G256V2, and qt=50 MQ2G256V2 are wired for dense WMMA prefill on gfx11 + gfx12 (gfx1100/1101/1102/1150/1151 and gfx1200/gfx1201) beside qt=44 MQ4G256V2 (see mq4-v2.md). Register disposition is passthrough for all four (qt-register.txt). Wire support ≠ product admission: MQ2V2 remains quality-rejected (§8.2).

One-line summary: same neutral-size Magnum V2 header as MQ4V2 — 8 B of dual half fp16 scale + fp16 zero per 128 weights — on top of the legacy bit-width payloads from MQ6/5/3/2 v1. Group byte counts and nominal codec bpw are unchanged from v1 at each bit width.

Wire implementation ≠ product admission. A format that loads, decodes, and passes parity can still be rejected as a product body. In particular MQ2V2 is measured and rejected for product quality (catastrophic KLD; decoded text coherent but degraded/repetitive). Do not call MQ2V2 shipping.

Product tier names, rung recipes, and the full 15-cell Qwen3.8 ladder live in ladder.md and docs/perf-checkpoints/2026-08-20-qwen38-mq-v2-product-ladder.md. This file is the byte contract a loader/kernel implementer needs.

1 · Family table

Constants (source of truth):

• crates/rdna-compute/src/dispatch.rs — MQ6G256V2_GROUP_BYTES=200, MQ5G256V2_GROUP_BYTES=168, MQ3G256V2_GROUP_BYTES=104, MQ2G256V2_GROUP_BYTES=72 (and MQ4V2_GROUP_BYTES=136 for qt=44).
• Encoders: quantize_mq6g256v2 / quantize_mq5g256v2 in crates/hipfire-quantize/src/quant_fwht.rs; quantize_mq3g256v2 / quantize_mq2g256v2 in crates/hipfire-quantize/src/quant_mq.rs.
• RAW_CODECS: qt 47→MQ6G256V2, 48→MQ5G256V2, 49→MQ3G256V2, 50→MQ2G256V2 in crates/hipfire-runtime/src/weight_backend.rs.
• qwen35 dtype_from_quant_type: 47/48/49/50 map to the same four DTypes and must not collapse to legacy MQ6/5/3/2.

qt=44 is specified fully in mq4-v2.md. It is listed here only so the family header contract is visible in one place.

2 · Shared header (all five V2 formats)

Little-endian throughout. Per 256-weight group:

Reconstruction:

h = (weight_index >= 128) ? 1 : 0
w = q * f32(scale[h]) + f32(zero[h])

Half 0 owns q[0..128); half 1 owns q[128..256).

Geometry invariants

• K % 256 == 0 (loader and encoder both assert).
• Tensor blob length = M * (K/256) * GROUP_BYTES for that format.
• Group alignment is 8-byte (header is four fp16s).
• Payload offset is always +8, same as v1 MQ*/HFQ* G256 affine formats.
• FWHT-256 rotation is baked into the weights at encode time; runtime rotates x on the Magnum path (RotationPlan::FwhtG256).

Why the halves land on lane boundaries

Same argument as MQ4V2: decode kernels index contiguous packs per lane so that weights 0–127 and 128–255 do not straddle a half-wave. Header dwords stay lane-invariant scalar loads. Derive the half select from the kernel’s own payload addressing; do not hard-code a lane id without re-deriving (see mq4-v2.md §4).

3 · Payload packing (legacy, bit-width specific)

Payload packing is unchanged from the matching v1 format. Only the header is V2.

3.1 · MQ6G256V2 — 6-bit, 4 values / 3 bytes

For i = 0,4,8,…,252 with byte offset bo = 8 + (i/4)*3:

q0..q3 in 0..63
out[bo+0] =  q0        | (q1 << 6)
out[bo+1] = (q1 >> 2)  | (q2 << 4)
out[bo+2] = (q2 >> 4)  | (q3 << 2)

192 payload bytes / group.

3.2 · MQ5G256V2 — 5-bit, 8 values / 5 bytes

For i = 0,8,16,…,248 with bo = 8 + (i/8)*5:

q0..q7 in 0..31
out[bo+0] =  q0        | (q1 << 5)
out[bo+1] = (q1 >> 3)  | (q2 << 2) | (q3 << 7)
out[bo+2] = (q3 >> 1)  | (q4 << 4)
out[bo+3] = (q4 >> 4)  | (q5 << 1) | (q6 << 6)
out[bo+4] = (q6 >> 2)  | (q7 << 3)

160 payload bytes / group.

3.3 · MQ3G256V2 — 3-bit LE bitstream, 8 values / 3 bytes

For chunk = 0..31, ci = chunk*8, bo = 8 + chunk*3, each qq[j] = q[ci+j] & 7:

b0 = (qq[0] & 7) | ((qq[1] & 7) << 3) | ((qq[2] & 3) << 6)
b1 = ((qq[2] >> 2) & 1) | ((qq[3] & 7) << 1) | ((qq[4] & 7) << 4) | ((qq[5] & 1) << 7)
b2 = ((qq[5] >> 1) & 3) | ((qq[6] & 7) << 2) | ((qq[7] & 7) << 5)

96 payload bytes / group.

3.4 · MQ2G256V2 — 2-bit, 4 values / byte

For i = 0..63:

byte = (q[4i] & 3) | ((q[4i+1] & 3) << 2) | ((q[4i+2] & 3) << 4) | ((q[4i+3] & 3) << 6)
out[8 + i] = byte

64 payload bytes / group.

3.5 · MQ4G256V2 — 4-bit nibbles

See mq4-v2.md §2: 128 B nibble payload at [8..136), byte-identical to qt=1/6/13.

4 · Encoder contract

For each 256-weight group, after FWHT-256 (cpu_fwht_256 with the tensor’s sign seeds):

for h in {0, 1}:
    lo = min(w[128h .. 128h+127])
    hi = max(w[128h .. 128h+127])
    levels_max = 2^bits - 1          # 63 / 31 / 15 / 7 / 3
    step_f32 = (hi > lo) ? (hi - lo) / levels_max : 0
    scale[h] = f16(step_f32)         # force 0 bits if hi == lo
    zero[h]  = f16(lo)
    # REQUIRED: quantize against ROUND-TRIPPED fp16 values
    st = f32(scale[h]); z = f32(zero[h])
    degenerate = (hi == lo) || (step_f32 == 0) || (st == 0)
    if degenerate:
        q[half] = 0
    else:
        for i in half h:
            q[i] = clamp(floor((w[i] - z) / st + 0.5), 0, levels_max)
pack header LE: s0, z0, s1, z1
pack payload with the bit-width rules in §3

Degenerate half: scale=0, zero=f16(lo), q=0 → decoder reproduces lo exactly.

Round-tripping s/z through fp16 before selecting q is mandatory. Skipping it makes encoder reconstruction disagree with the kernel.

v1 MQ6/5/3/2 used a single f32 scale+zero per 256 and the same payload packing. V2 only changes the header geometry and the per-half fit.

5 · Decoder contract

hA = load_u32(group + 0)   // s0 | z0
hB = load_u32(group + 4)   // s1 | z1
hs = half0 ? hA : hB       // half from payload address, not a raw lane id
sc = f16_to_f32(hs & 0xFFFF)
zp = f16_to_f32(hs >> 16)
// unpack q from payload (§3)
w  = q * sc + zp

Mis-routing a V2 tensor to a v1 kernel (f32 scale/zero per 256 at the same payload offset) decodes every weight near ~1e-14. Dispatch fail-closes that class of error in crates/hipfire-dispatch/src/families/gemm.rs (explicit qt47–50 vs v1 / cross-V2 key guards). Equal group stride across bit widths is not interchangeable either (e.g. MQ3V2 104 B vs MQ4V2 136 B).

6 · Support matrix (runtime)

Dense Magnum V2 (qt47–50; qt44 by reference in mq4-v2.md) is not gfx12-only. Batched prefill WMMA admits on HasWmma arches (gfx11 family + gfx12). Production weight-reuse (batch-tile) defaults are measured per arch / op / bit-width in rdna-compute::gemm::mqv2_prefill_batch_tile. Capture and replay deliberately bypass that adaptive policy and keep fixed launch contracts.

Prefill weight-reuse (production defaults)

Values are independent 16-token output tiles sharing one dequantized weight tile. Thresholds are N≥96 unless a band is noted. Source of truth: crates/rdna-compute/src/gemm.rs (mqv2_prefill_batch_tile).

Qwen4 MQ6 Halo tuning is default on gfx1151 inside the Qwen4 forward: unset there (or HIPFIRE_QWEN4_MQ6_X4_TILE=auto anywhere) uses the artifact-sweep table and 0 keeps the incumbent (entries are BV/RW/prefetch; base retains the incumbent):

The a/b/z region fold (48 + 48 + 6144 rows, K=2560) and the HC-write epilogue launch (2560,6144) pick from their own BV8 twins: regions 8/8/2 at N=8192 and 8/8/1 at N=2048; HC-write 8/8/2 at N=8192, base at N=2048. The cold three-process Halo sweep (BF16 HC streams) measured regions 7.177 → 6.972 ms and HC-write 10.502 → 10.100 ms at N=8192 (isolated-kernel medians, not full-prefill savings). The packed-dequant (_dq) entries measured no gain over the same tile without it and are not routed.

Unmeasured shapes and token counts retain the incumbent. The table uses one conservative tile for both output dtypes. Explicit overrides use BVxRWxprefetch with BV ∈ {8,12,16}, RW ∈ {4,8}, and prefetch depth ∈ {1,2}; BV12 overrides fall back to the incumbent unless N % 192 == 0. HIPFIRE_QWEN4_MQ6_X4_REGIONS independently controls the three-F32-output MQ6 a/b/z region fold on the admitted prepared-F16 route: on by default in the Qwen4 forward on gfx1151 (0 off), opt-in 1 on gfx1201. Both switches preserve independent WMMA chains and the existing shared rotation.

The Halo three-run artifact F32 sweep at N=1536 measured medians 1.559106 → 1.455083 ms for (2560,6144) and 2.216403 → 2.089681 ms for (10240,2560); the latter’s BF16 output measured 2.174003 → 2.021261 ms. These are isolated-kernel measurements, not full-prefill or MQ6-group savings. In the initial synthetic sweep, s_setprio slowed the first winner from 1.402878 to 1.620221 ms and was rejected. Artifact-bitwise checks cover all seven trunk shapes, F32/BF16 outputs, N=1536/512/1131/16, K=10240 row tails, extreme headers, zero guards, and poisoned output pads. The three-output fold also matches independent incumbent launches, including N=0.

MQ4V2 adaptive bands on gfx1100 (fuller narrative in mq4-v2.md §9):

• QKV / QKVZA: BT4 @ N≥96, BT12 @ N≥192
• gate_up: BT6 @ N≥96, BT12 @ N≥384
• residual: BT4 @ 96–223, BT6 @ 224–287, BT8 @ 288–415

Parity / kernel gates:

• Family wire oracle: crates/hipfire-runtime/examples/mqv2_family_parity.rs (byte count, header divergence, sentinel, plain GEMV B=1, B=16 batched lm-head vs CPU dequant).
• Ladder BT screens: crates/rdna-compute/examples/test_mqv2_bt_gfx11.rs, test_mqv2_qkv_bt_gfx1201.rs; MQ4V2 per-arch BT suites test_mq4v2_*_bt_gfx{1100,1151,1201}.rs.

7 · Codec MSE (tool-level)

Measured on 278,528 real post-FWHT G256 blocks from Qwen3.8-27B layer-20 down-proj (M=5120, K=17408) with tools/quant-design/optimize_mq_lowbit_ladder.hip. Each comparison keeps the payload, header budget, group stride, and codec bpw fixed; only the header changes from one G256 f32 affine pair to two G128 fp16 affine pairs.

The matched HIP microbench (tools/quant-design/bench_mq_lowbit_kernels.hip, 100 warmups, 1,000 interleaved samples, gfx1201) passed host/GPU sentinels for all formats. MQ5V2 was throughput-neutral versus MQ5 v1; MQ6V2 was throughput-neutral at B=1 and 2.4% faster in the B=16 shared-weight proxy.

Codec MSE is not a KLD proxy. Product quality is §8 / the ladder checkpoint.

8 · Product admission boundary (historical August sweep; not current quality pins)

The data in this section is a historical, fixture-bound August 2026 sweep, kept separate from current quality pins (see ladder.md and ../../AGENTS.md § 5). It records evidence, not shipping status. Full record: ../perf-checkpoints/2026-08-20-qwen38-mq-v2-product-ladder.md.

Evidence host: 4×gfx1201, ROCm/HIP 7.14. Fixture: Qwen3.8-27B, 26,895,998,464 params. Scoring: WT2 + v6sel, 24 chunks, prompt md5 253c7ac50857fe6d0e10fb0d2c5e35c0. Ladder summary generated 2026-08-20T14:13:16Z, commit bfe3cfead5856c5ee85e972b609712b0badbd1b1.

Acceptance rate from native generate is null (acceptance_reason: unavailable_in_native_generate_v1).

8.1 · Measured KLD / AR (body format = cell format)

Artifact sha256 (full):

Fixed-tier overlays (product recipe, not wire):

• mq2-pro: lm_head:mq6v2,ssm_out:mq6v2
• mq3-pro: ssm_out:mq6v2
• all other cells: none

8.2 · MQ2V2 product rejection

MQ2V2 codec math is valid (encoder round-trips, kernels decode, parity gate covers qt=50). Product quality is not:

• WT2/v6sel KLD ≈ 12–14 on every mq2 rung, substantially higher than mq3–mq6 on this fixture.
• Serve battery (serve-mq2.json): no attractor/empty flags, but outputs are coherent-yet-degraded / repetitive (runaway on the 5-prompt battery).
• Do not advertise MQ2V2 as a shipping body format. Wire support remains for research, drafts, and mechanical parity.

MQ3–MQ6 V2 serve batteries (serve-mq3.json … serve-mq6.json): coherent, no attractor/empty. Product rung selection still belongs in ladder.md, not here.

8.3 · DFlash same-bit vs mq4v2 control (decode tok/s medians)

Draft digests (from summary):

Median τ is present in the remote summary per cell; acceptance remains null.

9 · Name and coexistence with v1

Artifacts self-describe by qt. V2 never redefines a v1 number. Dedicated kernel symbols / TUs (not a compile-time header macro on v1 sources) keep byte consumption explicit per dtype.

10 · Implementer checklist

1. Header: four LE fp16s at +0/+2/+4/+6; payload at +8.
2. GROUP_BYTES and payload pack match §1–§3 exactly.
3. Encode with fp16 round-trip + degenerate half rule (§4).
4. Decode w = q*sc + zp with half from payload geometry (§5).
5. K % 256 == 0; blob length = M*(K/256)*GROUP_BYTES.
6. Register RAW_CODECS + supports_awq_sidecar for the dtype.
7. Route only to matching V2 kernel keys; refuse v1/cross-width keys.
8. Dense WMMA prefill on HasWmma (gfx11 base + gfx12); apply measured weight-reuse defaults per arch/bits; MoE mixed path stays fail-closed. Capture/replay retain fixed contracts.
9. Pass mqv2_family_parity before claiming wire readiness.
10. Never promote MQ2V2 as product-quality from wire readiness alone.

11 · References

• qt-register.txt — qt 47–50 passthrough rows
• mq4-v2.md — qt=44/45 detailed sibling
• ladder.md — product names / rungs / honesty invariant
• ../perf-checkpoints/2026-08-20-qwen38-mq-v2-product-ladder.md — immutable ladder checkpoint (historical fixture scope; measured ladder summary)
• docs/perf-checkpoints/2026-08-23-mq-v2-crossarch-prefill-reuse.md — multi-arch prefill weight-reuse (gfx1151 all ops bits2–6; gfx1201 QKV BT8 bits2/5/6, MQ3 base)
• crates/rdna-compute/src/dispatch.rs — group-byte constants, AWQ allow-list
• crates/rdna-compute/src/gemm.rs — mqv2_prefill_batch_tile production policy
• crates/hipfire-quantize/src/quant_fwht.rs — MQ6/5 V2 encoders
• crates/hipfire-quantize/src/quant_mq.rs — MQ3/2 V2 encoders
• crates/hipfire-runtime/examples/mqv2_family_parity.rs — parity gate
• crates/rdna-compute/examples/test_mqv2_bt_gfx11.rs, test_mqv2_qkv_bt_gfx1201.rs — ladder BT screens
• kld/VERIFY.md — public current quality-pin reproduction