MQ V2 family

Status: qt=47 MQ6G256V2, qt=48 MQ5G256V2, qt=49 MQ3G256V2, and qt=50 MQ2G256V2 are wired for dense WMMA prefill on gfx11 + gfx12 (gfx1100/1101/1102/1150/1151 and gfx1200/gfx1201) beside qt=44 MQ4G256V2 (see mq4-v2.md). Register disposition is passthrough for all four (qt-register.txt). Wire support ≠ product admission: MQ2V2 remains quality-rejected (§8.2).

One-line summary: same neutral-size Magnum V2 header as MQ4V2 — 8 B of dual half fp16 scale + fp16 zero per 128 weights — on top of the legacy bit-width payloads from MQ6/5/3/2 v1. Group byte counts and nominal codec bpw are unchanged from v1 at each bit width.

Wire implementation ≠ product admission. A format that loads, decodes, and passes parity can still be rejected as a product body. In particular MQ2V2 is measured and rejected for product quality (catastrophic KLD; decoded text coherent but degraded/repetitive). Do not call MQ2V2 shipping.

Product tier names, rung recipes, and the full 15-cell Qwen3.8 ladder live in ladder.md and docs/perf-checkpoints/2026-08-20-qwen38-mq-v2-product-ladder.md. This file is the byte contract a loader/kernel implementer needs.

1 · Family table

Constants (source of truth):

• crates/rdna-compute/src/dispatch.rs — MQ6G256V2_GROUP_BYTES=200, MQ5G256V2_GROUP_BYTES=168, MQ3G256V2_GROUP_BYTES=104, MQ2G256V2_GROUP_BYTES=72 (and MQ4V2_GROUP_BYTES=136 for qt=44).
• Encoders: quantize_mq6g256v2 / quantize_mq5g256v2 in crates/hipfire-quantize/src/quant_fwht.rs; quantize_mq3g256v2 / quantize_mq2g256v2 in crates/hipfire-quantize/src/quant_mq.rs.
• RAW_CODECS: qt 47→MQ6G256V2, 48→MQ5G256V2, 49→MQ3G256V2, 50→MQ2G256V2 in crates/hipfire-runtime/src/weight_backend.rs.
• qwen35 dtype_from_quant_type: 47/48/49/50 map to the same four DTypes and must not collapse to legacy MQ6/5/3/2.

qt=44 is specified fully in mq4-v2.md. It is listed here only so the family header contract is visible in one place.

2 · Shared header (all five V2 formats)

Little-endian throughout. Per 256-weight group:

Reconstruction:

h = (weight_index >= 128) ? 1 : 0
w = q * f32(scale[h]) + f32(zero[h])

Half 0 owns q[0..128); half 1 owns q[128..256).

Geometry invariants

• K % 256 == 0 (loader and encoder both assert).
• Tensor blob length = M * (K/256) * GROUP_BYTES for that format.
• Group alignment is 8-byte (header is four fp16s).
• Payload offset is always +8, same as v1 MQ*/HFQ* G256 affine formats.
• FWHT-256 rotation is baked into the weights at encode time; runtime rotates x on the Magnum path (RotationPlan::FwhtG256).

Why the halves land on lane boundaries

Same argument as MQ4V2: decode kernels index contiguous packs per lane so that weights 0–127 and 128–255 do not straddle a half-wave. Header dwords stay lane-invariant scalar loads. Derive the half select from the kernel’s own payload addressing; do not hard-code a lane id without re-deriving (see mq4-v2.md §4).

3 · Payload packing (legacy, bit-width specific)

Payload packing is unchanged from the matching v1 format. Only the header is V2.

3.1 · MQ6G256V2 — 6-bit, 4 values / 3 bytes

For i = 0,4,8,…,252 with byte offset bo = 8 + (i/4)*3:

q0..q3 in 0..63
out[bo+0] =  q0        | (q1 << 6)
out[bo+1] = (q1 >> 2)  | (q2 << 4)
out[bo+2] = (q2 >> 4)  | (q3 << 2)

192 payload bytes / group.

3.2 · MQ5G256V2 — 5-bit, 8 values / 5 bytes

For i = 0,8,16,…,248 with bo = 8 + (i/8)*5:

q0..q7 in 0..31
out[bo+0] =  q0        | (q1 << 5)
out[bo+1] = (q1 >> 3)  | (q2 << 2) | (q3 << 7)
out[bo+2] = (q3 >> 1)  | (q4 << 4)
out[bo+3] = (q4 >> 4)  | (q5 << 1) | (q6 << 6)
out[bo+4] = (q6 >> 2)  | (q7 << 3)

160 payload bytes / group.

3.3 · MQ3G256V2 — 3-bit LE bitstream, 8 values / 3 bytes

For chunk = 0..31, ci = chunk*8, bo = 8 + chunk*3, each qq[j] = q[ci+j] & 7:

b0 = (qq[0] & 7) | ((qq[1] & 7) << 3) | ((qq[2] & 3) << 6)
b1 = ((qq[2] >> 2) & 1) | ((qq[3] & 7) << 1) | ((qq[4] & 7) << 4) | ((qq[5] & 1) << 7)
b2 = ((qq[5] >> 1) & 3) | ((qq[6] & 7) << 2) | ((qq[7] & 7) << 5)

96 payload bytes / group.

3.4 · MQ2G256V2 — 2-bit, 4 values / byte

For i = 0..63:

byte = (q[4i] & 3) | ((q[4i+1] & 3) << 2) | ((q[4i+2] & 3) << 4) | ((q[4i+3] & 3) << 6)
out[8 + i] = byte

64 payload bytes / group.

3.5 · MQ4G256V2 — 4-bit nibbles

See mq4-v2.md §2: 128 B nibble payload at [8..136), byte-identical to qt=1/6/13.

4 · Encoder contract

For each 256-weight group, after FWHT-256 (cpu_fwht_256 with the tensor’s sign seeds):

for h in {0, 1}:
    lo = min(w[128h .. 128h+127])
    hi = max(w[128h .. 128h+127])
    levels_max = 2^bits - 1          # 63 / 31 / 15 / 7 / 3
    step_f32 = (hi > lo) ? (hi - lo) / levels_max : 0
    scale[h] = f16(step_f32)         # force 0 bits if hi == lo
    zero[h]  = f16(lo)
    # REQUIRED: quantize against ROUND-TRIPPED fp16 values
    st = f32(scale[h]); z = f32(zero[h])
    degenerate = (hi == lo) || (step_f32 == 0) || (st == 0)
    if degenerate:
        q[half] = 0
    else:
        for i in half h:
            q[i] = clamp(floor((w[i] - z) / st + 0.5), 0, levels_max)
pack header LE: s0, z0, s1, z1
pack payload with the bit-width rules in §3

Degenerate half: scale=0, zero=f16(lo), q=0 → decoder reproduces lo exactly.

Round-tripping s/z through fp16 before selecting q is mandatory. Skipping it makes encoder reconstruction disagree with the kernel.

v1 MQ6/5/3/2 used a single f32 scale+zero per 256 and the same payload packing. V2 only changes the header geometry and the per-half fit.

5 · Decoder contract

hA = load_u32(group + 0)   // s0 | z0
hB = load_u32(group + 4)   // s1 | z1
hs = half0 ? hA : hB       // half from payload address, not a raw lane id
sc = f16_to_f32(hs & 0xFFFF)
zp = f16_to_f32(hs >> 16)
// unpack q from payload (§3)
w  = q * sc + zp

Mis-routing a V2 tensor to a v1 kernel (f32 scale/zero per 256 at the same payload offset) decodes every weight near ~1e-14. Dispatch fail-closes that class of error in crates/hipfire-dispatch/src/families/gemm.rs (explicit qt47–50 vs v1 / cross-V2 key guards). Equal group stride across bit widths is not interchangeable either (e.g. MQ3V2 104 B vs MQ4V2 136 B).

6 · Support matrix (runtime)

Dense Magnum V2 (qt47–50; qt44 by reference in mq4-v2.md) is not gfx12-only. Batched prefill WMMA admits on HasWmma arches (gfx11 family + gfx12). Production weight-reuse (batch-tile) defaults are measured per arch / op / bit-width in rdna-compute::gemm::mqv2_prefill_batch_tile. Capture and replay deliberately bypass that adaptive policy and keep fixed launch contracts.

Prefill weight-reuse (production defaults)

Values are independent 16-token output tiles sharing one dequantized weight tile. Thresholds are N≥96 unless a band is noted. Source of truth: crates/rdna-compute/src/gemm.rs (mqv2_prefill_batch_tile).

MQ4V2 adaptive bands on gfx1100 (fuller narrative in mq4-v2.md §9):

• QKV / QKVZA: BT4 @ N≥96, BT12 @ N≥192
• gate_up: BT6 @ N≥96, BT12 @ N≥384
• residual: BT4 @ 96–223, BT6 @ 224–287, BT8 @ 288–415

Parity / kernel gates:

• Family wire oracle: crates/hipfire-runtime/examples/mqv2_family_parity.rs (byte count, header divergence, sentinel, plain GEMV B=1, B=16 batched lm-head vs CPU dequant).
• Ladder BT screens: crates/rdna-compute/examples/test_mqv2_bt_gfx11.rs, test_mqv2_qkv_bt_gfx1201.rs; MQ4V2 per-arch BT suites test_mq4v2_*_bt_gfx{1100,1151,1201}.rs.

7 · Codec MSE (tool-level)

Measured on 278,528 real post-FWHT G256 blocks from Qwen3.8-27B layer-20 down-proj (M=5120, K=17408) with tools/quant-design/optimize_mq_lowbit_ladder.hip. Each comparison keeps the payload, header budget, group stride, and codec bpw fixed; only the header changes from one G256 f32 affine pair to two G128 fp16 affine pairs.

The matched HIP microbench (tools/quant-design/bench_mq_lowbit_kernels.hip, 100 warmups, 1,000 interleaved samples, gfx1201) passed host/GPU sentinels for all formats. MQ5V2 was throughput-neutral versus MQ5 v1; MQ6V2 was throughput-neutral at B=1 and 2.4% faster in the B=16 shared-weight proxy.

Codec MSE is not a KLD proxy. Product quality is §8 / the ladder checkpoint.

8 · Product admission boundary (measured, not claimed shipping)

Evidence host: hiptrx, 4×gfx1201, ROCm/HIP 7.14. Fixture: Qwen3.8-27B, 26,895,998,464 params. Scoring: WT2 + v6sel, 24 chunks, prompt md5 253c7ac50857fe6d0e10fb0d2c5e35c0. Ladder summary generated 2026-08-20T14:13:16Z, commit bfe3cfead5856c5ee85e972b609712b0badbd1b1, source ssh://hiptrx/home/kaden/qcal/ladder-v2/summary.json.

Acceptance rate from native generate is null (acceptance_reason: unavailable_in_native_generate_v1).

8.1 · Measured KLD / AR (body format = cell format)

Artifact sha256 (full):

Fixed-tier overlays (product recipe, not wire):

• mq2-pro: lm_head:mq6v2,ssm_out:mq6v2
• mq3-pro: ssm_out:mq6v2
• all other cells: none

8.2 · MQ2V2 product rejection

MQ2V2 codec math is valid (encoder round-trips, kernels decode, parity gate covers qt=50). Product quality is not:

• WT2/v6sel KLD ≈ 12–14 on every mq2 rung (three orders of magnitude above mq3+).
• Serve battery (serve-mq2.json): no attractor/empty flags, but outputs are coherent-yet-degraded / repetitive (runaway on the 5-prompt battery).
• Do not advertise MQ2V2 as a shipping body format. Wire support remains for research, drafts, and mechanical parity.

MQ3–MQ6 V2 serve batteries (serve-mq3.json … serve-mq6.json): coherent, no attractor/empty. Product rung selection still belongs in ladder.md, not here.

8.3 · DFlash same-bit vs mq4v2 control (decode tok/s medians)

Draft digests (from summary):

Median τ is present in the remote summary per cell; acceptance remains null.

9 · Name and coexistence with v1

Artifacts self-describe by qt. V2 never redefines a v1 number. Dedicated kernel symbols / TUs (not a compile-time header macro on v1 sources) keep byte consumption explicit per dtype.

10 · Implementer checklist

1. Header: four LE fp16s at +0/+2/+4/+6; payload at +8.
2. GROUP_BYTES and payload pack match §1–§3 exactly.
3. Encode with fp16 round-trip + degenerate half rule (§4).
4. Decode w = q*sc + zp with half from payload geometry (§5).
5. K % 256 == 0; blob length = M*(K/256)*GROUP_BYTES.
6. Register RAW_CODECS + supports_awq_sidecar for the dtype.
7. Route only to matching V2 kernel keys; refuse v1/cross-width keys.
8. Dense WMMA prefill on HasWmma (gfx11 base + gfx12); apply measured weight-reuse defaults per arch/bits; MoE mixed path stays fail-closed. Capture/replay retain fixed contracts.
9. Pass mqv2_family_parity before claiming wire readiness.
10. Never promote MQ2V2 as product-quality from wire readiness alone.

11 · References

• qt-register.txt — qt 47–50 passthrough rows
• mq4-v2.md — qt=44/45 detailed sibling
• ladder.md — product names / rungs / honesty invariant
• docs/perf-checkpoints/2026-08-20-qwen38-mq-v2-product-ladder.md — immutable ladder checkpoint (peer-owned; historical fixture scope)
• docs/perf-checkpoints/2026-08-23-mq-v2-crossarch-prefill-reuse.md — multi-arch prefill weight-reuse (gfx1151 all ops bits2–6; gfx1201 QKV BT8 bits2/5/6, MQ3 base)
• crates/rdna-compute/src/dispatch.rs — group-byte constants, AWQ allow-list
• crates/rdna-compute/src/gemm.rs — mqv2_prefill_batch_tile production policy
• crates/hipfire-quantize/src/quant_fwht.rs — MQ6/5 V2 encoders
• crates/hipfire-quantize/src/quant_mq.rs — MQ3/2 V2 encoders
• crates/hipfire-runtime/examples/mqv2_family_parity.rs — parity gate
• crates/rdna-compute/examples/test_mqv2_bt_gfx11.rs, test_mqv2_qkv_bt_gfx1201.rs — ladder BT screens
• ssh://hiptrx/home/kaden/qcal/ladder-v2/summary.{json,md,csv} — measured ladder