hipfire 0.4.3 benchmarks

Qwen3.6-35B-A3B · prefill on the R9700, 0.4.2.1 vs 0.4.3

Prefill tok/s on one Radeon AI PRO R9700 (gfx1201), measured 2026-10-24 in a single timing hold with the three arms interleaved. Each cell is the median of 9 samples: 3 fresh hipfire bench processes with --runs 3. “0.4.2” here is v0.4.2.1 (6b363a56a1ae), the latest 0.4.2 release.

Command: hipfire bench <model> --matrix --json --pp 512,1024,4096,8192,16384 --tg 1 --ctx 1 --runs 3 --warmups 1 --spec off, synthetic token ids. The new qwen3.6:35b-a3b-mq4rs quant is where most of the gain comes from (3.5× to 5.5× over the same 0.4.3 engine on the old mq4r file). The 0.4.3 engine alone is 1.0× to 1.8× on mq4r and removes the 0.4.2 long-context falloff (0.4.2.1 loses 41 % from pp512 to pp16384; 0.4.3 on mq4r does not fall off). Source: v0.4.3 release notes, “Prefill perf comparison”.

Read this before comparing columns

Each version ran its default config, and the defaults differ. The 0.4.2.1 → 0.4.3 mq4r column is therefore “what a user gets from upgrading without changing the model file”, not a single-variable engine A/B.

The symmetric IU4 MoE prefill route is only admitted on the mq4rs file. Fast mq4rs prefill is gfx1201 only; on other GPUs the mq4rs file needs HIPFIRE_MOE_SYM_IU4=0 (about 5× slower prefill). mq4rs file: hipfire-models/qwen3.6-35b-a3b, 18.7 GB, sha256 8806a742d558ebe43256010096cd82cfac3a20be5218ad56a3f74267db001894; the mq4r arms used the registry-pinned file, sha256 4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428. WikiText-2 KLD, same harness: 0.129 for mq4rs against 0.206 for mq4r. Decode was not compared here (--tg 1).

Late prefill steps on mq4rs (pp16000)

The last lever steps of the cycle, on qwen3.6-35b-a3b-d53.mq4rs only and not part of the table above: pp16000 (two chunks, 8192 + 7808 rows), 3 fresh processes per arm, three separate same-session A/Bs.

The +15.6 % total is the sum of the three, not a single A/B; the card runs a few percent slower on some days than others. The routes are exact-gfx1201 defaults, byte-identical to the code they replace: KLD is unchanged on mq4rs (the four-corpus KLD sequence is byte-equal at every step). That does not hold for the asymmetric mq4r file: its WikiText-2 prefill KLD moves 0.2082 → 0.2115 (Δ +0.0034, CI95 [+0.0009, +0.0058]), inside the quality bar. Source: v0.4.3 release notes, “Detail” and Known issues.

Qwen3.6-35B-A3B mq4rs · decode on the R9700, 0.4.3 → 0.4.3.1

The symmetric-expert A3B file used to decode on the HIP graph. In 0.4.3.1 it takes the automatic PM4 replay route on an exact gfx1201 card (one GPU, no drafter), the same default .mq4r already had. Output is bit-exact against the HIP route. Nothing to configure; replay.backend = "hip" turns it off.

Measured on a Radeon AI PRO R9700 with hipfire bench <model> --spec off --matrix --pp 128 --ctx 128,8192 --tg 128 --runs 3 --warmups 2, default config for both arms (the HIP arm set with replay.backend = "hip"), 3 fresh processes per arm. Greedy ids are identical between the two routes on prose, code, copy and 1024-token reasoning prompts, and the replay harness reports logits, KV and recurrent state bit-equal. This changes the decode route only; prefill is unaffected (mq4rs prefill stays about 20K tok/s at 4K–8K prompts, as in 0.4.3). It does not apply to other GPUs. Source: v0.4.3.1 release notes.

The 0.4.3.1 release notes' validation run reports tg128 at ctx 128 of 231.4 / 232.3 / 231.9 tok/s over 3 fresh processes, with the PM4 route bit-exact against the HIP route. --spec mtp now also loads on mq4rs, but the default stays AR and an MTP-loaded request does not use PM4 replay.

Qwen3.8-Flash (Flash-Next) · prefill and decode on the R9700

Change-level campaigns from the 0.4.3 release notes, not a cumulative release matrix: each row names its card, fixture and arms. Numbers from different sessions are not chained.

Source: v0.4.3 release notes, “Headline numbers” rows 1–4 and Highlights. The 16,384-row chunk is the default on gfx1201 under compact scratch; for short prompts with long generations set HIPFIRE_PREFILL_CHUNK_ROWS=4096. With a 4096-row chunk the pass is limited by the PCIe expert stream and the W1–W4 wins mostly hide. Opt-outs: HIPFIRE_MQ6_I8=0, HIPFIRE_QWEN4_COMPACT_SCRATCH=0.

Expert hot set (on by default on gfx1201)

hipfire pull qwen3.8:flash-next fetches a 213 KB decode-routing sidecar and the load keeps the most-used experts resident in VRAM (gfx1201 only). On real text it is worth +14 % to +26 % MTP decode (fiction prose +14.4 %, glimmer coding prompt +26.3 %) and +7 % to +40 % plain AR, with no prefill loss (pp16384 +0.5 %) and byte-identical output. The copy-heavy code edit under MTP is the exception at −3.1 %. Fixture: one Radeon AI PRO R9700, split base with and without the expert hot-set sidecar, 16,384-row default chunk, q4 PLE rows, 128 tokens, 3 fresh processes per arm. Source: v0.4.3 release notes, Highlights.

The release notes also publish this earlier on/off table, measured before the forward-scratch diet, the W1–W4 waves, the compact-scratch default and the 16,384-row chunk, and not re-measured on the final build. Treat it as a direction, not a current figure.

HIPFIRE_QWEN4_EXPERT_HOTSET=0 restores whole-layer placement. A phase-aware budget is planned for 0.4.4. Source: v0.4.3 release notes, “Flash-Next on a single R9700”.

Strix Halo (gfx1151 + XDNA2) · Flash-Next 16K prefill

Client time to first token on a 16,355-token prompt, XDNA2 NPU trunk on, 3 fresh processes × 3 requests, all-9 median, engine commit 9d74311a53, measured 2026-10-23.

The first request in each process runs slower than steady state; in this run the first requests were 2,314.2, 2,316.0 and 2,208.9 tok/s. The 0.4.2 and 0.4.3 figures are separate measurements, not a controlled A/B. The 0.4.3.1 release notes report that Strix Halo load, prefill and generate match v0.4.3. Source: v0.4.3 release notes, Headline numbers and Known issues; v0.4.3.1 release notes, Validation.

Strix Halo decode and speculation (contributor measurements)

These figures are the contributor's own (@fivetide) and were not re-measured on Strix Halo hardware for this release.

Other measured changes in 0.4.3

Source: v0.4.3 release notes, Highlights, § Strix Halo, § 7900 XTX and narrow PCIe links, Fixes. The 7900 XTX narrow-link rows have one hold of evidence.

New Flash-Next tiers: mq3-xts and mq2-xts

Smaller split-base Flash-Next files than the 125 GB mq4-xts. They load their per-token embedding rows from a shared PLE sidecar (q4 29.1 GB, q8 54.4 GB). These tiers lose some quality against the default mq4-xts; measured KLD per tier is on the model cards.

Example from the model card: mq2-xts (6,306 hot experts) decode on a 7900 XTX measures KLD 0.1966 on WikiText-2. KLD numbers are not comparable across tiers or against MQ4 without the card's scope notes. Strix Halo refuses the mq2 tiers cleanly in this release. Model cards: hipfire-models/qwen3.8-flash-next. Source: v0.4.3 release notes, “New Flash-Next model tiers”.

Reproduce it

# install hipfire (latest release), or pin one
$ curl -L https://raw.githubusercontent.com/warpfront/hipfire/master/scripts/install.sh | bash
$ curl -L https://raw.githubusercontent.com/warpfront/hipfire/master/scripts/install.sh | bash -s -- --tag v0.4.3.1

# A3B prefill and decode on a gfx1201 card (R9700)
$ hipfire pull qwen3.6:35b-a3b-mq4rs
$ hipfire bench qwen3.6:35b-a3b-mq4rs --matrix --json --pp 512,1024,4096,8192,16384 \
    --tg 1 --ctx 1 --runs 3 --warmups 1 --spec off
$ hipfire bench qwen3.6:35b-a3b-mq4rs --spec off --matrix --pp 128 --ctx 128,8192 \
    --tg 128 --runs 3 --warmups 2

Each version's default config applies. Releases: v0.4.3 (CHANGELOG), v0.4.3.1 (CHANGELOG). Protocol: BENCHMARKS.md.

Known limits

• The A3B speed-up applies to files built with the symmetric expert recipe, which is qwen3.6:35b-a3b-mq4rs. Fast mq4rs prefill is gfx1201 only, its router and lm_head are 4-bit, and --spec mtp is opt-in there (the default stays AR).
• On the asymmetric qwen3.6:35b-a3b-mq4r file, 0.4.3 prefill numerics change slightly and greedy output can differ from 0.4.2 at near-ties. Users on the old mq4r should pull mq4rs.
• Flash-Next on the R9700 prefills faster and decodes 5–10 % slower by default (16,384-row chunk); the earlier hot-set on/off table predates the final defaults.
• On Strix Halo, each process exit leaves about 80 GiB in the kernel TTM page pool, so a back-to-back reload can be refused until that memory frees; the fix is planned for 0.4.4. The first request in each process also runs slower than steady state.
• Several Strix Halo and 7900 XTX results are the contributors' own measurements and were not re-measured on that hardware for this release.
• Qwen3.8-27B --spec mtp failed to load on v0.4.3 and loads again in 0.4.3.1.
• The PM4 replay decode speed-up does not apply to other GPUs, to MTP-loaded requests or to a drafter.

Source: v0.4.3 and v0.4.3.1 release notes, Known issues and Upgrade notes.