hipfire

Benchmarks · measured 2026-10-14

hipfire 0.4.2 benchmarks

hipfire 0.4.2 runs Qwen3.8-Flash (qwen3.8:flash-next) trunk input projections on the Strix Halo XDNA2 NPU by default, makes paired DFlash the 27B default and speeds it up on Strix Halo, and adds width-8 and IU4 verify for batching serves. Each number below names its card, fixture and source.

2,386tok/s
Qwen3.8-Flash 16K prefill on Strix Halo with the NPU
16,355-token prompt · TTFT 6,854 ms · median of 3 × 3
−10.1%
16K time to first token, NPU trunk auto vs off
Strix Halo · default serve config · 3 fresh processes per arm
135.3tok/s
27B DFlash code decode on Strix Halo
up from 97.9 (+38 %) · exact 16-row verify GEMMs
60.2tok/s
Qwen3.8-Flash output speed on Strix Halo
greedy decode with native MTP · median of 3 runs over 8 prompts

Strix Halo (gfx1151 + XDNA2) · 16K prefill with the NPU trunk

Two consecutive Flash-Next prefill banks run their GDN and QSA trunk input projections on the NPU while the GPU consumes the NPU's arenas directly. Client TTFT on a 16,355-token prompt, NPU trunk on, bounded memlock (LimitMEMLOCK=1318060032), 3 fresh processes × 3 requests.

build all-9 median TTFT prefill tok/s steady requests
v0.4.2 shipped tip 6ea3f5076 6,854 ms 2,386 6,849.5 ms
release gate 1be80ed54 (16,356 tokens) 7,085 ms 2,309 7,079 ms

Measured 2026-10-14. The tip is -3.4 % TTFT against the gate build, measured interleaved, and its greedy 16K output is byte-identical to the gate build. Source: release notes, Headline numbers row 1 and § Strix Halo NPU trunk (j327 shipped-tip measurement).

NPU trunk auto vs off

promptoff TTFTauto TTFTchange
9,330 tokens 4,625 ms 4,319 ms -6.6 %
16K (16,355 tokens) 7,878 ms 7,085 ms -10.1 %
24,438 tokens 11,944 ms 11,159 ms -6.6 %
56K 30,047 ms 27,260 ms -9.3 %

2026-10-13, default serve config, 3 fresh processes per arm, median of per-process medians. Greedy on equals the bit-exact GPU-only emulate twin; ΔKLD WT2 −0.0005 / code +0.0052, within the +0.015 cap. Source: release notes § Strix Halo NPU trunk, off → auto table (CHANGELOG NPU trunk entry).

Two exact additions after the gate build

HC fusion under the NPU pair route

The attention HC gate hand-off, the output-projection epilogue and the MoE HC row fold now fire under the NPU pair route as they do GPU-only; greedy on, emulate and off bytes match. 16K all-9 median TTFT 6,896 ms against 7,194 ms (2,372 against 2,274 tok/s) on base 93720b4fb, interleaved, 3 × 3 per arm.

PLE prefetch and huge-page arenas

Bank B's per-layer-embedding rows are fetched ahead and the trunk arenas ask for transparent huge pages; exact, no kill switch. 16K TTFT 7,054 ms against 7,111 ms: no slower, gain not resolved beyond noise. On a loaded host THP backed only about 44 % of the arenas.

Source: release notes § Strix Halo NPU trunk, PLE prefetch and HC fusion paragraphs.

Strix Halo · prefill and decode, 0.4.2 next to 0.4.1

The 0.4.1 Strix Halo release-matrix cells rerun on v0.4.2 with the same protocol: fresh-process medians of three repetitions, warm sample discarded, decode over the same eight committed prompts. The 0.4.1 column is copied from the 0.4.1 page.

cell 0.4.2 median tok/s min – max 0.4.1 median 0.4.1 min – max
synthetic pp8192, cold prefill 2156.1 2155.4 – 2157.8 2072.9 2072.7 – 2076.9
32K prompt file, cold prefill 2305.9 2298.4 – 2318.0 1776.5 1776.3 – 1780.3
decode, greedy AR 34.15 34.14 – 34.21 33.78 33.77 – 33.88
decode, greedy native MTP 60.20 60.14 – 60.36 58.67 58.54 – 58.76
decode, sampled native MTP + presence penaltya 60.90 60.13 – 61.91 56.98 45.55 – 57.21

Measured 2026-10-14 on build 6ea3f5076 (v0.4.2 code; daemon md5 e4ccd5c9393642134c307ebf2787d430), Qwen3.8-Flash (qwen3.8:flash-next, qwen3.8-flash-next.mq4-xts) with release defaults: q4 PLE, the trunk overlay and the NPU trunk on auto under the bounded memlock limit. The NPU pair route fires on the 32K prompt; pp8192 and decode sit below its crossover and run GPU-only. 32K prompt file md5 3028a2c129301fe50d285927a3603299. Both releases' prefill runs show the Strix Halo fast-PPT power limit engaging. Greedy AR and MTP output is bit-identical across fresh processes. a Sampled MTP uses T0.7 / top-p 0.8 / top-k 20, a presence penalty of 1.5 and seed 123. In 0.4.2 its token streams differ across fresh processes on 3 of the 8 prompts (known issue: the wall-time route chooser), so this row is a rate over slightly different texts. Receipts: rel-matrix-042/SUMMARY.md, REPORT-table.md and runs/<cell>/r{1,2,3}/result.json.

Measured changes in 0.4.2

Change-level campaigns, not a cumulative release matrix: each row names its own card, fixture and arms, and rows are not chained. Synthetic pp8192 is not prompt-file prefill.

changehardware and fixtureresult
27B DFlash, exact 16-row verify GEMMs Strix Halo · 27B mq4-xts + paired draft · 3 fresh processes per arm code 97.9 → 135.3 tok/s (+38.2 %); prose 19.0 → 26.3 tok/s; AR unchanged
27B paired DFlash by default One R9700 · mq4-xts · C1 sampled, effort xhigh, 4,096-token cap · 3 fresh processes per arm 44.00 → 74.49 tok/s (+69.3 %) vs MTP auto
DFlash width 8 on batching serves One R9700 · 27B mq4-xts + MQ4V2 draft · 64 code prompts × 256, C8 steady greedy 254.6 → 324.6 tok/s (+27.5 %); C1 114.8 → 107.8 (−6.1 %)
IU4 verify at width 8 (separate campaign) Same R9700 fixture · C8 steady greedy · 8 fresh processes per arm 340.1 → 374.5 tok/s (+9.9 %); C1 −7.7 %
Flash-Next EP2 owner-routed prefill (opt-in EP) 2× R9700 · Flash-Next · committed prompts · 3 fresh processes per arm p8k / p16k: one card 3,033 / 3,754 → EP2 3,943 / 4,308 tok/s
Deep appends reuse the prompt Strix Halo · Flash-Next · +512 tokens onto a 192K / 256K prompt 9.3 → 141.8 / 4.7 → 197.5 tok/s
gfx1100 Qwen3.5-4B decode fusions (#845) RX 7900 XTX · Qwen3.5-4B decode +5.5 % (MQ4), +3.0 % (MQ3), bit-identical
q4 PLE on Strix Halo (new default there) Flash-Next · 8K committed prompt, steady after the default prewarm · one run per cell 1828.4 tok/s (q8 about 1,599)

Sources, in row order: release notes, Headline numbers (CHANGELOG gfx1151 XLDS entry, 2026-10-13); release notes, Headline numbers (CHANGELOG 27B DFlash default entry, 2026-10-12); release notes, Headline numbers (CHANGELOG speculation.dflash_block entry, 2026-10-11); release notes, Headline numbers (CHANGELOG DFlash IU4 verify entry, 2026-10-11); release notes, Headline numbers (CHANGELOG EP owner-routed prefill entry, 2026-10-13); release notes, Headline numbers (CHANGELOG deep-append entry, 2026-10-11); release notes, Headline numbers (CHANGELOG #845 entry, 2026-10-13); release notes, Headline numbers (CHANGELOG PLE sidecar entry, 2026-10-11). Width 8 and IU4 each cost a lone request (C1) on a batching-configured serve; HIPFIRE_DFLASH_BLOCK=16 and HIPFIRE_CB_VERIFY_IU4=0 restore the previous behaviour.

NPU trunk: memlock and defaults

Bounded memlock

The armed trunk pins 1,053,843,456 B of userptr arenas. Grant the serve process LimitMEMLOCK=1318060032 (systemd) or ulimit -l 1287168; infinity is not needed. Short of it, auto logs WARN npu-trunk: off (...) and serves GPU-only, and on is a load error.

On by default on Strix Halo

fusions.npu_trunk / HIPFIRE_QWEN4_NPU_TRUNK defaults to auto: it arms only with an open NPU and matching keyed calibration, otherwise logs one reason and runs GPU-only. Other architectures do no NPU I/O. HIPFIRE_QWEN4_NPU_TRUNK=off is the kill switch.

Details: /docs/npu-trunk and docs/npu/npu-trunk.md ("Host memlock limit"). Source: release notes § Strix Halo NPU trunk, Memlock.

Reproduce it

# install hipfire 0.4.2 and pull the exact artifact
$ curl -L https://raw.githubusercontent.com/warpfront/hipfire/master/scripts/install.sh | bash
$ hipfire pull qwen3.8:flash-next       # 116.7 GiB

# Strix Halo: allow the NPU trunk's pinned arenas (bounded, not infinity)
$ ulimit -l 1287168
$ hipfire serve qwen3.8:flash-next
artifactchecksum
qwen3.8-flash-next.mq4-xts sha256 · 125,288,544,792 B8b15b6fede7d7c5bfed0db4720a8295bedda51bc93e545fa242bd50d0f200972

Model: hipfire-models/qwen3.8-flash-next. qwen3.8-flash-next.mq4-xts (MQ4 XTS) is the canonical Flash-Next pin; qwen3.8-flash-next-gptq3.mq4, the "GPTQ3 MQ4" file on the 0.4.1 page, is a byte-identical legacy alias. Protocol: BENCHMARKS.md, CHANGELOG, release notes.

Known limits

  • The NPU trunk needs about 1.05 GB of memlock; ssh sessions and many desktop logins start at 8 MiB. Tailscale SSH skips pam_limits, so use the systemd drop-in or prlimit there.
  • Width 8 and IU4 verify each cost a lone request on batching serves (−6.1 % and −7.7 % at C1, separate campaigns); neither is a universal speed-up.
  • The first request after load still lags steady state under memory reclaim; q4 PLE warming is bounded.
  • Two- and four-card EP is opt-in and experimental; 42-token TTFT on four cards is slower than one card.
  • Halo kernel pins cover the warm cache only; the cold-JIT first-chunk cost is not fixed.

Source: release notes, Highlights, § Strix Halo NPU trunk, § Width and IU4, § PLE, § Lanes and EP, § Kernels.