what runs on the NPU
Two consecutive Flash-Next prefill banks (8,192-row chunks, or a rebalanced last pair) run their GDN and QSA trunk input projections on the NPU, layer by layer, and the GPU reads the NPU's row-major arenas directly. Everything else stays on the GPU. Other architectures do no NPU I/O.
default and kill switch
| setting | values | default |
|---|---|---|
fusions.npu_trunk / HIPFIRE_QWEN4_NPU_TRUNK | off | auto | on | emulate | auto |
fusions.npu_trunk_calibration / HIPFIRE_QWEN4_NPU_TRUNK_CAL | path | <hipfire home>/npu-trunk/calibration.json, else the built-in Halo calibration |
auto arms only with an open NPU and a calibration whose key matches this device and build;
any refusal logs one npu-trunk: off (reason) line and the load runs GPU-only. on makes
a refusal a load error. emulate is the GPU-only bit-exact twin: greedy on equals
emulate. Turn it off with HIPFIRE_QWEN4_NPU_TRUNK=off or
fusions.npu_trunk = off. Full reference: CONFIG.md.
host memlock limit
The armed trunk pins 1,053,843,456 B (1005 MiB) of userptr arenas against
RLIMIT_MEMLOCK. At open it raises the soft limit to exactly what it needs when the hard limit
allows; otherwise auto logs WARN npu-trunk: off (...) with the bytes and the fix and
serves GPU-only. The 8 MiB default of ssh sessions and many desktop logins is too small. Grant a
bounded limit, 1,318,060,032 B (1257 MiB); infinity is not needed.
# systemd unit or `systemctl edit` drop-in for the service that runs `hipfire serve` [Service] LimitMEMLOCK=1318060032 # /etc/security/limits.d/90-hipfire-npu.conf (pam_limits logins; value in KiB) <user> soft memlock 1287168 <user> hard memlock 1287168 # or, in a shell whose hard limit already allows it $ ulimit -l 1287168
Tailscale SSH skips pam_limits and always starts at 8 MiB, so use the systemd drop-in or prlimit
there. Source: docs/npu/npu-trunk.md, "Host memlock limit".
PLE prefetch and HC fusion
Two exact changes ship on the NPU route in 0.4.2, with no setting to change:
- The fused HC kernels (attention HC gate hand-off, output-projection epilogue, MoE HC row fold) now also run under the NPU pair route. Greedy
on,emulateandoffbytes match. - Bank B's per-layer-embedding rows are fetched ahead and the trunk arenas request transparent huge pages. Greedy is byte-identical; there is no kill switch.
measured
16K prefill on the shipped v0.4.2 tip, NPU trunk on, bounded memlock, 16,355 tokens,
3 fresh processes × 3 requests: all-9 median TTFT 6,854 ms
(2,386 tok/s). auto vs off cuts 16K TTFT by 10.1 %.
Tables and sources on the 0.4.2 benchmarks page; see also the
release notes and CHANGELOG.