Fast local LLMs on the AMD GPU you already own.
hipfire is a Rust inference engine with hand-written HIP kernels for Radeon GPUs and Strix Halo. One binary. No Python, no PyTorch.
curl -fsSL https://hipfire.dev/install.sh | bash Linux with ROCm 6+ · Windows, NixOS and source builds
replay of a recorded run · real timing
Read recorded transcript
Qwen3.8-Flash on Strix Halo, native MTP, greedy, thinking off. This run: 482 output tokens at 67.6 tok/s server-measured decode; 183 ms client time to first token. The displayed command is the native CLI equivalent of the recorded streaming request.
~ $ hipfire run qwen3.8:flash-next "Write a minimal Python LRU cache with per-key TTL and two pytest tests"
```python
import time
from collections import OrderedDict
from typing import Any, Optional
class LRUCache:
"""A minimal LRU cache with per-key TTL support."""
def __init__(self, capacity: int):
self.capacity = capacity
self.cache: OrderedDict[str, tuple[Any, Optional[float]]] = OrderedDict()
def get(self, key: str) -> Optional[Any]:
"""Get value by key. Returns None if not found or expired."""
if key not in self.cache:
return None
value, expires_at = self.cache[key]
if expires_at is not None and time.time() > expires_at:
del self.cache[key]
return None
# Move to end (most recently used)
self.cache.move_to_end(key)
return value
def put(self, key: str, value: Any, ttl: Optional[float] = None) -> None:
"""Put value with optional TTL in seconds."""
expires_at = time.time() + ttl if ttl is not None else None
if key in self.cache:
self.cache.move_to_end(key)
self.cache[key] = (value, expires_at)
if len(self.cache) > self.capacity:
self.cache.popitem(last=False)
# Tests
import pytest
def test_lru_eviction():
cache = LRUCache(capacity=2)
cache.put("a", 1)
cache.put("b", 2)
# Access "a" to make it most recently used
assert cache.get("a") == 1
# Add "c", should evict "b" (least recently used)
cache.put("c", 3)
assert cache.get("b") is None
assert cache.get("a") == 1
assert cache.get("c") == 3
def test_ttl_expiry():
cache = LRUCache(capacity=10)
cache.put("x", 42, ttl=0.1)
assert cache.get("x") == 42
time.sleep(0.15)
assert cache.get("x") is None
```
hipfire 0.4.4, each tile one measurement on the named card, fixture, date and build, not a speed floor. Methods, fixtures and sources on the 0.4.4 benchmarks page.
What's new in 0.4.4
Faster 7900 XTX speculative decode, long-context DFlash and Flash-Next decode-share on the R9700.
Faster 7900 XTX speculative decode
Qwen3.8-27B with native MTP reaches 141.1 tok/s on prose-1K (7900 XTX, 2026-10-26). NUMERIC, not byte-identical; each change has a kill switch.
Long-context DFlash on the R9700
The native-fp8 split-KV verifier is default on: 27B decode at 128K goes 18.93 → 33.70 tok/s (one R9700, 3 fresh processes per arm). NUMERIC, not byte-identical.
Flash-Next decode-share on the R9700
On by default and byte-identical: MTP decode 61.82 vs 51.14 tok/s with it off (one R9700, 3 rounds of fresh processes, 2026-10-27).
Dense TP on the mesh executor
serve --tp N loads the generic mesh executor: TP2 pp2048 2,746.1 → 3,358.5 tok/s on R9700s (2026-10-26). Vision is on by default with a streamed tower.
Models and hardware
From one Strix Halo APU to a multi-GPU rig.
Models
- Qwen3.8-Flash
- Qwen3.8 27B
- Qwen3.6 35B-A3B
- Qwen 3.5
- DeepSeek V4 Flash
- Ornith 1.5
- FLUX.1 schnell · FLUX.2 Klein
- Bring your own
GPUs
- Primary targetRadeon RX 7900 XTX / XTgfx1100 · RDNA 3
- SupportedRadeon AI PRO R9700 · RX 9070 XTgfx1201 · RDNA 461.82 tok/s Qwen3.8-Flash-Next MTP decode (11-prompt suite, 2026-10-27)
- SupportedStrix Halo APUgfx1151 · RDNA 3.52,472.7 tok/s Qwen3.8-Flash 16K prefill with the NPU (2026-10-27)
- SupportedRadeon RX 6950 XTgfx1030 · RDNA 2
- ExperimentalRadeon RX 5700 XTgfx1010 · RDNA 1
- CommunityInstinct MI50 / MI60gfx906 · Vega 20
- PartialInstinct MI300Xgfx942 · CDNA 3
Why hipfire
Written for RDNA, not ported to it.
No PyTorch. No Python.
A Rust engine that calls the HIP runtime directly and runs kernels written and tuned per chip. Nothing to pip install.
One binary
On Radeon cards hipfire needs the HIP runtime and nothing else from ROCm: no rocBLAS, MIOpen or hipBLASLt to version-match.
OpenAI-compatible API
hipfire serve exposes /v1/chat/completions on :11435, with tools and stop sequences, so existing clients and agent harnesses connect unchanged.
Simple pull, run, serve
pull, run and serve with curated model tags, or quantize your own from Hugging Face safetensors.
Quickstart
Running in three commands.
Install, pull a model, and serve an OpenAI-compatible API on :11435.
$curl -fsSL https://hipfire.dev/install.sh | bash $hipfire pull qwen3.5:9b $hipfire serve -d # OpenAI-compatible API on :11435