hipfire
New in 0.4.4 7900 XTX MTP decode, long-context DFlash and Flash-Next decode-share

Fast local LLMs on the AMD GPU you already own.

hipfire is a Rust inference engine with hand-written HIP kernels for Radeon GPUs and Strix Halo. One binary. No Python, no PyTorch.

curl -fsSL https://hipfire.dev/install.sh | bash

Linux with ROCm 6+ · Windows, NixOS and source builds

hipfire 0.4.1

replay of a recorded run · real timing

Qwen3.8-Flash · Strix Halo · native MTP · greedy, thinking off this run · 482 tokens · 67.6 tok/s server decode · 183 ms to first token
Read recorded transcript

Qwen3.8-Flash on Strix Halo, native MTP, greedy, thinking off. This run: 482 output tokens at 67.6 tok/s server-measured decode; 183 ms client time to first token. The displayed command is the native CLI equivalent of the recorded streaming request.

~ $ hipfire run qwen3.8:flash-next "Write a minimal Python LRU cache with per-key TTL and two pytest tests"

```python
import time
from collections import OrderedDict
from typing import Any, Optional


class LRUCache:
    """A minimal LRU cache with per-key TTL support."""

    def __init__(self, capacity: int):
        self.capacity = capacity
        self.cache: OrderedDict[str, tuple[Any, Optional[float]]] = OrderedDict()

    def get(self, key: str) -> Optional[Any]:
        """Get value by key. Returns None if not found or expired."""
        if key not in self.cache:
            return None
        value, expires_at = self.cache[key]
        if expires_at is not None and time.time() > expires_at:
            del self.cache[key]
            return None
        # Move to end (most recently used)
        self.cache.move_to_end(key)
        return value

    def put(self, key: str, value: Any, ttl: Optional[float] = None) -> None:
        """Put value with optional TTL in seconds."""
        expires_at = time.time() + ttl if ttl is not None else None
        if key in self.cache:
            self.cache.move_to_end(key)
        self.cache[key] = (value, expires_at)
        if len(self.cache) > self.capacity:
            self.cache.popitem(last=False)


# Tests
import pytest


def test_lru_eviction():
    cache = LRUCache(capacity=2)
    cache.put("a", 1)
    cache.put("b", 2)
    # Access "a" to make it most recently used
    assert cache.get("a") == 1
    # Add "c", should evict "b" (least recently used)
    cache.put("c", 3)
    assert cache.get("b") is None
    assert cache.get("a") == 1
    assert cache.get("c") == 3


def test_ttl_expiry():
    cache = LRUCache(capacity=10)
    cache.put("x", 42, ttl=0.1)
    assert cache.get("x") == 42
    time.sleep(0.15)
    assert cache.get("x") is None
```

What's new in 0.4.4

Faster 7900 XTX speculative decode, long-context DFlash and Flash-Next decode-share on the R9700.

01

Faster 7900 XTX speculative decode

Qwen3.8-27B with native MTP reaches 141.1 tok/s on prose-1K (7900 XTX, 2026-10-26). NUMERIC, not byte-identical; each change has a kill switch.

02

Long-context DFlash on the R9700

The native-fp8 split-KV verifier is default on: 27B decode at 128K goes 18.93 → 33.70 tok/s (one R9700, 3 fresh processes per arm). NUMERIC, not byte-identical.

03

Flash-Next decode-share on the R9700

On by default and byte-identical: MTP decode 61.82 vs 51.14 tok/s with it off (one R9700, 3 rounds of fresh processes, 2026-10-27).

04

Dense TP on the mesh executor

serve --tp N loads the generic mesh executor: TP2 pp2048 2,746.1 → 3,358.5 tok/s on R9700s (2026-10-26). Vision is on by default with a streamed tower.

Models and hardware

From one Strix Halo APU to a multi-GPU rig.

Models

  • Qwen3.8-Flash
    2,472.7 tok/s 16K prefill on Strix Halo with the NPU (2026-10-27)
  • Qwen3.8 27B
    141.1 tok/s decode on a 7900 XTX with native MTP, prose-1K (2026-10-26)
  • Qwen3.6 35B-A3B
    MoE, recurrent state cached across turns
  • Qwen 3.5
    qwen3.5:9b is the quickstart model
  • DeepSeek V4 Flash
    multi-GPU, tensor + expert parallel
  • Ornith 1.5
    MoE / hybrid
  • FLUX.1 schnell · FLUX.2 Klein
    image generation on RDNA 3 / 3.5
  • Bring your own
    quantize Hugging Face safetensors
Model tags and BYO

GPUs

  • Radeon RX 7900 XTX / XT
    gfx1100 · RDNA 3
    Primary target
  • Radeon AI PRO R9700 · RX 9070 XT
    gfx1201 · RDNA 4
    61.82 tok/s Qwen3.8-Flash-Next MTP decode (11-prompt suite, 2026-10-27)
    Supported
  • Strix Halo APU
    gfx1151 · RDNA 3.5
    2,472.7 tok/s Qwen3.8-Flash 16K prefill with the NPU (2026-10-27)
    Supported
  • Radeon RX 6950 XT
    gfx1030 · RDNA 2
    Supported
  • Radeon RX 5700 XT
    gfx1010 · RDNA 1
    Experimental
  • Instinct MI50 / MI60
    gfx906 · Vega 20
    Community
  • Instinct MI300X
    gfx942 · CDNA 3
    Partial
Multi-GPU, including mixed architectures

Why hipfire

Written for RDNA, not ported to it.

No PyTorch. No Python.

A Rust engine that calls the HIP runtime directly and runs kernels written and tuned per chip. Nothing to pip install.

One binary

On Radeon cards hipfire needs the HIP runtime and nothing else from ROCm: no rocBLAS, MIOpen or hipBLASLt to version-match.

OpenAI-compatible API

hipfire serve exposes /v1/chat/completions on :11435, with tools and stop sequences, so existing clients and agent harnesses connect unchanged.

Simple pull, run, serve

pull, run and serve with curated model tags, or quantize your own from Hugging Face safetensors.

Also in the box: DFlash speculative decodingMagnum V2 quantizationcompressed KV cachemulti-GPU, mixed architecturesimage generationvision sidecarsNixOS module
The RDNA-native thesis

Quickstart

Running in three commands.

Install, pull a model, and serve an OpenAI-compatible API on :11435.

$curl -fsSL https://hipfire.dev/install.sh | bash
$hipfire pull qwen3.5:9b
$hipfire serve -d   # OpenAI-compatible API on :11435