Image generation

hipfire generates images from two diffusion families on AMD RDNA GPUs:

Both run through the same daemon message (img_generate), the same HTTP endpoint (POST /v1/images/generations) and the same CLI (hipfire img). Klein also edits: reference images condition every denoise step without being denoised.

The daemon loads HFQ component packs only. A diffusers pipe directory is the input of the packer (hipfire-quantize --flux-pipe), never a model path.

This guide is the shortest path to a first image on a developer box. The crate-level details (modules, A/B env flags, gate bars) are in crates/hipfire-arch-diffusion/README.md.

Quick start

Five commands from a clean checkout to a first image (Klein 4B, the smaller model; the download dominates the time):

cargo build --release
export HIPFIRE_DAEMON_BIN=$PWD/target/release/daemon
huggingface-cli download black-forest-labs/FLUX.2-klein-4B --local-dir ~/models/flux-klein-pipe
./target/release/hipfire-quantize --flux-pipe ~/models/flux-klein-pipe --output ~/models/flux-klein.hfq
./target/release/hipfire img ~/models/flux-klein-transformer.hfq "a red bicycle leaning on a stone wall, photo" --out bike.png

Edit that image with a reference (Klein only; --image goes before the prompt):

./target/release/hipfire img --image bike.png ~/models/flux-klein-transformer.hfq "make the bicycle blue" --out bike-blue.png

The sections below explain each step, the FLUX.1 path, the HTTP API, the tests and what to do when something fails.

1. Requirements

• An RDNA3 or RDNA3.5 GPU with the ROCm HIP runtime installed. hipfire dlopens HIP at run time, so the build needs no ROCm. RDNA4 (gfx12) is refused at load: the FLUX kernels use the gfx11 wave32 WMMA intrinsics. Hardware seen to run it (historical characterization, measurement dates not recorded; not an admission): gfx1151 (Radeon 8060S, unified memory; FLUX.1 schnell and Klein), gfx1150 (Radeon 890M, Klein only), gfx1100 (RX 7900 XT: kernels pass, FLUX.1 schnell does not fit in 24 GB — see § 10).
• Rust stable and disk for the pipe plus its packs: about 90 GB for FLUX.1 schnell (54 GB pipe + 34 GB packs), 30 GB for Klein 4B.
• huggingface-cli (or any HF download tool) for the weights.

2. Build

cargo build --release
# target/release/hipfire, target/release/daemon, target/release/hipfire-quantize

The CLI prefers an installed daemon under ~/.hipfire/bin. Point it at the fresh build for every command below:

export HIPFIRE_DAEMON_BIN=$PWD/target/release/daemon

3. Get the weights

Download a pipe root in the diffusers layout (transformer/, text_encoder*/, tokenizer*/, vae/, scheduler/):

# FLUX.1 schnell (Apache-2.0):
huggingface-cli download black-forest-labs/FLUX.1-schnell --local-dir ~/models/flux-schnell-pipe
# FLUX.2 Klein 4B (Apache-2.0; the 9B is non-commercial):
huggingface-cli download black-forest-labs/FLUX.2-klein-4B --local-dir ~/models/flux-klein-pipe

FLUX.1 dev is gated and non-commercial; the same commands work on its pipe.

4. Pack

The packer is CPU-only. It converts the weights to F16 (bias, scale and norm statistics stay F32) and embeds each component’s config, the scheduler config and the tokenizers in the pack headers, so the pipe directory is not needed after this step. Historical characterization (one 32-core CPU host, measurement date not recorded, no current claim): about 25 minutes for FLUX.1 schnell and 12 minutes for Klein 4B.

./target/release/hipfire-quantize --flux-pipe ~/models/flux-schnell-pipe --output ~/models/flux-schnell.hfq
# -> flux-schnell-transformer.hfq (40), -t5.hfq (41), -clip.hfq (42), -vae.hfq (43)
./target/release/hipfire-quantize --flux-pipe ~/models/flux-klein-pipe --output ~/models/flux-klein.hfq
# -> flux-klein-transformer.hfq (45), -qwen3.hfq (46), -vae.hfq (43)

The trunk pack is the model path. The sidecar packs are found next to it by name (<base>-t5.hfq and so on, or the shared t5-xxl.hfq / clip-l.hfq / qwen3.hfq / vae.hfq). Details: QUANTIZE.md.

5. First image

The first run compiles the kernels for your GPU and uploads the weights, so expect it to take longer than the numbers below.

./target/release/hipfire img ~/models/flux-schnell-transformer.hfq \
  "a tiny lighthouse on a rock at sunset, photo" \
  --width 512 --height 512 --steps 4 --seed 0 --out lighthouse.png --json

The JSON result names the file, size, seed, steps and wall time. The same seed gives a byte-identical PNG on the same GPU.

Flag defaults, from the CLI and daemon: --width / --height default to 1024 for plain txt2img (with --image, the reference image’s own size) and must be at most 8192; --steps defaults to the architecture default (4 for schnell, 28 for dev, 4 for Klein) and must be 1 to 128; --seed defaults to 0; --out defaults to hipfire-<seed>.png in the working directory; --image is repeatable up to 4 times and needs a Klein (arch 45) pack. A model argument that is a registry tag (flux.schnell:1) is pulled first when it is not already local. --backend gpu|cpu picks the transformer forward; the default is gpu when a GPU is available. cpu is the f32 reference oracle and takes minutes per step at real geometry.

Klein txt2img and reference edit (--image must come before the prompt):

./target/release/hipfire img ~/models/flux-klein-transformer.hfq \
  "a red bicycle leaning on a stone wall, photo" --width 512 --height 512 --out bike.png
./target/release/hipfire img --image bike.png ~/models/flux-klein-transformer.hfq \
  "make the bicycle blue" --out bike-blue.png

Historical characterization, not a current performance claim: one gfx1151 unified-memory host, f16 packs, 512x512, 4 steps, hipfire img, after warm-up; measurement date not recorded, not a default or a floor.

6. HTTP

./target/release/hipfire serve 127.0.0.1 11580 --model ~/models/flux-schnell-transformer.hfq --idle-timeout 0 &
curl -s -X POST http://127.0.0.1:11580/v1/images/generations \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"a tiny lighthouse on a rock at sunset","size":"512x512","steps":4,"seed":0,"response_format":"b64_json"}' \
  | jq -r '.data[0].b64_json' | base64 -d > lighthouse.png

n must be 1 and response_format must be b64_json. Besides size, the body accepts width / height, steps, seed and backend; model is only echoed, because the model must already be loaded (--model above). An images field is refused (use /v1/images/edits), as is negative_prompt and any sampler other than euler. Malformed bodies return 400 with a message; a text model on this endpoint, or a diffusion model on /v1/chat/completions, is refused.

Reference edit over HTTP is the OpenAI multipart route (Klein pack loaded; one to four image parts):

curl -s -X POST http://127.0.0.1:11580/v1/images/edits \
  -F image=@bike.png -F prompt="make the bicycle blue" -F steps=4 -F seed=0 \
  | jq -r .data[0].b64_json | base64 -d > bike-blue.png

The image bytes travel in the request. Neither route accepts a server path, so an HTTP client cannot make the daemon read a file. Field reference: SERVE.md.

7. Tests and gates

No GPU, no weights:

cargo test -p hipfire-arch-diffusion -p hipfire-loader -p hipfire-quantize --lib

Kernel parity gates on your GPU (each one compares against a CPU reference and prints PASS per case):

for e in test_rope_2d_flux test_modulate_f32 test_layernorm_modulate_parity \
         test_qk_rmsnorm_rope_parity test_copy_rows_strided_f32_parity \
         test_gemm_f16_x_f16_wmma_lds_parity test_gemm_wide_lds_parity \
         test_gemm_epilogue_parity test_attention_flux_vt_parity \
         test_attention_text_gqa test_vae_lds; do
  cargo run --release --features lab -p rdna-compute --example $e || break
done

Model-level gates read the pipe directory (they need the raw weights for the CPU oracle) and a GPU; they live under crates/hipfire-arch-diffusion/examples/ and are listed with their purpose in that crate’s Cargo.toml. The whole-denoise-loop gates compare against the committed ComfyUI golden latents:

cargo run --release --features lab -p hipfire-arch-diffusion --example gpu_flux_golden_latent -- \
  ~/models/flux-dev-pipe crates/hipfire-arch-diffusion/tests/fixtures/flux-golden
cargo run --release --features lab -p hipfire-arch-diffusion --example gpu_klein_golden_latent -- \
  ~/models/flux-klein-pipe crates/hipfire-arch-diffusion/tests/fixtures/klein-golden/4b

The serve harness runs the HTTP path end to end, with seeded within- and cross-process byte parity and the fail-closed 400 cases:

python3 scripts/serve_harness.py --mode images --model ~/models/flux-klein-transformer.hfq \
  --port 11530 --img-width 512 --img-height 512 --img-steps 4

8. Profiling knobs

HIPFIRE_IMG_PROFILE=1 prints per-stage wall time; HIPFIRE_PROFILE=1 adds a per-kernel-family table per denoise step. Every other HIPFIRE_FLUX_* / HIPFIRE_VAE_* flag is an A/B knob whose default is the measured winner; the table in the crate README lists them.

9. Known limits

• One image per request; euler (flow-match) is the only sampler, and negative_prompt is refused (no CFG pass).
• FLUX.1 runs f16 activations by default (HIPFIRE_FLUX_F16_ACT=0 for f32).
• Width and height must be multiples of 16: the VAE factor is 8 and the latent is packed in 2x2 patches. Klein never upscales a reference image; it is area-capped at 1 MP.
• Noise is hipfire’s own seeded generator, not torch-compatible: the same seed does not reproduce a diffusers or ComfyUI image. The golden-latent gates start from the fixture’s captured init latent for that reason.
• The flux.schnell:1 registry entry points at the public elphil/flux Hub repo (Apache-2.0); hipfire pull flux.schnell:1 fetches the packs directly. flux.dev:1 is a local-only placeholder (the dev weights cannot be redistributed), and there is no Klein registry entry: pack Klein locally with hipfire-quantize --flux-pipe (section 4).

10. Troubleshooting

• ... is a diffusers pipe directory, which is not loadable — pack it first (section 4) and pass the trunk pack.
• ... needs the t5 and clip sidecar packs next to it — the sidecars are looked up next to the trunk by name; keep the packer’s output together.
• FATAL: GPU GPU-... already reserved by holder PID ... — another daemon owns that physical GPU. Stop it or select a different card with hardware.devices; a private HOME does not bypass the lock.
• A very slow first image — kernel JIT plus the weight upload. Run once more before you read any timing.
• Out of device memory on FLUX.1 — the T5 and CLIP encoders fall back to the host (slow, correct), but the transformer itself needs about 24.3 GB resident (23.8 GB f16 weights + row padding + activations), so a card with 24 GB or less (RX 7900 XT/XTX) fails at the first activation alloc. FLUX.1 needs a unified-memory APU (Strix Halo) or a 32 GB+ RDNA3 card; Klein 4B fits.