Serve

hipfire serve starts the native Rust HTTP control plane, owns one GPU daemon process, and exposes an OpenAI-shaped chat API. Defaults come from typed TOML configuration (CONFIG.md); the HTTP surface is implemented by crates/hipfire-cli and the shared transport lives in crates/hipfire-client.

Truth state: shipped / ref-pinned for the HTTP contract and lifecycle below. Branch-only model routes (for example LFM framing details) are not implied by this page — see MODELS.md and VALIDATION.md.

Security (no auth / no TLS)

The native handler implements no authentication and no TLS. Anyone who can reach the bind address can call every endpoint, including chat completions. Default bind is 0.0.0.0 (all interfaces).

• Prefer loopback for local use: hipfire serve 127.0.0.1:11435
• Expose beyond localhost only behind a trusted network or an authenticated TLS reverse proxy you control. Do not publish the raw port to the internet.

Start and stop

hipfire serve                         # foreground; Ctrl-C stops (default bind 0.0.0.0)
hipfire serve 127.0.0.1:11435         # loopback-only (preferred local bind)
hipfire serve -d                      # background (setsid/nohup); polls /health up to 300s
hipfire serve qwen3.5:9b -d           # pre-warm a specific tag this run
hipfire serve 0.0.0.0:11435 -d        # all-interfaces bind — unauthenticated; see Security
hipfire serve --no-prewarm -d         # bind first; load on first request
hipfire serve --kv-mode q8 --idle-timeout 0 -d
hipfire serve --tp 2 -d               # expert-parallel load (multi-GPU arches)

hipfire ps                            # daemons / quantize / uploads + port state (Linux)
hipfire stop                          # SIGTERM tracked pid in serve.pid (safe default)
# Destructive — inspect first (hipfire ps, curl /health, port owner):
hipfire stop --force                  # also pkill -x daemon + fuser -k <port>/tcp
hipfire stop --all                    # --force plus pkill orphan quantize jobs
hipfire restart -d                    # stop --force semantics, then serve again

Destructive lifecycle flags. Plain hipfire stop only SIGTERMs the tracked pid after ownership checks — use that by default. --force, --all, and restart call reapOrphans: system-wide pkill -x daemon (exact name), and fuser -k <port>/tcp which SIGKILLs whichever process owns the port. --all also runs pkill -f against release quantize binaries. Before using them, run hipfire ps, curl -s http://127.0.0.1:<port>/health, and confirm the port owner so you do not kill an unrelated daemon or listener.

Flags (also accepted by restart):

Positional forms: [model] [host] [port], or host:port / [ipv6]:port. Without a model arg, pre-warm uses HIPFIRE_MODEL if set, else cfg.default_model.

Config and env owners for bind, idle, queue, and body limits: CONFIG.md, env-vars.md.

Lifecycle

1. Bind + pid record. Serve writes its numeric pid to ~/.hipfire/serve.pid. stop verifies that /proc/<pid>/cmdline still identifies a native hipfire serve process before signaling it.
2. Daemon. Spawns the Rust daemon example over stdio JSON.
3. Pre-warm (default). Loads the chosen model asynchronously. Failures log and leave the process serving; the model loads on the first real request.
4. HTTP. The native server accepts traffic. Only one generation holds the daemon lock at a time (bounded queue).
5. Idle eviction. When idle_timeout > 0, an interval unloads the model after idle seconds and only when no generation is in flight and the serve lock is free. Next request reloads.
6. Stop. hipfire stop validates pid ownership before SIGTERM. Stale reused pids are not killed; the pidfile is removed instead.

Detached readiness: parent polls GET /health for up to 60 seconds. /health means the native process is answering; inspect model and loading_model to distinguish ready, loading, and unloaded state.

Endpoints

Implemented paths (anything else → 404):

There is no /v1/completions route in the current CLI serve.

For the sealed MQ4R reproduction, default-model handoff, and copyable Hermes Agent / Pi custom-provider configuration, see GOLDEN-REDLINE.md.

GET /health

Always 200 while the HTTP server is up. model is the loaded tag/path or null when idle/unloaded; loading_model names an asynchronous pre-warm.

# Loopback example (safe default for local smoke):
curl -s http://127.0.0.1:11435/health

POST /v1/chat/completions

curl -N http://127.0.0.1:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5:9b",
    "messages": [{"role": "user", "content": "hi"}],
    "stream": true
  }'

• stream: true (typical clients): SSE data: {chat.completion.chunk} lines until data: [DONE].
• stream: false / omitted falsey: single chat.completion JSON body.
• Oversized body → 413 before the daemon lock (Content-Length or streamed cap at max_request_bytes).
• Saturated admission queue → 503 with Retry-After.
• Invalid JSON body → 400.

Request fields honored by the serve layer (non-exhaustive; sampling falls through to per-model / registry / daemon defaults when omitted):

POST /v1/images/generations

OpenAI-shaped image generation on a loaded diffusion checkpoint (arch 40 FLUX.1, arch 45 FLUX.2 Klein). Body fields: model (must already be loaded on this server), prompt, optional width / height (or OpenAI size: "WxH"), steps (defaults to the architecture default: 4 for flux.schnell, 28 for flux.dev), seed, n (must be 1), response_format (must be b64_json). References the same denoise path as the img_generate daemon message (hipfire img).

{"model": "flux.schnell:1", "prompt": "a tiny lighthouse on a rock, sunset", "size": "512x512", "response_format": "b64_json"}

Response: { "data": [ { "b64_json": "…", "width": 512, "height": 512, "seed": 0, "steps": 4 } ], "model": "…", "hipfire": { "ms": 1234 } }.

This body takes no image input. A request with an images field is refused with a 400 that points at /v1/images/edits.

POST /v1/images/edits

OpenAI-shaped reference edit (FLUX.2 Klein, arch 45 only): multipart/form-data with one to four image file parts (PNG or JPEG, at most 32 MB each after upload) plus the text fields of /v1/images/generations (prompt required; size, steps, seed, n, response_format). The image bytes travel in the request; the server never reads a file the client names. Each reference is area-capped at 1 MP and floored to a multiple of 16, and conditions every denoise step without being denoised. Without size the output takes the first reference’s size. The whole body counts against serve.max_request_bytes.

curl -s -X POST http://127.0.0.1:11580/v1/images/edits \
  -F image=.png -F prompt="make the bicycle blue" -F steps=4 -F seed=0 \
  | jq -r .data[0].b64_json | base64 -d > bike-blue.png

Response: the same shape as /v1/images/generations.

Reasoning request contract

Mode, effort, and cap are three independent axes — full key table and budget map in CONFIG.md. HTTP accepts the same meanings as CLI/config.

Normalization (warn + drop, not silent reinterpretation):

• Recognizable unsupported or cross-family values are dropped or aliased with [WARN: INVALID CONFIG]. Warnings appear in the server log and, on OpenAI responses, under hipfire.reasoning / hipfire.config_warnings metadata.
• Malformed types and out-of-range integers remain hard request errors.
• Effort is never converted into a token cap (including on Qwen3.6 and other non-effort-native templates).
• Qwen3.8 and Ornith 1.5 default to an uncapped think span unless the request sets a positive integer cap (semantic effort stays independent of that cap). DeepSeek V4 and Muse Glimmer expose semantic effort but no independent parent-defined cap; cap fields are dropped+warned.

Examples:

# Qwen3.8 — disable thinking natively (empty closed think block)
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen3.8:27b",
  "messages": [{"role": "user", "content": "hi"}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

# Qwen3.8 — semantic effort only (still uncapped unless max_think_tokens is set)
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen3.8:27b",
  "messages": [{"role": "user", "content": "plan a refactor"}],
  "reasoning_effort": "medium"
}'

# Ornith 1.5 — Qwen3.8-compatible semantic effort (default xhigh; still uncapped unless max_think_tokens)
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "ornith-1.5:35b-a3b",
  "messages": [{"role": "user", "content": "plan a refactor"}],
  "reasoning_effort": "low"
}'

# Qwen3.6 — no native effort: effort is dropped+warned; set an explicit cap if wanted
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen3.6:27b",
  "messages": [{"role": "user", "content": "hi"}],
  "reasoning_effort": "low",
  "max_think_tokens": 2048
}'

# DeepSeek V4 — mode + effort; medium/xhigh alias to high; no cap from effort
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "deepseek-v4-flash",
  "messages": [{"role": "user", "content": "hi"}],
  "thinking": {"type": "enabled"},
  "reasoning_effort": "high"
}'

# Gemma4 — boolean thinking (official default off); unsupported effort/cap dropped
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "gemma4:31b",
  "messages": [{"role": "user", "content": "hi"}],
  "chat_template_kwargs": {"enable_thinking": true}
}'

# Muse Glimmer — always reasons via Onyx; strength dial; off is dropped+warned
curl -s http://127.0.0.1:11435/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "muse-glimmer",
  "messages": [{"role": "user", "content": "hi"}],
  "reasoning_effort": "high"
}'

When messages contains no system or developer role, the serve layer inserts prompt.system from a per-model TOML override or the registry card’s recommended_settings.system_prompt. A client-supplied system/developer message always wins.

finish_reason values emitted to clients: stop, length, tool_calls.

Prefix-cache capable arches (daemon cache_capable, or arch allowlist deepseek4 / qwen3_5 / qwen3_5_moe) skip per-request reset so multi-turn LCP can hit. Other arches reset every request (stateless OpenAI shape).

Client notes (Zed / OpenAI-compatible agents)

Live-GPU endpoint validation on gfx1100 and gfx1201 exercised Zed-shaped streamed requests against the canonical Qwen3.6 35B-A3B MQ4R artifact. The matrix covered automatic and required tool calls, multiple calls in one turn, multi-turn reasoning_content replay, tool_choice: "none", usage chunks, and strict UTF-8 SSE/JSON decoding. This validates the OpenAI-compatible endpoint, not the Zed application UI itself.

Inbound assistant tool_calls[].function.arguments strings are parsed into JSON objects for Qwen template replay; outbound OpenAI arguments remain JSON strings. Tool-call deltas are emitted in bulk at the terminal chunk, finish_reason=tool_calls is produced, and disconnect cancels in-flight generation.

Current caveats for agent clients:

• Multi-turn reasoning replay: Qwen3.5/3.6-family arches replay assistant reasoning_content into the next turn (alongside muse_glimmer).
• Tool-argument streaming: progressive per-token argument deltas are not emitted; arguments arrive bulk-at-terminal with the tool-call chunk.
• parallel_tool_calls: the explicit request field is ignored.
• Zed temperature: Zed defaults sampling temperature to 1.0 unless the agent profile overrides it.

For Qwen3.5/3.6 agent use, prefer temperature 0.2–0.6 in the profile, and enable interleaved reasoning on Zed’s OpenAI-compatible capability when the client exposes that toggle.

Auto-routing from hipfire run

hipfire serve -d
hipfire run qwen3.5:9b "..."                 # uses HTTP when /health is up
HIPFIRE_LOCAL=1 hipfire run qwen3.5:9b "..." # force one-shot local daemon

run probes http://<probe-host>:<port>/health (500 ms). Probe host maps 0.0.0.0 / :: → 127.0.0.1. If serve is up, run POSTs /v1/chat/completions and does not spawn a second daemon. If serve is up but the request fails, run exits rather than colliding on the GPU lock. --json / --no-stream also force local control.

Model mismatch: serve reloads to the requested model on the chat path (cold start cost on that first switched request).

Production smoke (GPU)

User-facing serve behavior is a manual GPU route — not proven by container builds or no-GPU CI.

Branch-only: LFM framing harness

Branch-implemented at audited lfm-redline@692a726dde53508cb53de1a74c720e75a7c9f33e; absent from comparison base origin/beta@202282de8759dfa6963ea5184ad2bf2b9259cef6. Not a shipped/ref-pinned HTTP contract on this page.

Example (harness spawns its own serve by default on port 11520):

python3 scripts/serve_harness.py --model ~/.hipfire/models/<file> --tag qwen3.5:9b --mode battery

No-GPU CI (.github/workflows/no-gpu-ci.yml → scripts/no-gpu-ci.sh) never certifies GPU serve, model coherence, or throughput.

Multi-process notes

• One listener per bind; second serve on the same host/port exits.
• Prefer plain hipfire stop when the pidfile is healthy. Reach for hipfire stop --force / restart only after hipfire ps and /health show a stuck orphan or port holder — those paths are destructive (system-wide pkill -x daemon + fuser -k on the port). Avoid ad-hoc pkill unless you know the exact process tree.
• hipfire chat starts a tracked detached native service if none is healthy; stop it explicitly with hipfire stop when the session is done.

Logs and state files

tail -f ~/.hipfire/serve.log

Related

• CLI inventory: CLI.md
• Chat TUI attach behavior: CHAT.md
• Containers: CONTAINER.md
• NixOS service: NIXOS.md