blazar
Documentation

Compatibility Contract

The behaviors Blazar guarantees not to break.

8. DO NOT BREAK

Invariants current and future changes must preserve. Verified against Blazar v0.6.1 source. Commands and configuration spot-verified against v0.19.0 (2026-10-03).

Critical assumptions

  • Local-only trust model. No telemetry, no cloud endpoints, no accounts (login/signin verbs refuse by design). Anything that phones home breaks the product promise.
  • Auth iff keys. Gateway is open iff zero [[keys]] exist; /healthz + /health must stay auth-free. Management requires an unscoped key; the last key can never be removed; an authless gateway must keep refusing management.
  • Zero-fork engines. Official upstream binaries, sha256-verified, side-by-side versioned tags with update/use/rollback. Never patch or vendor engine sources. (Exception by design: capability lanes — user-installed third-party forks pinned to one immutable SHA, clearly provenance-stamped, auto-retired when mainstream catches up. They must stay pinned, additive, and never preferred over a mainstream build that serves the same architecture.)
  • Engine deletion is aside-first. The retirement sweep and engine rm share remove_engine_row_and_tree: directory renamed aside → row deleted → aside discarded, restore on row-delete failure. A live row must never point at a deleted dir, and engine rm must refuse while any live child serves the engine (or when the daemon is down with orphan pidfiles) — no zombie servers on unlinked trees.
  • Children are stock. All dialect differences are absorbed by gateway translation layers; per-child auth secrets live in run/<model>.apikey and die with the child.
  • Weights are plain files. No blob store, no copies. Modelfile-style overlays are config overlays. Hardlink aliases must stay zero-byte-cost.
  • LaneError string parity. Routing/teaching error Display strings are byte-identical across CLI and API bodies — the same strings flow into both. Rewording = breaking change.
  • # reservation. models.name never contains # (replica keys model#N); @vision and #N grammars must keep matching in evict_model.
  • Store invariants. set_active_engine must never leave zero active engines (even on error); key_usage writes are absolute whole-day snapshots in one txn; pre-v4 rows default engines.kind = 'llamacpp'.
  • Quantized-safetensors detection (QUANTIZED_SAFETENSORS_TOKENS = [awq, gptq, fp8], word-boundary, non-GGUF only) must NOT be replaced by the quant column (awq→4BIT, fp8-dynamic→BF16 mislabels).
  • Session banks are ctx-scoped (_auto-<ctx>): never restore across ctx shapes; banks >512 MiB dropped; save bounded at 2 s, idle instances only.
  • Exit code 3 on bind conflict (EXIT_BIND_CONFLICT) pairs with systemd RestartPreventExitStatus=3 — changing either breaks no-restart-loop behavior.
  • child_transport = "tcp" is byte-identical default. The UDS lane is opt-in; under tcp the child argv, dial URLs, auth minting, and proxy bytes are exactly the pre-UDS shapes. Any transport-conditional must branch on the endpoint, never change the tcp path’s behavior.
  • Anthropic malformed image blocks 400. translate_message rejects an image block whose source is neither base64 data nor a URL with invalid_request_error (fail-fast beats the old text-marker placeholder); well-formed images translate to real image_url parts and reach the engine.
  • Engine rows persist data-dir-relative server_path. Storage form is engines/<tag>/... (the out-of-tree local lane keeps its absolute user path); every decode anchors to an absolute live path via Manifest::anchor_server_path before spawns/probes/sweeps see it. A new row-write site that persists an absolute in-tree path must call relativize_server_path first — the boot walk heals stragglers, but the invariant holds at birth.
  • serve owns run/daemon.log. The daemon tees tracing to that file under every launcher (systemd, CLI wrapper, foreground); journald is a secondary stream. Rotation is 10 MiB → daemon.log.1 at daemon start.

Performance-sensitive areas

  • Gateway hot path (proxy_core_chat, ollama.rs): audit drops are counted never blocking; OTLP batches bounded; singleflight coalescing; WFQ/EDF admission — do not add unbounded allocation or per-token syscalls here.
  • Streaming translation: boundary-safe UTF-8 line buffering (LineBuffer); tool-call delta merge — regressions corrupt tool-calling clients.
  • hint_task 60 s metrics scrape with EWMA 0.7/0.3 feeding cache/spec hints — feeds scheduling decisions; a stalled scrape loop degrades routing.
  • PREFILL_HEAVY_BYTES = 64 KiB demotion threshold and SLO tiers (2/30/120 s) are tuned; changing them reshapes fairness contract.
  • KV/VRAM math (unified_ctx_verdict, estimate_kv_vram_charge, scale_kv_by_cache_type, 2a-bis floor (weights+64)×20/17, 700 MiB overhead): wrong math = either refused loads or OOM crashes. The num_loops × block_count multiplication for loop architectures must survive (nanbeige-class: num_loops=2).
  • -ub (ubatch) 512 default: raising it was a measured regression — leave alone unless re-measured.
  • cache_ram_mb 30% RAM auto-cap: an unclamped budget was live-measured to swap-kill a machine; keep the cap.
  • spec default ‘auto’ is target-side verified-lossless; do not flip blindly.

Known technical debt

  • Cosmetic misclassification: SIGKILL-after-grace on a sleeping llama-server logs “engine crashed (signal 9)” ERROR — cosmetic only, breaker-safe, documented.
  • SGLang JIT cache ignores SGLANG_CACHE_DIR (upstream behavior: caches to ~/.cache/sglang); vanilla-AWQ builds env-blocked on nvcc 12.0.
  • mistral.rs v0.9.3 --max-num-batched-tokens 4096 default cap: prompt >4096 produced empty 200s upstream; gateway now surfaces 502 and argv raises the step to min(serving ctx, 4096).
  • Engine update pruning keeps KEEP_TAGS=2 per lane and deletes older tags outright (delete, not archive).

Legacy or backward-compatible behavior

  • ollama migrant model:tag colon→dash resolution onto existing rows only (never creates mappings).
  • Pre-v4 SQLite DBs migrate via PRAGMA user_version; engines.kind backfilled 'llamacpp'.
  • api_keys legacy config → blazar migrate → [[keys]] (idempotent, with backup).
  • mmproj_policy accepts bool compat (true=attach, false=skip) alongside lazy/attach/skip spellings; explicit -mm in extra_args always wins.
  • ollama API compatibility itself (tags/ps/pull/chat/generate/show/delete + OLLAMA_HOST) is a compat surface: existing ollama clients must keep working unchanged.
  • OpenAI/Anthropic wire compatibility (chat, completions, embeddings, responses, messages, batches, files) — field-level regressions break real clients.

Logic that looks wrong but is intentional

  • MLX quant dirs never route to SGLang — they are quantized-safetensors-shaped and only mlx-lm decodes them; the mlx axis is checked before the quantized arm (MlxUnserved teaching beats silent misrouting).
  • Quantized safetensors never fall back to mistral.rs even when SGLang is missing — mistral.rs dequant is broken for AWQ/GPTQ/FP8 (live-proven); a teaching error beats garbage output.
  • The diffusion domain gate is bidirectional — a model row with a component set (--vae+--llm/--t5xxl…) routes to sdcpp ONLY, and no text row ever routes to sdcpp; text/embedding surfaces answer 400 with the images-endpoint teaching instead of booting sd-server into a guaranteed 404, and /v1/images refuses text models with the chat-endpoint teaching.
  • sd-server’s ready signal is /v1/models, not /health — sd.cpp answers 404 on /health (live-probed, master-890-74988b2); the SdCpp health check polls /v1/models and requires a data array. It also binds only AFTER model load, so connection-refused during poll is normal loading.
  • Underscore flags must survive the help-probe charset — sd.cpp’s --llm_vision/--clip_l/--clip_g/--clip_vision/--qwen2vl_vision were once silently dropped from the manifest (152 vs 157 flags), which the strict probed-flags argv gate then turned into silently degraded /v1/images/edits.
  • Memoryless device census rows never reach the GPU pool — sd.cpp’s --list-devices prints no memory numbers; a zero-total pool must fall back to nvidia-smi, and the host CPU row is not an accelerator (it once handed --device CPU to a CUDA llama-server).
  • deterministic=true forces slots=1` — multi-slot serving measurably perturbs near-tie logits (greedy probes flip); this is physics, not preference.
  • Concurrency lease held through SSE drain — a streamed response occupying a max_concurrent slot is deliberate backpressure, not a leak.
  • resolve_model_name returns input verbatim on miss — silent invention of mappings was rejected by design.
  • context_shift upstream default is DISABLED (llama.cpp b10833 change): --context-shift is emitted only when explicitly enabled.
  • --sampler-seq deliberately not a knob — samplers config is the semicolon chain (top_k;top_p;typical) instead.
  • Sentinel never alters bytes — even with sentinel_enforce, only non-streaming chat gets the 422 path; observation-only everywhere else.
  • Semantic cache never crosses API keys and bypasses structured outputs in both directions — correctness over hit-rate.
  • think:false injection for marker-carrying templates (local lane only, before translation) and the fail-open-on-unreadable-marker teaching 400 are deliberate asymmetries.

Security findings (hardcoded secrets in source)

None found — no hardcoded credentials, tokens, or key material discovered in workspace source during exploration (verified absent). Runtime secrets (child API keys, plm_ gateway keys) are generated from OS entropy at runtime and persisted only to run/<model>.apikey files or shown once at creation.

Improve this page on GitHub