xiaobaifupan/docs/governance/stage-13-llm-gateway.md

Stage 13: Unified LLM Gateway

Boundary

All runtime model calls now enter through backend/llm/gateway.py. Feature agents retain their deterministic context assembly, prompt content, and provider response parsing.

The gateway owns:

• membership and daily quota enforcement;
• primary and fallback model selection;
• fallback only before the first streamed delta;
• stable user-visible availability and interruption errors;
• one logical-call audit record with feature, model role, prompt version, latency, status, normalized error code, and token fields reserved for providers that report usage.

Administrator connection probes also cross the gateway boundary, but do not consume member quota or create usage records.

Compatibility

• Mentor and review-assistant stream payloads are unchanged.
• Heaven readings retain their primary/fallback notice and persistence behavior.
• Strategy compilation still falls back to the deterministic local compiler when both model profiles are unavailable, while access and quota failures remain blocking.
• Provider credentials and raw provider failures remain outside browser responses.

Residual Risk

The current OpenAI-compatible streaming providers do not consistently return token usage, so the audit schema records zero until transport adapters expose trustworthy token counts. Request cancellation remains bounded by the existing provider socket timeout.