refactor: unify LLM gateway policy

This commit is contained in:
leefer
2026-07-29 20:08:39 +08:00
parent 5f0b9735bd
commit a368174a75
10 changed files with 587 additions and 224 deletions
+7 -7
View File
@@ -262,16 +262,16 @@
"bytes": 364385,
"lines": 15549
},
{
"path": "server.py",
"bytes": 268447,
"lines": 5956
},
{
"path": "static/redesign-v2.css",
"bytes": 264960,
"lines": 8616
},
{
"path": "server.py",
"bytes": 263744,
"lines": 5856
},
{
"path": "static/index.html",
"bytes": 133569,
@@ -279,8 +279,8 @@
},
{
"path": "database.py",
"bytes": 121153,
"lines": 2829
"bytes": 121546,
"lines": 2839
},
{
"path": "screener.py",
+32
View File
@@ -0,0 +1,32 @@
# Stage 13: Unified LLM Gateway
## Boundary
All runtime model calls now enter through `backend/llm/gateway.py`. Feature agents retain
their deterministic context assembly, prompt content, and provider response parsing.
The gateway owns:
- membership and daily quota enforcement;
- primary and fallback model selection;
- fallback only before the first streamed delta;
- stable user-visible availability and interruption errors;
- one logical-call audit record with feature, model role, prompt version, latency, status,
normalized error code, and token fields reserved for providers that report usage.
Administrator connection probes also cross the gateway boundary, but do not consume member
quota or create usage records.
## Compatibility
- Mentor and review-assistant stream payloads are unchanged.
- Heaven readings retain their primary/fallback notice and persistence behavior.
- Strategy compilation still falls back to the deterministic local compiler when both model
profiles are unavailable, while access and quota failures remain blocking.
- Provider credentials and raw provider failures remain outside browser responses.
## Residual Risk
The current OpenAI-compatible streaming providers do not consistently return token usage, so
the audit schema records zero until transport adapters expose trustworthy token counts. Request
cancellation remains bounded by the existing provider socket timeout.