refactor: unify LLM gateway policy
This commit is contained in:
@@ -262,16 +262,16 @@
|
||||
"bytes": 364385,
|
||||
"lines": 15549
|
||||
},
|
||||
{
|
||||
"path": "server.py",
|
||||
"bytes": 268447,
|
||||
"lines": 5956
|
||||
},
|
||||
{
|
||||
"path": "static/redesign-v2.css",
|
||||
"bytes": 264960,
|
||||
"lines": 8616
|
||||
},
|
||||
{
|
||||
"path": "server.py",
|
||||
"bytes": 263744,
|
||||
"lines": 5856
|
||||
},
|
||||
{
|
||||
"path": "static/index.html",
|
||||
"bytes": 133569,
|
||||
@@ -279,8 +279,8 @@
|
||||
},
|
||||
{
|
||||
"path": "database.py",
|
||||
"bytes": 121153,
|
||||
"lines": 2829
|
||||
"bytes": 121546,
|
||||
"lines": 2839
|
||||
},
|
||||
{
|
||||
"path": "screener.py",
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# Stage 13: Unified LLM Gateway
|
||||
|
||||
## Boundary
|
||||
|
||||
All runtime model calls now enter through `backend/llm/gateway.py`. Feature agents retain
|
||||
their deterministic context assembly, prompt content, and provider response parsing.
|
||||
|
||||
The gateway owns:
|
||||
|
||||
- membership and daily quota enforcement;
|
||||
- primary and fallback model selection;
|
||||
- fallback only before the first streamed delta;
|
||||
- stable user-visible availability and interruption errors;
|
||||
- one logical-call audit record with feature, model role, prompt version, latency, status,
|
||||
normalized error code, and token fields reserved for providers that report usage.
|
||||
|
||||
Administrator connection probes also cross the gateway boundary, but do not consume member
|
||||
quota or create usage records.
|
||||
|
||||
## Compatibility
|
||||
|
||||
- Mentor and review-assistant stream payloads are unchanged.
|
||||
- Heaven readings retain their primary/fallback notice and persistence behavior.
|
||||
- Strategy compilation still falls back to the deterministic local compiler when both model
|
||||
profiles are unavailable, while access and quota failures remain blocking.
|
||||
- Provider credentials and raw provider failures remain outside browser responses.
|
||||
|
||||
## Residual Risk
|
||||
|
||||
The current OpenAI-compatible streaming providers do not consistently return token usage, so
|
||||
the audit schema records zero until transport adapters expose trustworthy token counts. Request
|
||||
cancellation remains bounded by the existing provider socket timeout.
|
||||
Reference in New Issue
Block a user