Files
总工andmultica-agent
ecec9df8e8
HEL-906: continuous-call ASR usage metering + one-shot window guard in dedicated worker seam
- asr_usage.py (stdlib-only): append-only JSONL ledger, per-call/window audio seconds counted from actually-sent frames (sample-rate aware, resend counted), connect attempt/failure/retry semantics (normal per-utterance reconnects are not retries), fail-closed default (real cap=0 unless a window is armed), pre-send blocking so no byte crosses the cap, ledger replay for restart recovery and cross-window isolation, resumable summarize CLI - stt_funasr.py: guard hooks on the real send path (check_connect before open_utt, precheck before send, count after send, recv-side death recorded, per-utterance flush aggregation); usage=None keeps prior behavior - worker.py: entrypoint builds guard from env per job, room name as call id, on_exhausted -> ctx.shutdown ends the call, window summary in start log - tests: 17 deterministic offline tests incl. fake livekit/websockets SDK driving the real _run seam, sample-rate variants, silence/cancel/failure/ reconnect/repeated-end, cap rejection before crossing, ledger persistence across restart, room deletion keeps history, other windows unaffected Co-authored-by: multica-agent <github@multica.ai>
HEL-874 rework: bounded verbatim readback reaches the real provider input, real pinned Mem0 2.2.1 SDK contract verified live and the adapter's five wiring defects fixed, budget gates made provable (atomic reserve/settle embed ledger + independent llm_ops cap), definite-cap real-check execution table, 8360 isolated release package prepared (not deployed), 2026-10-26 user originals recorded source-only - semantic.context now quotes chain-verified archive rows (original words + role + when, <=200-char window with visible trim marker, summary demoted to a one-line pointer, same min(600,budget) cap) and episodic.context flips from gist-first-120-chars to verbatim-first (<=240) + role + date, both proven by tests/test_readback_hel874.py capturing the actual messages handed to the provider through the real API path: a detail absent from the gist yet quoted verbatim from a turn pushed outside the recent-20 window, the same for a semantic chain hit, and an unshared companion row never leaking through FTS, semantic or any anchor (AM-001); tools/verify_mem0_real_hel874.py runs the FIXED install itself (mem0ai 2.2.1 2c81af3, real embedded qdrant, real fastembed from the existing cache, deterministic 127.0.0.1 fake gateway, temp ledgers only): before=31970c4 dies at construction (second QdrantClient on the same path rejected by the SDK - recorded), and source-level findings confirmed live and fixed - unscoped search raises ValueError in 2.2.1 (add now carries user_id scope, search passes filters+top_k since limit is silently ignored), payload reads ride the SDK's own vector_store.client, the old outer search debit double-counted (wrapper is now the single accounting point), the SDK's internal extraction LLM was ungated (LlmLedgerGuard with its own llm_ops table, cap default 0) and check-then-debit could overshoot under concurrency (EmbedLedgerGuard now reserves atomically in BEGIN IMMEDIATE, releases on failure/cancel, purges stale holds); after fixes all 12 checks pass (construction/anchor payload/user scope/1 LLM + 2-3 embeds per add measured/extraction input set provably only prior gated batches incl. old memories and rolling history/unshared canary zero outbound/search single debit/concurrency never over cap/close-reopen intact/both zero caps refuse with ZERO outbound LLM requests), evidence in docs/rework/hel874/; mood sensitivity returns to the original repo's continuous rule 1.35-0.70*strength (bazi.sensitivity_from_strength, foundation commit
1012d8b) replacing HEL-870's invented 1.25/0.8 step and is labeled an engineering heuristic, never user-approved bazi science (external agent formulas remain not adopted); docs/budget-plan-hel874.md gives definite caps for the one-time team check (55 requests/0 embeds), first-week semantic activation (<=60 extract LLM / <=180 embeds) and user daily use with code-level hard-gate facts and authorization sources (old 20-30/100-200 were estimates, not approved quota; frozen 39/60 and 498/500 untouched and never挪用); deploy/hel874-release.md + cybergril-hel874-8360.service.example prepare the isolated fresh-empty-data-dir release with health capability matrix and data-preserving rollback (NOT executed, PR#11 not merged, no user-facing loop, no experience address); sources/comment-evidence-full-experience-20261026.json adds the two 2026-10-26 user originals (01a17806, 01a1780e full-experience-then-iterate) with authorship verified and manifest appended only - baseline/current/coverage/changes untouched; verification: 256 python tests pass on 3.11.16 venv + requirements.lock offline (3 new real-provider-input readback cases incl. AM-001 negative, mem0 fake rebuilt to mirror the real 2.2.1 surfaces, 9 cases), ruff clean, web tsc + 42 vitest + build pass, local 8360 stub smoke (loopback, isolated temp DB) health R4-assembly all green then stopped, real-SDK script re-run 12/12 after final edits, dual-target requirement checks 4/4 pass with handoff-HEL-874.json; zero real model calls, zero real embeddings, zero outbound (fake gateway on loopback + existing local model cache only), authoritative ledgers and freezes untouched, no existing services or profiles touched
HEL-889 rework: shutdown never closes under a live worker, and a queued-cancelled worker is released by the authoritative future state — SemanticMemory now runs batch workers on its OWN dedicated ThreadPoolExecutor and keeps each work item's concurrent.futures.Future: cancelling the awaiting tick propagates to that future, and its post-cancel state is the deterministic queued/started oracle (the manager's review_hel886_edges.py reproduced the old false-liveness on the shared default executor: register _live, await to_thread, executor busy -> cancel tick cancels the QUEUED item, _run and its finally never run, _live=1 forever with the batch pending and recovery permanently skipping a worker that does not exist, restart the only way out; now a CANCELLED future proves nothing ever ran: _release_never_started removes the registry entry idempotently and returns the batch to queued with its attempt restored and zero sends, while a running/finished future keeps HEL-886 semantics — the worker thread owns the batch fate, single send, no reclaim, no resend), and aclose becomes a real ownership protocol instead of a bounded promise it could break: a closing flag makes tick refuse new claims and makes both backends' _add_group_sync check should_stop BEFORE each row's wire work (a _ShutdownAbort at a boundary keeps finished rows' sem_row_done proofs and requeues only the unsent remainder — never a mid-row cut, and a single-row batch still completes done); never-started workers are cancelled out of the executor queue under the same authoritative CANCELLED proof; started workers finish their current row and flush, and aclose WAITS for every live worker to actually exit before backend/executor close — drain_timeout only bounds the polite phase (after expiry: honest warning, keep waiting; a live worker is never treated as dead), bounded in practice because Mem0Backend now sets the gateway OpenAI client's timeout (gateway_timeout config, default 60s) instead of the 600s default that made the drain effectively open-ended; a cancelled aclose closes NOTHING (shielded finish, closing signal reset, layer stays usable), concurrent aclose calls serialize on an asyncio.Lock and are idempotent, and the app lifespan comment now states the ownership contract — the manager's fixed-SDK reproduction of the OLD timeout branch (loopback gateway accepts the first HTTP then withholds the answer, aclose(drain_timeout=0.05) returns with the worker alive, app closes the store, late worker dies on 'QdrantLocal instance is closed'/'Cannot operate on a closed database', reopen shows pending/row_done=0/llm_used=1) is closed: aclose does not return while the worker is alive, the worker's writes land BEFORE the app-order store.close(), and the reopened store shows done + row_done=1 + full entry_ids with llm_used=1 and zero late errors; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); verification: 294 python tests pass on 3.11.16 venv + requirements.lock offline (7 new tests/test_rework_hel889.py cases, all barrier-driven real product paths: queued-cancel release + in-process recovery via injected single-worker executor / shutdown-timeout never closes under the live worker and flushes before app-order store close / between-rows abort requeues and the next process sends ONLY the missing row / shutdown-before-start zero sends then resumes / cancelled aclose leaves everything open / concurrent aclose serializes / closed layer refuses new claims), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_shutdown_queued_hel889.py 12/12 (Q1 queued-cancel phantom released and recovered in-process with the manager's 41s lease probe, S1 fixed-SDK shutdown-timeout branch, S2 between-rows abort + missing-row-only resume proven by gateway POST counts, S3 cancelled aclose, S4 concurrent aclose), the manager's review_hel886_edges.py re-run VERBATIM on the pristine
f1b50ad first reproduced both defects exactly (docs/rework/hel889/edges-verbatim-before-fix-f1b50ad.json matches the manager's independent JSON), the same-shape rerun review_hel886_edges_rerun_hel889.py now passes both scenarios with every original result key (queued: done/attempts=1/registered 0/entries 1/row_done 1; timeout: aclose_returned_while_worker_alive=false, late_worker_error null, persisted done+entry_ids+row_done=1, llm_used=1) while the verbatim original stops at its own premise assert executor._work_queue.qsize()==1 — batch workers no longer ride the loop default executor, the encoded defect is structurally gone (original negative and baseline kept); regressions all green: verify_inflight_lifecycle_hel886.py 6/6 (single-send preserved), verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass, review_hel885_inflight.py still fails at its 2-add barrier exactly as documented in HEL-886; runbook §4 gains the HEL-889 ownership/queued-cancel semantics, config/semantic.example.yaml documents gateway_timeout, docs/requirements/handoff-HEL-889.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of
ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
HEL-899: custom voice via reference audio on the unified API — reuse .12 panel zero-shot registry (server-side proxy, cg- namespaced IDs never collide with the 11 builtin voices, panel stays the only voices.json writer) + CosyVoice fresh-synthesis preview (unknown voice pre-checked: no silent default fallback) + persisted selection/manifest/usage journal in instance data space with atomic writes and full rollback; session-open captures the effective voice (selection if valid else worker-file value else default, D1 domain untouched when user never chose) pinned per call, mid-call changes land at next call at the earliest; explicit allow_asr gates FunASR transcription, preview/ASR journaled, preview writes no chat/memory/receipt rows; zero new deps (raw-bytes upload endpoint), zero real calls in tests: 18 new cases (formats/size/duration refusals, collision+concurrency, panel down/reject/lying, tts down, rollback, restart persistence, call pinning, identity/shared-state untouched) + offline chain script 12/12, suite 370; req governance v1.12: CG-010 boundary updated to custom-voice-first per user comment 01a17de1 (verbatim evidence + changes + manifest), checks pass vs origin/main and base
8dc84f8 with handoff-HEL-899