zonggongandmultica-agent 152479b350 HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 01:43:54 +08:00
HEL-889 rework: shutdown never closes under a live worker, and a queued-cancelled worker is released by the authoritative future state — SemanticMemory now runs batch workers on its OWN dedicated ThreadPoolExecutor and keeps each work item's concurrent.futures.Future: cancelling the awaiting tick propagates to that future, and its post-cancel state is the deterministic queued/started oracle (the manager's review_hel886_edges.py reproduced the old false-liveness on the shared default executor: register _live, await to_thread, executor busy -> cancel tick cancels the QUEUED item, _run and its finally never run, _live=1 forever with the batch pending and recovery permanently skipping a worker that does not exist, restart the only way out; now a CANCELLED future proves nothing ever ran: _release_never_started removes the registry entry idempotently and returns the batch to queued with its attempt restored and zero sends, while a running/finished future keeps HEL-886 semantics — the worker thread owns the batch fate, single send, no reclaim, no resend), and aclose becomes a real ownership protocol instead of a bounded promise it could break: a closing flag makes tick refuse new claims and makes both backends' _add_group_sync check should_stop BEFORE each row's wire work (a _ShutdownAbort at a boundary keeps finished rows' sem_row_done proofs and requeues only the unsent remainder — never a mid-row cut, and a single-row batch still completes done); never-started workers are cancelled out of the executor queue under the same authoritative CANCELLED proof; started workers finish their current row and flush, and aclose WAITS for every live worker to actually exit before backend/executor close — drain_timeout only bounds the polite phase (after expiry: honest warning, keep waiting; a live worker is never treated as dead), bounded in practice because Mem0Backend now sets the gateway OpenAI client's timeout (gateway_timeout config, default 60s) instead of the 600s default that made the drain effectively open-ended; a cancelled aclose closes NOTHING (shielded finish, closing signal reset, layer stays usable), concurrent aclose calls serialize on an asyncio.Lock and are idempotent, and the app lifespan comment now states the ownership contract — the manager's fixed-SDK reproduction of the OLD timeout branch (loopback gateway accepts the first HTTP then withholds the answer, aclose(drain_timeout=0.05) returns with the worker alive, app closes the store, late worker dies on 'QdrantLocal instance is closed'/'Cannot operate on a closed database', reopen shows pending/row_done=0/llm_used=1) is closed: aclose does not return while the worker is alive, the worker's writes land BEFORE the app-order store.close(), and the reopened store shows done + row_done=1 + full entry_ids with llm_used=1 and zero late errors; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); verification: 294 python tests pass on 3.11.16 venv + requirements.lock offline (7 new tests/test_rework_hel889.py cases, all barrier-driven real product paths: queued-cancel release + in-process recovery via injected single-worker executor / shutdown-timeout never closes under the live worker and flushes before app-order store close / between-rows abort requeues and the next process sends ONLY the missing row / shutdown-before-start zero sends then resumes / cancelled aclose leaves everything open / concurrent aclose serializes / closed layer refuses new claims), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_shutdown_queued_hel889.py 12/12 (Q1 queued-cancel phantom released and recovered in-process with the manager's 41s lease probe, S1 fixed-SDK shutdown-timeout branch, S2 between-rows abort + missing-row-only resume proven by gateway POST counts, S3 cancelled aclose, S4 concurrent aclose), the manager's review_hel886_edges.py re-run VERBATIM on the pristine f1b50ad first reproduced both defects exactly (docs/rework/hel889/edges-verbatim-before-fix-f1b50ad.json matches the manager's independent JSON), the same-shape rerun review_hel886_edges_rerun_hel889.py now passes both scenarios with every original result key (queued: done/attempts=1/registered 0/entries 1/row_done 1; timeout: aclose_returned_while_worker_alive=false, late_worker_error null, persisted done+entry_ids+row_done=1, llm_used=1) while the verbatim original stops at its own premise assert executor._work_queue.qsize()==1 — batch workers no longer ride the loop default executor, the encoded defect is structurally gone (original negative and baseline kept); regressions all green: verify_inflight_lifecycle_hel886.py 6/6 (single-send preserved), verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass, review_hel885_inflight.py still fails at its 2-add barrier exactly as documented in HEL-886; runbook §4 gains the HEL-889 ownership/queued-cancel semantics, config/semantic.example.yaml documents gateway_timeout, docs/requirements/handoff-HEL-889.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 01:24:11 +08:00
HEL-885 rework: per-row progress matches what was really sent, cross-day and unknown-outcome accounting stay conservative, positive caps require the ledger authority, and the release dependency-fill command actually exists - every Memory.add now carries an anchor for exactly its own row (the HEL-882 whole-batch anchor let the first stored vector mark unsent rows complete; the manager's real-SDK partial reproduction showed the user's original words never leaving the process), a row finishing with a legitimately EMPTY extraction gets a durable sem_row_done record (r4.sql additive table) written inside the worker thread the moment its add completes, resume skips = vector-store anchors ∪ sem_row_done so no row is lost, none falsely complete, and finished work is never re-sent across budget-refusal/crash/restart/lease/concurrency, entry_ids merge across partial attempts, and the swallowed-failure detection is per-row; PersistentBudget.settle/release book on hold.day (the old write to state.day manufactured -80 used on the new day and re-opened headroom the reservation day had already spent; token_usage/request_usage refund the paying day, budget_total stays global) and the empty-budget_total TypeError crash in total_requests_used/total_tokens_used is fixed - the old one-shot-total test passed only because that crash surfaced as chat.error, it now books the total honestly; _atomic_reserve no longer purges aged reserve: rows (a crashed worker's hold is an UNKNOWN outcome - an hour-old timeout is not proof the call never happened; unknown consumption stays billed across restarts, only a provably-unsent release() ever removes a hold, embed_ops and llm_ops share the fix); Mem0Backend refuses construction before importing mem0 when any cap>0 lacks a (non-empty dict) authority block - ledger_authority_required - or when the prefix check fails, closing the fresh-empty-ledger bypass the manager reproduced live, config/semantic.example.yaml ships the authority block as REQUIRED with the explicit cumulative formula (embed_cap=500+approved-increment e.g. 680, embed_reserve=2 frozen untouched, min_used=498; never write 180 as the cumulative cap of a 498-used account) mirrored into docs/budget-plan-hel874.md §2 with the HEL-885 accounting rules; deploy/hel874-release.md §4 replaces the nonexistent uv pip download (rc=2, re-verified) with python3 -m pip download --only-binary :all: --platform manylinux2014_x86_64 --python-version 3.11 --implementation cp --abi cp311 + sha256 manifest + uv pip install --no-index --find-links offline merge + same-process co-import verdict, honest missing-materials list unchanged (no 3.11 wheels cached, zero downloads this card, no version swaps), and tools/check_release_offline_hel882.py gains R7 probing both command surfaces offline; verification: 281 python tests pass on 3.11.16 venv + requirements.lock offline (9 new tests/test_rework_hel885.py cases: partial batch claims only the sent row / empty-extraction row completes and is never resent / process restart sends only the missing row / concurrent ticks never double-send / cross-day settle and release book on the reservation day / aged unknown llm+embed reservations never refunded / missing-empty-malformed-wrong authority all fail closed; test_resume_never_loses_or_duplicates_rows rewritten to drive the REAL product path instead of hand-planting single-row anchors the product never wrote), ruff clean (src/tests/tools), web tsc + 42 vitest + build via npm ci --offline, pinned mem0 2.2.1 install on the deterministic boundary: contract re-run 12/12 (harness now hands a synthetic authority prefix - production defaults not relaxed), new tools/verify_semantic_recovery_hel885.py 7/7 (S0 authority fail-closed, S1 per-row partial with exactly 1 HTTP, S2 restart sends only the missing row proven by HTTP count since mem0 prompts legitimately carry old rows in rolling history, S3 lease recovery + concurrent ticks single-send, S4a/S4b unknown reservations kept), both manager review scripts re-run VERBATIM against this head (partial: completed=[2] only, resume done with 2 entries, HTTP=2, user words actually sent; extra: new-day used=0, over-cap request refused, aged reservation second_allowed=false, uv probe still rc=2), release offline check R1-R7 green, dual-target requirement checks 4/4 with handoff-HEL-885.json; zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 00:01:44 +08:00
HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 01:43:54 +08:00
HEL-869: read-only remediation evidence per manager independent review of 81d50f2 - recheck869.py reproduces the upstream rolling 10-message window from the fixed mem0 storage source (SHA256 5f5ecdad, byte-identical to the isolated venv install) in an independent in-memory SQLite with zero LLM zero embedding zero network (single 15-message batch truncates to 10; the real 9,1,1,2,2 add pattern keeps batch1 first-4 + later-6, dropping batch1 last-5) and verifies read-only that this round's mem0 messages hold 10/15 missing exactly t3 user/companion, t4 user/companion, t5 user with the restored public copy identical at 10, while the baseline main archive chat_message keeps all 15 verbatim per-role rows and the original calls inputs still carry all 15 - SDK messages is a rolling window, not a permanent quote archive, and no real user data was lost; erratum_HEL-869.json records six report-corrections without touching any original JSON (E1 S1 frozen criterion sampled only mem0 t1 so full-15 retention cannot be claimed, E2 this round used 46004 tokens = 44735+1269 while 163041 is the cumulative 39-call total, E3 raw QA6-13 map in order to plan QA1-8, E4 calls headline 'answered blue' is a stale run2 note vs this round's honest unknown answer, E5 in-process llm_calls_real=0/embed 39 are not authoritative vs persistent ledgers' 13 sent/40 dense, E6 S9 re-verified only the public instance with full private+public recovery reused from gate2 P5); README HEL-868 record annotated in place at the three corrected conclusions plus a HEL-869 section; plan-next-HEL-869.md delivers the decision-only minimal follow-up (main archive stays the permanent quote layer with reread via turn_ids anchors, mem0 interfaces reused, no upstream fork; batch-extraction line-level provenance proven by input-side pre-grouping where each add's anchor equals its actual input set with gate-side sha256 recompute - no gate bypass, no anchor trimming, no prompt-only promises; S6 verified via explicit-status questioning under the same frozen bounded grammar with no wording amnesty; Chinese custom_instructions deferred; upstream/ours responsibilities, risks incl. embed budget 498/500 remaining 2 as the hard blocker, and a staged minimal retest: stage0 offline zero-cost audit then stage1 ~14 requests/~40-46 embeddings pending quota decision); all three ledgers byte-identical before/after (03f92557/c9871277/9716c722), every e06dcaf result blob unchanged, no criteria/fixtures/orchestration edits, no real requests, no embeddings, no merge, no deploy; dual-target requirement checks 4/4 pass vs origin/main and origin/agent/agent/de8874a3803b with/without handoff-HEL-869.json
2026-10-26 04:09:58 +08:00
HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 01:43:54 +08:00
HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 01:43:54 +08:00
HEL-894 rework: a cancelled close owns nothing it did not start — aclose cancellation is now a two-phase ownership protocol - the manager's independent pinned-SDK review of ceb0f35 reproduced two combo holes in the HEL-889 close path: (1) cancelling aclose AFTER backend.aclose began made the shielded finish keep running unowned in the background while the outer except already released the lock and cleared closing — the layer was reported reusable (closing=false/closed=false/client usable), a new tick claimed a batch onto real HTTP, then the orphaned finish closed Qdrant under it (SDK-only branch: batch failed on 'QdrantLocal instance is closed' with row_done=0/llm_used=1; with the caller closing the store afterwards the late worker died on ProgrammingError 'Cannot operate on a closed database' and the reopened batch sat pending), and (2) cancelling a SECOND aclose that was still WAITING for the close lock ran the same except and cleared the FIRST closer's flag (first close still pending but signal=false, a new batch ran to done while the original worker was still on the wire). SemanticMemory.aclose now states the revocation boundary explicitly: the DRAIN phase (nothing closed yet) stays revocable — a cancelled caller leaves everything open, resets the closing signal and the layer remains usable (HEL-889 semantics preserved); the FINISH phase (backend.aclose begun) is irreversible — a cancelled caller keeps the closing markers (a half-closed layer is never reported usable, ticks keep refusing), keeps the lock, and waits — repeated cancels only re-shield — for the layer-owned close task it started (self._close_task, created exactly once with asyncio.create_task and stored on the layer, never dropped to the background) to land before propagating CancelledError, so resources close exactly once; _finish_close also joins this layer's OWN executor via asyncio.to_thread so a cancelled caller can never block the event loop on a synchronous shutdown wait (caller-injected executors stay caller-owned), and signal reset is ownership-gated by owns_close: only the call that actually acquired the lock and set the signal may clear it, and only while no close task exists — a lock waiter cancelled on acquisition touches nothing; module docstring, aclose docstring and the app lifespan comment state the same contract; verification: 299 python tests pass on 3.11.16 venv + requirements.lock offline (5 new tests/test_rework_hel894.py cases, all barrier-driven real product paths: finish-phase cancel keeps ownership and refuses new work with zero sends and zero budget / repeated cancels during finish land the close exactly once / a same-phase second aclose waits then returns idempotently while a third cancelled lock-waiter touches nothing / waiter-cancel + owner draining + refused new tick + original worker completing combo / drain-phase cancel still re-arms for reuse), ruff clean (src/tests/tools), no frontend changes (frontend checks not re-run, stated honestly); pinned mem0 2.2.1 on the deterministic boundary (512-dim stand-in before TextEmbedding construction, loopback-only): new tools/verify_close_ownership_hel894.py 16/16 reruns the manager's two negatives in the same shape with assertions restated for the new protocol (F1 SDK-only: after the cancel closing=true/closed=false/client usable and the cancelled caller does not return before its close lands; the new tick is refused with zero gateway arrivals, batch queued/owner NULL/attempts 0 and llm_used=0; after the gate the close lands exactly once with live_workers=0 and the reopened batch is intact queued/row_done=0/llm_used=0 — before the fix the same run gave closing=false, a claimed batch, QdrantLocal instance is closed, failed/pending with row_done=0/llm_used=1; F2 + store-close branch: same shape and no late error — before: ProgrammingError; W waiter cancel: closing stays set, the new batch stays queued, the original worker completes done/row_done=1 after release and the owner's close then lands once); the manager's review_hel889_cancel.py re-run VERBATIM on the pristine ceb0f35 first reproduced both defects exactly (docs/rework/hel894/before-ceb0f35.json matches the manager's independent JSON), and on the fixed tree stops honestly at its own premise assert server.arrived — 'a finish-phase-cancelled layer still starts new real HTTP' is structurally gone because tick refuses while closing is held (verbatim-after-fix.log; original negative and baseline kept); regressions all green: verify_shutdown_queued_hel889.py 12/12 (queued cancel, timeout drain, between-rows resume, drain-cancel re-arm, concurrent close all preserved), verify_inflight_lifecycle_hel886.py 6/6, verify_semantic_recovery_hel885.py 7/7, verify_mem0_real_hel874.py 12/12, check_release_offline_hel882.py R1-R7 15/15, both manager review_hel882_{partial,extra}.py scripts re-run verbatim pass (extra's uv rc=2 is its own expected negative), HEL-889 same-shape rerun review_hel886_edges_rerun_hel889.py passes; crash semantics unchanged and still honest (unknown wire outcomes billed, proof-gated retries, strict-once not claimed); runbook §4 gains the HEL-894 cancellation-ownership paragraph, docs/requirements/handoff-HEL-894.json written, dual-target requirement checks 4/4 (origin/main and origin/agent/agent/de8874a3803b, with and without handoff); zero real LLM, zero real embeddings, zero non-loopback outbound, authoritative ledgers and freezes untouched, not deployed, PR not merged, no visual changes
2026-10-27 01:43:54 +08:00
HEL-870 assembly: memory/persona/mood/life/proactive-contact joined into one runnable candidate on an additive R4 migration (no existing table or row replaced; no visual changes beyond mechanical proactive flag passthrough) - src/cybergril/memory/semantic.py productizes the audited Mem0 path per plan-next-HEL-869: input-side pre-grouping sends only public rows outbound (user rows always; companion rows only judged+acked with all-told candidates), every batch anchor turn_ids/turn_roles/input_sha256 equals that exact input set, and the read-time gate independently recomputes the fingerprint against the live archive and re-checks every companion row's share state, fail-closed rejecting no_payload/unknown_source/archive_missing/source_mismatch/hit_text_mismatch/anchor_turn_mismatch/role_text_conflict/companion_unshared with honest row_verbatim/row_anchor/batch provenance; main archive chat_message remains the ONLY permanent quote layer; sem_batch/sem_covered carry a lease-recovered idempotent job loop; EmbedLedgerGuard continues the single authoritative embed ledger and with embed_cap=0 (no approved budget) refuses every real embedding so the frozen 498/500 and 39/60 accounts and the HEL-835 gate stay untouched; fixed Mem0 2c81af3 is lazily imported with zero upstream fork, a deterministic stub backend exists for engineering verification and is never claimed semantic; MoodRuntime makes core.mood live (event impulses scaled by bazi strength sensitivity, heartbeat decay, state_snapshot persistence, restart recovery, model sees one word only); DyadRuntime makes the relationship ledger live (completed exchanges are rituals, daily regression, behavior hints stay instructions); life_activity adds start/continue/interrupt/resume/finish continuity across slots and restarts with chat-completion interrupts; reach_out implements the approved R4 scope from character.yaml (>=6h interval, hours 8-23, salience>=0.6, arousal>=0.5, intimacy>=0.4) with traceable already-public bases only, UNIQUE(basis) so restarts and duplicate tasks can never multiply messages, night suppression, absence-rule fail-closed, phi projection, and messages persisted as companion rows (proactive=1) fully synced on reopen; API health reports R4-assembly with per-capability honesty (voice/vision/weather/materials/backup stay false, semantic_memory names its backend); config/semantic.example.yaml sanitized config and docs/runbook-hel870.md (startup, locked deps, health, independent data dir, disable/rollback, experience chain list, budget request, R1-R7 seams); verification: 250 python tests pass on 3.11 venv + requirements.lock (61 new deterministic/fake-provider cases covering every gate negative, restart recovery, dedupe, red line and the full entry chain incl. reopen epoch sync), ruff clean, web typecheck + 42 vitest + build pass with the original author frontend untouched except the one-line proactive passthrough; dual-target requirement checks 4/4 pass vs origin/main and origin/agent/agent/de8874a3803b with handoff-HEL-870.json; zero real model calls, zero embeddings, zero outbound network this card; local unified-instance smoke on port 8360 (stub semantic) serves the built web and reports the life loop waiting; read-only server check recorded in the runbook (existing 8300-8350 services and voice ports observed, end-to-end voice availability NOT verified); not merged, not deployed, no production or shared services touched
2026-10-26 21:03:42 +08:00
2026-10-19 20:08:52 +08:00

CyberGril

一个有自己世界的赛博伴侣。

不是聊天机器人。区别在于:它在你不说话的时候也在运行。


先看一眼

晚晚 · 介绍动画

▶ 打开完整动画 — 22 秒,纯代码生成,无任何外部资源

docs/intro.html          单文件动画,双击即播
docs/preview.webp        渲染好的预览(800×450 / 10fps / 22s / 0.5MB)
tools/render_preview.py  重新渲染、换分辨率、导出静帧

动画里有一条贯穿的主线:强调色 = 当前效价 v。 情绪低落时画面整体转冷,而不是靠字幕告诉你"她现在很难过"。

幕 时间 讲什么
膜 0–6s 物质撞在膜上弹回,信息穿过去
心跳 6–12s 没有输入时它也在跳;难过恢复得慢
记忆 12–17s 记得的会沉下去,但沉不到底
名字 17–22s —

四张静帧:docs/01-membrane.png 02-heartbeat.png 03-memory.png 04-title.png

动画本身的技术取舍:时间轴是纯函数 render(t),不做 RAF 累加。 所以任意一帧都能独立渲染——这也是 render_preview.py 能逐帧截图、 而不需要真实录屏的原因。


它是什么

一个单用户、本地优先、常驻的存在,通过一层"膜"与你通信:

  • 有自己的时钟 —— 无输入时仍在演化、仍有生活
  • 情绪是动力学系统 —— 慢变量、有惯性、有代价(会心情不好)
  • 记忆有形状 —— 三层结构,会遗忘,且遗忘有下限
  • 关系是一等实体 —— 信任、债务、未修复的事
  • 本体不可谈判 —— 立场不因任何对话、情绪、施压而改变

三层不可混淆的东西

层 载体 变化速度 谁能改
命 bazi_core.derived 几乎不变 只有整合器,每次 ±0.02,且留痕
运 fortune_period 按周/月 排盘
状态 State (v/a/d/e) 按秒/分 tick 循环

混掉任何两层,角色就会失真:

  • 命被对话改 → 人设漂移,变成一个讨好你的东西
  • 状态被写成描述 → 模型自己编情绪,动力学失效

架构

contracts/   数据契约 —— 全项目唯一的共同语言
providers/   模型接入 —— 按能力建模
core/        状态核心 —— 独立时钟,不依赖用户输入
memory/      三层记忆 + 遗忘
dyad/        关系账本
think/       模型路由 + 预算 —— 四档分配,按关键时刻
config/      models.yaml(版本号) + character.yaml(世界法则)

设计判断(这一节比代码重要)

1. OpenAI 兼容是 adapter,不是标准

对 LLM 和 Embedding,OpenAI 的形状(/v1/chat/completions、/v1/embeddings) 已经是事实标准,内部契约就直接用它。

但对 ASR / TTS / 形象,OpenAI 的形状会砍掉这个项目最需要的能力:

能力 OpenAI 形状的问题 我们的做法
ASR /v1/audio/transcriptions 是「文件进→文本出」,没有 partial;而对话必须要 partial 才能做端点检测 内部契约是事件流(partial → final);OpenAI 适配器作为降级档,supports_partial=False
TTS /v1/audio/speech 只有 text/voice/speed,没有副语言标签;而 [laughter] [breath] 是"像活的"的核心 内部契约带 tags / emotion;不支持的 provider 在 plan() 里显式降级并上报
形象 没有事实标准 自定义 RenderCmd 协议,驱动源必须是 State,不是 LLM 文本

关键机制:静默降级是不允许的。

planned = tts.plan(request)
if planned.lost_paralinguistics:
    # 日志里会出现 [degraded] speech_tags: ...
    ...

静默降级会让你在调优时永远找不到"它为什么不像人了"。

2. 状态必须以数值注入,不能用散文描述

✗ "你现在有点难过"          → 模型会自己发挥,甚至编出更多情绪
✓ v=-0.62 a=0.41 d=0.30    → 模型知道边界
  + visible="低落"
  + constraint="不得声称与此不符的内在状态"

StateBlock 里那个 constraint 字段不是装饰,它是唯一能阻止 模型绕过状态机的机制(外加输出校验与重写)。

3. 遗忘有下限

mem_episodic.accessibility 的下限是 0.03,不是 0。

  • 正常检索拿不到 → 想不起来
  • 情绪线索命中时能捞出来 → 但还在

所有现成框架都是"要么记住要么删除",没有这个中间态。

4. 未修复的事变成行为,不变成台词

✗ "你上次伤害了我"          → 指责,而且廉价
✓ 主动性 -0.1,回复变短      → 它不说,但它不一样了

5. 缺席可以被提及,不能被使用

✗ "我一直在等你,等了好久"   → 制造负罪感,这是操控
✓ "这几天我这边发生了点事"   → 它有自己的世界,不是空转等你

这条写在 core/projection.py 的注释里,也有对应的测试。

6. 大模型按「关键时刻」分配,不按时间分配

先纠正一个常见误解:60 秒心跳不该调用任何 LLM。

心跳做的是 s = b + (s-b)*exp(-dt/τ) + I,四个 float 一次 exp, 微秒级。给它配模型是架构错误,不是资源问题。

真正的 token 消耗在两处:每轮对话的输入上下文,以及后台任务的频次。

所以用四档:

档 用什么 场景 成本
none 不调用 心跳、主动性判定、填充词、表情驱动 ≈0
low 本地小模型 生活事件、记忆压缩、事实抽取、输出初筛 电费
standard 中大模型 + 短上下文 投机响应的第二段 低
high 旗舰 主对话台词、主动联系、危机 按 token

路由判定必须便宜 —— 查表,不调模型。为了省 token 先调一次模型,是本末倒置。

关键约束:

  • 主对话的下限是 low,不是 none。 预算耗尽时可以变笨,不能没台词。
  • 危机场景永不降级。 这是产品约束,不是性能优化。
  • 降级要可预测。 单次请求放不下只降一档;到硬阈值才降到下限。

省钱的真正杠杆,按收益排序:

  1. Prompt 缓存 —— 本体/世界法则固定放前缀,命中后便宜一个数量级。杠杆最大。
  2. 上游压缩 —— 小模型先把 20 条记忆压成 3 条,大模型只收结果。输入从 4000 降到 800 tokens。
  3. 批处理 —— 记忆整合攒 30 分钟一次批量处理,不是 20 次单条调用。
  4. 换小模型 —— 收益反而最小,而且伤害人设。

还有一个漂亮的副作用:成本降级可以被叙事化。 大模型不可用时它"有点走神"—— 这既符合人设,又是真的(实现同构的虚构)。


快速开始

# 1. 安装
pip install -e ".[dev]"

# 2. 配置
cp .env.example .env
# 编辑 .env 填入 API key(不填也能跑,会回退到 echo provider)

# 3. 跑测试
pytest -q

# 4. 检查配置能被正确加载与校验
python -c "from cybergril import load_settings; print(load_settings('config').character.name)"

默认配置不需要任何 API key 就能跑通测试 —— 因为内置了 echo(LLM)/ scripted(ASR)/ silence(TTS)三个离线 provider。 它们的价值是把接口形状固定下来,让上层可以先跑通再换真后端。


换成真模型

只改 config/models.yaml,不动任何代码。

tiers:
  high:
    provider: openai_compat
    model: deepseek-v4.1-flash
    base_url: https://api.deepseek.com/v1
    api_key_env: DEEPSEEK_API_KEY
  low:
    provider: openai_compat
    model: qwen3.5-4b-instruct
    base_url: http://127.0.0.1:8000/v1

tts:
  provider: cosyvoice        # 需要自己实现这个 adapter,见下
  model: fun-cosyvoice3-0.5b

新增一个 provider 只需要三步:

# 1. 继承对应的基类
class MyTTS(TTSProvider):
    name = "mytts"

    # 2. 诚实地声明能力
    @property
    def capabilities(self) -> Capabilities:
        return Capabilities(
            modality=Modality.AUDIO,
            streaming=Streaming.CHUNKED,
            speech_tags=frozenset({"laughter", "breath"}),
        )

    # 3. 实现 stream()
    async def stream(self, request): ...

# 注册
registry.register("tts", "mytts", MyTTS)

能力必须诚实。 夸大能力会让上层做出错误的规划, 而 plan() 会据此剥掉你其实需要的东西。


配置分层

文件 放什么 什么时候改
config/models.yaml 唯一允许写模型版本号的地方 换模型(每季度复核)
config/character.yaml 世界法则、情绪参数、关系参数 换人格(几乎不动)

两个文件互不污染。这是"防止文档和代码一起过时"的机制。

关于版本号的一个约定:只有 models.yaml 允许写版本号。 代码里所有模型引用走配置,不硬编码。


测试

pytest -q                    # 全部
pytest tests/test_mood.py -v # 情绪动力学
pytest -k degradation -v     # 降级机制

测试守的是设计判断,不是实现细节:

文件 守什么
test_contracts.py 接口形状
test_providers.py 降级必须显式上报
test_mood.py 情绪有惯性;负向恢复更慢
test_bazi.py 命是数值;强弱影响敏感度
test_projection.py 未覆盖事件必须计数;投影不泄漏技术词
test_store.py 遗忘有下限;间隔效应

下一步

地基只包含:契约、provider 抽象、配置、情绪动力学、八字编译、投影、存储 schema、关系账本。

还没有的(按依赖顺序):

  1. core/tick.py —— 四个循环(tick / consolidate / life / reach_out)
  2. memory/retrieval.py —— mood-congruent 检索
  3. memory/consolidation.py —— 整合与身份提升
  4. think/ —— 上下文组装、生成、三道校验
  5. express/ —— TTS 管道、形象、barge-in
  6. api/ —— WebSocket 会话

规格

完整设计见 docs/SPEC.md。

许可

私有项目,未授权。

S1 前端演示

独立的 React / TypeScript 前端位于 web/,运行 cd web && npm ci && npm run dev 后从普通入口进入明确标识的本机演示。真实后台、语音、视觉和备份尚未接通;不自动连接模型或现有服务。安装、构建和测试命令见 web/README.md,实现与后续接线见 docs/frontend/IMPLEMENTATION.md。

S
Description
No description provided
Readme
198 MiB
Languages
Python 57.2%
TypeScript 22.7%
CSS 19.9%
JavaScript 0.1%
HTML 0.1%