Commit Graph
13 Commits
Author SHA1 Message Date
总工 d167ba927c Merge remote-tracking branch 'origin/hel-879-assembly' into hel-902-integration 2026-10-28 00:20:10 +08:00
总管andmultica-agent be149cd6f7 HEL-879 verify endpoint grants and correct final window evidence
Co-authored-by: multica-agent <github@multica.ai>
2026-10-28 00:11:44 +08:00
施工员2号andmultica-agent ce7279e68f HEL-899: custom voice via reference audio on the unified API — reuse .12 panel zero-shot registry (server-side proxy, cg- namespaced IDs never collide with the 11 builtin voices, panel stays the only voices.json writer) + CosyVoice fresh-synthesis preview (unknown voice pre-checked: no silent default fallback) + persisted selection/manifest/usage journal in instance data space with atomic writes and full rollback; session-open captures the effective voice (selection if valid else worker-file value else default, D1 domain untouched when user never chose) pinned per call, mid-call changes land at next call at the earliest; explicit allow_asr gates FunASR transcription, preview/ASR journaled, preview writes no chat/memory/receipt rows; zero new deps (raw-bytes upload endpoint), zero real calls in tests: 18 new cases (formats/size/duration refusals, collision+concurrency, panel down/reject/lying, tts down, rollback, restart persistence, call pinning, identity/shared-state untouched) + offline chain script 12/12, suite 370; req governance v1.12: CG-010 boundary updated to custom-voice-first per user comment 01a17de1 (verbatim evidence + changes + manifest), checks pass vs origin/main and base 8dc84f8 with handoff-HEL-899
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 23:34:00 +08:00
zonggongandmultica-agent f92a86f5d8 HEL-879 unified integration review: actual fc905fe worker frames (generated on the .12 real SDK, read-only SSH stdin) fed through the preserved HEL-898 frontend - fixtures + 9 vitest cases pass (incremental->final->record takeover, phi/no-receipt/interrupt/proactive-during-playout lifecycles, user-interim/hush/drop-reopen boundaries, replay/late/out-of-order frames all dropped); seq namespace analysis (job=room=dispatch=controller call, one entrypoint per job, reconnect keeps the bus, single dispatch site per fresh room - no same-room second-job reset path; residual gap: agent-left zombie UI, for the author); DOCUMENTED GAP: closed-64 squeeze lets a duplicate high-seq turn_start re-enter a retired speech (not producible by the current worker; unbounded-Set fix suggested for the author); browser layer NOT independently re-run (no GUI system libs locally, zero-install policy) - hel879_unified_shots.mjs harness committed for Edge replay, author's 61/61 + theme screenshots remain the interface evidence; regression pytest 352/352, vitest 108/108, typecheck/build/lint(0 err/1 known warning)/ruff clean, requirements checks 4/4 (origin/main and fc905fe, with/without updated handoff-HEL-879); R5 real-test package prepared for the main card (r5-test-package.md) with zero execution; mock &turn/&id params are additive test-infra extensions, default semantics unchanged
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 06:27:27 +08:00
zonggongandmultica-agent fc905fe6bd HEL-879 timing closure: the three contract-approval findings fixed as implementation-timing only (v2 envelope fields/event kinds untouched, web/src untouched, no re-audit of passed binding/backup/budget work) - (1) open_session re-kicks the cleanup owner AFTER registration+persist: a real dispatch await yields the loop and the pre-dispatch kick's cleaner already exited with nothing to watch (manager repro: 20ms real dispatch await, TTL 25ms, 120ms of silence -> DELETE=0, expired registration still present, cleaner=None; fixed: exactly one DELETE, registration settled, no dangling coroutine - the bounded auto-expiry/shared-cap mechanism itself is unchanged), new regressions reproduce first on the old code with a genuinely-yielding dispatch stand-in (asyncio.sleep, not the instantly-returning MockTransport) plus an earlier-deadline wake negative; (2) speak_proactive_row takes the handle the real sync AgentSession.say returns (iscoroutine only unwraps await-style stand-ins - the real SpeechHandle itself defines __await__, so awaiting it would wait out the whole playout), publishes proactive_start IMMEDIATELY, registers the handle's own end/cancel, then waits out that one row (sequential rows kept); .12 real-SDK probe: start visible while handle_done=false, end only after, cancel-only on interrupt, v2 fields identical; (3) _after_playout gates assistant_final on the completed+legal-same-turn-receipt conditions BEFORE anything is sent - incomplete streams, missing/partial receipts and stale epochs close with turn_cancel only (reason incomplete/no-receipt), never final-then-unack; the complete+receipt path keeps BOTH final and ack; phi/interrupt/failure/unattributed fail-closed unchanged; real-SDK check extended to 14/14 ALL PASS on .12 (proactive timing, proactive interrupt, no-receipt no-final negatives all driving the actual worker module's registered callbacks), before/after evidence in docs/rework/hel879/timing-fix/ and sdk-check-hel879-timing.log, reusable probe generator scripts/gen_hel879_timing_probe.py; regression at this head: pytest 352/352 x3 consecutive runs, ruff(src/tests/tools) clean, verify_hel877_voice_chain 8/8, requirements checks 4/4 (origin/main and c121c42, each with/without handoff-HEL-879.json, handoff appended with this round's scope); zero real LLM/embed/ASR/TTS, zero downloads, .12 touched read-only via SSH stdin only, not deployed, nothing merged, PR#11 and hel-870-assembly untouched, real caps stay 0
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 05:31:05 +08:00
zonggongandmultica-agent 3228b1e0d7 HEL-879 remaining closure: auto-expiry cleanup ownership, exact SDK speech-turn attribution, and the versioned v2 subtitle contract for HEL-898
A. Room lifecycle (总管 review items A-1/A-2): the single cleanup owner now watches
   ACTIVE session deadlines as well as debts - a session nobody touches expires into
   the ledger and has its room really deleted with zero further requests (old code:
   _cleanup_loop only scanned debts, expired rooms were never deleted until some
   request ran _prune). Explicit end and the background sweep share ONE persisted
   attempt/age cap and abandoned state - _sessions no longer bypasses it (the
   manager measured 2 background attempts + 3 more interactive deletes = 5); an
   abandoned debt raises CleanupAbandoned -> 503 with room/attempts/reason instead
   of pretending done-404, success clears both _sessions and _debts, restart
   continues the same persisted cap, close/cancel leaves no dangling coroutine, and
   only this instance's own rooms are ever touched. New wake event makes a fresh
   debt retry immediately instead of waiting out a deadline sleep.

B. Queued speech turns bind EXACTLY (总管 interleaving note_handle(H1)->H2->
   note_stream(S1)->S2 gave H1=None, H2='turn-one'): CyberGrillLLM.chat now reads
   the SDK's own agents_speech_handle ContextVar (livekit-agents 1.8.8 sets it in
   _create_speech_task around every speech pipeline; source-verified on .12 and
   asserted at runtime) so each ChatApiLLMStream carries speech_handle/speech_id of
   the very handle whose generation created it. TurnLedger binds only on that exact
   attribution - a newer speech NEVER retires an older pending handle, say/proactive
   generations bind nobody, missing attribution fails closed (no ack, no subtitle
   turn), interrupt/failure discards only its own binding.

C. Versioned subtitle contract v2 for the visual author's wiring card (HEL-898):
   every cg.subtitle payload gains {v:2, seq} (session-monotonic), turn identity
   (speech_id, turn_id, same-source message_id for proactive rows) and explicit
   lifecycle events - turn_start/assistant_interim/assistant_final/turn_end/
   turn_cancel(reason) plus proactive_start/proactive_end/proactive_cancel that
   ride the say SpeechHandle instead of the receipt branch. Accumulators are
   isolated per speech_id (a late delta after cancel is dropped producer-side);
   seq + speech_id give the frontend two independent stale-rejection rules. Full
   executable contract: docs/voice-subtitle-contract-v2.md with recorded samples in
   docs/rework/hel879/subtitle-samples/ (scripts/gen_subtitle_samples_hel879.py).
   web/src untouched.

Verification: the .12 real-SDK check now LOADS THE ACTUAL WORKER MODULE and drives
the callbacks IT registered on a real AgentSession (no hand-copied functions) -
both-queued delayed generations bind exactly, a say sandwiched between user turns
steals nothing, unattributed generations fail closed, interrupted turns cancel
without final/receipt (11/11 ALL PASS, source-level proof of the ContextVar wiring
included; evidence docs/rework/hel879/sdk-check-hel879-remaining.log). The old r2
lifespan test flake is root-caused (clock freeze and block were two separate portal
calls; a 10ms life tick could slip between them and claim the frozen day unblocked)
and fixed atomically without raising timeouts or dropping assertions - 12/12 solo,
full suite 350/350 three times in a row on .venv311; ruff back to the single
pre-existing error; voice chain 8/8; release offline R1-R7 with zero failures;
requirement checks 4/4 (origin/main and c121c42, each with/without handoff).
Zero real LLM/embed/ASR/TTS calls, zero downloads/installs, nothing deployed or
merged, real caps stay 0.

Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 05:10:52 +08:00
zonggongandmultica-agent 7c73d392b9 HEL-879 rework materials: hatchling cache correction with the offline mem0ai wheel actually built from the fixed source (sha256 698284b6..., zero downloads/installs, no 3.10 mixing, recorded in release-artifacts.json), hel895 release doc §0 now separates the historical HEL-895/896 candidate from the current hel-879-assembly candidate and §4/§5 carry the v3 snapshot semantics, budget §6.2.4 amended - a sealed library is never reopened by raising total_requests (contradiction removed), runbook §3.3/3.5/3.6/4.4/4.5 corrected (bounded cleanup ledger replaces the unproven rooms-die-naturally claim; assistant_interim kind; real-but-default-off proactive seam; single-owner dual launcher), one comment-only correction in web/src/voice/session.ts removing the unproven server-auto-cleanup claim (no logic/style change; CSS and JS bundle hashes byte-stable), CG-011 + her incremental subtitles now have a real minimal backend seam (voice/seam.py + worker/llm_chat_api wiring, proactive default off) with the precise frontend interface and acceptance list in docs/rework/hel879/voice-seam-interfaces.md, real-SDK check extended to 10 items (worker callback path on real SpeechHandle/SpeechCreatedEvent; interim seam through real LLMStream) and re-run on .12 ALL PASS, handoff-HEL-879.json appended with the rework scope; regression at this head: pytest 341/341, ruff clean except 1 pre-existing error identical at 7f0a739, web typecheck 0 errors / lint 0 errors 1 pre-existing warning / vitest 88/88 / build passes (CSS index-BpZaybfD.css unchanged), voice chain 8/8, budget split 10/10, release offline R1-R7, http pagination 7/7, real-service browser 9/9 with zero non-loopback outbound, pinned-SDK 16/16+12/12+6/6+7/7+12/12, requirements checks 4/4 (origin/main and c121c42, each with/without handoff)
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 04:27:04 +08:00
zonggongandmultica-agent 7f66015ac8 HEL-879 unified voice candidate assembled and verified on hel-879-assembly (isolated branch; PR#11 and hel-870-assembly untouched) - the merge brings exactly the audited HEL-877 voice backend (aba2aeb: src/cybergril/voice module with zero new dependencies, byte-identical to the author's head) and the HEL-897-verified voice frontend (c61d9c9: web/src byte-identical to the original author's head - the CSS hash change to index-BpZaybfD.css is entirely the author's own components.css +21 lines, no visual rework by this card); the one integration adjustment is test-harness alignment, not a product change: the voice adapter fixtures and verify_hel877_voice_chain.py built their router without a budget config, which the mainline's HEL-882 hard gate (guard.acquire before every real send) now correctly refuses at cap 0 - they now use the same tests/conftest.engineering_budget() every mainline test uses, no gate weakened, no src change on either author's side; full battery rerun on the assembled head: pytest 324/324 (299 mainline + 25 voice), ruff clean, web typecheck 0 errors / lint 0 errors 1 pre-existing warning / vitest 88/88 / build passes, real-server browser 9/9 with the HEL-896 precise-interaction revision (reach card over real HTTP, stage plays the reach-out, unread clears, message-id dedup, restart epoch+1, zero permission API calls, zero non-loopback outbound - evidence docs/rework/hel879/browser/), verify_hel877_voice_chain 8/8 offline full-chain, the .12 livekit-agents 1.8.8 real-SDK stdin check regenerated from THIS head's sources 8/8 ALL PASS (zero writes zero network), check_release_offline R1-R7 passes, pinned-SDK regressions on hel857-venv all green (close-ownership 16/16, shutdown-queued 12/12, inflight 6/6, semantic-recovery 7/7, mem0-real 12/12 - the seal change is exercised through the same PersistentBudget paths), requirement checks 4/4 (origin/main and c121c42, each with and without handoff-HEL-879.json); release-artifacts.json records the exact release code commit (01ff919), the tarball sha256 measured equal to the lock, and the cache-reusable wheels reverified with content digests after the disk-recovery cache incident; honest gaps kept open: real-call acceptance untested and unauthorized (D1 voice, trusted HTTPS device path, budgets), LiveKit room/worker release after API restart still lacks real-environment evidence (_prune only drops the in-memory registry, end_session does delete_room), incremental subtitles remain a contract gap not a completed feature, CG-011 proactive voice stays an explicit no-op; zero real LLM/embed/ASR/TTS, zero downloads, no mic, no service started or stopped on any shared machine, not deployed, nothing merged, authoritative ledgers/freezes/HEL-835 gate untouched, real caps stay 0
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 03:36:37 +08:00
施工员2号 aba2aebeb0 HEL-877: wire the audited .12 voice base into the one CyberGril chat chain as backend-only code (no deploy, no real voice call) - new independent src/cybergril/voice module: session+router add POST /api/voice/session / DELETE end / GET honest status on the unified instance (server-side room + dedicated cybergril-voice-agent dispatch + short-TTL join token via zero-new-dependency livekit_wire HS256/Twirp matching the installed SDK's wire format verified claim-for-claim on the .12 agent venv stdin-only; keys never reach the browser; unconfigured/no-profile/TTS-down refuse honestly and capabilities.voice stays false), chat_api_client speaks the exact real.ts RealChatAdapter protocol (per-turn health epoch, bounded 409 resync, robust incremental SSE parsing across multibyte/CRLF/multi-data chunk splits, fixed turn_id idempotency, immediate HTTP-stream close on barge-in without waiting for the next ASR final), llm_chat_api is a livekit llm.LLM adapter whose only backend is POST /api/chat so persona/mood/dyad/memory/life-sharing/budget gates and the one same-turn chat_lock govern text and voice identically (phi spoken verbatim via ChatChunk, never archived, no receipt), receipt.py confirms a sharing receipt only after real SDK playback completion with every negative (interrupted/failed/mid-playout/phi/incomplete/stale-epoch) fail-closed and exactly once per turn, stt_funasr/tts_cosyvoice reuse the proven .12 adapters verbatim with provenance, worker.py assembles the dedicated worker (independent voice_state_cybergril.json so D1 stays undecided, no second persona, cg.subtitle data-channel contract for the visual lead, CG-011 proactive seam left as an explicit no-op hook, built-in health port 8082); app.py keeps a minimal registration with voice_service injection for isolated tests; deployment drafts only (config/voice.example.yaml, worker.env.example, restart_cybergril.sh.example with exact-pattern process matching, serve_dual_8360_8361.py keeping HTTP 8360 and the HTTPS voice entry on ONE process sharing state/locks, runbook-hel877-voice.md contract+startup+rollback incl. the >=6GB GPU precondition and the do-not-copy historical fuser warning) - verification: 281 pytest pass on 3.11 venv + requirements.lock after merging the chief engineer's mid-flight HEL-874 rework e978e87 in full (parallel work preserved), verify_hel877_voice_chain.py 8/8 offline full-chain incl. socket-level barge-in fail-close and text/voice mutual same-gate cancellation, real-SDK check on the .12 livekit-agents 1.8.8 venv 8/8 ALL PASS via SSH stdin with zero writes zero network, dual-port single-process smoke on loopback 18360/18361 with a throwaway self-signed cert then stopped, ruff clean on all new code, dual-target requirement checks pass vs origin/hel-870-assembly (current tip e978e87) and origin/main with handoff-HEL-877.json; zero real LLM/embed/TTS/ASR requests, no mic/camera, no downloads/installs, no service start/stop, no 4080 access, no third-account or profile writes, web/ untouched, not merged, not deployed
Co-authored-by: multica-agent <github@multica.ai>

## HEL-877 交付摘要(R5 后台施工,未部署未合并)

- 新独立模块 src/cybergril/voice/:专属 livekit worker(agent_name=cybergril-voice-agent,复用 .12 FunASR/CosyVoice/silero 底座,不 fork 不另搭平台,无第二人格/记忆,不直连 LLM);llm_chat_api 适配器与文字前端 real.ts 同协议走 POST /api/chat(epoch 409 有界重同步、SSE 分片、turn_id 幂等、打断即断流);/api/voice/session 建房+专属派发+短时限 token 全服务端(密钥不进浏览器;缺配置/无档案/TTS 停机诚实拒绝;capabilities.voice 不伪造);receipt 仅在 SDK 真实播放完成后确认(打断/失败/φ 轮/旧 epoch 全 fail-closed)。
- 部署草案:/opt/voice-agent-cybergril 目录、worker 健康端口 8082、HTTP 8360 + HTTPS 8361 同进程双入口(deploy/voice/serve_dual_8360_8361.py,共用同一状态/锁)、启动顺序含 TTS GPU≥6GB 前置与回滚;本卡未执行任何启停。契约(开关/字幕/失败/打断/挂断 + CG-011 主动开口接缝)见 docs/runbook-hel877-voice.md §3。
- 自测(零真实请求):281 pytest 全过(含语音 25 例:socket 级打断断流、SSE 边界、重放零调用、文字/语音同门互打断、回执全负例、token 房间隔离);scripts/verify_hel877_voice_chain.py 8/8;.12 agent venv(livekit-agents 1.8.8)真 SDK 检查 8/8(stdin-only 零写入零网络);双端口同进程本地冒烟通过;ruff 干净;需求检查双目标通过(含 handoff-HEL-877.json)。
- 施工中总工 HEL-874(e978e87)推上同分支,本卡已完整并入,不覆盖并行成果。未验证如实列明:端到端语音(TTS 停机/GPU 3.9GB<5.6GB)、真 LiveKit server 的 dispatch/delete、移动端实机——待接线/审核卡真机验收,未经真实验收不称 R5 完成。
2026-10-26 22:42:19 +08:00
施工员2号andmultica-agent 23415fbcd0 HEL-876: voice-base read-only verification + R5/R6 readiness materials - docs/voice-r5-readiness-HEL876.md records the 2026-10-26 bounded read-only audit of 192.168.200.12 with timestamps: LiveKit/FunASR/8090 alive with health evidence, 8081 corrected as the voice-agent livekit worker (not a legacy lip-sync piece), CosyVoice TTS 9880 gracefully stopped since 2026-10-24 22:17 and NOT running, MuseTalk 8010 unit gone, ComfyUI 8188 holding 26.6GB of the 32GB GPU leaving 3.9GB free < ~5.6GB TTS requirement (the only hard resource gap, coordination not unilateral action), voice catalog (11 voices, rain selected) with whitelisted config field names only and no credentials read or printed, .11 gateway model-list probe and open-meteo outbound both 200; the R5 seam is fixed at src/cybergril/api/app.py:488 - the worker must reuse the same chat chain via a livekit LLM adapter speaking the identical RealChatAdapter protocol (epoch sync/serialized turns/barge-in cancel/idempotent replay/receipt round-trip), proven offline with zero model calls by scripts/verify_hel876_voice_seam.py (six checks all pass on 3.11 venv + requirements.lock installed from the local uv offline cache, ruff clean, explicitly NOT claimed as end-to-end voice verification); R6 seams located read-only: city placeholder ManagePanel.tsx:182 with no backend, no weather client (outbound now proven), demo-only material import :178/:183, full-backup boundary = companion.db + vec_public + mem0_public_history.db + embed ledger where copying the single db file is not a complete backup and Persona.fingerprint is the same-her check; three-column honest conclusions (verified / unverified / actually-missing) plus six zero-cost work items ready for dispatch; no product code touched (src/ web/ unchanged), no new product PR, candidate branch not merged, dual-target requirement checks 4/4 pass with handoff-HEL-876.json
Co-authored-by: multica-agent <github@multica.ai>
2026-10-26 21:42:41 +08:00
白栖知andmultica-agent c285a0fd07 fix(r2): keep her self-narration and life within the approved room
Co-authored-by: multica-agent <github@multica.ai>
2026-10-24 12:03:08 +08:00
白栖知andmultica-agent 4575b27515 HEL-827: implement durable life, receipt-gated disclosure and persona layers
Co-authored-by: multica-agent <github@multica.ai>
2026-10-24 09:04:40 +08:00
白栖知andmultica-agent 003d8c2cae HEL-822: land HEL-821 A1+A2 source onto PR#4 base (db571ba)
- 13 modified / 8 added / 1 deleted per CHANGES.json (author package 76bc120b)
- docs/requirements v1.10 + prototypes/hel-799 + PROVENANCE from bundle, verified byte-identical to 63799f0
- AGENTS.md + scripts/check_requirements.py restored from governance commit 63799f0

Co-authored-by: multica-agent <github@multica.ai>
2026-10-24 06:10:53 +08:00