施工员2号 and multica-agent
4a38106eef
HEL-911: fix AgentHandoff/shutdown seams, attribute & fix the 3.66s pre-generation wait
...
- worker _on_item: filter conversation_item_added by the real 1.8.8 item
discriminator (ChatMessage | AgentHandoff | _TypeDiscriminator); only
assistant str-part body feeds the echo filter (HEL-910 AttributeError)
- bind_interrupt_channel unload: async (JobContext shutdown callbacks are
awaited), idempotent, disconnect-safe; interrupt/receipt/budget paths
untouched (HEL-910 TypeError await None)
- TURN_HANDLING: explicit turn_detection=None — 1.8.8 silently installs an
English EOU TurnDetector whose unlikely verdicts on Chinese finals push
endpointing to max_delay=6.0 (measured 3.667/3.654s HEL-910 call2; offline
A/B on the real SDK: baseline 3.704s vs fixed 0.004s). Endpointing
0.8/6.0 kept verbatim; no VAD/buffer_turn/semantic change
- scripts/gen_hel911_wait_repro.py: offline reproduction harness (real
AgentSession + real silero VAD + HEL-910 wav and measured timings)
- tests: +3 real-SDK checks (11/11 on .12 agent venv; prior 8 regressions
kept, unload call sites await the new protocol)
- hel910_call1.mjs reliability: segmented probe/recorder dumps to disk,
timing-A trigger requires the server-committed voice turn row (not a
we-sent-it proxy); node --check only, no real run
- window-record.md §9: appended factual corrections (companion 9 not 18,
turn_start +6.099 not +4.10, assistant_final is the playout/receipt gate,
post-window ListRooms token failure is not empty-room evidence); original
record untouched
- docs/rework/hel911/summary.md + handoff-HEL-911.json; requirement checks
pass vs origin/main and 407fc81github@multica.ai >
2026-10-29 21:25:25 +08:00
总工
d167ba927c
Merge remote-tracking branch 'origin/hel-879-assembly' into hel-902-integration
2026-10-28 00:20:10 +08:00
总管 and multica-agent
be149cd6f7
HEL-879 verify endpoint grants and correct final window evidence
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-28 00:11:44 +08:00
施工员2号 and multica-agent
ce7279e68f
HEL-899: custom voice via reference audio on the unified API — reuse .12 panel zero-shot registry (server-side proxy, cg- namespaced IDs never collide with the 11 builtin voices, panel stays the only voices.json writer) + CosyVoice fresh-synthesis preview (unknown voice pre-checked: no silent default fallback) + persisted selection/manifest/usage journal in instance data space with atomic writes and full rollback; session-open captures the effective voice (selection if valid else worker-file value else default, D1 domain untouched when user never chose) pinned per call, mid-call changes land at next call at the earliest; explicit allow_asr gates FunASR transcription, preview/ASR journaled, preview writes no chat/memory/receipt rows; zero new deps (raw-bytes upload endpoint), zero real calls in tests: 18 new cases (formats/size/duration refusals, collision+concurrency, panel down/reject/lying, tts down, rollback, restart persistence, call pinning, identity/shared-state untouched) + offline chain script 12/12, suite 370; req governance v1.12: CG-010 boundary updated to custom-voice-first per user comment 01a17de1 (verbatim evidence + changes + manifest), checks pass vs origin/main and base 8dc84f8 with handoff-HEL-899
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-27 23:34:00 +08:00
zonggong and multica-agent
f92a86f5d8
HEL-879 unified integration review: actual fc905fe worker frames (generated on the .12 real SDK, read-only SSH stdin) fed through the preserved HEL-898 frontend - fixtures + 9 vitest cases pass (incremental->final->record takeover, phi/no-receipt/interrupt/proactive-during-playout lifecycles, user-interim/hush/drop-reopen boundaries, replay/late/out-of-order frames all dropped); seq namespace analysis (job=room=dispatch=controller call, one entrypoint per job, reconnect keeps the bus, single dispatch site per fresh room - no same-room second-job reset path; residual gap: agent-left zombie UI, for the author); DOCUMENTED GAP: closed-64 squeeze lets a duplicate high-seq turn_start re-enter a retired speech (not producible by the current worker; unbounded-Set fix suggested for the author); browser layer NOT independently re-run (no GUI system libs locally, zero-install policy) - hel879_unified_shots.mjs harness committed for Edge replay, author's 61/61 + theme screenshots remain the interface evidence; regression pytest 352/352, vitest 108/108, typecheck/build/lint(0 err/1 known warning)/ruff clean, requirements checks 4/4 (origin/main and fc905fe, with/without updated handoff-HEL-879); R5 real-test package prepared for the main card (r5-test-package.md) with zero execution; mock &turn/&id params are additive test-infra extensions, default semantics unchanged
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-27 06:27:27 +08:00
zonggong and multica-agent
fc905fe6bd
HEL-879 timing closure: the three contract-approval findings fixed as implementation-timing only (v2 envelope fields/event kinds untouched, web/src untouched, no re-audit of passed binding/backup/budget work) - (1) open_session re-kicks the cleanup owner AFTER registration+persist: a real dispatch await yields the loop and the pre-dispatch kick's cleaner already exited with nothing to watch (manager repro: 20ms real dispatch await, TTL 25ms, 120ms of silence -> DELETE=0, expired registration still present, cleaner=None; fixed: exactly one DELETE, registration settled, no dangling coroutine - the bounded auto-expiry/shared-cap mechanism itself is unchanged), new regressions reproduce first on the old code with a genuinely-yielding dispatch stand-in (asyncio.sleep, not the instantly-returning MockTransport) plus an earlier-deadline wake negative; (2) speak_proactive_row takes the handle the real sync AgentSession.say returns (iscoroutine only unwraps await-style stand-ins - the real SpeechHandle itself defines __await__, so awaiting it would wait out the whole playout), publishes proactive_start IMMEDIATELY, registers the handle's own end/cancel, then waits out that one row (sequential rows kept); .12 real-SDK probe: start visible while handle_done=false, end only after, cancel-only on interrupt, v2 fields identical; (3) _after_playout gates assistant_final on the completed+legal-same-turn-receipt conditions BEFORE anything is sent - incomplete streams, missing/partial receipts and stale epochs close with turn_cancel only (reason incomplete/no-receipt), never final-then-unack; the complete+receipt path keeps BOTH final and ack; phi/interrupt/failure/unattributed fail-closed unchanged; real-SDK check extended to 14/14 ALL PASS on .12 (proactive timing, proactive interrupt, no-receipt no-final negatives all driving the actual worker module's registered callbacks), before/after evidence in docs/rework/hel879/timing-fix/ and sdk-check-hel879-timing.log, reusable probe generator scripts/gen_hel879_timing_probe.py; regression at this head: pytest 352/352 x3 consecutive runs, ruff(src/tests/tools) clean, verify_hel877_voice_chain 8/8, requirements checks 4/4 (origin/main and c121c42, each with/without handoff-HEL-879.json, handoff appended with this round's scope); zero real LLM/embed/ASR/TTS, zero downloads, .12 touched read-only via SSH stdin only, not deployed, nothing merged, PR#11 and hel-870-assembly untouched, real caps stay 0
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-27 05:31:05 +08:00
zonggong and multica-agent
3228b1e0d7
HEL-879 remaining closure: auto-expiry cleanup ownership, exact SDK speech-turn attribution, and the versioned v2 subtitle contract for HEL-898
...
A. Room lifecycle (总管 review items A-1/A-2): the single cleanup owner now watches
ACTIVE session deadlines as well as debts - a session nobody touches expires into
the ledger and has its room really deleted with zero further requests (old code:
_cleanup_loop only scanned debts, expired rooms were never deleted until some
request ran _prune). Explicit end and the background sweep share ONE persisted
attempt/age cap and abandoned state - _sessions no longer bypasses it (the
manager measured 2 background attempts + 3 more interactive deletes = 5); an
abandoned debt raises CleanupAbandoned -> 503 with room/attempts/reason instead
of pretending done-404, success clears both _sessions and _debts, restart
continues the same persisted cap, close/cancel leaves no dangling coroutine, and
only this instance's own rooms are ever touched. New wake event makes a fresh
debt retry immediately instead of waiting out a deadline sleep.
B. Queued speech turns bind EXACTLY (总管 interleaving note_handle(H1)->H2->
note_stream(S1)->S2 gave H1=None, H2='turn-one'): CyberGrillLLM.chat now reads
the SDK's own agents_speech_handle ContextVar (livekit-agents 1.8.8 sets it in
_create_speech_task around every speech pipeline; source-verified on .12 and
asserted at runtime) so each ChatApiLLMStream carries speech_handle/speech_id of
the very handle whose generation created it. TurnLedger binds only on that exact
attribution - a newer speech NEVER retires an older pending handle, say/proactive
generations bind nobody, missing attribution fails closed (no ack, no subtitle
turn), interrupt/failure discards only its own binding.
C. Versioned subtitle contract v2 for the visual author's wiring card (HEL-898):
every cg.subtitle payload gains {v:2, seq} (session-monotonic), turn identity
(speech_id, turn_id, same-source message_id for proactive rows) and explicit
lifecycle events - turn_start/assistant_interim/assistant_final/turn_end/
turn_cancel(reason) plus proactive_start/proactive_end/proactive_cancel that
ride the say SpeechHandle instead of the receipt branch. Accumulators are
isolated per speech_id (a late delta after cancel is dropped producer-side);
seq + speech_id give the frontend two independent stale-rejection rules. Full
executable contract: docs/voice-subtitle-contract-v2.md with recorded samples in
docs/rework/hel879/subtitle-samples/ (scripts/gen_subtitle_samples_hel879.py).
web/src untouched.
Verification: the .12 real-SDK check now LOADS THE ACTUAL WORKER MODULE and drives
the callbacks IT registered on a real AgentSession (no hand-copied functions) -
both-queued delayed generations bind exactly, a say sandwiched between user turns
steals nothing, unattributed generations fail closed, interrupted turns cancel
without final/receipt (11/11 ALL PASS, source-level proof of the ContextVar wiring
included; evidence docs/rework/hel879/sdk-check-hel879-remaining.log). The old r2
lifespan test flake is root-caused (clock freeze and block were two separate portal
calls; a 10ms life tick could slip between them and claim the frozen day unblocked)
and fixed atomically without raising timeouts or dropping assertions - 12/12 solo,
full suite 350/350 three times in a row on .venv311; ruff back to the single
pre-existing error; voice chain 8/8; release offline R1-R7 with zero failures;
requirement checks 4/4 (origin/main and c121c42github@multica.ai >
2026-10-27 05:10:52 +08:00
zonggong and multica-agent
7c73d392b9
HEL-879 rework materials: hatchling cache correction with the offline mem0ai wheel actually built from the fixed source (sha256 698284b6..., zero downloads/installs, no 3.10 mixing, recorded in release-artifacts.json), hel895 release doc §0 now separates the historical HEL-895/896 candidate from the current hel-879-assembly candidate and §4/§5 carry the v3 snapshot semantics, budget §6.2.4 amended - a sealed library is never reopened by raising total_requests (contradiction removed), runbook §3.3/3.5/3.6/4.4/4.5 corrected (bounded cleanup ledger replaces the unproven rooms-die-naturally claim; assistant_interim kind; real-but-default-off proactive seam; single-owner dual launcher), one comment-only correction in web/src/voice/session.ts removing the unproven server-auto-cleanup claim (no logic/style change; CSS and JS bundle hashes byte-stable), CG-011 + her incremental subtitles now have a real minimal backend seam (voice/seam.py + worker/llm_chat_api wiring, proactive default off) with the precise frontend interface and acceptance list in docs/rework/hel879/voice-seam-interfaces.md, real-SDK check extended to 10 items (worker callback path on real SpeechHandle/SpeechCreatedEvent; interim seam through real LLMStream) and re-run on .12 ALL PASS, handoff-HEL-879.json appended with the rework scope; regression at this head: pytest 341/341, ruff clean except 1 pre-existing error identical at 7f0a739, web typecheck 0 errors / lint 0 errors 1 pre-existing warning / vitest 88/88 / build passes (CSS index-BpZaybfD.css unchanged), voice chain 8/8, budget split 10/10, release offline R1-R7, http pagination 7/7, real-service browser 9/9 with zero non-loopback outbound, pinned-SDK 16/16+12/12+6/6+7/7+12/12, requirements checks 4/4 (origin/main and c121c42, each with/without handoff)
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-27 04:27:04 +08:00
zonggong and multica-agent
7f66015ac8
HEL-879 unified voice candidate assembled and verified on hel-879-assembly (isolated branch; PR#11 and hel-870-assembly untouched) - the merge brings exactly the audited HEL-877 voice backend ( aba2aeb: src/cybergril/voice module with zero new dependencies, byte-identical to the author's head) and the HEL-897-verified voice frontend ( c61d9c9: web/src byte-identical to the original author's head - the CSS hash change to index-BpZaybfD.css is entirely the author's own components.css +21 lines, no visual rework by this card); the one integration adjustment is test-harness alignment, not a product change: the voice adapter fixtures and verify_hel877_voice_chain.py built their router without a budget config, which the mainline's HEL-882 hard gate (guard.acquire before every real send) now correctly refuses at cap 0 - they now use the same tests/conftest.engineering_budget() every mainline test uses, no gate weakened, no src change on either author's side; full battery rerun on the assembled head: pytest 324/324 (299 mainline + 25 voice), ruff clean, web typecheck 0 errors / lint 0 errors 1 pre-existing warning / vitest 88/88 / build passes, real-server browser 9/9 with the HEL-896 precise-interaction revision (reach card over real HTTP, stage plays the reach-out, unread clears, message-id dedup, restart epoch+1, zero permission API calls, zero non-loopback outbound - evidence docs/rework/hel879/browser/), verify_hel877_voice_chain 8/8 offline full-chain, the .12 livekit-agents 1.8.8 real-SDK stdin check regenerated from THIS head's sources 8/8 ALL PASS (zero writes zero network), check_release_offline R1-R7 passes, pinned-SDK regressions on hel857-venv all green (close-ownership 16/16, shutdown-queued 12/12, inflight 6/6, semantic-recovery 7/7, mem0-real 12/12 - the seal change is exercised through the same PersistentBudget paths), requirement checks 4/4 (origin/main and c121c42, each with and without handoff-HEL-879.json); release-artifacts.json records the exact release code commit ( 01ff919), the tarball sha256 measured equal to the lock, and the cache-reusable wheels reverified with content digests after the disk-recovery cache incident; honest gaps kept open: real-call acceptance untested and unauthorized (D1 voice, trusted HTTPS device path, budgets), LiveKit room/worker release after API restart still lacks real-environment evidence (_prune only drops the in-memory registry, end_session does delete_room), incremental subtitles remain a contract gap not a completed feature, CG-011 proactive voice stays an explicit no-op; zero real LLM/embed/ASR/TTS, zero downloads, no mic, no service started or stopped on any shared machine, not deployed, nothing merged, authoritative ledgers/freezes/HEL-835 gate untouched, real caps stay 0
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-27 03:36:37 +08:00
施工员2号
aba2aebeb0
HEL-877: wire the audited .12 voice base into the one CyberGril chat chain as backend-only code (no deploy, no real voice call) - new independent src/cybergril/voice module: session+router add POST /api/voice/session / DELETE end / GET honest status on the unified instance (server-side room + dedicated cybergril-voice-agent dispatch + short-TTL join token via zero-new-dependency livekit_wire HS256/Twirp matching the installed SDK's wire format verified claim-for-claim on the .12 agent venv stdin-only; keys never reach the browser; unconfigured/no-profile/TTS-down refuse honestly and capabilities.voice stays false), chat_api_client speaks the exact real.ts RealChatAdapter protocol (per-turn health epoch, bounded 409 resync, robust incremental SSE parsing across multibyte/CRLF/multi-data chunk splits, fixed turn_id idempotency, immediate HTTP-stream close on barge-in without waiting for the next ASR final), llm_chat_api is a livekit llm.LLM adapter whose only backend is POST /api/chat so persona/mood/dyad/memory/life-sharing/budget gates and the one same-turn chat_lock govern text and voice identically (phi spoken verbatim via ChatChunk, never archived, no receipt), receipt.py confirms a sharing receipt only after real SDK playback completion with every negative (interrupted/failed/mid-playout/phi/incomplete/stale-epoch) fail-closed and exactly once per turn, stt_funasr/tts_cosyvoice reuse the proven .12 adapters verbatim with provenance, worker.py assembles the dedicated worker (independent voice_state_cybergril.json so D1 stays undecided, no second persona, cg.subtitle data-channel contract for the visual lead, CG-011 proactive seam left as an explicit no-op hook, built-in health port 8082); app.py keeps a minimal registration with voice_service injection for isolated tests; deployment drafts only (config/voice.example.yaml, worker.env.example, restart_cybergril.sh.example with exact-pattern process matching, serve_dual_8360_8361.py keeping HTTP 8360 and the HTTPS voice entry on ONE process sharing state/locks, runbook-hel877-voice.md contract+startup+rollback incl. the >=6GB GPU precondition and the do-not-copy historical fuser warning) - verification: 281 pytest pass on 3.11 venv + requirements.lock after merging the chief engineer's mid-flight HEL-874 rework e978e87 in full (parallel work preserved), verify_hel877_voice_chain.py 8/8 offline full-chain incl. socket-level barge-in fail-close and text/voice mutual same-gate cancellation, real-SDK check on the .12 livekit-agents 1.8.8 venv 8/8 ALL PASS via SSH stdin with zero writes zero network, dual-port single-process smoke on loopback 18360/18361 with a throwaway self-signed cert then stopped, ruff clean on all new code, dual-target requirement checks pass vs origin/hel-870-assembly (current tip e978e87) and origin/main with handoff-HEL-877.json; zero real LLM/embed/TTS/ASR requests, no mic/camera, no downloads/installs, no service start/stop, no 4080 access, no third-account or profile writes, web/ untouched, not merged, not deployed
...
Co-authored-by: multica-agent <github@multica.ai >
## HEL-877 交付摘要（R5 后台施工，未部署未合并）
- 新独立模块 src/cybergril/voice/：专属 livekit worker（agent_name=cybergril-voice-agent，复用 .12 FunASR/CosyVoice/silero 底座，不 fork 不另搭平台，无第二人格/记忆，不直连 LLM）；llm_chat_api 适配器与文字前端 real.ts 同协议走 POST /api/chat（epoch 409 有界重同步、SSE 分片、turn_id 幂等、打断即断流）；/api/voice/session 建房+专属派发+短时限 token 全服务端（密钥不进浏览器；缺配置/无档案/TTS 停机诚实拒绝；capabilities.voice 不伪造）；receipt 仅在 SDK 真实播放完成后确认（打断/失败/φ 轮/旧 epoch 全 fail-closed）。
- 部署草案：/opt/voice-agent-cybergril 目录、worker 健康端口 8082、HTTP 8360 + HTTPS 8361 同进程双入口（deploy/voice/serve_dual_8360_8361.py，共用同一状态/锁）、启动顺序含 TTS GPU≥6GB 前置与回滚；本卡未执行任何启停。契约（开关/字幕/失败/打断/挂断 + CG-011 主动开口接缝）见 docs/runbook-hel877-voice.md §3。
- 自测（零真实请求）：281 pytest 全过（含语音 25 例：socket 级打断断流、SSE 边界、重放零调用、文字/语音同门互打断、回执全负例、token 房间隔离）；scripts/verify_hel877_voice_chain.py 8/8；.12 agent venv（livekit-agents 1.8.8）真 SDK 检查 8/8（stdin-only 零写入零网络）；双端口同进程本地冒烟通过；ruff 干净；需求检查双目标通过（含 handoff-HEL-877.json）。
- 施工中总工 HEL-874（e978e87）推上同分支，本卡已完整并入，不覆盖并行成果。未验证如实列明：端到端语音（TTS 停机/GPU 3.9GB<5.6GB）、真 LiveKit server 的 dispatch/delete、移动端实机——待接线/审核卡真机验收，未经真实验收不称 R5 完成。
2026-10-26 22:42:19 +08:00
施工员2号 and multica-agent
23415fbcd0
HEL-876: voice-base read-only verification + R5/R6 readiness materials - docs/voice-r5-readiness-HEL876.md records the 2026-10-26 bounded read-only audit of 192.168.200.12 with timestamps: LiveKit/FunASR/8090 alive with health evidence, 8081 corrected as the voice-agent livekit worker (not a legacy lip-sync piece), CosyVoice TTS 9880 gracefully stopped since 2026-10-24 22:17 and NOT running, MuseTalk 8010 unit gone, ComfyUI 8188 holding 26.6GB of the 32GB GPU leaving 3.9GB free < ~5.6GB TTS requirement (the only hard resource gap, coordination not unilateral action), voice catalog (11 voices, rain selected) with whitelisted config field names only and no credentials read or printed, .11 gateway model-list probe and open-meteo outbound both 200; the R5 seam is fixed at src/cybergril/api/app.py:488 - the worker must reuse the same chat chain via a livekit LLM adapter speaking the identical RealChatAdapter protocol (epoch sync/serialized turns/barge-in cancel/idempotent replay/receipt round-trip), proven offline with zero model calls by scripts/verify_hel876_voice_seam.py (six checks all pass on 3.11 venv + requirements.lock installed from the local uv offline cache, ruff clean, explicitly NOT claimed as end-to-end voice verification); R6 seams located read-only: city placeholder ManagePanel.tsx:182 with no backend, no weather client (outbound now proven), demo-only material import :178/:183, full-backup boundary = companion.db + vec_public + mem0_public_history.db + embed ledger where copying the single db file is not a complete backup and Persona.fingerprint is the same-her check; three-column honest conclusions (verified / unverified / actually-missing) plus six zero-cost work items ready for dispatch; no product code touched (src/ web/ unchanged), no new product PR, candidate branch not merged, dual-target requirement checks 4/4 pass with handoff-HEL-876.json
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-26 21:42:41 +08:00
白栖知 and multica-agent
c285a0fd07
fix(r2): keep her self-narration and life within the approved room
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-24 12:03:08 +08:00
白栖知 and multica-agent
4575b27515
HEL-827: implement durable life, receipt-gated disclosure and persona layers
...
Co-authored-by: multica-agent <github@multica.ai >
2026-10-24 09:04:40 +08:00
白栖知 and multica-agent
003d8c2cae
HEL-822: land HEL-821 A1+A2 source onto PR#4 base ( db571ba)
...
- 13 modified / 8 added / 1 deleted per CHANGES.json (author package 76bc120b)
- docs/requirements v1.10 + prototypes/hel-799 + PROVENANCE from bundle, verified byte-identical to 63799f063799f0github@multica.ai >
2026-10-24 06:10:53 +08:00
