Commit Graph
3 Commits
Author SHA1 Message Date
施工员2号andmultica-agent ce7279e68f HEL-899: custom voice via reference audio on the unified API — reuse .12 panel zero-shot registry (server-side proxy, cg- namespaced IDs never collide with the 11 builtin voices, panel stays the only voices.json writer) + CosyVoice fresh-synthesis preview (unknown voice pre-checked: no silent default fallback) + persisted selection/manifest/usage journal in instance data space with atomic writes and full rollback; session-open captures the effective voice (selection if valid else worker-file value else default, D1 domain untouched when user never chose) pinned per call, mid-call changes land at next call at the earliest; explicit allow_asr gates FunASR transcription, preview/ASR journaled, preview writes no chat/memory/receipt rows; zero new deps (raw-bytes upload endpoint), zero real calls in tests: 18 new cases (formats/size/duration refusals, collision+concurrency, panel down/reject/lying, tts down, rollback, restart persistence, call pinning, identity/shared-state untouched) + offline chain script 12/12, suite 370; req governance v1.12: CG-010 boundary updated to custom-voice-first per user comment 01a17de1 (verbatim evidence + changes + manifest), checks pass vs origin/main and base 8dc84f8 with handoff-HEL-899
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 23:34:00 +08:00
hel-879-workerandmultica-agent 11fd0e2dc2 HEL-879 R5 test package v2 (materials rework per manager's 4-item list; zero execution): (1) dependencies split into two tracks - R5 voice-only chain verified present (LiveKit 7880 plain ws / FunASR / agent venv livekit-agents 1.8.8 / CosyVoice weights, GPU 30520MiB free) vs FULL unified experience honestly listed as 41-item closure truly-missing (mem0ai fixed-source wheel offline-built sha256 698284b6 352586B, greenlet/pillow/truststore cache-reusable, 37 pending download authorization, hatchling resolved; existing .12 product venvs probed have NO mem0 and are system 3.12.3 - 'zero missing' does not hold for the full experience; fastembed models actually 190.4MB+0.05MB, hel895 doc's ~92MB flagged as erratum); (2) trusted HTTPS+WSS rewritten with evidence - retracted 'one self-signed click covers LiveKit WSS 7880' (7880 is plain ws shared by generic worker and 8090 demo, 7881 is not an HTTP endpoint, the 8090 https page itself hands browsers ws:// = mixed content with no proof it ever worked); proposed dedicated 8363 TLS bridge deploy/voice/wss_bridge.example.py (asyncio+ssl stdlib, zero new deps; local smoke TLS1.3 + byte transparency pass, wss-bridge-smoke.log), livekit_url=wss://...:8363, per-device per-port trust steps, ignore-cert switches explicitly banned, iOS/Android behavior marked pending real device; (3) budget per-category - acquire() is an atomic pre-send hold so failures/retries/interrupts/background all count (LLM call count != successful chat lines); LLM+embedding = code-enforced hard gates, ASR/TTS/session-creation honestly listed as ops-only caps (window<=2h, calls<=3, TTS<=200 segments; session-cap listed as engineering follow-up); proposed caps do not self-activate, caps stay 0; (4) engineering params decided by engineering - 8360/8361/8363/8082 and /opt/voice-agent-cybergril + /opt/cybergril-voice-certs all verified conflict-free read-only; D1 rewritten to the real 11 voice IDs with zero-synthesis reference-audio audition plan (8090 panel has NO audition and mutates shared state - v1 claim retracted; generic demo 'rain' is not this user's choice). .12 facts from read-only SSH probes, evidence sheet r5-facts-20261027.md (secrets redacted); web/src and src zero changes; 4x requirements checks pass (origin/main and f92a86f, with/without handoff); handoff appended; zero real LLM/embed/ASR/TTS/mic, zero download/install/deploy/merge, real caps stay 0
Co-authored-by: multica-agent <github@multica.ai>
2026-10-27 06:44:42 +08:00
施工员2号 aba2aebeb0 HEL-877: wire the audited .12 voice base into the one CyberGril chat chain as backend-only code (no deploy, no real voice call) - new independent src/cybergril/voice module: session+router add POST /api/voice/session / DELETE end / GET honest status on the unified instance (server-side room + dedicated cybergril-voice-agent dispatch + short-TTL join token via zero-new-dependency livekit_wire HS256/Twirp matching the installed SDK's wire format verified claim-for-claim on the .12 agent venv stdin-only; keys never reach the browser; unconfigured/no-profile/TTS-down refuse honestly and capabilities.voice stays false), chat_api_client speaks the exact real.ts RealChatAdapter protocol (per-turn health epoch, bounded 409 resync, robust incremental SSE parsing across multibyte/CRLF/multi-data chunk splits, fixed turn_id idempotency, immediate HTTP-stream close on barge-in without waiting for the next ASR final), llm_chat_api is a livekit llm.LLM adapter whose only backend is POST /api/chat so persona/mood/dyad/memory/life-sharing/budget gates and the one same-turn chat_lock govern text and voice identically (phi spoken verbatim via ChatChunk, never archived, no receipt), receipt.py confirms a sharing receipt only after real SDK playback completion with every negative (interrupted/failed/mid-playout/phi/incomplete/stale-epoch) fail-closed and exactly once per turn, stt_funasr/tts_cosyvoice reuse the proven .12 adapters verbatim with provenance, worker.py assembles the dedicated worker (independent voice_state_cybergril.json so D1 stays undecided, no second persona, cg.subtitle data-channel contract for the visual lead, CG-011 proactive seam left as an explicit no-op hook, built-in health port 8082); app.py keeps a minimal registration with voice_service injection for isolated tests; deployment drafts only (config/voice.example.yaml, worker.env.example, restart_cybergril.sh.example with exact-pattern process matching, serve_dual_8360_8361.py keeping HTTP 8360 and the HTTPS voice entry on ONE process sharing state/locks, runbook-hel877-voice.md contract+startup+rollback incl. the >=6GB GPU precondition and the do-not-copy historical fuser warning) - verification: 281 pytest pass on 3.11 venv + requirements.lock after merging the chief engineer's mid-flight HEL-874 rework e978e87 in full (parallel work preserved), verify_hel877_voice_chain.py 8/8 offline full-chain incl. socket-level barge-in fail-close and text/voice mutual same-gate cancellation, real-SDK check on the .12 livekit-agents 1.8.8 venv 8/8 ALL PASS via SSH stdin with zero writes zero network, dual-port single-process smoke on loopback 18360/18361 with a throwaway self-signed cert then stopped, ruff clean on all new code, dual-target requirement checks pass vs origin/hel-870-assembly (current tip e978e87) and origin/main with handoff-HEL-877.json; zero real LLM/embed/TTS/ASR requests, no mic/camera, no downloads/installs, no service start/stop, no 4080 access, no third-account or profile writes, web/ untouched, not merged, not deployed
Co-authored-by: multica-agent <github@multica.ai>

## HEL-877 交付摘要(R5 后台施工,未部署未合并)

- 新独立模块 src/cybergril/voice/:专属 livekit worker(agent_name=cybergril-voice-agent,复用 .12 FunASR/CosyVoice/silero 底座,不 fork 不另搭平台,无第二人格/记忆,不直连 LLM);llm_chat_api 适配器与文字前端 real.ts 同协议走 POST /api/chat(epoch 409 有界重同步、SSE 分片、turn_id 幂等、打断即断流);/api/voice/session 建房+专属派发+短时限 token 全服务端(密钥不进浏览器;缺配置/无档案/TTS 停机诚实拒绝;capabilities.voice 不伪造);receipt 仅在 SDK 真实播放完成后确认(打断/失败/φ 轮/旧 epoch 全 fail-closed)。
- 部署草案:/opt/voice-agent-cybergril 目录、worker 健康端口 8082、HTTP 8360 + HTTPS 8361 同进程双入口(deploy/voice/serve_dual_8360_8361.py,共用同一状态/锁)、启动顺序含 TTS GPU≥6GB 前置与回滚;本卡未执行任何启停。契约(开关/字幕/失败/打断/挂断 + CG-011 主动开口接缝)见 docs/runbook-hel877-voice.md §3。
- 自测(零真实请求):281 pytest 全过(含语音 25 例:socket 级打断断流、SSE 边界、重放零调用、文字/语音同门互打断、回执全负例、token 房间隔离);scripts/verify_hel877_voice_chain.py 8/8;.12 agent venv(livekit-agents 1.8.8)真 SDK 检查 8/8(stdin-only 零写入零网络);双端口同进程本地冒烟通过;ruff 干净;需求检查双目标通过(含 handoff-HEL-877.json)。
- 施工中总工 HEL-874(e978e87)推上同分支,本卡已完整并入,不覆盖并行成果。未验证如实列明:端到端语音(TTS 停机/GPU 3.9GB<5.6GB)、真 LiveKit server 的 dispatch/delete、移动端实机——待接线/审核卡真机验收,未经真实验收不称 R5 完成。
2026-10-26 22:42:19 +08:00