diff --git a/docs/rework/hel879/d1-audition/README.md b/docs/rework/hel879/d1-audition/README.md index fc0993c..6f07226 100644 --- a/docs/rework/hel879/d1-audition/README.md +++ b/docs/rework/hel879/d1-audition/README.md @@ -4,6 +4,14 @@ > 可离线播放的 HTML 试听页 + 音频包,作为主卡 D1 拍板材料。**零 TTS 合成、零 > 克隆、零共享音色切换、零下载安装**;对 .12 只做 SSH 只读复制,未改服务器任何 > 文件与状态。 +> +> **2026-10-27 勘误(主卡授权轮)**:白栖知反馈「让我解压附件包试听」的设计 +> 不合理(主卡评论 `01a17de1-2892-78be-a422-eb6a11ffebfd`)——他要的是可 +> 自定义音色,而不是从预设音色里挑一个。本包正名为「**非定制功能入口**」: +> 它只是既有 11 支登记音色的参考原声对照材料,**不是**用户音色定制/上传/录制 +> 的功能入口;正式的自定义音色(上传、录制、试听、应用)由独立施工分支接续, +> 未完成前不宣称可用。本轮 R5 有限测试由总管选定临时测试音色 `female_soft` +> (已核服务器登记;临时测试值,不是用户最终音色),不要求用户解压本包选 ID。 ## 包内容(随本卡评论附件交付,不入仓库大媒体) diff --git a/docs/rework/hel879/r5-window-20261027.md b/docs/rework/hel879/r5-window-20261027.md new file mode 100644 index 0000000..4f892bf --- /dev/null +++ b/docs/rework/hel879/r5-window-20261027.md @@ -0,0 +1,100 @@ +# HEL-879 · R5 有限真实语音测试窗口记录(2026-10-27) + +> 授权:白栖知主卡评论 `01a17de1-2892-78be-a422-eb6a11ffebfd`(同意最长 2 小时 / +> 3 次通话;音色由工程自定);额度沿《r5-test-package.md》§6 申请上限执行。 +> 窗口(UTC):**15:26:54 – 15:58:10**(31 分 16 秒,未用满 2 小时)。 +> 候选 `hel-879-assembly` @ `8dc84f8`;模型/运行时/认证与既有实例一致(同一 +> `192.168.200.11:8731` halogen 端点),零新增付费支出。 + +## 1. 部署布局(全部独立,已全部撤除) + +| 组件 | 端口/路径 | 说明 | +| --- | --- | --- | +| 统一实例(同进程双入口) | HTTP 8360 + HTTPS 8361 | `deploy/voice/serve_dual_8360_8361.py`,隔离数据 `/opt/cybergril-hel879/data/`;`.venv311`+uv CPython 3.11.16 自本机离线复制(.12 无 uv/node,零下载) | +| WSS TLS 桥 | 8363 → 127.0.0.1:7880 | `deploy/voice/wss_bridge.example.py`,与 8361 同一张自签证书 | +| 专属 worker | 健康口 8082 | `/opt/voice-agent-cybergril/`,复用 agent venv(livekit-agents 1.8.8),`voice_state_cybergril.json` = `female_soft`(临时测试值;未写通用助手 `voice_state.json`/`voices.json`) | +| 共享 TTS(CosyVoice) | 9880 | 窗口内从停机恢复;结束时核实无他人占用后按窗口前状态停回 | +| 测试档案 | — | 隔离实例自建测试伴侣(名字「小测」),非用户真实档案 | + +配置差异(均记录在案、可回滚):`models.yaml` 预算写入本轮批准值 +(dialogue 40、total 40/500k,其余类别 0);`CYBERGRIL_SEMANTIC=stub`(嵌入 0); +`CYBERGRIL_R2_ENABLED=1`(receipt 签发依赖生活运行时开启;life/background 预算 +仍为 0,实测无后台模型调用);`voice.yaml` `session_ttl_seconds: 900` +(真实验证 TTL 到期删房用;仓库默认 7200)。 + +## 2. 真机发现并修复的代码问题(有限返工,含证据) + +1. **Twirp 服务 token 缺少端点授权(401)**:`livekit_wire.mint_server_token` + 原按「无 grants 形态」铸造,但已装 SDK 的 `create_dispatch` 以 + `VideoGrants(room_admin=True, room=…)` 签名、`delete_room` 以 + `VideoGrants(room_create=True)` 签名;真机 LiveKit 拒绝无授权 bearer(401)。 + 离线期 8 项 SDK 检查只比对了 token 形态,未对真服务器发送,未暴露此问题。 + 修复:`mint_server_token` 增加 `video` grants 参数,两处调用按 SDK 同款授权。 +2. **protobuf 响应不可解析**:CreateDispatch 以 `application/protobuf` 请求时 + 服务器回二进制,原实现的 `json.loads` 崩溃。修复:改走 Twirp JSON + (`application/json`),零新依赖,dispatch id 正常解析;离线替身同步改为 + JSON 断言(`tests/test_voice_session.py`、`tests/test_voice_rework_hel879.py`)。 +3. **自签证书 `CA:TRUE` 被 rustls 拒绝**:openssl 默认给自签证书加了 + `basicConstraints=CA:TRUE`,livekit 客户端(rustls)报 `CaUsedAsEndEntity`。 + 重签为 `critical,CA:FALSE` + `SAN=IP:192.168.200.12` 后通过;测试客户端以 + `SSL_CERT_FILE` 进程级信任该证书(非系统信任库、非浏览器例外冒充)。 + +修复后全量 `pytest tests -q` **352/352** 通过、ruff(src/tests/scripts)0 错; +本文件与修复随本轮提交入库。 + +## 3. 验证结果 + +真链:预录参考录音(**音频回放,非用户设备麦克风/手机**)→ 真实 WebRTC → +LiveKit 7880(客户端经 8363 WSS 桥)→ FunASR 流式识别 → 统一 `POST /api/chat` +(真 LLM)→ CosyVoice 合成(female_soft)→ WebRTC 回流收音(48k WAV 落盘)。 + +| 项 | 结果 | 证据 | +| --- | --- | --- | +| 连续 ≥3 轮(同档案、同对话源、音色一致) | ✅ 通话2:haimianbaobao/nezha/rain 三轮全部 `turn_end`;跨通话记忆延续可见(通话2 回复接住通话1 话题) | `call1b.subtitles.jsonl`、`chat_message` 全表 | +| 完整播放 receipt | ✅ 4 条 `sharing_attempt` 全部 `ack_at` 落值、state=judged;被截断轮无回执(fail-closed,msg7=cancelled) | `receipts_table`、`/api/chat/received 200` | +| 开口打断 | ✅(发生真实性)通话1 三次真实 `turn_cancel(interrupted)`,用户开口截断在播回复、旧轮 user 行标 cancelled;**精确 ~1s 停声计时未取得**(量测设计缺陷+后两轮回复过短无以复测,见 §5) | `call1.subtitles.jsonl` seq68/87/92 | +| 挂断立即停麦无残音 | ✅ DELETE 200 → 房间真实删除(ListRooms 空)、删除后仅 2 帧在途(20ms)、本地 track 停发 | `call1b.json hangup` | +| 断线(不挂断)+ TTL 到期后台删房 | ✅ 掉线不删 → 会话过期自动转债删除;15:52/15:56/15:57 三笔全部结清,账本空、房间空 | 轮询记录(窗口观察) | +| API 重启清理 | ✅ 重启 epoch 4→5、档案/历史完整、voice 配置在;重启遗留会话按 TTL 由补偿清理全部结清 | 同上 | +| 文字/语音互斥 | ✅(机制)文字轮被后到语音轮取消(msg23=cancelled)——同一把 chat 锁双向互斥成立;**「文字打断语音」方向未单独取证**(触发时机竞态,见 §5) | `call3.json`、消息表 | +| HTTPS/WSS | ✅ 8361/8363 同证书 TLS1.3、SAN=IP 可达;WSS 桥真实承载信令与媒体建立(客户端全程经 8363 接入) | 窗口日志、preflight | +| 嵌入 0 / 语义 stub | ✅ `semantic_backend=stub`,全程零嵌入调用 | `/api/health` | +| 字幕 v2 全生命周期 | ✅ seq 单调(三次通话均验证)、interim→final→turn_end / 打断→turn_cancel 无 final | 各 `subtitles.jsonl` | + +## 4. 用量账(逐笔可查,硬闸全程在位) + +- **LLM**:14/40 次请求、42,717/500,000 tokens(`budget_total`/`request_usage`/ + `token_usage` 逐行;含失败/取消轮,预占语义如代码注释)。结束即封账: + `seal_budget_hel879.py` 已执行,封账标记随库归档,真实用量保留不清零。 +- **TTS**:12/200 合成段(`tts.log` POST /tts 计数;本地 GPU、零外部计费)。 +- **ASR**:仅窗口内 3 次通话使用(本地 CPU FunASR,零外部计费)。 +- **嵌入/新增付费**:0。 +- **语音会话**:建房 6 次 = **3 次真实通话**(含全部验证轮次)+ 3 次工具端 + 崩溃遗留的空会话(1 次手工 DELETE、2 次由 TTL 补偿自动删除;空会话无对话、 + 零模型消耗,如实计数不隐瞒)。 + +## 5. 未测 / 留待下轮(如实) + +- **精确 1 秒级打断停声计时**:本轮三次真实打断均已发生并留痕,但帧级停声 + 时间戳未随证据落盘(下一轮客户端应在每帧记录到达时间并按 speech 归属切分)。 +- **「文字打断语音」方向**:本轮竞态取证到反向(语音取消文字);机制同一把锁, + 下轮把文字触发改为「assistant_interim 出现后」即可定向取证。 +- **真实设备浏览器路径**(D3 用户动作):证书「高级→继续」两处(8361/8363)、 + 真机麦克风(iOS Safari/安卓 Chrome)未测——须用户设备动作,本轮交付地址与 + 最小步骤,不以 rustls/SSL_CERT_FILE 冒充浏览器例外。 +- 回复质量观察:长上下文后出现单字回复与一条英文串扰(与测试档案 boundary + 文案提到「测试/临时」相关)——记录为内容质量观察项,非链路故障。 + +## 6. 收尾与回滚 + +- 窗口结束:worker/桥/双入口进程全部停止,8360/8361/8363/8082/9880 复核无监听; + TTS 按窗口前状态停回(精确 PID 终止,未用 fuser -k,核过无他人占用)。 +- 受保护服务零接触:8300/8310/8320/8350/7880/7881/10095 原样运行,8340 保持 + 停机,ComfyUI 保持暂停(GPU 回到 97MiB 占用)。 +- 数据封存:预算封账 + `release_snapshot_hel896.py` 停服一致性快照 + (`/opt/cybergril-hel879/archives/hel879-r5-window-close-*`,rows_match 全过); + 证据包 `/opt/cybergril-hel879/hel879-r5-evidence-20261027.tar.gz` + (sha256 `b23af43b…c394`,含三次通话字幕/JSON/音频/worker/TTS 日志/清理账)。 +- 回滚:全部为本轮新建独立进程/目录,删除 `/opt/cybergril-hel879`、 + `/opt/voice-agent-cybergril`、`/opt/cybergril-voice-certs` 即完全退场;Git 侧 + 为本分支正常追加提交,PR#11 与既有实例未动。 diff --git a/docs/rework/hel879/r5-window/call1.json b/docs/rework/hel879/r5-window/call1.json new file mode 100644 index 0000000..4b5c36f --- /dev/null +++ b/docs/rework/hel879/r5-window/call1.json @@ -0,0 +1,96 @@ +{ + "rounds": [ + { + "play_label": "xiaoyun", + "reply": { + "kind": "turn_cancel", + "speech_id": "speech_b687b8055307", + "reason": "interrupted" + } + }, + { + "play_label": "bailu", + "reply": { + "kind": "turn_cancel", + "speech_id": "speech_8f12c4a50957", + "reason": "no-receipt" + } + }, + { + "play_label": "jiaojiao", + "reply": { + "kind": "turn_cancel", + "speech_id": "speech_4c73b7d1829c", + "reason": "interrupted" + } + } + ], + "interrupt": { + "outcome": "interrupted", + "trigger": "audio", + "publish_ms": 1793115246711, + "cancel_ms": 1793115249971, + "cancel_latency_s": 3.26, + "last_agent_frame_ms": 1793115252008, + "audio_stop_latency_s": 5.3, + "frames_after_cancel": 204 + }, + "hangup": { + "delete_status": 200, + "delete_body": { + "ended": true, + "room": "cg-voice-qeghdnxehn" + }, + "delete_ms": 1793115252011, + "agent_frames_after_delete": 2, + "room_removed": true + }, + "call": { + "tag": "call1", + "session": { + "url": "wss://192.168.200.12:8363", + "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2aWRlbyI6eyJyb29tSm9pbiI6dHJ1ZSwicm9vbSI6ImNnLXZvaWNlLXFlZ2hkbnhlaG4iLCJjYW5QdWJsaXNoIjp0cnVlLCJjYW5TdWJzY3JpYmUiOnRydWUsImNhblB1Ymxpc2hEYXRhIjp0cnVlfSwic3ViIjoidXNlci1nd2h2ZXpjeiIsImlzcyI6ImRldmtleSIsIm5iZiI6MTc5MzExNTIwMywiZXhwIjoxNzkzMTE1NTAzLCJuYW1lIjoi55So5oi3In0.D91S6SXWBtoVoSC6TEsULuK8WNHQu36O_vZQS1urwbw", + "room": "cg-voice-qeghdnxehn", + "session_id": "Ey8pBEFHbrO8i6Lbc0-47mh7", + "agent_name": "cybergril-voice-agent", + "epoch": 3, + "expires_at": 1793116103636 + }, + "subtitle_count": 96, + "seq_monotonic": true, + "agent_frames": 4766, + "agent_audio_seconds": 47.7, + "agent_audio_rate": 48000, + "agent_joined_ms": 1793115204287, + "agent_left_ms": null, + "subtitles_file": "/opt/cybergril-hel879/evidence/call1.subtitles.jsonl", + "agent_audio_file": "/opt/cybergril-hel879/evidence/call1.agent.wav" + }, + "budget_after": { + "at_ms": 1793115254555, + "token_usage": [ + { + "day": "2026-10-27", + "scene": "dialogue_main", + "used": 11416 + } + ], + "request_usage": [ + { + "day": "2026-10-27", + "category": "dialogue", + "requests": 4 + } + ], + "budget_total": [ + { + "key": "total_requests", + "value": 4 + }, + { + "key": "total_tokens", + "value": 11416 + } + ] + } +} \ No newline at end of file diff --git a/docs/rework/hel879/r5-window/call1.subtitles.jsonl b/docs/rework/hel879/r5-window/call1.subtitles.jsonl new file mode 100644 index 0000000..8bf6b1d --- /dev/null +++ b/docs/rework/hel879/r5-window/call1.subtitles.jsonl @@ -0,0 +1,96 @@ +{"v": 2, "seq": 1, "kind": "interim", "role": "user", "text": "盯", "_at_ms": 1793115205011} +{"v": 2, "seq": 2, "kind": "interim", "role": "user", "text": "着你看怎", "_at_ms": 1793115205596} +{"v": 2, "seq": 3, "kind": "interim", "role": "user", "text": "么啦", "_at_ms": 1793115206196} +{"v": 2, "seq": 4, "kind": "user_final", "role": "user", "text": "盯着你看怎么啦", "_at_ms": 1793115207385} +{"v": 2, "seq": 5, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_b687b8055307", "_at_ms": 1793115207410} +{"v": 2, "seq": 6, "kind": "interim", "role": "user", "text": "难得", "_at_ms": 1793115207554} +{"v": 2, "seq": 7, "kind": "interim", "role": "user", "text": "看你这么", "_at_ms": 1793115208153} +{"v": 2, "seq": 8, "kind": "interim", "role": "user", "text": "有意思的", "_at_ms": 1793115208743} +{"v": 2, "seq": 9, "kind": "interim", "role": "user", "text": "样子", "_at_ms": 1793115209349} +{"v": 2, "seq": 10, "kind": "assistant_interim", "role": "assistant", "text": "没", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209395} +{"v": 2, "seq": 11, "kind": "assistant_interim", "role": "assistant", "text": "没怎么", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209436} +{"v": 2, "seq": 12, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209439} +{"v": 2, "seq": 13, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209478} +{"v": 2, "seq": 14, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209479} +{"v": 2, "seq": 15, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209522} +{"v": 2, "seq": 16, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209563} +{"v": 2, "seq": 17, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209565} +{"v": 2, "seq": 18, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209605} +{"v": 2, "seq": 19, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209611} +{"v": 2, "seq": 20, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209650} +{"v": 2, "seq": 21, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209688} +{"v": 2, "seq": 22, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209731} +{"v": 2, "seq": 23, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209773} +{"v": 2, "seq": 24, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查我", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209816} +{"v": 2, "seq": 25, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查我有没有", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209816} +{"v": 2, "seq": 26, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查我有没有乱", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209857} +{"v": 2, "seq": 27, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查我有没有乱动", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209900} +{"v": 2, "seq": 28, "kind": "assistant_interim", "role": "assistant", "text": "没怎么啦,工程师。你盯着看,我就当你在检查我有没有乱动。", "speech_id": "speech_b687b8055307", "_at_ms": 1793115209947} +{"v": 2, "seq": 29, "kind": "user_final", "role": "user", "text": "难得看你这么有意思的样子吗", "_at_ms": 1793115210503} +{"v": 2, "seq": 30, "kind": "turn_cancel", "role": "assistant", "text": "", "speech_id": "speech_b687b8055307", "reason": "interrupted", "_at_ms": 1793115210511} +{"v": 2, "seq": 31, "kind": "interim", "role": "user", "text": "烈会军校", "_at_ms": 1793115212136} +{"v": 2, "seq": 32, "kind": "interim", "role": "user", "text": "校啊招", "_at_ms": 1793115212733} +{"v": 2, "seq": 33, "kind": "interim", "role": "user", "text": "啊这种类", "_at_ms": 1793115213338} +{"v": 2, "seq": 34, "kind": "interim", "role": "user", "text": "型我都是", "_at_ms": 1793115213936} +{"v": 2, "seq": 35, "kind": "interim", "role": "user", "text": "那种比较", "_at_ms": 1793115214541} +{"v": 2, "seq": 36, "kind": "interim", "role": "user", "text": "呃豪", "_at_ms": 1793115215744} +{"v": 2, "seq": 37, "kind": "interim", "role": "user", "text": "爽比", "_at_ms": 1793115216340} +{"v": 2, "seq": 38, "kind": "interim", "role": "user", "text": "较开", "_at_ms": 1793115216943} +{"v": 2, "seq": 39, "kind": "interim", "role": "user", "text": "朗的那种", "_at_ms": 1793115217541} +{"v": 2, "seq": 40, "kind": "interim", "role": "user", "text": "类型", "_at_ms": 1793115218142} +{"v": 2, "seq": 41, "kind": "interim", "role": "user", "text": "阳光大", "_at_ms": 1793115218753} +{"v": 2, "seq": 42, "kind": "interim", "role": "user", "text": "女孩", "_at_ms": 1793115219344} +{"v": 2, "seq": 43, "kind": "interim", "role": "user", "text": "那种感", "_at_ms": 1793115220543} +{"v": 2, "seq": 44, "kind": "interim", "role": "user", "text": "觉", "_at_ms": 1793115221139} +{"v": 2, "seq": 45, "kind": "interim", "role": "user", "text": "都给", "_at_ms": 1793115221743} +{"v": 2, "seq": 46, "kind": "user_final", "role": "user", "text": "烈火军校啊,朝阳啊这种类型我都是那种比较呃豪爽,比较开朗的那种类型,阳光大女孩那种感觉就", "_at_ms": 1793115223193} +{"v": 2, "seq": 47, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115226967} +{"v": 2, "seq": 48, "kind": "assistant_interim", "role": "assistant", "text": "原来", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228249} +{"v": 2, "seq": 49, "kind": "assistant_interim", "role": "assistant", "text": "原来你是", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228291} +{"v": 2, "seq": 50, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228333} +{"v": 2, "seq": 51, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228335} +{"v": 2, "seq": 52, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228376} +{"v": 2, "seq": 53, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228377} +{"v": 2, "seq": 54, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228420} +{"v": 2, "seq": 55, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228464} +{"v": 2, "seq": 56, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228506} +{"v": 2, "seq": 57, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228508} +{"v": 2, "seq": 58, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228548} +{"v": 2, "seq": 59, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228590} +{"v": 2, "seq": 60, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228634} +{"v": 2, "seq": 61, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228636} +{"v": 2, "seq": 62, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228674} +{"v": 2, "seq": 63, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228716} +{"v": 2, "seq": 64, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股豪", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228717} +{"v": 2, "seq": 65, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股豪爽", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228758} +{"v": 2, "seq": 66, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股豪爽劲儿", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228760} +{"v": 2, "seq": 67, "kind": "assistant_interim", "role": "assistant", "text": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股豪爽劲儿。", "speech_id": "speech_8f12c4a50957", "_at_ms": 1793115228801} +{"v": 2, "seq": 68, "kind": "turn_cancel", "role": "assistant", "text": "", "speech_id": "speech_8f12c4a50957", "reason": "no-receipt", "_at_ms": 1793115239147} +{"v": 2, "seq": 69, "kind": "interim", "role": "user", "text": "怎么还", "_at_ms": 1793115240503} +{"v": 2, "seq": 70, "kind": "interim", "role": "user", "text": "有没事", "_at_ms": 1793115241090} +{"v": 2, "seq": 71, "kind": "interim", "role": "user", "text": "的", "_at_ms": 1793115241700} +{"v": 2, "seq": 72, "kind": "user_final", "role": "user", "text": "怎么还有没事的", "_at_ms": 1793115242576} +{"v": 2, "seq": 73, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115242585} +{"v": 2, "seq": 74, "kind": "interim", "role": "user", "text": "是我不够", "_at_ms": 1793115243790} +{"v": 2, "seq": 75, "kind": "assistant_interim", "role": "assistant", "text": "哈哈", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243833} +{"v": 2, "seq": 76, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243874} +{"v": 2, "seq": 77, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243875} +{"v": 2, "seq": 78, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243916} +{"v": 2, "seq": 79, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243917} +{"v": 2, "seq": 80, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真听", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115243958} +{"v": 2, "seq": 81, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真听你", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115244002} +{"v": 2, "seq": 82, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真听你说话", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115244044} +{"v": 2, "seq": 83, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真听你说话呀", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115244045} +{"v": 2, "seq": 84, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,因为我在认真听你说话呀。", "speech_id": "speech_4c73b7d1829c", "_at_ms": 1793115244086} +{"v": 2, "seq": 85, "kind": "interim", "role": "user", "text": "努力", "_at_ms": 1793115244386} +{"v": 2, "seq": 86, "kind": "user_final", "role": "user", "text": "是我不够努力吗", "_at_ms": 1793115245511} +{"v": 2, "seq": 87, "kind": "turn_cancel", "role": "assistant", "text": "", "speech_id": "speech_4c73b7d1829c", "reason": "interrupted", "_at_ms": 1793115245516} +{"v": 2, "seq": 88, "kind": "interim", "role": "user", "text": "盯着你看", "_at_ms": 1793115248139} +{"v": 2, "seq": 89, "kind": "interim", "role": "user", "text": "怎么", "_at_ms": 1793115248742} +{"v": 2, "seq": 90, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_7a9c0f804ca0", "_at_ms": 1793115249065} +{"v": 2, "seq": 91, "kind": "user_final", "role": "user", "text": "盯着你看怎么啦", "_at_ms": 1793115249963} +{"v": 2, "seq": 92, "kind": "turn_cancel", "role": "assistant", "text": "", "speech_id": "speech_7a9c0f804ca0", "reason": "interrupted", "_at_ms": 1793115249971} +{"v": 2, "seq": 93, "kind": "interim", "role": "user", "text": "难得", "_at_ms": 1793115250198} +{"v": 2, "seq": 94, "kind": "interim", "role": "user", "text": "看你这么", "_at_ms": 1793115250808} +{"v": 2, "seq": 95, "kind": "interim", "role": "user", "text": "有意思的", "_at_ms": 1793115251389} +{"v": 2, "seq": 96, "kind": "interim", "role": "user", "text": "样子", "_at_ms": 1793115251989} diff --git a/docs/rework/hel879/r5-window/call1b.json b/docs/rework/hel879/r5-window/call1b.json new file mode 100644 index 0000000..546099f --- /dev/null +++ b/docs/rework/hel879/r5-window/call1b.json @@ -0,0 +1,97 @@ +{ + "rounds": [ + { + "play_label": "haimianbaobao", + "reply": { + "kind": "turn_end", + "speech_id": "speech_e69e351f1c44", + "reason": null + } + }, + { + "play_label": "nezha", + "reply": { + "kind": "turn_end", + "speech_id": "speech_ff61928cdce0", + "reason": null + } + }, + { + "play_label": "rain", + "reply": { + "kind": "turn_end", + "speech_id": "speech_2f9dc8b9848d", + "reason": null + } + } + ], + "interrupt": { + "outcome": "no-cancel", + "trigger": "audio", + "publish_ms": 1793115490292, + "cancel_ms": null, + "cancel_latency_s": null, + "old_audio_stop_ms": 1793115512315, + "old_audio_stop_latency_s": 22.02, + "old_frames_during_interrupt": 2203, + "frames_after_cancel_plus1s5": 0 + }, + "hangup": { + "delete_status": 200, + "delete_body": { + "ended": true, + "room": "cg-voice-5c4y64wxkl" + }, + "delete_ms": 1793115512327, + "agent_frames_after_delete": 2, + "room_removed": true + }, + "call": { + "tag": "call1", + "session": { + "url": "wss://192.168.200.12:8363", + "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2aWRlbyI6eyJyb29tSm9pbiI6dHJ1ZSwicm9vbSI6ImNnLXZvaWNlLTVjNHk2NHd4a2wiLCJjYW5QdWJsaXNoIjp0cnVlLCJjYW5TdWJzY3JpYmUiOnRydWUsImNhblB1Ymxpc2hEYXRhIjp0cnVlfSwic3ViIjoidXNlci12ZHl3c2h2ZSIsImlzcyI6ImRldmtleSIsIm5iZiI6MTc5MzExNTQzMSwiZXhwIjoxNzkzMTE1NzMxLCJuYW1lIjoi55So5oi3In0.PjOElCid-0VLPw8w0iQml4ca9QkLqoCwu9EJf2pz4DE", + "room": "cg-voice-5c4y64wxkl", + "session_id": "jCiBIdbaj2_2_AjxrwChbQ4A", + "agent_name": "cybergril-voice-agent", + "epoch": 4, + "expires_at": 1793116331245 + }, + "subtitle_count": 98, + "seq_monotonic": true, + "agent_frames": 8042, + "agent_audio_seconds": 80.4, + "agent_audio_rate": 48000, + "agent_joined_ms": 1793115431859, + "agent_left_ms": null, + "subtitles_file": "/opt/cybergril-hel879/evidence/call1b.subtitles.jsonl", + "agent_audio_file": "/opt/cybergril-hel879/evidence/call1b.agent.wav" + }, + "budget_after": { + "at_ms": 1793115514868, + "token_usage": [ + { + "day": "2026-10-27", + "scene": "dialogue_main", + "used": 23504 + } + ], + "request_usage": [ + { + "day": "2026-10-27", + "category": "dialogue", + "requests": 8 + } + ], + "budget_total": [ + { + "key": "total_requests", + "value": 8 + }, + { + "key": "total_tokens", + "value": 23504 + } + ] + } +} \ No newline at end of file diff --git a/docs/rework/hel879/r5-window/call1b.subtitles.jsonl b/docs/rework/hel879/r5-window/call1b.subtitles.jsonl new file mode 100644 index 0000000..6178dc3 --- /dev/null +++ b/docs/rework/hel879/r5-window/call1b.subtitles.jsonl @@ -0,0 +1,98 @@ +{"v": 2, "seq": 1, "kind": "interim", "role": "user", "text": "没", "_at_ms": 1793115432559} +{"v": 2, "seq": 2, "kind": "interim", "role": "user", "text": "错", "_at_ms": 1793115433170} +{"v": 2, "seq": 3, "kind": "interim", "role": "user", "text": "我就是个", "_at_ms": 1793115433760} +{"v": 2, "seq": 4, "kind": "interim", "role": "user", "text": "笨蛋", "_at_ms": 1793115434360} +{"v": 2, "seq": 5, "kind": "interim", "role": "user", "text": "派大星", "_at_ms": 1793115434960} +{"v": 2, "seq": 6, "kind": "interim", "role": "user", "text": "我们", "_at_ms": 1793115435559} +{"v": 2, "seq": 7, "kind": "interim", "role": "user", "text": "一起去", "_at_ms": 1793115436165} +{"v": 2, "seq": 8, "kind": "interim", "role": "user", "text": "抓水母", "_at_ms": 1793115436768} +{"v": 2, "seq": 9, "kind": "interim", "role": "user", "text": "吧我", "_at_ms": 1793115437365} +{"v": 2, "seq": 10, "kind": "interim", "role": "user", "text": "们一起度", "_at_ms": 1793115437963} +{"v": 2, "seq": 11, "kind": "interim", "role": "user", "text": "过快乐的", "_at_ms": 1793115438565} +{"v": 2, "seq": 12, "kind": "interim", "role": "user", "text": "一天", "_at_ms": 1793115439163} +{"v": 2, "seq": 13, "kind": "user_final", "role": "user", "text": "没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天", "_at_ms": 1793115440793} +{"v": 2, "seq": 14, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115440831} +{"v": 2, "seq": 15, "kind": "assistant_interim", "role": "assistant", "text": "好", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115442938} +{"v": 2, "seq": 16, "kind": "assistant_interim", "role": "assistant", "text": "好呀", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115442940} +{"v": 2, "seq": 17, "kind": "assistant_interim", "role": "assistant", "text": "好呀,", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115442981} +{"v": 2, "seq": 18, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115442984} +{"v": 2, "seq": 19, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443024} +{"v": 2, "seq": 20, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443067} +{"v": 2, "seq": 21, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443068} +{"v": 2, "seq": 22, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443109} +{"v": 2, "seq": 23, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443110} +{"v": 2, "seq": 24, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443152} +{"v": 2, "seq": 25, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443154} +{"v": 2, "seq": 26, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443195} +{"v": 2, "seq": 27, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443237} +{"v": 2, "seq": 28, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443238} +{"v": 2, "seq": 29, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443279} +{"v": 2, "seq": 30, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443321} +{"v": 2, "seq": 31, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443322} +{"v": 2, "seq": 32, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443363} +{"v": 2, "seq": 33, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443406} +{"v": 2, "seq": 34, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443449} +{"v": 2, "seq": 35, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443450} +{"v": 2, "seq": 36, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443491} +{"v": 2, "seq": 37, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443534} +{"v": 2, "seq": 38, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海边", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443577} +{"v": 2, "seq": 39, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海边晒太阳", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443578} +{"v": 2, "seq": 40, "kind": "assistant_interim", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海边晒太阳。", "speech_id": "speech_e69e351f1c44", "_at_ms": 1793115443620} +{"v": 2, "seq": 41, "kind": "assistant_final", "role": "assistant", "text": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海边晒太阳。", "speech_id": "speech_e69e351f1c44", "turn_id": "b72de17e-bf42-470e-b396-c219bd7593f9", "_at_ms": 1793115457080} +{"v": 2, "seq": 42, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_e69e351f1c44", "turn_id": "b72de17e-bf42-470e-b396-c219bd7593f9", "_at_ms": 1793115457081} +{"v": 2, "seq": 43, "kind": "interim", "role": "user", "text": "天雷滚", "_at_ms": 1793115458260} +{"v": 2, "seq": 44, "kind": "interim", "role": "user", "text": "滚我好", "_at_ms": 1793115458865} +{"v": 2, "seq": 45, "kind": "interim", "role": "user", "text": "怕怕", "_at_ms": 1793115459463} +{"v": 2, "seq": 46, "kind": "interim", "role": "user", "text": "劈", "_at_ms": 1793115460064} +{"v": 2, "seq": 47, "kind": "interim", "role": "user", "text": "的我浑", "_at_ms": 1793115460657} +{"v": 2, "seq": 48, "kind": "interim", "role": "user", "text": "身掉渣", "_at_ms": 1793115461256} +{"v": 2, "seq": 49, "kind": "interim", "role": "user", "text": "渣突", "_at_ms": 1793115461862} +{"v": 2, "seq": 50, "kind": "interim", "role": "user", "text": "破天", "_at_ms": 1793115462458} +{"v": 2, "seq": 51, "kind": "interim", "role": "user", "text": "劫我笑", "_at_ms": 1793115463067} +{"v": 2, "seq": 52, "kind": "interim", "role": "user", "text": "哈哈", "_at_ms": 1793115463659} +{"v": 2, "seq": 53, "kind": "interim", "role": "user", "text": "逆", "_at_ms": 1793115464263} +{"v": 2, "seq": 54, "kind": "interim", "role": "user", "text": "天改命", "_at_ms": 1793115464867} +{"v": 2, "seq": 55, "kind": "interim", "role": "user", "text": "我吹喇", "_at_ms": 1793115465459} +{"v": 2, "seq": 56, "kind": "interim", "role": "user", "text": "叭滴", "_at_ms": 1793115466062} +{"v": 2, "seq": 57, "kind": "interim", "role": "user", "text": "答滴答", "_at_ms": 1793115466663} +{"v": 2, "seq": 58, "kind": "interim", "role": "user", "text": "滴滴", "_at_ms": 1793115467270} +{"v": 2, "seq": 59, "kind": "user_final", "role": "user", "text": "天雷滚滚,我好怕怕劈的我浑身掉渣渣,突破天劫,我笑哈哈逆天改命,我吹喇叭滴答滴答,滴滴答", "_at_ms": 1793115468935} +{"v": 2, "seq": 60, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115468946} +{"v": 2, "seq": 61, "kind": "assistant_interim", "role": "assistant", "text": "哈哈", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470187} +{"v": 2, "seq": 62, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470230} +{"v": 2, "seq": 63, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470231} +{"v": 2, "seq": 64, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470273} +{"v": 2, "seq": 65, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470315} +{"v": 2, "seq": 66, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得还挺", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470316} +{"v": 2, "seq": 67, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得还挺有", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470357} +{"v": 2, "seq": 68, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得还挺有节奏", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470359} +{"v": 2, "seq": 69, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得还挺有节奏感的", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470400} +{"v": 2, "seq": 70, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你唱得还挺有节奏感的!", "speech_id": "speech_ff61928cdce0", "_at_ms": 1793115470443} +{"v": 2, "seq": 71, "kind": "assistant_final", "role": "assistant", "text": "哈哈,你唱得还挺有节奏感的!", "speech_id": "speech_ff61928cdce0", "turn_id": "baa6c5c7-b483-49d4-905c-d6b7e5dd7786", "_at_ms": 1793115476107} +{"v": 2, "seq": 72, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_ff61928cdce0", "turn_id": "baa6c5c7-b483-49d4-905c-d6b7e5dd7786", "_at_ms": 1793115476109} +{"v": 2, "seq": 73, "kind": "interim", "role": "user", "text": "你是不是", "_at_ms": 1793115477463} +{"v": 2, "seq": 74, "kind": "interim", "role": "user", "text": "早就想", "_at_ms": 1793115478068} +{"v": 2, "seq": 75, "kind": "interim", "role": "user", "text": "睡我", "_at_ms": 1793115478661} +{"v": 2, "seq": 76, "kind": "interim", "role": "user", "text": "了", "_at_ms": 1793115479267} +{"v": 2, "seq": 77, "kind": "user_final", "role": "user", "text": "你是不是早就想睡我了呀", "_at_ms": 1793115480206} +{"v": 2, "seq": 78, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115480215} +{"v": 2, "seq": 79, "kind": "assistant_interim", "role": "assistant", "text": "哈哈", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481774} +{"v": 2, "seq": 80, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481776} +{"v": 2, "seq": 81, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481816} +{"v": 2, "seq": 82, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481816} +{"v": 2, "seq": 83, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481857} +{"v": 2, "seq": 84, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481859} +{"v": 2, "seq": 85, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太快了", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481903} +{"v": 2, "seq": 86, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太快了,", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481906} +{"v": 2, "seq": 87, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太快了,我都", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481945} +{"v": 2, "seq": 88, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太快了,我都跟不上", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481987} +{"v": 2, "seq": 89, "kind": "assistant_interim", "role": "assistant", "text": "哈哈,你思维跳跃得太快了,我都跟不上。", "speech_id": "speech_2f9dc8b9848d", "_at_ms": 1793115481988} +{"v": 2, "seq": 90, "kind": "assistant_final", "role": "assistant", "text": "哈哈,你思维跳跃得太快了,我都跟不上。", "speech_id": "speech_2f9dc8b9848d", "turn_id": "a91bbbb6-45b0-4671-84a2-027cc194f420", "_at_ms": 1793115489067} +{"v": 2, "seq": 91, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_2f9dc8b9848d", "turn_id": "a91bbbb6-45b0-4671-84a2-027cc194f420", "_at_ms": 1793115489068} +{"v": 2, "seq": 92, "kind": "interim", "role": "user", "text": "盯着你看", "_at_ms": 1793115491717} +{"v": 2, "seq": 93, "kind": "interim", "role": "user", "text": "怎么啦", "_at_ms": 1793115492309} +{"v": 2, "seq": 94, "kind": "user_final", "role": "user", "text": "盯着你看怎么啦", "_at_ms": 1793115493388} +{"v": 2, "seq": 95, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_0d075d37bc31", "_at_ms": 1793115493398} +{"v": 2, "seq": 96, "kind": "assistant_interim", "role": "assistant", "text": "没", "speech_id": "speech_0d075d37bc31", "_at_ms": 1793115494608} +{"v": 2, "seq": 97, "kind": "assistant_final", "role": "assistant", "text": "没", "speech_id": "speech_0d075d37bc31", "turn_id": "5bd95c66-b42b-4571-bfc2-fbf376c90747", "_at_ms": 1793115501161} +{"v": 2, "seq": 98, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_0d075d37bc31", "turn_id": "5bd95c66-b42b-4571-bfc2-fbf376c90747", "_at_ms": 1793115501163} diff --git a/docs/rework/hel879/r5-window/call3.json b/docs/rework/hel879/r5-window/call3.json new file mode 100644 index 0000000..187e1ae --- /dev/null +++ b/docs/rework/hel879/r5-window/call3.json @@ -0,0 +1,303 @@ +{ + "barges": [ + { + "label": "bajie", + "outcome": "reply-too-short-to-barge" + }, + { + "label": "sunwukong", + "outcome": "reply-too-short-to-barge" + }, + { + "label": "haimianbaobao", + "outcome": "reply-too-short-to-barge" + } + ], + "text_turn": { + "turn_id": "cb1bbe27-795f-45e3-b7bf-f2d6371c0830", + "status": 200, + "deltas": 0, + "final": null, + "ms": 114 + }, + "voice_reply_after_text": { + "kind": "turn_end", + "speech_id": "speech_9aa32dc387c3", + "reason": null + }, + "voice_cancelled_by_text": false, + "drop": { + "dropped_ms": 1793116007019, + "room": "cg-voice-oleganioz7", + "session_id": "Si6UZJbAENsNExR-B1cfpIEX", + "agent_frames_after_drop": 1 + }, + "call": { + "tag": "call3", + "session": { + "url": "wss://192.168.200.12:8363", + "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ2aWRlbyI6eyJyb29tSm9pbiI6dHJ1ZSwicm9vbSI6ImNnLXZvaWNlLW9sZWdhbmlvejciLCJjYW5QdWJsaXNoIjp0cnVlLCJjYW5TdWJzY3JpYmUiOnRydWUsImNhblB1Ymxpc2hEYXRhIjp0cnVlfSwic3ViIjoidXNlci11cWZ5dWhudCIsImlzcyI6ImRldmtleSIsIm5iZiI6MTc5MzExNTY5MCwiZXhwIjoxNzkzMTE1OTkwLCJuYW1lIjoi55So5oi3In0.wXERZSpOZVIidVRq-8dkIbb7e2gPa0HUX8_ZV-kHTN4", + "room": "cg-voice-oleganioz7", + "session_id": "Si6UZJbAENsNExR-B1cfpIEX", + "agent_name": "cybergril-voice-agent", + "epoch": 4, + "expires_at": 1793116590248 + }, + "subtitle_count": 104, + "seq_monotonic": true, + "agent_frames": 31510, + "agent_audio_seconds": 315.1, + "agent_audio_rate": 48000, + "agent_joined_ms": 1793115690867, + "agent_left_ms": null, + "subtitles_file": "/opt/cybergril-hel879/evidence/call3.subtitles.jsonl", + "agent_audio_file": "/opt/cybergril-hel879/evidence/call3.agent.wav" + }, + "messages": [ + { + "id": 1, + "role": "user", + "t": "盯着你看怎么啦", + "status": "sent", + "turn_id": "61605d50-17f0-4d9e-b407-60574304a827" + }, + { + "id": 2, + "role": "companion", + "t": "没怎么啦,工程师。你盯着看,我就当你在检查我有没有乱动。", + "status": "sent", + "turn_id": "61605d50-17f0-4d9e-b407-60574304a827" + }, + { + "id": 3, + "role": "user", + "t": "难得看你这么有意思的样子吗 烈火军校啊,朝阳啊这种类型我都是那种比较呃豪爽,比较开朗的那种类型,阳光大女孩那种感觉就", + "status": "sent", + "turn_id": "eb182bcd-3441-42d8-ba60-9eced3a6a5a5" + }, + { + "id": 4, + "role": "companion", + "t": "原来你是那种阳光大女孩的类型啊,难怪感觉你说话都透着一股豪爽劲儿。", + "status": "sent", + "turn_id": "eb182bcd-3441-42d8-ba60-9eced3a6a5a5" + }, + { + "id": 5, + "role": "user", + "t": "怎么还有没事的", + "status": "sent", + "turn_id": "0e1a5226-4ca6-44bb-a343-39647f0112e3" + }, + { + "id": 6, + "role": "companion", + "t": "哈哈,因为我在认真听你说话呀。", + "status": "sent", + "turn_id": "0e1a5226-4ca6-44bb-a343-39647f0112e3" + }, + { + "id": 7, + "role": "user", + "t": "是我不够努力吗", + "status": "cancelled", + "turn_id": "47d8d0ce-7ac7-4202-9757-64f5f4fb1300" + }, + { + "id": 8, + "role": "user", + "t": "没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天", + "status": "sent", + "turn_id": "b72de17e-bf42-470e-b396-c219bd7593f9" + }, + { + "id": 9, + "role": "companion", + "t": "好呀,那我们这就出发去抓水母!不过我可不想住进那个菠萝屋,我只喜欢在海边晒太阳。", + "status": "sent", + "turn_id": "b72de17e-bf42-470e-b396-c219bd7593f9" + }, + { + "id": 10, + "role": "user", + "t": "天雷滚滚,我好怕怕劈的我浑身掉渣渣,突破天劫,我笑哈哈逆天改命,我吹喇叭滴答滴答,滴滴答", + "status": "sent", + "turn_id": "baa6c5c7-b483-49d4-905c-d6b7e5dd7786" + }, + { + "id": 11, + "role": "companion", + "t": "哈哈,你唱得还挺有节奏感的!", + "status": "sent", + "turn_id": "baa6c5c7-b483-49d4-905c-d6b7e5dd7786" + }, + { + "id": 12, + "role": "user", + "t": "你是不是早就想睡我了呀", + "status": "sent", + "turn_id": "a91bbbb6-45b0-4671-84a2-027cc194f420" + }, + { + "id": 13, + "role": "companion", + "t": "哈哈,你思维跳跃得太快了,我都跟不上。", + "status": "sent", + "turn_id": "a91bbbb6-45b0-4671-84a2-027cc194f420" + }, + { + "id": 14, + "role": "user", + "t": "盯着你看怎么啦", + "status": "sent", + "turn_id": "5bd95c66-b42b-4571-bfc2-fbf376c90747" + }, + { + "id": 15, + "role": "companion", + "t": "没", + "status": "sent", + "turn_id": "5bd95c66-b42b-4571-bfc2-fbf376c90747" + }, + { + "id": 16, + "role": "user", + "t": "师傅,哎,这取经路比高,老庄种地还苦,俺老猪饿的能吞下石龙素包子,猴哥,你筋斗云块,要不先去化个缘", + "status": "sent", + "turn_id": "9bba6412-6679-48c2-9f1c-4bfc07f58aea" + }, + { + "id": 17, + "role": "companion", + "t": "师傅,", + "status": "sent", + "turn_id": "9bba6412-6679-48c2-9f1c-4bfc07f58aea" + }, + { + "id": 18, + "role": "user", + "t": "俺老孙来也,俺乃500年前大闹天宫的齐天大圣孙悟空,这里山势险峻,只怕藏有妖精妖怪,哪里逃吃?俺老孙吧", + "status": "sent", + "turn_id": "16006868-5ee7-49cc-b9c9-1a8dcfe74d08" + }, + { + "id": 19, + "role": "companion", + "t": "好", + "status": "sent", + "turn_id": "16006868-5ee7-49cc-b9c9-1a8dcfe74d08" + }, + { + "id": 20, + "role": "user", + "t": "没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天", + "status": "sent", + "turn_id": "fdf28a03-19a8-4fcb-ba8c-24587736c3f0" + }, + { + "id": 21, + "role": "companion", + "t": "好", + "status": "sent", + "turn_id": "fdf28a03-19a8-4fcb-ba8c-24587736c3f0" + }, + { + "id": 22, + "role": "user", + "t": "你是不是早就想睡我了呀", + "status": "sent", + "turn_id": "623bab59-0edc-4751-9feb-a9a7e4a11233" + }, + { + "id": 23, + "role": "user", + "t": "(文字通道)我改用打字了,这条应该立刻打断你正在说的话。", + "status": "cancelled", + "turn_id": "cb1bbe27-795f-45e3-b7bf-f2d6371c0830" + }, + { + "id": 25, + "role": "companion", + "t": "Considering the limited time by the user, I have to give the", + "status": "sent", + "turn_id": "623bab59-0edc-4751-9feb-a9a7e4a11233" + } + ], + "receipts": [ + { + "message_id": 9, + "turn_id": "b72de17e-bf42-470e-b396-c219bd7593f9", + "ack_at": 1793115457081, + "state": "judged" + }, + { + "message_id": 11, + "turn_id": "baa6c5c7-b483-49d4-905c-d6b7e5dd7786", + "ack_at": 1793115476109, + "state": "judged" + }, + { + "message_id": 13, + "turn_id": "a91bbbb6-45b0-4671-84a2-027cc194f420", + "ack_at": 1793115489069, + "state": "judged" + }, + { + "message_id": 15, + "turn_id": "5bd95c66-b42b-4571-bfc2-fbf376c90747", + "ack_at": 1793115501163, + "state": "judged" + }, + { + "message_id": 17, + "turn_id": "9bba6412-6679-48c2-9f1c-4bfc07f58aea", + "ack_at": 1793115715591, + "state": "judged" + }, + { + "message_id": 19, + "turn_id": "16006868-5ee7-49cc-b9c9-1a8dcfe74d08", + "ack_at": 1793115816209, + "state": "judged" + }, + { + "message_id": 21, + "turn_id": "fdf28a03-19a8-4fcb-ba8c-24587736c3f0", + "ack_at": 1793115908179, + "state": "judged" + }, + { + "message_id": 25, + "turn_id": "623bab59-0edc-4751-9feb-a9a7e4a11233", + "ack_at": 1793116005932, + "state": "pending" + } + ], + "budget": { + "at_ms": 1793116007051, + "token_usage": [ + { + "day": "2026-10-27", + "scene": "dialogue_main", + "used": 42717 + } + ], + "request_usage": [ + { + "day": "2026-10-27", + "category": "dialogue", + "requests": 14 + } + ], + "budget_total": [ + { + "key": "total_requests", + "value": 14 + }, + { + "key": "total_tokens", + "value": 42717 + } + ] + } +} \ No newline at end of file diff --git a/docs/rework/hel879/r5-window/call3.subtitles.jsonl b/docs/rework/hel879/r5-window/call3.subtitles.jsonl new file mode 100644 index 0000000..1d560c3 --- /dev/null +++ b/docs/rework/hel879/r5-window/call3.subtitles.jsonl @@ -0,0 +1,104 @@ +{"v": 2, "seq": 1, "kind": "interim", "role": "user", "text": "师", "_at_ms": 1793115691554} +{"v": 2, "seq": 2, "kind": "interim", "role": "user", "text": "傅", "_at_ms": 1793115692163} +{"v": 2, "seq": 3, "kind": "interim", "role": "user", "text": "哎这", "_at_ms": 1793115692756} +{"v": 2, "seq": 4, "kind": "interim", "role": "user", "text": "取经", "_at_ms": 1793115693359} +{"v": 2, "seq": 5, "kind": "interim", "role": "user", "text": "路比", "_at_ms": 1793115693964} +{"v": 2, "seq": 6, "kind": "interim", "role": "user", "text": "高老", "_at_ms": 1793115694564} +{"v": 2, "seq": 7, "kind": "interim", "role": "user", "text": "庄种", "_at_ms": 1793115695157} +{"v": 2, "seq": 8, "kind": "interim", "role": "user", "text": "地还", "_at_ms": 1793115695762} +{"v": 2, "seq": 9, "kind": "interim", "role": "user", "text": "苦俺", "_at_ms": 1793115696359} +{"v": 2, "seq": 10, "kind": "interim", "role": "user", "text": "老中", "_at_ms": 1793115696958} +{"v": 2, "seq": 11, "kind": "interim", "role": "user", "text": "饿的能", "_at_ms": 1793115697557} +{"v": 2, "seq": 12, "kind": "interim", "role": "user", "text": "吞下", "_at_ms": 1793115698168} +{"v": 2, "seq": 13, "kind": "interim", "role": "user", "text": "石龙", "_at_ms": 1793115698760} +{"v": 2, "seq": 14, "kind": "interim", "role": "user", "text": "素包", "_at_ms": 1793115699353} +{"v": 2, "seq": 15, "kind": "interim", "role": "user", "text": "子", "_at_ms": 1793115699967} +{"v": 2, "seq": 16, "kind": "interim", "role": "user", "text": "猴哥", "_at_ms": 1793115700562} +{"v": 2, "seq": 17, "kind": "interim", "role": "user", "text": "你筋", "_at_ms": 1793115701159} +{"v": 2, "seq": 18, "kind": "interim", "role": "user", "text": "斗云", "_at_ms": 1793115701768} +{"v": 2, "seq": 19, "kind": "interim", "role": "user", "text": "快", "_at_ms": 1793115702352} +{"v": 2, "seq": 20, "kind": "interim", "role": "user", "text": "要不", "_at_ms": 1793115702961} +{"v": 2, "seq": 21, "kind": "interim", "role": "user", "text": "先", "_at_ms": 1793115703557} +{"v": 2, "seq": 22, "kind": "interim", "role": "user", "text": "去画个", "_at_ms": 1793115704166} +{"v": 2, "seq": 23, "kind": "user_final", "role": "user", "text": "师傅,哎,这取经路比高,老庄种地还苦,俺老猪饿的能吞下石龙素包子,猴哥,你筋斗云块,要不先去化个缘", "_at_ms": 1793115706500} +{"v": 2, "seq": 24, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_ba7b3f730ffd", "_at_ms": 1793115710205} +{"v": 2, "seq": 25, "kind": "assistant_interim", "role": "assistant", "text": "师傅", "speech_id": "speech_ba7b3f730ffd", "_at_ms": 1793115711789} +{"v": 2, "seq": 26, "kind": "assistant_interim", "role": "assistant", "text": "师傅,", "speech_id": "speech_ba7b3f730ffd", "_at_ms": 1793115711850} +{"v": 2, "seq": 27, "kind": "assistant_final", "role": "assistant", "text": "师傅,", "speech_id": "speech_ba7b3f730ffd", "turn_id": "9bba6412-6679-48c2-9f1c-4bfc07f58aea", "_at_ms": 1793115715589} +{"v": 2, "seq": 28, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_ba7b3f730ffd", "turn_id": "9bba6412-6679-48c2-9f1c-4bfc07f58aea", "_at_ms": 1793115715590} +{"v": 2, "seq": 29, "kind": "interim", "role": "user", "text": "俺老", "_at_ms": 1793115792459} +{"v": 2, "seq": 30, "kind": "interim", "role": "user", "text": "孙来", "_at_ms": 1793115793061} +{"v": 2, "seq": 31, "kind": "interim", "role": "user", "text": "也俺", "_at_ms": 1793115793659} +{"v": 2, "seq": 32, "kind": "interim", "role": "user", "text": "乃五百", "_at_ms": 1793115794261} +{"v": 2, "seq": 33, "kind": "interim", "role": "user", "text": "年前", "_at_ms": 1793115794859} +{"v": 2, "seq": 34, "kind": "interim", "role": "user", "text": "大闹", "_at_ms": 1793115795470} +{"v": 2, "seq": 35, "kind": "interim", "role": "user", "text": "天宫的", "_at_ms": 1793115796055} +{"v": 2, "seq": 36, "kind": "interim", "role": "user", "text": "齐天", "_at_ms": 1793115796661} +{"v": 2, "seq": 37, "kind": "interim", "role": "user", "text": "霸圣", "_at_ms": 1793115797263} +{"v": 2, "seq": 38, "kind": "interim", "role": "user", "text": "孙悟", "_at_ms": 1793115797864} +{"v": 2, "seq": 39, "kind": "interim", "role": "user", "text": "空这", "_at_ms": 1793115798464} +{"v": 2, "seq": 40, "kind": "interim", "role": "user", "text": "里山", "_at_ms": 1793115799067} +{"v": 2, "seq": 41, "kind": "interim", "role": "user", "text": "势险", "_at_ms": 1793115799657} +{"v": 2, "seq": 42, "kind": "interim", "role": "user", "text": "峻", "_at_ms": 1793115800262} +{"v": 2, "seq": 43, "kind": "interim", "role": "user", "text": "只怕藏", "_at_ms": 1793115800858} +{"v": 2, "seq": 44, "kind": "interim", "role": "user", "text": "有妖", "_at_ms": 1793115801462} +{"v": 2, "seq": 45, "kind": "interim", "role": "user", "text": "精妖", "_at_ms": 1793115802053} +{"v": 2, "seq": 46, "kind": "interim", "role": "user", "text": "怪", "_at_ms": 1793115802654} +{"v": 2, "seq": 47, "kind": "interim", "role": "user", "text": "哪里", "_at_ms": 1793115803271} +{"v": 2, "seq": 48, "kind": "interim", "role": "user", "text": "逃", "_at_ms": 1793115803858} +{"v": 2, "seq": 49, "kind": "interim", "role": "user", "text": "吃完老", "_at_ms": 1793115804464} +{"v": 2, "seq": 50, "kind": "interim", "role": "user", "text": "我所", "_at_ms": 1793115805066} +{"v": 2, "seq": 51, "kind": "interim", "role": "user", "text": "一", "_at_ms": 1793115805654} +{"v": 2, "seq": 52, "kind": "user_final", "role": "user", "text": "俺老孙来也,俺乃500年前大闹天宫的齐天大圣孙悟空,这里山势险峻,只怕藏有妖精妖怪,哪里逃吃?俺老孙吧", "_at_ms": 1793115807317} +{"v": 2, "seq": 53, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_d1c8f4816b10", "_at_ms": 1793115810983} +{"v": 2, "seq": 54, "kind": "assistant_interim", "role": "assistant", "text": "好", "speech_id": "speech_d1c8f4816b10", "_at_ms": 1793115812231} +{"v": 2, "seq": 55, "kind": "assistant_final", "role": "assistant", "text": "好", "speech_id": "speech_d1c8f4816b10", "turn_id": "16006868-5ee7-49cc-b9c9-1a8dcfe74d08", "_at_ms": 1793115816208} +{"v": 2, "seq": 56, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_d1c8f4816b10", "turn_id": "16006868-5ee7-49cc-b9c9-1a8dcfe74d08", "_at_ms": 1793115816213} +{"v": 2, "seq": 57, "kind": "interim", "role": "user", "text": "没", "_at_ms": 1793115893204} +{"v": 2, "seq": 58, "kind": "interim", "role": "user", "text": "错我", "_at_ms": 1793115893809} +{"v": 2, "seq": 59, "kind": "interim", "role": "user", "text": "就是个笨", "_at_ms": 1793115894415} +{"v": 2, "seq": 60, "kind": "interim", "role": "user", "text": "蛋", "_at_ms": 1793115895004} +{"v": 2, "seq": 61, "kind": "interim", "role": "user", "text": "派大", "_at_ms": 1793115895614} +{"v": 2, "seq": 62, "kind": "interim", "role": "user", "text": "星我们一", "_at_ms": 1793115896206} +{"v": 2, "seq": 63, "kind": "interim", "role": "user", "text": "起去抓", "_at_ms": 1793115896811} +{"v": 2, "seq": 64, "kind": "interim", "role": "user", "text": "水母", "_at_ms": 1793115897406} +{"v": 2, "seq": 65, "kind": "interim", "role": "user", "text": "吧我们", "_at_ms": 1793115898018} +{"v": 2, "seq": 66, "kind": "interim", "role": "user", "text": "一起度", "_at_ms": 1793115898608} +{"v": 2, "seq": 67, "kind": "interim", "role": "user", "text": "过快乐的一", "_at_ms": 1793115899211} +{"v": 2, "seq": 68, "kind": "user_final", "role": "user", "text": "没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天", "_at_ms": 1793115901261} +{"v": 2, "seq": 69, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_8ac7cd94c5db", "_at_ms": 1793115901269} +{"v": 2, "seq": 70, "kind": "assistant_interim", "role": "assistant", "text": "好", "speech_id": "speech_8ac7cd94c5db", "_at_ms": 1793115902769} +{"v": 2, "seq": 71, "kind": "assistant_final", "role": "assistant", "text": "好", "speech_id": "speech_8ac7cd94c5db", "turn_id": "fdf28a03-19a8-4fcb-ba8c-24587736c3f0", "_at_ms": 1793115908179} +{"v": 2, "seq": 72, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_8ac7cd94c5db", "turn_id": "fdf28a03-19a8-4fcb-ba8c-24587736c3f0", "_at_ms": 1793115908181} +{"v": 2, "seq": 73, "kind": "interim", "role": "user", "text": "你是不是", "_at_ms": 1793115991657} +{"v": 2, "seq": 74, "kind": "interim", "role": "user", "text": "早就想", "_at_ms": 1793115992268} +{"v": 2, "seq": 75, "kind": "interim", "role": "user", "text": "睡我", "_at_ms": 1793115992857} +{"v": 2, "seq": 76, "kind": "interim", "role": "user", "text": "了", "_at_ms": 1793115993462} +{"v": 2, "seq": 77, "kind": "user_final", "role": "user", "text": "你是不是早就想睡我了呀", "_at_ms": 1793115994404} +{"v": 2, "seq": 78, "kind": "turn_start", "role": "assistant", "text": "", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994412} +{"v": 2, "seq": 79, "kind": "assistant_interim", "role": "assistant", "text": "Considering", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994844} +{"v": 2, "seq": 80, "kind": "assistant_interim", "role": "assistant", "text": "Considering the", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994878} +{"v": 2, "seq": 81, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994911} +{"v": 2, "seq": 82, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994944} +{"v": 2, "seq": 83, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115994977} +{"v": 2, "seq": 84, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995010} +{"v": 2, "seq": 85, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995044} +{"v": 2, "seq": 86, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user,", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995077} +{"v": 2, "seq": 87, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995111} +{"v": 2, "seq": 88, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995144} +{"v": 2, "seq": 89, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995178} +{"v": 2, "seq": 90, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995212} +{"v": 2, "seq": 91, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995245} +{"v": 2, "seq": 92, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995278} +{"v": 2, "seq": 93, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995312} +{"v": 2, "seq": 94, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995345} +{"v": 2, "seq": 95, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995378} +{"v": 2, "seq": 96, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995412} +{"v": 2, "seq": 97, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995445} +{"v": 2, "seq": 98, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995478} +{"v": 2, "seq": 99, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now.", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995511} +{"v": 2, "seq": 100, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995549} +{"v": 2, "seq": 101, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995583} +{"v": 2, "seq": 102, "kind": "assistant_interim", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n\n\n", "speech_id": "speech_9aa32dc387c3", "_at_ms": 1793115995614} +{"v": 2, "seq": 103, "kind": "assistant_final", "role": "assistant", "text": "Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n\n\n", "speech_id": "speech_9aa32dc387c3", "turn_id": "623bab59-0edc-4751-9feb-a9a7e4a11233", "_at_ms": 1793116005934} +{"v": 2, "seq": 104, "kind": "turn_end", "role": "assistant", "text": "", "speech_id": "speech_9aa32dc387c3", "turn_id": "623bab59-0edc-4751-9feb-a9a7e4a11233", "_at_ms": 1793116005935} diff --git a/docs/rework/hel879/r5-window/evidence-window.txt b/docs/rework/hel879/r5-window/evidence-window.txt new file mode 100644 index 0000000..fea49b4 --- /dev/null +++ b/docs/rework/hel879/r5-window/evidence-window.txt @@ -0,0 +1,2 @@ +WINDOW_START=2026-10-27T15:26:54Z +WINDOW_END=2026-10-27T15:58:10Z diff --git a/docs/rework/hel879/r5-window/worker.log b/docs/rework/hel879/r5-window/worker.log new file mode 100644 index 0000000..25fcb17 --- /dev/null +++ b/docs/rework/hel879/r5-window/worker.log @@ -0,0 +1,409 @@ +{"message": "starting worker", "level": "INFO", "name": "livekit.agents", "version": "1.8.8", "rtc-version": "1.1.20", "pid": 8800, "timestamp": "2026-10-27T15:21:02.628498+00:00"} +{"message": "plugin registered", "level": "INFO", "name": "livekit.agents", "plugin": "livekit.plugins.silero", "version": "1.8.8", "pid": 8800, "timestamp": "2026-10-27T15:21:02.628622+00:00"} +{"message": "preloading plugins", "level": "INFO", "name": "livekit.agents", "packages": ["livekit.plugins.silero", "livekit.agents.ipc._preload", "livekit.agents.ipc._preload_freeze"], "pid": 8800, "timestamp": "2026-10-27T15:21:02.628690+00:00"} +{"message": "HTTP server listening on :8082", "level": "INFO", "name": "livekit.agents", "pid": 8800, "timestamp": "2026-10-27T15:21:02.632394+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 8879, "timestamp": "2026-10-27T15:21:03.868641+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 8881, "timestamp": "2026-10-27T15:21:03.871820+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 8883, "timestamp": "2026-10-27T15:21:03.874482+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 8885, "timestamp": "2026-10-27T15:21:03.877329+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 8885, "elapsed_time": 0.14, "timestamp": "2026-10-27T15:21:04.013188+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 8883, "elapsed_time": 0.14, "timestamp": "2026-10-27T15:21:04.013380+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 8879, "elapsed_time": 0.14, "timestamp": "2026-10-27T15:21:04.013613+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 8881, "elapsed_time": 0.14, "timestamp": "2026-10-27T15:21:04.016022+00:00"} +/opt/voice-agent/agent/venv/lib/python3.12/site-packages/jwt/api_jwt.py:149: InsecureKeyLengthWarning: The HMAC key is 28 bytes long, which is below the minimum recommended length of 32 bytes for SHA256. See RFC 7518 Section 3.2. + return self._jws.encode( +{"message": "registered worker", "level": "INFO", "name": "livekit.agents", "agent_name": "cybergril-voice-agent", "deployment": "", "id": "AW_LGViCVagSNNu", "url": "ws://127.0.0.1:7880", "region": "", "protocol": 17, "pid": 8800, "timestamp": "2026-10-27T15:21:04.039801+00:00"} +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_dcfbVLNh9cxc", "dispatch_id": "AD_pFPtM5kSSpZH", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:28:57.444695+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 10801, "timestamp": "2026-10-27T15:28:57.518855+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 10801, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:28:57.639650+00:00"} +Nvidia Decoder is supported. +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:28:57.893684+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:28:57.895149+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_FBBgf5Si2JYd", "dispatch_id": "AD_iJTKQsucFwSK", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:32:32.446385+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 11093, "timestamp": "2026-10-27T15:32:32.549290+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 11093, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:32:32.665948+00:00"} +Nvidia Decoder is supported. +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:32:32.853330+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:32:32.855790+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_QhLhdpZKohT3", "dispatch_id": "AD_6Esm9sgx9sbT", "room": "cg-voice-74blnuc9c6", "room_id": "RM_q3vSy5gNpvMA", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:32:39.512686+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 11326, "timestamp": "2026-10-27T15:32:39.616162+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 11326, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:32:39.737803+00:00"} +Nvidia Decoder is supported. +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 8879, "job_id": "AJ_QhLhdpZKohT3", "room": "cg-voice-74blnuc9c6", "room_id": "RM_q3vSy5gNpvMA", "timestamp": "2026-10-27T15:32:39.969275+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 8879, "job_id": "AJ_QhLhdpZKohT3", "room": "cg-voice-74blnuc9c6", "room_id": "RM_q3vSy5gNpvMA", "timestamp": "2026-10-27T15:32:39.971493+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8879, "job_id": "AJ_QhLhdpZKohT3", "room": "cg-voice-74blnuc9c6", "room_id": "RM_q3vSy5gNpvMA", "timestamp": "2026-10-27T15:32:59.437282+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 8879, "job_id": "AJ_QhLhdpZKohT3", "room": "cg-voice-74blnuc9c6", "room_id": "RM_q3vSy5gNpvMA", "timestamp": "2026-10-27T15:33:01.444726+00:00"} +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_4vPjEVFaLPMq", "dispatch_id": "AD_MVu9k7fFLQyy", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:33:23.836716+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 11830, "timestamp": "2026-10-27T15:33:23.895593+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 11830, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:33:24.010796+00:00"} +Nvidia Decoder is supported. +{"message": "RoomIO linked to participant", "level": "INFO", "name": "livekit.agents", "lk.pii.participant_identity": "user-gwhvezcz", "lk.pii.room_name": "cg-voice-qeghdnxehn", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:33:24.256405+00:00"} +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:33:24.260750+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:33:24.261961+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +[stt] utterance open, frame sr=24000 +[stt] interim: 盯 +[stt->session] interim_transcript: 盯 +[stt] interim: 着你看怎 +[stt->session] interim_transcript: 着你看怎 +[stt] interim: 么啦 +[stt->session] interim_transcript: 么啦 +[stt] FINAL(merged): 盯着你看怎么啦 +[stt->session] final_transcript: 盯着你看怎么啦 +[stt] interim: 难得 +[stt->session] interim_transcript: 难得 +[stt] interim: 看你这么 +[stt->session] interim_transcript: 看你这么 +[stt] interim: 有意思的 +[stt->session] interim_transcript: 有意思的 +[stt] interim: 样子 +[stt->session] interim_transcript: 样子 +[stt] FINAL(merged): 难得看你这么有意思的样子吗 +[stt->session] final_transcript: 难得看你这么有意思的样子吗 +{"message": "transcript arrives after turn has been committed. consider raising `min_delay` in the endpointing options to accommodate a slow stt. subsequent occurrences will log at debug level.", "level": "WARNING", "name": "livekit.agents", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:33:30.501478+00:00"} +[stt] interim: 烈会军校 +[stt->session] interim_transcript: 烈会军校 +[stt] interim: 校啊招 +[stt->session] interim_transcript: 校啊招 +[stt] interim: 啊这种类 +[stt->session] interim_transcript: 啊这种类 +[stt] interim: 型我都是 +[stt->session] interim_transcript: 型我都是 +[stt] interim: 那种比较 +[stt->session] interim_transcript: 那种比较 +[stt] interim: 呃豪 +[stt->session] interim_transcript: 呃豪 +[stt] interim: 爽比 +[stt->session] interim_transcript: 爽比 +[stt] interim: 较开 +[stt->session] interim_transcript: 较开 +[stt] interim: 朗的那种 +[stt->session] interim_transcript: 朗的那种 +[stt] interim: 类型 +[stt->session] interim_transcript: 类型 +[stt] interim: 阳光大 +[stt->session] interim_transcript: 阳光大 +[stt] interim: 女孩 +[stt->session] interim_transcript: 女孩 +[stt] interim: 那种感 +[stt->session] interim_transcript: 那种感 +[stt] interim: 觉 +[stt->session] interim_transcript: 觉 +[stt] interim: 都给 +[stt->session] interim_transcript: 都给 +[stt] FINAL(merged): 烈火军校啊,朝阳啊这种类型我都是那种比较呃豪爽,比较开朗的那种类型,阳光大女孩那种感觉就 +[stt->session] final_transcript: 烈火军校啊,朝阳啊这种类型我都是那种比较呃豪爽,比较开朗的那种类型,阳光大女孩那 +[tts] first audio after 3.335s (原来你是那种阳光大女孩的类型啊,难怪感觉...) +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:33:57.436181+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:33:57.437201+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:33:57.437449+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 8885, "job_id": "AJ_dcfbVLNh9cxc", "room": "probe-room-2", "room_id": "RM_mNZagsczYXao", "timestamp": "2026-10-27T15:33:59.447143+00:00"} +[stt] interim: 怎么还 +[stt->session] interim_transcript: 怎么还 +[stt] interim: 有没事 +[stt->session] interim_transcript: 有没事 +[stt] interim: 的 +[stt->session] interim_transcript: 的 +[stt] FINAL(merged): 怎么还有没事的 +[stt->session] final_transcript: 怎么还有没事的 +[stt] interim: 是我不够 +[stt->session] interim_transcript: 是我不够 +[stt] interim: 努力 +[stt->session] interim_transcript: 努力 +[stt] FINAL(merged): 是我不够努力吗 +[stt->session] final_transcript: 是我不够努力吗 +[stt] interim: 盯着你看 +[stt->session] interim_transcript: 盯着你看 +[stt] interim: 怎么 +[stt->session] interim_transcript: 怎么 +[stt] FINAL(merged): 盯着你看怎么啦 +[stt->session] final_transcript: 盯着你看怎么啦 +[stt] interim: 难得 +[stt->session] interim_transcript: 难得 +[stt] interim: 看你这么 +[stt->session] interim_transcript: 看你这么 +[stt] interim: 有意思的 +[stt->session] interim_transcript: 有意思的 +[stt] interim: 样子 +[stt->session] interim_transcript: 样子 +{"message": "closing agent session due to participant disconnect (disable via `RoomInputOptions.close_on_disconnect=False`)", "level": "INFO", "name": "livekit.agents", "room": "cg-voice-qeghdnxehn", "participant": "user-gwhvezcz", "reason": "ROOM_DELETED", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.027601+00:00"} +{"message": "skipping user input, speech scheduling is paused", "level": "WARNING", "name": "livekit.agents", "lk.pii.user_input": "盯着你看怎么啦", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.040356+00:00"} +{"message": "failed to send session event\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "level": "WARNING", "name": "livekit.agents", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.042202+00:00"} +[stt] FINAL(merged): 难得看你这么有意思的样子吗 +[stt->session] final_transcript: 难得看你这么有意思的样子吗 +{"message": "failed to send session event\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "level": "WARNING", "name": "livekit.agents", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.890538+00:00"} +{"message": "skipping user input, speech scheduling is paused", "level": "WARNING", "name": "livekit.agents", "lk.pii.user_input": "难得看你这么有意思的样子吗", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.891932+00:00"} +{"message": "failed to send session event\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "level": "WARNING", "name": "livekit.agents", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 477, in _write_loop\n await self._transport.send_message(msg)\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/agents/voice/remote_session.py\", line 153, in send_message\n raise RuntimeError(\"room session transport is closed\")\nRuntimeError: room session transport is closed", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.892740+00:00"} +{"message": "Task exception was never retrieved\nfuture: exception=PublishDataError('engine: connection error: engine is closed')>\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/participant.py\", line 359, in publish_data\n raise PublishDataError(cb.publish_data.error)\nlivekit.rtc.participant.PublishDataError: engine: connection error: engine is closed", "level": "ERROR", "name": "asyncio", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/participant.py\", line 359, in publish_data\n raise PublishDataError(cb.publish_data.error)\nlivekit.rtc.participant.PublishDataError: engine: connection error: engine is closed", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.894879+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 8881, "job_id": "AJ_4vPjEVFaLPMq", "room": "cg-voice-qeghdnxehn", "room_id": "RM_EMAorR7C2evE", "timestamp": "2026-10-27T15:34:12.911612+00:00"} +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_HpxBoFkqfBdm", "dispatch_id": "AD_QPVzQMzStdbx", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:37:07.605078+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 12418, "timestamp": "2026-10-27T15:37:07.631804+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 12418, "elapsed_time": 0.11, "timestamp": "2026-10-27T15:37:07.743701+00:00"} +Nvidia Decoder is supported. +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:37:08.025546+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:37:08.026426+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_MLg4JBtoku27", "dispatch_id": "AD_uMQVHwFMfKqf", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:37:11.445956+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 12620, "timestamp": "2026-10-27T15:37:11.462186+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 12620, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:37:11.583631+00:00"} +Nvidia Decoder is supported. +{"message": "RoomIO linked to participant", "level": "INFO", "name": "livekit.agents", "lk.pii.participant_identity": "user-vdywshve", "lk.pii.room_name": "cg-voice-5c4y64wxkl", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:37:11.840352+00:00"} +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:37:11.842501+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:37:11.844023+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +[stt] utterance open, frame sr=24000 +[stt] interim: 没 +[stt->session] interim_transcript: 没 +[stt] interim: 错 +[stt->session] interim_transcript: 错 +[stt] interim: 我就是个 +[stt->session] interim_transcript: 我就是个 +[stt] interim: 笨蛋 +[stt->session] interim_transcript: 笨蛋 +[stt] interim: 派大星 +[stt->session] interim_transcript: 派大星 +[stt] interim: 我们 +[stt->session] interim_transcript: 我们 +[stt] interim: 一起去 +[stt->session] interim_transcript: 一起去 +[stt] interim: 抓水母 +[stt->session] interim_transcript: 抓水母 +[stt] interim: 吧我 +[stt->session] interim_transcript: 吧我 +[stt] interim: 们一起度 +[stt->session] interim_transcript: 们一起度 +[stt] interim: 过快乐的 +[stt->session] interim_transcript: 过快乐的 +[stt] interim: 一天 +[stt->session] interim_transcript: 一天 +[stt] FINAL(merged): 没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天 +[stt->session] final_transcript: 没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天 +[tts] first audio after 3.909s (好呀,那我们这就出发去抓水母!不过我可不...) +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:37:32.436074+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:37:32.436405+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:37:32.436558+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "room disconnected", "pid": 8883, "job_id": "AJ_FBBgf5Si2JYd", "room": "probe-room-3", "room_id": "RM_msdxb3QWFMYf", "timestamp": "2026-10-27T15:37:34.449378+00:00"} +[stt] interim: 天雷滚 +[stt->session] interim_transcript: 天雷滚 +[stt] interim: 滚我好 +[stt->session] interim_transcript: 滚我好 +[stt] interim: 怕怕 +[stt->session] interim_transcript: 怕怕 +[stt] interim: 劈 +[stt->session] interim_transcript: 劈 +[stt] interim: 的我浑 +[stt->session] interim_transcript: 的我浑 +[stt] interim: 身掉渣 +[stt->session] interim_transcript: 身掉渣 +[stt] interim: 渣突 +[stt->session] interim_transcript: 渣突 +[stt] interim: 破天 +[stt->session] interim_transcript: 破天 +[stt] interim: 劫我笑 +[stt->session] interim_transcript: 劫我笑 +[stt] interim: 哈哈 +[stt->session] interim_transcript: 哈哈 +[stt] interim: 逆 +[stt->session] interim_transcript: 逆 +[stt] interim: 天改命 +[stt->session] interim_transcript: 天改命 +[stt] interim: 我吹喇 +[stt->session] interim_transcript: 我吹喇 +[stt] interim: 叭滴 +[stt->session] interim_transcript: 叭滴 +[stt] interim: 答滴答 +[stt->session] interim_transcript: 答滴答 +[stt] interim: 滴滴 +[stt->session] interim_transcript: 滴滴 +[stt] FINAL(merged): 天雷滚滚,我好怕怕劈的我浑身掉渣渣,突破天劫,我笑哈哈逆天改命,我吹喇叭滴答滴答,滴滴答 +[stt->session] final_transcript: 天雷滚滚,我好怕怕劈的我浑身掉渣渣,突破天劫,我笑哈哈逆天改命,我吹喇叭滴答滴答 +[tts] first audio after 3.008s (哈哈,你唱得还挺有节奏感的!...) +[stt] interim: 你是不是 +[stt->session] interim_transcript: 你是不是 +[stt] interim: 早就想 +[stt->session] interim_transcript: 早就想 +[stt] interim: 睡我 +[stt->session] interim_transcript: 睡我 +[stt] interim: 了 +[stt->session] interim_transcript: 了 +[stt] FINAL(merged): 你是不是早就想睡我了呀 +[stt->session] final_transcript: 你是不是早就想睡我了呀 +[tts] first audio after 3.304s (哈哈,你思维跳跃得太快了,我都跟不上。...) +[stt] interim: 盯着你看 +[stt->session] interim_transcript: 盯着你看 +[stt] interim: 怎么啦 +[stt->session] interim_transcript: 怎么啦 +[stt] FINAL(merged): 盯着你看怎么啦 +[stt->session] final_transcript: 盯着你看怎么啦 +[tts] first audio after 2.405s (没...) +{"message": "closing agent session due to participant disconnect (disable via `RoomInputOptions.close_on_disconnect=False`)", "level": "INFO", "name": "livekit.agents", "room": "cg-voice-5c4y64wxkl", "participant": "user-vdywshve", "reason": "ROOM_DELETED", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:38:32.340079+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:38:32.351146+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:38:32.352194+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:38:32.353282+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 11093, "job_id": "AJ_MLg4JBtoku27", "room": "cg-voice-5c4y64wxkl", "room_id": "RM_YDaqgLrwdfsB", "timestamp": "2026-10-27T15:38:34.358814+00:00"} +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_a8uXggcKPamC", "dispatch_id": "AD_fxrSDuKoYdTr", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:41:10.456804+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 13261, "timestamp": "2026-10-27T15:41:10.492898+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 13261, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:41:10.613693+00:00"} +Nvidia Decoder is supported. +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:10.904892+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:10.905825+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:30.437268+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:30.437449+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:30.437534+00:00"} +{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_sfYen6Augi9u", "dispatch_id": "AD_UmNB9yQbmDEK", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "agent_name": "cybergril-voice-agent", "resuming": false, "enable_recording": false, "pid": 8800, "timestamp": "2026-10-27T15:41:30.448381+00:00"} +{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 13614, "timestamp": "2026-10-27T15:41:30.549122+00:00"} +{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 13614, "elapsed_time": 0.12, "timestamp": "2026-10-27T15:41:30.671152+00:00"} +Nvidia Decoder is supported. +{"message": "RoomIO linked to participant", "level": "INFO", "name": "livekit.agents", "lk.pii.participant_identity": "user-uqfyuhnt", "lk.pii.room_name": "cg-voice-oleganioz7", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:41:30.841632+00:00"} +{"message": "adaptive interruption is disabled by default in production mode", "level": "INFO", "name": "livekit.agents", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:41:30.843597+00:00"} +{"message": "failed to emit event conversation_item_added\nTraceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "level": "ERROR", "name": "livekit", "exc_info": "Traceback (most recent call last):\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/livekit/rtc/event_emitter.py\", line 70, in emit\n callback(*callback_args)\n File \"/opt/voice-agent-cybergril/cybergril/voice/worker.py\", line 193, in _on_item\n if ev.item.role == \"assistant\":\n ^^^^^^^^^^^^\n File \"/opt/voice-agent/agent/venv/lib/python3.12/site-packages/pydantic/main.py\", line 1105, in __getattr__\n raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')\nAttributeError: 'AgentHandoff' object has no attribute 'role'", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:41:30.844442+00:00"} +[cybergril-worker] session started (api=http://127.0.0.1:8360, voice=female_soft, agent_name=cybergril-voice-agent) +[stt] utterance open, frame sr=24000 +[stt] interim: 师 +[stt->session] interim_transcript: 师 +[stt] interim: 傅 +[stt->session] interim_transcript: 傅 +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 11326, "job_id": "AJ_a8uXggcKPamC", "room": "cg-voice-6d0grxxwno", "room_id": "RM_bHsWETGZc5cQ", "timestamp": "2026-10-27T15:41:32.444706+00:00"} +[stt] interim: 哎这 +[stt->session] interim_transcript: 哎这 +[stt] interim: 取经 +[stt->session] interim_transcript: 取经 +[stt] interim: 路比 +[stt->session] interim_transcript: 路比 +[stt] interim: 高老 +[stt->session] interim_transcript: 高老 +[stt] interim: 庄种 +[stt->session] interim_transcript: 庄种 +[stt] interim: 地还 +[stt->session] interim_transcript: 地还 +[stt] interim: 苦俺 +[stt->session] interim_transcript: 苦俺 +[stt] interim: 老中 +[stt->session] interim_transcript: 老中 +[stt] interim: 饿的能 +[stt->session] interim_transcript: 饿的能 +[stt] interim: 吞下 +[stt->session] interim_transcript: 吞下 +[stt] interim: 石龙 +[stt->session] interim_transcript: 石龙 +[stt] interim: 素包 +[stt->session] interim_transcript: 素包 +[stt] interim: 子 +[stt->session] interim_transcript: 子 +[stt] interim: 猴哥 +[stt->session] interim_transcript: 猴哥 +[stt] interim: 你筋 +[stt->session] interim_transcript: 你筋 +[stt] interim: 斗云 +[stt->session] interim_transcript: 斗云 +[stt] interim: 快 +[stt->session] interim_transcript: 快 +[stt] interim: 要不 +[stt->session] interim_transcript: 要不 +[stt] interim: 先 +[stt->session] interim_transcript: 先 +[stt] interim: 去画个 +[stt->session] interim_transcript: 去画个 +[stt] FINAL(merged): 师傅,哎,这取经路比高,老庄种地还苦,俺老猪饿的能吞下石龙素包子,猴哥,你筋斗云块,要不先去化个缘 +[stt->session] final_transcript: 师傅,哎,这取经路比高,老庄种地还苦,俺老猪饿的能吞下石龙素包子,猴哥,你筋斗云 +[tts] first audio after 2.480s (师傅,...) +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_reliable' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:42:07.442254+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_lossy' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:42:07.442585+00:00"} +{"message": "livekit::rtc_engine::rtc_session:704:livekit::rtc_engine::rtc_session - publisher data channel '_data_track' closed unexpectedly", "level": "ERROR", "name": "livekit", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:42:07.442726+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 10801, "job_id": "AJ_HpxBoFkqfBdm", "room": "cg-voice-ymtapbx42a", "room_id": "RM_5BfQDxZuTsu4", "timestamp": "2026-10-27T15:42:09.450806+00:00"} +[stt] interim: 俺老 +[stt->session] interim_transcript: 俺老 +[stt] interim: 孙来 +[stt->session] interim_transcript: 孙来 +[stt] interim: 也俺 +[stt->session] interim_transcript: 也俺 +[stt] interim: 乃五百 +[stt->session] interim_transcript: 乃五百 +[stt] interim: 年前 +[stt->session] interim_transcript: 年前 +[stt] interim: 大闹 +[stt->session] interim_transcript: 大闹 +[stt] interim: 天宫的 +[stt->session] interim_transcript: 天宫的 +[stt] interim: 齐天 +[stt->session] interim_transcript: 齐天 +[stt] interim: 霸圣 +[stt->session] interim_transcript: 霸圣 +[stt] interim: 孙悟 +[stt->session] interim_transcript: 孙悟 +[stt] interim: 空这 +[stt->session] interim_transcript: 空这 +[stt] interim: 里山 +[stt->session] interim_transcript: 里山 +[stt] interim: 势险 +[stt->session] interim_transcript: 势险 +[stt] interim: 峻 +[stt->session] interim_transcript: 峻 +[stt] interim: 只怕藏 +[stt->session] interim_transcript: 只怕藏 +[stt] interim: 有妖 +[stt->session] interim_transcript: 有妖 +[stt] interim: 精妖 +[stt->session] interim_transcript: 精妖 +[stt] interim: 怪 +[stt->session] interim_transcript: 怪 +[stt] interim: 哪里 +[stt->session] interim_transcript: 哪里 +[stt] interim: 逃 +[stt->session] interim_transcript: 逃 +[stt] interim: 吃完老 +[stt->session] interim_transcript: 吃完老 +[stt] interim: 我所 +[stt->session] interim_transcript: 我所 +[stt] interim: 一 +[stt->session] interim_transcript: 一 +[stt] FINAL(merged): 俺老孙来也,俺乃500年前大闹天宫的齐天大圣孙悟空,这里山势险峻,只怕藏有妖精妖怪,哪里逃吃?俺老孙吧 +[stt->session] final_transcript: 俺老孙来也,俺乃500年前大闹天宫的齐天大圣孙悟空,这里山势险峻,只怕藏有妖精妖 +[tts] first audio after 2.608s (好...) +[stt] interim: 没 +[stt->session] interim_transcript: 没 +[stt] interim: 错我 +[stt->session] interim_transcript: 错我 +[stt] interim: 就是个笨 +[stt->session] interim_transcript: 就是个笨 +[stt] interim: 蛋 +[stt->session] interim_transcript: 蛋 +[stt] interim: 派大 +[stt->session] interim_transcript: 派大 +[stt] interim: 星我们一 +[stt->session] interim_transcript: 星我们一 +[stt] interim: 起去抓 +[stt->session] interim_transcript: 起去抓 +[stt] interim: 水母 +[stt->session] interim_transcript: 水母 +[stt] interim: 吧我们 +[stt->session] interim_transcript: 吧我们 +[stt] interim: 一起度 +[stt->session] interim_transcript: 一起度 +[stt] interim: 过快乐的一 +[stt->session] interim_transcript: 过快乐的一 +[stt] FINAL(merged): 没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天 +[stt->session] final_transcript: 没错,我就是个笨蛋派大星,我们一起去抓水母吧,我们一起度过快乐的一天 +[tts] first audio after 2.300s (好...) +[stt] interim: 你是不是 +[stt->session] interim_transcript: 你是不是 +[stt] interim: 早就想 +[stt->session] interim_transcript: 早就想 +[stt] interim: 睡我 +[stt->session] interim_transcript: 睡我 +[stt] interim: 了 +[stt->session] interim_transcript: 了 +[stt] FINAL(merged): 你是不是早就想睡我了呀 +[stt->session] final_transcript: 你是不是早就想睡我了呀 +{"message": "failed to generate LLM completion: chat stream ended without final or error (caused by ChatApiConnectionError: chat stream ended without final or error), retrying in 0.1s", "level": "WARNING", "name": "livekit.agents", "llm": "cybergril.voice.llm_chat_api.CyberGrillLLM", "attempt": 1, "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:46:34.451048+00:00"} +[tts] first audio after 3.847s (Considering the limi...) +[tts] first audio after 2.284s (...) +{"message": "closing agent session due to participant disconnect (disable via `RoomInputOptions.close_on_disconnect=False`)", "level": "INFO", "name": "livekit.agents", "room": "cg-voice-oleganioz7", "participant": "user-uqfyuhnt", "reason": "CLIENT_INITIATED", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:46:46.019622+00:00"} +{"message": "process exiting", "level": "INFO", "name": "livekit.agents", "reason": "parent process shutdown", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:47:06.441875+00:00"} +{"message": "livekit_ffi::server:279:livekit_ffi::server - Attempted to drop unknown FFI handle: 16", "level": "WARNING", "name": "livekit", "pid": 11830, "job_id": "AJ_sfYen6Augi9u", "room": "cg-voice-oleganioz7", "room_id": "RM_anqqGcUZrCbe", "timestamp": "2026-10-27T15:47:06.446226+00:00"} +{"message": "draining worker", "level": "INFO", "name": "livekit.agents", "id": "AW_LGViCVagSNNu", "timeout": 3600, "pid": 8800, "timestamp": "2026-10-27T15:58:10.576647+00:00"} +{"message": "shutting down worker", "level": "INFO", "name": "livekit.agents", "id": "AW_LGViCVagSNNu", "pid": 8800, "timestamp": "2026-10-27T15:58:10.577287+00:00"} diff --git a/src/cybergril/voice/livekit_wire.py b/src/cybergril/voice/livekit_wire.py index 97cbeca..9b1c9ba 100644 --- a/src/cybergril/voice/livekit_wire.py +++ b/src/cybergril/voice/livekit_wire.py @@ -82,12 +82,26 @@ def mint_token( def mint_server_token(api_key: str, api_secret: str, ttl_seconds: float = 60.0, - now: float | None = None) -> str: - """Server-API bearer token; matches Service._auth_header() with no grants.""" + now: float | None = None, *, + video: dict[str, Any] | None = None) -> str: + """Server-API bearer token; claims mirror Service._auth_header(). + + ``video`` grants are required per endpoint exactly like the installed SDK + signs them (R5 real-machine finding 2026-10-27: a grantless bearer is + rejected 401 by CreateDispatch — the SDK sends VideoGrants(room_admin=True, + room=...) there and VideoGrants(room_create=True) for DeleteRoom). + """ now = time.time() if now is None else now + video_claims: dict[str, Any] = {} + if video: + video_claims = {_camel(k): v for k, v in video.items() if v is not None and v != ""} + claims: dict[str, Any] = {"iss": api_key, "sub": "", + "nbf": int(now), "exp": int(now + ttl_seconds)} + if video_claims: + claims["video"] = video_claims return _jwt_hs256( api_secret, - {"iss": api_key, "sub": "", "nbf": int(now), "exp": int(now + ttl_seconds)}, + claims, ) @@ -125,18 +139,20 @@ async def create_dispatch( ) -> DispatchResult: """Twirp AgentDispatchService/CreateDispatch with an explicit agent_name. - Body is livekit.CreateAgentDispatchRequest: field 1 agent_name, field 2 room, - field 3 metadata (field numbers verified against the installed protocol pkg). + Sent as Twirp JSON (content-type application/json): the server answers + JSON, so the dispatch id parses without a protobuf dependency (R5 + real-machine finding: protobuf content-type returns a binary body the + JSON parser cannot read). """ - body = _proto_string(1, agent_name) + _proto_string(2, room) + payload: dict[str, Any] = {"agentName": agent_name, "room": room} if metadata: - body += _proto_string(3, metadata) + payload["metadata"] = metadata response = await http.post( f"{internal_url.rstrip('/')}{TWIRP_PREFIX}/{DISPATCH_SERVICE}/CreateDispatch", - content=body, + json=payload, headers={ - "authorization": f"Bearer {mint_server_token(api_key, api_secret)}", - "content-type": "application/protobuf", + # SDK parity: VideoGrants(room_admin=True, room=...) + "authorization": f"Bearer {mint_server_token(api_key, api_secret, video={'room_admin': True, 'room': room})}", }, timeout=timeout, ) @@ -184,10 +200,10 @@ async def delete_room( """ response = await http.post( f"{internal_url.rstrip('/')}{TWIRP_PREFIX}/{ROOM_SERVICE}/DeleteRoom", - content=_proto_string(1, room), + json={"room": room}, headers={ - "authorization": f"Bearer {mint_server_token(api_key, api_secret)}", - "content-type": "application/protobuf", + # SDK parity: VideoGrants(room_create=True) + "authorization": f"Bearer {mint_server_token(api_key, api_secret, video={'room_create': True})}", }, timeout=timeout, ) diff --git a/tests/test_voice_rework_hel879.py b/tests/test_voice_rework_hel879.py index 2e945d2..adc34cd 100644 --- a/tests/test_voice_rework_hel879.py +++ b/tests/test_voice_rework_hel879.py @@ -177,7 +177,7 @@ class StandinLiveKit: if isinstance(error, int): return httpx.Response(error, text="room service error") raise error - self.deleted.append(_proto_field1(request.content)) + self.deleted.append(json.loads(request.content or b"{}").get("room", "")) return httpx.Response(200, content=b"") return httpx.Response(404, text="unknown") diff --git a/tests/test_voice_session.py b/tests/test_voice_session.py index 70ddf15..10deffd 100644 --- a/tests/test_voice_session.py +++ b/tests/test_voice_session.py @@ -44,16 +44,18 @@ class FakeLiveKit: return httpx.Response(500, text="livekit down") path = request.url.path if path == "/twirp/livekit.AgentDispatchService/CreateDispatch": - body = request.content - agent_name, room = _parse_dispatch(body) + body = json.loads(request.content or b"{}") + agent_name = body.get("agentName", "") + room = body.get("room", "") self._dispatch_seq += 1 self.dispatches.append({"agent_name": agent_name, "room": room, - "proto": True, "content_type": + "json": True, "content_type": request.headers.get("content-type", "")}) return httpx.Response(200, json={"id": f"AD_{self._dispatch_seq}", "agentName": agent_name, "room": room}) if path == "/twirp/livekit.RoomService/DeleteRoom": - self.deleted_rooms.append(_parse_room(request.content)) + body = json.loads(request.content or b"{}") + self.deleted_rooms.append(body.get("room", "")) return httpx.Response(200, content=b"") return httpx.Response(404, text="unknown twirp route") @@ -174,8 +176,8 @@ def test_open_session_dispatches_dedicated_worker(instance): # dedicated dispatch, not the generic cascade-voice-agent assert livekit.dispatches[-1]["agent_name"] == "cybergril-voice-agent" assert livekit.dispatches[-1]["room"] == body["room"] - assert livekit.dispatches[-1]["proto"] is True - assert livekit.dispatches[-1]["content_type"] == "application/protobuf" + assert livekit.dispatches[-1]["json"] is True + assert livekit.dispatches[-1]["content_type"] == "application/json" assert body["url"] == "ws://voice.test:7880" assert body["agent_name"] == "cybergril-voice-agent" # no livekit key material leaks into the browser payload