HEL-908 R1/R2 rework (HEL-909 review): whole-turn reasoning gate + cg.interrupt data channel
R1 — reasoning separation truly closed. The streaming hold leaked wholesale
on real token arrival (review A1: 'private reasoning '*20 beyond the 128-char
bound, no sentence ender; review A2: a Chinese sentence ender released the
hold) AND the adapter consumed the stray closer, so the app-level turn split
could never recover — the previous 'persistence and receipt always clean'
claim was false and is retracted. Per the engineering ruling the adapter now
has two explicit endpoint modes (GOV-006, no heuristics, no new engine):
buffer_turn (TurnReasoningGate) emits NOTHING during generation, splits the
whole turn at stream end and only then delivers the clean visible text; it
is set explicitly for all tiers in config/models.yaml (first visible text
waits for the full generation; cancelled/timed-out turns emit nothing and
end exactly as before — recorded and test-pinned, low latency NOT claimed).
stream (default, backward compatible) keeps explicit-tag/field semantics and
passes stray closers through unmodified so the turn-level split keeps its
signal. Both review strings and the recorded id=12 stream replay clean at
every chunk size through the real adapter and the real app into chat.delta,
the archive and the receipt; normal zh/en preserved byte-for-byte; unclosed
explicit thinking still fails honestly.
R2 — an authoritative interrupt when the LLM already finalized and TTS is
still playing (review evidence C: zero interrupts, old audio played out).
The page's existing hush() entry (ChatPanel untouched) now also publishes
{"v":1} on the room's own reliable data channel topic cg.interrupt
(fire-and-forget: a dropped publish never blocks or breaks the text send);
the worker subscribes data_received, validates topic/sender/payload and
calls session.interrupt(force=True) — the SDK's own programmatic interrupt,
force confirmed against the installed 1.8.8 source (non-force raises
RuntimeError on allow_interruptions=False speech). Failures log loudly; the
listener is unloaded via add_shutdown_callback; the server-cancel path in
llm_chat_api also interrupts with force and logs failures.
Verification (offline, zero real requests): real-SDK harness 8/8 including a
real SpeechHandle with a genuinely pending playout wait released by the
cg.interrupt message after a clean chat.final (evidence-C shape) plus the
full validation matrix; .venv311 full suite 480 passed / 8 skipped (2
test_r3 failures pre-exist at 5625de6); web npm ci from the existing lock,
typecheck 0 errors, lint 0 errors + 1 pre-existing warning, vitest 148/148,
build ok; ruff clean; check_requirements vs origin/main and b8ec832 with the
corrected handoff.
Co-authored-by: multica-agent <github@multica.ai>
This commit is contained in:
@@ -23,6 +23,9 @@ tiers:
|
|||||||
api_key_env: null
|
api_key_env: null
|
||||||
timeout: 60.0
|
timeout: 60.0
|
||||||
temperature: 0.85
|
temperature: 0.85
|
||||||
|
# HEL-908/909:该端点会把隐式开头思考(无 <think>、模板预填)混入 content,
|
||||||
|
# 只能整轮缓冲后分离;首段可见文本因此等整轮生成完成(见 reasoning.py)。
|
||||||
|
reasoning: buffer_turn
|
||||||
|
|
||||||
# ---- 中档:可以指向同一个模型,但配更短的上下文预算 ----
|
# ---- 中档:可以指向同一个模型,但配更短的上下文预算 ----
|
||||||
# 省钱的杠杆通常不是换更小的模型,而是少喂上下文。
|
# 省钱的杠杆通常不是换更小的模型,而是少喂上下文。
|
||||||
@@ -33,6 +36,7 @@ tiers:
|
|||||||
api_key_env: null
|
api_key_env: null
|
||||||
timeout: 40.0
|
timeout: 40.0
|
||||||
temperature: 0.7
|
temperature: 0.7
|
||||||
|
reasoning: buffer_turn
|
||||||
|
|
||||||
# ---- 本地小模型:后台任务的主力 ----
|
# ---- 本地小模型:后台任务的主力 ----
|
||||||
# 没有本地服务时,把它指向 openai_compat 的远端小模型即可。
|
# 没有本地服务时,把它指向 openai_compat 的远端小模型即可。
|
||||||
@@ -43,6 +47,7 @@ tiers:
|
|||||||
api_key_env: null
|
api_key_env: null
|
||||||
timeout: 20.0
|
timeout: 20.0
|
||||||
temperature: 0.4
|
temperature: 0.4
|
||||||
|
reasoning: buffer_turn
|
||||||
|
|
||||||
# ---- 场景 → 档位 ----
|
# ---- 场景 → 档位 ----
|
||||||
# 改这里就能调成本结构,不用改代码。
|
# 改这里就能调成本结构,不用改代码。
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"baseline_version": "1.12",
|
"baseline_version": "1.12",
|
||||||
"read_commit": "5625de6606a607a9d4fd583fe8a52d93d6d2b4ea",
|
"read_commit": "5625de6606a607a9d4fd583fe8a52d93d6d2b4ea",
|
||||||
"task": "HEL-908:HEL-797 已授权定点返修,修复 HEL-907 窗口实证的两项缺陷。其一『她在播语音时网页发文字→文字轮被取消、无回复、旧语音继续约32秒』:实证根因是取消发生在首 delta 前(worker.log:108 的 SDK 重试警告在文字 POST 后 13ms),服务端取消收尾无终态帧,worker 适配器把它当连接错误,livekit-agents 1.8.8 LLMStream._main_task 静默重试同一 turn_id,重试重新激活已取消轮并反杀新文字轮、重新生成整段语音播完;原报告『_retry_on_chunk_sent 守卫从未生效』按总管只读核对与实测源码确认不成立(守卫仅在有输出后适用)。其二『Considering the limi…/</think> 进入字幕与 TTS』:端点把隐式开头(无 <think>、模板预填)的推理原文+stray </think> 混入 content(DB id=12 原文),适配器此前无任何 reasoning 分离。全部修复离线复现→验证,零真实请求、零部署、零依赖变更。",
|
"task": "HEL-908 两轮定点返修(第二轮为 HEL-909 审核不通过后的集中返修一次)。第一轮修复文字打断丢字/旧轮复活/旧语音32秒(chat.cancelled 终态帧+cancelled 轮拒绝重新激活+worker 499 不重试+SpeechHandle.interrupt 停播,HEL-909 复核通过)。第二轮按总管裁定修两个核心缺口:R1 思考分流改显式整轮缓冲模式(原流式 hold 在中文句号/超128字符形状下整段泄漏且 closer 被适配层提前删除、轮级 split 无法兜底,第一轮『落库与 receipt 始终干净』表述不实已删除);R2 补上 LLM 已 final、TTS 仍在播时的权威打断通道(网页既有 hush() 入口经房间数据通道发 cg.interrupt,worker 订阅校验后 session.interrupt(force=True))。全部离线、零真实请求、零部署、零新依赖。",
|
||||||
"requirement_ids": [
|
"requirement_ids": [
|
||||||
"CG-006",
|
"CG-006",
|
||||||
"CG-010",
|
"CG-010",
|
||||||
@@ -22,27 +22,27 @@
|
|||||||
"GOV-006",
|
"GOV-006",
|
||||||
"DEL-001"
|
"DEL-001"
|
||||||
],
|
],
|
||||||
"deliverable": "双侧最小修复+离线证据。服务端 src/cybergril/api/app.py:取消路径显式下发 chat.cancelled 终态帧(取消≠断网);chat_lock 拒绝 status=cancelled 的旧轮重新取得活动轮(迟到重试不再反杀新轮,幂等回 chat.cancelled;failed 轮仍可重试);落库/chat.final/receipt 前轮级 split_reasoning 兜底。worker:chat_api_client.py 识别 chat.cancelled 终态;llm_chat_api.py 以 SDK 自身 499 语义正常结束(不重试不抛出)并经 SpeechHandle.interrupt()(LiveKit 既有打断路径)停掉已缓存音频,server_cancelled 重入闸零网络短路;worker.py 零改动。思考分流:新增 providers/llm/reasoning.py(流式状态机+轮级分流,复用 reasoning_content 字段与 <think> 约定,不造推理引擎、不改提示词、不删英文),openai_compat.py 接入并把 reasoning_content 增量导入 LLMDelta.reasoning(contracts 新增诊断字段,绝不持久化/下发/播报)。测试:tests/test_rework_hel908.py(app 级取消语义与泄漏回放)、tests/test_reasoning_separation_hel908.py(37 例含记录流逐字回放)、tests/test_sdk_livekit_hel908.py(真实 SDK harness,pytest 可收集、agent venv 可独立运行);既有 test_voice_chat_adapter.py 一例更新到新终态语义、test_api.py 断连用例内容补句号(意图不变)。证据 docs/rework/hel908/(summary.md+tests-remote.log)。",
|
"deliverable": "同分支追加提交(agent/2/15e79a2458bb,第一轮 b8ec832 之上)。R1:providers/llm/reasoning.py 重做为双显式模式——TurnReasoningGate(buffer_turn,整轮缓冲零中途外发,流结束 split_reasoning 后一次性下发干净正文;取消/超时行为不变)与无启发式 ReasoningSeparator(stream 默认:仅滤显式 <think> 块跨chunk安全,stray closer 原样透传保住轮级兜底信号);openai_compat 显式 reasoning 模式参数;LLMConfig.reasoning 字面量校验+models.yaml 三档显式 buffer_turn(未配置=既有语义,向后兼容)。R2:web/src/voice/session.ts+livekit.ts 纯传输接线(VoiceRoom.interrupt(),既有 hush() 附带 publishData reliable topic=cg.interrupt,fire-and-forget 不阻塞文字发送,零视觉改动);worker.py bind_interrupt_channel(本房间 data_received 订阅,topic/发送者/载荷三重校验,session.interrupt(force=True),失败大声留痕,add_shutdown_callback 卸载);llm_chat_api._interrupt_owner 改 force=True+留痕。文档:docs/rework/hel908/summary.md 重写(删除不实表述、保留原始证据与纠偏、记录等待代价与边界),tests-remote-r2.log、web-r2.log、check-requirements.log。",
|
||||||
"acceptance": [
|
"acceptance": [
|
||||||
"问题1离线复现→修复:ScriptedLLM 首delta前挂起模拟记录时序——文字 POST 后被取消的语音流收到 chat.cancelled 终态干净结束(不再异常);迟到同 turn_id 重试被拒绝且零额外模型调用;文字轮保留并获回复(user 行 sent+companion 行落库);语音轮无 phantom companion 行",
|
"R1 评审复现串逐字/小块经真实 OpenAICompatLLM→真实 app→持久化/receipt:'private reasoning '*20+'</think>回答。' 与 '我先想一下。内部思考</think>回答。' 的 chat.delta、落库行、chat.final、receipt text_hash 全部只含干净正文(buffer_turn 生成中零外发,修复前同输入已被本卡离线复现整段泄漏);记录流 id=12 按 1~1000 任意分块同样全干净;正常中英文正文逐字节保留;未闭合显式思考→空正文→诚实 chat.error;stream 模式显式块/推理字段/跨 chunk 语义保持,stray closer 透传可被 app 轮级 split 清理",
|
||||||
"首delta后取消(真实 socket uvicorn):已发 delta 保留、随后终态取消;真实 SDK harness 5/5——首delta前取消不重试(1次POST)、SpeechHandle.interrupt() 恰一次、取消后重入立即499零网络、真实连接错误仍按 SDK 有限重试(max_retry+1后诚实失败)、完成轮流式+receipt不变",
|
"R1 等待代价如实记录并测试固定:buffer_turn 首个可见正文等整轮生成完成(生成中零 chat.delta);取消轮照常 chat.cancelled 零泄漏、超时照常 chat.error;无 closer 无字段的推理不可识别边界已声明,不虚称全好",
|
||||||
"问题2离线复现→修复:DB id=12 原文(隐式开头推理+stray </think>)经真实 OpenAICompatLLM+真实 app 全链回放——chat.delta 拼接、落库行、chat.final、receipt text_hash 全部只绑干净可见回复;未闭合思考→空正文→诚实 chat.error(user 行 failed,无 phantom 回复);中英文正常正文逐字节保留(不以删英文掩盖)",
|
"R2 真实 SDK 证明 8/8(agent venv 1.8.8):真实 SpeechHandle(allow_interruptions=False 覆盖 force 语义)+真实 pending 播放等待(wait_if_not_interrupted)——完整轮 chat.final 干净结束后(评审 evidence C 形状)注入 cg.interrupt 数据消息,播放等待被真实 interrupt 释放、句柄 interrupted、force=True 断言成立;非 force 在该句柄上 RuntimeError 复核;校验矩阵(错 topic/服务端来源/自身/坏载荷/错版本)零触发;卸载后零触发;session 未运行大声失败不崩溃",
|
||||||
"既有韧性不破坏:已完成轮幂等重放仍零额外模型调用;failed 轮仍可重试;receipt 校验语义不变(专项用例)",
|
"R2 app 集成记录:完整语音轮后的文字轮为普通轮(无任何行被取消、无 phantom 行);web 侧 hush() 在 speaking/thinking 才发消息且本地静音反馈保持,下一回复恢复正常收听路径未动;ChatPanel/视觉零改动",
|
||||||
"规范环境全量:.venv311(requirements.lock 一致,只读复用)445 passed/5 skipped;2 个 test_r3 失败在施工基准 5625de6 上同样失败(预先存在,证据留存);ruff src/ tests/ 全绿;零真实请求、零部署、未触碰任何现网服务/端口/依赖/锁文件"
|
"回归:.venv311 全量 480 passed/8 skipped(2 个 test_r3 失败在基准 5625de6 同样失败,预先存在);第一轮 5 项 SDK 检查全保持(取消终态/不复活/不重试/真实连接错误仍有限重试/完成轮流式+receipt 不变);web npm ci(既有锁文件)typecheck 0 错、lint 0 错 1 既有警告、vitest 148/148、build 成功;ruff 全绿;models.yaml 新键经 ModelsConfig 实际解析"
|
||||||
],
|
],
|
||||||
"deferred": [
|
"deferred": [
|
||||||
"修复后真实端到端复测(真实房间、真人打断时延、真机 Safari/Chrome)须按授权窗口安排,本卡零真实请求边界内未做;停播证明是真实 SDK SpeechHandle.interrupt() 断言,不冒充端到端毫秒指标",
|
"真实端到端复测(真实房间/真人打断时延/真机/真实端点思考出现频率与长回复起播延迟)须按授权窗口安排;R2 离线证明是真实 SDK 句柄/等待语义+实装源码核对,不冒充端到端毫秒指标",
|
||||||
"思考分流流式残留:隐式开头且 </think> 落在128字符hold外的推理前缀仍会部分流出到字幕/TTS(流式下无法提前判定),落库与 receipt 始终干净;彻底消除需上游自行分流(reasoning_content 字段或服务端 reasoning parser),零真实请求下无法探测该端点能力——最小缺口如实声明",
|
"无 closer、无 reasoning 字段、无开头标签的推理任何客户端模式都不可识别,彻底解决需上游自行分流(字段或服务端 parser),零真实请求边界内无法探测——最小缺口如实声明",
|
||||||
"被新轮打断的可见界面状态(文字轮显示『被新对话取代』等)属界面核心,交视觉总监;本卡前端零改动(新帧 kind 被现有 ChatPanel 静默忽略,行为不差于现状)",
|
"chat.cancelled 的前端可见呈现与 ChatPanel 取消判定 limit 扩大(HEL-909 建议 #3/#4,非阻塞)未动:前者属界面核心交视觉总监,后者纯展示层既有行为",
|
||||||
"scripts/gen_hel877_sdk_check.py 在施工基准上即因 HEL-906 的 stt_funasr→asr_usage 导入而失效(harness 模块清单未随更新,base_exit=2 证据留存),交总管决定是否另行安排维护;本卡 SDK 验证由新 harness 覆盖",
|
"gen_hel877_sdk_check harness 在基准上即因 HEL-906 导入失效(前轮已留证),交总管决定维护安排;本卡 SDK 验证由 test_sdk_livekit_hel908.py 覆盖(8 项,agent venv 可独立复跑)",
|
||||||
"R1–R7 完整功能(真实用户麦克风打断、视频/口型、记忆长跑等)继续按获批依赖承接,本卡不构成整产品完成"
|
"R1–R7 完整功能、真实用户麦克风、视频/口型、记忆长跑继续按获批依赖承接;本卡不构成整产品完成"
|
||||||
],
|
],
|
||||||
"requirement_changes": [],
|
"requirement_changes": [],
|
||||||
"verification": [
|
"verification": [
|
||||||
"docs/rework/hel908/summary.md:根因时间轴(worker.log:108 13ms 证据、DB id=5/6/8、_retry_on_chunk_sent 纠偏对照源码)、最小变更表、残留声明",
|
"docs/rework/hel908/summary.md:两轮根因/纠偏/方案/等待代价/边界全记录;第一轮不实表述已删除,原始证据与纠偏保留",
|
||||||
"docs/rework/hel908/tests-remote.log:.venv311 全量 445 passed/5 skipped/2 pre-existing failed + agent venv SDK harness 5/5 PASS 实况",
|
"docs/rework/hel908/tests-remote-r2.log:.venv311 全量 480 passed/8 skipped/2 pre-existing + agent venv SDK harness 8/8 实况",
|
||||||
|
"docs/rework/hel908/web-r2.log:typecheck/lint/vitest 148/148/build",
|
||||||
"tests/test_sdk_livekit_hel908.py 可在任何装有 livekit-agents 1.8.8 的环境复跑(pytest 或 --standalone),命令写在文件头",
|
"tests/test_sdk_livekit_hel908.py 可在任何装有 livekit-agents 1.8.8 的环境复跑(pytest 或 --standalone),命令写在文件头",
|
||||||
"预先存在失败的基准证据:test_r3 两例与 gen_hel877 harness 在 5625de6 上同样失败(本卡留存对照运行记录)",
|
"需求检查:check_requirements.py 对 origin/main 与 b8ec832 各带 --handoff 本文件通过(日志 docs/rework/hel908/check-requirements.log)"
|
||||||
"需求检查:check_requirements.py 对 origin/main 与 5625de6 各带 --handoff 本文件通过(日志随交付)"
|
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
||||||
== --base-ref origin/main --handoff ==
|
== --base-ref origin/main --handoff ==
|
||||||
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
||||||
== --base-ref 5625de6 ==
|
== --base-ref b8ec832 ==
|
||||||
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
||||||
== --base-ref 5625de6 --handoff ==
|
== --base-ref b8ec832 --handoff ==
|
||||||
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。
|
||||||
|
|||||||
+110
-62
@@ -1,87 +1,135 @@
|
|||||||
# HEL-908 · 定点返修记录:交谈打断丢字 + 内部思考外泄(2026-10-28)
|
# HEL-908 · 定点返修记录:交谈打断丢字 + 内部思考外泄(2026-10-28)
|
||||||
|
|
||||||
> 授权:本卡 HEL-908(主卡 HEL-797 定点返修)。全部验证离线、零真实请求、零部署。
|
> 授权:本卡 HEL-908(主卡 HEL-797 定点返修;2026-10-28 HEL-909 集中返修一次)。
|
||||||
> 施工基准:`agent/agent/47adf6961f8a` @ `5625de6606a607a9d4fd583fe8a52d93d6d2b4ea`(完整 SHA 已核)。
|
> 全部验证离线、零真实请求、零部署。
|
||||||
> 需求基准 v1.12/60 项,本卡零需求变更。
|
> 施工基准:`agent/agent/47adf6961f8a` @ `5625de6606…`(完整 SHA 已核);
|
||||||
|
> 第一轮交付 `b8ec832`;HEL-909 审核不通过后在同分支追加本轮提交。
|
||||||
|
> 需求基准 v1.12/60 项,两轮均零需求变更。
|
||||||
|
|
||||||
## 0. 结论速览
|
## 0. 结论速览(第二轮后)
|
||||||
|
|
||||||
| 项 | 结果 |
|
| 项 | 结果 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| 问题1:她在播语音时网页发文字 → 文字轮被取消、无回复、旧语音继续约32秒 | ✅ 已修复并离线复现→验证(服务端+worker 双侧,穿真实 SDK 路径) |
|
| 问题1:文字打断丢字 / 旧轮复活 / 旧语音约32秒 | ✅ 第一轮已修(HEL-909 复核通过,含真实 SDK 证据) |
|
||||||
| 问题2:`Considering the limi…`/`</think>` 进入字幕与 TTS | ✅ 已修复并离线复现→验证(记录流逐字回放);1 项有界残留已如实声明(§4) |
|
| R1:隐式思考整段进流式/落库/receipt(HEL-909 不通过项) | ✅ 已按裁定改为**显式整轮缓冲模式**(`buffer_turn`),评审复现串 A1/A2 逐字/小块经真实适配器→真实 app→落库/receipt 全干净 |
|
||||||
| HEL-907 原报告“`_retry_on_chunk_sent` 守卫从未生效”的推断 | ❌ 不成立(按总管只读核对纠偏,本卡实证确认,见 §1.2) |
|
| R2:LLM 已 final、TTS 仍在播时文字无法停旧声(HEL-909 不通过项) | ✅ 网页既有 `hush()` 入口经房间数据通道发 `cg.interrupt`(reliable),worker 订阅→`session.interrupt(force=True)`(LiveKit 既有打断),真实 SDK 播放等待取消证明 8/8 |
|
||||||
| 既有有限重试/已完成幂等重放/receipt 校验 | ✅ 未被破坏(专项测试证明) |
|
| 第一轮交付中“落库与 receipt 始终干净”的表述 | ❌ 不实,已删除(原流式 hold 方案在中文句号/超128字符形状下整段泄漏且 closer 被适配层提前删除,轮级 split 无法兜底——HEL-909 A1/A2 证实) |
|
||||||
| 真实端到端复测(真人麦克风/真房间) | ⛔ 本卡零真实请求边界内未做,交回总管按窗口安排 |
|
| 既有有限重试/幂等重放/receipt 校验 | ✅ 未破坏(专项测试持续通过) |
|
||||||
|
| 真实端到端复测(真人/真房间/时延) | ⛔ 未做,交回总管按授权窗口安排 |
|
||||||
|
|
||||||
## 1. 问题1根因(源码级 + 实证时间轴)
|
## 1. 问题1根因与修复(第一轮,HEL-909 复核通过)
|
||||||
|
|
||||||
### 1.1 事实链(HEL-907 证据 + 本卡新增核对)
|
**事实链**:`worker.log:108` 的 SDK 重试警告在文字 POST 后 13ms——取消发生在首 delta 前
|
||||||
|
(`output_sent=False`),服务端取消收尾无终态帧,worker 把它当连接错误,livekit-agents
|
||||||
|
1.8.8 `LLMStream._main_task` 静默重试同一 `turn_id`,重试重新激活已取消轮、反杀新文字轮、
|
||||||
|
重新生成整段语音播完(DB id=5/6/8;t0 后 32.0 秒音频)。原报告“`_retry_on_chunk_sent`
|
||||||
|
守卫从未生效”的推断不成立:守卫只在已有输出后适用(.12 实装源码核对一致)。
|
||||||
|
|
||||||
- `worker.log:108`:`failed to generate LLM completion: chat stream ended without final or error …, retrying in 0.1s`,时间 `12:17:58.455`;用户文字 POST t0=`12:17:58.442`(call1.json `t0_text_post_ms=1793189878442`)——**SSE 中止与 SDK 重试发生在文字 POST 后 13ms**。
|
**修复(双侧)**:服务端取消显式下发 `chat.cancelled` 终态帧;`cancelled` 轮永久拒绝
|
||||||
- 该警告来自 livekit-agents 1.8.8 `LLMStream._main_task` 的 APIError 重试分支(attempt 1)→ 证明重试走的是 SDK 自己的重试环,重试同一 `turn_id` 重新执行 `_run` → `send_turn`(api.log:health+POST 同端口 38996 两次)。
|
重新激活(`failed` 轮仍可重试);worker 以 SDK 自身 499 路径正常结束不重试,
|
||||||
- DB:语音轮 user 行 id=5 两度 pending(重试 `ON CONFLICT … DO UPDATE` 重新激活),文字轮 user 行 id=6 `cancelled`,语音重试生成完整回复 id=8(ts=882175,重试 POST 后约 2.6 秒重新生成)并经 TTS 全部播完(t0 后 32.0 秒音频,`old_audio_stop_latency_s=32.015`)。
|
`SpeechHandle.interrupt()` 停掉已缓存音频;`server_cancelled` 重入闸零网络短路。
|
||||||
- 关键事实:**取消发生在首 delta 之前**——12:17:58.442 时该轮一个 chat.delta 都没到(TTS 首包在其后 3.5 秒;字幕全部增量来自重试后的重新生成),因此 `LLMStream._event_ch.output_sent=False`。
|
|
||||||
|
|
||||||
### 1.2 纠偏:原报告“`_retry_on_chunk_sent=False` 永远不起作用”不成立
|
## 2. R1:思考分流(第二轮重做)
|
||||||
|
|
||||||
对照 .12 实际安装的 livekit-agents 1.8.8 源码(只读核对 `livekit/agents/llm/llm.py`):
|
**第一轮方案为何被否**:流式 hold(128 字符/中文句末解除)在真实 token 到达方式下
|
||||||
|
必现泄漏——`'private reasoning '*20+'</think>回答。'`(超128、无句末标点)与
|
||||||
|
`'我先想一下。内部思考</think>回答。'`(中文句号解除 hold)都会把思考整段放出,且
|
||||||
|
stray closer 被适配层删除后,app 的轮级 `split_reasoning` 永远剥不掉前缀;第一轮交付
|
||||||
|
声称“落库与 receipt 始终干净”不成立(HEL-909 A1/A2/A7,本卡已在修复前离线复现同结果)。
|
||||||
|
|
||||||
- `_LLMEventChannel.send_nowait` 在**送入**含 content 的 chunk 时置 `output_sent=True`;
|
**本轮方案(按总管工程裁定:确定的整轮缓冲,不再用长度/标点/英文片段当判据)**:
|
||||||
- `_main_task` 对 APIError:`if not self._retry_on_chunk_sent and self._event_ch.output_sent: e.retryable=False`;`APIStatusError` 499 直接正常返回(不重试不抛出)。
|
|
||||||
|
|
||||||
即守卫**只在已有输出后才适用**;本缺陷路径取消发生在首 chunk 前(`output_sent=False`),守卫根本不在适用范围——不是“守卫失效”,是“取消被当成了临时断网”。本卡 SDK harness 的 `real_connection_errors_still_retry` 用例同时证明:真实连接错误的重试路径未被本次修复全局禁用。
|
- `providers/llm/reasoning.py` 两个显式模式:
|
||||||
|
- `TurnReasoningGate`(`buffer_turn`):生成期间**不外发任何内容**,流结束后
|
||||||
|
`split_reasoning` 整轮分离,一次性下发干净正文;`reasoning_content` 字段增量仍走
|
||||||
|
`LLMDelta.reasoning` 诊断通道(绝不持久化/下发/播报)。
|
||||||
|
- `ReasoningSeparator`(`stream`,默认、向后兼容):仅滤显式 `<think>` 块(跨 chunk
|
||||||
|
安全);stray closer **原样透传**(第一轮的“提前删除 closer”正是缺陷根源,透传保住
|
||||||
|
信号让 app 轮级 split 可兜底)。不再有任何 hold 启发式。
|
||||||
|
- 端点模式显式化:`config/models.yaml` 三档显式 `reasoning: buffer_turn`
|
||||||
|
(`LLMConfig.reasoning` 字面量校验,未配置=既有 `stream` 语义,其他部署不受影响)。
|
||||||
|
- 验证:A1/A2/记录流(id=12)按 1~1000 任意分块经**真实 OpenAICompatLLM → 真实 app**回放
|
||||||
|
——chat.delta、落库行、chat.final、receipt text_hash 全部只绑干净正文;正常中英文
|
||||||
|
逐字节保留;未闭合显式思考→空正文→诚实 `chat.error`;`stream` 模式显式块/字段/跨
|
||||||
|
chunk 语义保持。
|
||||||
|
- **等待代价(如实记录)**:`buffer_turn` 下首个可见正文要等整轮生成完成(生成中零
|
||||||
|
chat.delta;字幕与 TTS 首包不再先于全文);取消/超时行为:被取消轮照常 `chat.cancelled`
|
||||||
|
收尾(零泄漏),`stream_timeout`(60s)超时照常 `chat.error` 诚实失败。低延迟未实测、
|
||||||
|
也不宣称。
|
||||||
|
- **边界(如实)**:无 closer、无 reasoning 字段、无开头标签的推理在任何客户端模式下都
|
||||||
|
无法与正文区分,只有上游自行分流可解;本轮保证的是已实证“带 closer”的形状绝不进入
|
||||||
|
chat.delta/字幕/TTS/落库/receipt。
|
||||||
|
|
||||||
### 1.3 修复(双侧最小改动)
|
## 3. R2:LLM 已 final、TTS 仍在播时的权威打断(第二轮新增)
|
||||||
|
|
||||||
- 服务端 `src/cybergril/api/app.py`:
|
**缺口(HEL-909 evidence C)**:第一轮唯一打断点在“仍活跃 SSE 收到 cancelled”;旧轮
|
||||||
1. `generate()` 取消路径在结束 SSE 前**显式下发终态帧 `chat.cancelled`**(取消≠断网:取消是协议事件,断网才走连接错误语义;客户端已断开时保持原样静默结束)。
|
拿到 final 后服务端已清 active,播放句柄仍 pending 时新文字没有任何权威打断路径——旧语音
|
||||||
2. `chat_lock` 段新增:`status='cancelled'` 的旧轮**拒绝重新激活**,直接幂等回 `chat.cancelled`,且**不触碰当前活动轮**——同一已取消 `turn_id` 永远无法再取得活动轮所有权(迟到的重试再也不能反杀新文字轮)。`failed` 轮仍可重试(既有韧性不变,专项用例覆盖)。
|
照旧播完。
|
||||||
- worker `src/cybergril/voice/`:
|
|
||||||
- `chat_api_client.py`:`_consume` 识别 `chat.cancelled` 终态帧 → `ChatApiTurn.cancelled=True`,不再当 `ChatApiConnectionError`。
|
|
||||||
- `llm_chat_api.py`:取消终态 → 复用 SDK 自身的 499(Client Closed Request)语义 `raise APIStatusError(status_code=499)`,`_main_task` 正常返回、**不重试不抛出**;同时经 `SpeechHandle.interrupt()`(SDK 既有打断路径,VAD 抢话同款)**停掉已缓存的旧音频播放**,不止停止生成。另加 `server_cancelled` 重入闸:一旦该流得知取消,`_run` 再被任何路径重入都立即 499、零网络调用。
|
|
||||||
- `worker.py`:零改动(interrupt 经适配器归属到确切 SpeechHandle;`_after_playout` 对 interrupted 的 fail-closed 收口、字幕 turn_cancel、回执放弃全部走既有路径)。
|
|
||||||
|
|
||||||
## 2. 问题2根因与修复
|
**修复(复用 LiveKit 既有通道与打断,无第二套调度器)**:
|
||||||
|
|
||||||
- 证据:DB 行 id=12(call3)原文 = `"Considering the limited time by the user, I have to give the solution based on the thinking directly now.\n</think>\n\n大圣,您这定身法…"`——**`content` 里只有 stray `</think>`、没有开头 `<think>`**(服务端 chat template 预填了开头),推理原文因此流进 chat.delta → 字幕与 TTS(tts-window-tail.log 两个 segment 起头即 `Considering the limi…` 与 `</think>`)。同一端点同窗口亦有完全干净回复(id=14),证明泄漏是按次混入而非固定。
|
- web(仅 `web/src/voice/session.ts` + `livekit.ts` 纯传输接线):`VoiceRoom` 增
|
||||||
- 定位:`openai_compat.py` 适配器此前只读 `delta.content`,**没有任何 reasoning 分离**;`reasoning_content` 字段(DeepSeek 系约定)也未读。
|
`interrupt()`;既有 `hush()`(ChatPanel 文字发送时的既有入口,零界面改动)在本地静音
|
||||||
- 修复(复用 OpenAI 兼容生态的两个既有分流约定,不造推理引擎、不改提示词、不删英文):
|
反馈之外经 `localParticipant.publishData('{"v":1}', {reliable:true, topic:'cg.interrupt'})`
|
||||||
- 新增 `src/cybergril/providers/llm/reasoning.py`:流式 `ReasoningSeparator` 状态机(hold/think/visible;跨 chunk 标签安全;stray closer=隐式开头思考的收口;中文句末/128 字符解除 hold)+ 轮级 `split_reasoning`。
|
发出本通打断消息——fire-and-forget,publish 未完成/失败/断线不产生未处理异常、不阻塞
|
||||||
- `openai_compat.py`:`reasoning_content`/`reasoning` 字段增量 → `LLMDelta.reasoning`(仅诊断,绝不持久化/下发/播报);`content` 过分离器后才是 `delta.text`。
|
文字发送(失败丢弃:本地静音与服务端轮次门仍在)。本通绑定由房间隔离天然成立。
|
||||||
- `contracts/utterance.py`:`LLMDelta.reasoning` 字段(默认空,向后兼容)。
|
- worker(`worker.py` 新增 `bind_interrupt_channel`):订阅本房间 `data_received`;
|
||||||
- `app.py`:落库/chat.final/receipt 前再做轮级 `split_reasoning`(对超出 hold 的流式残留兜底,receipt 绑定干净文本)。
|
校验 topic 精确 `cg.interrupt`、发送者为远端用户参与者(服务端/自身来源忽略)、载荷
|
||||||
- 边界:原始模型输出 →(适配器分流)→ 可见正文才可持久化/字幕/TTS;reasoning 仅存在于 `LLMDelta.reasoning` 诊断通道,落地即弃。
|
`{"v":1}` 精确匹配;通过即 `session.interrupt(force=True)`(SDK 现成 programmatic
|
||||||
|
打断;`AgentActivity.interrupt` 实装源码核 对 force 透传到 `_current_speech.interrupt
|
||||||
|
(force=…)`,真实句柄在 `allow_interruptions=False` 且非 force 时抛 RuntimeError——故
|
||||||
|
force)。失败大声留痕(print 落 worker.log),不静默假装成功;`add_shutdown_callback`
|
||||||
|
注册卸载,会话结束移除监听。`llm_chat_api._interrupt_owner` 同步改为 force=True+留痕。
|
||||||
|
- **真实 SDK 证明(8/8,agent venv 1.8.8)**:真实 `SpeechHandle`(allow_interruptions=
|
||||||
|
False,覆盖 force 语义)+ 真实 pending 播放等待(`wait_if_not_interrupted`)——完整轮
|
||||||
|
(chat.final 干净结束、无取消路径触碰句柄=评审 C 形状)后注入 `cg.interrupt` 数据消息,
|
||||||
|
播放等待被真实 interrupt 释放、句柄 interrupted;校验矩阵(错误 topic/服务端来源/自身/
|
||||||
|
坏载荷/错版本)全部不触发;卸载后不再触发;session 未运行时大声失败不崩溃。非 Spy
|
||||||
|
计数:打断走真实句柄的真实 interrupt 语义(`HandleSessionBridge` 只站在
|
||||||
|
`AgentSession.interrupt` 的位置,其到句柄的转发按实装源码核对)。
|
||||||
|
- **app 集成记录**:完整语音轮之后文字轮即普通轮(无任何行被取消、无 phantom 行)——
|
||||||
|
该时序在 app 缝合面正常,缺的只是 worker 侧通道,本轮已闭合。
|
||||||
|
|
||||||
## 3. 最小变更表
|
## 4. 变更表(本轮追加)
|
||||||
|
|
||||||
| 文件 | 变更 |
|
| 文件 | 变更 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `src/cybergril/api/app.py` | 取消路径下发 `chat.cancelled`;拒绝 cancelled 轮重新激活;落库前轮级分流兜底 |
|
| `src/cybergril/providers/llm/reasoning.py` | 重做:`TurnReasoningGate`(整轮缓冲)+ 无启发式 `ReasoningSeparator`(stray closer 透传);`split_reasoning` 保持 |
|
||||||
| `src/cybergril/voice/chat_api_client.py` | `chat.cancelled` 终态帧解析(`ChatApiTurn.cancelled`) |
|
| `src/cybergril/providers/llm/openai_compat.py` | 显式 `reasoning` 模式参数(`stream`/`buffer_turn`,默认前者) |
|
||||||
| `src/cybergril/voice/llm_chat_api.py` | 499 取消终态(不重试)+ `SpeechHandle.interrupt()` 停播 + `server_cancelled` 重入闸 |
|
| `src/cybergril/config.py` + `think/wiring.py` + `config/models.yaml` | `LLMConfig.reasoning` 字面量 + 透传;三档显式 `buffer_turn` |
|
||||||
| `src/cybergril/providers/llm/reasoning.py` | 新增:流式思考分流状态机 + 轮级分流 |
|
| `src/cybergril/voice/worker.py` | 新增 `bind_interrupt_channel`(订阅/校验/force 打断/留痕/卸载),entrypoint 接线 |
|
||||||
| `src/cybergril/providers/llm/openai_compat.py` | 接入分流器;读 `reasoning_content` 字段 |
|
| `src/cybergril/voice/llm_chat_api.py` | `_interrupt_owner` force=True+失败留痕 |
|
||||||
| `src/cybergril/contracts/utterance.py` | `LLMDelta.reasoning` 诊断字段 |
|
| `web/src/voice/session.ts` / `livekit.ts` | `VoiceRoom.interrupt()`;`hush()` 附带发 `cg.interrupt`(纯传输接线,零视觉改动) |
|
||||||
| `tests/test_rework_hel908.py` | 新增:app 级取消语义(首 delta 前取消/首 delta 后取消/迟到重试拒绝/文字轮存活/think 泄漏回放/未闭合思考 fail-closed/重放与失败重试不变) |
|
| `tests/test_reasoning_separation_hel908.py` | 重写为双模式(A1/A2/记录流任意分块、中英文保留、跨 chunk、字段、未闭合、stray 透传) |
|
||||||
| `tests/test_reasoning_separation_hel908.py` | 新增:分流单测(记录流逐字回放、跨 chunk 标签、推理字段、未闭合、中英文正文保留、残留声明) |
|
| `tests/test_rework_hel908.py` | A1/A2 经真实适配器→真实 app→落库/receipt 回放;evidence-C 时序 app 记录;adapter fixture 按 buffer_turn |
|
||||||
| `tests/test_sdk_livekit_hel908.py` | 新增:真实 SDK 路径 harness(pytest 可收集;agent venv 无 pytest 可独立运行) |
|
| `tests/test_sdk_livekit_hel908.py` | +3 项 R2 真实 SDK 检查(共 8) |
|
||||||
| `tests/test_voice_chat_adapter.py` | 既有用例更新到新终态语义(被取消方收到 `chat.cancelled` 干净结束,不再异常) |
|
| `tests/test_voice_asr_usage.py` | FakeCtx 补 `add_shutdown_callback`/`room.on|off`,新增通道接线断言 |
|
||||||
| `tests/test_api.py` | 断连用例的上游内容补句号(hold 语义下首帧可流出;用例意图——断连关闭上游——不变) |
|
| `web/tests/*.test.ts`(5 个) | `VoiceRoom` 桩补 `interrupt()`;voice.test 断言 hush 时通道被调用 |
|
||||||
|
|
||||||
## 4. 验证(全部离线、零真实请求)
|
## 5. 验证(全部离线、零真实请求)
|
||||||
|
|
||||||
- **规范环境**:`.12 /opt/cybergril-hel879/current/.venv311`(requirements.lock 逐字一致,只读复用;代码在 `/tmp` 副本运行,未装任何依赖)。
|
- **规范环境** `.venv311`(只读复用):全量 **480 passed / 8 skipped / 2 failed**——两个
|
||||||
- 全量:**445 passed / 5 skipped**(5 skipped=SDK harness 需 livekit,见下)/ 2 failed=`test_r3` 基线/迁移两例——**在施工基准 5625de6 上同样失败(预先存在,与本卡无关,已留存证据)**。
|
`test_r3` 基线/迁移失败在基准 `5625de6` 上同样失败(预先存在,前轮已留证)。
|
||||||
- `tests-remote.log`:全量 + SDK harness 输出。
|
- **真实 SDK harness**(agent venv,livekit-agents 1.8.8):**8/8**(前轮 5 项回归全保持 +
|
||||||
- **真实 SDK 路径**(`.12 /opt/voice-agent/agent/venv`,livekit-agents 1.8.8,只读复用):`tests/test_sdk_livekit_hel908.py --standalone` **5/5 PASS**——首 delta 前取消不重试(1 次 POST)、首 delta 后取消保留已发 chunk 且 `SpeechHandle.interrupt()` 恰好一次、取消后重入立即 499 零网络、**真实连接错误仍按 SDK 有限重试(max_retry+1 次后诚实失败)**、完成轮流式与 receipt 不变。
|
R2 三项新证)。
|
||||||
- **记录流回放**:`tests/test_reasoning_separation_hel908.py` 37/37(含 id=12 原文按 1/2/3/5/7/13/1000 字符分块全零泄漏;`</think>` 跨 SSE 帧拆分;中英文正文逐字节保留;未闭合思考→空正文→诚实 `chat.error`)。
|
- **web**:`npm ci`(既有锁文件+缓存,零新依赖/零 lock 变更);typecheck 0 错;lint
|
||||||
- 本地环境(py3.10,与 lock 版本不一致):reasoning 37/37、ruff 全绿;app 级用例因 starlette/anyio 版本差异在本地环境基线即不可运行(已在 .12 规范环境执行)。
|
0 错 1 既有警告(`ChatPanel.tsx:252` react-hooks,非本卡引入);vitest **148/148**;
|
||||||
- ruff:`src/ tests/` 全绿。
|
build 成功。
|
||||||
- `scripts/gen_hel877_sdk_check.py` 在 agent venv 于**施工基准上即失败**(HEL-906 引入 `stt_funasr→asr_usage` 导入后 harness 模块清单未更新,与本卡无关,base_exit=2 证据留存);本卡 SDK 验证由新 harness 覆盖。
|
- ruff `src/ tests/` 全绿;`config/models.yaml` 新键经 `ModelsConfig` 实际解析验证。
|
||||||
|
- 本地环境(py3.10 与 lock 版本不一致):reasoning 69/69 与 ruff 本地跑通;app 级用例按
|
||||||
|
项目既有方式在 .12 规范环境执行。
|
||||||
|
- 日志:`tests-remote-r2.log`(全量+SDK harness)、`web-r2.log`、
|
||||||
|
`check-requirements.log`(本轮重跑)。
|
||||||
|
|
||||||
## 5. 残留与未测(如实)
|
## 6. 残留与未测(如实)
|
||||||
|
|
||||||
1. **思考分流流式残留**:隐式开头(无 `<think>`)且收尾 `</think>` 落在 128 字符 hold 之外的推理,前缀仍会部分流出到字幕/TTS(无法在流式下提前判定);落库文本与 receipt 始终干净(轮级兜底,专项用例声明该边界)。彻底消除需上游自行分流(`reasoning_content` 字段或服务端 reasoning parser)——零真实请求边界内无法探测该端点能力,这是**最小缺口**,不虚称已完全修好。
|
1. `buffer_turn` 的首段等待代价:整轮生成完成前无任何可见正文(已在 §2 记录并测试固定
|
||||||
2. 修复后的真实端到端复测(真实房间/真人打断时延/真机)未做,交回总管按授权窗口安排;停播路径用真实 SDK 的 `SpeechHandle.interrupt()` 断言证明,不冒充端到端毫秒指标。
|
该行为);对长回复的字幕/TTS 起播延迟未在真实端到端测量——真实端点体验须授权窗口。
|
||||||
3. 前端零改动(新帧 kind 被现有 ChatPanel 静默忽略,行为不差于现状);把“被新轮打断”做成可见界面状态属界面核心,交视觉总监。
|
2. 无 closer、无字段的推理不可识别(§2 边界);`stream` 模式下 stray closer 透传意味着
|
||||||
|
该模式端点若真出现隐式思考,字幕/TTS 会看到标记原文(落库/receipt 由轮级 split 清理)
|
||||||
|
——产品端点已配 `buffer_turn`,不受此影响。
|
||||||
|
3. R2 真实房间端到端(页面 publish→worker 停播的实际时延、断线姿态)未测;离线证明是
|
||||||
|
真实 SDK 句柄/等待语义 + 实装源码核对,不冒充端到端毫秒指标。
|
||||||
|
4. `chat.cancelled` 前端呈现(建议 #4)、ChatPanel 取消判定 limit(建议 #3)等非阻塞
|
||||||
|
建议未动(后者纯展示层;前者属界面核心,交视觉总监)。
|
||||||
|
5. R1–R7 完整功能、真实用户麦克风、视频/口型、记忆长跑仍按获批依赖承接。
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||||
|
=========================== short test summary info ============================
|
||||||
|
FAILED tests/test_r3.py::test_populated_migration_idempotence_and_old_code_rollback
|
||||||
|
FAILED tests/test_r3.py::test_three_endpoint_bytes_equal_product_baseline - s...
|
||||||
|
2 failed, 480 passed, 8 skipped, 1 warning in 51.72s
|
||||||
|
=== SDK harness (agent venv, livekit-agents 1.8.8) ===
|
||||||
|
failed to generate LLM completion: message='chat api failure: network is genuinely down', status_code=502, retryable=True, retrying in 0.1s
|
||||||
|
failed to generate LLM completion: message='chat api failure: network is genuinely down', status_code=502, retryable=True, retrying in 0.05s
|
||||||
|
PASS pre_first_delta_cancel_never_retries
|
||||||
|
PASS mid_stream_cancel_interrupts_handle_and_never_retries
|
||||||
|
PASS reentry_after_cancel_short_circuits_without_network
|
||||||
|
PASS real_connection_errors_still_retry_not_globally_disabled
|
||||||
|
PASS completed_turn_still_streams_with_receipt
|
||||||
|
PASS interrupt_message_stops_pending_playout_after_final
|
||||||
|
PASS interrupt_channel_validation_and_unload
|
||||||
|
[cybergril-worker] interrupt message failed: RuntimeError: AgentSession isn't running
|
||||||
|
[cybergril-worker] interrupt message failed: RuntimeError: AgentSession isn't running
|
||||||
|
PASS interrupt_message_survives_session_not_running
|
||||||
|
8/8 passed
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
|
||||||
|
✖ 1 problem (0 errors, 1 warning)
|
||||||
|
|
||||||
|
Test Files 12 passed (12)
|
||||||
|
Tests 148 passed (148)
|
||||||
|
✓ built in 720ms
|
||||||
@@ -34,6 +34,11 @@ class LLMConfig(BaseModel):
|
|||||||
temperature: float | None = 0.85
|
temperature: float | None = 0.85
|
||||||
max_tokens: int | None = None
|
max_tokens: int | None = None
|
||||||
languages: list[str] = Field(default_factory=lambda: ["zh", "en"])
|
languages: list[str] = Field(default_factory=lambda: ["zh", "en"])
|
||||||
|
reasoning: Literal["stream", "buffer_turn"] = "stream"
|
||||||
|
"""HEL-908/909 思考分流端点模式:stream=只滤显式 <think> 块(流式,适合
|
||||||
|
自带 reasoning_content 字段或显式标签的端点);buffer_turn=整轮缓冲后
|
||||||
|
一次性下发干净正文(隐式开头思考端点必须用它,代价是首段可见文本
|
||||||
|
等整轮生成完成,详见 providers/llm/reasoning.py)。"""
|
||||||
|
|
||||||
|
|
||||||
def _default_tiers() -> dict[str, LLMConfig]:
|
def _default_tiers() -> dict[str, LLMConfig]:
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ from ...contracts import (
|
|||||||
)
|
)
|
||||||
from ..base import ProviderError
|
from ..base import ProviderError
|
||||||
from .base import LLMProvider
|
from .base import LLMProvider
|
||||||
from .reasoning import ReasoningSeparator
|
from .reasoning import ReasoningSeparator, TurnReasoningGate
|
||||||
|
|
||||||
|
|
||||||
class OpenAICompatLLM(LLMProvider):
|
class OpenAICompatLLM(LLMProvider):
|
||||||
@@ -41,6 +41,7 @@ class OpenAICompatLLM(LLMProvider):
|
|||||||
timeout: float = 60.0,
|
timeout: float = 60.0,
|
||||||
languages: frozenset[str] = frozenset({"zh", "en"}),
|
languages: frozenset[str] = frozenset({"zh", "en"}),
|
||||||
max_context: int | None = None,
|
max_context: int | None = None,
|
||||||
|
reasoning: str = "stream",
|
||||||
) -> None:
|
) -> None:
|
||||||
self.base_url = base_url.rstrip("/")
|
self.base_url = base_url.rstrip("/")
|
||||||
self.model = model
|
self.model = model
|
||||||
@@ -52,6 +53,13 @@ class OpenAICompatLLM(LLMProvider):
|
|||||||
)
|
)
|
||||||
self._languages = languages
|
self._languages = languages
|
||||||
self._max_context = max_context
|
self._max_context = max_context
|
||||||
|
if reasoning not in ("stream", "buffer_turn"):
|
||||||
|
raise ValueError(f"unknown reasoning mode {reasoning!r}")
|
||||||
|
# HEL-908/909: explicit, conservative, backward-compatible endpoint
|
||||||
|
# mode — "stream" (default) filters explicit <think> blocks live;
|
||||||
|
# "buffer_turn" gates the whole turn and emits only the clean visible
|
||||||
|
# text at the end (for endpoints that stream implicit-open reasoning).
|
||||||
|
self._reasoning = reasoning
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _headers(api_key: str | None) -> dict[str, str]:
|
def _headers(api_key: str | None) -> dict[str, str]:
|
||||||
@@ -108,18 +116,23 @@ class OpenAICompatLLM(LLMProvider):
|
|||||||
f"LLM {self.name} HTTP {resp.status_code}: {body[:400]}"
|
f"LLM {self.name} HTTP {resp.status_code}: {body[:400]}"
|
||||||
)
|
)
|
||||||
|
|
||||||
# HEL-908: reasoning/user-content separation at the adapter seam —
|
# HEL-908/909: reasoning/user-content separation at the adapter
|
||||||
# reasoning_content deltas (DeepSeek-style field) go to the
|
# seam. reasoning_content deltas (DeepSeek-style field) always go
|
||||||
# reasoning side-channel, inline <think> conventions are stripped
|
# to the reasoning side-channel; inline content passes the mode's
|
||||||
# cross-chunk; only visible text ever reaches delta.text.
|
# gate — "buffer_turn" emits nothing until the whole turn is
|
||||||
separator = ReasoningSeparator()
|
# safely separated, "stream" filters explicit <think> blocks live
|
||||||
|
# and passes stray closers through unmodified.
|
||||||
|
gate: ReasoningSeparator | TurnReasoningGate = (
|
||||||
|
TurnReasoningGate() if self._reasoning == "buffer_turn"
|
||||||
|
else ReasoningSeparator()
|
||||||
|
)
|
||||||
|
|
||||||
async for line in resp.aiter_lines():
|
async for line in resp.aiter_lines():
|
||||||
if not line.startswith("data:"):
|
if not line.startswith("data:"):
|
||||||
continue
|
continue
|
||||||
data = line[5:].strip()
|
data = line[5:].strip()
|
||||||
if data == "[DONE]":
|
if data == "[DONE]":
|
||||||
tail_visible, tail_reasoning = separator.finish()
|
tail_visible, tail_reasoning = gate.finish()
|
||||||
if tail_visible or tail_reasoning:
|
if tail_visible or tail_reasoning:
|
||||||
yield LLMDelta(text=tail_visible, reasoning=tail_reasoning)
|
yield LLMDelta(text=tail_visible, reasoning=tail_reasoning)
|
||||||
yield LLMDelta(done=True)
|
yield LLMDelta(done=True)
|
||||||
@@ -135,7 +148,14 @@ class OpenAICompatLLM(LLMProvider):
|
|||||||
field_reasoning = (
|
field_reasoning = (
|
||||||
delta.get("reasoning_content") or delta.get("reasoning") or ""
|
delta.get("reasoning_content") or delta.get("reasoning") or ""
|
||||||
)
|
)
|
||||||
visible, dropped = separator.feed(text) if text else ("", "")
|
if isinstance(gate, TurnReasoningGate):
|
||||||
|
if text:
|
||||||
|
gate.feed(text)
|
||||||
|
if field_reasoning:
|
||||||
|
gate.feed_field(field_reasoning)
|
||||||
|
visible, dropped = "", ""
|
||||||
|
else:
|
||||||
|
visible, dropped = gate.feed(text) if text else ("", "")
|
||||||
reasoning = dropped + field_reasoning
|
reasoning = dropped + field_reasoning
|
||||||
finish = choice.get("finish_reason")
|
finish = choice.get("finish_reason")
|
||||||
usage = obj.get("usage") or {}
|
usage = obj.get("usage") or {}
|
||||||
|
|||||||
@@ -3,35 +3,46 @@
|
|||||||
The dialogue endpoint (halogen-qwen3.8-flash-next, OpenAI-compatible) is a
|
The dialogue endpoint (halogen-qwen3.8-flash-next, OpenAI-compatible) is a
|
||||||
Qwen3-family model served raw: some replies open a reasoning phase inline in
|
Qwen3-family model served raw: some replies open a reasoning phase inline in
|
||||||
``content``. The recorded evidence (HEL-907 window, companion.db row id=12 and
|
``content``. The recorded evidence (HEL-907 window, companion.db row id=12 and
|
||||||
tts-window-tail.log) shows the exact shape:
|
tts-window-tail.log) shows the shape:
|
||||||
|
|
||||||
Considering the limited time by the user, ... now.\\n</think>\\n\\n大圣,您这定身法…
|
Considering the limited time by the user, ... now.\\n</think>\\n\\n大圣,您这定身法…
|
||||||
|
|
||||||
i.e. reasoning text arrives in ``content`` with NO opening ``<think>`` (the
|
i.e. reasoning text arrives in ``content`` with NO opening ``<think>`` (the
|
||||||
serving chat template pre-fills it) and a stray ``</think>`` closes it, while
|
serving chat template pre-fills it) and a stray ``</think>`` closes it; other
|
||||||
other replies from the same endpoint carry no tags at all, and DeepSeek-style
|
replies from the same endpoint carry no tags at all, and DeepSeek-style
|
||||||
endpoints instead stream a separate ``reasoning_content`` delta field.
|
endpoints instead stream a separate ``reasoning_content`` delta field.
|
||||||
|
|
||||||
This module is the single boundary between raw model output and user-visible
|
HEL-909 review established that this implicit-open shape CANNOT be separated
|
||||||
content: visible text is the ONLY thing that may be persisted as her reply,
|
safely while streaming: an unopened reasoning phase is indistinguishable from
|
||||||
streamed as chat.delta (subtitles) or spoken (TTS). Reasoning is exposed on
|
visible text until its ``</think>`` arrives (Chinese reasoning contains
|
||||||
``LLMDelta.reasoning`` for diagnostics and dropped everywhere else — it is
|
sentence-enders and routinely exceeds any length bound — the earlier
|
||||||
never persisted. This is the same ``<think>`` text convention vLLM/SGLang
|
128-char/sentence-end hold was proven to leak, and the adapter consumed the
|
||||||
reasoning parsers implement server-side (GOV-006: reuse the mature
|
closer so the turn-level split could not recover). Two explicit, conservative
|
||||||
convention, do not build a reasoning engine): no prompt changes, no model
|
modes now exist (selected per endpoint in models.yaml, GOV-006: reuse the
|
||||||
behavior changes, and nothing is deleted except the tag-delimited reasoning
|
mature conventions, no second reasoning engine):
|
||||||
itself.
|
|
||||||
|
|
||||||
Streaming caveat (documented residual, offline-verifiable): an implicit-open
|
``buffer_turn`` Whole-turn gate: NOTHING is emitted during generation; at
|
||||||
reasoning phase is indistinguishable from visible text until its ``</think>``
|
stream end ``split_reasoning`` decides and the clean visible
|
||||||
arrives, so the separator holds back the beginning of every stream until
|
text is delivered as one delta. Guarantees the recorded
|
||||||
either a tag appears, a Chinese sentence ends, or HOLD_LIMIT characters
|
closer-bearing shape never reaches chat.delta, subtitles,
|
||||||
accumulate. The recorded leak (closer at ~104 chars) is fully covered;
|
TTS, persistence or the receipt. Cost (measured, not
|
||||||
reasoning whose closer lies beyond HOLD_LIMIT streams partially visible —
|
assumed): the first visible text waits for the entire
|
||||||
``split_reasoning`` still cleans the archived reply and the receipt text at
|
generation; a cancelled/timed-out turn emits nothing at all
|
||||||
turn level (api/app.py). Removing the residual needs the upstream to separate
|
and ends exactly as any other cancelled/failed turn.
|
||||||
reasoning itself (reasoning_content field or a server-side parser), which
|
This is the mode the product's tiers configure.
|
||||||
cannot be probed under the zero-real-request rule of this rework.
|
|
||||||
|
``stream`` Streaming separator for endpoints that separate reasoning
|
||||||
|
properly (explicit ``<think>`` tags or a ``reasoning_content``
|
||||||
|
field): explicit blocks are filtered live, cross-chunk safe.
|
||||||
|
A stray ``</think>`` with no open is passed through
|
||||||
|
UNMODIFIED — deleting it was the HEL-909 defect; the signal
|
||||||
|
must survive so api/app.py's turn-level split can act on it.
|
||||||
|
|
||||||
|
Boundary stated honestly: reasoning that carries NO closer and NO field and
|
||||||
|
NO opening tag is indistinguishable from a real reply in ANY client-side
|
||||||
|
mode; only the upstream can separate that (field or server-side parser).
|
||||||
|
Normal Chinese and English user-visible content is preserved byte-for-byte
|
||||||
|
in both modes; nothing is deleted except tag-delimited reasoning.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -39,13 +50,109 @@ from __future__ import annotations
|
|||||||
THINK_OPEN = "<think>"
|
THINK_OPEN = "<think>"
|
||||||
THINK_CLOSE = "</think>"
|
THINK_CLOSE = "</think>"
|
||||||
|
|
||||||
# Held prefix before the stream is known visible: comfortably above the
|
|
||||||
# recorded implicit-open reasoning length (~104 chars incl. </think>).
|
class ReasoningSeparator:
|
||||||
HOLD_LIMIT = 128
|
"""Streaming separation for ``stream`` mode (explicit tags only).
|
||||||
# A finished Chinese sentence ends the hold: her replies are Chinese, and the
|
|
||||||
# recorded reasoning contains none of these before its closer. ASCII "." and
|
``feed``/``finish`` return ``(visible_delta, reasoning_delta)``. Starts in
|
||||||
# newline are deliberately NOT flush triggers (both occur inside reasoning).
|
visible mode — no length or punctuation heuristics (removed after HEL-909:
|
||||||
_SENTENCE_ENDS = "。!?!?"
|
they leaked on real token arrival). Explicit ``<think>`` blocks are
|
||||||
|
filtered live; a stray ``</think>`` passes through so downstream
|
||||||
|
turn-level splitting keeps its signal.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._in_think = False
|
||||||
|
self._buffer = ""
|
||||||
|
self._reasoning = ""
|
||||||
|
self._strip_pending = False # drop leading ws until content (post-think)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def reasoning(self) -> str:
|
||||||
|
"""All reasoning seen so far (diagnostics only; never persisted)."""
|
||||||
|
return self._reasoning
|
||||||
|
|
||||||
|
def feed(self, chunk: str) -> tuple[str, str]:
|
||||||
|
self._buffer += chunk
|
||||||
|
return self._drain(final=False)
|
||||||
|
|
||||||
|
def finish(self) -> tuple[str, str]:
|
||||||
|
return self._drain(final=True)
|
||||||
|
|
||||||
|
def _drain(self, *, final: bool) -> tuple[str, str]:
|
||||||
|
visible = ""
|
||||||
|
reasoning = ""
|
||||||
|
while True:
|
||||||
|
if self._in_think:
|
||||||
|
close_at = self._buffer.find(THINK_CLOSE)
|
||||||
|
if close_at != -1:
|
||||||
|
dropped = self._buffer[: close_at + len(THINK_CLOSE)]
|
||||||
|
self._reasoning += dropped
|
||||||
|
reasoning += dropped
|
||||||
|
self._buffer = self._buffer[close_at + len(THINK_CLOSE):]
|
||||||
|
self._in_think = False
|
||||||
|
self._strip_pending = True
|
||||||
|
continue
|
||||||
|
keep = 0 if final else _partial_tag_tail(self._buffer, (THINK_CLOSE,))
|
||||||
|
cut = len(self._buffer) - keep
|
||||||
|
if cut > 0:
|
||||||
|
self._reasoning += self._buffer[:cut]
|
||||||
|
reasoning += self._buffer[:cut]
|
||||||
|
self._buffer = self._buffer[cut:]
|
||||||
|
return visible, reasoning
|
||||||
|
if self._strip_pending and self._buffer:
|
||||||
|
stripped = self._buffer.lstrip()
|
||||||
|
if stripped:
|
||||||
|
self._buffer = stripped
|
||||||
|
self._strip_pending = False
|
||||||
|
else: # whitespace only so far; more may follow
|
||||||
|
self._buffer = ""
|
||||||
|
open_at = self._buffer.find(THINK_OPEN)
|
||||||
|
if open_at != -1:
|
||||||
|
visible += self._buffer[:open_at]
|
||||||
|
self._buffer = self._buffer[open_at + len(THINK_OPEN):]
|
||||||
|
self._in_think = True
|
||||||
|
continue
|
||||||
|
if final:
|
||||||
|
visible += self._buffer
|
||||||
|
self._buffer = ""
|
||||||
|
return visible, reasoning
|
||||||
|
keep = _partial_tag_tail(self._buffer, (THINK_OPEN,))
|
||||||
|
cut = len(self._buffer) - keep
|
||||||
|
if cut > 0:
|
||||||
|
visible += self._buffer[:cut]
|
||||||
|
self._buffer = self._buffer[cut:]
|
||||||
|
return visible, reasoning
|
||||||
|
|
||||||
|
|
||||||
|
class TurnReasoningGate:
|
||||||
|
"""Whole-turn gate for ``buffer_turn`` mode (HEL-909 R1 ruling).
|
||||||
|
|
||||||
|
Emits nothing while the stream runs; at the end ``split_reasoning``
|
||||||
|
decides and the clean visible text is the ONLY content ever emitted.
|
||||||
|
``reasoning`` deltas from a separate ``reasoning_content`` field are
|
||||||
|
accumulated alongside and never become text.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._raw = ""
|
||||||
|
self._field_reasoning = ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def reasoning(self) -> str:
|
||||||
|
"""All reasoning seen so far (diagnostics only; never persisted)."""
|
||||||
|
return self._field_reasoning
|
||||||
|
|
||||||
|
def feed(self, chunk: str) -> tuple[str, str]:
|
||||||
|
self._raw += chunk
|
||||||
|
return "", ""
|
||||||
|
|
||||||
|
def feed_field(self, reasoning: str) -> None:
|
||||||
|
self._field_reasoning += reasoning
|
||||||
|
|
||||||
|
def finish(self) -> tuple[str, str]:
|
||||||
|
visible, inline = split_reasoning(self._raw)
|
||||||
|
return visible, self._field_reasoning + inline
|
||||||
|
|
||||||
|
|
||||||
def _partial_tag_tail(buffer: str, tags: tuple[str, ...]) -> int:
|
def _partial_tag_tail(buffer: str, tags: tuple[str, ...]) -> int:
|
||||||
@@ -58,132 +165,13 @@ def _partial_tag_tail(buffer: str, tags: tuple[str, ...]) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
class ReasoningSeparator:
|
|
||||||
"""Stateful streaming separator for one upstream reply.
|
|
||||||
|
|
||||||
``feed``/``finish`` return ``(visible_delta, reasoning_delta)``; the
|
|
||||||
concatenation of all visible deltas plus the finish tail is the visible
|
|
||||||
text, everything else is reasoning. Untagged content is always preserved
|
|
||||||
byte-for-byte (Chinese or English) — only tag-delimited reasoning is
|
|
||||||
dropped.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, *, hold_limit: int = HOLD_LIMIT) -> None:
|
|
||||||
self._hold_limit = hold_limit
|
|
||||||
self._mode = "hold" # hold | think | visible
|
|
||||||
self._buffer = ""
|
|
||||||
self._reasoning = ""
|
|
||||||
self._strip_pending = False # drop leading ws until real content (post-think)
|
|
||||||
|
|
||||||
@property
|
|
||||||
def reasoning(self) -> str:
|
|
||||||
"""All reasoning seen so far (diagnostics only; never persisted)."""
|
|
||||||
return self._reasoning
|
|
||||||
|
|
||||||
def feed(self, chunk: str) -> tuple[str, str]:
|
|
||||||
self._buffer += chunk
|
|
||||||
return self._drain(final=False)
|
|
||||||
|
|
||||||
def finish(self) -> tuple[str, str]:
|
|
||||||
"""End of stream: resolve remaining buffer (partial tags are text)."""
|
|
||||||
return self._drain(final=True)
|
|
||||||
|
|
||||||
def _drain(self, *, final: bool) -> tuple[str, str]:
|
|
||||||
visible = ""
|
|
||||||
reasoning = ""
|
|
||||||
while True:
|
|
||||||
if self._mode == "hold":
|
|
||||||
open_at = self._buffer.find(THINK_OPEN)
|
|
||||||
close_at = self._buffer.find(THINK_CLOSE)
|
|
||||||
if close_at != -1 and (open_at == -1 or close_at < open_at):
|
|
||||||
# implicit-open (template pre-filled <think>) or a normal
|
|
||||||
# block that opened at/after position 0: everything through
|
|
||||||
# the closer is reasoning
|
|
||||||
dropped = self._buffer[: close_at + len(THINK_CLOSE)]
|
|
||||||
self._reasoning += dropped
|
|
||||||
reasoning += dropped
|
|
||||||
self._buffer = self._buffer[close_at + len(THINK_CLOSE):]
|
|
||||||
self._mode = "visible"
|
|
||||||
self._strip_pending = True
|
|
||||||
continue
|
|
||||||
if open_at != -1:
|
|
||||||
visible += self._buffer[:open_at]
|
|
||||||
self._buffer = self._buffer[open_at + len(THINK_OPEN):]
|
|
||||||
self._mode = "think"
|
|
||||||
continue
|
|
||||||
if final:
|
|
||||||
self._mode = "visible" # untagged short reply: never swallow
|
|
||||||
continue
|
|
||||||
sentence_end = min(
|
|
||||||
(
|
|
||||||
i
|
|
||||||
for i in (self._buffer.find(c) for c in _SENTENCE_ENDS)
|
|
||||||
if i != -1
|
|
||||||
),
|
|
||||||
default=-1,
|
|
||||||
)
|
|
||||||
if sentence_end != -1 or len(self._buffer) >= self._hold_limit:
|
|
||||||
self._mode = "visible"
|
|
||||||
continue
|
|
||||||
return visible, reasoning
|
|
||||||
if self._mode == "think":
|
|
||||||
close_at = self._buffer.find(THINK_CLOSE)
|
|
||||||
if close_at != -1:
|
|
||||||
dropped = self._buffer[: close_at + len(THINK_CLOSE)]
|
|
||||||
self._reasoning += dropped
|
|
||||||
reasoning += dropped
|
|
||||||
self._buffer = self._buffer[close_at + len(THINK_CLOSE):]
|
|
||||||
self._mode = "visible"
|
|
||||||
self._strip_pending = True
|
|
||||||
continue
|
|
||||||
keep = 0 if final else _partial_tag_tail(self._buffer, (THINK_CLOSE,))
|
|
||||||
cut = len(self._buffer) - keep
|
|
||||||
if cut > 0:
|
|
||||||
self._reasoning += self._buffer[:cut]
|
|
||||||
self._buffer = self._buffer[cut:]
|
|
||||||
return visible, ""
|
|
||||||
# visible
|
|
||||||
if self._strip_pending and self._buffer:
|
|
||||||
stripped = self._buffer.lstrip()
|
|
||||||
if stripped:
|
|
||||||
self._buffer = stripped
|
|
||||||
self._strip_pending = False
|
|
||||||
else: # whitespace only so far; more may follow
|
|
||||||
self._buffer = ""
|
|
||||||
open_at = self._buffer.find(THINK_OPEN)
|
|
||||||
close_at = self._buffer.find(THINK_CLOSE)
|
|
||||||
if close_at != -1 and (open_at == -1 or close_at < open_at):
|
|
||||||
# stray closer after output already streamed: drop the marker;
|
|
||||||
# split_reasoning still cleans the turn-level text
|
|
||||||
visible += self._buffer[:close_at]
|
|
||||||
self._reasoning += THINK_CLOSE
|
|
||||||
reasoning += THINK_CLOSE
|
|
||||||
self._buffer = self._buffer[close_at + len(THINK_CLOSE):]
|
|
||||||
self._strip_pending = True
|
|
||||||
continue
|
|
||||||
if open_at != -1:
|
|
||||||
visible += self._buffer[:open_at]
|
|
||||||
self._buffer = self._buffer[open_at + len(THINK_OPEN):]
|
|
||||||
self._mode = "think"
|
|
||||||
continue
|
|
||||||
if final:
|
|
||||||
visible += self._buffer
|
|
||||||
self._buffer = ""
|
|
||||||
return visible, reasoning
|
|
||||||
keep = _partial_tag_tail(self._buffer, (THINK_OPEN, THINK_CLOSE))
|
|
||||||
cut = len(self._buffer) - keep
|
|
||||||
if cut > 0:
|
|
||||||
visible += self._buffer[:cut]
|
|
||||||
self._buffer = self._buffer[cut:]
|
|
||||||
return visible, reasoning
|
|
||||||
|
|
||||||
|
|
||||||
def split_reasoning(text: str) -> tuple[str, str]:
|
def split_reasoning(text: str) -> tuple[str, str]:
|
||||||
"""Turn-level final split of one complete reply: everything through the
|
"""Turn-level final split of one complete reply: everything through the
|
||||||
FIRST ``</think>`` (opened explicitly or implicitly) is reasoning, later
|
FIRST ``</think>`` (opened explicitly or implicitly) is reasoning, later
|
||||||
explicit ``<think>...</think>`` blocks are reasoning too, the rest is the
|
explicit ``<think>...</think>`` blocks are reasoning too, the rest is the
|
||||||
visible reply. Belt-and-braces for streams that leaked past HOLD_LIMIT —
|
visible reply. In ``stream`` mode the closer survives the adapter, so this
|
||||||
the archived reply and its receipt always bind the clean text."""
|
split is the real persistence guard; in ``buffer_turn`` mode it has
|
||||||
|
already been applied inside the gate (kept as the one shared definition)."""
|
||||||
rest = text
|
rest = text
|
||||||
reasoning = ""
|
reasoning = ""
|
||||||
close_at = rest.find(THINK_CLOSE)
|
close_at = rest.find(THINK_CLOSE)
|
||||||
|
|||||||
@@ -58,6 +58,7 @@ def build_llm(cfg: LLMConfig, *, environ: Mapping[str, str] | None = None) -> LL
|
|||||||
api_key=api_key,
|
api_key=api_key,
|
||||||
timeout=cfg.timeout,
|
timeout=cfg.timeout,
|
||||||
languages=frozenset(cfg.languages),
|
languages=frozenset(cfg.languages),
|
||||||
|
reasoning=cfg.reasoning,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -121,12 +121,21 @@ class ChatApiLLMStream(LLMStream):
|
|||||||
"""Stop this generation's cached playout through the SDK's own
|
"""Stop this generation's cached playout through the SDK's own
|
||||||
SpeechHandle.interrupt (the same path user barge-in takes), so a
|
SpeechHandle.interrupt (the same path user barge-in takes), so a
|
||||||
server-side cancel stops already-synthesized audio — not just
|
server-side cancel stops already-synthesized audio — not just
|
||||||
generation. Any failure here must not mask the cancelled end."""
|
generation. force=True: a programmatic user-facing interrupt must
|
||||||
|
also stop speech created with allow_interruptions=False (a non-force
|
||||||
|
interrupt raises RuntimeError there, HEL-909 evidence B2). A failure
|
||||||
|
is logged loudly, never silently faked."""
|
||||||
handle = self.speech_handle
|
handle = self.speech_handle
|
||||||
if handle is None:
|
if handle is None:
|
||||||
return
|
return
|
||||||
with contextlib.suppress(Exception):
|
try:
|
||||||
handle.interrupt()
|
handle.interrupt(force=True)
|
||||||
|
except Exception as exc:
|
||||||
|
print(
|
||||||
|
f"[cybergril-worker] speech interrupt after server cancel failed: "
|
||||||
|
f"{type(exc).__name__}: {exc}",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
async def _run(self) -> None:
|
async def _run(self) -> None:
|
||||||
if self.server_cancelled:
|
if self.server_cancelled:
|
||||||
|
|||||||
@@ -43,6 +43,7 @@ import contextlib
|
|||||||
import inspect
|
import inspect
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
from collections.abc import Callable
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from livekit.agents import Agent, AgentSession, JobContext, JobProcess, WorkerOptions, cli
|
from livekit.agents import Agent, AgentSession, JobContext, JobProcess, WorkerOptions, cli
|
||||||
@@ -213,6 +214,67 @@ def bind_voice_session(session: AgentSession, *, llm: CyberGrillLLM,
|
|||||||
stt.agent_speaking = getattr(ev, "new_state", "") == "speaking"
|
stt.agent_speaking = getattr(ev, "new_state", "") == "speaking"
|
||||||
|
|
||||||
|
|
||||||
|
INTERRUPT_TOPIC = "cg.interrupt"
|
||||||
|
"""R2(HEL-909):网页发文字时经房间数据通道发出的本通打断消息 topic。
|
||||||
|
web/src/voice 的 hush() 在本地静音反馈之外 publish 它(reliable);worker 在
|
||||||
|
本房间订阅。只复用 LiveKit 自己的数据通道与 session.interrupt,不建第二套
|
||||||
|
调度引擎。"""
|
||||||
|
|
||||||
|
|
||||||
|
def bind_interrupt_channel(room, session) -> Callable[[], None]:
|
||||||
|
"""订阅本房间的 cg.interrupt 打断消息,走 SDK 自己的 programmatic 打断.
|
||||||
|
|
||||||
|
绑定与校验(HEL-909 R2 裁定):
|
||||||
|
- 本通绑定是通道固有的:数据消息只到达本房间的订阅者,旧通的消息物理
|
||||||
|
上到不了新通的 worker;
|
||||||
|
- topic 必须精确等于 cg.interrupt(cg.subtitle 等一律忽略);
|
||||||
|
- 发送者必须是远端参与者(用户网页);worker 自己/服务端来源不处理;
|
||||||
|
- 载荷必须是 {"v": 1},其他形状丢弃;
|
||||||
|
- session.interrupt(force=True):程序性用户打断按 SDK 文档越过
|
||||||
|
allow_interruptions=False 的排队语音(真实句柄在非 force 下会抛
|
||||||
|
RuntimeError,HEL-909 evidence B2);失败一律大声留痕,不静默假装
|
||||||
|
成功。session.interrupt 未启动时抛 RuntimeError 同样留痕返回。
|
||||||
|
|
||||||
|
返回卸载函数(JobContext.add_shutdown_callback 注册,会话结束卸载)。
|
||||||
|
"""
|
||||||
|
def on_data(msg) -> None:
|
||||||
|
if getattr(msg, "topic", None) != INTERRUPT_TOPIC:
|
||||||
|
return
|
||||||
|
participant = getattr(msg, "participant", None)
|
||||||
|
if participant is None or getattr(participant, "is_local", False):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
payload = json.loads(bytes(msg.data).decode("utf-8"))
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
if not isinstance(payload, dict) or payload.get("v") != 1:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
fut = session.interrupt(force=True)
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[cybergril-worker] interrupt message failed: "
|
||||||
|
f"{type(exc).__name__}: {exc}", flush=True)
|
||||||
|
return
|
||||||
|
if inspect.isawaitable(fut):
|
||||||
|
|
||||||
|
async def _drain(f) -> None:
|
||||||
|
try:
|
||||||
|
await f
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"[cybergril-worker] interrupt completion failed: "
|
||||||
|
f"{type(exc).__name__}: {exc}", flush=True)
|
||||||
|
|
||||||
|
asyncio.get_running_loop().create_task(_drain(fut))
|
||||||
|
|
||||||
|
room.on("data_received", on_data)
|
||||||
|
|
||||||
|
def unload() -> None:
|
||||||
|
with contextlib.suppress(Exception):
|
||||||
|
room.off("data_received", on_data)
|
||||||
|
|
||||||
|
return unload
|
||||||
|
|
||||||
|
|
||||||
async def speak_proactive_row(session: AgentSession, bus: SubtitleBus, row: dict) -> None:
|
async def speak_proactive_row(session: AgentSession, bus: SubtitleBus, row: dict) -> None:
|
||||||
"""念一条已落库 proactive 行,并给同源 message_id 的字幕起止/打断事件.
|
"""念一条已落库 proactive 行,并给同源 message_id 的字幕起止/打断事件.
|
||||||
|
|
||||||
@@ -312,6 +374,10 @@ async def entrypoint(ctx: JobContext) -> None:
|
|||||||
)
|
)
|
||||||
bind_voice_session(session, llm=llm, ledger=ledger, interim=interim,
|
bind_voice_session(session, llm=llm, ledger=ledger, interim=interim,
|
||||||
bus=bus, stt=stt, receipts=receipts)
|
bus=bus, stt=stt, receipts=receipts)
|
||||||
|
# R2:本通文字打断通道(cg.interrupt)——LLM 已 final、TTS 仍在播时,
|
||||||
|
# 服务端已无活动轮可取消,这里是唯一权威打断路径;会话结束卸载。
|
||||||
|
ctx.add_shutdown_callback(
|
||||||
|
bind_interrupt_channel(ctx.room, session))
|
||||||
|
|
||||||
await ctx.connect()
|
await ctx.connect()
|
||||||
await session.start(room=ctx.room, agent=Agent(instructions=INSTRUCTIONS_PLACEHOLDER))
|
await session.start(room=ctx.room, agent=Agent(instructions=INSTRUCTIONS_PLACEHOLDER))
|
||||||
|
|||||||
@@ -1,12 +1,24 @@
|
|||||||
"""HEL-908 reasoning separation tests: the adapter seam between raw model
|
"""HEL-908/909 reasoning separation tests: the adapter seam between raw model
|
||||||
output and user-visible content, against the REAL ReasoningSeparator and the
|
output and user-visible content, against the REAL ReasoningSeparator /
|
||||||
REAL OpenAICompatLLM adapter (fake SSE transports, zero network).
|
TurnReasoningGate and the REAL OpenAICompatLLM adapter (fake SSE transports,
|
||||||
|
zero network).
|
||||||
|
|
||||||
The recorded evidence (HEL-907 window, companion.db row id=12 and
|
Two explicit modes (models.yaml chooses per endpoint):
|
||||||
tts-window-tail.log) is replayed verbatim: the dialogue endpoint streamed
|
|
||||||
"Considering the limited time by the user, ... now.\\n</think>\\n\\n大圣,…"
|
- ``buffer_turn`` (the product's tiers): the recorded implicit-open shape —
|
||||||
into ``content`` — implicit-open reasoning (no ``<think>``, the chat template
|
reasoning with NO ``<think>`` opener and a stray ``</think>`` closer — can
|
||||||
pre-fills it) that reached subtitles AND TTS.
|
NEVER be separated safely while streaming (HEL-909 review: sentence-enders
|
||||||
|
and >128-char holds leaked on real token arrival, and the adapter consumed
|
||||||
|
the closer so the turn-level split could not recover). The gate emits
|
||||||
|
nothing during generation, splits the whole turn at the end, and only the
|
||||||
|
clean visible text is ever emitted.
|
||||||
|
- ``stream`` (default): explicit ``<think>`` blocks and the DeepSeek-style
|
||||||
|
``reasoning_content`` field; a stray closer passes through UNMODIFIED so
|
||||||
|
api/app.py's turn-level split keeps its signal.
|
||||||
|
|
||||||
|
The recorded evidence (HEL-907 companion.db row id=12 / tts-window-tail.log)
|
||||||
|
and both HEL-909 review reproductions are replayed verbatim at every chunk
|
||||||
|
size.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -22,8 +34,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
|||||||
|
|
||||||
from cybergril.providers.llm.openai_compat import OpenAICompatLLM
|
from cybergril.providers.llm.openai_compat import OpenAICompatLLM
|
||||||
from cybergril.providers.llm.reasoning import (
|
from cybergril.providers.llm.reasoning import (
|
||||||
HOLD_LIMIT,
|
|
||||||
ReasoningSeparator,
|
ReasoningSeparator,
|
||||||
|
TurnReasoningGate,
|
||||||
split_reasoning,
|
split_reasoning,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -43,30 +55,89 @@ RECORDED_VISIBLE = (
|
|||||||
"连肉都没有,怕硌着您的金箍棒。"
|
"连肉都没有,怕硌着您的金箍棒。"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# HEL-909 review reproductions A1/A2: the common shapes the old hold leaked.
|
||||||
|
A1_RAW = "private reasoning " * 20 + "</think>回答。"
|
||||||
|
A2_RAW = "我先想一下。内部思考</think>回答。"
|
||||||
|
|
||||||
def run_stream(text: str, chunk_size: int) -> tuple[str, str]:
|
|
||||||
|
def run_gate(raw: str, chunk_size: int) -> tuple[list[str], str]:
|
||||||
|
"""Feed raw through a whole-turn gate; returns (visible deltas, reasoning)."""
|
||||||
|
gate = TurnReasoningGate()
|
||||||
|
emitted: list[str] = []
|
||||||
|
for i in range(0, len(raw), chunk_size):
|
||||||
|
visible, _ = gate.feed(raw[i : i + chunk_size])
|
||||||
|
emitted.append(visible)
|
||||||
|
visible, _ = gate.finish()
|
||||||
|
emitted.append(visible)
|
||||||
|
return emitted, gate.reasoning + split_reasoning(raw)[1]
|
||||||
|
|
||||||
|
|
||||||
|
def run_separator(raw: str, chunk_size: int) -> tuple[str, str]:
|
||||||
separator = ReasoningSeparator()
|
separator = ReasoningSeparator()
|
||||||
visible_parts: list[str] = []
|
visible_parts: list[str] = []
|
||||||
for i in range(0, len(text), chunk_size):
|
for i in range(0, len(raw), chunk_size):
|
||||||
v, _ = separator.feed(text[i : i + chunk_size])
|
v, _ = separator.feed(raw[i : i + chunk_size])
|
||||||
visible_parts.append(v)
|
visible_parts.append(v)
|
||||||
v, _ = separator.finish()
|
v, _ = separator.finish()
|
||||||
visible_parts.append(v)
|
visible_parts.append(v)
|
||||||
return "".join(visible_parts), separator.reasoning
|
return "".join(visible_parts), separator.reasoning
|
||||||
|
|
||||||
|
|
||||||
# ── streaming separator ──────────────────────────────────────────────────────
|
# ── buffer_turn: the recorded and review shapes never leak ───────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("raw", [RECORDED_RAW, A1_RAW, A2_RAW],
|
||||||
|
ids=["recorded-id12", "review-A1", "review-A2"])
|
||||||
|
@pytest.mark.parametrize("chunk", [1, 2, 3, 5, 7, 13, 1000])
|
||||||
|
def test_buffer_turn_emits_only_clean_text_at_any_chunk_size(raw, chunk):
|
||||||
|
emitted, reasoning = run_gate(raw, chunk)
|
||||||
|
assert "".join(emitted) in ("回答。", RECORDED_VISIBLE)
|
||||||
|
# nothing AT ALL is emitted before the finish — the stream cannot leak
|
||||||
|
assert emitted[:-1] == [""] * (len(emitted) - 1)
|
||||||
|
assert reasoning.endswith("</think>"), "the reasoning landed on the diagnostics side"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("chunk", [1, 2, 3, 5, 7, 13, 1000])
|
@pytest.mark.parametrize("chunk", [1, 2, 3, 5, 7, 13, 1000])
|
||||||
def test_recorded_stream_leaks_nothing_at_any_chunk_size(chunk):
|
def test_buffer_turn_recorded_stream_zero_leak(chunk):
|
||||||
visible, reasoning = run_stream(RECORDED_RAW, chunk)
|
emitted, reasoning = run_gate(RECORDED_RAW, chunk)
|
||||||
assert visible == RECORDED_VISIBLE
|
assert "".join(emitted) == RECORDED_VISIBLE
|
||||||
assert reasoning == RECORDED_REASONING
|
assert reasoning == RECORDED_REASONING
|
||||||
|
|
||||||
|
|
||||||
|
def test_buffer_turn_unclosed_explicit_think_yields_nothing():
|
||||||
|
gate = TurnReasoningGate()
|
||||||
|
gate.feed("<think>只想不说到结尾")
|
||||||
|
visible, reasoning = gate.finish()
|
||||||
|
assert visible == ""
|
||||||
|
assert "<think>" in reasoning
|
||||||
|
|
||||||
|
|
||||||
|
def test_buffer_turn_normal_chinese_and_english_preserved():
|
||||||
|
zh = "今天水母网我背,你负责举着泡泡喊“出发”。\n\n抓完水母我们就坐在海滩上。"
|
||||||
|
en = ("Sure, listening to you. A reply that happens to be English is "
|
||||||
|
"normal content and must survive the gate untouched.")
|
||||||
|
for text in (zh, en):
|
||||||
|
gate = TurnReasoningGate()
|
||||||
|
gate.feed(text)
|
||||||
|
visible, _ = gate.finish()
|
||||||
|
assert visible == text
|
||||||
|
assert gate.reasoning == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_buffer_turn_separates_field_reasoning_too():
|
||||||
|
gate = TurnReasoningGate()
|
||||||
|
gate.feed_field("先在心里盘算")
|
||||||
|
gate.feed(RECORDED_RAW)
|
||||||
|
visible, reasoning = gate.finish()
|
||||||
|
assert visible == RECORDED_VISIBLE
|
||||||
|
assert reasoning.startswith("先在心里盘算")
|
||||||
|
|
||||||
|
|
||||||
|
# ── stream mode: explicit tags / field; stray closer passes through ─────────
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("split_at", range(1, len("<think>隐藏</think>可见")))
|
@pytest.mark.parametrize("split_at", range(1, len("<think>隐藏</think>可见")))
|
||||||
def test_explicit_tags_split_across_chunks(split_at):
|
def test_stream_explicit_tags_split_across_chunks(split_at):
|
||||||
text = "<think>隐藏</think>可见"
|
text = "<think>隐藏</think>可见"
|
||||||
head, tail = text[:split_at], text[split_at:]
|
head, tail = text[:split_at], text[split_at:]
|
||||||
separator = ReasoningSeparator()
|
separator = ReasoningSeparator()
|
||||||
@@ -76,64 +147,37 @@ def test_explicit_tags_split_across_chunks(split_at):
|
|||||||
assert v1 + v2 + v3 == "可见"
|
assert v1 + v2 + v3 == "可见"
|
||||||
|
|
||||||
|
|
||||||
def test_explicit_think_block_anywhere_in_stream():
|
def test_stream_explicit_block_anywhere_is_filtered():
|
||||||
visible, _ = run_stream("先说。<think>中途思考</think>后说。", 3)
|
visible, reasoning = run_separator("先说。<think>中途思考</think>后说。", 3)
|
||||||
assert visible == "先说。后说。"
|
assert visible == "先说。后说。"
|
||||||
|
assert "中途思考" in reasoning
|
||||||
|
|
||||||
|
|
||||||
def test_unclosed_think_yields_no_visible_text():
|
def test_stream_unclosed_think_yields_no_visible_text():
|
||||||
separator = ReasoningSeparator()
|
separator = ReasoningSeparator()
|
||||||
visible, _ = separator.feed("<think>只想不说到结尾")
|
visible, _ = separator.feed("<think>只想不说到结尾")
|
||||||
tail_visible, _ = separator.finish()
|
tail, _ = separator.finish()
|
||||||
assert visible + tail_visible == ""
|
assert visible + tail == ""
|
||||||
|
|
||||||
|
|
||||||
def test_safe_no_think_chinese_streams_immediately():
|
def test_stream_normal_chinese_and_english_passthrough():
|
||||||
separator = ReasoningSeparator()
|
zh = "哈哈,那走吧,派大星。今天水母网我背。"
|
||||||
visible, _ = separator.feed("哈哈,那走吧。")
|
en = "Plain English reply, kept fully. Second sentence.\nAnd a newline."
|
||||||
assert visible == "哈哈,那走吧。"
|
for text in (zh, en):
|
||||||
|
visible, reasoning = run_separator(text, 4)
|
||||||
|
assert visible == text
|
||||||
|
assert reasoning == ""
|
||||||
|
|
||||||
|
|
||||||
def test_normal_chinese_body_preserved_byte_for_byte():
|
def test_stream_stray_closer_passes_through_untouched():
|
||||||
text = "今天水母网我背,你负责举着泡泡喊“出发”。\n\n抓完水母我们就坐在海滩上。"
|
"""The HEL-909 defect was the adapter DELETING the closer; the signal must
|
||||||
visible, reasoning = run_stream(text, 4)
|
survive so the app-level turn split can still act on it."""
|
||||||
assert visible == text
|
visible, reasoning = run_separator(A2_RAW, 5)
|
||||||
|
assert visible == A2_RAW, "stream mode passes stray closers through"
|
||||||
assert reasoning == ""
|
assert reasoning == ""
|
||||||
|
|
||||||
|
|
||||||
def test_normal_english_body_preserved_byte_for_byte():
|
# ── turn-level final split (shared definition; real persistence guard) ──────
|
||||||
text = (
|
|
||||||
"Sure, listening to you. A reply that happens to be English is normal "
|
|
||||||
"user-facing content and must survive the separator untouched, "
|
|
||||||
"including newlines\nand punctuation."
|
|
||||||
)
|
|
||||||
visible, reasoning = run_stream(text, 5)
|
|
||||||
assert visible == text
|
|
||||||
assert reasoning == ""
|
|
||||||
|
|
||||||
|
|
||||||
def test_untagged_short_stream_flushes_at_finish():
|
|
||||||
separator = ReasoningSeparator()
|
|
||||||
v, _ = separator.feed("嗯") # below HOLD_LIMIT, no sentence end: held
|
|
||||||
assert v == ""
|
|
||||||
v, _ = separator.finish()
|
|
||||||
assert v == "嗯"
|
|
||||||
|
|
||||||
|
|
||||||
def test_residual_beyond_hold_limit_is_cleaned_at_turn_level():
|
|
||||||
"""Documented residual: implicit-open reasoning whose closer lies beyond
|
|
||||||
HOLD_LIMIT streams partially visible; the turn-level split still returns
|
|
||||||
the clean reply for persistence and the receipt."""
|
|
||||||
long_reasoning = "r" * (HOLD_LIMIT + 50) + "</think>"
|
|
||||||
text = long_reasoning + "干净的回答。"
|
|
||||||
streamed_visible, _ = run_stream(text, 64)
|
|
||||||
assert streamed_visible != "干净的回答。" # partial streaming leak (known)
|
|
||||||
final_visible, dropped = split_reasoning(text)
|
|
||||||
assert final_visible == "干净的回答。"
|
|
||||||
assert long_reasoning in dropped
|
|
||||||
|
|
||||||
|
|
||||||
# ── turn-level final split ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
def test_split_recorded_row_is_clean():
|
def test_split_recorded_row_is_clean():
|
||||||
@@ -142,6 +186,11 @@ def test_split_recorded_row_is_clean():
|
|||||||
assert reasoning == RECORDED_REASONING
|
assert reasoning == RECORDED_REASONING
|
||||||
|
|
||||||
|
|
||||||
|
def test_split_review_shapes_are_clean():
|
||||||
|
assert split_reasoning(A1_RAW)[0] == "回答。"
|
||||||
|
assert split_reasoning(A2_RAW)[0] == "回答。"
|
||||||
|
|
||||||
|
|
||||||
def test_split_explicit_blocks_and_unclosed():
|
def test_split_explicit_blocks_and_unclosed():
|
||||||
assert split_reasoning("<think>a</think>答") == ("答", "<think>a</think>")
|
assert split_reasoning("<think>a</think>答") == ("答", "<think>a</think>")
|
||||||
assert split_reasoning("<think>a") == ("", "<think>a")
|
assert split_reasoning("<think>a") == ("", "<think>a")
|
||||||
@@ -165,82 +214,95 @@ def sse_response(deltas: list[dict]) -> httpx.Response:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
async def collect_provider(provider: OpenAICompatLLM) -> tuple[str, str]:
|
def chunked(raw: str, size: int) -> list[dict]:
|
||||||
text = reasoning = ""
|
return [{"content": raw[i : i + size]} for i in range(0, len(raw), size)]
|
||||||
async for delta in provider.stream([type("M", (), {"role": "user", "content": "你好"})()]):
|
|
||||||
text += delta.text
|
|
||||||
|
def make_provider(mode: str, pieces: list[dict]) -> OpenAICompatLLM:
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
if request.url.path.endswith("/models"):
|
||||||
|
return httpx.Response(200, json={"data": [{"id": "m"}]})
|
||||||
|
return sse_response(pieces)
|
||||||
|
|
||||||
|
provider = OpenAICompatLLM(base_url="http://isolated/v1", model="m", reasoning=mode)
|
||||||
|
provider._client = httpx.AsyncClient(
|
||||||
|
base_url="http://isolated/v1", transport=httpx.MockTransport(handler)
|
||||||
|
)
|
||||||
|
return provider
|
||||||
|
|
||||||
|
|
||||||
|
async def collect(provider: OpenAICompatLLM) -> tuple[list[str], str, int]:
|
||||||
|
"""Returns (visible delta sequence — empty strings kept in place so the
|
||||||
|
timing of emissions is visible, reasoning, index of first non-empty)."""
|
||||||
|
text_deltas: list[str] = []
|
||||||
|
reasoning = ""
|
||||||
|
async for delta in provider.stream(
|
||||||
|
[type("M", (), {"role": "user", "content": "你好"})()]
|
||||||
|
):
|
||||||
|
text_deltas.append(delta.text)
|
||||||
reasoning += delta.reasoning
|
reasoning += delta.reasoning
|
||||||
return text, reasoning
|
first = next((i for i, t in enumerate(text_deltas) if t), len(text_deltas))
|
||||||
|
return text_deltas, reasoning, first
|
||||||
|
|
||||||
|
|
||||||
async def test_adapter_separates_reasoning_field_from_content():
|
@pytest.mark.parametrize("mode,raw", [
|
||||||
seen: list[httpx.Request] = []
|
("buffer_turn", RECORDED_RAW),
|
||||||
|
("buffer_turn", A1_RAW),
|
||||||
def handler(request: httpx.Request) -> httpx.Response:
|
("buffer_turn", A2_RAW),
|
||||||
seen.append(request)
|
])
|
||||||
return sse_response(
|
@pytest.mark.parametrize("chunk", [1, 7, 60])
|
||||||
[
|
async def test_adapter_buffer_turn_never_leaks_review_shapes(mode, raw, chunk):
|
||||||
{"reasoning_content": "先在心里盘算一下"},
|
provider = make_provider(mode, chunked(raw, chunk))
|
||||||
{"content": ""},
|
|
||||||
{"content": RECORDED_VISIBLE[:4]},
|
|
||||||
{"content": RECORDED_VISIBLE[4:]},
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
provider = OpenAICompatLLM(base_url="http://isolated/v1", model="m")
|
|
||||||
provider._client = httpx.AsyncClient(
|
|
||||||
base_url="http://isolated/v1", transport=httpx.MockTransport(handler)
|
|
||||||
)
|
|
||||||
try:
|
try:
|
||||||
text, reasoning = await collect_provider(provider)
|
text_deltas, reasoning, _ = await collect(provider)
|
||||||
assert text == RECORDED_VISIBLE
|
assert "".join(text_deltas) in ("回答。", RECORDED_VISIBLE)
|
||||||
assert reasoning == "先在心里盘算一下"
|
# the whole generation ran inside the gate: the first visible delta is
|
||||||
|
# the LAST content delta (documented wait cost, not a hidden leak)
|
||||||
|
non_empty = [t for t in text_deltas if t]
|
||||||
|
assert len(non_empty) == 1
|
||||||
|
assert "private reasoning" not in "".join(text_deltas)
|
||||||
|
assert "我先想一下" not in "".join(text_deltas)
|
||||||
|
assert "Considering" not in "".join(text_deltas)
|
||||||
|
assert reasoning # the reasoning went to the diagnostics channel only
|
||||||
finally:
|
finally:
|
||||||
await provider.aclose()
|
await provider.aclose()
|
||||||
|
|
||||||
|
|
||||||
async def test_adapter_filters_recorded_inline_leak():
|
async def test_adapter_stream_mode_filters_explicit_blocks_only():
|
||||||
def handler(request: httpx.Request) -> httpx.Response:
|
pieces = [{"content": "先说。"}, {"content": "<th"}, {"content": "ink>隐藏"},
|
||||||
# recorded id=12 shape: reasoning text + stray closer, no opening tag;
|
{"content": "</th"}, {"content": "ink>后说。"}]
|
||||||
# chunks split the closing tag across SSE frames
|
provider = make_provider("stream", pieces)
|
||||||
return sse_response(
|
|
||||||
[
|
|
||||||
{"content": RECORDED_RAW[:60]},
|
|
||||||
{"content": RECORDED_RAW[60:104]},
|
|
||||||
{"content": RECORDED_RAW[104:112]},
|
|
||||||
{"content": RECORDED_RAW[112:]},
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
provider = OpenAICompatLLM(base_url="http://isolated/v1", model="m")
|
|
||||||
provider._client = httpx.AsyncClient(
|
|
||||||
base_url="http://isolated/v1", transport=httpx.MockTransport(handler)
|
|
||||||
)
|
|
||||||
try:
|
try:
|
||||||
text, reasoning = await collect_provider(provider)
|
text_deltas, reasoning, _ = await collect(provider)
|
||||||
assert text == RECORDED_VISIBLE, "reasoning must never reach delta.text"
|
assert "".join(text_deltas) == "先说。后说。"
|
||||||
assert "Considering" in reasoning and "</think>" in reasoning
|
assert "隐藏" in reasoning
|
||||||
finally:
|
finally:
|
||||||
await provider.aclose()
|
await provider.aclose()
|
||||||
|
|
||||||
|
|
||||||
async def test_adapter_preserves_english_content_and_flags():
|
async def test_adapter_stream_mode_routes_reasoning_field():
|
||||||
def handler(request: httpx.Request) -> httpx.Response:
|
pieces = [{"reasoning_content": "先盘算"}, {"content": RECORDED_VISIBLE[:4]},
|
||||||
return sse_response(
|
{"content": RECORDED_VISIBLE[4:]}]
|
||||||
[
|
provider = make_provider("stream", pieces)
|
||||||
{"content": "Plain English reply, kept fully. "},
|
|
||||||
{"reasoning_content": "internal note"},
|
|
||||||
{"content": "Second sentence."},
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
provider = OpenAICompatLLM(base_url="http://isolated/v1", model="m")
|
|
||||||
provider._client = httpx.AsyncClient(
|
|
||||||
base_url="http://isolated/v1", transport=httpx.MockTransport(handler)
|
|
||||||
)
|
|
||||||
try:
|
try:
|
||||||
text, reasoning = await collect_provider(provider)
|
text_deltas, reasoning, _ = await collect(provider)
|
||||||
assert text == "Plain English reply, kept fully. Second sentence."
|
assert "".join(text_deltas) == RECORDED_VISIBLE
|
||||||
assert reasoning == "internal note"
|
assert "先盘算" in reasoning
|
||||||
finally:
|
finally:
|
||||||
await provider.aclose()
|
await provider.aclose()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_adapter_buffer_turn_routes_reasoning_field():
|
||||||
|
pieces = [{"reasoning_content": "先盘算"}, *chunked(RECORDED_RAW, 13)]
|
||||||
|
provider = make_provider("buffer_turn", pieces)
|
||||||
|
try:
|
||||||
|
text_deltas, reasoning, _ = await collect(provider)
|
||||||
|
assert "".join(text_deltas) == RECORDED_VISIBLE
|
||||||
|
assert reasoning.startswith("先盘算")
|
||||||
|
finally:
|
||||||
|
await provider.aclose()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_adapter_rejects_unknown_mode():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
OpenAICompatLLM(base_url="http://x/v1", model="m", reasoning="nonsense")
|
||||||
|
|||||||
@@ -55,6 +55,13 @@ RECORDED_VISIBLE = (
|
|||||||
RECORDED_CHUNKS = [RECORDED_RAW[:60], RECORDED_RAW[60:104],
|
RECORDED_CHUNKS = [RECORDED_RAW[:60], RECORDED_RAW[60:104],
|
||||||
RECORDED_RAW[104:112], RECORDED_RAW[112:]]
|
RECORDED_RAW[104:112], RECORDED_RAW[112:]]
|
||||||
|
|
||||||
|
# HEL-909 review reproductions A1/A2 — the shapes the old streaming hold
|
||||||
|
# leaked into persistence and the receipt on real token arrival.
|
||||||
|
A1_RAW = "private reasoning " * 20 + "</think>回答。"
|
||||||
|
A2_RAW = "我先想一下。内部思考</think>回答。"
|
||||||
|
A1_CHUNKS = [A1_RAW[i:i + 13] for i in range(0, len(A1_RAW), 13)]
|
||||||
|
A2_CHUNKS = [A2_RAW[i:i + 5] for i in range(0, len(A2_RAW), 5)]
|
||||||
|
|
||||||
|
|
||||||
class ScriptedLLM(R2LLM):
|
class ScriptedLLM(R2LLM):
|
||||||
"""R2 controlled provider with HEL-908 scripted dialogue behaviour."""
|
"""R2 controlled provider with HEL-908 scripted dialogue behaviour."""
|
||||||
@@ -120,8 +127,10 @@ def sse_chunks(raw_pieces: list[str]) -> bytes:
|
|||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
def adapter_app(tmp_path):
|
def adapter_app(tmp_path):
|
||||||
"""The REAL OpenAICompatLLM behind the real app, fed by a scripted
|
"""The REAL OpenAICompatLLM behind the real app, fed by a scripted
|
||||||
OpenAI-compatible SSE gateway (the seam where the reasoning separation
|
OpenAI-compatible SSE gateway — in buffer_turn mode, exactly what
|
||||||
lives, exercised end to end)."""
|
config/models.yaml configures for the product's tiers (the endpoint
|
||||||
|
streams implicit-open reasoning, so the whole turn is gated before any
|
||||||
|
content is emitted)."""
|
||||||
from cybergril.providers.llm.openai_compat import OpenAICompatLLM
|
from cybergril.providers.llm.openai_compat import OpenAICompatLLM
|
||||||
|
|
||||||
state = {"answers": [sse_chunks(RECORDED_CHUNKS)]}
|
state = {"answers": [sse_chunks(RECORDED_CHUNKS)]}
|
||||||
@@ -134,7 +143,8 @@ def adapter_app(tmp_path):
|
|||||||
200, headers={"content-type": "text/event-stream"}, content=answer
|
200, headers={"content-type": "text/event-stream"}, content=answer
|
||||||
)
|
)
|
||||||
|
|
||||||
provider = OpenAICompatLLM(base_url="http://recorded/v1", model="recorded")
|
provider = OpenAICompatLLM(base_url="http://recorded/v1", model="recorded",
|
||||||
|
reasoning="buffer_turn")
|
||||||
provider._client = httpx.AsyncClient(
|
provider._client = httpx.AsyncClient(
|
||||||
base_url="http://recorded/v1", transport=httpx.MockTransport(handler)
|
base_url="http://recorded/v1", transport=httpx.MockTransport(handler)
|
||||||
)
|
)
|
||||||
@@ -306,6 +316,32 @@ async def test_think_leak_replay_clean_deltas_archive_and_receipt(adapter_app):
|
|||||||
assert await api_client(app).acknowledge(turn.receipt) is True
|
assert await api_client(app).acknowledge(turn.receipt) is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("raw,pieces,label", [
|
||||||
|
(A1_RAW, A1_CHUNKS, "review-A1-long-implicit"),
|
||||||
|
(A2_RAW, A2_CHUNKS, "review-A2-chinese-sentence"),
|
||||||
|
])
|
||||||
|
async def test_review_shapes_never_reach_stream_archive_or_receipt(
|
||||||
|
adapter_app, raw, pieces, label):
|
||||||
|
"""HEL-909 A1/A2 verbatim, small chunks (real token arrival): through the
|
||||||
|
real OpenAICompatLLM -> real app -> persistence and receipt. These exact
|
||||||
|
shapes leaked wholesale in the previous delivery; the buffer_turn gate
|
||||||
|
keeps them out of chat.delta, the archive and the receipt."""
|
||||||
|
app, state, epoch = adapter_app
|
||||||
|
state["answers"] = [sse_chunks(pieces)]
|
||||||
|
deltas: list[str] = []
|
||||||
|
turn = await api_client(app).send_turn(label, epoch=epoch,
|
||||||
|
on_delta=deltas.append)
|
||||||
|
assert turn.completed
|
||||||
|
assert turn.text == "回答。", "the archived reply is the clean visible text"
|
||||||
|
assert "".join(deltas) == "回答。", "chat.delta (subtitles/TTS) stays clean"
|
||||||
|
assert "private reasoning" not in turn.text and "内部思考" not in turn.text
|
||||||
|
archived = [r for r in rows(app) if r["role"] == "companion"]
|
||||||
|
assert archived and archived[0]["text"] == "回答。"
|
||||||
|
assert turn.receipt and turn.receipt["text_hash"] == \
|
||||||
|
hashlib.sha256("回答。".encode()).hexdigest()
|
||||||
|
assert await api_client(app).acknowledge(turn.receipt) is True
|
||||||
|
|
||||||
|
|
||||||
async def test_unclosed_think_fails_closed(adapter_app):
|
async def test_unclosed_think_fails_closed(adapter_app):
|
||||||
"""A reply that never leaves its think phase has no visible content: the
|
"""A reply that never leaves its think phase has no visible content: the
|
||||||
turn fails honestly instead of archiving reasoning as her words."""
|
turn fails honestly instead of archiving reasoning as her words."""
|
||||||
@@ -343,6 +379,24 @@ async def test_completed_replay_and_failed_retry_still_work(rework):
|
|||||||
assert len(provider.calls) == calls + 2
|
assert len(provider.calls) == calls + 2
|
||||||
|
|
||||||
|
|
||||||
|
async def test_text_after_clean_voice_final_is_a_normal_turn(rework):
|
||||||
|
"""R2 app-integration record (evidence C sequence): the voice turn
|
||||||
|
already finished clean — the server has no active turn left to cancel,
|
||||||
|
so only the worker's cg.interrupt channel can stop leftover playout
|
||||||
|
(proved on the real SDK in test_sdk_livekit_hel908). At the app seam the
|
||||||
|
text turn simply proceeds: nothing cancelled, no phantom rows."""
|
||||||
|
app, provider, epoch = rework
|
||||||
|
voice = await api_client(app).send_turn("语音已经说完", epoch=epoch)
|
||||||
|
assert voice.completed
|
||||||
|
text = await api_client(app).send_turn("文字随后到达", epoch=epoch)
|
||||||
|
assert text.completed and text.text == provider.reply
|
||||||
|
table = rows(app)
|
||||||
|
assert not [r for r in table if r["status"] == "cancelled"], \
|
||||||
|
"after a clean final there is nothing to cancel at the app seam"
|
||||||
|
assert len([r for r in table
|
||||||
|
if r["role"] == "companion" and r["status"] == "sent"]) == 2
|
||||||
|
|
||||||
|
|
||||||
async def test_client_treats_cancelled_frame_as_terminal(rework):
|
async def test_client_treats_cancelled_frame_as_terminal(rework):
|
||||||
"""Unit-level protocol check: chat.cancelled ends the turn client-side
|
"""Unit-level protocol check: chat.cancelled ends the turn client-side
|
||||||
without raising (a raise is what made livekit retry it as a network
|
without raising (a raise is what made livekit retry it as a network
|
||||||
|
|||||||
@@ -127,9 +127,11 @@ class InterruptSpy:
|
|||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.interrupts = 0
|
self.interrupts = 0
|
||||||
|
self.forced: list[bool] = []
|
||||||
|
|
||||||
def interrupt(self):
|
def interrupt(self, *, force: bool = False):
|
||||||
self.interrupts += 1
|
self.interrupts += 1
|
||||||
|
self.forced.append(force)
|
||||||
return self
|
return self
|
||||||
|
|
||||||
|
|
||||||
@@ -179,6 +181,8 @@ async def test_mid_stream_cancel_interrupts_handle_and_never_retries():
|
|||||||
assert stream.server_cancelled
|
assert stream.server_cancelled
|
||||||
assert spy.interrupts == 1, \
|
assert spy.interrupts == 1, \
|
||||||
"server-side cancel must stop the cached playout via SpeechHandle.interrupt"
|
"server-side cancel must stop the cached playout via SpeechHandle.interrupt"
|
||||||
|
assert spy.forced == [True], \
|
||||||
|
"programmatic user interrupt goes through with force (HEL-909 R2)"
|
||||||
assert backend.chat_posts == 1
|
assert backend.chat_posts == 1
|
||||||
|
|
||||||
|
|
||||||
@@ -212,6 +216,141 @@ async def test_completed_turn_still_streams_with_receipt():
|
|||||||
assert backend.chat_posts == 1
|
assert backend.chat_posts == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ── R2 (HEL-909): the cg.interrupt data channel for "LLM final, TTS playing" ─
|
||||||
|
|
||||||
|
|
||||||
|
class HandleSessionBridge:
|
||||||
|
"""Stands exactly where AgentSession.interrupt stands in production. The
|
||||||
|
SDK's AgentActivity.interrupt (verified read-only on .12,
|
||||||
|
agent_activity.py:1932-1934) forwards force to the LIVE SpeechHandle's
|
||||||
|
own interrupt(); everything below this delegate is real SDK machinery —
|
||||||
|
the real handle, the real pending playout wait, the real interrupt
|
||||||
|
semantics (no Spy counting masquerading as a stopped buffer)."""
|
||||||
|
|
||||||
|
def __init__(self, handle) -> None:
|
||||||
|
self.handle = handle
|
||||||
|
self.calls: list[bool] = []
|
||||||
|
|
||||||
|
def interrupt(self, *, force: bool = False):
|
||||||
|
self.calls.append(force)
|
||||||
|
return self.handle.interrupt(force=force)
|
||||||
|
|
||||||
|
|
||||||
|
def _data_packet(topic: str, payload: bytes, sender: str = "user-1"):
|
||||||
|
"""Structured stand-in for rtc's DataPacket (offline: no room server).
|
||||||
|
sender: 'user-1' remote user page | 'self' the agent | 'server'."""
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
participant = None
|
||||||
|
if sender != "server":
|
||||||
|
participant = SimpleNamespace(is_local=sender == "self", identity=sender)
|
||||||
|
return SimpleNamespace(topic=topic, data=payload, participant=participant)
|
||||||
|
|
||||||
|
|
||||||
|
def _quiet_await(handle) -> None:
|
||||||
|
"""Clean the interrupt-timeout timer a real interrupt arms (5s) so the
|
||||||
|
loop can close without pending-callback noise."""
|
||||||
|
timer = getattr(handle, "_interrupt_timeout_handle", None)
|
||||||
|
if timer is not None:
|
||||||
|
timer.cancel()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_interrupt_message_stops_pending_playout_after_final():
|
||||||
|
"""Reviewer evidence-C shape: the old voice turn already ended CLEAN
|
||||||
|
(chat.final, no cancel — nothing ever touched its handle; the server has
|
||||||
|
no active turn left to cancel), while a real playout wait is still
|
||||||
|
hanging. The page's cg.interrupt data message must stop it through the
|
||||||
|
SDK's own interrupt, with force."""
|
||||||
|
from livekit import rtc
|
||||||
|
from livekit.agents.voice import SpeechHandle
|
||||||
|
|
||||||
|
from cybergril.voice.worker import bind_interrupt_channel
|
||||||
|
|
||||||
|
# allow_interruptions=False: proves the force semantics on a real handle
|
||||||
|
handle = SpeechHandle.create(allow_interruptions=False)
|
||||||
|
bridge = HandleSessionBridge(handle)
|
||||||
|
room = rtc.Room()
|
||||||
|
unload = bind_interrupt_channel(room, bridge)
|
||||||
|
|
||||||
|
playout = asyncio.Future() # the playout work the TTS pipeline would owe
|
||||||
|
waiter = asyncio.create_task(handle.wait_if_not_interrupted([playout]))
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
assert not waiter.done(), "the playout wait is genuinely pending"
|
||||||
|
|
||||||
|
room.emit("data_received", _data_packet("cg.interrupt", b'{"v":1}'))
|
||||||
|
await asyncio.wait_for(waiter, timeout=5) # real interrupt releases the pending playout wait
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
assert handle.interrupted
|
||||||
|
assert bridge.calls == [True], "programmatic user interrupt goes with force"
|
||||||
|
|
||||||
|
# real force semantics on this SDK: non-force on a no-interruptions
|
||||||
|
# speech raises (HEL-909 evidence B2); force does not.
|
||||||
|
fresh = SpeechHandle.create(allow_interruptions=False)
|
||||||
|
with raises(RuntimeError):
|
||||||
|
fresh.interrupt()
|
||||||
|
fresh.interrupt(force=True)
|
||||||
|
_quiet_await(fresh)
|
||||||
|
_quiet_await(handle)
|
||||||
|
unload()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_interrupt_channel_validation_and_unload():
|
||||||
|
"""Topic/sender/payload validation, and the shutdown callback removal."""
|
||||||
|
from livekit import rtc
|
||||||
|
from livekit.agents.voice import SpeechHandle
|
||||||
|
|
||||||
|
from cybergril.voice.worker import bind_interrupt_channel
|
||||||
|
|
||||||
|
handle = SpeechHandle.create(allow_interruptions=True)
|
||||||
|
bridge = HandleSessionBridge(handle)
|
||||||
|
room = rtc.Room()
|
||||||
|
unload = bind_interrupt_channel(room, bridge)
|
||||||
|
|
||||||
|
for topic, payload, sender in [
|
||||||
|
("cg.subtitle", b'{"v":1}', "user-1"), # wrong topic (our own bus)
|
||||||
|
("cg.interrupt", b'{"v":1}', "server"), # no participant
|
||||||
|
("cg.interrupt", b'{"v":1}', "self"), # the agent itself
|
||||||
|
("cg.interrupt", b"not json", "user-1"), # broken payload
|
||||||
|
("cg.interrupt", b'{"v":2}', "user-1"), # unknown version
|
||||||
|
("cg.interrupt", b'{"v":"1"}', "user-1"), # wrong type
|
||||||
|
]:
|
||||||
|
room.emit("data_received", _data_packet(topic, payload, sender))
|
||||||
|
await asyncio.sleep(0.02)
|
||||||
|
assert bridge.calls == [] and not handle.interrupted, \
|
||||||
|
"nothing but the user page's valid same-room message interrupts"
|
||||||
|
|
||||||
|
room.emit("data_received", _data_packet("cg.interrupt", b'{"v":1}'))
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
assert bridge.calls == [True] and handle.interrupted
|
||||||
|
|
||||||
|
unload()
|
||||||
|
room.emit("data_received", _data_packet("cg.interrupt", b'{"v":1}'))
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
assert bridge.calls == [True], "the unload callback removed the listener"
|
||||||
|
_quiet_await(handle)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_interrupt_message_survives_session_not_running():
|
||||||
|
"""session.interrupt raises RuntimeError before session.start; the
|
||||||
|
channel logs loudly and keeps the room listener alive (no crash, no
|
||||||
|
silent success)."""
|
||||||
|
from livekit import rtc
|
||||||
|
|
||||||
|
from cybergril.voice.worker import bind_interrupt_channel
|
||||||
|
|
||||||
|
class NotRunning:
|
||||||
|
def interrupt(self, *, force: bool = False):
|
||||||
|
raise RuntimeError("AgentSession isn't running")
|
||||||
|
|
||||||
|
room = rtc.Room()
|
||||||
|
unload = bind_interrupt_channel(room, NotRunning())
|
||||||
|
room.emit("data_received", _data_packet("cg.interrupt", b'{"v":1}'))
|
||||||
|
await asyncio.sleep(0.05) # logged, swallowed at the boundary, listener intact
|
||||||
|
room.emit("data_received", _data_packet("cg.interrupt", b'{"v":1}'))
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
unload()
|
||||||
|
|
||||||
|
|
||||||
# ── standalone runner (agent venv has no pytest) ─────────────────────────────
|
# ── standalone runner (agent venv has no pytest) ─────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
@@ -226,6 +365,12 @@ async def _standalone() -> int:
|
|||||||
test_real_connection_errors_still_retry_not_globally_disabled),
|
test_real_connection_errors_still_retry_not_globally_disabled),
|
||||||
("completed_turn_still_streams_with_receipt",
|
("completed_turn_still_streams_with_receipt",
|
||||||
test_completed_turn_still_streams_with_receipt),
|
test_completed_turn_still_streams_with_receipt),
|
||||||
|
("interrupt_message_stops_pending_playout_after_final",
|
||||||
|
test_interrupt_message_stops_pending_playout_after_final),
|
||||||
|
("interrupt_channel_validation_and_unload",
|
||||||
|
test_interrupt_channel_validation_and_unload),
|
||||||
|
("interrupt_message_survives_session_not_running",
|
||||||
|
test_interrupt_message_survives_session_not_running),
|
||||||
]
|
]
|
||||||
failures = 0
|
failures = 0
|
||||||
for name, check in checks:
|
for name, check in checks:
|
||||||
|
|||||||
@@ -269,14 +269,28 @@ class FakeCtx:
|
|||||||
"""JobContext 替身:记录 shutdown 调用."""
|
"""JobContext 替身:记录 shutdown 调用."""
|
||||||
|
|
||||||
def __init__(self, room_name: str) -> None:
|
def __init__(self, room_name: str) -> None:
|
||||||
|
self.room_listeners: list[tuple[str, object]] = []
|
||||||
|
self.shutdown_callbacks: list[object] = []
|
||||||
self.room = SimpleNamespace(
|
self.room = SimpleNamespace(
|
||||||
name=room_name,
|
name=room_name,
|
||||||
local_participant=SimpleNamespace(
|
local_participant=SimpleNamespace(
|
||||||
publish_data=self._publish_data),
|
publish_data=self._publish_data),
|
||||||
|
on=self._room_on,
|
||||||
|
off=self._room_off,
|
||||||
)
|
)
|
||||||
self.proc = SimpleNamespace(userdata={"vad": object()})
|
self.proc = SimpleNamespace(userdata={"vad": object()})
|
||||||
self.shutdown_calls: list[str] = []
|
self.shutdown_calls: list[str] = []
|
||||||
|
|
||||||
|
def _room_on(self, event: str, cb) -> None:
|
||||||
|
self.room_listeners.append((event, cb))
|
||||||
|
|
||||||
|
def _room_off(self, event: str, cb) -> None:
|
||||||
|
self.room_listeners = [(e, c) for e, c in self.room_listeners
|
||||||
|
if not (e == event and c is cb)]
|
||||||
|
|
||||||
|
def add_shutdown_callback(self, callback) -> None:
|
||||||
|
self.shutdown_callbacks.append(callback)
|
||||||
|
|
||||||
async def _publish_data(self, data, topic=None, reliable=False) -> None:
|
async def _publish_data(self, data, topic=None, reliable=False) -> None:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -844,6 +858,10 @@ async def test_worker_entrypoint_wires_guard(fake_sdk, tmp_path, monkeypatch, ca
|
|||||||
guard = stt_obj._usage
|
guard = stt_obj._usage
|
||||||
assert isinstance(guard, AsrUsageGuard)
|
assert isinstance(guard, AsrUsageGuard)
|
||||||
assert guard.policy.window_id == "hel902-w1"
|
assert guard.policy.window_id == "hel902-w1"
|
||||||
|
# HEL-909 R2:本通打断通道挂在房间 data_received 上,会话结束有卸载回调
|
||||||
|
assert any(e == "data_received" for e, _ in ctx.room_listeners), \
|
||||||
|
"cg.interrupt 通道必须挂在本房间 data_received"
|
||||||
|
assert ctx.shutdown_callbacks, "通道卸载必须注册到 add_shutdown_callback"
|
||||||
out = capsys.readouterr().out
|
out = capsys.readouterr().out
|
||||||
assert "asr_window=" in out and "hel902-w1" in out
|
assert "asr_window=" in out and "hel902-w1" in out
|
||||||
# 触线回调真的接到 ctx.shutdown:预检越限帧 → 结束本通 job
|
# 触线回调真的接到 ctx.shutdown:预检越限帧 → 结束本通 job
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
import type { MicTrack, RoomEvents, VoiceRoom, VoiceTransport } from './session';
|
import type { MicTrack, RoomEvents, VoiceRoom, VoiceTransport } from './session';
|
||||||
|
|
||||||
export const LIVEKIT_SRC = '/vendor/livekit-client-2.22.4.umd.min.js';
|
export const LIVEKIT_SRC = '/vendor/livekit-client-2.22.4.umd.min.js';
|
||||||
const SUBTITLES = 'cg.subtitle', AGENT_STATE = 'lk.agent.state';
|
const SUBTITLES = 'cg.subtitle', INTERRUPT = 'cg.interrupt', AGENT_STATE = 'lk.agent.state';
|
||||||
|
|
||||||
// The few parts of the client used here (typed locally; the bundle carries no type package).
|
// The few parts of the client used here (typed locally; the bundle carries no type package).
|
||||||
interface LkTrack { kind: string; mediaStreamTrack: MediaStreamTrack; attach(): HTMLMediaElement; detach(): HTMLMediaElement[]; stop(): void }
|
interface LkTrack { kind: string; mediaStreamTrack: MediaStreamTrack; attach(): HTMLMediaElement; detach(): HTMLMediaElement[]; stop(): void }
|
||||||
@@ -16,7 +16,7 @@ interface LkRoom {
|
|||||||
startAudio(): Promise<void>;
|
startAudio(): Promise<void>;
|
||||||
canPlaybackAudio: boolean;
|
canPlaybackAudio: boolean;
|
||||||
remoteParticipants: Map<string, LkParticipant>;
|
remoteParticipants: Map<string, LkParticipant>;
|
||||||
localParticipant: { publishTrack(track: LkTrack, options?: { source?: string }): Promise<unknown> };
|
localParticipant: { publishTrack(track: LkTrack, options?: { source?: string }): Promise<unknown>; publishData(data: Uint8Array, options?: { reliable?: boolean; topic?: string }): Promise<void> };
|
||||||
}
|
}
|
||||||
export interface Livekit {
|
export interface Livekit {
|
||||||
Room: new (options?: Record<string, unknown>) => LkRoom;
|
Room: new (options?: Record<string, unknown>) => LkRoom;
|
||||||
@@ -93,6 +93,13 @@ export function livekitTransport(lk: Livekit): VoiceTransport {
|
|||||||
const voiceRoom: VoiceRoom = {
|
const voiceRoom: VoiceRoom = {
|
||||||
async publish(mic) { const track = tracks.get(mic); if (!track) throw new Error('unknown microphone'); await room.localParticipant.publishTrack(track, { source: lk.Track.Source.Microphone }); },
|
async publish(mic) { const track = tracks.get(mic); if (!track) throw new Error('unknown microphone'); await room.localParticipant.publishTrack(track, { source: lk.Track.Source.Microphone }); },
|
||||||
hush(quiet) { hushed = quiet; players().forEach(el => { el.muted = quiet; }); },
|
hush(quiet) { hushed = quiet; players().forEach(el => { el.muted = quiet; }); },
|
||||||
|
// HEL-909 R2: same-call interrupt for her worker (reliable, exact topic, payload {"v":1}).
|
||||||
|
// Fire-and-forget by contract: a failed or dropped publish must never break the text
|
||||||
|
// send that triggered it — the local mute above and the server's turn gate still hold.
|
||||||
|
interrupt() {
|
||||||
|
if (closing) return;
|
||||||
|
try { void room.localParticipant.publishData(new TextEncoder().encode('{"v":1}'), { reliable: true, topic: INTERRUPT }).catch(() => {}); } catch { /* transport already closed: nothing of this call can still be playing. */ }
|
||||||
|
},
|
||||||
resumeAudio: () => room.startAudio(),
|
resumeAudio: () => room.startAudio(),
|
||||||
close,
|
close,
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -69,7 +69,10 @@ export interface RoomEvents {
|
|||||||
dropped(): void;
|
dropped(): void;
|
||||||
audioBlocked(blocked: boolean): void;
|
audioBlocked(blocked: boolean): void;
|
||||||
}
|
}
|
||||||
export interface VoiceRoom { publish(mic: MicTrack): Promise<void>; hush(on: boolean): void; resumeAudio(): Promise<void>; close(): Promise<void> }
|
// interrupt(): HEL-909 R2 — ask HER worker (over the room's own data channel, reliable)
|
||||||
|
// to stop any speech still playing. The call is one-way and never throws: it only
|
||||||
|
// supplements the local mute; the text send itself must never be blocked by it.
|
||||||
|
export interface VoiceRoom { publish(mic: MicTrack): Promise<void>; hush(on: boolean): void; interrupt(): void; resumeAudio(): Promise<void>; close(): Promise<void> }
|
||||||
export interface VoiceTransport { capture(): Promise<MicTrack>; join(url: string, token: string, on: RoomEvents, signal: AbortSignal): Promise<VoiceRoom> }
|
export interface VoiceTransport { capture(): Promise<MicTrack>; join(url: string, token: string, on: RoomEvents, signal: AbortSignal): Promise<VoiceRoom> }
|
||||||
|
|
||||||
export interface VoiceAvailability { available: boolean; reason: string }
|
export interface VoiceAvailability { available: boolean; reason: string }
|
||||||
@@ -338,6 +341,11 @@ export class VoiceController {
|
|||||||
if (this.view.her) this.show({ her: null }, live.gen);
|
if (this.view.her) this.show({ her: null }, live.gen);
|
||||||
if (!live.room || (this.view.phase !== 'speaking' && this.view.phase !== 'thinking')) return;
|
if (!live.room || (this.view.phase !== 'speaking' && this.view.phase !== 'thinking')) return;
|
||||||
live.hushed = true; live.room.hush(true);
|
live.hushed = true; live.room.hush(true);
|
||||||
|
// HEL-909 R2: also tell the worker itself — when her turn already finished
|
||||||
|
// generating (server has no active turn to cancel) only the worker can stop
|
||||||
|
// the speech still playing. Fire-and-forget by contract: interrupt() never
|
||||||
|
// throws and never delays the text send.
|
||||||
|
live.room.interrupt();
|
||||||
this.show({ spoken: '' }, live.gen);
|
this.show({ spoken: '' }, live.gen);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ class Room implements VoiceRoom {
|
|||||||
closed = 0; published: MicTrack[] = []; hushed: boolean[] = []; on!: RoomEvents;
|
closed = 0; published: MicTrack[] = []; hushed: boolean[] = []; on!: RoomEvents;
|
||||||
async publish(mic: MicTrack) { this.published.push(mic); }
|
async publish(mic: MicTrack) { this.published.push(mic); }
|
||||||
hush(on: boolean) { this.hushed.push(on); }
|
hush(on: boolean) { this.hushed.push(on); }
|
||||||
|
interrupt() {}
|
||||||
async resumeAudio() {}
|
async resumeAudio() {}
|
||||||
async close() { this.closed++; }
|
async close() { this.closed++; }
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -110,7 +110,7 @@ describe('her worker leaves while the room stays open (real adapter, SDK event n
|
|||||||
});
|
});
|
||||||
|
|
||||||
describe('controller: agentLeft only for this call, only once she was there', () => {
|
describe('controller: agentLeft only for this call, only once she was there', () => {
|
||||||
class Room implements VoiceRoom { closed = 0; on!: RoomEvents; async publish(_m: MicTrack) {} hush() {} async resumeAudio() {} async close() { this.closed++; } }
|
class Room implements VoiceRoom { closed = 0; on!: RoomEvents; async publish(_m: MicTrack) {} hush() {} interrupt() {} async resumeAudio() {} async close() { this.closed++; } }
|
||||||
const make = () => {
|
const make = () => {
|
||||||
const list: Room[] = [], views: VoiceView[] = [];
|
const list: Room[] = [], views: VoiceView[] = [];
|
||||||
const transport = { capture: async () => ({ stop() {}, onEnded() {} }), join: async (_u: string, _t: string, on: RoomEvents) => { const x = new Room(); x.on = on; list.push(x); return x; } };
|
const transport = { capture: async () => ({ stop() {}, onEnded() {} }), join: async (_u: string, _t: string, on: RoomEvents) => { const x = new Room(); x.on = on; list.push(x); return x; } };
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ describe('外接服务 reads only what /api/voice/status can tell', () => {
|
|||||||
|
|
||||||
// ── The call's own voice ──
|
// ── The call's own voice ──
|
||||||
class Mic implements MicTrack { stopped = 0; stop() { this.stopped++; } onEnded() {} }
|
class Mic implements MicTrack { stopped = 0; stop() { this.stopped++; } onEnded() {} }
|
||||||
class Room implements VoiceRoom { closed = 0; on!: RoomEvents; async publish() {} hush() {} async resumeAudio() {} async close() { this.closed++; } }
|
class Room implements VoiceRoom { closed = 0; on!: RoomEvents; async publish() {} hush() {} interrupt() {} async resumeAudio() {} async close() { this.closed++; } }
|
||||||
function rig() {
|
function rig() {
|
||||||
const views: VoiceView[] = [], rooms: Room[] = [], voices: unknown[] = [], gates: (ReturnType<typeof gate> | undefined)[] = [];
|
const views: VoiceView[] = [], rooms: Room[] = [], voices: unknown[] = [], gates: (ReturnType<typeof gate> | undefined)[] = [];
|
||||||
let made = 0;
|
let made = 0;
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ test('old room disconnect cleanup cannot overwrite a newly opened call', async (
|
|||||||
capture: async () => ({ stop() {}, onEnded() {} }),
|
capture: async () => ({ stop() {}, onEnded() {} }),
|
||||||
join: async (_url, _token, callbacks) => {
|
join: async (_url, _token, callbacks) => {
|
||||||
events.push(callbacks);
|
events.push(callbacks);
|
||||||
return { publish: async () => {}, close: async () => {}, hush() {}, resumeAudio: async () => {} };
|
return { publish: async () => {}, close: async () => {}, hush() {}, interrupt() {}, resumeAudio: async () => {} };
|
||||||
},
|
},
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -16,9 +16,10 @@ const gate = () => { let open!: () => void; const promise = new Promise<void>(r
|
|||||||
const flush = async (n = 8) => { for (let i = 0; i < n; i++) await Promise.resolve(); await new Promise(r => setTimeout(r, 0)); };
|
const flush = async (n = 8) => { for (let i = 0; i < n; i++) await Promise.resolve(); await new Promise(r => setTimeout(r, 0)); };
|
||||||
class Mic implements MicTrack { stopped = 0; ended?: () => void; stop() { this.stopped++; } onEnded(cb: () => void) { this.ended = cb; } }
|
class Mic implements MicTrack { stopped = 0; ended?: () => void; stop() { this.stopped++; } onEnded(cb: () => void) { this.ended = cb; } }
|
||||||
class Room implements VoiceRoom {
|
class Room implements VoiceRoom {
|
||||||
closed = 0; published: MicTrack[] = []; hushed: boolean[] = []; on!: RoomEvents;
|
closed = 0; published: MicTrack[] = []; hushed: boolean[] = []; interrupted = 0; on!: RoomEvents;
|
||||||
async publish(mic: MicTrack) { this.published.push(mic); }
|
async publish(mic: MicTrack) { this.published.push(mic); }
|
||||||
hush(on: boolean) { this.hushed.push(on); }
|
hush(on: boolean) { this.hushed.push(on); }
|
||||||
|
interrupt() { this.interrupted++; }
|
||||||
async resumeAudio() {}
|
async resumeAudio() {}
|
||||||
async close() { this.closed++; }
|
async close() { this.closed++; }
|
||||||
}
|
}
|
||||||
@@ -376,8 +377,10 @@ describe('subtitles and the two channels', () => {
|
|||||||
it('typing over her spoken answer silences what is left of it until she is listening again', async () => {
|
it('typing over her spoken answer silences what is left of it until she is listening again', async () => {
|
||||||
const r = await connected(), room = r.transport.rooms[0];
|
const r = await connected(), room = r.transport.rooms[0];
|
||||||
r.c.hush(); expect(room.hushed).toEqual([]); // listening: nothing to silence
|
r.c.hush(); expect(room.hushed).toEqual([]); // listening: nothing to silence
|
||||||
|
expect(room.interrupted).toBe(0); // nor anything to tell her worker
|
||||||
room.on.agentState('speaking'); r.c.hush();
|
room.on.agentState('speaking'); r.c.hush();
|
||||||
expect(room.hushed).toEqual([true]);
|
expect(room.hushed).toEqual([true]);
|
||||||
|
expect(room.interrupted).toBe(1); // HEL-909 R2: the worker is told too
|
||||||
room.on.agentState('speaking'); expect(room.hushed).toEqual([true]);
|
room.on.agentState('speaking'); expect(room.hushed).toEqual([true]);
|
||||||
room.on.agentState('listening'); expect(room.hushed).toEqual([true, false]);
|
room.on.agentState('listening'); expect(room.hushed).toEqual([true, false]);
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user