R1 — reasoning separation truly closed. The streaming hold leaked wholesale
on real token arrival (review A1: 'private reasoning '*20 beyond the 128-char
bound, no sentence ender; review A2: a Chinese sentence ender released the
hold) AND the adapter consumed the stray closer, so the app-level turn split
could never recover — the previous 'persistence and receipt always clean'
claim was false and is retracted. Per the engineering ruling the adapter now
has two explicit endpoint modes (GOV-006, no heuristics, no new engine):
buffer_turn (TurnReasoningGate) emits NOTHING during generation, splits the
whole turn at stream end and only then delivers the clean visible text; it
is set explicitly for all tiers in config/models.yaml (first visible text
waits for the full generation; cancelled/timed-out turns emit nothing and
end exactly as before — recorded and test-pinned, low latency NOT claimed).
stream (default, backward compatible) keeps explicit-tag/field semantics and
passes stray closers through unmodified so the turn-level split keeps its
signal. Both review strings and the recorded id=12 stream replay clean at
every chunk size through the real adapter and the real app into chat.delta,
the archive and the receipt; normal zh/en preserved byte-for-byte; unclosed
explicit thinking still fails honestly.
R2 — an authoritative interrupt when the LLM already finalized and TTS is
still playing (review evidence C: zero interrupts, old audio played out).
The page's existing hush() entry (ChatPanel untouched) now also publishes
{"v":1} on the room's own reliable data channel topic cg.interrupt
(fire-and-forget: a dropped publish never blocks or breaks the text send);
the worker subscribes data_received, validates topic/sender/payload and
calls session.interrupt(force=True) — the SDK's own programmatic interrupt,
force confirmed against the installed 1.8.8 source (non-force raises
RuntimeError on allow_interruptions=False speech). Failures log loudly; the
listener is unloaded via add_shutdown_callback; the server-cancel path in
llm_chat_api also interrupts with force and logs failures.
Verification (offline, zero real requests): real-SDK harness 8/8 including a
real SpeechHandle with a genuinely pending playout wait released by the
cg.interrupt message after a clean chat.final (evidence-C shape) plus the
full validation matrix; .venv311 full suite 480 passed / 8 skipped (2
test_r3 failures pre-exist at 5625de6); web npm ci from the existing lock,
typecheck 0 errors, lint 0 errors + 1 pre-existing warning, vitest 148/148,
build ok; ruff clean; check_requirements vs origin/main and b8ec832 with the
corrected handoff.
Co-authored-by: multica-agent <github@multica.ai>
1096 lines
45 KiB
Python
1096 lines
45 KiB
Python
"""HEL-906 连续通话 ASR 实际用量计数与窗口闸的确定性离线测试.
|
||
|
||
两层验证(复审要求"接缝必须真的被专属 worker 调用,不能只有独立计数类
|
||
测试"):
|
||
|
||
1. 纯计数/闸/账本恢复(guard 单元):fail-closed 默认、按实际帧样本计时、
|
||
触线前拒绝、重试口径、落盘恢复、跨窗口隔离、崩溃少计有界。
|
||
2. 真实接缝:用假 livekit/websockets 模块(隔离假 provider、假帧)导入
|
||
真实的 voice/stt_funasr.py 与 voice/worker.py,驱动真实 ``_run`` 送数
|
||
路径——建连检查、发送前拒绝、保守预记/确证送达、接收侧死亡、流异常
|
||
重试、utterance 聚合落账全部走产品代码;worker entrypoint 布线(guard
|
||
注入、房间名标识、触线 ctx.shutdown、未布防拒绝)逐项断言。
|
||
3. 2026-10-28 总管定点复核返工:账本写失败→重放不得回退预算(复现脚本的
|
||
回归路径)、读失败/损坏/中途丢失 fail-closed 停机并保全已知消耗、
|
||
guard_stop 跨重启仍停、假 ws 先记 payload 再 raise/取消后预算仍包含
|
||
该帧(保守计费不退还、确证量单列)。
|
||
|
||
零真实 ASR/TTS/LiveKit 调用(fake 连接全部在进程内),零安装。
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import asyncio
|
||
import enum
|
||
import json
|
||
import subprocess
|
||
import sys
|
||
import types
|
||
from pathlib import Path
|
||
from types import SimpleNamespace
|
||
|
||
import httpx
|
||
import pytest
|
||
|
||
from cybergril.voice.asr_usage import (
|
||
AsrUsageGuard,
|
||
AsrWindowClosed,
|
||
AsrWindowPolicy,
|
||
build_usage_from_env,
|
||
policy_from_env,
|
||
summarize_ledger,
|
||
)
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# fake livekit / websockets SDK(只满足本仓库代码用到的表面,进程内假 provider)
|
||
# ---------------------------------------------------------------------------
|
||
|
||
_FAKE_MODULES = (
|
||
"livekit",
|
||
"livekit.rtc",
|
||
"livekit.agents",
|
||
"livekit.agents.stt",
|
||
"livekit.agents.llm",
|
||
"livekit.agents.tts",
|
||
"livekit.agents.types",
|
||
"livekit.plugins",
|
||
"livekit.plugins.silero",
|
||
"websockets",
|
||
)
|
||
|
||
|
||
class FakeFrame:
|
||
def __init__(self, data: bytes, sample_rate: int, samples_per_channel: int) -> None:
|
||
self.data = data
|
||
self.sample_rate = sample_rate
|
||
self.samples_per_channel = samples_per_channel
|
||
|
||
|
||
class FakeResampler:
|
||
"""线性比例重采样替身:输出帧数与输入样本数严格成比例(16bit/单声道)."""
|
||
|
||
def __init__(self, in_rate: int, out_rate: int) -> None:
|
||
self.out_rate = out_rate
|
||
self.ratio = out_rate / in_rate
|
||
self._in_samples = 0
|
||
self._out_samples = 0
|
||
|
||
def push(self, frame: FakeFrame) -> list[FakeFrame]:
|
||
self._in_samples += frame.samples_per_channel
|
||
target = round(self._in_samples * self.ratio)
|
||
n = target - self._out_samples
|
||
if n <= 0:
|
||
return []
|
||
self._out_samples = target
|
||
return [FakeFrame(b"\x00" * (n * 2), self.out_rate, n)]
|
||
|
||
|
||
class FakeWS:
|
||
"""进程内 FunASR websocket 假 provider:记录每个出站字节.
|
||
|
||
fail_send 模拟「帧可能已出站」的不确定失败:先把 payload 记入 sent
|
||
再 raise/cancel(控制消息不受 fail_bytes_after 影响)。
|
||
"""
|
||
|
||
def __init__(self, server_msgs: list[str] | None = None, recv_fail: bool = False) -> None:
|
||
self.sent: list[bytes | str] = []
|
||
self.server_msgs = list(server_msgs or [])
|
||
self.recv_fail = recv_fail
|
||
self.closed = False
|
||
self.fail_send = "" # "" | "raise" | "cancel"
|
||
self.fail_bytes_after: int | None = None # 第 N 个音频帧之后开始失败
|
||
|
||
async def send(self, payload: bytes | str) -> None:
|
||
self.sent.append(payload) # 先记:即使随后失败,帧也可能已经出站
|
||
if isinstance(payload, (bytes, bytearray)) and self.fail_send:
|
||
if self.fail_bytes_after is not None and self.fail_bytes_after > 0:
|
||
self.fail_bytes_after -= 1
|
||
return # 前 N 个音频帧正常送达
|
||
self.fail_bytes_after = None # 计数耗尽:之后的音频帧都按失败语义
|
||
if self.fail_send == "raise":
|
||
raise OSError("send failed after outbound")
|
||
if self.fail_send == "cancel":
|
||
raise asyncio.CancelledError()
|
||
|
||
async def close(self) -> None:
|
||
self.closed = True
|
||
|
||
def __aiter__(self):
|
||
return self
|
||
|
||
async def __anext__(self):
|
||
while True:
|
||
if self.closed:
|
||
raise StopAsyncIteration
|
||
if self.recv_fail:
|
||
raise ConnectionError("recv died")
|
||
if self.server_msgs:
|
||
return self.server_msgs.pop(0)
|
||
await asyncio.sleep(0.005)
|
||
|
||
|
||
class FakeWebsockets:
|
||
"""websockets.connect 假 provider:可编程失败与假服务端响应."""
|
||
|
||
def __init__(self) -> None:
|
||
self.attempts = 0
|
||
self.fail_attempts: set[int] = set()
|
||
self.sockets: list[FakeWS] = []
|
||
self.ws_factory = None
|
||
|
||
def connect(self, url: str, max_size: int | None = None):
|
||
async def _connect() -> FakeWS:
|
||
self.attempts += 1
|
||
if self.attempts in self.fail_attempts:
|
||
raise OSError(f"connect refused (attempt {self.attempts})")
|
||
if self.ws_factory is not None:
|
||
ws = self.ws_factory()
|
||
else:
|
||
ws = FakeWS(server_msgs=[json.dumps(
|
||
{"mode": "2pass-offline", "text": "测试终稿"})])
|
||
self.sockets.append(ws)
|
||
return ws
|
||
|
||
return _connect()
|
||
|
||
def bytes_sent(self) -> list[bytes]:
|
||
return [p for ws in self.sockets for p in ws.sent if isinstance(p, bytes)]
|
||
|
||
|
||
class _InCh:
|
||
def __init__(self) -> None:
|
||
self.q: asyncio.Queue = asyncio.Queue()
|
||
|
||
def push(self, item) -> None:
|
||
self.q.put_nowait(item)
|
||
|
||
def end(self) -> None:
|
||
self.q.put_nowait(None)
|
||
|
||
def __aiter__(self):
|
||
return self
|
||
|
||
async def __anext__(self):
|
||
return await self.q.get()
|
||
|
||
|
||
class _OutCh:
|
||
def __init__(self) -> None:
|
||
self.q: asyncio.Queue = asyncio.Queue()
|
||
|
||
def send_nowait(self, item) -> None:
|
||
self.q.put_nowait(item)
|
||
|
||
def empty(self) -> bool:
|
||
return self.q.empty()
|
||
|
||
def get_nowait(self):
|
||
return self.q.get_nowait()
|
||
|
||
|
||
class SpeechEventType(enum.Enum):
|
||
START_OF_SPEECH = "start_of_speech"
|
||
INTERIM_TRANSCRIPT = "interim_transcript"
|
||
FINAL_TRANSCRIPT = "final_transcript"
|
||
END_OF_SPEECH = "end_of_speech"
|
||
|
||
|
||
class SpeechData:
|
||
def __init__(self, *, language: str, text: str) -> None:
|
||
self.language = language
|
||
self.text = text
|
||
|
||
|
||
class SpeechEvent:
|
||
def __init__(self, *, type, alternatives) -> None:
|
||
self.type = type
|
||
self.alternatives = alternatives
|
||
|
||
|
||
def _kwargs_ns(**kwargs):
|
||
return SimpleNamespace(**kwargs)
|
||
|
||
|
||
class FakeRecognizeStream:
|
||
class _FlushSentinel:
|
||
pass
|
||
|
||
def __init__(self, *, stt=None, conn_options=None) -> None:
|
||
self._stt = stt
|
||
self._conn_options = conn_options
|
||
self._input_ch = _InCh()
|
||
self._event_ch = _OutCh()
|
||
self._run_task: asyncio.Task | None = None
|
||
|
||
def _ensure_run(self) -> asyncio.Task:
|
||
if self._run_task is None:
|
||
self._run_task = asyncio.get_running_loop().create_task(self._run())
|
||
return self._run_task
|
||
|
||
def __aiter__(self):
|
||
self._ensure_run()
|
||
return self
|
||
|
||
async def __anext__(self):
|
||
self._ensure_run()
|
||
while True:
|
||
if not self._event_ch.empty():
|
||
return self._event_ch.get_nowait()
|
||
task = self._run_task
|
||
if task.done():
|
||
exc = task.exception()
|
||
if exc is not None:
|
||
raise exc
|
||
raise StopAsyncIteration
|
||
await asyncio.sleep(0.005)
|
||
|
||
|
||
class FakeAgentSession:
|
||
last: FakeAgentSession | None = None # 测试替身的类级记录
|
||
|
||
def __init__(self, **kwargs) -> None:
|
||
self.kwargs = kwargs
|
||
self.handlers: dict[str, list] = {}
|
||
self.started = None
|
||
FakeAgentSession.last = self
|
||
|
||
def on(self, ev: str):
|
||
def deco(fn):
|
||
self.handlers.setdefault(ev, []).append(fn)
|
||
return fn
|
||
|
||
return deco
|
||
|
||
async def start(self, *, room=None, agent=None) -> None:
|
||
self.started = (room, agent)
|
||
|
||
|
||
class FakeCtx:
|
||
"""JobContext 替身:记录 shutdown 调用."""
|
||
|
||
def __init__(self, room_name: str) -> None:
|
||
self.room_listeners: list[tuple[str, object]] = []
|
||
self.shutdown_callbacks: list[object] = []
|
||
self.room = SimpleNamespace(
|
||
name=room_name,
|
||
local_participant=SimpleNamespace(
|
||
publish_data=self._publish_data),
|
||
on=self._room_on,
|
||
off=self._room_off,
|
||
)
|
||
self.proc = SimpleNamespace(userdata={"vad": object()})
|
||
self.shutdown_calls: list[str] = []
|
||
|
||
def _room_on(self, event: str, cb) -> None:
|
||
self.room_listeners.append((event, cb))
|
||
|
||
def _room_off(self, event: str, cb) -> None:
|
||
self.room_listeners = [(e, c) for e, c in self.room_listeners
|
||
if not (e == event and c is cb)]
|
||
|
||
def add_shutdown_callback(self, callback) -> None:
|
||
self.shutdown_callbacks.append(callback)
|
||
|
||
async def _publish_data(self, data, topic=None, reliable=False) -> None:
|
||
return None
|
||
|
||
async def connect(self) -> None:
|
||
return None
|
||
|
||
def shutdown(self, reason: str = "", message: str = "") -> None:
|
||
self.shutdown_calls.append(reason or message or "shutdown")
|
||
|
||
|
||
class _VAD:
|
||
@classmethod
|
||
def load(cls):
|
||
return cls()
|
||
|
||
|
||
def _install_fake_sdk(monkeypatch) -> SimpleNamespace:
|
||
rtc_mod = types.ModuleType("livekit.rtc")
|
||
rtc_mod.AudioFrame = FakeFrame
|
||
rtc_mod.AudioResampler = FakeResampler
|
||
|
||
stt_mod = types.ModuleType("livekit.agents.stt")
|
||
stt_mod.STT = type("STT", (), {"__init__": lambda self, *, capabilities=None: setattr(
|
||
self, "capabilities", capabilities)})
|
||
stt_mod.STTCapabilities = _kwargs_ns
|
||
stt_mod.SpeechData = SpeechData
|
||
stt_mod.SpeechEvent = SpeechEvent
|
||
stt_mod.SpeechEventType = SpeechEventType
|
||
stt_mod.RecognizeStream = FakeRecognizeStream
|
||
|
||
llm_mod = types.ModuleType("livekit.agents.llm")
|
||
for name in ("LLM", "ChatChunk", "ChatContext", "ChoiceDelta", "LLMStream", "Tool"):
|
||
setattr(llm_mod, name, type(name, (), {"__init__": lambda self, *a, **k: None}))
|
||
|
||
tts_mod = types.ModuleType("livekit.agents.tts")
|
||
tts_mod.TTS = type("TTS", (), {"__init__": lambda self, *, capabilities=None,
|
||
sample_rate=None, num_channels=None: None})
|
||
tts_mod.TTSCapabilities = _kwargs_ns
|
||
for name in ("AudioEmitter", "ChunkedStream"):
|
||
setattr(tts_mod, name, type(name, (), {}))
|
||
|
||
types_mod = types.ModuleType("livekit.agents.types")
|
||
types_mod.APIConnectOptions = type("APIConnectOptions", (), {})
|
||
types_mod.DEFAULT_API_CONNECT_OPTIONS = types_mod.APIConnectOptions()
|
||
types_mod.NOT_GIVEN = object()
|
||
types_mod.NotGivenOr = object
|
||
|
||
agents_mod = types.ModuleType("livekit.agents")
|
||
agents_mod.Agent = type("Agent", (), {"__init__": lambda self, *a, **k: None})
|
||
agents_mod.AgentSession = FakeAgentSession
|
||
for name in ("JobContext", "JobProcess"):
|
||
setattr(agents_mod, name, type(name, (), {}))
|
||
agents_mod.WorkerOptions = type("WorkerOptions", (), {"__init__": lambda self, **k: None})
|
||
agents_mod.cli = type("cli", (), {"run_app": staticmethod(lambda *a, **k: None)})
|
||
agents_mod.APIConnectionError = type("APIConnectionError", (Exception,), {})
|
||
agents_mod.APIStatusError = type("APIStatusError", (Exception,), {})
|
||
agents_mod.stt = stt_mod
|
||
agents_mod.llm = llm_mod
|
||
agents_mod.tts = tts_mod
|
||
agents_mod.types = types_mod
|
||
|
||
silero_mod = types.ModuleType("livekit.plugins.silero")
|
||
silero_mod.VAD = _VAD
|
||
plugins_mod = types.ModuleType("livekit.plugins")
|
||
plugins_mod.silero = silero_mod
|
||
|
||
livekit_mod = types.ModuleType("livekit")
|
||
livekit_mod.rtc = rtc_mod
|
||
livekit_mod.agents = agents_mod
|
||
livekit_mod.plugins = plugins_mod
|
||
|
||
websockets_mod = types.ModuleType("websockets")
|
||
fake_ws_module = FakeWebsockets()
|
||
websockets_mod.connect = fake_ws_module.connect
|
||
|
||
for name, mod in (
|
||
("livekit", livekit_mod),
|
||
("livekit.rtc", rtc_mod),
|
||
("livekit.agents", agents_mod),
|
||
("livekit.agents.stt", stt_mod),
|
||
("livekit.agents.llm", llm_mod),
|
||
("livekit.agents.tts", tts_mod),
|
||
("livekit.agents.types", types_mod),
|
||
("livekit.plugins", plugins_mod),
|
||
("livekit.plugins.silero", silero_mod),
|
||
("websockets", websockets_mod),
|
||
):
|
||
monkeypatch.setitem(sys.modules, name, mod)
|
||
for cached in ("cybergril.voice.stt_funasr", "cybergril.voice.worker",
|
||
"cybergril.voice.tts_cosyvoice", "cybergril.voice.llm_chat_api"):
|
||
monkeypatch.delitem(sys.modules, cached, raising=False)
|
||
return SimpleNamespace(websockets=fake_ws_module, rtc=rtc_mod, stt=stt_mod)
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# helpers
|
||
# ---------------------------------------------------------------------------
|
||
|
||
def make_policy(**overrides) -> AsrWindowPolicy:
|
||
values = dict(window_id="w-test", max_audio_seconds=1000.0,
|
||
max_connect_retries_per_call=6, flush_seconds=5.0,
|
||
resync_seconds=1e9)
|
||
values.update(overrides)
|
||
return AsrWindowPolicy(**values)
|
||
|
||
|
||
def make_guard(tmp_path: Path, *, on_exhausted=None, **overrides) -> AsrUsageGuard:
|
||
return AsrUsageGuard(make_policy(**overrides), tmp_path / "asr-usage.jsonl",
|
||
on_exhausted=on_exhausted)
|
||
|
||
|
||
def frame_16k(n_samples: int = 160) -> FakeFrame:
|
||
return FakeFrame(b"\x00" * (n_samples * 2), 16000, n_samples)
|
||
|
||
|
||
def frame_48k(n_samples: int = 480) -> FakeFrame:
|
||
return FakeFrame(b"\x00" * (n_samples * 2), 48000, n_samples)
|
||
|
||
|
||
def read_records(tmp_path: Path) -> list[dict]:
|
||
path = tmp_path / "asr-usage.jsonl"
|
||
if not path.exists():
|
||
return []
|
||
return [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines()
|
||
if line.strip()]
|
||
|
||
|
||
async def drive(module, stt, script, *, timeout: float = 10.0):
|
||
"""按脚本驱动真实 RecognizeStream(帧/flush/None=结束),等待跑完."""
|
||
stream = stt.stream()
|
||
task = stream._ensure_run()
|
||
for item in script:
|
||
if item is None:
|
||
stream._input_ch.end()
|
||
elif isinstance(item, str) and item == "flush":
|
||
stream._input_ch.push(module.RecognizeStream._FlushSentinel())
|
||
else:
|
||
stream._input_ch.push(item)
|
||
await asyncio.sleep(0)
|
||
await asyncio.wait_for(task, timeout=timeout)
|
||
return stream
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 1) guard 单元:计数口径、fail-closed、触线、恢复、隔离
|
||
# ---------------------------------------------------------------------------
|
||
|
||
def test_policy_env_armed_and_fail_closed_default():
|
||
assert policy_from_env({}).armed is False # 未布防 = 真实 cap=0
|
||
policy = policy_from_env({
|
||
"CYBERGRIL_ASR_WINDOW_ID": "hel902-w1",
|
||
"CYBERGRIL_ASR_WINDOW_MAX_AUDIO_SECONDS": "2700",
|
||
"CYBERGRIL_ASR_MAX_CONNECT_RETRIES_PER_CALL": "6",
|
||
})
|
||
assert policy.armed
|
||
assert policy.max_audio_seconds == 2700.0
|
||
assert policy.max_connect_retries_per_call == 6
|
||
# 配置损坏按 fail-closed(0)处理,不放宽
|
||
bad = policy_from_env({"CYBERGRIL_ASR_WINDOW_ID": "w",
|
||
"CYBERGRIL_ASR_WINDOW_MAX_AUDIO_SECONDS": "abc"})
|
||
assert bad.max_audio_seconds == 0.0
|
||
|
||
|
||
def test_unarmed_guard_blocks_everything_and_records(tmp_path):
|
||
guard = make_guard(tmp_path, window_id="")
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.begin_stream("room-a")
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.check_connect("room-a")
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.precheck_audio("room-a", 16000, 160)
|
||
blocks = [r for r in read_records(tmp_path) if r["kind"] == "block"]
|
||
assert {(b["what"], b["reason"]) for b in blocks} == {
|
||
("new_stream", "window-not-armed"),
|
||
("new_connection", "window-not-armed"),
|
||
("audio", "window-not-armed"),
|
||
}
|
||
|
||
|
||
def test_guard_audio_counted_by_actual_frame_samples(tmp_path):
|
||
guard = make_guard(tmp_path)
|
||
call = guard.begin_stream("room-a")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
# 16k 直接路径:160 样本 = 0.01s;重采样路径按输出帧 16k 计
|
||
for _ in range(10):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
guard.note_audio_delivered(call, 16000, 160, 320)
|
||
for _ in range(5):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
guard.note_audio_delivered(call, 16000, 160, 320) # 重发按重复计
|
||
guard.stream_end(call, "closed")
|
||
assert guard.calls[call].audio_seconds == pytest.approx(0.15)
|
||
assert guard.calls[call].confirmed_audio_seconds == pytest.approx(0.15)
|
||
audio = [r for r in read_records(tmp_path) if r["kind"] == "audio"]
|
||
assert sum(r["seconds"] for r in audio) == pytest.approx(0.15)
|
||
assert sum(r["confirmed_seconds"] for r in audio) == pytest.approx(0.15)
|
||
|
||
|
||
def test_guard_uncertain_outbound_not_refunded(tmp_path):
|
||
"""发送前预记、失败/取消不退还(保守计费);确证量只记成功返回的帧."""
|
||
guard = make_guard(tmp_path)
|
||
call = guard.begin_stream("room-a")
|
||
guard.reserve_audio(call, 16000, 160) # 发送失败的帧:保守预记保留
|
||
guard.reserve_audio(call, 16000, 160)
|
||
guard.note_audio_delivered(call, 16000, 160) # 只有这帧确证送达
|
||
guard.flush_audio(call)
|
||
assert guard.calls[call].audio_seconds == pytest.approx(0.02)
|
||
assert guard.calls[call].confirmed_audio_seconds == pytest.approx(0.01)
|
||
audio = [r for r in read_records(tmp_path) if r["kind"] == "audio"][-1]
|
||
assert audio["seconds"] == pytest.approx(0.02)
|
||
assert audio["confirmed_seconds"] == pytest.approx(0.01)
|
||
|
||
|
||
def test_guard_blocks_crossing_frame_before_send(tmp_path):
|
||
stops: list[str] = []
|
||
guard = make_guard(tmp_path, max_audio_seconds=2.0, on_exhausted=stops.append)
|
||
call = guard.begin_stream("room-a")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
guard.reserve_audio(call, 16000, 16000, 32000) # 1.0s
|
||
guard.note_audio_delivered(call, 16000, 16000, 32000)
|
||
guard.flush_audio(call)
|
||
guard.reserve_audio(call, 16000, 16000) # 恰好等于上限:允许(≤上限)
|
||
guard.note_audio_delivered(call, 16000, 16000, 32000)
|
||
guard.flush_audio(call)
|
||
with pytest.raises(AsrWindowClosed): # 第 3 秒会在发送前拒绝
|
||
guard.reserve_audio(call, 16000, 16000)
|
||
assert stops == ["window-audio-exhausted"] # 回调恰好一次
|
||
# 同窗新通也立即拒绝(新连接/新音频都不放行)
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.begin_stream("room-b")
|
||
blocks = [r for r in read_records(tmp_path) if r["kind"] == "block"]
|
||
assert {b["reason"] for b in blocks} == {"window-audio-exhausted"}
|
||
assert guard.window_audio_seconds == pytest.approx(2.0) # 一个字节没多
|
||
|
||
|
||
def test_guard_retry_semantics_only_after_failure(tmp_path):
|
||
guard = make_guard(tmp_path, max_connect_retries_per_call=2)
|
||
call = guard.begin_stream("room-a")
|
||
|
||
guard.check_connect(call) # 初次建连不是 retry
|
||
guard.note_connect_result(call, ok=True)
|
||
guard.check_connect(call) # 正常说完一轮后的下一次建连也不是 retry
|
||
guard.note_connect_result(call, ok=True)
|
||
assert guard.calls[call].connect_retries == 0
|
||
|
||
guard.check_connect(call) # 失败 → retry_pending
|
||
guard.note_connect_result(call, ok=False, error="OSError: refused")
|
||
guard.check_connect(call) # 失败后的再建连 = retry #1
|
||
guard.note_connect_result(call, ok=True)
|
||
guard.check_connect(call) # 再次失败 → retry_pending
|
||
guard.note_connect_result(call, ok=False, error="OSError: refused")
|
||
guard.check_connect(call) # retry #2(用尽上限)
|
||
guard.note_connect_result(call, ok=True)
|
||
cu = guard.calls[call]
|
||
assert (cu.connect_attempts, cu.connect_failures, cu.connect_retries) == (6, 2, 2)
|
||
|
||
guard.check_connect(call) # 上次成功 → 非 retry:不受 retry 上限约束
|
||
guard.note_connect_result(call, ok=True)
|
||
guard.check_connect(call) # 失败 → 置 retry_pending
|
||
guard.note_connect_result(call, ok=False, error="OSError: refused")
|
||
with pytest.raises(AsrWindowClosed): # retry #3 在建连前被拒
|
||
guard.check_connect(call)
|
||
cu = guard.calls[call]
|
||
assert (cu.connect_attempts, cu.connect_failures, cu.connect_retries) == (8, 3, 2)
|
||
|
||
# 新的一通有独立 retry 预算(每通口径)
|
||
call2 = guard.begin_stream("room-b")
|
||
guard.check_connect(call2)
|
||
guard.note_connect_result(call2, ok=True)
|
||
assert guard.calls["room-b"].connect_attempts == 1
|
||
|
||
|
||
def test_guard_ledger_recovery_same_window_no_reset(tmp_path):
|
||
guard1 = make_guard(tmp_path, max_audio_seconds=3.0)
|
||
call = guard1.begin_stream("room-a")
|
||
guard1.check_connect(call)
|
||
guard1.note_connect_result(call, ok=True)
|
||
guard1.check_connect(call)
|
||
guard1.note_connect_result(call, ok=False, error="x")
|
||
for _ in range(150):
|
||
guard1.reserve_audio(call, 16000, 160, 320)
|
||
guard1.note_audio_delivered(call, 16000, 160, 320)
|
||
guard1.flush_audio(call)
|
||
guard1.stream_end(call, "closed")
|
||
# 模拟进程重启:同窗口同账本重建 guard,预算与逐通历史都在
|
||
guard2 = make_guard(tmp_path, max_audio_seconds=3.0)
|
||
assert guard2.window_audio_seconds == pytest.approx(1.5)
|
||
assert guard2.window_confirmed_seconds == pytest.approx(1.5)
|
||
cu = guard2.calls["room-a"]
|
||
assert (cu.connect_attempts, cu.connect_failures, cu.connect_retries) == (2, 1, 0)
|
||
assert cu.audio_seconds == pytest.approx(1.5)
|
||
assert cu.confirmed_audio_seconds == pytest.approx(1.5)
|
||
assert cu.retry_pending is True # 最后一次连接事实是失败
|
||
# 耗尽同窗口剩余预算后拒绝;重开同窗口 id 不能重置
|
||
call = guard2.begin_stream("room-a")
|
||
guard2.check_connect(call)
|
||
guard2.note_connect_result(call, ok=True) # retry_pending=True → 这是 retry
|
||
with pytest.raises(AsrWindowClosed):
|
||
for _ in range(200):
|
||
guard2.reserve_audio(call, 16000, 160, 320)
|
||
guard2.note_audio_delivered(call, 16000, 160, 320)
|
||
guard2.flush_audio(call)
|
||
|
||
|
||
def test_guard_window_isolation_does_not_stop_others(tmp_path):
|
||
"""W1 耗尽不误停别的窗口/会话(一次性窗口各有独立预算)."""
|
||
g1 = make_guard(tmp_path, window_id="w1", max_audio_seconds=0.05)
|
||
with pytest.raises(AsrWindowClosed):
|
||
call = g1.begin_stream("room-a")
|
||
g1.check_connect(call)
|
||
g1.note_connect_result(call, ok=True)
|
||
for _ in range(10):
|
||
g1.reserve_audio(call, 16000, 160, 320)
|
||
g1.note_audio_delivered(call, 16000, 160, 320)
|
||
g2 = make_guard(tmp_path, window_id="w2", max_audio_seconds=100.0)
|
||
call = g2.begin_stream("room-a") # 同账本、不同窗口:完全不受 W1 影响
|
||
g2.check_connect(call)
|
||
g2.note_connect_result(call, ok=True)
|
||
for _ in range(10):
|
||
g2.reserve_audio(call, 16000, 160, 320)
|
||
g2.note_audio_delivered(call, 16000, 160, 320)
|
||
g2.stream_end(call, "closed")
|
||
assert g2.calls["room-a"].audio_seconds == pytest.approx(0.1)
|
||
|
||
|
||
def test_guard_crash_undercount_bounded_by_flush(tmp_path):
|
||
"""如实边界:崩溃前 ≤flush 阈值的已发音频未落账(有界少计)."""
|
||
guard = make_guard(tmp_path, flush_seconds=5.0)
|
||
call = guard.begin_stream("room-a")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
for _ in range(30):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
guard.note_audio_delivered(call, 16000, 160, 320) # 0.3s < flush 阈值
|
||
# 模拟进程直接消失(没有 stream_end/flush)
|
||
replayed = summarize_ledger(tmp_path / "asr-usage.jsonl", "w-test")
|
||
assert replayed["windows"]["w-test"]["audio_seconds"] == 0.0
|
||
# 正常路径(flush/stream_end)计数完整落账
|
||
guard2 = make_guard(tmp_path, flush_seconds=5.0)
|
||
call = guard2.begin_stream("room-b")
|
||
guard2.reserve_audio(call, 16000, 16000, 32000)
|
||
guard2.note_audio_delivered(call, 16000, 16000, 32000)
|
||
guard2.stream_end(call, "closed")
|
||
assert summarize_ledger(tmp_path / "asr-usage.jsonl", "w-test")["windows"][
|
||
"w-test"]["calls"]["room-b"]["audio_seconds"] == pytest.approx(1.0)
|
||
|
||
|
||
def test_summarize_cli_reproducible(tmp_path, monkeypatch):
|
||
guard = make_guard(tmp_path)
|
||
call = guard.begin_stream("room-a")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
guard.reserve_audio(call, 16000, 16000, 32000)
|
||
guard.note_audio_delivered(call, 16000, 16000, 32000)
|
||
guard.stream_end(call, "closed")
|
||
repo_root = Path(__file__).resolve().parents[1]
|
||
proc = subprocess.run(
|
||
[sys.executable, "-m", "cybergril.voice.asr_usage",
|
||
str(tmp_path / "asr-usage.jsonl"), "--window", "w-test"],
|
||
capture_output=True, text=True, timeout=60,
|
||
env={"PYTHONPATH": str(repo_root / "src"), "PATH": "/usr/bin:/bin"},
|
||
check=True,
|
||
)
|
||
out = json.loads(proc.stdout)
|
||
call_summary = out["windows"]["w-test"]["calls"]["room-a"]
|
||
assert call_summary["audio_seconds"] == pytest.approx(1.0)
|
||
assert call_summary["connect_attempts"] == 1
|
||
assert out["parse_errors"] == 0
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 2) 真实接缝:stt_funasr._run 在假 provider/假帧上走完整送数路径
|
||
# ---------------------------------------------------------------------------
|
||
|
||
@pytest.fixture
|
||
def fake_sdk(monkeypatch):
|
||
return _install_fake_sdk(monkeypatch)
|
||
|
||
|
||
async def test_seam_counts_fed_audio_both_sample_rates(fake_sdk, tmp_path):
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
guard = make_guard(tmp_path)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="cg-voice-abc")
|
||
await drive(funasr, stt, [frame_16k() for _ in range(30)] + ["flush"]
|
||
+ [frame_48k() for _ in range(30)] + [None])
|
||
cu = guard.calls["cg-voice-abc"]
|
||
# 16k 直接路径 0.3s + 48k→16k 重采样路径 0.3s(按实际输出帧计时)
|
||
assert cu.audio_seconds == pytest.approx(0.6)
|
||
assert cu.confirmed_audio_seconds == pytest.approx(0.6) # 全部送达成功
|
||
assert cu.connect_attempts == 2 # 两轮 utterance 两次建连
|
||
assert (cu.connect_failures, cu.connect_retries) == (0, 0)
|
||
records = read_records(tmp_path)
|
||
audio_records = [r for r in records if r["kind"] == "audio"]
|
||
assert [round(r["seconds"], 6) for r in audio_records] == [0.3, 0.3] # utterance 边界聚合
|
||
assert [round(r["confirmed_seconds"], 6) for r in audio_records] == [0.3, 0.3]
|
||
assert all(r["call"] == "cg-voice-abc" for r in records)
|
||
ends = [r for r in records if r["kind"] == "call_end"]
|
||
assert [r["reason"] for r in ends] == ["closed"]
|
||
# 假 provider 侧:每帧恰好一次出站,cfg/is_speaking 各一条文本
|
||
audio_payloads = fake_sdk.websockets.bytes_sent()
|
||
assert len(audio_payloads) == 60
|
||
assert all(len(p) == 320 for p in audio_payloads)
|
||
|
||
|
||
async def test_seam_silence_and_cancel_counted(fake_sdk, tmp_path):
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
guard = make_guard(tmp_path)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="room-silent")
|
||
# 静音数据帧只要真实送出就计数;不 flush 直接结束(用户取消/挂断)
|
||
silence = FakeFrame(b"\x00" * 320, 16000, 160)
|
||
await drive(funasr, stt, [silence] * 20 + [None])
|
||
cu = guard.calls["room-silent"]
|
||
assert cu.audio_seconds == pytest.approx(0.2)
|
||
records = read_records(tmp_path)
|
||
assert [r["reason"] for r in records if r["kind"] == "call_end"] == ["closed"]
|
||
# 反复结束:第二通历史不覆盖第一通
|
||
stt2 = funasr.FunASRSTT(usage=guard, call_label="room-silent-2")
|
||
await drive(funasr, stt2, [frame_16k(1600)] + [None]) # 0.1s
|
||
assert guard.calls["room-silent-2"].audio_seconds == pytest.approx(0.1)
|
||
assert len([r for r in read_records(tmp_path)
|
||
if r["kind"] == "call_start"]) == 2
|
||
|
||
|
||
async def test_seam_connect_failure_marks_error_and_retry_denied(fake_sdk, tmp_path):
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
guard = make_guard(tmp_path, max_connect_retries_per_call=2)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
fake_sdk.websockets.fail_attempts = {1, 3, 4}
|
||
|
||
with pytest.raises(OSError): # 初次建连失败 → 流异常结束
|
||
await drive(funasr, stt, [frame_16k(), None])
|
||
records = read_records(tmp_path)
|
||
assert records[-1]["kind"] == "call_end" and records[-1]["reason"] == "error"
|
||
|
||
stt2 = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
await drive(funasr, stt2, [frame_16k(), "flush", None]) # retry#1 成功
|
||
assert guard.calls["room-flaky"].connect_retries == 1
|
||
|
||
stt3 = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
with pytest.raises(OSError): # 上次成功后这次是普通建连,但连接失败
|
||
await drive(funasr, stt3, [frame_16k(), None])
|
||
|
||
stt4 = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
with pytest.raises(OSError): # retry#2(用尽上限)且仍失败
|
||
await drive(funasr, stt4, [frame_16k(), None])
|
||
|
||
stt5 = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
attempts_before = fake_sdk.websockets.attempts
|
||
with pytest.raises(AsrWindowClosed): # retry#3 建连前被拒:连假 provider 都不碰
|
||
await drive(funasr, stt5, [frame_16k(), None])
|
||
assert fake_sdk.websockets.attempts == attempts_before
|
||
blocks = [r for r in read_records(tmp_path) if r["kind"] == "block"]
|
||
assert blocks[-1]["reason"] == "connect-retries-exhausted"
|
||
cu = guard.calls["room-flaky"]
|
||
assert (cu.connect_attempts, cu.connect_failures, cu.connect_retries) == (4, 3, 2)
|
||
|
||
stt5 = funasr.FunASRSTT(usage=guard, call_label="room-flaky")
|
||
with pytest.raises(AsrWindowClosed): # 该通 retry 预算已用尽:再建连被拒
|
||
await drive(funasr, stt5, [frame_16k(), None])
|
||
# 新的一通有独立 retry 预算:两次正常 utterance 建连都放行
|
||
stt6 = funasr.FunASRSTT(usage=guard, call_label="room-flaky-2")
|
||
await drive(funasr, stt6, [frame_16k(), "flush", frame_16k(), "flush", None])
|
||
assert guard.calls["room-flaky-2"].connect_attempts == 2
|
||
assert guard.calls["room-flaky-2"].connect_retries == 0
|
||
|
||
|
||
async def test_seam_window_exhaustion_stops_before_crossing(fake_sdk, tmp_path):
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
stops: list[str] = []
|
||
guard = AsrUsageGuard(
|
||
make_policy(max_audio_seconds=0.05),
|
||
tmp_path / "asr-usage.jsonl", on_exhausted=stops.append)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="room-cap")
|
||
with pytest.raises(AsrWindowClosed):
|
||
await drive(funasr, stt, [frame_16k() for _ in range(6)] + [None])
|
||
# 0.05s = 5 帧放行,第 6 帧发送前拒绝:出站字节恰好 5 帧(provider 未被触线帧触碰)
|
||
assert len(fake_sdk.websockets.bytes_sent()) == 5
|
||
assert guard.calls["room-cap"].audio_seconds == pytest.approx(0.05)
|
||
assert guard.calls["room-cap"].confirmed_audio_seconds == pytest.approx(0.05)
|
||
assert stops == ["window-audio-exhausted"]
|
||
records = read_records(tmp_path)
|
||
assert records[-1]["kind"] == "call_end"
|
||
assert records[-1]["reason"] == "window_exhausted"
|
||
assert any(r["kind"] == "block" for r in records)
|
||
# 本窗口后续会话:新流在建流即被拒
|
||
stt2 = funasr.FunASRSTT(usage=guard, call_label="room-cap-2")
|
||
with pytest.raises(AsrWindowClosed):
|
||
await drive(funasr, stt2, [frame_16k(), None])
|
||
|
||
|
||
async def test_seam_recv_death_recorded_as_connection_died(fake_sdk, tmp_path):
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
guard = make_guard(tmp_path)
|
||
fake_sdk.websockets.ws_factory = lambda: FakeWS(recv_fail=True)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="room-dead")
|
||
await drive(funasr, stt, [frame_16k(), "flush", None], timeout=6)
|
||
records = read_records(tmp_path)
|
||
died = [r for r in records if r["kind"] == "connection_died"]
|
||
assert len(died) == 1 and died[0]["call"] == "room-dead"
|
||
assert guard.calls["room-dead"].connect_failures == 1
|
||
assert guard.calls["room-dead"].retry_pending is True
|
||
# 死亡后的下一次建连计为 retry(接收侧死亡未伴随建连失败也如实计数)
|
||
fake_sdk.websockets.ws_factory = None
|
||
stt2 = funasr.FunASRSTT(usage=guard, call_label="room-dead")
|
||
await drive(funasr, stt2, [frame_16k(), "flush", None])
|
||
assert guard.calls["room-dead"].connect_retries == 1
|
||
|
||
|
||
async def test_ledger_survives_room_deletion(fake_sdk, tmp_path):
|
||
"""删房成功会话账本移除该通;ASR 账本逐通历史原样保留."""
|
||
import cybergril.voice.stt_funasr as funasr
|
||
from cybergril.voice.session import VoiceConfig, VoiceSessionService
|
||
|
||
guard = make_guard(tmp_path)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="cg-voice-keep")
|
||
await drive(funasr, stt, [frame_16k(1600), "flush", None])
|
||
before = read_records(tmp_path)
|
||
|
||
service = VoiceSessionService(
|
||
config=VoiceConfig(livekit_url="ws://v.test", livekit_internal="http://v.test",
|
||
api_key="devkey", api_secret="s"),
|
||
http=httpx.AsyncClient(transport=httpx.MockTransport(_fake_twirp)))
|
||
session = await service.open_session(epoch=1)
|
||
ended = await service.end_session(session.session_id)
|
||
assert ended is not None
|
||
assert service.get(session.session_id) is None # 会话账本已移除
|
||
assert read_records(tmp_path) == before # ASR 账本一字未动
|
||
await service.aclose()
|
||
|
||
|
||
def _fake_twirp(request: httpx.Request) -> httpx.Response:
|
||
if request.url.path.endswith("CreateDispatch"):
|
||
return httpx.Response(200, json={"id": "AD_1"})
|
||
return httpx.Response(200, content=b"")
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 3) worker 接线:entrypoint 真的构造并注入 guard,触线真的关 job
|
||
# ---------------------------------------------------------------------------
|
||
|
||
async def test_worker_entrypoint_wires_guard(fake_sdk, tmp_path, monkeypatch, capsys):
|
||
import cybergril.voice.worker as worker
|
||
|
||
ledger = tmp_path / "worker-asr.jsonl"
|
||
env = {"CYBERGRIL_ASR_WINDOW_ID": "hel902-w1",
|
||
"CYBERGRIL_ASR_WINDOW_MAX_AUDIO_SECONDS": "2700",
|
||
"CYBERGRIL_ASR_MAX_CONNECT_RETRIES_PER_CALL": "6",
|
||
"CYBERGRIL_ASR_USAGE_LEDGER": str(ledger)}
|
||
monkeypatch.setattr(worker, "build_usage_from_env",
|
||
lambda on_exhausted=None: build_usage_from_env(
|
||
env, on_exhausted=on_exhausted, ledger_path=ledger))
|
||
ctx = FakeCtx("cg-voice-room1")
|
||
await worker.entrypoint(ctx)
|
||
captured = FakeAgentSession.last
|
||
assert captured is not None and captured.started is not None
|
||
stt_obj = captured.kwargs["stt"]
|
||
assert stt_obj._call_label == "cg-voice-room1" # 房间名 = 每通标识
|
||
guard = stt_obj._usage
|
||
assert isinstance(guard, AsrUsageGuard)
|
||
assert guard.policy.window_id == "hel902-w1"
|
||
# HEL-909 R2:本通打断通道挂在房间 data_received 上,会话结束有卸载回调
|
||
assert any(e == "data_received" for e, _ in ctx.room_listeners), \
|
||
"cg.interrupt 通道必须挂在本房间 data_received"
|
||
assert ctx.shutdown_callbacks, "通道卸载必须注册到 add_shutdown_callback"
|
||
out = capsys.readouterr().out
|
||
assert "asr_window=" in out and "hel902-w1" in out
|
||
# 触线回调真的接到 ctx.shutdown:预检越限帧 → 结束本通 job
|
||
call = guard.begin_stream("cg-voice-room1")
|
||
guard.reserve_audio(call, 16000, 16000, 32000)
|
||
guard.flush_audio(call)
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.reserve_audio(call, 16000, 16000 * 2700) # 会越 2700s 上限的帧
|
||
assert ctx.shutdown_calls, "on_exhausted 必须真的调用 ctx.shutdown 结束本通"
|
||
|
||
|
||
async def test_worker_entrypoint_default_is_fail_closed(fake_sdk, monkeypatch, capsys):
|
||
import cybergril.voice.worker as worker
|
||
|
||
monkeypatch.setattr(
|
||
worker, "build_usage_from_env",
|
||
lambda on_exhausted=None: build_usage_from_env({}, on_exhausted=on_exhausted,
|
||
ledger_path=None))
|
||
ctx = FakeCtx("cg-voice-room2")
|
||
await worker.entrypoint(ctx)
|
||
guard = FakeAgentSession.last.kwargs["stt"]._usage
|
||
assert guard.policy.armed is False # 未布防 = 真实 cap=0
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.begin_stream("cg-voice-room2")
|
||
assert "asr_window=" in capsys.readouterr().out
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 4) 2026-10-28 总管定点复核返工:持久化 fail-closed 与不确定出站口径
|
||
# ---------------------------------------------------------------------------
|
||
|
||
class FakeClock:
|
||
def __init__(self, t: float = 0.0) -> None:
|
||
self.t = t
|
||
|
||
def __call__(self) -> float:
|
||
return self.t
|
||
|
||
def advance(self, dt: float) -> None:
|
||
self.t += dt
|
||
|
||
|
||
def _rework_guard(tmp_path: Path, clock: FakeClock, *, on_exhausted=None,
|
||
**overrides) -> AsrUsageGuard:
|
||
values = dict(window_id="w-rework", max_audio_seconds=10.0,
|
||
flush_seconds=1e9, resync_seconds=5.0)
|
||
values.update(overrides)
|
||
return AsrUsageGuard(AsrWindowPolicy(**values), tmp_path / "asr-usage.jsonl",
|
||
clock=clock, on_exhausted=on_exhausted)
|
||
|
||
|
||
def _feed(guard: AsrUsageGuard, call: str, seconds: float, *, deliver: bool = True) -> None:
|
||
frames = round(seconds * 100)
|
||
for _ in range(frames):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
if deliver:
|
||
guard.note_audio_delivered(call, 16000, 160, 320)
|
||
|
||
|
||
def test_ledger_write_failure_never_regresses_budget(tmp_path, monkeypatch):
|
||
"""总管复现的回归路径:写失败→flush 丢缓冲→重放按旧账回退,修复后不再发生."""
|
||
import pathlib
|
||
|
||
stops: list[str] = []
|
||
clock = FakeClock(0.0)
|
||
guard = _rework_guard(tmp_path, clock, max_audio_seconds=1.5,
|
||
on_exhausted=stops.append)
|
||
call = guard.begin_stream("room")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
_feed(guard, call, 0.8)
|
||
guard.flush_audio(call) # 0.8s 落账成功
|
||
assert guard.window_audio_seconds == pytest.approx(0.8)
|
||
|
||
real_open = pathlib.Path.open
|
||
|
||
def fail_append(self, mode="r", *args, **kwargs):
|
||
if "a" in (mode or ""):
|
||
raise OSError("append refused (injected)")
|
||
return real_open(self, mode, *args, **kwargs)
|
||
|
||
monkeypatch.setattr(pathlib.Path, "open", fail_append)
|
||
|
||
_feed(guard, call, 0.5)
|
||
guard.flush_audio(call) # 0.5s 落账失败:内存保全、立即停机、不许回退
|
||
assert guard.window_audio_seconds == pytest.approx(1.3)
|
||
assert guard.persistence_stop_reason == "ledger-write-failed"
|
||
assert stops == ["ledger-write-failed"] # 停机回调恰好一次并带原因
|
||
|
||
clock.advance(6.0) # 越过 resync:重放不得把内存拉回旧账本值 0.8
|
||
with pytest.raises(AsrWindowClosed) as ei:
|
||
_feed(guard, call, 0.6) # 复现脚本里的「再送 0.6s」:现在被拒
|
||
assert "ledger-write-failed" in str(ei.value)
|
||
assert guard.window_audio_seconds == pytest.approx(1.3) # 无预算倒退
|
||
# 写失败注入仍生效期间 block 记录同样写不落账——停机事实以日志与内存为准
|
||
assert not [r for r in read_records(tmp_path) if r["kind"] == "block"]
|
||
# 停机状态在 summary/summary_line 可见(操作员可核对)
|
||
assert guard.summary()["persistence_stop_reason"] == "ledger-write-failed"
|
||
assert "ledger-write-failed" in guard.summary_line()
|
||
|
||
|
||
def test_ledger_read_failure_stops_outbound_preserving_memory(tmp_path, monkeypatch):
|
||
import pathlib
|
||
|
||
clock = FakeClock(0.0)
|
||
guard = _rework_guard(tmp_path, clock)
|
||
call = guard.begin_stream("room")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
_feed(guard, call, 1.0)
|
||
guard.flush_audio(call)
|
||
|
||
def boom(self, *args, **kwargs):
|
||
raise OSError("read refused (injected)")
|
||
|
||
monkeypatch.setattr(pathlib.Path, "read_text", boom)
|
||
clock.advance(6.0) # resync 触发严格读:读失败 → fail-closed
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
assert guard.persistence_stop_reason == "ledger-read-failed"
|
||
assert guard.window_audio_seconds == pytest.approx(1.0) # 不清零、不倒退
|
||
with pytest.raises(AsrWindowClosed): # 新通也不放行
|
||
guard.begin_stream("room-2")
|
||
|
||
|
||
def test_ledger_corruption_stops_outbound(tmp_path):
|
||
clock = FakeClock(0.0)
|
||
guard = _rework_guard(tmp_path, clock)
|
||
call = guard.begin_stream("room")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
_feed(guard, call, 1.0)
|
||
guard.flush_audio(call)
|
||
|
||
with (tmp_path / "asr-usage.jsonl").open("a", encoding="utf-8") as fh:
|
||
fh.write("{corrupt-not-json\n") # 中途损坏一行
|
||
clock.advance(6.0)
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
assert guard.persistence_stop_reason == "ledger-corrupt"
|
||
assert guard.window_audio_seconds == pytest.approx(1.0) # 损坏不套用、内存保全
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.check_connect("room")
|
||
|
||
|
||
def test_ledger_missing_after_seen_stops_outbound(tmp_path):
|
||
clock = FakeClock(0.0)
|
||
guard = _rework_guard(tmp_path, clock)
|
||
call = guard.begin_stream("room")
|
||
guard.check_connect(call)
|
||
guard.note_connect_result(call, ok=True)
|
||
_feed(guard, call, 1.0)
|
||
guard.flush_audio(call) # 账本已存在(读过/写过)
|
||
|
||
(tmp_path / "asr-usage.jsonl").unlink() # 中途消失 = 已落账事实丢失
|
||
clock.advance(6.0)
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.reserve_audio(call, 16000, 160, 320)
|
||
assert guard.persistence_stop_reason == "ledger-missing"
|
||
assert guard.window_audio_seconds == pytest.approx(1.0)
|
||
# 停机事实已落回重建的账本(guard_stop):汇总 CLI 可见
|
||
assert summarize_ledger(tmp_path / "asr-usage.jsonl", "w-rework")[
|
||
"windows"]["w-rework"]["stopped"] is True
|
||
|
||
|
||
def test_guard_stop_record_keeps_window_stopped_across_restart(tmp_path):
|
||
ledger = tmp_path / "asr-usage.jsonl"
|
||
lines = [
|
||
{"kind": "audio", "window": "w", "call": "room", "seconds": 2.0,
|
||
"confirmed_seconds": 2.0, "t": 1},
|
||
{"kind": "guard_stop", "window": "w", "reason": "ledger-corrupt", "t": 2},
|
||
]
|
||
ledger.write_text("".join(json.dumps(x) + "\n" for x in lines), encoding="utf-8")
|
||
guard = AsrUsageGuard(
|
||
AsrWindowPolicy(window_id="w", max_audio_seconds=10.0), ledger)
|
||
# 同窗重启不恢复出站;已知消耗(2.0s)仍被重放保全
|
||
assert guard.persistence_stop_reason == "ledger-corrupt"
|
||
assert guard.window_audio_seconds == pytest.approx(2.0)
|
||
with pytest.raises(AsrWindowClosed):
|
||
guard.begin_stream("room-new")
|
||
|
||
|
||
async def test_seam_uncertain_outbound_counted_conservatively(fake_sdk, tmp_path):
|
||
"""假 ws 先记下 payload 再 raise/取消:预算仍包含该帧,确证量不包含."""
|
||
import cybergril.voice.stt_funasr as funasr
|
||
|
||
guard = make_guard(tmp_path)
|
||
|
||
def flaky(mode: str, ok_frames: int) -> FakeWS:
|
||
ws = FakeWS(server_msgs=[])
|
||
ws.fail_send = mode
|
||
ws.fail_bytes_after = ok_frames
|
||
return ws
|
||
|
||
# 场景 1:第 3 帧送出后 raise(帧可能已出站)
|
||
fake_sdk.websockets.ws_factory = lambda: flaky("raise", 2)
|
||
stt = funasr.FunASRSTT(usage=guard, call_label="room-raise")
|
||
with pytest.raises(OSError):
|
||
await drive(funasr, stt, [frame_16k() for _ in range(3)] + [None])
|
||
cu = guard.calls["room-raise"]
|
||
assert cu.audio_seconds == pytest.approx(0.03) # 保守计费含失败帧
|
||
assert cu.confirmed_audio_seconds == pytest.approx(0.02) # 确证只有前两帧
|
||
ws_raise = fake_sdk.websockets.sockets[-1]
|
||
assert len([p for p in ws_raise.sent if isinstance(p, bytes)]) == 3 # 帧确实已出站
|
||
rec = [r for r in read_records(tmp_path)
|
||
if r["kind"] == "audio" and r["call"] == "room-raise"][-1]
|
||
assert rec["seconds"] == pytest.approx(0.03)
|
||
assert rec["confirmed_seconds"] == pytest.approx(0.02)
|
||
assert [r for r in read_records(tmp_path)
|
||
if r["kind"] == "call_end" and r["call"] == "room-raise"][-1]["reason"] == "error"
|
||
|
||
# 场景 2:第 2 帧送出后被取消:同样不退还预记
|
||
fake_sdk.websockets.ws_factory = lambda: flaky("cancel", 1)
|
||
stt2 = funasr.FunASRSTT(usage=guard, call_label="room-cancel")
|
||
with pytest.raises(asyncio.CancelledError):
|
||
await drive(funasr, stt2, [frame_16k() for _ in range(2)] + [None])
|
||
cu2 = guard.calls["room-cancel"]
|
||
assert cu2.audio_seconds == pytest.approx(0.02)
|
||
assert cu2.confirmed_audio_seconds == pytest.approx(0.01)
|
||
ws_cancel = fake_sdk.websockets.sockets[-1]
|
||
assert len([p for p in ws_cancel.sent if isinstance(p, bytes)]) == 2
|
||
|
||
# 触线拒绝仍然不碰 provider:越限帧在 reserve 阶段就拒绝,ws.send 不执行
|
||
guard_cap = AsrUsageGuard(
|
||
make_policy(max_audio_seconds=0.01), tmp_path / "cap.jsonl")
|
||
fake_sdk.websockets.ws_factory = None
|
||
stt3 = funasr.FunASRSTT(usage=guard_cap, call_label="room-cap2")
|
||
with pytest.raises(AsrWindowClosed):
|
||
await drive(funasr, stt3, [frame_16k() for _ in range(3)] + [None])
|
||
ws_cap = fake_sdk.websockets.sockets[-1]
|
||
assert len([p for p in ws_cap.sent if isinstance(p, bytes)]) == 1 # 只有第 1 帧出站
|