Contracts (the shared language): - percept / state / context / utterance / render / audio / capabilities - Capability model instead of vendor schemas (see below) Providers (capability-based abstraction): - LLM + Embedding: OpenAI-compatible shape IS the standard, adopted directly - ASR: internal contract is an event stream (partial -> final), because /v1/audio/transcriptions has no partial and dialogue needs it for endpointing. OpenAI adapter degrades to supports_partial=False. - TTS: internal contract carries tags/emotion/style, because /v1/audio/speech has no paralinguistics and [laughter]/[breath] are the core of feeling alive. Unsupported providers are stripped AND reported via Degradation -- silent degradation is forbidden. - Avatar: no de-facto standard, custom RenderCmd protocol driven by State. Core: - mood: dynamical system, slow variables, negativity bias (grudge), numbing after prolonged negativity - bazi: four pillars -> numeric vector; 'ming' is data, not prose - projection: phi, system event -> in-world statement; uncovered events are counted as projection gaps Storage: - SQLite schema, three memory layers, dyad ledger - accessibility floor 0.03 (not 0): forgettable but still there Config split: - models.yaml is the ONLY place model versions live - character.yaml holds world rules / emotion params, no version numbers 58 tests pass; ruff clean.
64 lines
1.7 KiB
Python
64 lines
1.7 KiB
Python
"""音频契约。
|
||
|
||
单独成文件,因为音频是唯一一个「流」语义贯穿全链路的模态:
|
||
麦克风 → AEC → VAD → ASR → ... → TTS → 扬声器。
|
||
每一步都是块,每一步都有格式,格式错了就是静默失真。
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
from dataclasses import dataclass, field
|
||
|
||
SAMPLE_RATE_16K = 16_000
|
||
SAMPLE_RATE_48K = 48_000
|
||
|
||
#: Silero VAD v6 只吃 16kHz。麦克风来的 48k 必须先重采样,否则静默识别失败。
|
||
VAD_RATE = SAMPLE_RATE_16K
|
||
|
||
|
||
@dataclass(frozen=True, slots=True)
|
||
class AudioFormat:
|
||
sample_rate: int = SAMPLE_RATE_16K
|
||
channels: int = 1
|
||
sample_width: int = 2 # bytes,2 = int16
|
||
|
||
@property
|
||
def bytes_per_second(self) -> int:
|
||
return self.sample_rate * self.channels * self.sample_width
|
||
|
||
def __str__(self) -> str:
|
||
return f"{self.sample_rate}Hz/{self.channels}ch/{self.sample_width * 8}bit"
|
||
|
||
|
||
PCM16_16K = AudioFormat(SAMPLE_RATE_16K, 1, 2)
|
||
PCM16_48K = AudioFormat(SAMPLE_RATE_48K, 1, 2)
|
||
|
||
|
||
@dataclass(frozen=True, slots=True)
|
||
class AudioChunk:
|
||
"""一段 PCM 音频。"""
|
||
|
||
data: bytes
|
||
format: AudioFormat = PCM16_16K
|
||
ts: int = 0
|
||
"""采集时间戳(unix ms)。用于把 ASR 结果与副语言结果按时间对齐。"""
|
||
|
||
@property
|
||
def duration_ms(self) -> float:
|
||
return len(self.data) / self.format.bytes_per_second * 1000
|
||
|
||
@property
|
||
def is_empty(self) -> bool:
|
||
return not self.data
|
||
|
||
|
||
@dataclass(frozen=True, slots=True)
|
||
class AudioFrameStats:
|
||
"""一帧的能量统计。VAD 不需要看波形,只需要这些。"""
|
||
|
||
ts: int
|
||
rms: float
|
||
peak: float
|
||
is_speech: bool = False
|
||
extra: dict[str, float] = field(default_factory=dict)
|