diff --git a/deploy/hel919-continuation/README.md b/deploy/hel919-continuation/README.md index b4cfb0a..f9a6077 100644 --- a/deploy/hel919-continuation/README.md +++ b/deploy/hel919-continuation/README.md @@ -13,15 +13,15 @@ - **产品基线(本卡起点,不回退、不合并、不改 PR 目标)**: `agent/2/182d2402a8df` @ `2e91a3c91e90944991886b21f7167a6913f41efb` (含 HEL-916 两轮返工 e74531c/2e91a3c、HEL-914 ffbc8c2)。 -- **运行包提交**:基线之后的追加提交。除本包文件外,产品代码只含**极小 - scope 扩展**(`PersistentBudget` 可选 `scope` 参数 + app 读 - `CYBERGRIL_BUDGET_SCOPE` + 封账工具 `--scope`;缺省 "" 时键名与语义与旧 - 完全一致,旧实例零影响),改动仅限: - `src/cybergril/life/model.py`、`src/cybergril/api/app.py`、 +- **运行包提交**:基线之后追加 scope 隔离、请求出口截止和资料保全修复。 + `PersistentBudget` 读取 scope/deadline,app 从环境注入;语义 SDK 的嵌入 + 出口也检查同一期限;封账工具支持 `--scope`。缺省空 scope 保持旧行为。 + 相对产品基线的代码改动限于 `src/cybergril/life/model.py`、 + `src/cybergril/api/app.py`、`src/cybergril/memory/semantic.py`、 `tools/seal_budget_hel879.py`: ```bash git merge-base --is-ancestor 2e91a3c91e90944991886b21f7167a6913f41efb HEAD && echo ok - git diff --stat 2e91a3c91e90944991886b21f7167a6913f41efb HEAD -- src web config tools # 只有上述 3 个文件 + git diff --stat 2e91a3c91e90944991886b21f7167a6913f41efb HEAD -- src web config tools # 只有上述 4 个文件 ``` - 需求基准 v1.12(`docs/requirements/`),检查见 `handoff-HEL-919.json`。 @@ -74,10 +74,19 @@ python3 deploy/hel919-continuation/migrate_same_her.py migrate \ --db <源实例库> --base /opt/cybergril-hel919 \ --data-dir <源向量库目录>=sem/vec --data-dir <源推导史目录>=sem/hist -# 3) 回退(只动副本,源库永不回退) +# 3) 回退(只动副本;有迁移后变化时默认拒绝,源库永不回退) python3 deploy/hel919-continuation/migrate_same_her.py rollback --base /opt/cybergril-hel919 ``` +回退先核数据库逻辑快照(含 WAL 中已提交内容)和资料文件哈希,更新已有记忆 +或向量也算变化,不能只数新增聊天。没有快照的旧迁移记录同样默认拒绝。 +确需强制回退时,操作者显式加 `--force`;工具先在 +`backups/rollback-<时间>-<唯一后缀>/data/` 一致性复制数据库、向量/推导史及迁移 +记录,核源与备份逻辑快照相等,并写 `manifest.json` 后才回退。归档失败、 +核验失败或实例仍活跃均拒绝删除。要恢复本次新资料,先停止实例,核归档 +manifest 中每个文件 SHA256,然后将归档的 `data/` 恢复到一个**新的空目录**; +不得覆盖原实例。自动化测试实际从该归档恢复并读回新增聊天和向量。 + 迁移铁律(工具内建自检,任一不过自动回退并报错): - 空库/缺出生底色/缺出生诠释 → **拒绝**(不走新建人格,也不因已有档案拒绝); - 同一位她前后可核:身份指纹、bazi/persona/记忆/聊天/生活/关系逐表计数相等; @@ -103,22 +112,28 @@ bash deploy/hel919-continuation/instance_ctl.sh watch # 到点自动 cl 安全属性(替身演练 + 单测证明,见 §6): - **默认关闭**:无 `window-active` 标记 / 预算全 0 → 一切真实出口在发送前 拒绝(fail-closed),start 也被守卫拒绝; -- **过期拒绝**:deadline 持久在 window.json,到期 watch 自动 close,此后 - 重启被守卫拒绝;进程重启不重置期限、不新增用量(账在库内); +- **过期拒绝**:start 先拉起专属 watcher 并核 PID/命令/目录/就绪记录,失败 + 不启动 API;再把 window.json 的 deadline 注入 `CYBERGRIL_WINDOW_DEADLINE`。 + 新 scope 把期限钉在自身 `budget_total` 键,重启不能延长。主聊、后台、SDK + 提取在 acquire 处、嵌入在 SDK 出口处到期拒绝,**即使进程和 watcher 仍活着** + 也不新增调用。缺失/非法期限同样拒绝;在途结算允许完成,不倒改旧账; - **只停自己**:stop/close 只操作身份核验过的本实例 pid(cmdline+cwd 双验, PID 复用不动手);不调用旧 force_end / worker / bridge / tts 脚本,不触碰 8300/8310/8320/8350、7880、8081、9880、8090 等共享服务; - **有界收尾**:在途排空上限 HEL919_DRAIN_SEC(默认 30s); - **收尾可核**:close 写 close.log / window.json closed_at / 封账标记, - 运行日志在 logs/。 + 运行日志在 logs/;watcher 句柄为 logs/expiry.pid、就绪证据为 expiry.ready。 + watch 负责到期停服,出站闸不依赖它按时被调度。机器故障下独立进程存续 + 仅尽力保证;实际部署可再安装下方 systemd 定时模板,不能据模板宣称已布防。 ## 5. 获批启用(总管操作;enablement-plan.md 是完整安排) 1. `check` 通过后,按批准写 etc/models.yaml 预算值(新授权范围); 2. 生成 window.json(approved/started_at/deadline_at/deadline_epoch/ authorization_comment)+ `touch window-active`; -3. 生成 expiry timer(`cybergril-hel919-expiry.template.service` + 由 - deadline 换算的 OnCalendar)→ `instance_ctl.sh start`。 +3. `instance_ctl.sh start` 自动布防并核实自有 watcher 后才启动;可另用 + `cybergril-hel919-expiry.template.service` 和该 deadline 生成 systemd timer。 + timer 与 watcher 均只 close 本实例,重复 close 幂等。 ## 6. 离线验证(零真实调用) @@ -131,5 +146,7 @@ python3 scripts/check_requirements.py --base-ref 2e91a3c91e90944991886b21f7167a6 覆盖:空档案拒绝、同一位她前后保留、旧账冻结与源库不变、幂等/冲突/回退、 新用量隔离且重启不重置、默认关闭零出站、过期拒绝、stop 只操作自有句柄、 -队列只盘数量与 ID。**未测到**:真实 LLM/ASR/TTS 行为、真实部署、浏览器 +队列只盘数量与 ID;实际 API/后台在活进程中过期零 provider 出站、SDK 提取/ +嵌入过期拒绝、回退新增默认拒绝、强制回退归档可恢复及备份失败不删。 +**未测到**:真实 LLM/ASR/TTS 行为、真实部署、浏览器 端到端、Safari/手机;真实自然度由用户体验判断,离线证据不冒充。 diff --git a/deploy/hel919-continuation/enablement-plan.md b/deploy/hel919-continuation/enablement-plan.md index 01c8141..a717e86 100644 --- a/deploy/hel919-continuation/enablement-plan.md +++ b/deploy/hel919-continuation/enablement-plan.md @@ -19,15 +19,20 @@ ## 2. 运行边界(工程自选,原有共池直接复用) - **窗口**:单次 ≤ 72 小时(deadline 写入 window.json,进程重启不重置; - 到期 expiry 自动收尾,重启被 window_guard 拒绝)。 + start 先核自有 watcher 已就绪,运行中请求出口按同一期限拒绝新增调用, + 到期 watcher 自动收尾;期限持久在新 scope,重启不得延长)。 - **额度**:只复用既有共池机制——models.yaml `daily_tokens` 与 `daily_requests` 四类(dialogue / life / background / semantic)。这些 上限真实控制的合计数就是:每日总 token + 每类每日真实 provider 出口 请求次数(含失败/取消/重试,预占不退)。建议起步值:dialogue 200/日、 - life 40/日、background 120/日、semantic 40/日、daily_tokens 300k/日; - 总管可按首日实际 `budget_total`/`request_usage` 读数调整,不需要用户 - 拍板金额。不为每个子类另造新系统。 -- **不承诺**:「8 轮必然花多少」不承诺;历史全类别 345186/747 不当作每轮 + life 40/日、background 120/日、semantic 0/日、daily_tokens 300k/日; + 本次先保持 SDK 语义提取/嵌入关闭,使用已明示的 R3 词面 FTS 通道,保留原 + 向量及其出处,不把它说成真实语义记忆体验。累计 total_requests=1080、 + total_tokens=900000(最长三日的同一总盘,不按重启重置)。这不是新增费用 + 审批;这些数值只是未启用模板在实际试用前需明确填入的运行边界。 + 以上均为待批准建议;不得按首日读数自动提高上限。需要调整时说明具体 + 影响后再取批准,无需用户提供金额或套餐资料。不为每个子类另造新系统。 +- **不承诺**:「8 轮必然花多少」不承诺;历史全类别 345186/7 轮不当作每轮 主聊成本引用(那是全历史合计,不是本轮口径)。 - **语义账**:如需真实记忆检索,semantic.yaml 按累计算式续唯一权威账 (不动冻结余额);默认可保持 0,先用 R3 词面 FTS 通道体验文字接续。 @@ -46,7 +51,7 @@ - 任何观察异常:总管 `instance_ctl.sh close` 立即收尾(同到期路径)。 - 到期/封账后:**新聊天与新记忆保留**在接续库,不自动覆盖原实例、不删作 测试垃圾;原实例与其旧账、旧封账、归档全程未动。 -- 回退:`migrate_same_her.py rollback` 只回退接续副本到迁移前状态,源库 - 永不受影响。 +- 回退:默认拒绝删除迁移后有变化的接续资料;显式 --force 也必须先一致性 + 归档数据库/向量/推导史、核验成功可还原才回退。归档失败不删,源库永不受影响。 - 真实自然度的体验判断由白栖知决定;离线证据只证明接线正确,不冒充体验 通过。 diff --git a/deploy/hel919-continuation/instance_ctl.sh b/deploy/hel919-continuation/instance_ctl.sh index 962ddb6..3be2a3c 100644 --- a/deploy/hel919-continuation/instance_ctl.sh +++ b/deploy/hel919-continuation/instance_ctl.sh @@ -33,6 +33,8 @@ DRAIN=${HEL919_DRAIN_SEC:-30} PKG=$CODE/deploy/hel919-continuation LOG=$BASE/logs/api.log PIDF=$BASE/logs/api.pid +WATCH_PIDF=$BASE/logs/expiry.pid +WATCH_READY=$BASE/logs/expiry.ready MARKER="serve_dual_8360_8361.py" PY=${HEL919_PY:-python3} # 守卫/检查用系统 python(仅标准库) @@ -53,6 +55,61 @@ ours() { } pidfile_pid() { [ -f "$PIDF" ] && cat "$PIDF" 2>/dev/null; } +watch_ours() { + local p=$1 dl=$2 cmd cw + case "$p" in ''|*[!0-9]*) return 1 ;; esac + case "$dl" in ''|*[!0-9]*) return 1 ;; esac + kill -0 "$p" 2>/dev/null || return 1 + cmd=$(tr '\0' ' ' < "/proc/$p/cmdline" 2>/dev/null) || return 1 + cw=$(readlink "/proc/$p/cwd" 2>/dev/null) || return 1 + case "$cmd" in *"$PKG/instance_ctl.sh watch $dl "*) ;; *) return 1 ;; esac + [ "$(readlink -f "$cw")" = "$(readlink -f "$CODE")" ] +} + +disarm_watch() { + local p dl ready_pid + [ -f "$WATCH_PIDF" ] || return 0 + p=$(cat "$WATCH_PIDF") + read -r ready_pid dl < "$WATCH_READY" 2>/dev/null || return 1 + [ "$p" = "$ready_pid" ] || return 1 + if kill -0 "$p" 2>/dev/null; then + if [ "$p" != "$$" ]; then + watch_ours "$p" "$dl" || return 1 + kill "$p" || return 1 + fi + fi + rm -f "$WATCH_PIDF" "$WATCH_READY" +} + +arm_watch() { + local dl=$1 p i=0 + if [ -f "$WATCH_PIDF" ]; then + p=$(cat "$WATCH_PIDF") + if kill -0 "$p" 2>/dev/null; then + watch_ours "$p" "$dl" && [ "$(cat "$WATCH_READY" 2>/dev/null)" = "$p $dl" ] && return 0 + echo "REFUSED: 截止句柄不是本实例当前窗口" >&2 + return 1 + fi + fi + rm -f "$WATCH_PIDF" "$WATCH_READY" + ( + cd "$CODE" || exit 1 + nohup bash "$PKG/instance_ctl.sh" watch "$dl" >> "$BASE/logs/watch.log" 2>&1 & + echo "$!" > "$WATCH_PIDF" + ) + p=$(cat "$WATCH_PIDF") + while [ "$i" -lt 20 ]; do + if watch_ours "$p" "$dl" && [ "$(cat "$WATCH_READY" 2>/dev/null)" = "$p $dl" ]; then + return 0 + fi + i=$((i+1)); sleep 0.1 + done + watch_ours "$p" "$dl" && kill "$p" + rm -f "$WATCH_PIDF" "$WATCH_READY" + echo "REFUSED: 截止 watcher 未就绪,API 不启动" >&2 + return 1 +} + check() { local rc=0 for d in "$BASE" "$CODE" "$BASE/etc" "$BASE/data" "$BASE/logs"; do @@ -90,7 +147,7 @@ EOF } start() { - local p + local p deadline p=$(pidfile_pid) if [ -n "$p" ] && kill -0 "$p" 2>/dev/null; then if ours "$p"; then echo "REFUSED: 本实例已在运行 (pid $p) — 先 stop" >&2 @@ -101,7 +158,10 @@ start() { [ -x "$VENV" ] || { echo "REFUSED: venv python 不可执行: $VENV" >&2; exit 1; } mkdir -p "$BASE/logs" "$BASE/data" # 授权核验:默认关闭;未批准/过期窗口一律拒绝启动 - "$PY" "$PKG/window_guard_hel919.py" "$BASE" || exit 1 + deadline=$("$PY" "$PKG/window_guard_hel919.py" "$BASE" --deadline) || exit 1 + # Verify a real, identity-checked watcher BEFORE permitting API startup. + arm_watch "$deadline" || exit 1 + "$PY" "$PKG/window_guard_hel919.py" "$BASE" || { disarm_watch; exit 1; } cd "$CODE" || exit 1 set -a export CYBERGRIL_DB=$BASE/data/companion.db @@ -115,6 +175,7 @@ start() { export TRANSFORMERS_OFFLINE=1 export CYBERGRIL_MODELS=$BASE/etc/models.yaml export CYBERGRIL_BUDGET_SCOPE=${HEL919_SCOPE:-hel919} # 新账独立 budget_total 键;旧键逐字保留 + export CYBERGRIL_WINDOW_DEADLINE=$deadline export CYBERGRIL_VOICE_CONFIG=$BASE/etc/voice.yaml export CYBERGRIL_WEB=$CODE/web/dist export PYTHONPATH=$CODE/src @@ -133,6 +194,7 @@ start() { tail -20 "$LOG" >&2 if ours "$p"; then kill "$p"; sleep 2; ours "$p" && kill -9 "$p"; fi rm -f "$PIDF" + disarm_watch || true echo "已清理自己刚起的进程(日志保留)" >&2 exit 1 } @@ -178,6 +240,7 @@ if p.exists(): p.write_text(json.dumps(w, indent=2) + "\n") EOF [ -f "$BASE/window-active" ] && rm -f "$BASE/window-active" + disarm_watch || rc=1 echo "hel919 window closed(本实例新账已显式封账;资料与全部账目保留)" return $rc } @@ -206,8 +269,13 @@ case "${1:-}" in DL=${2:?deadline epoch seconds required} case "$DL" in *[!0-9]*|'') usage ;; esac mkdir -p "$BASE/logs" + BOUND=$("$PY" "$PKG/window_guard_hel919.py" "$BASE" --deadline --allow-expired) || exit 1 + [ "$DL" = "$BOUND" ] || { echo "REFUSED: watch deadline 与 window.json 不同" >&2; exit 1; } + cd "$CODE" || exit 1 + echo "$$" > "$WATCH_PIDF" + printf '%s %s\n' "$$" "$DL" > "$WATCH_READY" echo "[$(date -u +%FT%TZ)] watch start, deadline $DL (pid $$)" >> "$BASE/logs/watch.log" - while [ "$(date +%s)" -lt "$DL" ]; do sleep 30; done + while [ "$(date +%s)" -lt "$DL" ]; do sleep 1; done echo "[$(date -u +%FT%TZ)] DEADLINE REACHED — close" >> "$BASE/logs/watch.log" close ;; diff --git a/deploy/hel919-continuation/manifest.json b/deploy/hel919-continuation/manifest.json index 0459730..6a2a164 100644 --- a/deploy/hel919-continuation/manifest.json +++ b/deploy/hel919-continuation/manifest.json @@ -5,26 +5,39 @@ "product_baseline": { "branch": "agent/2/182d2402a8df", "commit": "2e91a3c91e90944991886b21f7167a6913f41efb", - "includes": ["e74531cf98540dd35198753fa8db87ac75d57a94", "ffbc8c24963c23b0ce108d4f7563871568825936"] + "includes": [ + "e74531cf98540dd35198753fa8db87ac75d57a94", + "ffbc8c24963c23b0ce108d4f7563871568825936" + ] }, - "package_rule": "基线之后新增 deploy/hel919-continuation/、tests/test_hel919_continuation.py、docs/requirements/handoff-HEL-919.json;产品代码仅极小 scope 扩展:src/cybergril/life/model.py(PersistentBudget 可选 scope,缺省 ''=旧键旧语义)、src/cybergril/api/app.py(读 CYBERGRIL_BUDGET_SCOPE)、tools/seal_budget_hel879.py(--scope);其余 src/web/config/tools 零改动", + "package_rule": "基线2e91a3c后:接续运行包、测试及交接;产品接缝仅scope隔离、持久deadline与API/语义SDK出站检查(life/model.py、api/app.py、memory/semantic.py、seal_budget_hel879.py),旧空scope行为保留。总管从99c5de1接手补齐原审核未完成的截止和回退保护,未部署。", "files": [ - "README.md", "manifest.json", "migrate_same_her.py", "instance_ctl.sh", - "window_guard_hel919.py", "models.template.yaml", "semantic.template.yaml", - "cybergril-hel919-expiry.template.service", "queue_policy.md", - "enablement-plan.md", "test_ctl_scripts.sh" + "README.md", + "manifest.json", + "migrate_same_her.py", + "instance_ctl.sh", + "window_guard_hel919.py", + "models.template.yaml", + "semantic.template.yaml", + "cybergril-hel919-expiry.template.service", + "queue_policy.md", + "enablement-plan.md", + "test_ctl_scripts.sh" ], "reuse": [ "deploy/hel913-unified(启停/身份核验/模板口径)", "deploy/hel914-user(窗口守卫形态;空档案限制由 migrate_same_her.py 取代)", - "src/cybergril/think/budget.py + life/model.py PersistentBudget(持久预算,零改动复用)", + "think/budget.py与life/model.py原持久预算;最小scope与期限扩展,不建独立账务平台", "tools/seal_budget_hel879.py(显式封账,close 复用)", "config/semantic.example.yaml(权威账续账算式)" ], "defaults": { - "deployed": false, "listening_ports": [], "real_calls": 0, + "deployed": false, + "listening_ports": [], + "real_calls": 0, "budgets": "models/semantic 模板全 0(fail-closed)", - "shared_services_touched": [] + "shared_services_touched": [], + "deadline": "新scope缺失/非法/过期拒绝;有效期限持久化,重启不能延长" }, "boundaries": "不动 8300/8310/8320/8350、7880/8081、9880/8090、836x/8082/9880;不改模型/认证/运行时;不安装依赖;启用需总管获批窗口(enablement-plan.md)" } diff --git a/deploy/hel919-continuation/migrate_same_her.py b/deploy/hel919-continuation/migrate_same_her.py index be3a9a6..e294b99 100644 --- a/deploy/hel919-continuation/migrate_same_her.py +++ b/deploy/hel919-continuation/migrate_same_her.py @@ -9,16 +9,14 @@ deploy/hel914-user/activate.sh 只接受空库,这是它不能续用的原因 三条铁律: 1. **源库只读**。一律以 `mode=ro` 打开,任何路径都不写源库;旧封账、旧 计数、冻结值、旧失败记录原样保留在源库,迁移后逐项核对不变。 -2. **旧账冻结、新账隔离**。迁移副本中,旧 `budget_total` 的 - sealed/sealed_* 标记被移入冻结出处键 `cont_prior_sealed*`(仅删副本里的 - live sealed 键,源库不动);token_usage/request_usage 历史行与 - total_requests/total_tokens 计数原样保留,新用量由既有 PersistentBudget - 机制在**同一副本**上继续累计——新旧在同一份账里天然可分(cont_migrated_at - 之前/之后),不存在第二份权威账并发消费。语义嵌入权威账(embed_ops) +2. **旧账冻结、新账隔离**。旧 sealed*/total_* 原键原值保留, + 新用量由 PersistentBudget 的 hel919 scope 单独记账。 + 语义嵌入权威账(embed_ops) **不复制**:它仍是唯一权威账文件,经 semantic.template.yaml 的累计上限 算式续账(见 README §语义账)。 3. **幂等与回退**。同一源重复迁移 = 安全 no-op;不同源拒绝;迁移前目标库 - 如已存在先做一致性备份,`rollback` 按 continuation.json 精确还原或清除。 + 如已存在先做一致性备份。回退发现任何迁移后变化默认拒绝;--force 也须 + 先一致性归档数据库与资料目录并核验成功,才允许回退。 队列盘点只输出**数量与待处理 ID**(persona_job/growth_job/mem_job/ chat_affect_job/sem_batch/sharing_attempt/reach_out/life_activity),不输出 @@ -42,6 +40,7 @@ import hashlib import json import shutil import sqlite3 +import tempfile import time from pathlib import Path @@ -206,6 +205,73 @@ def _refuse(msg: str, **extra) -> None: raise SystemExit(json.dumps({"ok": False, **extra, "error": msg}, ensure_ascii=False)) +def _data_path(data: Path, rel: str) -> Path: + path = Path(rel) + if not rel or path.is_absolute() or ".." in path.parts or path == Path("."): + _refuse("invalid_data_directory", refused="bad_spec") + target = data / path + try: + target.resolve().relative_to(data.resolve()) + except ValueError: + _refuse("data_directory_outside_instance", refused="bad_spec") + return target + + +def _data_snapshot(data: Path) -> dict: + """Logical database + file hashes; detects updates as well as new rows. + + Raw SQLite file hashes miss WAL changes and change on harmless checkpoints. + Only digests are recorded; no private row contents enter the report. + """ + conn = _ro(data / "companion.db") + try: + digest = hashlib.sha256() + for line in conn.iterdump(): + digest.update(line.encode("utf-8")) + digest.update(b"\n") + finally: + conn.close() + files = {} + for path in sorted(data.rglob("*")): + if path.is_symlink(): + _refuse("symlink_in_data: preserve manually before rollback", refused="unsafe_data") + rel = str(path.relative_to(data)) + if rel in {"companion.db", "companion.db-wal", "companion.db-shm", "continuation.json"}: + continue + if path.is_file(): + files[rel] = _sha256(path) + return {"database": digest.hexdigest(), "files": files} + + +def _archive_before_rollback(base: Path, expected: dict) -> Path: + """Archive first, verify both copies, and leave source intact on any error.""" + backups = base / "backups" + backups.mkdir(parents=True, exist_ok=True) + archive = Path(tempfile.mkdtemp(prefix=f"rollback-{int(time.time())}-", dir=backups)) + data = base / "data" + saved = archive / "data" + try: + saved.mkdir() + _consistency_copy(data / "companion.db", saved / "companion.db") + for path in data.iterdir(): + if path.name in {"companion.db", "companion.db-wal", "companion.db-shm"}: + continue + if path.is_dir(): + shutil.copytree(path, saved / path.name) + else: + shutil.copy2(path, saved / path.name) + if _data_snapshot(saved) != expected or _data_snapshot(data) != expected: + _refuse("rollback_archive_verification_failed", refused="archive_failed", + partial_archive=str(archive)) + (archive / "manifest.json").write_text(json.dumps( + {"complete": True, "snapshot": expected, "files": _dir_manifest(saved)}, + ensure_ascii=False, indent=2) + "\n") + except (OSError, sqlite3.Error) as exc: + _refuse(f"rollback_archive_failed:{type(exc).__name__}", refused="archive_failed", + partial_archive=str(archive)) + return archive + + def migrate(db: Path, base: Path, data_dirs: list[tuple[str, str]]) -> dict: pre = preflight(db) # 含空库/缺出生/缺诠释拒绝 src_sha = pre["db_sha256"] @@ -255,7 +321,7 @@ def migrate(db: Path, base: Path, data_dirs: list[tuple[str, str]]) -> dict: # 3) 向量/推导史等数据目录随实例迁移(可选;含逐文件哈希清单)。 copied_dirs = {} for src_dir, rel in data_dirs: - s, d = Path(src_dir), data / rel + s, d = Path(src_dir), _data_path(data, rel) if not s.is_dir(): _refuse(f"data_dir_missing:{s}", refused="data_dir_missing") if d.exists(): @@ -274,9 +340,10 @@ def migrate(db: Path, base: Path, data_dirs: list[tuple[str, str]]) -> dict: "migrated_at": now_iso, "queue_policy": "不删队列、不静默禁用;未批准能力 cap=0 保持关闭;" "旧排队工作执行时计入新账,不执行则保留并在队列盘点中如实报告", + "rollback_snapshot": _data_snapshot(data), } cont.write_text(json.dumps(record, ensure_ascii=False, indent=1) + "\n") - # 4) 自检:同一位她前后可核 + 旧账冻结 + 副本 live seal 已移除。 + # 4) 自检:同一位她前后可核,旧账和旧封账逐字保留。 conn = _ro(tdb) try: post = {"identity": _identity(conn), "provenance": _counts(conn, SAME_HER_TABLES), @@ -309,27 +376,40 @@ def _int_or_hash(sha: str) -> int: return int(sha[:15], 16) -def rollback(base: Path) -> dict: +def rollback(base: Path, *, force: bool = False) -> dict: data = base / "data" cont = data / "continuation.json" if not cont.exists(): _refuse("nothing_to_rollback: continuation.json 不存在", refused="no_continuation") + for live in (base / "window-active", base / "logs" / "api.pid"): + if live.exists(): + _refuse("refused_live_instance", refused="live_instance") rec = json.loads(cont.read_text()) tdb = data / "companion.db" existed = rec["target"].get("target_db_existed_before") backup = rec["target"].get("backup") + if existed and (not backup or not Path(backup).is_file()): + _refuse("pre_migration_backup_missing", refused="backup_missing") + # Validate every deletion target before touching the database or directories. + dirs = [_data_path(data, rel) for rel in rec["target"].get("data_dirs", {})] + current = _data_snapshot(data) + changed = current != rec.get("rollback_snapshot") + if changed and not force: + _refuse("post_migration_data_changed: use --force only after reviewing preservation", + refused="new_data") + archive = _archive_before_rollback(base, current) if force else None if existed and backup: - Path(backup).exists() or _refuse(f"backup_missing:{backup}", refused="backup_missing") _consistency_copy(Path(backup), tdb) note = "目标库已从迁移前一致性备份还原" else: - tdb.exists() and tdb.unlink() + for name in ("companion.db", "companion.db-wal", "companion.db-shm"): + (data / name).unlink(missing_ok=True) note = "迁移前目标库不存在:迁移副本已删除(源库从未被写过)" - for rel in rec["target"].get("data_dirs", {}): - d = data / rel + for d in dirs: d.exists() and shutil.rmtree(d) cont.unlink() - return {"ok": True, "rolled_back": note, "source_untouched": rec["source"]["db"]} + return {"ok": True, "rolled_back": note, "source_untouched": rec["source"]["db"], + "preserved_archive": str(archive) if archive else None} def main() -> int: @@ -344,6 +424,7 @@ def main() -> int: help="随实例迁移的数据目录(如向量库/推导史),SRC=目标 data 下相对路径") r = sub.add_parser("rollback") r.add_argument("--base", required=True, type=Path) + r.add_argument("--force", action="store_true", help="先一致性归档并核验后,再回退变化过的副本") args = ap.parse_args() if args.cmd == "preflight": out = preflight(args.db) @@ -352,12 +433,11 @@ def main() -> int: for pair in args.data_dir: src, sep, rel = pair.partition("=") (src and sep and rel) or _refuse(f"bad_data_dir_spec:{pair}", refused="bad_spec") - rel.startswith("/") or ".." not in Path(rel).parts or _refuse( - f"bad_data_dir_rel:{rel}", refused="bad_spec") + _data_path(args.base / "data", rel) dirs.append((src, rel)) out = migrate(args.db, args.base, dirs) else: - out = rollback(args.base) + out = rollback(args.base, force=args.force) print(json.dumps(out, ensure_ascii=False, indent=1)) return 0 diff --git a/deploy/hel919-continuation/test_ctl_scripts.sh b/deploy/hel919-continuation/test_ctl_scripts.sh index cc4cbfb..5952139 100644 --- a/deploy/hel919-continuation/test_ctl_scripts.sh +++ b/deploy/hel919-continuation/test_ctl_scripts.sh @@ -12,7 +12,12 @@ PKG=$(cd "$(dirname "$0")" && pwd) ROOT=$(cd "$PKG/../.." && pwd) TMP=$(mktemp -d /tmp/hel919-ctl-test.XXXXXX) PASS=0; FAIL=0 -cleanup() { rm -rf "$TMP"; } +cleanup() { + if [ -n "${BASE:-}" ] && [ -n "${CTL:-}" ]; then + env HEL919_BASE="$BASE" HEL919_DRAIN_SEC=3 bash "$CTL" close >/dev/null 2>&1 || true + fi + rm -rf "$TMP" +} trap cleanup EXIT ok() { PASS=$((PASS+1)); echo "PASS $1"; } @@ -36,7 +41,15 @@ mkdir -p "$BASE/etc" "$BASE/data" "$BASE/logs" "$CODE" cp -a "$ROOT/deploy" "$ROOT/tools" "$ROOT/src" "$CODE/" CTL="$PKG/instance_ctl.sh" STUB=$TMP/stub-python -printf '#!/usr/bin/env bash\nsleep 5\n' > "$STUB"; chmod +x "$STUB" +cat > "$STUB" <<'EOF' +#!/usr/bin/env bash +if [ -n "${HEL919_TEST_ENV_RECORD:-}" ]; then + printf '%s\n' "$CYBERGRIL_WINDOW_DEADLINE" > "$HEL919_TEST_ENV_RECORD" + cat "$HEL919_BASE/logs/expiry.ready" >> "$HEL919_TEST_ENV_RECORD" +fi +sleep 5 +EOF +chmod +x "$STUB" printf 'dummy cert\n' > "$TMP/c.pem"; printf 'dummy key\n' > "$TMP/c.key" ENV="HEL919_BASE=$BASE HEL919_VENV=$STUB HEL919_CERT=$TMP/c.pem HEL919_KEY=$TMP/c.key HEL919_PY=python3" @@ -85,8 +98,14 @@ rm -f "$BASE/window-active" "$BASE/window.json" # ---- S4. 失败启动收尾:健康不过 → 只清自己、留日志 ---- mkwin 1 99999999999 env $ENV HEL919_VENV=$STUB HEL919_CERT=$TMP/c.pem HEL919_KEY=$TMP/c.key \ + HEL919_TEST_ENV_RECORD="$TMP/api-environment" \ bash "$CTL" start >/dev/null 2>&1; expect_rc 1 $? "failed start (no health) exits 1" [ ! -f "$BASE/logs/api.pid" ] && ok "failed start removed pid file" || bad "failed start left pid file" +[ "$(head -1 "$TMP/api-environment" 2>/dev/null)" = "99999999999" ] \ + && ok "start injected persistent deadline into API" || bad "API deadline not injected" +grep -Eq '^[0-9]+ 99999999999$' "$TMP/api-environment" \ + && ok "watch ready before API was launched" || bad "API launched before watch readiness" +[ ! -f "$BASE/logs/expiry.pid" ] && ok "failed start disarmed its own watcher" || bad "failed start orphaned watcher" # ---- S5. stop 只操作身份核验过的自有句柄 ---- # 自有句柄:stub 以 marker 参数启动、cwd 在本实例代码目录 @@ -141,16 +160,16 @@ with c: c.execute("DELETE FROM budget_total WHERE key=?", (k,)) # 重置 S6 的新账封账,模拟窗口内运行 c.close() EOF -mkwin 1 $(( $(date +%s) - 5 )) # 窗口已到期(deadline 持久在 window.json) +EXPIRED=$(( $(date +%s) - 5 )) +mkwin 1 "$EXPIRED" # 窗口已到期(deadline 持久在 window.json) pushd "$CODE" >/dev/null "$STUB" serve_dual_8360_8361.py --http-port 1 >/dev/null 2>&1 & RUN=$! popd >/dev/null sleep 0.3 echo "$RUN" > "$BASE/logs/api.pid" -kill -0 "$RUN" 2>/dev/null && ok "running instance alive before deadline" || bad "running instance not alive" -NOW=$(date +%s) -env $ENV HEL919_DRAIN_SEC=5 bash "$CTL" watch $NOW >/dev/null 2>&1; expect_rc 0 $? "watch closed expired running instance" +kill -0 "$RUN" 2>/dev/null && ok "running instance alive before expiry close" || bad "running instance not alive" +env $ENV HEL919_DRAIN_SEC=5 bash "$CTL" watch "$EXPIRED" >/dev/null 2>&1; expect_rc 0 $? "watch closed expired running instance" kill -0 "$RUN" 2>/dev/null && bad "expired running instance still alive (new egress not stopped)" || ok "expired running instance stopped (no further egress possible)" [ ! -f "$BASE/logs/api.pid" ] && ok "watch removed pid file" || bad "watch left pid file" [ ! -f "$BASE/window-active" ] && ok "watch removed window-active" || bad "watch left window-active" @@ -166,6 +185,19 @@ EOF [ $? -eq 0 ] && ok "expiry sealed NEW scope only; old keys verbatim (no rename)" || bad "expiry seal state wrong" env $ENV bash "$CTL" start >/dev/null 2>&1; expect_rc 1 $? "restart after expiry refused (deadline persisted)" +# ---- S9. 不可核验的截止句柄:API 绝不先开,外来进程不受影响 ---- +mkwin 1 99999999999 +( sleep 300 ) & FOREIGN_WATCH=$! +echo "$FOREIGN_WATCH" > "$BASE/logs/expiry.pid" +printf '%s 99999999999\n' "$FOREIGN_WATCH" > "$BASE/logs/expiry.ready" +env $ENV HEL919_TEST_ENV_RECORD="$TMP/should-not-launch" bash "$CTL" start >/dev/null 2>&1 +expect_rc 1 $? "start refused unverifiable expiry arming" +[ ! -f "$TMP/should-not-launch" ] && ok "unarmed API never launched" || bad "unarmed API launched" +kill -0 "$FOREIGN_WATCH" 2>/dev/null && ok "foreign expiry pid untouched" || bad "foreign expiry pid killed" +kill "$FOREIGN_WATCH" 2>/dev/null +wait "$FOREIGN_WATCH" 2>/dev/null || true +rm -f "$BASE/logs/expiry.pid" "$BASE/logs/expiry.ready" + echo "----" echo "PASS=$PASS FAIL=$FAIL" [ "$FAIL" -eq 0 ] diff --git a/deploy/hel919-continuation/window_guard_hel919.py b/deploy/hel919-continuation/window_guard_hel919.py index cd724d0..b2a4baf 100644 --- a/deploy/hel919-continuation/window_guard_hel919.py +++ b/deploy/hel919-continuation/window_guard_hel919.py @@ -7,31 +7,43 @@ - 当前时间 ≥ deadline_epoch → 拒绝(过期窗口重启不放行:期限持久在 window.json,进程重启不重置;新用量由库内 PersistentBudget 持久计数, 重启同样不重置、不新增)。 -运行中到达截止由 expiry watch(instance_ctl.sh watch)有界收尾:停本实例 -进程后不存在可出站进程,满足「到期拒绝新出站」;本守卫保证此后无法重启。 +运行中由 PersistentBudget.check_window 在请求出口拒绝新增工作; +expiry watch 另负责有界停服,启动前必须确认它已真实布防。 用法: python3 window_guard_hel919.py """ from __future__ import annotations import json +import math import sys import time from pathlib import Path -base = Path(sys.argv[1]) if len(sys.argv) > 1 else None -if base is None: - raise SystemExit("usage: window_guard_hel919.py ") -if not (base / "window-active").is_file(): - raise SystemExit("hel919 window inactive: default closed") -wp = base / "window.json" -try: - w = json.loads(wp.read_text()) -except Exception as exc: - raise SystemExit(f"hel919 window unreadable: {exc}") from exc -if w.get("instance") != "hel919": - raise SystemExit("hel919 window: wrong instance") -if not w.get("approved"): - raise SystemExit("hel919 window not approved") -if time.time() >= int(w["deadline_epoch"]): - raise SystemExit(f"hel919 window expired at {w.get('deadline_at')}") + +def load_deadline(base: Path, *, allow_expired=False) -> int: + if not (base / "window-active").is_file(): + raise SystemExit("hel919 window inactive: default closed") + try: + w = json.loads((base / "window.json").read_text()) + raw = w["deadline_epoch"] + deadline = float(raw) + if isinstance(raw, bool) or not math.isfinite(deadline) or deadline <= 0 or not deadline.is_integer(): + raise ValueError("invalid deadline") + except (OSError, ValueError, TypeError, KeyError) as exc: + raise SystemExit(f"hel919 window unreadable: {exc}") from exc + if w.get("instance") != "hel919": + raise SystemExit("hel919 window: wrong instance") + if not w.get("approved"): + raise SystemExit("hel919 window not approved") + if not allow_expired and time.time() >= deadline: + raise SystemExit("hel919 window expired") + return int(deadline) + + +if __name__ == "__main__": + if len(sys.argv) < 2: + raise SystemExit("usage: window_guard_hel919.py [--deadline] [--allow-expired]") + deadline = load_deadline(Path(sys.argv[1]), allow_expired="--allow-expired" in sys.argv[2:]) + if "--deadline" in sys.argv[2:]: + print(deadline) diff --git a/docs/requirements/handoff-HEL-919.json b/docs/requirements/handoff-HEL-919.json index 2563858..2349d8b 100644 --- a/docs/requirements/handoff-HEL-919.json +++ b/docs/requirements/handoff-HEL-919.json @@ -1,7 +1,7 @@ { "baseline_version": "1.12", - "read_commit": "2e91a3c91e90944991886b21f7167a6913f41efb", - "task": "HEL-919 补齐同一位她继续体验的启停接续包(HEL-915 工程收口)", + "read_commit": "99c5de1d602970a34c5d7b937b187e16b57bda9d", + "task": "HEL-919 接续运行包;HEL-920复核后由总管直接补齐原未完成的截止与回退保护(HEL-915)", "requirement_ids": [ "CG-006", "CG-008", @@ -23,25 +23,24 @@ "GOV-006", "DEL-001" ], - "deliverable": "deploy/hel919-continuation/ 接续运行包(基线 2e91a3c 之后新增本目录 + tests/test_hel919_continuation.py + 本 handoff;产品代码仅极小 scope 扩展:PersistentBudget 可选 scope 参数(缺省 \"\"=旧键旧语义,旧实例零影响)、app 读 CYBERGRIL_BUDGET_SCOPE、seal_budget_hel879.py --scope,共 3 文件,其余零改动):1) migrate_same_her.py 同一位她迁移(preflight/migrate/rollback)——源库只读(mode=ro + SQLite backup API),拒绝空档案/缺出生/缺诠释,不因已有档案拒绝;身份指纹(companion+bazi_core+persona_interp)与逐表出处计数(bazi/persona/记忆/聊天/生活/关系)前后核对相等;向量/推导史目录 --data-dir 随实例复制含逐文件哈希;同一源幂等 no-op、不同源拒绝、迁移前目标库一致性备份、rollback 精确还原、自检不过自动回退;对活实例(window-active/api.pid)拒绝迁移。2) 旧账保留+新账隔离(总管复核返工后口径):源库逐字节不变;副本内旧 budget_total 键(含 sealed*/total_*)逐字保留——不改、不删、不改名,只新增 cont_scope/cont_source_sha256/cont_migrated_at 三个出处标记;接续实例以 CYBERGRIL_BUDGET_SCOPE=hel919 运行,新用量计数与到期封账全部落在独立 hel919:* budget_total 键(PersistentBudget scope 扩展),与旧键零交叉,旧口径读者在源库与副本上都依旧被旧封账拒绝;重启不重置(scope 计数持久在库);close 用 tools/seal_budget_hel879.py --scope hel919 只封新账;语义嵌入权威账不复制,semantic.template.yaml 按既有累计算式续唯一权威账(冻结余额/authority 前缀保留,42提取/111嵌入类历史余额不可借)。3) queue_policy.md 背景工作盘点(原文压缩/聊天情绪分类/分享判断/生活/成长/persona/mem_job/SDK提取嵌入/主动联系 → 实际表与预算类别)与接续政策:不删队列、不静默禁用,未批准 cap=0 显式关闭,执行计入新账,不执行保留说明,chat_affect_cutoff 历史切断点不重算;盘点只含 pending 数量与 ID。4) instance_ctl.sh 唯一入口 check/start/stop/close/status/watch + window_guard_hel919.py:默认关闭(无 window-active 或预算全 0 一律 fail-closed,check 拒绝未批非零预算);启动需 approved window.json,过期拒绝、到期 watch/timer 只 close 本实例;stop 只杀身份核验(cmdline+cwd 双验,PID 复用不动手)的本实例 pid,有界排空 HEL919_DRAIN_SEC(默认30s),绝不调用旧 force_end/worker/bridge/tts,不触碰 8300/8310/8320/8350、7880/8081、9880/8090;重复 start 拒绝;close 幂等并记 closed_at/close.log。5) enablement-plan.md 给总管的启用安排:文字/生活/情绪小范围,谁准备(施工员2号离线)/谁操作(总管唯一入口)/白栖知何时参与(体验阶段)与观察点(记得吗/生活连续/情绪连贯/边界承接/失败表现)、≤72h 窗口 + 原有共池上限直接复用(daily_tokens + 四类 daily_requests 合计口径,建议起步值列明、总管可按首日读数调整)、失败停止与资料保留(新聊天记忆保留、不自动覆盖原实例、rollback 只回退副本);无人民币金额、不承诺固定每轮成本、不把历史全类别合计当每轮口径。本轮取消费用前置依据:白栖知 HEL-915 评论 01a1a482-1338-719d-8acd-9df70355e95c 原话,属操作裁定、非需求变更。", + "deliverable": "保留99c5de1已审scope隔离和非空同一档案迁移成果。总管直接补齐原两项遗漏:1) PersistentBudget为非空scope绑定有限deadline并持久钉住,API从CYBERGRIL_WINDOW_DEADLINE注入;主聊/后台/SDK提取acquire及语义SDK单条/批量嵌入出口到点拒绝,缺失/非法期限拒绝,在途settle仍可完成,旧空scope保持原行为。start先启动并核自有watcher的PID/命令/工作目录和ready文件,再将同一deadline传给API,布防不可核验则不启动;失败启动清理自有watcher,不碰外来句柄。2) migrate记录数据库逻辑快照及资料文件SHA256;rollback比较实际内容(不只行数),新聊天、更新记忆、向量或用量变化默认拒绝;显式--force须先以SQLite backup一致性归档数据库(含WAL已提交内容)、资料目录及continuation记录,核归档及源快照一致并写完整manifest后才回退,备份失败不删;活实例及越界资料路径拒绝。README/manifest/启用安排与handoff同步,不自动提额。全部只用合成资料,零真实调用、未部署。", "acceptance": [ - "tests/test_hel919_continuation.py 12/12 通过(隔离合成旧档案+替身,零真实调用/零网络):空档案/缺出生/缺诠释拒绝;同一位她身份指纹与逐表计数前后相等;源库逐字节不变、副本 budget_total 与源逐键相等仅新增 cont_* 三键(旧封账 sealed=1 原样保留)、token/request 历史行原样;旧口径(scope=\"\")在源库与迁移副本上都仍被旧封账拒绝(旧账未被绕过);新 scope 计数独立(hel919:total_* 键)、settle 修正、进程重开不重置;新账封账只写 hel919:sealed* 且旧键 untouched;幂等 no-op/不同源拒绝/回退精确还原(含迁移前已存在目标库的备份还原与目标不存在时删副本);window_guard 默认关闭/过期/未批准/错实例/损坏 JSON 全拒绝、有效窗口放行;队列盘点 JSON 不含任何聊天/生活/诠释文本;CLI 通路 preflight/migrate/rollback rc 与 JSON 输出正确", - "bash deploy/hel919-continuation/test_ctl_scripts.sh 41/41 通过(stub 替身,零真实服务/端口/出站):包文件齐全与语法;未批非零预算 check 拒绝、显式 HEL919_ALLOW_BUDGET=1 放行;非法子命令 rc=2;无窗口/过期窗口 start 拒绝且不留 pid;失败启动只清自己留日志;stop 杀身份核验过的自有 pid、外来活 pid 拒绝动手且不受影响;close 只封本实例新 scope(hel919:sealed=1,旧 sealed 原样)、window.json 记 closed_at、幂等、封窗后 start 拒绝;S8 运行中到期直接证据:活的自有句柄进程在 watch 到点后被 close 停止(进程确认死亡=不再可能有新出站)、pid/window-active 移除、新 scope 封账、重启被拒", - "python3 -m ruff check deploy/hel919-continuation tests/test_hel919_continuation.py 通过(0 错误)", - "受影响面核对:产品代码改动仅 scope 扩展 3 文件(life/model.py、api/app.py、seal_budget_hel879.py,缺省语义与旧完全一致);tests/test_hel916.py 16/16、test_hard_gates_hel882 + test_rework_hel885/886/889/894 + test_store 共 70/70 复测通过;实跑记录 docs/rework/hel919/offline-run.txt", - "双基线需求检查通过:--base-ref origin/main 与 --base-ref 2e91a3c91e90944991886b21f7167a6913f41efb 各含本 handoff" + "本轮tests/test_hel919_continuation.py:25 passed,含原12项及新增截止/回退反例。活的TestClient API在过点后聊天与life/pattern背景调用零provider出站;SDK单条/批量嵌入和提取同样过点拒绝且不新增计数;缺失/非法期限拒绝;同scope重启改deadline拒绝,在途结算保留。", + "本轮启停替身:47 passed / 0 failed。除原测试外,核API启动前watch已ready、持久deadline实际注入、失败启动清理自有watch、不可核验布防拒绝且不触碰外来PID。watch停止进程仅作为停服证据,不能代替请求出口截止反例。", + "本轮回退反例:新聊天和原行记忆更新默认拒绝;强制回退先归档可恢复,实际恢复读回新增聊天及改变的向量;一致性归档故障不删除任何资料;活实例仍拒绝。", + "本轮受影响合集92 passed(包含接续专项25项,不重复相加):hard_gates_hel882、rework_hel885/886/889/894、semantic_recovery_hel882、hel916、store及接续专项;准确完整命令/结果存docs/rework/hel919/takeover-run.txt。未重跑全仓。", + "变更Python文件ruff通过;origin/main和实际产品基线2e91a3c双基线check_requirements带本handoff通过;结构检查不等同独立功能审核或用户验收。" ], "deferred": [ - "实际启用(部署、写获批预算、开窗口、真实体验)归总管按 enablement-plan.md 获批后执行;本卡只交离线可执行准备,cap=0、未部署、未监听端口、未动任何现有实例与用户资料", - "真实模型下的接续自然度(她是否'还是她')由白栖知实际体验判断;离线证据只证明接线与账目正确,不冒充体验通过", - "语音、天气/素材、完整备份恢复、摄像头、口型等未完功能承接不变(HEL-915 正文清单继续有效);本包不做用量优化研究", - "语义真实提取/嵌入(semantic cap>0)默认关闭;如需启用由总管按权威账现状填累计算式数值,min_used 保证冻结余额不可动" + "本轮总管自测通过后仅交已授权原两项补齐的必要独立验证,不冒充总工已通过;旧HEL-920修复1通过项和其他成果继续保留,不再全库审核。", + "真实模型自然度、.12实际落盘、真实启动/停服、systemd timer及浏览器/真机体验未执行;实际启用仍须具体安排获批,不能把离线cap=0测试当体验通过。", + "用户01a1a482-1338-719d-8acd-9df70355e95c已取消费用前置,不索价格套餐、不为金额阻塞;旧额度仍封账,不能从历史余额自动续用。", + "完整v1.12/60项、R1–R7以及语音/天气/素材/完整备份恢复/摄像头/口型等未完承接不变;本包强制回退归档不冒充完整产品导出恢复交付。" ], "requirement_changes": [], "verification": [ - "离线证据:docs/rework/hel919/offline-run.txt(pytest 12/12 + 启停替身 41/41 实跑输出,含 S8 运行中到期收尾直接证据);ruff 0 错误;双基线 check_requirements 通过", - "迁移自检内建:同一份 JSON 里给出 same_her/provenance_preserved/old_counts_preserved/totals_preserved/prior_seal_frozen/live_seal_removed/queues_preserved 七项布尔,任一 False 自动回退并拒绝", - "边界:全部测试用隔离合成档案('测试她/替身她'),未读、未复制、未写任何真实用户资料;未调用 LLM/ASR/TTS/嵌入;未建 window/房间;未恢复任何停机服务;未安装依赖", - "未测到:真实部署布局在 .12 上的实际落盘、systemd timer 实际触发、真实 provider 出站、浏览器端到端、Safari/手机、多日运行;均如实未验证,留待获批启用窗口由总管执行并记录" + "起点99c5de1;原HEL-920复核1通过,2与3未完成,总管从该真实成果直接接手,没有重跑失败run或更改队员配置。AGENTS/需求入口/current/coverage/changes与相关sources前后不改;只有用户取消费用前置的操作裁定继续有效。", + "实际命令/输出见docs/rework/hel919/takeover-run.txt;原offline-run.txt保留当时结果。首次启停检查因封账工具期待budget_sealed却先收到缺期限拒绝而3项失败,已恢复“封账优先于期限检查”的既有语义,复跑47/47;不抹去首错。", + "所有写入均隔离合成库和替身,未读复制真实用户资料,未LLM/ASR/TTS/嵌入真实调用,未建window/房间/服务timer,未安装模型依赖、未碰4080、未部署/合PR/换界面。" ] } diff --git a/docs/rework/hel919/takeover-run.txt b/docs/rework/hel919/takeover-run.txt new file mode 100644 index 0000000..54810b2 --- /dev/null +++ b/docs/rework/hel919/takeover-run.txt @@ -0,0 +1,76 @@ +HEL-919 总管接手补齐原遗漏;起点99c5de1d602970a34c5d7b937b187e16b57bda9d +全部离线:合成旧档案、替身provider/SDK/进程/时钟;未部署、未读用户库、零真实调用。 + +$ python3 -m pytest tests/test_hel919_continuation.py tests/test_hard_gates_hel882.py tests/test_rework_hel885.py tests/test_rework_hel886.py tests/test_rework_hel889.py tests/test_rework_hel894.py tests/test_semantic_recovery_hel882.py tests/test_hel916.py tests/test_store.py -q +........................................................................ [ 78%] +.................... [100%] +92 passed in 3.84s + +其中接续专项单跑25 passed;完整合集92 passed(含该25项),两者不是相加。 + +$ bash deploy/hel919-continuation/test_ctl_scripts.sh +PASS syntax instance_ctl.sh +PASS package file README.md +PASS package file manifest.json +PASS package file migrate_same_her.py +PASS package file window_guard_hel919.py +PASS package file models.template.yaml +PASS package file semantic.template.yaml +PASS package file queue_policy.md +PASS package file enablement-plan.md +PASS package file cybergril-hel919-expiry.template.service +PASS package file test_ctl_scripts.sh +PASS compile tools +PASS migrate refuses missing db +PASS synthetic migrate ok +PASS check ok (all budgets 0) +PASS check refuses unapproved nonzero budget +PASS check passes nonzero with explicit allow +PASS invalid subcommand refused (rc=2) +PASS invalid subcommand started nothing +PASS start refused without window (default closed) +PASS refused start left no pid +PASS start refused with expired window +PASS expired start left no pid +PASS failed start (no health) exits 1 +PASS failed start removed pid file +PASS start injected persistent deadline into API +PASS watch ready before API was launched +PASS failed start disarmed its own watcher +PASS stop own verified pid +PASS own pid stopped +PASS stop refuses non-own live pid +PASS foreign pid untouched +PASS close ok (no running api) +{"ok": true, "old_sealed": 1, "new_scope_sealed": 1} +PASS close sealed NEW scope only; old ledger verbatim +PASS window.json records closed_at +PASS close idempotent +PASS start refused after close (window-active removed) +PASS running instance alive before expiry close +PASS watch closed expired running instance +PASS expired running instance stopped (no further egress possible) +PASS watch removed pid file +PASS watch removed window-active +{"ok": true} +PASS expiry sealed NEW scope only; old keys verbatim (no rename) +PASS restart after expiry refused (deadline persisted) +PASS start refused unverifiable expiry arming +PASS unarmed API never launched +PASS foreign expiry pid untouched +---- +PASS=47 FAIL=0 + +$ python3 -m ruff check src/cybergril/life/model.py src/cybergril/api/app.py src/cybergril/memory/semantic.py deploy/hel919-continuation/migrate_same_her.py deploy/hel919-continuation/window_guard_hel919.py tests/test_hel919_continuation.py +All checks passed! + +$ python3 scripts/check_requirements.py --base-ref origin/main --handoff docs/requirements/handoff-HEL-919.json +需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。 +$ python3 scripts/check_requirements.py --base-ref 2e91a3c91e90944991886b21f7167a6913f41efb --handoff docs/requirements/handoff-HEL-919.json +需求结构检查通过:60项;来源和覆盖完整。仍须人工核对授权语义与实际体验。 + +先前首错:第一次启停替身44 PASS/3 FAIL,close/幂等close/watch的封账自检收到缺期限而非budget_sealed;已恢复封账优先于期限检查的原语义,再跑47 PASS/0 FAIL。未修改全局测试shim或降低断言。原offline-run.txt保留旧交付结果。 + +必要反例:TestClient进程仍活、watch未运行,时钟到deadline后新主聊/生活/聊天分类请求被拒,provider.calls及请求数不增加;SDK单条/批量嵌入/提取同样拒绝且不新增计数。缺失/非法deadline拒绝,scope期限持久钉住,重启改期拒绝;在途settle可完成。 +回退:新增聊天、更新原行记忆、向量改变默认拒绝;force先一致性归档(含已提交WAL),实际恢复读回新聊天/向量;备份故障不删,活实例拒绝。资料路径限定在实例data内。 +未验证:真实模型自然度、.12实际部署与启用、真实timer、浏览器真机/多日体验;均须另行获批实际启用后核证。 diff --git a/src/cybergril/api/app.py b/src/cybergril/api/app.py index bd5061d..74d9bc5 100644 --- a/src/cybergril/api/app.py +++ b/src/cybergril/api/app.py @@ -236,7 +236,8 @@ def create_app( # authorization under a separate budget_total scope; "" keeps the # exact legacy keys for every existing instance. router.guard = PersistentBudget( - store, router.guard, scope=os.getenv("CYBERGRIL_BUDGET_SCOPE", "")) + store, router.guard, scope=os.getenv("CYBERGRIL_BUDGET_SCOPE", ""), + deadline=os.getenv("CYBERGRIL_WINDOW_DEADLINE")) app.state.sharing = Sharing(store, router, models) app.state.persona = Persona(store, router, models, enabled=life_config.enabled) app.state.public = PublicView(store) diff --git a/src/cybergril/life/model.py b/src/cybergril/life/model.py index 7570173..68d6243 100644 --- a/src/cybergril/life/model.py +++ b/src/cybergril/life/model.py @@ -4,6 +4,7 @@ from __future__ import annotations import asyncio import json import logging +import math import time from contextlib import aclosing @@ -30,10 +31,30 @@ class PersistentBudget(BudgetGuard): more generous). """ - def __init__(self, store, previous, scope: str = ""): + def __init__(self, store, previous, scope: str = "", *, deadline=None, clock=None): super().__init__(previous.config) self.store = store + self.scope = scope + self.clock = clock or time.time self._k = (lambda key: f"{scope}:{key}") if scope else (lambda key: key) + # A continuation cannot send without a finite, persistent deadline. + # Legacy/unscoped instances retain their original behavior. Constructing + # a guard without a deadline remains possible for readout/sealing only. + try: + self.deadline = float(deadline) if not isinstance(deadline, bool) else None + except (TypeError, ValueError, OverflowError): + self.deadline = None + if self.deadline is not None and (not math.isfinite(self.deadline) or self.deadline <= 0): + self.deadline = None + if scope and self.deadline is not None: + with self.store.conn: + self.store.execute("BEGIN IMMEDIATE") + self.store.execute("INSERT OR IGNORE INTO budget_total VALUES(?,?)", + (self._k("deadline_epoch"), self.deadline)) + bound = self.store.query_one("SELECT value FROM budget_total WHERE key=?", + (self._k("deadline_epoch"),))["value"] + if bound != self.deadline: + raise BudgetStop("window_deadline_changed") # Persisted rows already include previous usage when reusing an injected router. if not isinstance(previous, PersistentBudget): for scene, used in previous.state.by_scene.items(): @@ -122,6 +143,7 @@ class PersistentBudget(BudgetGuard): self._roll() with self.store.conn: self.store.execute("BEGIN IMMEDIATE") + self.check_window() if self.used + tokens >= self.limit * self.config.hard_stop_ratio: raise BudgetStop("budget_stop") self.store.execute( @@ -150,6 +172,7 @@ class PersistentBudget(BudgetGuard): raise BudgetStop( f"budget_sealed:{sealed['requests']}{suffix}(封账于 " f"{sealed['sealed_at']},真实用量保留,不伪造、不清零)") + self.check_window() row = self.store.query_one( "SELECT coalesce(sum(used),0) n FROM token_usage WHERE day=?", (day,) ) @@ -192,6 +215,23 @@ class PersistentBudget(BudgetGuard): return BudgetHold(scene=scene, category=category, reserved_tokens=tokens, requests=requests, day=day) + def check_window(self): + """Refuse new work at the wire boundary; in-flight settlement is allowed. + + The deadline is pinned in this scope, so a restart cannot extend it. + No counters change on refusal, even while the process/watch is alive. + """ + if not self.scope: + return + if self.deadline is None: + raise BudgetStop("window_deadline_missing_or_invalid") + row = self.store.query_one("SELECT value FROM budget_total WHERE key=?", + (self._k("deadline_epoch"),)) + if not row or row["value"] != self.deadline: + raise BudgetStop("window_deadline_changed") + if self.clock() >= self.deadline: + raise BudgetStop("window_expired") + def settle(self, hold, *, actual_tokens=None): """Real usage adjusts the reserved tokens; unknown usage keeps the reserve. diff --git a/src/cybergril/memory/semantic.py b/src/cybergril/memory/semantic.py index f9484cb..7fc8ba6 100644 --- a/src/cybergril/memory/semantic.py +++ b/src/cybergril/memory/semantic.py @@ -758,6 +758,8 @@ class Mem0Backend: failures.append(reason) def counted_embed(text, memory_action=None): + if request_guard is not None and hasattr(request_guard, "check_window"): + request_guard.check_window() try: hold = ledger.reserve(1, str(memory_action or "embed")) except RuntimeError as exc: @@ -778,6 +780,8 @@ class Mem0Backend: return result def counted_batch(texts, memory_action=None): + if request_guard is not None and hasattr(request_guard, "check_window"): + request_guard.check_window() n = len(texts) if isinstance(texts, (list, tuple)) else 1 try: hold = ledger.reserve(n, f"batch:{memory_action or 'embed'}") diff --git a/tests/test_hel919_continuation.py b/tests/test_hel919_continuation.py index fb468b0..24fcfde 100644 --- a/tests/test_hel919_continuation.py +++ b/tests/test_hel919_continuation.py @@ -246,7 +246,8 @@ def test_new_usage_isolated_persistent_and_old_seal_still_enforced(synth): legacy.close() # 接续实例(scope="hel919"):新授权范围生效,新计数落在独立 hel919:* 键 store = Store(tdb, init=False) - guard = PersistentBudget(store, BudgetGuard(cfg), scope="hel919") + deadline = int(time.time()) + 3600 + guard = PersistentBudget(store, BudgetGuard(cfg), scope="hel919", deadline=deadline) assert guard.total_requests_used() == 0 # 新 scope 从零起,不借旧计数也不清旧键 hold = guard.acquire("dialogue_main", tokens=500) guard.settle(hold, actual_tokens=400) @@ -257,7 +258,7 @@ def test_new_usage_isolated_persistent_and_old_seal_still_enforced(synth): store.close() # 进程"重启":重开库,新 scope 计数不重置;旧口径读者依旧被旧封账拒绝 store2 = Store(tdb, init=False) - guard2 = PersistentBudget(store2, BudgetGuard(cfg), scope="hel919") + guard2 = PersistentBudget(store2, BudgetGuard(cfg), scope="hel919", deadline=deadline) assert guard2.total_requests_used() == 1 and guard2.total_tokens_used() == 400 hold = guard2.acquire("life_event", tokens=100) guard2.settle(hold, actual_tokens=100) @@ -267,7 +268,7 @@ def test_new_usage_isolated_persistent_and_old_seal_still_enforced(synth): store2.close() # 新账封账(close 路径的库内效果):新 scope 拒绝一切;旧键依旧原样 store3 = Store(tdb, init=False) - guard3 = PersistentBudget(store3, BudgetGuard(cfg), scope="hel919") + guard3 = PersistentBudget(store3, BudgetGuard(cfg), scope="hel919", deadline=deadline) marker = guard3.seal() with pytest.raises(BudgetStop): guard3.acquire("dialogue_main", tokens=1) @@ -278,6 +279,169 @@ def test_new_usage_isolated_persistent_and_old_seal_still_enforced(synth): assert marker["requests"] == 2 +def test_rollback_refuses_new_chat_and_memory_by_default(synth): + src, base = synth + _migrate(src, base) + target = base / "data" / "companion.db" + store = Store(target, init=False) + with store.conn: + store.execute("INSERT INTO chat_message(ts,role,text,turn_id,status) " + "VALUES(9999,'user','new synthetic conversation','new','sent')") + # Same row count: updating a memory is also new experience to preserve. + store.execute("UPDATE mem_episodic SET gist='updated synthetic memory'") + store.close() + with pytest.raises(SystemExit, match="post_migration_data_changed"): + MIG.rollback(base) + conn = sqlite3.connect(target) + assert conn.execute("SELECT count(*) FROM chat_message").fetchone()[0] == 4 + assert conn.execute("SELECT gist FROM mem_episodic").fetchone()[0] == "updated synthetic memory" + conn.close() + + +def test_force_rollback_archives_database_wal_and_vectors_and_is_restorable(synth, tmp_path): + src, base = synth + vec, _ = build_semantic_dirs(tmp_path / "sem") + _migrate(src, base, dirs=[(vec, "sem/vec")]) + data = base / "data" + store = Store(data / "companion.db", init=False) + # Keep this connection open: the new chat may only be present in the WAL. + with store.conn: + store.execute("INSERT INTO chat_message(ts,role,text,turn_id,status) " + "VALUES(9999,'user','WAL synthetic conversation','new','sent')") + (data / "sem/vec/v.bin").write_bytes(b"new-synthetic-vector") + with pytest.raises(SystemExit, match="post_migration_data_changed"): + MIG.rollback(base) + result = MIG.rollback(base, force=True) + store.close() + archive = Path(result["preserved_archive"]) + assert json.loads((archive / "manifest.json").read_text())["complete"] + assert not (data / "companion.db").exists() + restored = tmp_path / "restored" + import shutil + shutil.copytree(archive / "data", restored) + conn = sqlite3.connect(restored / "companion.db") + assert conn.execute("SELECT text FROM chat_message WHERE turn_id='new'").fetchone()[0] == "WAL synthetic conversation" + conn.close() + assert (restored / "sem/vec/v.bin").read_bytes() == b"new-synthetic-vector" + assert json.loads((restored / "continuation.json").read_text())["scope"] == "hel919" + + +def test_force_rollback_archive_failure_leaves_all_data(synth, monkeypatch): + src, base = synth + _migrate(src, base) + target = base / "data/companion.db" + before = target.read_bytes() + def broken_backup(*args): + raise OSError("synthetic archive disk failure") + monkeypatch.setattr(MIG, "_consistency_copy", broken_backup) + with pytest.raises(SystemExit, match="rollback_archive_failed"): + MIG.rollback(base, force=True) + assert target.read_bytes() == before + assert (base / "data/continuation.json").exists() + + +def test_rollback_refuses_running_instance(synth): + src, base = synth + _migrate(src, base) + (base / "window-active").touch() + with pytest.raises(SystemExit, match="refused_live_instance"): + MIG.rollback(base, force=True) + + +@pytest.mark.parametrize("deadline", [None, "missing", float("nan"), float("inf"), -1, True]) +def test_scoped_deadline_missing_or_invalid_refuses_without_counters(tmp_path, deadline): + from conftest import engineering_budget + + store = Store(tmp_path / "invalid-window.db") + guard = PersistentBudget(store, BudgetGuard(engineering_budget()), scope="hel919", deadline=deadline) + with pytest.raises(BudgetStop, match="window_deadline_missing_or_invalid"): + guard.acquire("dialogue_main", tokens=1) + assert guard.total_requests_used() == 0 and guard.used == 0 + store.close() + + +def test_deadline_is_pinned_and_allows_inflight_settlement(synth): + from conftest import engineering_budget + + src, base = synth + _migrate(src, base) + store = Store(base / "data/companion.db", init=False) + now = [100] + guard = PersistentBudget(store, BudgetGuard(engineering_budget()), scope="hel919", + deadline=200, clock=lambda: now[0]) + hold = guard.acquire("dialogue_main", tokens=500) + now[0] = 200 + with pytest.raises(BudgetStop, match="window_expired"): + guard.acquire("pattern_detect", tokens=200) + guard.settle(hold, actual_tokens=400) # already in flight may drain after deadline + assert guard.total_requests_used() == 1 and guard.total_tokens_used() == 400 + with pytest.raises(BudgetStop, match="window_deadline_changed"): + PersistentBudget(store, BudgetGuard(engineering_budget()), scope="hel919", deadline=300) + restarted = PersistentBudget(store, BudgetGuard(engineering_budget()), scope="hel919", + deadline=200, clock=lambda: now[0]) + with pytest.raises(BudgetStop, match="window_expired"): + restarted.acquire("life_event", tokens=100) + assert restarted.total_requests_used() == 1 + store.close() + + +def test_live_api_and_background_refuse_at_deadline_before_provider(tmp_path, monkeypatch): + from fastapi.testclient import TestClient + from test_api import CREATE, events, turn + from test_hard_gates_hel882 import gated_app + + from cybergril.life.model import structured + from cybergril.think.scenes import Scene + + deadline = int(time.time()) + 300 + monkeypatch.setenv("CYBERGRIL_BUDGET_SCOPE", "hel919") + monkeypatch.setenv("CYBERGRIL_WINDOW_DEADLINE", str(deadline)) + app, provider = gated_app(tmp_path) + with TestClient(app) as client: + assert client.post("/api/companion", json=CREATE).status_code == 201 + guard = app.state.life.router.guard + assert events(turn(client, "synthetic before deadline"))[-1]["t"] == "chat.final" + calls, count = len(provider.calls), guard.total_requests_used() + # The process remains alive. No watcher runs in this in-process test. + guard.clock = lambda: deadline + assert client.get("/api/companion").status_code == 200 + assert events(turn(client, "synthetic after deadline"))[-1]["t"] == "chat.error" + for scene in (Scene.LIFE_EVENT, Scene.PATTERN_DETECT): + with pytest.raises(BudgetStop, match="window_expired"): + client.portal.call(structured, app.state.life.router, app.state.life.models, + scene, "synthetic offline check", {}) + assert len(provider.calls) == calls + assert guard.total_requests_used() == count + + +def test_sdk_embedding_and_extraction_refuse_after_deadline(tmp_path, monkeypatch): + from conftest import engineering_budget + from test_semantic_recovery_hel882 import _make_ledger, config_for, install_fake + + from cybergril.memory.semantic import Mem0Backend + + install_fake(monkeypatch) + ledger = tmp_path / "semantic-ledger.db" + _make_ledger(ledger) + store = Store(tmp_path / "semantic-product.db") + now = [100] + guard = PersistentBudget(store, BudgetGuard(engineering_budget()), scope="hel919", + deadline=200, clock=lambda: now[0]) + backend = Mem0Backend(config_for(tmp_path, ledger), request_guard=guard) + assert backend._memory.embedding_model.embed("synthetic before deadline") == [0.0] + used = backend.ledger.used() + now[0] = 200 + with pytest.raises(BudgetStop, match="window_expired"): + backend._memory.embedding_model.embed("synthetic after deadline") + with pytest.raises(BudgetStop, match="window_expired"): + backend._memory.embedding_model.embed_batch(["synthetic after deadline"]) + with pytest.raises(BudgetStop, match="window_expired"): + backend._memory.llm.generate_response(messages=[{"role": "user", "content": "synthetic"}]) + assert backend.ledger.used() == used + assert guard.total_requests_used() == 0 + store.close() + + # ---------------------------------------------------------------- window guard def _guard(base: Path):