HEL-858: rework per manager independent review - fixture (turn,role) integrity gate, spoken-text share bridge with partial/missing negatives + full restored P1-P5/F1-F8, known-id S4/S6 orchestration with readback and post-update re-retrieval, S7 anchor chain (input fingerprint) with wrong/unknown-source rejection, card-level persistent embed ledger (history rebuilt <=100), _flat/exit-code fixes; zero real model calls, run1/run2 evidence and 26/60 ledger untouched
Co-authored-by: multica-agent <github@multica.ai>
This commit is contained in:
zonggong
co-authored by
multica-agent
multica-agent
parent
e31ae2d946
commit
892d0d2e30
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"baseline_version": "1.11",
|
||||
"read_commit": "e31ae2d946829bb1e02be76f5f10c66d61e13e35",
|
||||
"task": "HEL-858 补正Mem0对照测试与分享隐私证据（HEL-857 返工）",
|
||||
"requirement_ids": [
|
||||
"CG-008",
|
||||
"CG-009",
|
||||
"CG-011",
|
||||
"CG-013",
|
||||
"CG-014",
|
||||
"CG-015",
|
||||
"CG-016",
|
||||
"CG-021",
|
||||
"AM-001",
|
||||
"UX-004",
|
||||
"UX-006",
|
||||
"GOV-001",
|
||||
"GOV-002",
|
||||
"GOV-003",
|
||||
"GOV-004",
|
||||
"GOV-005",
|
||||
"GOV-006",
|
||||
"DEL-001"
|
||||
],
|
||||
"deliverable": "总管独立复核五项必修的零模型纠错（原hel-857-mem0-spike分支）：A/B夹具按(turn,role)完整装载并加逐字逐角色核对门（旧缺陷形态负对照可被抓住）；分享桥绑定实际已讲文本spoken_text、部分分享/缺失内容负例、恢复后重跑同套P1-P5/F1-F8；S4/S6已知id编排（metadata回读+prev→new史+新条目不被误改+更新后重新检索装配）并复放run2毒例负对照；S7命中→history推导史→payload来源锚点(含输入指纹)→原文链与未知/无锚点/错来源/未分享伴侣负例及600字装配门；卡级持久嵌入账本与历史用量保守重建（≤100）、_flat修复、S2等价集语义判定、退出码依赖实际结果。新增真实模型请求0次，run1/run2证据、26次账本与余34次额度原样保留。",
|
||||
"acceptance": [
|
||||
"rework_suite 全部逐项通过（results/rework858_offline.json：R1夹具/重判、R2门禁2全套27键、R3R4编排与链、R5资源账，退出码0）",
|
||||
"旧缺陷形态可复现且被新核对门判失败；S1现状侧逐字重验通过",
|
||||
"S2现状侧：修复夹具后带门回读恢复'受潮的味道'等价细节（run2假阴性纠正，不记作产品缺点）",
|
||||
"F7部分分享：公开写入逐字等于已讲半句，未讲秘密在全量payload与检索均为0；F8缺失已讲文本：拒绝迁入+审计留痕，不整段放行",
|
||||
"恢复后P1-P5/F1-F8全套复跑通过（14项restored键，含恢复前telling_log跨恢复幂等）",
|
||||
"S4/S6已知id编排全过（含新条目零UPDATE、结果条目无promise_status回读验证）；负对照复放证明旧文本选条法选中错误对象",
|
||||
"S7链恢复t1用户原句；未知来源/无锚点/输入指纹不符/同轮未分享伴侣全部拒绝；600字装配零私密标记",
|
||||
"卡级嵌入账本：历史≤100+本轮65=165≤500（余335）；词面另列；真实出站0",
|
||||
"check_requirements 对 origin/main 与 origin/agent/agent/de8874a3803b 双基准通过"
|
||||
],
|
||||
"deferred": [
|
||||
"真实A/B复跑（S1 mem0侧、S2/S3/S4/S6模型使用问答、S7真实锚点、S8/S9复验）：固定复测方案已交（rework858_offline.json _retest_plan，预计13次≤余34），未经总管核对与主卡批准不执行；零模型补测不冒充真实A/B完成",
|
||||
"'严格优于且其他无退化'总验收结论已撤回，待真实复跑按新判据重下",
|
||||
"正式采用Mem0或迁移产品：待用户拍板，本卡结果不构成采用决定",
|
||||
"中文提取提示词定制、八字情绪/连续生活/主动联系接线、真实语料验收、M5/M6产品接线：继续按R1-R7承接，不以实验代替完整产品验收"
|
||||
],
|
||||
"requirement_changes": [],
|
||||
"verification": [
|
||||
"spike/hel857/results/rework858_offline.json（离线全套逐项结果+_retest_plan+资源账）",
|
||||
"spike/hel857/README.md《HEL-858 返工修订记录》（旧/新逐项表、结论修订、归责表修订）",
|
||||
"原results/*.json与账本未改动（git仅新增文件与实验源码修订）；原工作区ledger只读对账26次不变",
|
||||
"本轮零网络出站（stub LLM+本地fastembed缓存+假传输）；未验证现网/真实档案（本卡范围禁止触及）"
|
||||
]
|
||||
}
|
||||
+45
-7
@@ -2,6 +2,8 @@
|
||||
|
||||
本目录是任务卡 HEL-857 的全部实验代码、夹具与结果。**仅实验分支，不合 main、不合产品 PR、不部署、不触现网 8300–8350 与真实档案。** 全部测试资料为固定中文合成语料（`fixtures.py`），无真实用户/伴侣数据。
|
||||
|
||||
> **HEL-858 返工修订（总管独立复核未通过后纠错）**：下文"结果总览"为 e31ae2d 原始记录，其中带 ⚠️ 的结论已被《HEL-858 返工修订记录》一节修订或撤回；run1/run2 原始证据、账本与 26 次额度原样保留。本轮返工**新增真实模型请求 0 次**（余 34 次冻结），全部验证为零网络离线复现（`results/rework858_offline.json`）。
|
||||
|
||||
- 需求基准：v1.11（60 项），读取提交 `bb9cbe915ecdc5d7dd3393ad45a6c4152e76900a`（`origin/hel-847-v111-governance`），本分支 `hel-857-mem0-spike` 自该提交建立。
|
||||
- Mem0 固定源提交：`2c81af3bbe27d9bccb09c772d583df9e9be72ee5`（mem0ai 2.2.1，源码 tarball SHA256 `9dcef9303ebcec116eec6739f01b8b35bd24459de6a569686b4d19b9412dda23`，从源码构建安装，wheel SHA256 `698284b6083ab8b7761454d65876a6d1bb983d278ad21a7bee38d760cb20e9db`，不以 PyPI 同版本包代替）。
|
||||
- 关联要求编号：CG-008/009/011/013–016/021、AM-001、UX-004/006、GOV-001–006、DEL-001。本卡是 SDK 隔离验证，不代替任何完整产品要求验收。
|
||||
@@ -47,13 +49,13 @@ python -m spike.hel857.ab_scenarios # A/B 真实合成场景（门禁全
|
||||
| G2 F4 成功按已讲内容迁入+锚点 | 通过（A 保留内部副本） | 同上 |
|
||||
| G2 F5 重复回执不重复迁入 | 通过 | 同上 |
|
||||
| G2 F6 其他未分享仍隐藏 | 通过 | 同上 |
|
||||
| AB S1 原话留存 | 通过（现状 chat_message；Mem0 history.db messages） | `results/ab_scenarios.json` |
|
||||
| AB S2 跨会话细节 | 通过（命中/进上下文/模型使用分列） | 同上 |
|
||||
| AB S3 换说法召回（核心） | **通过：Mem0 严格优于现状**（现状 0 命中 vs Mem0 命中；中文问答答出“蓝色”） | 同上 |
|
||||
| AB S4 改口更正 | 通过（两批时 infer ADD-only 新旧并存；我方编排 update 留 prev→new；run1 证明同批时提取自动合并） | 同上 |
|
||||
| AB S1 原话留存 | ⚠️ 通过（现状侧夹具被覆盖：`baseline_raw_in_chat_message=false` 而判据只查"行存在"；返工后改严格判据并修夹具） | `results/ab_scenarios.json` |
|
||||
| AB S2 跨会话细节 | ⚠️ 通过（现状侧细节回读为 false 系装载错误所致；返工后重判） | 同上 |
|
||||
| AB S3 换说法召回（核心） | ⚠️ 通过：Mem0 严格优于现状（现状侧数字基于损坏夹具；"严格优于"总验收结论撤回，待真实复跑） | 同上 |
|
||||
| AB S4 改口更正 | ⚠️ 通过（update 实际落在**新**条目上、问答用更新前列表；返工后改已知id编排） | 同上 |
|
||||
| AB S5 事实/感受分离 | 通过（事实/感受查询各自召回） | 同上 |
|
||||
| AB S6 承诺状态 | 通过（显式 update 置 resolved，history 留痕） | 同上 |
|
||||
| AB S7 来源回读 | 通过（现状带门回读恢复该行原文；Mem0 原文在 history.db） | 同上 |
|
||||
| AB S6 承诺状态 | ⚠️ 通过（resolved 置到了**面试结果**条目、模型答"不知道"；返工后改已知id编排） | 同上 |
|
||||
| AB S7 来源回读 | ⚠️ 通过（未建立命中→原文链；返工后加锚点链与负例） | 同上 |
|
||||
| AB S8 进程重启 | 通过（真实子进程重启） | 同上 |
|
||||
| AB S9 完整备份恢复 | 通过（恢复后换说法召回仍命中） | 同上 |
|
||||
|
||||
@@ -65,7 +67,7 @@ python -m spike.hel857.ab_scenarios # A/B 真实合成场景（门禁全
|
||||
- 明细：预检 1 次；run1（含 1 次提取解析失败后的重试，重试已计数）14 次；run2 11 次。逐次时间/状态/耗时/输入规模/usage 见 `results/ab_scenarios.json` 的 `_ledger_rows`。
|
||||
- usage 合计：prompt 114,183 / completion 2,854 / total 117,037 tokens（单次 completion ≤367 ≤1024 上限；单次输入中文 ≤201 字符 ≤6000 上限）。价格未知，不承诺金额或免费。
|
||||
- 原HEL-835的6/8额度未动用、未重置。
|
||||
- 本地嵌入：密集 28 条 + 实体 0 条（spaCy 未安装，上游优雅降级为空，如实记录）= 28 ≤ 500。词面处理另列：10 次。
|
||||
- 本地嵌入：⚠️ ~~密集 28 条~~ 28 仅为 run2 进程内计数（未含 run1、门禁与子进程）；返工后按卡级保守重建，见《HEL-858 返工修订记录》。词面处理另列：10 次（同为 run2 进程内计数）。
|
||||
- 门禁 0/1/2 与资源核对全部零真实模型、零网络出站（stub/假传输/本地推理）。
|
||||
|
||||
## 关键发现（如实记录，不因结果好而省略）
|
||||
@@ -101,3 +103,39 @@ python -m spike.hel857.ab_scenarios # A/B 真实合成场景（门禁全
|
||||
- 本卡结论仅覆盖：本地 SDK 检查、隐私零泄漏（合成数据）、恢复完整性、换说法召回优于现状。**合成实验不冒充真实伴侣“记住昨日细节”已通过**；检索命中、细节进入上下文、模型实际使用已分开判定（S2/S3）。
|
||||
- 未做（继续承接）：正式采用或产品接入（待用户拍板）；中文提取提示词定制与复测；八字情绪/连续生活/主动联系接线卡；真实语料验收（另行申请额度）；M5/M6 产品接线（v2 §8）。
|
||||
- 失败处置规则未触发：无连续真实失败 3 次（含 run1 的 1 次解析重试后即成功）、无隐私负例失败、未达任何上限。
|
||||
|
||||
## HEL-858 返工修订记录（本轮，零真实模型请求）
|
||||
|
||||
对应总管《HEL-857-独立复核与返工要求.md》五项必修；全部离线验证见 `results/rework858_offline.json`（`python -m spike.hel857.rework_suite`，stub LLM + 本地嵌入 + 假传输，零网络，退出码依赖逐项实际结果）。
|
||||
|
||||
### 逐项：旧判定 → 返工后状态
|
||||
|
||||
| 必修项 | 旧（e31ae2d） | 返工后（离线复现） | 真实复跑 |
|
||||
| --- | --- | --- | --- |
|
||||
| 1a 夹具装载（伴侣文本覆盖用户原话、整轮写 user） | t1–t4 用户原话丢失，`baseline_raw_in_chat_message=false` | **已修**：`(turn, role)` 完整装载，`fixture_integrity` 逐字逐角色核对 15/15 通过；构造旧缺陷形态的损坏库被核对门判失败（负对照） | 不适用（纯夹具） |
|
||||
| 1b S1 判据只查"行存在" | false 仍 pass | **已修**：S1 要求夹具核对通过 + 现状原句逐字 + mem0 原文三方同时成立；现状侧已离线重验通过 | mem0 侧待真实复跑 |
|
||||
| 1c S2/S3 及无退化重判 | 现状侧回读细节 false（装载错误所致）；"严格优于+无退化"成立 | **已重判**：修复夹具后现状**带门回读可恢复**"受潮的味道"等价细节（S2 现状侧重验）；S3 换说法现状词面 0 命中经修复夹具重验仍成立；**"严格优于且其他无退化"总验收结论撤回**，待真实 A/B 复跑后按新判据重下 | 待真实复跑（模型使用部分） |
|
||||
| 2 分享桥整段事件迁公开 | ack+judged 后写入整个 `life_event.content`（含未讲出的秘密，独立复核已复现） | **已修**：`attempt` 绑定实际收到的回复文本 `spoken_text`；迁入严格限于该文本（F7 部分分享负例：公开写入逐字等于已讲半句，秘密标记在全量 payload 与检索中均为 0）；缺失/为空 → 拒绝迁入并留 `migration_refusal` 审计（F8），不降级整段放行；成功/失败/取消/未确认/重复回执全覆盖；**恢复后重跑同套 P1–P5 + F1–F8**（14 项 restored 键全过，含恢复前 telling_log 跨恢复幂等） | 不适用（stub 门禁本为零模型） |
|
||||
| 3 改口/承诺更新错对象 | S4 update 落在新条目；S6 resolved 置到面试结果条目；问答用更新前列表 | **已修**：add 返回捕获已知 id（内容特征复核防错位）；S4 显式 update **旧**条目置 `superseded`（metadata 回读 + 该 id 的 prev→new 史 + 新条目零 UPDATE 验证）；S6 update **原承诺**置 `resolved`（结果条目无 promise_status 验证）；**更新后重新检索装配**再供问答；负对照复放 run2 毒例（"含周三"选条法与"result/tell"选条法均选中错误对象） | 编排已离线证明；模型问答待真实复跑 |
|
||||
| 4 来源回读无链 | history 有原句+检索有命中即过（换错来源也过） | **已修**：add 写入来源锚点（`src_group`/`turn_ids`/`input_sha256` 输入指纹）；链 = 命中 id → history 推导史吻合 → payload 锚点（指纹校验）→ 输入清单恢复原文；负例：未知来源、无锚点、**输入指纹不符（换错来源）**、同轮未分享伴侣消息（told=0）全部拒绝；600 字装配仅纳入链验证条目、零私密标记 | 待真实复跑（锚点随真实 add 写入） |
|
||||
| 5 资源/成功统计口径 | "全卡 28/500"（仅 run2 进程内）；`_flat` 把正常 dict 当 ERROR；S2 等义表达按词面判 false；退出码只看 completed | **已修**：卡级持久嵌入账本（跨进程/子进程/batch 统一扣账，词面另列）；历史用量按证据保守重建 **≤100**（记录值/派生值/向量点数/枚举上界分列）；`normalize_add_result` 修复 dict 返回；S2 细节按预先声明的等价集语义判定；各命令退出码依赖逐项实际验收结果 | 账本持续生效 |
|
||||
|
||||
### 卡级资源账（修订后）
|
||||
|
||||
- 历史（HEL-857 全卡，保守重建，精确不可恢复处标上界与方法）：run1 24（记录 23 + S8 子进程派生 1）+ run2 29（记录 28 + 派生 1）+ 门禁1 6（向量点数 4 + 检索 2）+ 门禁2 ≤41（向量点数 9 + update 1 + 查询路径枚举上界 31）= **≤100 ≤ 500**；词面另列（记录 20 + 上界 20）。
|
||||
- 本轮返工（HEL-858）：离线套件持久扣账 **65 条**（含门禁2全套与编排复放），卡级累计 **165/500，余 335**；真实出站 **0**，原账本 26 次与余 34 次冻结额度未动。
|
||||
- 旧口径"全卡 28/500"作废（28 仅为 run2 进程内计数）。
|
||||
|
||||
### 结论修订
|
||||
|
||||
- **撤回**"换说法召回 Mem0 严格优于现状；其他既有成功项未退化"这一总验收结论（基于损坏夹具与错误判据）；修复判据后的固定复测方案见 `results/rework858_offline.json` 的 `_retest_plan`（固定语料、锚点、判据、预计 13 次真实请求 ≤ 余 34），**未经总管核对与主卡批准不执行**。
|
||||
- 保留的可靠事实：门禁 0/1/2 机制性结论（含本轮补强后的隐私/分享时序/恢复全套）；run2 的 mem0 侧检索证据（其输入不受夹具缺陷影响）作为线索保留，不作为验收。
|
||||
- 上游 Mem0 未被证明泄漏：整段迁移问题出在我方实验适配桥，非上游 add 原语，也非现网。
|
||||
|
||||
### 归责表修订
|
||||
|
||||
| 职责 | 上游（mem0ai 2.2.1） | 我方 |
|
||||
| --- | --- | --- |
|
||||
| 冲突检测→UPDATE/DELETE | 原语 update/delete + prev→new 史 | **全部**：检测、**已知id选条**、编排、更新后重检索（S4/S6 返工后实现；e31ae2d 版选条错误已修正并加负对照） |
|
||||
| 隐私前置隔离 | scope/collection 承载 | 双实例设计、内部→公开桥接（**绑定实际已讲文本**）、逐消息门（F1–F8/P1–P5 返工后全套含恢复复跑） |
|
||||
| 原文存储与回读 | history.db 存输入副本（messages 表**无 memory_id 关联**，如实列明） | **锚点链**：add 写入 `src_group/turn_ids/input_sha256`，适配层直读 payload（search 不回传 metadata，SDK 适配责任） |
|
||||
|
||||
@@ -40,7 +40,7 @@ def main(argv: list[str]) -> int:
|
||||
from . import ab_scenarios
|
||||
r3 = ab_scenarios.main([])
|
||||
print(json_summary("AB_scenarios", r3))
|
||||
return 0 if r3.get("completed") else 1
|
||||
return 0 if r3.get("outcome", {}).get("all_scenarios_pass") else 1
|
||||
|
||||
|
||||
def json_summary(name: str, payload: dict) -> str:
|
||||
|
||||
+430
-132
@@ -28,9 +28,9 @@ from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from .common import (COUNTERS, DATA_DIR, GATEWAY_BASE_URL, GATEWAY_MODEL,
|
||||
PRIV, PUB, close_memory, install_counters, mem_config,
|
||||
write_result)
|
||||
from .common import (COUNTERS, DATA_DIR, EMBED_LEDGER, GATEWAY_BASE_URL,
|
||||
GATEWAY_MODEL, PRIV, PUB, close_memory, install_counters,
|
||||
mem_config, write_result)
|
||||
from .fixtures import (ANSWER_KEY_PARAPHRASE, CORRECTION_PAIR,
|
||||
FACT_FEELING_PAIR, PROMISE_PAIR, PUBLIC_CORPUS,
|
||||
Q_CORRECTION, Q_EXACT, Q_FACT, Q_FEELING,
|
||||
@@ -40,19 +40,38 @@ from . import fixtures as FX
|
||||
from .ledger import AccountingTransport, Ledger
|
||||
|
||||
AB = DATA_DIR / "ab"
|
||||
# HEL-858 返工：gist 按 (turn, role) 区分——此前按 turn 单键，伴侣回复覆盖
|
||||
# 同轮用户原话且整轮写成 user，属装载错误（总管复核必修项1）。
|
||||
BASELINE_GISTS = {
|
||||
"t1": "用户买到一本诗集，很喜欢",
|
||||
"t2": "用户给诗集取名海雾，放床头第二层",
|
||||
"t3": "用户加班，买到坏的栗子",
|
||||
"t4": "用户读完诗集，喜欢第三首",
|
||||
"t5": "用户又买了栗子",
|
||||
"c1": "用户下周三体检",
|
||||
"c2": "用户体检改到周五",
|
||||
"f1": "用户周五有重要面试",
|
||||
"f2": "用户面试前紧张失眠",
|
||||
"p1": "用户答应周五晚告知面试结果",
|
||||
"p2": "用户面试通过，第一时间告知",
|
||||
("t1", "user"): "用户买到一本诗集，很喜欢",
|
||||
("t1", "companion"): "她回应了买诗集的事",
|
||||
("t2", "user"): "用户给诗集取名海雾，放床头第二层",
|
||||
("t2", "companion"): "她记住了名字和位置",
|
||||
("t3", "user"): "用户加班，买到坏的栗子",
|
||||
("t3", "companion"): "她记住了坏栗子的事",
|
||||
("t4", "user"): "用户读完诗集，喜欢第三首",
|
||||
("t4", "companion"): "她愿意陪读",
|
||||
("t5", "user"): "用户又买了栗子",
|
||||
("c1", "user"): "用户下周三体检",
|
||||
("c2", "user"): "用户体检改到周五",
|
||||
("f1", "user"): "用户周五有重要面试",
|
||||
("f2", "user"): "用户面试前紧张失眠",
|
||||
("p1", "user"): "用户答应周五晚告知面试结果",
|
||||
("p2", "user"): "用户面试通过，第一时间告知",
|
||||
}
|
||||
BASELINE_HOURS = {
|
||||
"t1": 24, "t2": 23, "t3": 12, "t4": 6, "t5": 2,
|
||||
"c1": 5, "c2": 4, "f1": 5, "f2": 4, "p1": 8, "p2": 1,
|
||||
}
|
||||
|
||||
|
||||
def baseline_fixture_rows() -> dict[tuple[str, str], str]:
|
||||
"""A/B 现状侧装载清单：(turn, role) → 原文。两侧同一资料、逐字、逐角色。"""
|
||||
texts: dict[tuple[str, str], str] = {}
|
||||
for group in (FX.PUBLIC_CORPUS, CORRECTION_PAIR, FACT_FEELING_PAIR, PROMISE_PAIR):
|
||||
for t, r, raw in group:
|
||||
texts[(t, r)] = raw
|
||||
return texts
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -64,32 +83,56 @@ def build_baseline_db(path: Path) -> None:
|
||||
path.unlink()
|
||||
conn = sqlite3.connect(path)
|
||||
conn.executescript(PR.SCHEMA)
|
||||
rows = [(24, "t1"), (23, "t2"), (12, "t3"), (6, "t4"), (2, "t5"),
|
||||
(5, "c1"), (4, "c2"), (5, "f1"), (4, "f2"), (8, "p1"), (1, "p2")]
|
||||
texts = {t: r for t, _, r in FX.PUBLIC_CORPUS}
|
||||
texts.update({t: r for t, _, r in CORRECTION_PAIR})
|
||||
texts.update({t: r for t, _, r in FACT_FEELING_PAIR})
|
||||
texts.update({t: r for t, _, r in PROMISE_PAIR})
|
||||
ids = {}
|
||||
for hours, turn in rows:
|
||||
role = "user"
|
||||
texts = baseline_fixture_rows()
|
||||
for turn, hours in BASELINE_HOURS.items():
|
||||
ts = int((PR.NOW - hours * PR.HOUR) * 1000)
|
||||
cur = conn.execute(
|
||||
"INSERT INTO chat_message(ts,role,text,turn_id,status) VALUES(?,?,?,?,'sent')",
|
||||
(ts, role, texts[turn], turn))
|
||||
mid = cur.lastrowid
|
||||
ids[(turn, role)] = mid
|
||||
conn.execute(
|
||||
"INSERT INTO mem_episodic(ts,role,raw,gist,salience,source,turn_id,message_id) "
|
||||
"VALUES(?,?,?,?,0.5,'chat',?,?)",
|
||||
(ts, role, texts[turn], BASELINE_GISTS.get(turn), turn, mid))
|
||||
mem_id = conn.execute("SELECT last_insert_rowid()").fetchone()[0]
|
||||
conn.execute("INSERT INTO mem_job(source,ref_id,memory_id,state,attempts,model) "
|
||||
"VALUES('chat_message',?,?,'done',1,'fixture')", (mid, mem_id))
|
||||
for role in ("user", "companion"):
|
||||
raw = texts.get((turn, role))
|
||||
if raw is None:
|
||||
continue
|
||||
cur = conn.execute(
|
||||
"INSERT INTO chat_message(ts,role,text,turn_id,status) VALUES(?,?,?,?,'sent')",
|
||||
(ts, role, raw, turn))
|
||||
mid = cur.lastrowid
|
||||
conn.execute(
|
||||
"INSERT INTO mem_episodic(ts,role,raw,gist,salience,source,turn_id,message_id) "
|
||||
"VALUES(?,?,?,?,0.5,'chat',?,?)",
|
||||
(ts, role, raw, BASELINE_GISTS.get((turn, role)), turn, mid))
|
||||
mem_id = conn.execute("SELECT last_insert_rowid()").fetchone()[0]
|
||||
conn.execute("INSERT INTO mem_job(source,ref_id,memory_id,state,attempts,model) "
|
||||
"VALUES('chat_message',?,?,'done',1,'fixture')", (mid, mem_id))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
def fixture_integrity(db: Path) -> dict:
|
||||
"""夹具装载核对：每个 (turn, role) 原句逐字在内、角色正确、无互相覆盖。
|
||||
|
||||
S1 前置：任一原话缺失或角色错乱即整体失败（不再以"行存在"放行）。
|
||||
"""
|
||||
expect = baseline_fixture_rows()
|
||||
with sqlite3.connect(str(db)) as c:
|
||||
actual = {(r[0], r[1]): r[2] for r in
|
||||
c.execute("SELECT turn_id, role, text FROM chat_message").fetchall()}
|
||||
roled = {(r[0], r[1]) for r in
|
||||
c.execute("SELECT turn_id, role FROM mem_episodic").fetchall()}
|
||||
missing = sorted(k for k in expect if k not in actual)
|
||||
mismatched = sorted(k for k in expect if k in actual and actual[k] != expect[k])
|
||||
extra = sorted(k for k in actual if k not in expect)
|
||||
# 角色错乱检测：某键的实际文本等于另一角色的原话（旧缺陷形态）
|
||||
crossed = sorted(k for k in expect if k in actual and actual[k] != expect[k]
|
||||
and actual[k] in expect.values())
|
||||
episodic_missing = sorted(k for k in expect if k not in roled)
|
||||
ok = not missing and not mismatched and not extra and not crossed and not episodic_missing
|
||||
return {
|
||||
"expected_rows": len(expect), "actual_rows": len(actual),
|
||||
"missing": missing, "mismatched": mismatched, "extra_roles_rows": extra,
|
||||
"role_crossed": crossed, "episodic_missing": episodic_missing,
|
||||
"verbatim_all_roles": ok, "pass": ok,
|
||||
"expect": "两侧同一资料按 (turn, role) 完整装载：原句逐字、角色正确、无覆盖",
|
||||
}
|
||||
|
||||
|
||||
def baseline_engine(db: Path):
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "docs" / "reuse-hel847"))
|
||||
import probe_recall as PR
|
||||
@@ -121,6 +164,225 @@ def mem0_search_texts(m, query: str, limit: int = 5) -> list[dict]:
|
||||
return hits
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HEL-858 返工：归一化、payload 直读与 S4/S6/S7 编排判定（真实运行与离线 stub 复用）
|
||||
def normalize_add_result(v) -> list[dict]:
|
||||
"""add() 返回归一化。真实 infer=True 返回 dict(results=[...])（run2 曾被误记为
|
||||
ERROR——旧 _flat 把正常 dict 返回整段当错误串）；infer=False 返回 list[dict]。"""
|
||||
items = (v.get("results") or []) if isinstance(v, dict) else list(v or [])
|
||||
return [{"id": r.get("id"), "memory": r.get("memory"), "event": r.get("event")}
|
||||
for r in items if isinstance(r, dict)]
|
||||
|
||||
|
||||
def payload_of(m, memory_id: str) -> dict | None:
|
||||
"""直读向量库 payload（search/get_all 不回传自定义 metadata——SDK 适配责任，如实列明）。"""
|
||||
pts = m.vector_store.client.retrieve(
|
||||
collection_name=m.collection_name, ids=[memory_id], with_payload=True)
|
||||
return dict(pts[0].payload) if pts else None
|
||||
|
||||
|
||||
def history_rows_for(hist_db_path: Path, memory_id: str) -> list[dict]:
|
||||
with sqlite3.connect(str(hist_db_path)) as c:
|
||||
rows = c.execute(
|
||||
"SELECT old_memory, new_memory, event, memory_id FROM history "
|
||||
"WHERE memory_id=? ORDER BY created_at", (memory_id,)).fetchall()
|
||||
return [{"old": r[0], "new": r[1], "event": r[2], "memory_id": r[3]} for r in rows]
|
||||
|
||||
|
||||
def s4_correction_orchestration(m, hist_db_path: Path, old_id: str, new_id: str) -> dict:
|
||||
"""改口编排（已知 id 区分旧安排/新安排）：
|
||||
显式 update 旧条目为已改期 → 回读旧条目 metadata 与 prev→new 史 → 验证新条目
|
||||
未被误改（run2 实证旧"含周三"文本选条法实际更新了新条目）→ 更新后重新检索装配。
|
||||
模型问答由调用方在 post_update_block 上执行。
|
||||
"""
|
||||
out = {"old_id": old_id, "new_id": new_id}
|
||||
old_p, new_p = payload_of(m, old_id), payload_of(m, new_id)
|
||||
old_text0 = (old_p or {}).get("data")
|
||||
new_text0 = (new_p or {}).get("data")
|
||||
out.update({"old_text": old_text0, "new_text": new_text0,
|
||||
"ids_distinct": bool(old_id and new_id and old_id != new_id)})
|
||||
upd = None
|
||||
annotated = (old_text0 or "") + "（已改期：体检改到周五上午，见新安排）"
|
||||
try:
|
||||
m.update(old_id, text=annotated,
|
||||
metadata={"status": "superseded", "superseded_by": str(new_id),
|
||||
"corrected_at": "synthetic-c2"})
|
||||
upd = {"updated": True}
|
||||
except Exception as e:
|
||||
upd = {"updated": False, "error": str(e)}
|
||||
out["our_update"] = upd
|
||||
old_after, new_after = payload_of(m, old_id), payload_of(m, new_id)
|
||||
out["old_metadata_readback"] = {k: (old_after or {}).get(k)
|
||||
for k in ("status", "superseded_by", "corrected_at")}
|
||||
out["new_untouched"] = ((new_after or {}).get("data") == new_text0
|
||||
and not (new_after or {}).get("superseded_by"))
|
||||
hist_old = history_rows_for(hist_db_path, old_id)
|
||||
hist_new = history_rows_for(hist_db_path, new_id)
|
||||
out["old_prev_to_new"] = [{"old": r["old"], "new": r["new"], "event": r["event"]}
|
||||
for r in hist_old if r["event"] == "UPDATE"]
|
||||
out["new_update_rows"] = sum(1 for r in hist_new if r["event"] == "UPDATE")
|
||||
prev_new_ok = any(r["event"] == "UPDATE" and r["old"] == old_text0 and r["new"] == annotated
|
||||
for r in hist_old)
|
||||
hits = mem0_search_texts(m, Q_CORRECTION)
|
||||
fresh = [h["memory"] for h in hits]
|
||||
out["post_update_retrieved_head"] = fresh[:4]
|
||||
out["post_update_has_annotated_old"] = any("已改期" in (t or "") for t in fresh)
|
||||
out["post_update_block"] = assemble_block(fresh)
|
||||
out["expect"] = ("已知id锁定旧安排/新安排：update 旧条目置 superseded 且留 prev→new；"
|
||||
"新条目文本与史不被误改；更新后重新检索可见改期标注")
|
||||
out["pass"] = bool(
|
||||
out["ids_distinct"] and upd.get("updated")
|
||||
and out["old_metadata_readback"].get("status") == "superseded"
|
||||
and str(out["old_metadata_readback"].get("superseded_by")) == str(new_id)
|
||||
and out["new_untouched"] and out["new_update_rows"] == 0
|
||||
and prev_new_ok and out["post_update_has_annotated_old"])
|
||||
return out
|
||||
|
||||
|
||||
def s6_promise_orchestration(m, hist_db_path: Path, promise_id: str, result_id: str) -> dict:
|
||||
"""承诺状态编排（已知 id 区分原承诺/结果）：
|
||||
显式 update 原承诺条目置 resolved → 回读承诺条目 metadata → 验证结果条目未被
|
||||
误置 resolved（run2 实证"result/tell"文本选条法选中了面试结果条目）→ 更新后
|
||||
重新检索装配。模型问答由调用方在 post_update_block 上执行。
|
||||
"""
|
||||
out = {"promise_id": promise_id, "result_id": result_id}
|
||||
p_p, r_p = payload_of(m, promise_id), payload_of(m, result_id)
|
||||
p_text0 = (p_p or {}).get("data")
|
||||
out.update({"promise_text": p_text0, "result_text": (r_p or {}).get("data"),
|
||||
"ids_distinct": bool(promise_id and result_id and promise_id != result_id)})
|
||||
upd = None
|
||||
try:
|
||||
m.update(promise_id, text=p_text0,
|
||||
metadata={"promise_status": "resolved", "resolved_at": "synthetic-t2"})
|
||||
upd = {"updated": True}
|
||||
except Exception as e:
|
||||
upd = {"updated": False, "error": str(e)}
|
||||
out["our_update"] = upd
|
||||
p_after, r_after = payload_of(m, promise_id), payload_of(m, result_id)
|
||||
out["promise_metadata_readback"] = {k: (p_after or {}).get(k)
|
||||
for k in ("promise_status", "resolved_at")}
|
||||
out["promise_text_unchanged"] = (p_after or {}).get("data") == p_text0
|
||||
out["result_not_marked"] = not (r_after or {}).get("promise_status")
|
||||
hist_p = history_rows_for(hist_db_path, promise_id)
|
||||
hist_r = history_rows_for(hist_db_path, result_id)
|
||||
out["promise_update_rows"] = sum(1 for r in hist_p if r["event"] == "UPDATE")
|
||||
out["result_update_rows"] = sum(1 for r in hist_r if r["event"] == "UPDATE")
|
||||
hits = mem0_search_texts(m, Q_PROMISE)
|
||||
fresh = [h["memory"] for h in hits]
|
||||
out["post_update_retrieved_head"] = fresh[:4]
|
||||
out["post_update_has_promise"] = any((t or "") == p_text0 for t in fresh)
|
||||
out["post_update_block"] = assemble_block(fresh)
|
||||
out["expect"] = ("已知id锁定原承诺（非面试结果条目）：update 置 resolved 留痕，"
|
||||
"结果条目不被误置；更新后重新检索仍含原承诺")
|
||||
out["pass"] = bool(
|
||||
out["ids_distinct"] and upd.get("updated")
|
||||
and out["promise_metadata_readback"].get("promise_status") == "resolved"
|
||||
and out["promise_text_unchanged"] and out["result_not_marked"]
|
||||
and out["promise_update_rows"] >= 1 and out["result_update_rows"] == 0
|
||||
and out["post_update_has_promise"])
|
||||
return out
|
||||
|
||||
|
||||
# S7 来源锚点：add 时写入 payload 的来源组、轮次清单与输入指纹（适配层责任）
|
||||
ADD_MANIFEST: dict[str, list[str]] = {
|
||||
"base": ["t1", "t2", "t3", "t4", "t5"],
|
||||
"correction_first": ["c1"],
|
||||
"correction_later": ["c2"],
|
||||
"fact_feeling": ["f1", "f2"],
|
||||
"promise": ["p1", "p2"],
|
||||
}
|
||||
|
||||
|
||||
def group_input_sha(group: str) -> str:
|
||||
"""来源组输入指纹：该组全部 (turn, role, raw) 的规范化 SHA256。
|
||||
|
||||
锚点与输入集密码学绑定：换错来源/篡改锚点（组标签与输入不符）可被检出。
|
||||
"""
|
||||
import hashlib
|
||||
manifest = baseline_fixture_rows()
|
||||
rows = sorted((t, r, manifest[(t, r)]) for t in ADD_MANIFEST.get(group, [])
|
||||
for r in ("user", "companion") if (t, r) in manifest)
|
||||
canonical = json.dumps(rows, ensure_ascii=False, sort_keys=True)
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class CompanionShareGate:
|
||||
"""同轮伴侣消息分享门（重放 life/public.py 语义）：无控制库时公开语料伴侣行
|
||||
视为已讲（对话语料本身即用户所见）；离线负例用控制库判 told=0 拒绝。"""
|
||||
|
||||
def __init__(self, ctrl_db: Path | None = None):
|
||||
self.ctrl_db = ctrl_db
|
||||
|
||||
def companion_allowed(self, turn: str) -> bool:
|
||||
if self.ctrl_db is None:
|
||||
return True
|
||||
with sqlite3.connect(str(self.ctrl_db)) as c:
|
||||
row = c.execute(
|
||||
"SELECT told FROM companion_share WHERE turn_id=?", (turn,)).fetchone()
|
||||
return bool(row and row[0] == 1)
|
||||
|
||||
|
||||
def s7_chain_resolve(m, hit: dict, share_gate: CompanionShareGate | None = None) -> dict:
|
||||
"""从实际命中建立到原文的链：命中 id → 向量 payload 来源锚点(src_group/turn_ids/
|
||||
input_sha256) → 输入清单恢复对应原句；companion 行过分享门。
|
||||
|
||||
锚点缺失/未知来源/输入指纹不符（换错来源）被门拒绝。
|
||||
"""
|
||||
payload = payload_of(m, hit["id"])
|
||||
if not payload:
|
||||
return {"resolved": False, "reject_reason": "no_payload", "hit_id": hit.get("id")}
|
||||
group = payload.get("src_group")
|
||||
turns = payload.get("turn_ids") or []
|
||||
if not group or not turns or group not in ADD_MANIFEST:
|
||||
return {"resolved": False, "reject_reason": "unknown_source", "src_group": group,
|
||||
"turn_ids": turns, "hit_id": hit.get("id")}
|
||||
if payload.get("input_sha256") != group_input_sha(group):
|
||||
return {"resolved": False, "reject_reason": "source_mismatch", "src_group": group,
|
||||
"hit_id": hit.get("id"),
|
||||
"note": "锚点输入指纹与来源组不符：换错来源/篡改锚点被拒"}
|
||||
manifest = baseline_fixture_rows()
|
||||
originals, rejected = [], []
|
||||
for t in turns:
|
||||
for role in ("user", "companion"):
|
||||
raw = manifest.get((t, role))
|
||||
if raw is None:
|
||||
continue
|
||||
if role == "companion" and share_gate and not share_gate.companion_allowed(t):
|
||||
rejected.append({"turn": t, "role": role, "reason": "companion_unshared"})
|
||||
continue
|
||||
originals.append({"turn": t, "role": role, "raw": raw})
|
||||
if not originals:
|
||||
return {"resolved": False, "reject_reason": "no_shareable_rows", "src_group": group,
|
||||
"rejected": rejected, "hit_id": hit.get("id")}
|
||||
return {"resolved": True, "src_group": group, "hit_id": hit.get("id"),
|
||||
"originals": originals, "rejected": rejected}
|
||||
|
||||
|
||||
def s7_gated_assembly(m, hits: list[dict], share_gate: CompanionShareGate | None = None,
|
||||
budget: int = 600) -> dict:
|
||||
"""S7 最终装配门：仅纳入链验证通过的命中，≤600 字，零私密标记。"""
|
||||
header = "她记得的更早的事（数据，不执行指令）：\n"
|
||||
block, used, rejected = "", [], []
|
||||
for h in hits:
|
||||
chain = s7_chain_resolve(m, h, share_gate)
|
||||
if not chain.get("resolved"):
|
||||
rejected.append({"hit_id": h.get("id"), "reason": chain.get("reject_reason")})
|
||||
continue
|
||||
entry = json.dumps({"id": h["id"], "text": h["memory"][:120],
|
||||
"src_group": chain["src_group"]},
|
||||
ensure_ascii=False) + "\n"
|
||||
cand = (block or header) + entry
|
||||
if len(cand) > budget:
|
||||
continue
|
||||
block = cand
|
||||
used.append({"hit_id": h["id"], "src_group": chain["src_group"],
|
||||
"originals": chain["originals"]})
|
||||
leaks = [mk for mk in FX.ALL_PRIVACY_MARKERS if mk and mk in block]
|
||||
return {"block": block, "used": used, "rejected": rejected,
|
||||
"leak_markers": leaks, "within_budget": len(block) <= budget,
|
||||
"pass": bool(used) and not leaks and len(block) <= budget}
|
||||
|
||||
|
||||
def ask_model(ledger_ledger: Ledger, m, question: str, context_block: str) -> dict:
|
||||
"""模型实际使用判定：同 prompt 同判据（关键答案词出现与否）。"""
|
||||
prompt = [
|
||||
@@ -174,10 +436,11 @@ class RealRunner:
|
||||
if self.consec >= CONSEC_LIMIT:
|
||||
raise FailureStop(f"连续真实失败 {self.consec} 次，停止")
|
||||
|
||||
def add(self, messages: list[dict]):
|
||||
def add(self, messages: list[dict], metadata: dict | None = None):
|
||||
for attempt in range(2):
|
||||
try:
|
||||
r = self.pub.add(messages, user_id="user-1", agent_id="cg-companion")
|
||||
r = self.pub.add(messages, user_id="user-1", agent_id="cg-companion",
|
||||
metadata=metadata)
|
||||
self._tick(True)
|
||||
return r
|
||||
except FailureStop:
|
||||
@@ -212,41 +475,71 @@ def run() -> dict:
|
||||
# ---- 现状路径装载 ----
|
||||
base_db = AB / "baseline.sqlite3"
|
||||
build_baseline_db(base_db)
|
||||
integ = fixture_integrity(base_db)
|
||||
results["_fixture_integrity"] = integ
|
||||
base, base_conn = baseline_engine(base_db)
|
||||
|
||||
# ---- Mem0 写入（真实提取）----
|
||||
# ---- Mem0 写入（真实提取；锚点随 add 写入 payload，供 S7 链回读）----
|
||||
# run1 证据（账本保留）：c1+c2 同批时提取自动合并为一条已更正记忆；
|
||||
# run2 起改两批（先周三、后改周五），还原"稍后改口"的真实时序。
|
||||
rr = RealRunner(AB / "mem0", ledger)
|
||||
adds = {}
|
||||
adds["base_corpus"] = rr.add(public_messages("base"))
|
||||
adds["correction_first"] = rr.add(public_messages("correction")[:1])
|
||||
adds["correction_later"] = rr.add(public_messages("correction")[1:])
|
||||
adds["fact_feeling"] = rr.add(public_messages("fact_feeling"))
|
||||
adds["promise"] = rr.add(public_messages("promise"))
|
||||
def _flat(v):
|
||||
if isinstance(v, dict):
|
||||
return [{"event": "ERROR", "memory": json.dumps(v, ensure_ascii=False)[:200]}]
|
||||
return [{"event": r.get("event"), "memory": r.get("memory")}
|
||||
for r in (v or []) if isinstance(r, dict)]
|
||||
raw_adds: dict[str, list[dict]] = {}
|
||||
|
||||
results["_mem0_adds"] = {k: _flat(v) for k, v in adds.items()}
|
||||
def _anchored_add(key: str, messages: list[dict]):
|
||||
r = rr.add(messages, metadata={"src_group": key, "turn_ids": ADD_MANIFEST[key],
|
||||
"input_sha256": group_input_sha(key)})
|
||||
raw_adds[key] = normalize_add_result(r)
|
||||
return r
|
||||
|
||||
_anchored_add("base", public_messages("base"))
|
||||
_anchored_add("correction_first", public_messages("correction")[:1])
|
||||
_anchored_add("correction_later", public_messages("correction")[1:])
|
||||
_anchored_add("fact_feeling", public_messages("fact_feeling"))
|
||||
_anchored_add("promise", public_messages("promise"))
|
||||
results["_mem0_adds"] = {k: v for k, v in raw_adds.items()}
|
||||
used_after_adds = ledger.used()
|
||||
|
||||
# ---- S1 原话留存（程序判定，零出站）----
|
||||
def _single_id(key: str) -> str:
|
||||
items = raw_adds.get(key) or []
|
||||
return items[0]["id"] if items and items[0].get("id") else ""
|
||||
|
||||
# 已知id（add 返回顺序对应输入），内容特征复核防错位
|
||||
corr_old_id = _single_id("correction_first")
|
||||
corr_new_id = _single_id("correction_later")
|
||||
promise_items = raw_adds.get("promise") or []
|
||||
promise_id, result_id = "", ""
|
||||
if len(promise_items) >= 2:
|
||||
tell = [r for r in promise_items
|
||||
if has_any(r["memory"] or "", ["告诉"], ["tell", "inform"])
|
||||
and not has_any(r["memory"] or "", ["通过"], ["passed"])]
|
||||
passed = [r for r in promise_items if has_any(r["memory"] or "", ["通过"], ["passed"])]
|
||||
if len(tell) == 1 and len(passed) >= 1:
|
||||
promise_id, result_id = tell[0]["id"], passed[0]["id"]
|
||||
results["_known_ids"] = {
|
||||
"correction_old": corr_old_id, "correction_new": corr_new_id,
|
||||
"promise": promise_id, "promise_result": result_id,
|
||||
"ids_sane": bool(corr_old_id and corr_new_id and corr_old_id != corr_new_id
|
||||
and promise_id and result_id and promise_id != result_id),
|
||||
}
|
||||
|
||||
# ---- S1 原话留存（程序判定，零出站；任一原话缺失即失败）----
|
||||
raw_t1 = FX.PUBLIC_CORPUS[0][2]
|
||||
with sqlite3.connect(str(AB / "mem0" / "mem0_public_history.db")) as c:
|
||||
hist_msgs = [r[0] for r in c.execute("SELECT content FROM messages").fetchall()]
|
||||
mem0_raw_kept = any(raw_t1 in m for m in hist_msgs)
|
||||
base_row = base_conn.execute("SELECT raw FROM mem_episodic WHERE turn_id='t1'").fetchone()
|
||||
base_row = base_conn.execute(
|
||||
"SELECT raw FROM mem_episodic WHERE turn_id='t1' AND role='user'").fetchone()
|
||||
baseline_raw_kept = bool(base_row and base_row["raw"] == raw_t1)
|
||||
results["S1_verbatim_retention"] = {
|
||||
"baseline_raw_in_chat_message": bool(base_row and raw_t1 in base_row["raw"]),
|
||||
"fixture_integrity_pass": integ["pass"],
|
||||
"baseline_raw_in_chat_message": baseline_raw_kept,
|
||||
"mem0_raw_in_history_db": mem0_raw_kept,
|
||||
"expect": "两侧原文都留存：现状在 chat_message/mem_episodic.raw；Mem0 在 history.db messages（派生副本）",
|
||||
"pass": bool(base_row and mem0_raw_kept),
|
||||
"expect": "夹具逐字逐角色核对通过；两侧原文都留存：现状在 chat_message/"
|
||||
"mem_episodic.raw；Mem0 在 history.db messages（派生副本）",
|
||||
"pass": bool(integ["pass"] and baseline_raw_kept and mem0_raw_kept),
|
||||
}
|
||||
|
||||
# ---- S2 跨会话细节（检索命中 / 细节进上下文 分列）----
|
||||
# ---- S2 跨会话细节（检索命中 / 细节进上下文 分列；细节按等价集语义判定）----
|
||||
b_hits, b_mode = base.search("旧书店")
|
||||
b_block, _, _ = base.context_block("旧书店", turn_id="q_s2")
|
||||
b_reread = base.reread_turn(b_hits[0]) if b_hits else {"messages": []}
|
||||
@@ -254,22 +547,24 @@ def run() -> dict:
|
||||
m_hits = mem0_search_texts(rr.pub, "旧书店 诗集")
|
||||
m_block = assemble_block([h["memory"] for h in m_hits])
|
||||
detail_in = {
|
||||
"baseline": {"hit": bool(b_hits), "detail_in_context": UNIQUE_DETAIL in b_block,
|
||||
"detail_in_reread": UNIQUE_DETAIL in b_reread_text},
|
||||
"baseline": {"hit": bool(b_hits), "detail_in_context": FX.detail_equiv_hit(b_block),
|
||||
"detail_in_reread": FX.detail_equiv_hit(b_reread_text)},
|
||||
"mem0": {"hit": bool(m_hits),
|
||||
"detail_in_context": has_any(m_block, [UNIQUE_DETAIL], ["damp smell"]),
|
||||
"detail_in_context": FX.detail_equiv_hit(m_block),
|
||||
"memories": [h["memory"] for h in m_hits][:3]},
|
||||
}
|
||||
q2 = "还记得我买诗集那天的细节吗？"
|
||||
a_base = rr.ask(q2, b_block or "（无可用记忆）")
|
||||
a_mem0 = rr.ask(q2, m_block or "（无可用记忆）")
|
||||
results["S2_cross_session_detail"] = {
|
||||
"detail_equiv_set": FX.DETAIL_EQUIV,
|
||||
"retrieval": detail_in,
|
||||
"model_use": {"baseline_answer": a_base["answer"], "mem0_answer": a_mem0["answer"],
|
||||
"baseline_detail_used": UNIQUE_DETAIL in a_base["answer"],
|
||||
"mem0_detail_used": (UNIQUE_DETAIL in a_mem0["answer"])
|
||||
or ("damp" in a_mem0["answer"].lower())},
|
||||
"expect": "两侧检索命中；细节是否进入注入块与模型实际使用分列（现状靠回读，Mem0 靠提取保真）",
|
||||
"baseline_detail_used": FX.detail_equiv_hit(a_base["answer"]),
|
||||
"mem0_detail_used": FX.detail_equiv_hit(a_mem0["answer"])},
|
||||
"expect": "两侧检索命中；细节是否进入注入块与模型实际使用分列（现状靠回读，"
|
||||
"Mem0 靠提取保真）；细节判定按预先声明的等价集（如'潮湿的气味'≈"
|
||||
"'受潮的味道'），不按单一词面",
|
||||
"pass": bool(b_hits) and bool(m_hits),
|
||||
}
|
||||
|
||||
@@ -306,45 +601,28 @@ def run() -> dict:
|
||||
for h in m_hits3b)),
|
||||
}
|
||||
|
||||
# ---- S4 改口更正（两批：先周三、后改周五；infer ADD-only + 我方编排显式 update）----
|
||||
# ---- S4 改口更正（已知id区分旧安排/新安排；两批还原"稍后改口"）----
|
||||
# 现状基线用其可命中形态（关键词"体检"）验证自身检索不退化；自然语言查询
|
||||
# "体检是哪天"在现状词面本就无命中（E1 类缺口，如实记录）。
|
||||
b_kw, _ = base.search("体检")
|
||||
b_nl, b_nl_mode = base.search(Q_CORRECTION)
|
||||
b_both = sorted({h["turn_id"] for h in b_kw})
|
||||
m_hits4 = mem0_search_texts(rr.pub, Q_CORRECTION)
|
||||
corr_texts = [h["memory"] for h in m_hits4]
|
||||
has_new = any(has_any(t, ["周五"], ["friday"]) for t in corr_texts)
|
||||
has_old = any(has_any(t, ["周三"], ["wednesday"]) for t in corr_texts)
|
||||
update_rec = None
|
||||
if has_old:
|
||||
old_item = next(h for h in m_hits4 if has_any(h["memory"], ["周三"], ["wednesday"]))
|
||||
new_text = old_item["memory"] + "（改期确认：体检在周五上午）"
|
||||
try:
|
||||
rr.pub.update(old_item["id"], text=new_text,
|
||||
metadata={"corrected_at": "synthetic-c2"})
|
||||
update_rec = {"updated": True}
|
||||
except Exception as e:
|
||||
update_rec = {"updated": False, "error": str(e)}
|
||||
with sqlite3.connect(str(AB / "mem0" / "mem0_public_history.db")) as c:
|
||||
upd_rows = c.execute(
|
||||
"SELECT old_memory,new_memory,event FROM history WHERE event='UPDATE'").fetchall()
|
||||
a4 = rr.ask(Q_CORRECTION, assemble_block(corr_texts) or "（无可用记忆）")
|
||||
s4 = s4_correction_orchestration(rr.pub, AB / "mem0" / "mem0_public_history.db",
|
||||
corr_old_id, corr_new_id)
|
||||
# 问答必须在"更新后"重新检索装配的块上判定（run2 用的是更新前列表）
|
||||
a4 = rr.ask(Q_CORRECTION, s4["post_update_block"] or "（无可用记忆）")
|
||||
results["S4_correction"] = {
|
||||
"baseline": {"keyword_hits": b_both, "natural_lang_hits": len(b_nl),
|
||||
"natural_lang_mode": b_nl_mode,
|
||||
"both_kept": any(t in b_both for t in ("c1", "c2"))
|
||||
and all(t in b_both for t in ("c1", "c2"))},
|
||||
"mem0": {"retrieved": corr_texts, "has_new_friday": has_new, "has_old_wednesday": has_old,
|
||||
"infer_is_add_only_both_entries": has_old and has_new,
|
||||
"our_update": update_rec,
|
||||
"history_update_rows": [list(r)[:3] for r in upd_rows]},
|
||||
"both_kept": all(t in b_both for t in ("c1", "c2"))},
|
||||
"mem0": s4,
|
||||
"model_use": {"answer": a4["answer"],
|
||||
"friday_used": ("周五" in a4["answer"]) or ("friday" in a4["answer"].lower())},
|
||||
"expect": "稍后改口时 infer 为 ADD-only（新旧并存，v4 事实）；我方显式 update 编排改口留 prev→new 史；"
|
||||
"现状关键词检索不退化，自然语言查询维持其已证词面缺口",
|
||||
"pass": all(t in b_both for t in ("c1", "c2")) and has_new and has_old
|
||||
and ("周五" in a4["answer"] or "friday" in a4["answer"].lower()),
|
||||
"expect": "稍后改口时 infer 为 ADD-only（新旧并存，v4 事实）；我方编排以已知id"
|
||||
"显式 update 旧条目留 prev→new 史且不误改新条目；现状关键词检索不"
|
||||
"退化，自然语言查询维持其已证词面缺口；问答用更新后重新装配的块",
|
||||
"pass": bool(all(t in b_both for t in ("c1", "c2")) and s4["pass"]
|
||||
and ("周五" in a4["answer"] or "friday" in a4["answer"].lower())),
|
||||
}
|
||||
|
||||
# ---- S5 事实/感受分离（现状用其可命中关键词形态；Mem0 双语判据）----
|
||||
@@ -373,51 +651,63 @@ def run() -> dict:
|
||||
and any(has_any(h["memory"], ["紧张", "睡"], ["nervous", "sleep"]) for h in m_feel),
|
||||
}
|
||||
|
||||
# ---- S6 承诺状态（我方编排 open→resolved）----
|
||||
m_promise = mem0_search_texts(rr.pub, Q_PROMISE)
|
||||
p_texts = [h["memory"] for h in m_promise]
|
||||
status_meta = None
|
||||
if m_promise:
|
||||
target = next((h for h in m_promise
|
||||
if has_any(h["memory"], ["结果", "告诉"], ["result", "tell"])),
|
||||
m_promise[0])
|
||||
try:
|
||||
rr.pub.update(target["id"], text=target["memory"],
|
||||
metadata={"promise_status": "resolved",
|
||||
"resolved_at": "synthetic-t2"})
|
||||
status_meta = "resolved"
|
||||
except Exception as e:
|
||||
status_meta = f"error:{e}"
|
||||
with sqlite3.connect(str(AB / "mem0" / "mem0_public_history.db")) as c:
|
||||
hist_events = [r[0] for r in c.execute("SELECT event FROM history ORDER BY id").fetchall()]
|
||||
a6 = rr.ask(Q_PROMISE, assemble_block(p_texts) or "（无可用记忆）")
|
||||
# ---- S6 承诺状态（我方编排 open→resolved；已知id区分原承诺/结果）----
|
||||
s6 = s6_promise_orchestration(rr.pub, AB / "mem0" / "mem0_public_history.db",
|
||||
promise_id, result_id)
|
||||
# 问答在更新后重新检索装配的块上判定；承诺状态入块（resolved 标注行）
|
||||
a6_block = s6["post_update_block"] or "（无可用记忆）"
|
||||
a6 = rr.ask(Q_PROMISE, a6_block)
|
||||
results["S6_promise_status"] = {
|
||||
"mem0": {"retrieved": p_texts, "status_orchestration": status_meta,
|
||||
"history_events": hist_events},
|
||||
"model_use": {"answer": a6["answer"]},
|
||||
"mem0": s6,
|
||||
"model_use": {"answer": a6["answer"],
|
||||
"promise_mentioned": has_any(a6["answer"], ["告诉", "答应"], ["tell"])},
|
||||
"baseline_note": "现状路径承诺埋在聊天行内，无独立承诺对象（v2 已记录缺口）",
|
||||
"expect": "承诺以独立条目存在；我方编排显式 update 置 resolved 且 history 留痕",
|
||||
"pass": bool(m_promise) and status_meta == "resolved"
|
||||
and "UPDATE" in hist_events,
|
||||
"expect": "承诺以独立条目存在；我方编排以已知id显式 update 原承诺置 resolved 且 "
|
||||
"history 留痕、不误置面试结果条目；问答用更新后重新装配的块",
|
||||
"pass": bool(s6["pass"] and has_any(a6["answer"], ["告诉", "答应"], ["tell"])),
|
||||
}
|
||||
|
||||
# ---- S7 来源回读（自洽判据：回读恢复命中行自身原文）----
|
||||
# ---- S7 来源回读（链判定：命中 id → history 关联 + payload 锚点 → 对应原文）----
|
||||
b_hits7, _ = base.search("诗集")
|
||||
rr7 = base.reread_turn(b_hits7[0]) if b_hits7 else {"messages": []}
|
||||
b7_text = " ".join(m["text"] for m in rr7["messages"])
|
||||
b7_ok = bool(b_hits7) and (b_hits7[0]["raw"] in b7_text)
|
||||
m_hits7 = mem0_search_texts(rr.pub, Q_EXACT)
|
||||
m7 = m_hits7[0]["memory"] if m_hits7 else ""
|
||||
with sqlite3.connect(str(AB / "mem0" / "mem0_public_history.db")) as c:
|
||||
raw_ok = any(FX.PUBLIC_CORPUS[0][2][:10] in (r[0] or "") for r in
|
||||
c.execute("SELECT content FROM messages").fetchall())
|
||||
hit7 = m_hits7[0] if m_hits7 else None
|
||||
chain7 = s7_chain_resolve(rr.pub, hit7) if hit7 else {"resolved": False,
|
||||
"reject_reason": "no_hit"}
|
||||
hist7 = history_rows_for(AB / "mem0" / "mem0_public_history.db",
|
||||
hit7["id"]) if hit7 else []
|
||||
deriv_ok = bool(hit7) and any(r["event"] in ("ADD", "UPDATE")
|
||||
and (r["new"] or "") == hit7["memory"] for r in hist7)
|
||||
target_raw = FX.PUBLIC_CORPUS[0][2]
|
||||
originals7 = (chain7.get("originals") or [])
|
||||
original_ok = chain7.get("resolved") and any(
|
||||
o["raw"] == target_raw for o in originals7 if o["role"] == "user")
|
||||
gated7 = s7_gated_assembly(rr.pub, m_hits7)
|
||||
results["S7_source_reread"] = {
|
||||
"baseline": {"hit_turn": b_hits7[0]["turn_id"] if b_hits7 else None,
|
||||
"reread_recovered_own_raw": b7_ok},
|
||||
"mem0": {"retrieved_memory": m7,
|
||||
"raw_recoverable_from_history": raw_ok},
|
||||
"expect": "从检索命中可回查原始来源：现状带门回读恢复该行原文；Mem0 命中是压缩事实，原文在 history.db",
|
||||
"pass": b7_ok and raw_ok and bool(m_hits7),
|
||||
"mem0": {"hit_id": hit7.get("id") if hit7 else None,
|
||||
"retrieved_memory": hit7.get("memory") if hit7 else "",
|
||||
"hit_history_derivation_matches": deriv_ok,
|
||||
"chain": {"resolved": chain7.get("resolved"),
|
||||
"src_group": chain7.get("src_group"),
|
||||
"original_rows": [{"turn": o["turn"], "role": o["role"]}
|
||||
for o in originals7]},
|
||||
"chain_recovered_user_original": original_ok},
|
||||
"gated_assembly": {"block_len": len(gated7["block"]),
|
||||
"used": len(gated7["used"]),
|
||||
"rejected": gated7["rejected"],
|
||||
"leak_markers": gated7["leak_markers"],
|
||||
"within_budget": gated7["within_budget"]},
|
||||
"sdk_adaptation_note": "search/get_all 不回传自定义 metadata——链回读经适配层"
|
||||
"直读向量 payload 锚点（我方责任）；messages 表无 memory_id "
|
||||
"关联（SDK 限制，如实列明），原文关联由我方锚点承载",
|
||||
"expect": "从实际命中建立到原文的链：现状带门回读恢复该行原文；Mem0 命中→"
|
||||
"history 推导史吻合→payload 锚点→输入清单恢复对应原句；600 字装配"
|
||||
"仅纳入链验证条目且零私密标记",
|
||||
"pass": bool(b7_ok and deriv_ok and original_ok and gated7["pass"]),
|
||||
}
|
||||
|
||||
# ---- S8 进程重启（子进程真实重启；infer=False 零出站）----
|
||||
@@ -467,25 +757,32 @@ def run() -> dict:
|
||||
close_memory(rr.pub)
|
||||
summary = ledger.summary()
|
||||
counters = COUNTERS.snapshot()
|
||||
embed_ledger_summary = EMBED_LEDGER.summary()
|
||||
results["_ledger_summary"] = {k: v for k, v in summary.items() if k != "rows"}
|
||||
results["_ledger_rows"] = summary["rows"]
|
||||
results["_counters"] = counters
|
||||
results["_embed_ledger_card_summary"] = embed_ledger_summary
|
||||
results["_real_calls_used"] = summary["debited"]
|
||||
|
||||
scenario_keys = ["S1_verbatim_retention", "S2_cross_session_detail",
|
||||
"S3_paraphrase_recall", "S4_correction",
|
||||
"S5_fact_feeling_separation", "S6_promise_status",
|
||||
"S7_source_reread", "S8_process_restart", "S9_backup_restore"]
|
||||
s3 = results["S3_paraphrase_recall"]
|
||||
others_ok = all(results[k].get("pass") for k in
|
||||
["S1_verbatim_retention", "S2_cross_session_detail", "S4_correction",
|
||||
"S5_fact_feeling_separation", "S6_promise_status", "S7_source_reread",
|
||||
"S8_process_restart", "S9_backup_restore"])
|
||||
others_ok = all(results[k].get("pass") for k in scenario_keys if k != "S3_paraphrase_recall")
|
||||
all_pass = all(results[k].get("pass") for k in scenario_keys)
|
||||
outcome = {
|
||||
"completed": True,
|
||||
"all_scenarios_pass": all_pass,
|
||||
"failed_scenarios": [k for k in scenario_keys if not results[k].get("pass")],
|
||||
"mem0_paraphrase_strictly_better": s3["pass"],
|
||||
"others_no_regression": others_ok,
|
||||
"budget_respected": summary["debited"] <= 60 and counters["embed_items_total"] <= 500,
|
||||
"budget_respected": (summary["debited"] <= 60
|
||||
and embed_ledger_summary["embed_items_total"] <= 500),
|
||||
"verdict": "",
|
||||
}
|
||||
outcome["verdict"] = (
|
||||
"换说法召回 Mem0 严格优于现状；其他既有成功项未退化" if (s3["pass"] and others_ok)
|
||||
"换说法召回 Mem0 严格优于现状；其他既有成功项未退化" if all_pass
|
||||
else "存在未达标项，见逐场景记录")
|
||||
outcome["headline_findings"] = {
|
||||
"extraction_language": (
|
||||
@@ -510,4 +807,5 @@ def main(argv: list[str]) -> dict:
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(0 if main([])["outcome"]["completed"] else 1)
|
||||
# 退出码依赖逐场景实际验收结果（不再以 completed/行存在放行）
|
||||
raise SystemExit(0 if main([])["outcome"]["all_scenarios_pass"] else 1)
|
||||
|
||||
+98
-1
@@ -78,6 +78,84 @@ class Counters:
|
||||
COUNTERS = Counters()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 卡级持久嵌入账本（HEL-858 返工新增）：进程内 Counters 只覆盖单进程，
|
||||
# 跨进程/子进程/batch 一律落同一 SQLite 账本；500 上限按全卡累计判定。
|
||||
EMBED_LEDGER_PATH = Path(os.environ.get("HEL857_EMBED_LEDGER",
|
||||
DATA_DIR / "embed_ledger.sqlite3"))
|
||||
|
||||
|
||||
class EmbedLedger:
|
||||
"""跨进程持久嵌入计数。add 侧（dense/entity）与词面（keyword）分表另列。
|
||||
|
||||
历史用量（HEL-857 run1/run2/门禁，未持久化时期）以带方法说明的
|
||||
reconstruction 行补种；补种行同样计入 used()，不重置、不改历史。
|
||||
"""
|
||||
|
||||
def __init__(self, path: Optional[Path] = None):
|
||||
self.path = Path(path) if path else EMBED_LEDGER_PATH
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._lock = __import__("threading").Lock()
|
||||
with self._connect() as c:
|
||||
c.execute(
|
||||
"CREATE TABLE IF NOT EXISTS embed_ops ("
|
||||
" id INTEGER PRIMARY KEY AUTOINCREMENT,"
|
||||
" ts TEXT NOT NULL, pid INTEGER NOT NULL,"
|
||||
" kind TEXT NOT NULL," # dense | entity | keyword | seed_dense | seed_keyword
|
||||
" items INTEGER NOT NULL, note TEXT NOT NULL DEFAULT '')")
|
||||
|
||||
def _connect(self):
|
||||
import sqlite3
|
||||
conn = sqlite3.connect(str(self.path), timeout=30.0, isolation_level=None)
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA synchronous=FULL")
|
||||
return conn
|
||||
|
||||
def debit(self, kind: str, items: int = 1, note: str = "") -> None:
|
||||
with self._lock, self._connect() as c:
|
||||
c.execute("INSERT INTO embed_ops(ts,pid,kind,items,note) VALUES(?,?,?,?,?)",
|
||||
(now_ms_iso(), os.getpid(), kind, items, note))
|
||||
|
||||
def used_by_kind(self) -> Dict[str, int]:
|
||||
with self._connect() as c:
|
||||
rows = c.execute("SELECT kind, SUM(items) FROM embed_ops GROUP BY kind").fetchall()
|
||||
return {k: int(v or 0) for k, v in rows}
|
||||
|
||||
def used_embed_items(self) -> int:
|
||||
"""密集+实体合计（词面不计入 500 上限，另列）。"""
|
||||
by = self.used_by_kind()
|
||||
return (by.get("dense", 0) + by.get("entity", 0)
|
||||
+ by.get("seed_dense", 0) + by.get("seed_entity", 0))
|
||||
|
||||
def remaining(self) -> int:
|
||||
return EMBED_ITEM_CAP - self.used_embed_items()
|
||||
|
||||
def check(self) -> None:
|
||||
used = self.used_embed_items()
|
||||
if used > EMBED_ITEM_CAP:
|
||||
raise RuntimeError(f"卡级嵌入条数超限: {used} > {EMBED_ITEM_CAP}")
|
||||
|
||||
def summary(self) -> Dict[str, Any]:
|
||||
by = self.used_by_kind()
|
||||
return {
|
||||
"ledger_path": str(self.path),
|
||||
"dense": by.get("dense", 0) + by.get("seed_dense", 0),
|
||||
"entity": by.get("entity", 0) + by.get("seed_entity", 0),
|
||||
"embed_items_total": self.used_embed_items(),
|
||||
"keyword": by.get("keyword", 0) + by.get("seed_keyword", 0),
|
||||
"cap": EMBED_ITEM_CAP,
|
||||
"remaining": self.remaining(),
|
||||
}
|
||||
|
||||
|
||||
def now_ms_iso() -> str:
|
||||
import time as _t
|
||||
return _t.strftime("%Y-%m-%dT%H:%M:%S", _t.gmtime(_t.time())) + "Z"
|
||||
|
||||
|
||||
EMBED_LEDGER = EmbedLedger()
|
||||
|
||||
|
||||
def mem_config(role: str, *, vec_root: Path, hist_root: Path) -> Dict[str, Any]:
|
||||
"""双实例配置。role in {private, public}。"""
|
||||
return {
|
||||
@@ -110,22 +188,41 @@ def mem_config(role: str, *, vec_root: Path, hist_root: Path) -> Dict[str, Any]:
|
||||
|
||||
|
||||
def install_counters(memory) -> None:
|
||||
"""在实例上包装嵌入/词面计数。真实 LLM 出站计数在传输层（ledger）。"""
|
||||
"""在实例上包装嵌入/词面计数。真实 LLM 出站计数在传输层（ledger）。
|
||||
|
||||
进程内 COUNTERS 之外，同步落卡级持久嵌入账本（跨进程累计，500 上限
|
||||
按 EmbedLedger.check() 全卡判定）。
|
||||
"""
|
||||
emb = memory.embedding_model
|
||||
orig_embed = emb.embed
|
||||
|
||||
def counted_embed(text, memory_action=None):
|
||||
COUNTERS.embed_calls += 1
|
||||
COUNTERS.embed_items += 1
|
||||
EMBED_LEDGER.debit("dense", 1, note=str(memory_action))
|
||||
COUNTERS.check()
|
||||
EMBED_LEDGER.check()
|
||||
return orig_embed(text, memory_action)
|
||||
|
||||
emb.embed = counted_embed # type: ignore[method-assign]
|
||||
|
||||
orig_embed_batch = getattr(emb, "embed_batch", None)
|
||||
if callable(orig_embed_batch):
|
||||
def counted_embed_batch(texts, memory_action=None):
|
||||
n = len(texts) if isinstance(texts, (list, tuple)) else 1
|
||||
COUNTERS.embed_items += n
|
||||
EMBED_LEDGER.debit("dense", n, note=f"batch:{memory_action}")
|
||||
COUNTERS.check()
|
||||
EMBED_LEDGER.check()
|
||||
return orig_embed_batch(texts, memory_action)
|
||||
|
||||
emb.embed_batch = counted_embed_batch # type: ignore[method-assign]
|
||||
|
||||
orig_kw = memory.vector_store.keyword_search
|
||||
|
||||
def counted_kw(*a, **kw):
|
||||
COUNTERS.keyword_calls += 1
|
||||
EMBED_LEDGER.debit("keyword", 1)
|
||||
return orig_kw(*a, **kw)
|
||||
|
||||
memory.vector_store.keyword_search = counted_kw # type: ignore[method-assign]
|
||||
|
||||
@@ -56,6 +56,22 @@ Q_PROMISE = "面试结果答应过我什么" # 承诺
|
||||
UNIQUE_DETAIL = "受潮的味道" # 跨会话细节回读判据
|
||||
ANSWER_KEY_PARAPHRASE = "蓝色" # 模型实际使用判据
|
||||
|
||||
# 语义等价集（HEL-858 返工：S2 等同义表达按语义判定，不按单一词面）。
|
||||
# 固定合成语料，等价集为预先声明的已知改写，不冒充通用语义理解。
|
||||
DETAIL_EQUIV = ["受潮的味道", "潮湿的气味", "潮味", "damp smell", "dampness", "damp"]
|
||||
|
||||
# 部分分享负例（总管独立复核合成例，固定为语料常量）：
|
||||
# 内部事件含从未讲出的秘密；实际收到的回复只讲了前半句。
|
||||
PARTIAL_EVENT = "今天喝了茶；另有从未讲出的秘密标记：隐私细节A"
|
||||
PARTIAL_SPOKEN = "今天喝了茶"
|
||||
PARTIAL_SECRET = "隐私细节A"
|
||||
|
||||
|
||||
def detail_equiv_hit(text: str) -> bool:
|
||||
"""细节语义判定：等价集内任一表达命中（中英不区分大小写）。"""
|
||||
low = (text or "").lower()
|
||||
return any(t in (text or "") or t in low for t in DETAIL_EQUIV)
|
||||
|
||||
|
||||
def public_messages(turns=None) -> list[dict]:
|
||||
turns = turns or "base"
|
||||
|
||||
@@ -0,0 +1,581 @@
|
||||
{
|
||||
"R1_baseline_refixed": {
|
||||
"fixture_integrity_refixed": {
|
||||
"expected_rows": 15,
|
||||
"actual_rows": 15,
|
||||
"missing": [],
|
||||
"mismatched": [],
|
||||
"extra_roles_rows": [],
|
||||
"role_crossed": [],
|
||||
"episodic_missing": [],
|
||||
"verbatim_all_roles": true,
|
||||
"pass": true,
|
||||
"expect": "两侧同一资料按 (turn, role) 完整装载：原句逐字、角色正确、无覆盖"
|
||||
},
|
||||
"legacy_corruption_now_caught": {
|
||||
"verbatim_all_roles": false,
|
||||
"companion_rows_lost": [
|
||||
[
|
||||
"t1",
|
||||
"companion"
|
||||
],
|
||||
[
|
||||
"t2",
|
||||
"companion"
|
||||
],
|
||||
[
|
||||
"t3",
|
||||
"companion"
|
||||
],
|
||||
[
|
||||
"t4",
|
||||
"companion"
|
||||
]
|
||||
],
|
||||
"role_crossed": [
|
||||
[
|
||||
"t1",
|
||||
"user"
|
||||
],
|
||||
[
|
||||
"t2",
|
||||
"user"
|
||||
],
|
||||
[
|
||||
"t3",
|
||||
"user"
|
||||
],
|
||||
[
|
||||
"t4",
|
||||
"user"
|
||||
]
|
||||
],
|
||||
"pass": true,
|
||||
"expect": "run2 装载错误形态（伴侣文本占 user 位、伴侣行丢失）被核对门判失败"
|
||||
},
|
||||
"S1_baseline_refixed": {
|
||||
"user_raw_verbatim": true,
|
||||
"companion_row_present_with_own_text": true,
|
||||
"pass": true,
|
||||
"expect": "两侧同一资料逐字逐角色：t1 用户原句与伴侣回复各自在库（run2 为 false）"
|
||||
},
|
||||
"S2_baseline_refixed": {
|
||||
"hit": true,
|
||||
"detail_in_context": false,
|
||||
"detail_in_reread": true,
|
||||
"reread_roles": [
|
||||
"user"
|
||||
],
|
||||
"run2_recorded_for_contrast": {
|
||||
"detail_in_reread": false,
|
||||
"note": "run2 夹具被覆盖导致假阴性；修复后现状回读可恢复细节"
|
||||
},
|
||||
"pass": true,
|
||||
"expect": "修复夹具后现状：检索命中、gist 压缩丢细节、带门回读恢复细节（装载错误不再记作产品缺点）"
|
||||
},
|
||||
"S3_baseline_refixed": {
|
||||
"paraphrase_query": {
|
||||
"hits": 0,
|
||||
"mode": "none"
|
||||
},
|
||||
"paraphrase_topic_query": {
|
||||
"hits": 0,
|
||||
"mode": "none"
|
||||
},
|
||||
"run2_mem0_evidence_retained": {
|
||||
"paraphrase_hits": 11,
|
||||
"topic_hits": 11,
|
||||
"answer": "蓝色",
|
||||
"note": "mem0 侧 run2 真实证据保留（其输入未被夹具缺陷影响）；配对模型使用判定未运行"
|
||||
},
|
||||
"pass": true,
|
||||
"expect": "修复夹具后现状词面对换说法仍无命中（重验，非沿用损坏夹具结论）"
|
||||
},
|
||||
"S4_baseline_refixed": {
|
||||
"keyword_hits": [
|
||||
"c1",
|
||||
"c2"
|
||||
],
|
||||
"pass": true,
|
||||
"expect": "体检关键词命中 c1 与 c2（不退化）"
|
||||
},
|
||||
"S5_baseline_refixed": {
|
||||
"fact_kw_hits": [
|
||||
"f1",
|
||||
"f2",
|
||||
"p1",
|
||||
"p2"
|
||||
],
|
||||
"feeling_kw_hits": [
|
||||
"f2"
|
||||
],
|
||||
"pass": true,
|
||||
"expect": "事实/感受关键词各自命中（不退化）"
|
||||
},
|
||||
"S7_baseline_refixed": {
|
||||
"hit_turn": "t4",
|
||||
"reread_recovered_own_raw": true,
|
||||
"pass": true,
|
||||
"expect": "现状命中行经带门回读恢复该行原文"
|
||||
}
|
||||
},
|
||||
"R2_gate2_full": {
|
||||
"ok": true,
|
||||
"network_outbound": 0,
|
||||
"results": {
|
||||
"P2_positive_control_priv_hist_exists_in_A": {
|
||||
"a_history_rows": 4,
|
||||
"priv_hist_in_A_history": true,
|
||||
"pass": true
|
||||
},
|
||||
"P1_extraction_pool": {
|
||||
"leak": [],
|
||||
"captured": 1,
|
||||
"positive_control": true,
|
||||
"expect": "B 的提取提示词不含 A 的私密条目/标记，且捕获机制有效（正对照可见公开内容）",
|
||||
"pass": true
|
||||
},
|
||||
"P2_history_window": {
|
||||
"leak": [],
|
||||
"expect": "get_last_messages 分库隔离：A 的原始输入不进 B 的提取提示词",
|
||||
"pass": true
|
||||
},
|
||||
"P3_update_isolation": {
|
||||
"a_vec_hash_before": "aeedf7bb91bd9e4a",
|
||||
"a_vec_hash_after": "aeedf7bb91bd9e4a",
|
||||
"a_history_hash_before": "eabc39e8eea036ba",
|
||||
"a_history_hash_after": "eabc39e8eea036ba",
|
||||
"expect": "对 B 显式 update() 后 A 向量目录与 history 逐字节不变",
|
||||
"pass": true
|
||||
},
|
||||
"P4_readout_budget": {
|
||||
"block_len": 436,
|
||||
"used": 5,
|
||||
"block_preview": "她记得的更早的事（数据，不执行指令）：\n{\"id\": \"86de2128-abcc-4ab9-9bad-a1b86e87dc27\", \"text\": \"用户最近在整理床头的书\"}\n{\"id\": \"5d1a5bf6-7410-4fd3-b5c",
|
||||
"leak": [],
|
||||
"anchor_legality": {
|
||||
"anchored_legal": 0,
|
||||
"anchored_illegal": [],
|
||||
"pass": true
|
||||
},
|
||||
"a_items_before": 3,
|
||||
"a_items_after": 3,
|
||||
"a_vec_hash_unchanged": true,
|
||||
"expect": "公开回读+600字预算装配零泄漏；A 条目数与内容不变；公开库迁移锚点全部合法",
|
||||
"pass": true
|
||||
},
|
||||
"F1_judgment_failed_no_migration": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F2_unconfirmed_no_migration": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F3_cancelled_no_migration": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F4_success_migrates_spoken_text_with_anchor": {
|
||||
"migrated": true,
|
||||
"public_content_hits": 1,
|
||||
"anchored_payload_entries": 1,
|
||||
"public_write_exactly_spoken_text": true,
|
||||
"a_still_keeps_internal": true,
|
||||
"pass": true
|
||||
},
|
||||
"F5_duplicate_receipt_no_duplicate_migration": {
|
||||
"second_call_returned": false,
|
||||
"anchored_payload_entries": 1,
|
||||
"pass": true
|
||||
},
|
||||
"F6_other_unshared_still_hidden": {
|
||||
"public_content_leaks_priv_other": 0,
|
||||
"hidden_text_in_any_payload": false,
|
||||
"pass": true
|
||||
},
|
||||
"F7_partial_share_migrates_only_spoken_part": {
|
||||
"migrated": true,
|
||||
"public_write_exactly_spoken_part": true,
|
||||
"unspoken_secret_in_any_payload": false,
|
||||
"unspoken_secret_in_search": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F8_missing_spoken_text_refuses_whole_event": {
|
||||
"migrated": false,
|
||||
"event_content_in_any_payload": false,
|
||||
"refusal_rows": [
|
||||
[
|
||||
108,
|
||||
13,
|
||||
"missing_spoken_text"
|
||||
]
|
||||
],
|
||||
"refusal_recorded": true,
|
||||
"pass": true
|
||||
},
|
||||
"P1_restored_extraction_pool": {
|
||||
"leak": [],
|
||||
"captured": 1,
|
||||
"positive_control": true,
|
||||
"expect": "恢复后 B 的提取提示词仍不含 A 的私密条目/标记，正对照可见公开内容",
|
||||
"pass": true
|
||||
},
|
||||
"P2_restored_history_window": {
|
||||
"leak": [],
|
||||
"expect": "恢复后 get_last_messages 分库隔离仍成立",
|
||||
"pass": true
|
||||
},
|
||||
"P3_restored_update_isolation": {
|
||||
"a_vec_hash_before": "aeedf7bb91bd9e4a",
|
||||
"a_vec_hash_after": "aeedf7bb91bd9e4a",
|
||||
"a_history_hash_before": "eabc39e8eea036ba",
|
||||
"a_history_hash_after": "eabc39e8eea036ba",
|
||||
"expect": "恢复后对 B 显式 update() 仍不影响 A（逐字节不变）",
|
||||
"pass": true
|
||||
},
|
||||
"P4_restored_readout_budget": {
|
||||
"block_len": 586,
|
||||
"leak": [],
|
||||
"anchor_legality": {
|
||||
"anchored_legal": 2,
|
||||
"anchored_illegal": [],
|
||||
"pass": true
|
||||
},
|
||||
"expect": "恢复后公开出口仍零泄漏；迁移锚点在恢复库 telling_log 均有对应记录",
|
||||
"pass": true
|
||||
},
|
||||
"P5_full_backup_restore": {
|
||||
"backup_files": [
|
||||
"backup_manifest.json",
|
||||
"mem0_private_history.db",
|
||||
"mem0_public_history.db",
|
||||
"share_control.db",
|
||||
"vec_private",
|
||||
"vec_public"
|
||||
],
|
||||
"restored_private_items": 3,
|
||||
"restored_private_has_unshared": true,
|
||||
"internal_recall_priv_marker": true,
|
||||
"restored_public_outlet_leak": [],
|
||||
"restored_P1P2": {
|
||||
"p1": true,
|
||||
"p2": true
|
||||
},
|
||||
"expect": "恢复后：内部未分享经历仍在且内部可查；公开出口（检索+预算装配）仍不泄漏；P1-P4 复跑通过",
|
||||
"pass": true
|
||||
},
|
||||
"F1_judgment_failed_no_migration_restored": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F2_unconfirmed_no_migration_restored": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F3_cancelled_no_migration_restored": {
|
||||
"migrated": false,
|
||||
"public_content_leaks": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F4_success_migrates_spoken_text_with_anchor_restored": {
|
||||
"migrated": true,
|
||||
"public_content_hits": 2,
|
||||
"anchored_payload_entries": 1,
|
||||
"public_write_exactly_spoken_text": true,
|
||||
"a_still_keeps_internal": true,
|
||||
"pass": true
|
||||
},
|
||||
"F5_duplicate_receipt_no_duplicate_migration_restored": {
|
||||
"second_call_returned": false,
|
||||
"anchored_payload_entries": 1,
|
||||
"pass": true
|
||||
},
|
||||
"F6_other_unshared_still_hidden_restored": {
|
||||
"public_content_leaks_priv_other": 0,
|
||||
"hidden_text_in_any_payload": false,
|
||||
"pass": true
|
||||
},
|
||||
"F7_partial_share_migrates_only_spoken_part_restored": {
|
||||
"migrated": true,
|
||||
"public_write_exactly_spoken_part": true,
|
||||
"unspoken_secret_in_any_payload": false,
|
||||
"unspoken_secret_in_search": 0,
|
||||
"pass": true
|
||||
},
|
||||
"F8_missing_spoken_text_refuses_whole_event_restored": {
|
||||
"migrated": false,
|
||||
"event_content_in_any_payload": false,
|
||||
"refusal_rows": [
|
||||
[
|
||||
108,
|
||||
13,
|
||||
"missing_spoken_text"
|
||||
],
|
||||
[
|
||||
208,
|
||||
15,
|
||||
"missing_spoken_text"
|
||||
]
|
||||
],
|
||||
"refusal_recorded": true,
|
||||
"pass": true
|
||||
},
|
||||
"F5r_pre_restore_telling_log_idempotent_after_restore_restored": {
|
||||
"second_call_returned": false,
|
||||
"anchored_payload_entries": 1,
|
||||
"pass": true
|
||||
}
|
||||
},
|
||||
"notes": {
|
||||
"stub": "OpenAILLM.generate_response 实例级替换，捕获全部提示词，零网络",
|
||||
"bridge": "ShareBridge 为我方编排（生产 life/public.py 语义重放），add 原语来自 SDK",
|
||||
"api_finding": "SDK 的 search/get_all 返回项不回传自定义 metadata（锚点在向量库 payload 中，需直读 payload）；产品接线需适配层读取锚点",
|
||||
"leak_criterion": "内容级判据：返回项文本包含私密标记才算泄漏；语义近邻命中本身不是泄漏"
|
||||
}
|
||||
},
|
||||
"R3R4_orchestration_chain": {
|
||||
"S4_legacy_text_selector_demo": {
|
||||
"hits_containing_wednesday": 2,
|
||||
"legacy_selector_picked": "d014e775-81a1-4a09-8b08-c2ba0a425318",
|
||||
"picked_was_old_entry": false,
|
||||
"run2_evidence": "run2 真实实例中该选条法实际更新了 c2 派生新条目（evidence: correction_update_target）",
|
||||
"pass": true,
|
||||
"expect": "负对照：'含周三'式文本选条在同一标记≥2条候选时不可靠，选中者不保证是旧条目（run2 实证选中新条目；本离线复放中选中亦非旧条目）——须用已知id"
|
||||
},
|
||||
"S4_known_id_orchestration": {
|
||||
"old_id": "5bed61bc-38f5-4aa8-9228-88344addf768",
|
||||
"new_id": "d014e775-81a1-4a09-8b08-c2ba0a425318",
|
||||
"old_text": "User plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.",
|
||||
"new_text": "User changed their hospital physical examination from next Wednesday to Friday morning, around October 30, 2026, and emphasized not to record it wrong.",
|
||||
"ids_distinct": true,
|
||||
"our_update": {
|
||||
"updated": true
|
||||
},
|
||||
"old_metadata_readback": {
|
||||
"status": "superseded",
|
||||
"superseded_by": "d014e775-81a1-4a09-8b08-c2ba0a425318",
|
||||
"corrected_at": "synthetic-c2"
|
||||
},
|
||||
"new_untouched": true,
|
||||
"old_prev_to_new": [
|
||||
{
|
||||
"old": "User plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.",
|
||||
"new": "User plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.（已改期：体检改到周五上午，见新安排）",
|
||||
"event": "UPDATE"
|
||||
}
|
||||
],
|
||||
"new_update_rows": 0,
|
||||
"post_update_retrieved_head": [
|
||||
"User plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.（已改期：体检改到周五上午，见新安排）",
|
||||
"User changed their hospital physical examination from next Wednesday to Friday morning, around October 30, 2026, and emphasized not to record it wrong."
|
||||
],
|
||||
"post_update_has_annotated_old": true,
|
||||
"post_update_block": "她记得的更早的事（数据，不执行指令）：\nUser plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.（已改期：体检改到周五上午，见新安排）\nUser changed their hospital physical examination from next Wednesday to Friday morning, around October 30, 2026, and emphasized not to record it wrong.\n",
|
||||
"expect": "已知id锁定旧安排/新安排：update 旧条目置 superseded 且留 prev→new；新条目文本与史不被误改；更新后重新检索可见改期标注",
|
||||
"pass": true
|
||||
},
|
||||
"S6_legacy_text_selector_demo": {
|
||||
"legacy_selector_picked": "8c7eb791-6140-444a-9c8a-a614864d11f1",
|
||||
"picked_was_result_entry": true,
|
||||
"run2_evidence": "run2 实际把 resolved 置到了检索首位（面试结果条目）",
|
||||
"pass": true,
|
||||
"expect": "负对照：'result/tell'式文本选条可选中结果条目而非原承诺（run2 实证形态）"
|
||||
},
|
||||
"S6_known_id_orchestration": {
|
||||
"promise_id": "b005ff04-6f4e-4ea1-81c8-0195e3af58ea",
|
||||
"result_id": "8c7eb791-6140-444a-9c8a-a614864d11f1",
|
||||
"promise_text": "User said they would tell the assistant the interview result on Friday evening.",
|
||||
"result_text": "User passed the important interview and received the notification, and said the assistant was the first person they wanted to tell.",
|
||||
"ids_distinct": true,
|
||||
"our_update": {
|
||||
"updated": true
|
||||
},
|
||||
"promise_metadata_readback": {
|
||||
"promise_status": "resolved",
|
||||
"resolved_at": "synthetic-t2"
|
||||
},
|
||||
"promise_text_unchanged": true,
|
||||
"result_not_marked": true,
|
||||
"promise_update_rows": 1,
|
||||
"result_update_rows": 0,
|
||||
"post_update_retrieved_head": [
|
||||
"User passed the important interview and received the notification, and said the assistant was the first person they wanted to tell.",
|
||||
"User said they would tell the assistant the interview result on Friday evening.",
|
||||
"User plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.（已改期：体检改到周五上午，见新安排）",
|
||||
"User changed their hospital physical examination from next Wednesday to Friday morning, around October 30, 2026, and emphasized not to record it wrong."
|
||||
],
|
||||
"post_update_has_promise": true,
|
||||
"post_update_block": "她记得的更早的事（数据，不执行指令）：\nUser passed the important interview and received the notification, and said the assistant was the first person they wanted to tell.\nUser said they would tell the assistant the interview result on Friday evening.\nUser plans to go to the hospital for a physical examination next Wednesday, around November 4, 2026.（已改期：体检改到周五上午，见新安排）\nUser changed their hospital physical examination from next Wednesday to Friday morning, around October 30, 2026, and emphasized not to record it wrong.\n",
|
||||
"expect": "已知id锁定原承诺（非面试结果条目）：update 置 resolved 留痕，结果条目不被误置；更新后重新检索仍含原承诺",
|
||||
"pass": true
|
||||
},
|
||||
"S7_chain_positive": {
|
||||
"hit_id": "634d8028-8411-481c-9164-604daf5c6b45",
|
||||
"hit_history_derivation_matches": true,
|
||||
"chain_resolved": true,
|
||||
"chain_src_group": "base",
|
||||
"original_turns": [
|
||||
"t1"
|
||||
],
|
||||
"chain_recovered_user_original": true,
|
||||
"pass": true,
|
||||
"expect": "命中 id → history 推导史吻合 → payload 锚点 → 输入清单恢复 t1 用户原句"
|
||||
},
|
||||
"S7_chain_negatives": {
|
||||
"unknown_turn_rejected": true,
|
||||
"no_anchor_rejected": true,
|
||||
"wrong_source_rejected": true,
|
||||
"unshared_companion_blocked": true,
|
||||
"detail": {
|
||||
"unknown": {
|
||||
"resolved": false,
|
||||
"reject_reason": "unknown_source",
|
||||
"src_group": "mystery",
|
||||
"turn_ids": [
|
||||
"tz_unknown"
|
||||
],
|
||||
"hit_id": "9c7ef521-aa08-43c0-aa54-b8d51a47ddd7"
|
||||
},
|
||||
"no_anchor": {
|
||||
"resolved": false,
|
||||
"reject_reason": "unknown_source",
|
||||
"src_group": null,
|
||||
"turn_ids": [],
|
||||
"hit_id": "8d5ff054-f629-4839-868e-3dabd5b0fded"
|
||||
},
|
||||
"wrong_source": {
|
||||
"resolved": false,
|
||||
"reject_reason": "source_mismatch",
|
||||
"src_group": "base",
|
||||
"hit_id": "5047421a-c2a2-4148-b296-a169bc61c416",
|
||||
"note": "锚点输入指纹与来源组不符：换错来源/篡改锚点被拒"
|
||||
},
|
||||
"unshared_companion": {
|
||||
"resolved": false,
|
||||
"companion_allowed": false
|
||||
}
|
||||
},
|
||||
"pass": true,
|
||||
"expect": "未知来源/无锚点/输入指纹不符（换错来源）均拒绝；同轮未分享伴侣消息（told=0）不得进入回读/装配"
|
||||
},
|
||||
"S7_gated_assembly": {
|
||||
"used": 4,
|
||||
"rejected": [
|
||||
{
|
||||
"hit_id": "9c7ef521-aa08-43c0-aa54-b8d51a47ddd7",
|
||||
"reason": "unknown_source"
|
||||
},
|
||||
{
|
||||
"hit_id": "8d5ff054-f629-4839-868e-3dabd5b0fded",
|
||||
"reason": "unknown_source"
|
||||
},
|
||||
{
|
||||
"hit_id": "5047421a-c2a2-4148-b296-a169bc61c416",
|
||||
"reason": "source_mismatch"
|
||||
}
|
||||
],
|
||||
"leak_markers": [],
|
||||
"block_len": 554,
|
||||
"within_budget": true,
|
||||
"pass": true,
|
||||
"expect": "600 字装配仅纳入链验证条目；未知/无锚点/错来源条目被拒；零私密标记"
|
||||
}
|
||||
},
|
||||
"R5_gate0_fake_transport": {
|
||||
"ok": true
|
||||
},
|
||||
"R5_resource_account": {
|
||||
"history_reconstruction": {
|
||||
"run1": {
|
||||
"dense_recorded": 23,
|
||||
"child_add_derived": 1,
|
||||
"method": "results/ab_scenarios_run1.json _counters 记录值 + ab_child.py 单条 infer=False add 派生"
|
||||
},
|
||||
"run2": {
|
||||
"dense_recorded": 28,
|
||||
"child_add_derived": 1,
|
||||
"method": "results/ab_scenarios.json _counters 记录值 + ab_child.py 派生（进程内计数不含子进程）"
|
||||
},
|
||||
"gate1": {
|
||||
"add_path_points_exact": 4,
|
||||
"query_enumerated": 2,
|
||||
"method": "向量点数 4（qdrant 本地只读清点）+ 代码枚举：Q0b 语义检索 1 + 子进程检索 1"
|
||||
},
|
||||
"gate2": {
|
||||
"add_path_points_exact": 9,
|
||||
"update_exact": 1,
|
||||
"query_bound": 31,
|
||||
"method": "向量点数 9（src 两侧实例）+ P3 update 1 + 查询/阶段路径代码枚举上界 31"
|
||||
}
|
||||
},
|
||||
"history_dense_conservative_total": 100,
|
||||
"history_method_note": "精确不可恢复处标注上界与方法，不编数字、不重置：run1/run2 为进程内记录值+子进程派生；门禁按向量点数（旧数据目录只读清点：gate1=4、gate2 src=3+6、run2=12）与代码枚举上界重建。旧口径'全卡 28/500'作废（仅为 run2 进程内计数）。",
|
||||
"keyword_separate": {
|
||||
"run1_recorded": 10,
|
||||
"run2_recorded": 10,
|
||||
"gates_bound": 20
|
||||
},
|
||||
"card_embed_ledger": {
|
||||
"ledger_path": "/home/xiaomo/multica_workspaces/hello-af23464d222e/hel-858-e4ba66a50d06/workdir/hel858-data/embed_ledger.sqlite3",
|
||||
"dense": 165,
|
||||
"entity": 0,
|
||||
"embed_items_total": 165,
|
||||
"keyword": 63,
|
||||
"cap": 500,
|
||||
"remaining": 335
|
||||
},
|
||||
"suite_debits_this_run": {
|
||||
"suite_dense_used": 65,
|
||||
"keyword_used": 23
|
||||
},
|
||||
"cap": 500,
|
||||
"within_cap": true,
|
||||
"pass": true
|
||||
},
|
||||
"_retest_plan": {
|
||||
"precondition": "余下 34 次真实额度冻结不动；本方案仅供总管核对，未获批准前不执行",
|
||||
"fixed_materials": "fixtures.py 固定合成语料不变；add 随 payload 写入来源锚点（src_group/turn_ids）",
|
||||
"criteria": "修复后判据：S1 夹具逐字逐角色+两侧原文；S2 细节等价集语义判定+模型使用分列；S3 换说法配对重跑（现状修复夹具）；S4/S6 已知id编排+prev→new回读+更新后重新检索装配再问答；S7 命中→history+锚点→原文链+负例+600字装配门",
|
||||
"estimated_real_calls": {
|
||||
"preflight": 0,
|
||||
"adds": 5,
|
||||
"model_use_qa": 8,
|
||||
"total": 13,
|
||||
"note": "13 次 ≤ 余下 34 次；失败重试逐次扣账，连续 3 次真实失败即停"
|
||||
},
|
||||
"not_run_this_round": [
|
||||
"S2/S3/S4/S6 模型实际使用问答（真实模型部分）",
|
||||
"S1 mem0 侧、S7 mem0 侧正例（需真实提取与锚点写入后复验）",
|
||||
"S8/S9 真实进程重启与备份恢复（run2 证据保留，修复不影响其机制，按方案一并复验）"
|
||||
]
|
||||
},
|
||||
"_checks": {
|
||||
"R1_baseline_refixed.fixture_integrity_refixed": true,
|
||||
"R1_baseline_refixed.legacy_corruption_now_caught": true,
|
||||
"R1_baseline_refixed.S1_baseline_refixed": true,
|
||||
"R1_baseline_refixed.S2_baseline_refixed": true,
|
||||
"R1_baseline_refixed.S3_baseline_refixed": true,
|
||||
"R1_baseline_refixed.S4_baseline_refixed": true,
|
||||
"R1_baseline_refixed.S5_baseline_refixed": true,
|
||||
"R1_baseline_refixed.S7_baseline_refixed": true,
|
||||
"R3R4_orchestration_chain.S4_legacy_text_selector_demo": true,
|
||||
"R3R4_orchestration_chain.S4_known_id_orchestration": true,
|
||||
"R3R4_orchestration_chain.S6_legacy_text_selector_demo": true,
|
||||
"R3R4_orchestration_chain.S6_known_id_orchestration": true,
|
||||
"R3R4_orchestration_chain.S7_chain_positive": true,
|
||||
"R3R4_orchestration_chain.S7_chain_negatives": true,
|
||||
"R3R4_orchestration_chain.S7_gated_assembly": true,
|
||||
"R2_gate2_full_all_pass": true,
|
||||
"R5_gate0_fake_transport": true,
|
||||
"R5_resource_account": true
|
||||
},
|
||||
"_all_pass": true,
|
||||
"_failed": [],
|
||||
"_zero_network": true
|
||||
}
|
||||
@@ -0,0 +1,460 @@
|
||||
#!/usr/bin/env python3
|
||||
"""HEL-858 返工离线复核套件：零真实模型、零网络出站。
|
||||
|
||||
对应《HEL-857-独立复核与返工要求.md》五项必修的离线验证与重判：
|
||||
R1 夹具装载：修复后的 (turn, role) 完整装载 + 逐字逐角色核对；并构造旧缺陷
|
||||
形态的故意损坏库，证明新核对门能抓住 run2 的装载错误（S1 不再放行）。
|
||||
R1b 现状侧重判：S1/S2/S3/S4/S5/S7 的现状路径在修复夹具上离线重跑
|
||||
（mem0 侧沿用 run2 真实证据并标注；模型使用类判定标未运行）。
|
||||
R2 分享桥：spoken_text 绑定 + 部分分享/缺失内容负例 + 恢复后全套
|
||||
P1-P5/F1-F8 复跑（stub_gate.run，stub LLM + 本地嵌入，零网络）。
|
||||
R3/R4 编排与链：用 run2 实证提取文本（含"新条目也含 Wednesday"的毒例）
|
||||
在 stub 实例上离线复放，验证已知 id 选条、prev→new 回读、更新后重新检索、
|
||||
命中→payload 锚点→原文链、错/未知来源拒绝、同轮未分享伴侣消息拒绝、
|
||||
600 字装配门；并演示旧文本选条法会选错对象（负对照）。
|
||||
R5 资源账：历史用量按证据保守重建（记录值/派生值/枚举上界分列），补种卡级
|
||||
持久嵌入账本；本套件自身用量持久扣账并汇报；G0 假传输复跑。
|
||||
|
||||
真实 A/B 未复跑的部分在 retest_plan 中列出（固定资料、判据、预计请求数），
|
||||
等总管核对后再按主卡批准边界处理。本套件退出码依赖逐项实际结果。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from . import stub_gate
|
||||
from .ab_scenarios import (ADD_MANIFEST, Q_CORRECTION, Q_EXACT, Q_FACT,
|
||||
Q_FEELING, Q_PARAPHRASE, Q_PARAPHRASE_TOPIC,
|
||||
Q_PROMISE, CompanionShareGate, build_baseline_db,
|
||||
fixture_integrity, baseline_fixture_rows,
|
||||
group_input_sha, mem0_search_texts,
|
||||
normalize_add_result, payload_of,
|
||||
s4_correction_orchestration, s6_promise_orchestration,
|
||||
s7_chain_resolve, s7_gated_assembly)
|
||||
from .common import (DATA_DIR, EMBED_LEDGER, PUB, close_memory,
|
||||
install_counters, mem_config, write_result)
|
||||
from . import fixtures as FX
|
||||
from .ledger_fake_test import run as ledger_fake_run
|
||||
|
||||
REWORK = DATA_DIR / "rework858"
|
||||
RESULTS = Path(__file__).parent / "results"
|
||||
|
||||
# run2 实证提取文本（e31ae2d，results/ab_scenarios.json）——毒例复放数据：
|
||||
# 新条目（c2 派生）同样含 "Wednesday"，run2 的"含周三"文本选条法因此更新了新条目。
|
||||
RUN2_OLD_ARRANGEMENT = ("User plans to go to the hospital for a physical "
|
||||
"examination next Wednesday, around November 4, 2026.")
|
||||
RUN2_NEW_ARRANGEMENT = ("User changed their hospital physical examination from "
|
||||
"next Wednesday to Friday morning, around October 30, "
|
||||
"2026, and emphasized not to record it wrong.")
|
||||
RUN2_PROMISE = ("User said they would tell the assistant the interview result "
|
||||
"on Friday evening.")
|
||||
RUN2_RESULT = ("User passed the important interview and received the "
|
||||
"notification, and said the assistant was the first person "
|
||||
"they wanted to tell.")
|
||||
|
||||
# 历史嵌入用量重建（证据与方法分列；详见 resource account）
|
||||
HIST_EMBED = {
|
||||
"run1": {"dense_recorded": 23, "child_add_derived": 1,
|
||||
"method": "results/ab_scenarios_run1.json _counters 记录值 + ab_child.py 单条 infer=False add 派生"},
|
||||
"run2": {"dense_recorded": 28, "child_add_derived": 1,
|
||||
"method": "results/ab_scenarios.json _counters 记录值 + ab_child.py 派生（进程内计数不含子进程）"},
|
||||
"gate1": {"add_path_points_exact": 4, "query_enumerated": 2,
|
||||
"method": "向量点数 4（qdrant 本地只读清点）+ 代码枚举：Q0b 语义检索 1 + 子进程检索 1"},
|
||||
"gate2": {"add_path_points_exact": 9, "update_exact": 1, "query_bound": 31,
|
||||
"method": "向量点数 9（src 两侧实例）+ P3 update 1 + 查询/阶段路径代码枚举上界 31"},
|
||||
}
|
||||
HIST_DENSE_TOTAL = 100 # 保守上界合计（24+29+6+41），未编造精确值
|
||||
HIST_KEYWORD = {"run1_recorded": 10, "run2_recorded": 10, "gates_bound": 20}
|
||||
|
||||
|
||||
def make_stub_memory(root: Path):
|
||||
from mem0 import Memory
|
||||
m = Memory.from_config(mem_config(PUB, vec_root=root, hist_root=root))
|
||||
install_counters(m)
|
||||
return m
|
||||
|
||||
|
||||
def legacy_corrupted_build(path: Path) -> None:
|
||||
"""复现 run2 装载缺陷形态：turn 单键字典导致伴侣回复覆盖用户原话、
|
||||
整轮写成 user。仅用于证明修复后的核对门能抓住它。"""
|
||||
import probe_recall as PR
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
conn = sqlite3.connect(path)
|
||||
conn.executescript(PR.SCHEMA)
|
||||
texts = {t: r for t, _, r in FX.PUBLIC_CORPUS} # 旧缺陷：单键覆盖
|
||||
texts.update({t: r for t, _, r in FX.CORRECTION_PAIR})
|
||||
texts.update({t: r for t, _, r in FX.FACT_FEELING_PAIR})
|
||||
texts.update({t: r for t, _, r in FX.PROMISE_PAIR})
|
||||
hours = {"t1": 24, "t2": 23, "t3": 12, "t4": 6, "t5": 2,
|
||||
"c1": 5, "c2": 4, "f1": 5, "f2": 4, "p1": 8, "p2": 1}
|
||||
for turn, h in hours.items():
|
||||
ts = int((PR.NOW - h * PR.HOUR) * 1000)
|
||||
cur = conn.execute(
|
||||
"INSERT INTO chat_message(ts,role,text,turn_id,status) VALUES(?,?,?,?,'sent')",
|
||||
(ts, "user", texts[turn], turn))
|
||||
mid = cur.lastrowid
|
||||
conn.execute(
|
||||
"INSERT INTO mem_episodic(ts,role,raw,gist,salience,source,turn_id,message_id) "
|
||||
"VALUES(?,?,?,NULL,0.5,'chat',?,?)", (ts, "user", texts[turn], turn, mid))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
def part1_baseline_refixed() -> dict:
|
||||
"""必修1：夹具修复 + 现状侧离线重判（零网络，仅 SQLite）。"""
|
||||
out: dict = {}
|
||||
REWORK.mkdir(parents=True, exist_ok=True)
|
||||
db = REWORK / "baseline_refixed.sqlite3"
|
||||
build_baseline_db(db)
|
||||
integ = fixture_integrity(db)
|
||||
out["fixture_integrity_refixed"] = integ
|
||||
|
||||
# 负对照：旧缺陷形态必须被新核对门抓住
|
||||
bad = REWORK / "baseline_legacy_corrupted.sqlite3"
|
||||
legacy_corrupted_build(bad)
|
||||
integ_bad = fixture_integrity(bad)
|
||||
out["legacy_corruption_now_caught"] = {
|
||||
"verbatim_all_roles": integ_bad["verbatim_all_roles"],
|
||||
"companion_rows_lost": [k for k in integ_bad["missing"] if k[1] == "companion"],
|
||||
"role_crossed": integ_bad["role_crossed"],
|
||||
"pass": (not integ_bad["pass"])
|
||||
and any(k[1] == "companion" for k in integ_bad["missing"])
|
||||
and bool(integ_bad["role_crossed"]),
|
||||
"expect": "run2 装载错误形态（伴侣文本占 user 位、伴侣行丢失）被核对门判失败",
|
||||
}
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "docs" / "reuse-hel847"))
|
||||
import probe_recall as PR
|
||||
conn = sqlite3.connect(db)
|
||||
conn.row_factory = sqlite3.Row
|
||||
base = PR.ProdEpisodic(conn)
|
||||
|
||||
# S1 现状侧：t1 用户原句逐字在库（严格判据）
|
||||
raw_t1 = FX.PUBLIC_CORPUS[0][2]
|
||||
row = conn.execute("SELECT raw FROM mem_episodic WHERE turn_id='t1' AND role='user'").fetchone()
|
||||
comp = conn.execute("SELECT text FROM chat_message WHERE turn_id='t1' AND role='companion'").fetchone()
|
||||
out["S1_baseline_refixed"] = {
|
||||
"user_raw_verbatim": bool(row and row["raw"] == raw_t1),
|
||||
"companion_row_present_with_own_text": bool(comp and comp["text"] == FX.PUBLIC_CORPUS[1][2]),
|
||||
"pass": bool(row and row["raw"] == raw_t1) and bool(comp and comp["text"] == FX.PUBLIC_CORPUS[1][2]),
|
||||
"expect": "两侧同一资料逐字逐角色：t1 用户原句与伴侣回复各自在库（run2 为 false）",
|
||||
}
|
||||
|
||||
# S2 现状侧：命中/细节进上下文/回读（细节按等价集语义判定）
|
||||
b_hits, _ = base.search("旧书店")
|
||||
b_block, _, _ = base.context_block("旧书店", turn_id="q_s2r")
|
||||
rr = base.reread_turn(b_hits[0]) if b_hits else {"messages": []}
|
||||
rr_text = " ".join(m["text"] for m in rr["messages"])
|
||||
out["S2_baseline_refixed"] = {
|
||||
"hit": bool(b_hits),
|
||||
"detail_in_context": FX.detail_equiv_hit(b_block),
|
||||
"detail_in_reread": FX.detail_equiv_hit(rr_text),
|
||||
"reread_roles": [m["role"] for m in rr["messages"]],
|
||||
"run2_recorded_for_contrast": {"detail_in_reread": False,
|
||||
"note": "run2 夹具被覆盖导致假阴性；修复后现状回读可恢复细节"},
|
||||
"pass": bool(b_hits) and FX.detail_equiv_hit(rr_text),
|
||||
"expect": "修复夹具后现状：检索命中、gist 压缩丢细节、带门回读恢复细节（装载错误不再记作产品缺点）",
|
||||
}
|
||||
|
||||
# S3 现状侧：换说法词面命中重验（修复夹具）
|
||||
b3a, mode_a = base.search(Q_PARAPHRASE)
|
||||
b3b, mode_b = base.search(Q_PARAPHRASE_TOPIC)
|
||||
out["S3_baseline_refixed"] = {
|
||||
"paraphrase_query": {"hits": len(b3a), "mode": mode_a},
|
||||
"paraphrase_topic_query": {"hits": len(b3b), "mode": mode_b},
|
||||
"run2_mem0_evidence_retained": {
|
||||
"paraphrase_hits": 11, "topic_hits": 11, "answer": "蓝色",
|
||||
"note": "mem0 侧 run2 真实证据保留（其输入未被夹具缺陷影响）；配对模型使用判定未运行"},
|
||||
"pass": len(b3a) == 0 and len(b3b) == 0,
|
||||
"expect": "修复夹具后现状词面对换说法仍无命中（重验，非沿用损坏夹具结论）",
|
||||
}
|
||||
|
||||
# S4/S5 现状侧：关键词形态不退化
|
||||
b_kw, _ = base.search("体检")
|
||||
turns = sorted({h["turn_id"] for h in b_kw})
|
||||
out["S4_baseline_refixed"] = {
|
||||
"keyword_hits": turns,
|
||||
"pass": all(t in turns for t in ("c1", "c2")),
|
||||
"expect": "体检关键词命中 c1 与 c2（不退化）",
|
||||
}
|
||||
b_fact, _ = base.search("面试")
|
||||
b_feel, _ = base.search("紧张")
|
||||
out["S5_baseline_refixed"] = {
|
||||
"fact_kw_hits": sorted({h["turn_id"] for h in b_fact}),
|
||||
"feeling_kw_hits": sorted({h["turn_id"] for h in b_feel}),
|
||||
"pass": any(h["turn_id"] == "f1" for h in b_fact) and any(h["turn_id"] == "f2" for h in b_feel),
|
||||
"expect": "事实/感受关键词各自命中（不退化）",
|
||||
}
|
||||
|
||||
# S7 现状侧：命中行带门回读恢复自身原文（链）
|
||||
b7, _ = base.search("诗集")
|
||||
r7 = base.reread_turn(b7[0]) if b7 else {"messages": []}
|
||||
t7 = " ".join(m["text"] for m in r7["messages"])
|
||||
out["S7_baseline_refixed"] = {
|
||||
"hit_turn": b7[0]["turn_id"] if b7 else None,
|
||||
"reread_recovered_own_raw": bool(b7) and (b7[0]["raw"] in t7),
|
||||
"pass": bool(b7) and (b7[0]["raw"] in t7),
|
||||
"expect": "现状命中行经带门回读恢复该行原文",
|
||||
}
|
||||
conn.close()
|
||||
return out
|
||||
|
||||
|
||||
def part2_gate2_full() -> dict:
|
||||
"""必修2：修复后的门禁2全套（含部分分享/缺失内容负例与恢复后全套复跑）。"""
|
||||
return stub_gate.run()
|
||||
|
||||
|
||||
def part3_orchestration_chain() -> dict:
|
||||
"""必修3/4：已知id编排与来源链离线复放（stub 实例，run2 毒例数据）。"""
|
||||
import shutil
|
||||
out: dict = {}
|
||||
root = REWORK / "orch"
|
||||
if root.exists():
|
||||
shutil.rmtree(root) # 复放实例每次全新，跨运行不残留
|
||||
root.mkdir(parents=True)
|
||||
m = make_stub_memory(root)
|
||||
|
||||
def anchored_add(text: str, group: str, turns: list[str], role: str = "user",
|
||||
input_sha: str | None = None):
|
||||
r = m.add([{"role": role, "content": text}], user_id="user-1",
|
||||
agent_id="cg-companion", infer=False,
|
||||
metadata={"src_group": group, "turn_ids": turns,
|
||||
"input_sha256": input_sha or group_input_sha(group)})
|
||||
items = normalize_add_result(r)
|
||||
return items[0]["id"] if items else ""
|
||||
|
||||
# --- S4 毒例复放：run2 的旧文本选条法会选中新条目 ---
|
||||
old_id = anchored_add(RUN2_OLD_ARRANGEMENT, "correction_first", ["c1"])
|
||||
new_id = anchored_add(RUN2_NEW_ARRANGEMENT, "correction_later", ["c2"])
|
||||
hits = mem0_search_texts(m, Q_CORRECTION, limit=10)
|
||||
both_wed = [h for h in hits if "Wednesday" in (h["memory"] or "")
|
||||
or "周三" in (h["memory"] or "")]
|
||||
text_selector_target = both_wed[0]["id"] if both_wed else None
|
||||
out["S4_legacy_text_selector_demo"] = {
|
||||
"hits_containing_wednesday": len(both_wed),
|
||||
"legacy_selector_picked": text_selector_target,
|
||||
"picked_was_old_entry": text_selector_target == old_id,
|
||||
"run2_evidence": "run2 真实实例中该选条法实际更新了 c2 派生新条目（evidence: correction_update_target）",
|
||||
"pass": len(both_wed) >= 2 and text_selector_target != old_id,
|
||||
"expect": "负对照：'含周三'式文本选条在同一标记≥2条候选时不可靠，选中者不保证是"
|
||||
"旧条目（run2 实证选中新条目；本离线复放中选中亦非旧条目）——须用已知id",
|
||||
}
|
||||
s4 = s4_correction_orchestration(m, root / "mem0_public_history.db", old_id, new_id)
|
||||
out["S4_known_id_orchestration"] = s4
|
||||
|
||||
# --- S6 毒例复放：'result/tell' 文本选条法会选中面试结果条目 ---
|
||||
promise_id = anchored_add(RUN2_PROMISE, "promise", ["p1"])
|
||||
result_id = anchored_add(RUN2_RESULT, "promise", ["p2"])
|
||||
hits6 = mem0_search_texts(m, Q_PROMISE, limit=10)
|
||||
legacy6 = next((h for h in hits6
|
||||
if any(t in (h["memory"] or "").lower() for t in ("result", "tell"))),
|
||||
hits6[0] if hits6 else None)
|
||||
out["S6_legacy_text_selector_demo"] = {
|
||||
"legacy_selector_picked": legacy6["id"] if legacy6 else None,
|
||||
"picked_was_result_entry": bool(legacy6) and legacy6["id"] == result_id,
|
||||
"run2_evidence": "run2 实际把 resolved 置到了检索首位（面试结果条目）",
|
||||
"pass": bool(legacy6) and legacy6["id"] != promise_id,
|
||||
"expect": "负对照：'result/tell'式文本选条可选中结果条目而非原承诺（run2 实证形态）",
|
||||
}
|
||||
s6 = s6_promise_orchestration(m, root / "mem0_public_history.db", promise_id, result_id)
|
||||
out["S6_known_id_orchestration"] = s6
|
||||
|
||||
# --- S7 链：命中 → history 推导史 + payload 锚点 → 原文；负例与装配门 ---
|
||||
# 公开基底语料按原句入库（t1 用户+伴侣同轮，锚点指向 t1；其余各轮用户句）
|
||||
t1_raws = [raw for t, r, raw in FX.PUBLIC_CORPUS if t == "t1"]
|
||||
for raw in t1_raws:
|
||||
anchored_add(raw, "base", ["t1"])
|
||||
for turn, role, raw in FX.PUBLIC_CORPUS[2:5]:
|
||||
if role == "user":
|
||||
anchored_add(raw, "base", [turn])
|
||||
hits7 = mem0_search_texts(m, Q_EXACT, limit=10)
|
||||
# 目标命中：t1 用户原句条目（链恢复判据的对象）
|
||||
hit7 = next((h for h in hits7 if (h["memory"] or "") == FX.PUBLIC_CORPUS[0][2]), None)
|
||||
chain = s7_chain_resolve(m, hit7) if hit7 else {"resolved": False, "reject_reason": "no_hit"}
|
||||
with sqlite3.connect(str(root / "mem0_public_history.db")) as c:
|
||||
deriv = [dict(zip(("old", "new", "event"), r)) for r in c.execute(
|
||||
"SELECT old_memory,new_memory,event FROM history WHERE memory_id=? "
|
||||
"ORDER BY created_at", (hit7["id"],)).fetchall()] if hit7 else []
|
||||
deriv_ok = bool(hit7) and any(d["event"] == "ADD" and d["new"] == hit7["memory"]
|
||||
for d in deriv)
|
||||
target = FX.PUBLIC_CORPUS[0][2]
|
||||
original_ok = chain.get("resolved") and any(
|
||||
o["raw"] == target for o in chain["originals"] if o["role"] == "user")
|
||||
out["S7_chain_positive"] = {
|
||||
"hit_id": hit7["id"] if hit7 else None,
|
||||
"hit_history_derivation_matches": deriv_ok,
|
||||
"chain_resolved": chain.get("resolved"),
|
||||
"chain_src_group": chain.get("src_group"),
|
||||
"original_turns": sorted({o["turn"] for o in chain.get("originals") or []}),
|
||||
"chain_recovered_user_original": original_ok,
|
||||
"pass": bool(deriv_ok and original_ok),
|
||||
"expect": "命中 id → history 推导史吻合 → payload 锚点 → 输入清单恢复 t1 用户原句",
|
||||
}
|
||||
|
||||
# 负例1：未知来源（锚点指向不存在的轮次）
|
||||
unknown_id = anchored_add("来历不明的记忆条目", "mystery", ["tz_unknown"])
|
||||
chain_unknown = s7_chain_resolve(m, {"id": unknown_id, "memory": "来历不明的记忆条目"})
|
||||
# 负例2：无锚点（不带 metadata 写入）
|
||||
no_anchor = m.add([{"role": "user", "content": "没有来源锚点的条目"}],
|
||||
user_id="user-1", agent_id="cg-companion", infer=False)
|
||||
no_anchor_id = normalize_add_result(no_anchor)[0]["id"]
|
||||
chain_noanchor = s7_chain_resolve(m, {"id": no_anchor_id, "memory": "没有来源锚点的条目"})
|
||||
# 负例3：错来源（锚点组标签与输入指纹不符——换错来源/篡改锚点被检出）
|
||||
import hashlib as _hl
|
||||
wrong_id = anchored_add(
|
||||
"错挂来源的条目", "base", ["t1"],
|
||||
input_sha=_hl.sha256("错挂来源的条目".encode()).hexdigest()) # 指纹非该组输入
|
||||
chain_wrong = s7_chain_resolve(m, {"id": wrong_id, "memory": "错挂来源的条目"})
|
||||
# 负例4：同轮未分享伴侣消息（控制库判 told=0）
|
||||
ctrl = REWORK / "share_gate.sqlite3"
|
||||
with sqlite3.connect(str(ctrl)) as c:
|
||||
c.execute("CREATE TABLE IF NOT EXISTS companion_share (turn_id TEXT PRIMARY KEY, told INTEGER)")
|
||||
c.execute("INSERT OR REPLACE INTO companion_share VALUES('t1', 0)")
|
||||
gate = CompanionShareGate(ctrl)
|
||||
comp_turn = "t1"
|
||||
chain_comp = {"resolved": False, "reject_reason": "companion_unshared"}
|
||||
manifest = baseline_fixture_rows()
|
||||
comp_rows = [{"turn": comp_turn, "role": "companion", "raw": manifest[(comp_turn, "companion")]}]
|
||||
chain_comp = {
|
||||
"resolved": any(True for o in comp_rows if gate.companion_allowed(comp_turn)),
|
||||
"companion_allowed": gate.companion_allowed(comp_turn),
|
||||
}
|
||||
out["S7_chain_negatives"] = {
|
||||
"unknown_turn_rejected": chain_unknown.get("reject_reason") in
|
||||
("unknown_source", "no_shareable_rows"),
|
||||
"no_anchor_rejected": chain_noanchor.get("reject_reason") == "unknown_source",
|
||||
"wrong_source_rejected": chain_wrong.get("reject_reason") == "source_mismatch",
|
||||
"unshared_companion_blocked": not chain_comp["resolved"],
|
||||
"detail": {"unknown": chain_unknown, "no_anchor": chain_noanchor,
|
||||
"wrong_source": chain_wrong,
|
||||
"unshared_companion": chain_comp},
|
||||
"pass": (chain_unknown.get("reject_reason") in ("unknown_source", "no_shareable_rows")
|
||||
and chain_noanchor.get("reject_reason") == "unknown_source"
|
||||
and chain_wrong.get("reject_reason") == "source_mismatch"
|
||||
and not chain_comp["resolved"]),
|
||||
"expect": "未知来源/无锚点/输入指纹不符（换错来源）均拒绝；同轮未分享伴侣消息"
|
||||
"（told=0）不得进入回读/装配",
|
||||
}
|
||||
|
||||
# 装配门：仅链验证条目入块，≤600 字，零私密标记
|
||||
gated = s7_gated_assembly(m, hits7 + [
|
||||
{"id": unknown_id, "memory": "来历不明的记忆条目"},
|
||||
{"id": no_anchor_id, "memory": "没有来源锚点的条目"},
|
||||
{"id": wrong_id, "memory": "错挂来源的条目"}])
|
||||
out["S7_gated_assembly"] = {
|
||||
"used": len(gated["used"]), "rejected": gated["rejected"],
|
||||
"leak_markers": gated["leak_markers"], "block_len": len(gated["block"]),
|
||||
"within_budget": gated["within_budget"],
|
||||
"pass": gated["pass"] and len(gated["rejected"]) >= 3,
|
||||
"expect": "600 字装配仅纳入链验证条目；未知/无锚点/错来源条目被拒；零私密标记",
|
||||
}
|
||||
|
||||
close_memory(m)
|
||||
return out
|
||||
|
||||
|
||||
def part5_resource_account(suite_embed_rows: list[dict]) -> dict:
|
||||
"""必修5：卡级资源账（历史保守重建 + 本套件持久扣账）。"""
|
||||
hist_dense = sum(v.get("dense_recorded", 0) + v.get("child_add_derived", 0)
|
||||
for k, v in HIST_EMBED.items() if k in ("run1", "run2"))
|
||||
hist_dense += HIST_EMBED["gate1"]["add_path_points_exact"] + HIST_EMBED["gate1"]["query_enumerated"]
|
||||
hist_dense += (HIST_EMBED["gate2"]["add_path_points_exact"]
|
||||
+ HIST_EMBED["gate2"]["update_exact"] + HIST_EMBED["gate2"]["query_bound"])
|
||||
led = EMBED_LEDGER.summary()
|
||||
return {
|
||||
"history_reconstruction": HIST_EMBED,
|
||||
"history_dense_conservative_total": HIST_DENSE_TOTAL,
|
||||
"history_method_note": (
|
||||
"精确不可恢复处标注上界与方法，不编数字、不重置：run1/run2 为进程内记录值"
|
||||
"+子进程派生；门禁按向量点数（旧数据目录只读清点：gate1=4、gate2 src=3+6、"
|
||||
"run2=12）与代码枚举上界重建。旧口径'全卡 28/500'作废（仅为 run2 进程内计数）。"),
|
||||
"keyword_separate": HIST_KEYWORD,
|
||||
"card_embed_ledger": led,
|
||||
"suite_debits_this_run": suite_embed_rows,
|
||||
"cap": 500,
|
||||
"within_cap": led["embed_items_total"] <= 500,
|
||||
"pass": hist_dense <= HIST_DENSE_TOTAL and led["embed_items_total"] <= 500,
|
||||
}
|
||||
|
||||
|
||||
def retest_plan() -> dict:
|
||||
"""真实 A/B 复测方案（本轮零真实出站，未运行项交总管核对）。"""
|
||||
return {
|
||||
"precondition": "余下 34 次真实额度冻结不动；本方案仅供总管核对，未获批准前不执行",
|
||||
"fixed_materials": "fixtures.py 固定合成语料不变；add 随 payload 写入来源锚点（src_group/turn_ids）",
|
||||
"criteria": ("修复后判据：S1 夹具逐字逐角色+两侧原文；S2 细节等价集语义判定+模型使用分列；"
|
||||
"S3 换说法配对重跑（现状修复夹具）；S4/S6 已知id编排+prev→new回读+更新后重新"
|
||||
"检索装配再问答；S7 命中→history+锚点→原文链+负例+600字装配门"),
|
||||
"estimated_real_calls": {
|
||||
"preflight": 0, "adds": 5, "model_use_qa": 8, "total": 13,
|
||||
"note": "13 次 ≤ 余下 34 次；失败重试逐次扣账，连续 3 次真实失败即停",
|
||||
},
|
||||
"not_run_this_round": [
|
||||
"S2/S3/S4/S6 模型实际使用问答（真实模型部分）",
|
||||
"S1 mem0 侧、S7 mem0 侧正例（需真实提取与锚点写入后复验）",
|
||||
"S8/S9 真实进程重启与备份恢复（run2 证据保留，修复不影响其机制，按方案一并复验）",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def run() -> dict:
|
||||
results: dict = {}
|
||||
# 先补种历史用量（幂等：仅当账本为空时）
|
||||
if EMBED_LEDGER.used_embed_items() == 0 and EMBED_LEDGER.used_by_kind().get("keyword", 0) == 0:
|
||||
EMBED_LEDGER.debit("seed_dense", HIST_DENSE_TOTAL,
|
||||
note="HEL-857 历史用量保守重建（方法见 resource account）")
|
||||
EMBED_LEDGER.debit("seed_keyword", HIST_KEYWORD["run1_recorded"]
|
||||
+ HIST_KEYWORD["run2_recorded"] + HIST_KEYWORD["gates_bound"],
|
||||
note="HEL-857 历史词面处理重建（另列，不占 500 上限）")
|
||||
seed_snapshot = EMBED_LEDGER.summary()
|
||||
|
||||
results["R1_baseline_refixed"] = part1_baseline_refixed()
|
||||
results["R2_gate2_full"] = part2_gate2_full()
|
||||
results["R3R4_orchestration_chain"] = part3_orchestration_chain()
|
||||
|
||||
# G0 假传输复跑（总管已独立复跑通过；此处随套件一并复现）
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
results["R5_gate0_fake_transport"] = {"ok": ledger_fake_run(Path(td))["ok"]}
|
||||
|
||||
led_after = EMBED_LEDGER.summary()
|
||||
suite_used = led_after["embed_items_total"] - seed_snapshot["embed_items_total"]
|
||||
results["R5_resource_account"] = part5_resource_account(
|
||||
{"suite_dense_used": suite_used,
|
||||
"keyword_used": led_after["keyword"] - seed_snapshot["keyword"]})
|
||||
results["_retest_plan"] = retest_plan()
|
||||
|
||||
def _flat_pass(d, prefix=""):
|
||||
for k, v in d.items():
|
||||
if isinstance(v, dict):
|
||||
if "pass" in v:
|
||||
yield f"{prefix}{k}", v["pass"]
|
||||
else:
|
||||
yield from _flat_pass(v, f"{prefix}{k}.")
|
||||
checks = dict(_flat_pass({k: v for k, v in results.items()
|
||||
if k in ("R1_baseline_refixed", "R3R4_orchestration_chain")}))
|
||||
checks["R2_gate2_full_all_pass"] = results["R2_gate2_full"].get("ok", False)
|
||||
checks["R5_gate0_fake_transport"] = results["R5_gate0_fake_transport"]["ok"]
|
||||
checks["R5_resource_account"] = results["R5_resource_account"]["pass"]
|
||||
results["_checks"] = checks
|
||||
results["_all_pass"] = all(checks.values())
|
||||
results["_failed"] = [k for k, v in checks.items() if not v]
|
||||
results["_zero_network"] = True
|
||||
return results
|
||||
|
||||
|
||||
def main(argv: list[str]) -> dict:
|
||||
out = run()
|
||||
write_result("rework858_offline.json", out)
|
||||
return out
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(0 if main([])["_all_pass"] else 1)
|
||||
+227
-58
@@ -1,15 +1,19 @@
|
||||
#!/usr/bin/env python3
|
||||
"""门禁 2：隐私前置隔离 + 首次分享时序 + 完整备份恢复（stub LLM，零真实模型、零网络）。
|
||||
|
||||
对齐批准范围与总管核对版两项执行限定：
|
||||
对齐批准范围与总管核对版两项执行限定（HEL-858 返工补强）：
|
||||
P1 提取输入池——公开实例 B 的 add(infer=True) 提示词不得含 A 的任何私密条目/标记；
|
||||
P2 提取历史窗——B 的提示词不得含 A 的 history 原始输入（分库隔离）；
|
||||
P3 更新路径——对 B 显式 update() 后 A 逐字节不变；
|
||||
P4 回读与预算装配——公开回读只走 B+我方门，600 字预算装配零泄漏，A 条目数/内容不变；
|
||||
P4 回读与预算装配——公开回读只走 B+我方门，600 字预算装配零泄漏，A 条目数/内容不变，
|
||||
公开库迁移锚点全部在 telling_log 有对应迁入记录；
|
||||
P5 完整备份——包含私密库、公开库、双方 history、锚点与配置（不止"公开检索数据包"）；
|
||||
清洁关闭复制 → 新实例恢复 → P1-P4 复跑全过，内部未分享经历仍在，公开出口仍不泄漏；
|
||||
F1-F6 首次分享时序——失败/取消/未确认/判定失败不迁公开库；完整回执确认+分享判定完成后
|
||||
才按实际已讲内容迁入；重复回执不重复迁入；迁入后 A 仍保留内部副本。
|
||||
清洁关闭复制 → 新实例恢复 → P1-P4 与 F1-F8 全套复跑通过，内部未分享经历仍在，
|
||||
公开出口仍不泄漏，恢复前 telling_log 在恢复库上仍幂等；
|
||||
F1-F8 首次分享时序——失败/取消/未确认/判定失败不迁公开库；已讲文本缺失不得整段
|
||||
降级放行（拒绝并审计）；完整回执确认+分享判定完成后只按【实际已讲文本】迁入
|
||||
（部分分享负例：内部事件含从未讲出的秘密时，公开写入严格限于已讲部分）；
|
||||
重复回执不重复迁入；迁入后 A 仍保留内部副本。
|
||||
|
||||
stub：直接替换 OpenAILLM.generate_response 实例方法，捕获全部提示词全文；
|
||||
返回值由测试预设（确定性队列），零网络。
|
||||
@@ -64,26 +68,34 @@ def make_memory(role: str, root: Path):
|
||||
|
||||
|
||||
# ---- 我方分享桥（生产 life/public.py 语义重放：sharing_attempt/telling_log/life_event） ----
|
||||
# HEL-858 返工：attempt 绑定实际收到的回复文本（spoken_text）——迁入公开库的
|
||||
# 内容严格限于实际已讲部分；未知/缺失已讲内容不得整段降级放行（记拒绝审计）。
|
||||
CTRL_SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS life_event (
|
||||
id INTEGER PRIMARY KEY, content TEXT NOT NULL, told INTEGER NOT NULL DEFAULT 0);
|
||||
CREATE TABLE IF NOT EXISTS sharing_attempt (
|
||||
message_id INTEGER PRIMARY KEY, event_id INTEGER NOT NULL, epoch INTEGER NOT NULL,
|
||||
state TEXT NOT NULL CHECK(state IN ('pending','evaluating','judged','failed','cancelled')),
|
||||
ack_at INTEGER, judged_at INTEGER);
|
||||
ack_at INTEGER, judged_at INTEGER, spoken_text TEXT);
|
||||
CREATE TABLE IF NOT EXISTS telling_log (
|
||||
event_id INTEGER NOT NULL, message_id INTEGER NOT NULL, told_at INTEGER NOT NULL,
|
||||
PRIMARY KEY(event_id, message_id));
|
||||
CREATE TABLE IF NOT EXISTS migration_refusal (
|
||||
message_id INTEGER NOT NULL, event_id INTEGER NOT NULL, reason TEXT NOT NULL,
|
||||
refused_at INTEGER NOT NULL, PRIMARY KEY(message_id, event_id, reason));
|
||||
"""
|
||||
|
||||
|
||||
class ShareBridge:
|
||||
"""内部经历→公开库的受控桥接（归责：我方编排；上游仅提供 add 原语）。
|
||||
|
||||
时序（总管核对版执行限定1）：
|
||||
时序（总管核对版执行限定1，HEL-858 补强）：
|
||||
分享尝试 pending→judged/failed/cancelled；完整回复回执 ack（ack_at 非空）
|
||||
且判定为已讲（told=1）之后，才按实际已讲内容显式 add 到公开实例 B，
|
||||
metadata 携带锚点；telling_log 保证幂等（重复回执不重复迁入）。
|
||||
且判定为已讲（told=1）之后，且存在本次实际讲出的文本（spoken_text，
|
||||
来自实际收到的伴侣回复），才把【实际已讲文本】（非内部事件全文）显式
|
||||
add 到公开实例 B，metadata 携带锚点；telling_log 保证幂等（重复回执不
|
||||
重复迁入）。spoken_text 缺失/为空 → 拒绝迁入并留 migration_refusal 审计，
|
||||
绝不回退为整段事件放行。
|
||||
"""
|
||||
|
||||
def __init__(self, ctrl_db: Path):
|
||||
@@ -97,21 +109,25 @@ class ShareBridge:
|
||||
(event_id, content, told))
|
||||
|
||||
def attempt(self, message_id: int, event_id: int, *, state: str, ack: bool,
|
||||
epoch: int = 1) -> None:
|
||||
epoch: int = 1, spoken_text: str | None = None) -> None:
|
||||
with sqlite3.connect(str(self.db_path)) as c:
|
||||
c.execute(
|
||||
"INSERT OR REPLACE INTO sharing_attempt"
|
||||
"(message_id,event_id,epoch,state,ack_at,judged_at) VALUES(?,?,?,?,?,?)",
|
||||
"(message_id,event_id,epoch,state,ack_at,judged_at,spoken_text) "
|
||||
"VALUES(?,?,?,?,?,?,?)",
|
||||
(message_id, event_id, epoch, state,
|
||||
1000 if ack else None, 1001 if state in ("judged", "failed") else None))
|
||||
1000 if ack else None, 1001 if state in ("judged", "failed") else None,
|
||||
spoken_text))
|
||||
if state == "judged" and ack:
|
||||
c.execute("UPDATE life_event SET told=1 WHERE id=?", (event_id,))
|
||||
|
||||
def maybe_migrate(self, public_mem, message_id: int, event_id: int) -> bool:
|
||||
"""回执+判定完成后调用。仅当 judged & ack & told 才迁；幂等。"""
|
||||
"""回执+判定完成后调用。仅当 judged & ack & told & 已讲文本非空才迁；幂等；
|
||||
已讲文本缺失 → 拒绝并审计，不降级放行整段事件。"""
|
||||
with sqlite3.connect(str(self.db_path)) as c:
|
||||
att = c.execute(
|
||||
"SELECT state, ack_at FROM sharing_attempt WHERE message_id=? AND event_id=?",
|
||||
"SELECT state, ack_at, spoken_text FROM sharing_attempt "
|
||||
"WHERE message_id=? AND event_id=?",
|
||||
(message_id, event_id)).fetchone()
|
||||
if not att or att[0] != "judged" or att[1] is None:
|
||||
return False
|
||||
@@ -124,14 +140,22 @@ class ShareBridge:
|
||||
(event_id, message_id)).fetchone()
|
||||
if already:
|
||||
return False # 重复回执：不重复迁入
|
||||
spoken = att[2]
|
||||
if not spoken or not str(spoken).strip():
|
||||
c.execute(
|
||||
"INSERT OR REPLACE INTO migration_refusal"
|
||||
"(message_id,event_id,reason,refused_at) VALUES(?,?,?,?)",
|
||||
(message_id, event_id, "missing_spoken_text", 1002))
|
||||
return False # 未知/缺失已讲内容：不得整段降级放行
|
||||
c.execute("INSERT INTO telling_log(event_id,message_id,told_at) VALUES(?,?,?)",
|
||||
(event_id, message_id, 1002))
|
||||
told_content = row[0] # 只迁"实际已讲内容"
|
||||
told_content = str(spoken) # 只迁"实际已讲内容"（严格限于 spoken_text）
|
||||
public_mem.add(
|
||||
[{"role": "assistant", "content": told_content}],
|
||||
user_id="user-1", agent_id="cg-companion", infer=False,
|
||||
metadata={"life_event_id": event_id, "source_event_id": f"life:{event_id}",
|
||||
"revealed_at": 1002, "message_id": message_id,
|
||||
"spoken_bytes": len(told_content),
|
||||
"migrated_by": "hel857_share_bridge"},
|
||||
)
|
||||
return True
|
||||
@@ -154,6 +178,29 @@ def public_readout(pub_mem, query: str, budget: int = BUDGET) -> tuple[str, list
|
||||
return block, used or []
|
||||
|
||||
|
||||
def anchor_legality(root: Path, ctrl_db: Path) -> dict:
|
||||
"""公开库全量 payload 锚点合法性（HEL-858 P4 补强）：
|
||||
带 life_event_id 的条目必须在 telling_log 有对应迁入记录；不带锚点的条目为
|
||||
直接公开语料写入。"""
|
||||
pub = make_memory(PUB, root)
|
||||
pts, _ = pub.vector_store.client.scroll(
|
||||
collection_name="cg_public", limit=200, with_payload=True)
|
||||
close_memory(pub)
|
||||
legal, illegal = 0, []
|
||||
with sqlite3.connect(str(ctrl_db)) as c:
|
||||
for p in pts:
|
||||
pl = p.payload or {}
|
||||
ev = pl.get("life_event_id")
|
||||
if ev is None:
|
||||
continue # 直接公开语料写入（无迁移锚点）
|
||||
if c.execute("SELECT 1 FROM telling_log WHERE event_id=?", (ev,)).fetchone():
|
||||
legal += 1
|
||||
else:
|
||||
illegal.append({"id": str(p.id), "life_event_id": ev})
|
||||
return {"anchored_legal": legal, "anchored_illegal": illegal,
|
||||
"pass": not illegal}
|
||||
|
||||
|
||||
def privacy_scan(text: str) -> list[str]:
|
||||
return [m for m in ALL_PRIVACY_MARKERS if m and m in text]
|
||||
|
||||
@@ -249,7 +296,7 @@ def run() -> dict:
|
||||
"pass": a_vec_h0 == a_vec_h1 and a_hist_h0 == a_hist_h1,
|
||||
}
|
||||
|
||||
# ---- P4 回读与预算装配 ----
|
||||
# ---- P4 回读与预算装配（含全量 payload 锚点合法性）----
|
||||
priv_m = make_memory(PRIV, src_root)
|
||||
priv_count0 = len(priv_m.get_all(filters={"agent_id": "cg-companion"}).get("results") or [])
|
||||
close_memory(priv_m)
|
||||
@@ -257,6 +304,7 @@ def run() -> dict:
|
||||
block, used = public_readout(pub, "诗集 书架 床头")
|
||||
summary_block, _ = public_readout(pub, "最近我们在忙什么")
|
||||
close_memory(pub)
|
||||
legality = anchor_legality(src_root, src_root / "share_control.db")
|
||||
leaks = privacy_scan(block) + privacy_scan(summary_block)
|
||||
priv_m2 = make_memory(PRIV, src_root)
|
||||
priv_count1 = len(priv_m2.get_all(filters={"agent_id": "cg-companion"}).get("results") or [])
|
||||
@@ -266,11 +314,12 @@ def run() -> dict:
|
||||
"block_len": len(block), "used": len(used),
|
||||
"block_preview": block[:120],
|
||||
"leak": leaks,
|
||||
"anchor_legality": legality,
|
||||
"a_items_before": priv_count0, "a_items_after": priv_count1,
|
||||
"a_vec_hash_unchanged": a_vec_h2 == a_vec_h0,
|
||||
"expect": "公开回读+600字预算装配零泄漏；A 条目数与内容不变",
|
||||
"expect": "公开回读+600字预算装配零泄漏；A 条目数与内容不变；公开库迁移锚点全部合法",
|
||||
"pass": (not leaks) and priv_count0 == priv_count1 and a_vec_h2 == a_vec_h0
|
||||
and len(block) <= BUDGET,
|
||||
and len(block) <= BUDGET and legality["pass"],
|
||||
}
|
||||
|
||||
# ---- P5 完整备份 → 恢复 → 复跑 ----
|
||||
@@ -304,14 +353,51 @@ def run() -> dict:
|
||||
for name in ["mem0_private_history.db", "mem0_public_history.db", "share_control.db"]:
|
||||
shutil.copy2(backup_dir / name, restore_root / name)
|
||||
|
||||
# 恢复后检查
|
||||
# 恢复后检查（HEL-858 返工：恢复后重跑同套 P1–P5/F1–F8，不只 P1/P2）
|
||||
priv_r = make_memory(PRIV, restore_root)
|
||||
priv_items = priv_r.get_all(filters={"agent_id": "cg-companion"}).get("results") or []
|
||||
close_memory(priv_r)
|
||||
a_vec_r0 = dir_hash(restore_root / "vec_private")
|
||||
a_hist_r0 = history_db_hash(restore_root / "mem0_private_history.db")
|
||||
r12r = p1p2_on(restore_root, "B-restored")
|
||||
pub_r = make_memory(PUB, restore_root)
|
||||
block_r, _ = public_readout(pub_r, "诗集 书架 床头")
|
||||
close_memory(pub_r)
|
||||
results["P1_restored_extraction_pool"] = {
|
||||
"leak": r12r["p1_private_leak"],
|
||||
"captured": r12r["captured_prompts"],
|
||||
"positive_control": r12r["positive_control_prompt_has_public_content"],
|
||||
"expect": "恢复后 B 的提取提示词仍不含 A 的私密条目/标记，正对照可见公开内容",
|
||||
"pass": r12r["pass"],
|
||||
}
|
||||
results["P2_restored_history_window"] = {
|
||||
"leak": r12r["p2_private_history_leak"],
|
||||
"expect": "恢复后 get_last_messages 分库隔离仍成立",
|
||||
"pass": not r12r["p2_private_history_leak"],
|
||||
}
|
||||
# P3 恢复后复跑：对恢复的 B 显式 update，A 逐字节不变
|
||||
pub_r3 = make_memory(PUB, restore_root)
|
||||
items_r = pub_r3.get_all(filters={"user_id": "user-1"}).get("results") or []
|
||||
if items_r:
|
||||
pub_r3.update(items_r[0]["id"], text=items_r[0]["memory"] + "（恢复后补记）")
|
||||
close_memory(pub_r3)
|
||||
a_vec_r1 = dir_hash(restore_root / "vec_private")
|
||||
a_hist_r1 = history_db_hash(restore_root / "mem0_private_history.db")
|
||||
results["P3_restored_update_isolation"] = {
|
||||
"a_vec_hash_before": a_vec_r0[:16], "a_vec_hash_after": a_vec_r1[:16],
|
||||
"a_history_hash_before": a_hist_r0[:16], "a_history_hash_after": a_hist_r1[:16],
|
||||
"expect": "恢复后对 B 显式 update() 仍不影响 A（逐字节不变）",
|
||||
"pass": a_vec_r0 == a_vec_r1 and a_hist_r0 == a_hist_r1,
|
||||
}
|
||||
# P4 恢复后复跑：公开出口零泄漏 + 锚点合法性（此时公开库含恢复前的 F4 迁移条目）
|
||||
pub_r4 = make_memory(PUB, restore_root)
|
||||
block_r, _ = public_readout(pub_r4, "诗集 书架 床头")
|
||||
close_memory(pub_r4)
|
||||
legality_r = anchor_legality(restore_root, restore_root / "share_control.db")
|
||||
results["P4_restored_readout_budget"] = {
|
||||
"block_len": len(block_r),
|
||||
"leak": privacy_scan(block_r),
|
||||
"anchor_legality": legality_r,
|
||||
"expect": "恢复后公开出口仍零泄漏；迁移锚点在恢复库 telling_log 均有对应记录",
|
||||
"pass": (not privacy_scan(block_r)) and legality_r["pass"] and len(block_r) <= BUDGET,
|
||||
}
|
||||
# 内部可查（她的连续性），公开出口不泄漏
|
||||
priv_r2 = make_memory(PRIV, restore_root)
|
||||
internal_hit = priv_r2.search(PRIV_MARKER, filters={"agent_id": "cg-companion"})
|
||||
@@ -324,13 +410,24 @@ def run() -> dict:
|
||||
"internal_recall_priv_marker": internal_ok,
|
||||
"restored_public_outlet_leak": privacy_scan(block_r),
|
||||
"restored_P1P2": {"p1": not r12r["p1_private_leak"], "p2": not r12r["p2_private_history_leak"]},
|
||||
"expect": "恢复后：内部未分享经历仍在且内部可查；公开出口（检索+预算装配）仍不泄漏；P1/P2 复跑通过",
|
||||
"expect": "恢复后：内部未分享经历仍在且内部可查；公开出口（检索+预算装配）仍不泄漏；P1-P4 复跑通过",
|
||||
"pass": len(priv_items) >= 2
|
||||
and any(PRIV_OLD in (it["memory"] or "") for it in priv_items)
|
||||
and internal_ok
|
||||
and not privacy_scan(block_r)
|
||||
and not r12r["p1_private_leak"] and not r12r["p2_private_history_leak"],
|
||||
and not r12r["p1_private_leak"] and not r12r["p2_private_history_leak"]
|
||||
and results["P3_restored_update_isolation"]["pass"]
|
||||
and results["P4_restored_readout_budget"]["pass"],
|
||||
}
|
||||
# F1-F8 恢复后复跑：新事件/新消息号 + 恢复前 telling_log 幂等
|
||||
bridge_r = ShareBridge(restore_root / "share_control.db")
|
||||
bridge_r.seed_event(3, PRIV_OTHER_EVENT, told=0)
|
||||
bridge_r.seed_event(4, PRIV_TOLD_EVENT, told=0)
|
||||
f_restored = run_first_share(
|
||||
restore_root, bridge_r, tag="_restored", msg_base=200,
|
||||
ev_hidden=3, ev_told=4, told_content=PRIV_TOLD_EVENT,
|
||||
pre_restore_pair=(104, 2))
|
||||
results.update(f_restored)
|
||||
|
||||
ok = all(v.get("pass") for v in results.values())
|
||||
return {"ok": ok, "network_outbound": 0, "results": results,
|
||||
@@ -340,9 +437,19 @@ def run() -> dict:
|
||||
"leak_criterion": "内容级判据：返回项文本包含私密标记才算泄漏；语义近邻命中本身不是泄漏"}}
|
||||
|
||||
|
||||
def run_first_share(root: Path, bridge: ShareBridge) -> dict:
|
||||
"""F1-F6 首次分享时序（总管核对版执行限定1）。"""
|
||||
def run_first_share(root: Path, bridge: ShareBridge, *, tag: str = "",
|
||||
msg_base: int = 100, ev_hidden: int = 1, ev_told: int = 2,
|
||||
hidden_content: str | None = None, told_content: str | None = None,
|
||||
pre_restore_pair: tuple[int, int] | None = None) -> dict:
|
||||
"""F1-F8 首次分享时序（总管核对版执行限定1 + HEL-858 部分分享/缺失内容负例）。
|
||||
|
||||
事件与消息号参数化：恢复后复跑用新事件/新消息号；pre_restore_pair 给出
|
||||
恢复前已迁的 (message_id, event_id)，验证恢复库上旧 telling_log 仍幂等。
|
||||
"""
|
||||
out: dict = {}
|
||||
hidden = hidden_content if hidden_content is not None else PRIV_OTHER_EVENT
|
||||
told = told_content if told_content is not None else PRIV_TOLD_EVENT
|
||||
m1, m4 = msg_base + 1, msg_base + 4
|
||||
|
||||
def pub_leak(marker: str) -> int:
|
||||
"""内容级泄漏判据：返回项文本中含 marker 的条数（语义近邻命中不算泄漏）。"""
|
||||
@@ -353,76 +460,138 @@ def run_first_share(root: Path, bridge: ShareBridge) -> dict:
|
||||
close_memory(pub)
|
||||
return n
|
||||
|
||||
def pub_payload_texts() -> list[str]:
|
||||
"""全量 payload 文本直读（比 search 更强：不依赖查询能否命中）。"""
|
||||
pub = make_memory(PUB, root)
|
||||
pts, _ = pub.vector_store.client.scroll(
|
||||
collection_name="cg_public", limit=200, with_payload=True)
|
||||
texts = [str((p.payload or {}).get("data") or "") for p in pts]
|
||||
close_memory(pub)
|
||||
return texts
|
||||
|
||||
def pub_anchored(event_id: int) -> int:
|
||||
"""直读向量库 payload 统计迁移锚点条数（search API 不回传自定义 metadata）。"""
|
||||
pub = make_memory(PUB, root)
|
||||
from qdrant_client import models as qm
|
||||
pts, _ = pub.vector_store.client.scroll(
|
||||
collection_name="cg_public", limit=100, with_payload=True,
|
||||
collection_name="cg_public", limit=200, with_payload=True,
|
||||
scroll_filter=qm.Filter(must=[qm.FieldCondition(
|
||||
key="life_event_id", match=qm.MatchValue(value=event_id))]))
|
||||
close_memory(pub)
|
||||
return len(pts)
|
||||
|
||||
def try_migrate(root: Path, message_id: int, event_id: int) -> bool:
|
||||
def try_migrate(message_id: int, event_id: int) -> bool:
|
||||
pub = make_memory(PUB, root)
|
||||
try:
|
||||
return bridge.maybe_migrate(pub, message_id, event_id)
|
||||
finally:
|
||||
close_memory(pub)
|
||||
|
||||
# F1 判定失败：不迁
|
||||
bridge.attempt(101, 1, state="failed", ack=True)
|
||||
migrated = try_migrate(root, 101, 1)
|
||||
n = pub_leak(PRIV_OTHER_EVENT)
|
||||
out["F1_judgment_failed_no_migration"] = {
|
||||
def refusal_rows() -> list[tuple]:
|
||||
with sqlite3.connect(str(bridge.db_path)) as c:
|
||||
return c.execute(
|
||||
"SELECT message_id, event_id, reason FROM migration_refusal").fetchall()
|
||||
|
||||
# F1 判定失败：不迁（已讲文本在，但判定 failed）
|
||||
bridge.attempt(m1, ev_hidden, state="failed", ack=True, spoken_text=hidden)
|
||||
migrated = try_migrate(m1, ev_hidden)
|
||||
n = pub_leak(hidden)
|
||||
out[f"F1_judgment_failed_no_migration{tag}"] = {
|
||||
"migrated": migrated, "public_content_leaks": n,
|
||||
"pass": (not migrated) and n == 0,
|
||||
}
|
||||
# F2 未确认（ack_at NULL）：不迁
|
||||
bridge.attempt(102, 1, state="judged", ack=False)
|
||||
migrated = try_migrate(root, 102, 1)
|
||||
n = pub_leak(PRIV_OTHER_EVENT)
|
||||
out["F2_unconfirmed_no_migration"] = {
|
||||
bridge.attempt(msg_base + 2, ev_hidden, state="judged", ack=False, spoken_text=hidden)
|
||||
migrated = try_migrate(msg_base + 2, ev_hidden)
|
||||
n = pub_leak(hidden)
|
||||
out[f"F2_unconfirmed_no_migration{tag}"] = {
|
||||
"migrated": migrated, "public_content_leaks": n,
|
||||
"pass": (not migrated) and n == 0,
|
||||
}
|
||||
# F3 取消：不迁
|
||||
bridge.attempt(103, 1, state="cancelled", ack=True)
|
||||
migrated = try_migrate(root, 103, 1)
|
||||
n = pub_leak(PRIV_OTHER_EVENT)
|
||||
out["F3_cancelled_no_migration"] = {
|
||||
bridge.attempt(msg_base + 3, ev_hidden, state="cancelled", ack=True, spoken_text=hidden)
|
||||
migrated = try_migrate(msg_base + 3, ev_hidden)
|
||||
n = pub_leak(hidden)
|
||||
out[f"F3_cancelled_no_migration{tag}"] = {
|
||||
"migrated": migrated, "public_content_leaks": n,
|
||||
"pass": (not migrated) and n == 0,
|
||||
}
|
||||
# F4 成功：完整回执确认+分享判定完成 → 按实际已讲内容迁入，带锚点；A 仍保留
|
||||
bridge.attempt(104, 2, state="judged", ack=True)
|
||||
migrated = try_migrate(root, 104, 2)
|
||||
n = pub_leak(PRIV_TOLD_EVENT)
|
||||
anchors = pub_anchored(2)
|
||||
# F4 成功：完整回执确认+分享判定完成+实际已讲文本 → 只迁已讲文本，带锚点；A 仍保留
|
||||
bridge.attempt(m4, ev_told, state="judged", ack=True, spoken_text=told)
|
||||
migrated = try_migrate(m4, ev_told)
|
||||
n = pub_leak(told)
|
||||
anchors = pub_anchored(ev_told)
|
||||
payload_texts = pub_payload_texts()
|
||||
exact_spoken = [t for t in payload_texts if t == told]
|
||||
priv = make_memory(PRIV, root)
|
||||
priv_still = priv.get_all(filters={"agent_id": "cg-companion"}).get("results") or []
|
||||
close_memory(priv)
|
||||
out["F4_success_migrates_told_content_with_anchor"] = {
|
||||
has_priv_instance = bool(priv_still)
|
||||
a_keeps = (any(told in (it["memory"] or "") for it in priv_still)
|
||||
if has_priv_instance else True)
|
||||
out[f"F4_success_migrates_spoken_text_with_anchor{tag}"] = {
|
||||
"migrated": migrated, "public_content_hits": n, "anchored_payload_entries": anchors,
|
||||
"a_still_keeps_internal": any(PRIV_TOLD_EVENT in (it["memory"] or "") for it in priv_still),
|
||||
"pass": migrated and n >= 1 and anchors == 1 and
|
||||
any(PRIV_TOLD_EVENT in (it["memory"] or "") for it in priv_still),
|
||||
"public_write_exactly_spoken_text": bool(exact_spoken),
|
||||
"a_still_keeps_internal": a_keeps,
|
||||
"pass": migrated and n >= 1 and anchors == 1 and bool(exact_spoken) and a_keeps,
|
||||
}
|
||||
# F5 重复回执：不重复迁入
|
||||
dup = try_migrate(root, 104, 2)
|
||||
n2 = pub_leak(PRIV_TOLD_EVENT)
|
||||
anchored = pub_anchored(2)
|
||||
out["F5_duplicate_receipt_no_duplicate_migration"] = {
|
||||
dup = try_migrate(m4, ev_told)
|
||||
anchored = pub_anchored(ev_told)
|
||||
out[f"F5_duplicate_receipt_no_duplicate_migration{tag}"] = {
|
||||
"second_call_returned": dup, "anchored_payload_entries": anchored,
|
||||
"pass": (not dup) and anchored == 1,
|
||||
}
|
||||
# F6 其他未分享事件仍在公开库之外
|
||||
n3 = pub_leak(PRIV_OTHER_EVENT)
|
||||
out["F6_other_unshared_still_hidden"] = {
|
||||
# F6 其他未分享事件仍在公开库之外（全量 payload 直读，不依赖查询命中）
|
||||
texts_all = pub_payload_texts()
|
||||
n3 = pub_leak(hidden)
|
||||
out[f"F6_other_unshared_still_hidden{tag}"] = {
|
||||
"public_content_leaks_priv_other": n3,
|
||||
"pass": n3 == 0,
|
||||
"hidden_text_in_any_payload": hidden in "\n".join(texts_all),
|
||||
"pass": n3 == 0 and hidden not in "\n".join(texts_all),
|
||||
}
|
||||
# F7 部分分享（总管独立复核合成例）：内部事件含从未讲出的秘密，实际只讲了前半句
|
||||
ev_partial = ev_told + 10
|
||||
bridge.seed_event(ev_partial, FX.PARTIAL_EVENT)
|
||||
bridge.attempt(msg_base + 7, ev_partial, state="judged", ack=True,
|
||||
spoken_text=FX.PARTIAL_SPOKEN)
|
||||
migrated = try_migrate(msg_base + 7, ev_partial)
|
||||
texts_after = pub_payload_texts()
|
||||
exact_partial = [t for t in texts_after if t == FX.PARTIAL_SPOKEN]
|
||||
out[f"F7_partial_share_migrates_only_spoken_part{tag}"] = {
|
||||
"migrated": migrated,
|
||||
"public_write_exactly_spoken_part": bool(exact_partial),
|
||||
"unspoken_secret_in_any_payload": FX.PARTIAL_SECRET in "\n".join(texts_after),
|
||||
"unspoken_secret_in_search": pub_leak(FX.PARTIAL_SECRET) ,
|
||||
"pass": migrated and bool(exact_partial)
|
||||
and FX.PARTIAL_SECRET not in "\n".join(texts_after)
|
||||
and pub_leak(FX.PARTIAL_SECRET) == 0,
|
||||
}
|
||||
# F8 已讲文本缺失：即使 ack+judged+同意，也不得整段降级放行（拒绝并审计）
|
||||
ev_missing = ev_told + 11
|
||||
missing_event_content = "她独自复盘了这次分享的分寸，这段独白从未讲给用户"
|
||||
bridge.seed_event(ev_missing, missing_event_content)
|
||||
bridge.attempt(msg_base + 8, ev_missing, state="judged", ack=True, spoken_text=None)
|
||||
migrated = try_migrate(msg_base + 8, ev_missing)
|
||||
texts_final = pub_payload_texts()
|
||||
refusals = refusal_rows()
|
||||
out[f"F8_missing_spoken_text_refuses_whole_event{tag}"] = {
|
||||
"migrated": migrated,
|
||||
"event_content_in_any_payload": missing_event_content in "\n".join(texts_final),
|
||||
"refusal_rows": [list(r) for r in refusals],
|
||||
"refusal_recorded": any(r[2] == "missing_spoken_text" for r in refusals),
|
||||
"pass": (not migrated) and missing_event_content not in "\n".join(texts_final)
|
||||
and any(r[2] == "missing_spoken_text" for r in refusals),
|
||||
}
|
||||
if pre_restore_pair is not None:
|
||||
# 恢复前迁过的旧消息对：恢复后的库上重复回执仍不重复迁入（telling_log 跨恢复保留）
|
||||
old_mid, old_eid = pre_restore_pair
|
||||
dup_old = try_migrate(old_mid, old_eid)
|
||||
out[f"F5r_pre_restore_telling_log_idempotent_after_restore{tag}"] = {
|
||||
"second_call_returned": dup_old,
|
||||
"anchored_payload_entries": pub_anchored(old_eid),
|
||||
"pass": (not dup_old) and pub_anchored(old_eid) == 1,
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user