fix(perf): verify retained parent digest identity

Require declared summary/page parent digests to match their linked Evidence
IDs at SDK acceptance and factual grading. Keep the fixture faithful to the
producer and reject mismatched, malformed or missing parent digests.
Original delivery observations and measured model data remain unchanged.

Validation: 30 context/grading tests, six actual SDK before/after fault and
normal-control probes, plus four stored original-model factual regrades.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
N0zoM1z0
2026-10-14 02:40:31 +08:00
committed by morluto
co-authored by factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> Kaoru0822
parent a4634ba21b
commit e5a5c17e52
4 changed files with 28 additions and 2 deletions
@@ -575,6 +575,11 @@ const pcapObservation = (evidence, requestedView) => {
!evidence.evidence_links.includes(value.parent_evidence_id))
)
throw new Error("PCAP retained parent lacks a distinct Evidence link");
if (
requestedView &&
value.parent_digest !== value.parent_evidence_id.slice(3)
)
throw new Error("PCAP retained parent digest differs from its identity");
const parameters = requestedView
? { parent_evidence_id: value.parent_evidence_id, view: requestedView }
: { path: values.path };
@@ -126,7 +126,7 @@ validated.
Both real CLI complete results match full MCP normalized results, subjects,
parameters, IDs and limitations. Portable CLI summary/URI views match MCP retained
normalized facts and subjects; source forms/caller parameters remain distinct.
Source bytes stay unchanged. Twenty-nine context/grading regressions pass, including
Source bytes stay unchanged. Thirty context/grading regressions pass, including
cited complete metadata/counts/UTF-16 diagnostics and exact ordered response ID
lists that reject parent lookups, duplicates and nested representations. URI
prefix coverage must come from its cited page, even if an uncited page has the
@@ -159,7 +159,7 @@ the preceding driver accepts inconsistent coverage, while the corrected driver
rejects wrong status, next offset or exhaustion. Unmodified exhausted and
continuable URI pages still pass. The grader also requires each cited receipt's
subject digest and local path to identify the reference capture, and uses the
reference page's verified coverage. Twenty-nine grading/context regressions pass,
reference page's verified coverage. Thirty grading/context regressions pass,
including missing or malformed subjects and faithful but inconsistent coverage;
all four original model answers still pass factual regrading. Strict resource
grades remain false where terminal whole-scope counters are unknown.
@@ -244,3 +244,9 @@ faithful receipts with missing or incorrect outer artifact identities. Private
mutations over actual SDK replies reproduce the previous false success and
verify rejection after the shared observation guard; normal deliveries and all
four original model answers retain their recorded facts.
Declared retained parent digests must match the digest encoded in the linked
parent Evidence ID for both summary and page views. SDK acceptance and factual
grading reject mismatches and missing digests while preserving complete
deliveries. Before/after private actual-reply mutations verify both view roles;
unmodified deliveries and four stored original-model factual regrades pass.
@@ -300,6 +300,7 @@ def main():
len(cited) == 1 or (len(cited) == 2 and all(
isinstance(record['normalized_result'].get('parent_evidence_id'), str)
and re.fullmatch(r'ev_[a-f0-9]{64}', record['normalized_result']['parent_evidence_id'])
and record['normalized_result'].get('parent_digest') == record['normalized_result']['parent_evidence_id'][3:]
and record['normalized_result']['parent_evidence_id'] != record['evidence_id']
and isinstance(record.get('evidence_links'), list)
and record['normalized_result']['parent_evidence_id'] in record['evidence_links']
@@ -100,6 +100,7 @@ class PcapGraderWorkflowTests(unittest.TestCase):
'source_coverage': 'complete', 'artifact': capture['target'],
'limitations': ['source', 'view'], 'unknowns': ['unrecorded traffic']}}
for record in [summary, page]:
record['normalized_result']['parent_digest'] = parent[3:]
record['parameters'] = {'parent_evidence_id': parent, 'view': copy.deepcopy(record['normalized_result']['view'])}
observations = [self.observation(value) for value in [summary, page]]
reference = {'artifact_sha256': 'd' * 64, 'capture': capture,
@@ -355,6 +356,19 @@ class PcapGraderWorkflowTests(unittest.TestCase):
answer, records, reference = self.fixture();records[index]['normalized_result'][field] = wrong
self.assertFalse(self.grade(answer, records, reference, tools)['all_factual_checks_pass'])
def test_retained_parent_digest_must_match_its_declared_identity(self):
for index in [0, 1]:
for digest in [None, {}, [], True, 'digest', 'e' * 64]:
with self.subTest(index=index, digest=digest):
answer, records, reference = self.fixture()
records[index]['normalized_result']['parent_digest'] = digest
report = self.grade(answer, records, reference)
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
self.assertFalse(report['all_factual_checks_pass'])
answer, records, reference = self.fixture()
records[index]['normalized_result'].pop('parent_digest')
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
def test_summary_outer_artifact_must_identify_the_selected_capture(self):
for wrong in [None, {}, [], 'artifact',
{'path': '/fixture/other.pcapng', 'sha256': 'd' * 64},