Compare commits

..
Author SHA1 Message Date
morluto 4c99912e8d chore(main): merge pinned CI checkpoint 2895a18 2026-10-14 03:12:47 +08:00
morluto b3a60b6b03 test(ci): provide proc metadata for Linux hopper fixtures 2026-10-14 03:02:01 +08:00
morluto d5b5441a3a fix(hopper): refresh runtime cleanup against main 2026-10-14 02:27:23 +08:00
morluto 4f83c64da8 fix(hopper): merge current main into runtime cleanup 2026-10-14 02:12:58 +08:00
Morluto 780191226c test(hopper): run Python-named fixture through Node eval 2026-10-14 02:03:25 +08:00
morluto 1738989590 test: isolate hopper scanner process fixtures 2026-10-14 01:26:30 +08:00
morluto 3f18439acd fix(types): declare hopper inventory proc root 2026-10-14 01:24:48 +08:00
morluto dd401a2804 test(process): isolate hopper inventory proc root 2026-10-14 01:06:02 +08:00
morluto 2bc898fae2 chore: merge captured main into Hopper runtime cleanup 2026-10-13 21:45:06 +08:00
N0zoM1z0andKaoru0822 b2e743ef24 test(hopper): reap orphan fixtures independently of container init
Keep a test-owned Linux subreaper until token-verified process-group cleanup completes, then wait for each adopted child. Observe from a separate verifier so the orphan remains outside its lineage. Assert disappearance from procfs after reaping.

Reproduced two failures and eight retained zombies without an init reaper. The corrected fourteen process cases pass with no residual children; typecheck and lint pass.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 12:25:38 +00:00
morluto 51b3418f2b chore: merge upstream CI baseline fixes 2026-10-13 20:13:27 +08:00
N0zoM1z0andKaoru0822 3bb09b8d2d fix(verification): retain command-less runtime observations
Use the kernel command name for processes without argv or an executable link. Preserve uncertainty for command-less Python interpreters and detect Hopper/Xvfb zombies without signaling. Real unreaped-child regressions keep cleanup with their waiting parent.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 3deda4781d fix(hopper): parse Python helper interpreter options
Resolve the script operand using CPython option boundaries, including combined flags, separate or attached option values, and --. Keep command/module arguments distinct and defer unknown invocation failures rather than declaring them clean.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 2ac4795df5 test(hopper): cover a reparented Python helper command
Launch a real Python child with the helper filename, wait for readiness before the Node parent exits, bind its procfs arguments, require non-signaling detection, and clean it through the existing run-token/group identity primitive.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 a588dc6376 fix(hopper): distinguish runtime processes from argument text
Parse procfs argument boundaries rather than searching rendered ps command text. Keep executable identity checks and deferred permission failures, eliminate the extra ps process, and put shared process observations in the serial process-boundary lane.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
14 changed files with 526 additions and 638 deletions
+2 -3
View File
@@ -389,9 +389,8 @@ adds in-place reparse, breakaway, environment, and token observations.
The separate `scripts/verify-windows-native-inspection.mjs` lane uses the
`.cache/windows-native-inspection-fixture` artifact built on Linux with
`node scripts/build-windows-native.mjs --inspection-fixture`. Its controlled
first deletion failure checks retained NTFS ownership across a same-volume
rename, later-query handle-based recovery, successful-admission caching, and
environment-teardown cleanup. It neither
first deletion failure checks retained NTFS ownership, later-query recovery,
successful-admission caching, and environment-teardown cleanup. It neither
establishes the cause of the intermittent #1701 host failure nor replaces
verification with the shipped production artifact.
`npm run verify:ghidra:windows:package` additionally packs and installs REA into
-3
View File
@@ -197,9 +197,6 @@ if (tier === "reused") {
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
if (file !== ".github/workflows/ci.yml") continue;
}
// The Notes example is also the source-owned agent evaluation fixture.
if (file.startsWith("website/public/examples/notes-electron/"))
selected.code = true;
if (owners.length === 1 && owners[0] === "website") continue;
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
if (file === "package.json") {
+74 -18
View File
@@ -1,8 +1,36 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { readdir, stat } from "node:fs/promises";
import { readFile, readdir, stat } from "node:fs/promises";
import { basename } from "node:path";
const execFileAsync = promisify(execFile);
// CPython's interface option ends interpreter parsing; following values are
// script arguments. -W/-X consume either an attached value or the next operand.
// https://docs.python.org/3.12/using/cmdline.html
const pythonScriptOperand = (arguments_) => {
for (let index = 1; index < arguments_.length; index++) {
const option = arguments_[index];
if (option === "--") return arguments_[index + 1];
if (option === "-" || option === "") return undefined;
if (!option.startsWith("-")) return option;
if (option === "--check-hash-based-pycs") {
index++;
continue;
}
if (/^--(?:help(?:-env|-xoptions|-all)?|version)$/u.test(option))
return undefined;
if (option.startsWith("--"))
throw new Error(`Unknown Python helper invocation option: ${option}`);
for (let character = 1; character < option.length; character++) {
const flag = option[character];
if ("cmh?V".includes(flag)) return undefined;
if (flag === "W" || flag === "X") {
if (character === option.length - 1) index++;
break;
}
if (!"bBdEiIOPqRsSuvx".includes(flag))
throw new Error(`Unknown Python helper invocation option: ${option}`);
}
}
return undefined;
};
/**
* Observe leftover demo processes on a dedicated clean Linux runner, including
@@ -18,9 +46,6 @@ export const assertCleanLinuxHopperRuntime = async (
],
procRoot = "/proc",
) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = [];
for (const path of launcherPaths) {
try {
@@ -29,20 +54,55 @@ export const assertCleanLinuxHopperRuntime = async (
if (error?.code !== "ENOENT") throw error;
}
}
if (launchers.length === 0) return;
// /proc exposes the executed file even when argv[0] names a removed symlink.
// Compare file identity, not a display path or a process name.
let inspectionError;
for (const entry of await readdir(procRoot, { withFileTypes: true })) {
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
let executable;
let retainedRuntime;
try {
executable = await stat(`${procRoot}/${entry.name}/exe`, {
bigint: true,
});
// procfs preserves argument boundaries. A caller merely mentioning a
// runtime in its script text or other arguments is not that runtime.
const commandLine = await readFile(
`${procRoot}/${entry.name}/cmdline`,
"utf8",
);
const arguments_ = commandLine.split("\0");
// The kernel command field identifies renamed runtime processes such
// as Xvfb. Keep it separate from arguments so ordinary script text does
// not count as a retained runtime.
const kernelCommand = (
await readFile(`${procRoot}/${entry.name}/comm`, "utf8")
).replace(/\n$/u, "");
if (
commandLine.length === 0 &&
/^python3(?:\.\d+)?$/u.test(kernelCommand)
)
throw new Error(
"Cannot identify a command-less Python process on the clean Linux runner",
);
retainedRuntime =
kernelCommand === "Hopper" ||
kernelCommand === "Xvfb" ||
arguments_[0] === "/opt/hopper/bin/Hopper" ||
basename(arguments_[0] ?? "") === "Xvfb" ||
(/^python3(?:\.\d+)?$/u.test(basename(arguments_[0] ?? "")) &&
arguments_.some(
(value) => basename(value) === "hopper-demo-x11.py",
) &&
basename(pythonScriptOperand(arguments_) ?? "") ===
"hopper-demo-x11.py");
if (!retainedRuntime && launchers.length > 0) {
const executable = await stat(`${procRoot}/${entry.name}/exe`, {
bigint: true,
});
retainedRuntime = launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
);
}
} catch (error) {
if (error?.code === "ENOENT" || error?.code === "ESRCH") continue;
// Other users' executables may be unreadable. The setup lane launches
// Other users' process facts may be unreadable. The setup lane launches
// providers as the caller; unreadable caller-owned processes fail closed.
if (error?.code === "EACCES") {
const owner = await stat(`${procRoot}/${entry.name}`).catch((cause) => {
@@ -56,11 +116,7 @@ export const assertCleanLinuxHopperRuntime = async (
inspectionError ??= error;
continue;
}
if (
launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
)
)
if (retainedRuntime)
throw new Error("Clean Linux runner retained a Hopper demo process");
}
if (inspectionError !== undefined) throw inspectionError;
+4 -3
View File
@@ -1,3 +1,7 @@
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
type VerifierCall = (
name: string,
arguments_?: Record<string, unknown>,
@@ -25,6 +29,3 @@ export function verifyHopperStringObjects(
}[];
readonly displayEscapesExcludedFromMatching: true;
}>;
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
-12
View File
@@ -155,18 +155,6 @@ try {
requiredAnswerTermGroups: [["profileapi"], ["profile:read"]],
prompt: `請靜態分析已交付的 Electron 程式 ${targets.javascript},說明 renderer 如何透過 preload 讀取 profile,以及 main handler 的關係。不要執行目標或安裝分析引擎;引用完整的產出 Evidence,保留 runtime 未知事項。`,
},
{
id: "notes-export",
expectedFirstTool: "analyze_javascript_application",
requiresEvidence: true,
requiredAnswerTermGroups: [],
requiredToolSubsequence: [
"analyze_javascript_application",
"trace_application_feature",
"trace_javascript_semantics",
],
prompt: `Use REA on the shipped Electron application at ${targets.notesElectron} to investigate its notes export API, preload exposure and main handler. Do not read target files directly or launch Electron. Use the returned application Evidence to trace the exact notes:export IPC channel in both directions, then query semantic relations for the property invoke with forward influence, excluding ambiguous dynamic edges and without an expected sink. Report the selected static facts, source locations and coverage limits using the closed claim contract.`,
},
{
id: "javascript-module-view",
expectedFirstTool: "analyze_javascript_application",
+2 -5
View File
@@ -62,11 +62,8 @@ assert.equal(first.retryable,true);
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
const [name,...extra]=readdirSync(temp);assert.ok(name);assert.equal(extra.length,0);
const path=join(temp,name);const identity=statSync(path,{bigint:true});
// This private directory permits rename while its retained handle still owns
// cleanup. Recovery must follow the object identity after its path changes.
const replacement=path+'-replacement';renameSync(path,replacement);
assert.equal(statSync(replacement,{bigint:true}).ino,identity.ino);
assert.equal(readdirSync(temp).length,1);
assert.throws(()=>renameSync(path,path+'-replacement'));
assert.equal(statSync(path,{bigint:true}).ino,identity.ino);
if(process.argv[2]!=='teardown'){
const second=systemWindowsNativeAuthority();assert.equal(second.available,true);
assert.equal(readdirSync(temp).length,0);
+16 -24
View File
@@ -6,7 +6,6 @@ import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
type AnalysisInputIssue,
} from "../../domain/analysisErrorCore.js";
import { createEvidence, type Evidence } from "../../domain/evidence.js";
import {
@@ -42,29 +41,22 @@ export class WasmArtifactService {
cause: input.error,
}),
);
const pathIssues = [
{ value: input.data.path, path: ["path"] },
...input.data.glue_paths.map((value, index) => ({
value,
path: ["glue_paths", index],
})),
...input.data.candidate_paths.map((value, index) => ({
value,
path: ["candidate_paths", index],
})),
].flatMap<AnalysisInputIssue>(({ value, path }) =>
isAbsolute(value)
? []
: [
{
path,
reason: "invalid_format",
message: "Expected an absolute filesystem path on this host.",
},
],
);
if (pathIssues.length > 0)
return err(new AnalysisInputError(OPERATION, undefined, pathIssues));
if (
![
input.data.path,
...input.data.glue_paths,
...input.data.candidate_paths,
].every(isAbsolute)
)
return err(
new AnalysisInputError(OPERATION, undefined, [
{
path: ["path"],
reason: "invalid_format",
message: "Expected an absolute filesystem path on this host.",
},
]),
);
const inspected = await this.provider.inspect(input.data, options);
if (!inspected.ok) return inspected;
if (options?.signal?.aborted)
+12 -144
View File
@@ -8,11 +8,6 @@ import type { JsonValue } from "../domain/jsonValue.js";
import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js";
import type { ApplicationNode } from "../domain/javascript/javascriptApplicationGraphSchemas.js";
import { javaScriptExportShapeComparisonResultSchema } from "../domain/javascript/javascriptExportShapeComparisonSchemas.js";
import {
applicationFeatureTraceResultSchema,
featureSeedMatchMode,
} from "../domain/javascript/javascriptFeatureTraceSchemas.js";
import { javaScriptSemanticTraceResultSchema } from "../domain/javascript/javascriptSemanticTraceSchemas.js";
import {
managedArtifactInspectionSchema,
managedMemberInspectionSchema,
@@ -33,8 +28,6 @@ export interface AgentEvaluationFixtureTargets {
readonly javascriptSha256: string;
readonly javascriptShapeLeft: string;
readonly javascriptShapeRight: string;
readonly notesElectron: string;
readonly notesElectronSha256: string;
readonly managed?: string;
readonly managedSha256?: string;
}
@@ -57,8 +50,6 @@ export const createAgentEvaluationFixtures = async (
const javascript = join(root, "desktop-app.asar");
const javascriptShapeLeft = join(root, "parser-v1");
const javascriptShapeRight = join(root, "parser-v2");
const notesDirectory = join(root, "notes-electron");
const notesElectron = join(root, "notes-electron.asar");
await Promise.all([
mkdir(join(desktop, "renderer"), { recursive: true }),
mkdir(javascriptShapeLeft, { recursive: true }),
@@ -76,14 +67,8 @@ export const createAgentEvaluationFixtures = async (
join(repositoryRoot, "tests/fixtures/replay/parser-v2.mjs"),
join(javascriptShapeRight, "parser.mjs"),
),
cp(
join(repositoryRoot, "website/public/examples/notes-electron"),
notesDirectory,
{ recursive: true },
),
]);
await createPackage(desktop, javascript);
await createPackage(notesDirectory, notesElectron);
return {
javascript,
javascriptSha256: createHash("sha256")
@@ -91,10 +76,6 @@ export const createAgentEvaluationFixtures = async (
.digest("hex"),
javascriptShapeLeft,
javascriptShapeRight,
notesElectron,
notesElectronSha256: createHash("sha256")
.update(await readFile(notesElectron))
.digest("hex"),
};
};
@@ -208,116 +189,21 @@ const runtimeSemantics = (value: JsonValue): JsonValue | undefined => {
: undefined;
};
const featureModules = (value: JsonValue): JsonValue | undefined => {
const result = applicationFeatureTraceResultSchema.safeParse(value);
if (!result.success || result.data.graph === null) return undefined;
if (
result.data.seed.kind !== "channel" ||
result.data.seed.value !== "notes:export" ||
featureSeedMatchMode(result.data.seed) !== "exact"
)
return undefined;
const paths = result.data.graph.nodes
.filter((node) => node.kind === "javascript-module")
.flatMap((node) => node.observations)
.filter(({ properties }) => properties.semantic_role === "source-module")
.map(({ properties }) => properties.path);
if (paths.length === 0 || !paths.every((path) => typeof path === "string"))
return undefined;
return {
paths: [...new Set(paths)].sort(compareUnicodeCodePoints),
coverage: result.data.coverage.status,
};
};
const semanticBoundary = (value: JsonValue): JsonValue | undefined => {
const result = javaScriptSemanticTraceResultSchema.safeParse(value);
if (!result.success) return undefined;
return {
status: result.data.status,
coverage: result.data.coverage.status,
relevant_unknowns: result.data.summary.relevant_unknowns,
};
};
const notesClaims = (
const asarClaims = (
targets: AgentEvaluationFixtureTargets,
): readonly FixtureClaimExpectation[] => {
const subject = {
path: targets.notesElectron,
sha256: targets.notesElectronSha256,
};
const derived = {
authority: "analyst-inference",
confidence: "inferred",
linkedSubjects: [
{ operation: "analyze_javascript_application", ...subject },
],
} satisfies Pick<
FixtureClaimExpectation["source"],
"authority" | "confidence" | "linkedSubjects"
>;
// Fixture source establishes bridge/module facts; the semantic claim checks
// the bounded static-query contract, not execution or CSV formatter behavior.
return [
...bridgeClaims(subject, "notes", ["exportCsv"], "notes:export"),
{
id: "feature_modules",
expectedValue: {
paths: ["csv.js", "main.js", "preload.js", "renderer.js"],
coverage: "complete-within-source",
},
source: {
...derived,
operation: "trace_application_feature",
parameters: {
direction: "both",
},
select: featureModules,
},
},
{
id: "semantic_boundary",
expectedValue: {
status: "ambiguous",
coverage: "partial",
relevant_unknowns: 1,
},
source: {
...derived,
operation: "trace_javascript_semantics",
parameters: {
query: {
seed: { kind: "property", name: "invoke" },
direction: "forward-influence",
include_ambiguous_dynamic_edges: false,
expected: null,
},
},
select: semanticBoundary,
},
},
];
};
const bridgeClaims = (
subject: { readonly path: string; readonly sha256: string },
key: string,
members: readonly string[],
channel: string,
): readonly FixtureClaimExpectation[] => {
const source = {
operation: "analyze_javascript_application",
authority: "shipped-artifact",
confidence: "derived",
subject,
subject: { path: targets.javascript, sha256: targets.javascriptSha256 },
} satisfies Omit<FixtureClaimExpectation["source"], "select">;
return [
{
id: "renderer_api",
expectedValue: {
key,
members: [...members],
key: "profileApi",
members: ["read"],
world: "main",
source: "preload.js",
},
@@ -326,7 +212,7 @@ const bridgeClaims = (
{
id: "renderer_ipc",
expectedValue: {
channel,
channel: "profile:read",
operation: "invoke",
source: "preload.js",
},
@@ -335,7 +221,7 @@ const bridgeClaims = (
{
id: "main_ipc",
expectedValue: {
channel,
channel: "profile:read",
operation: "handle",
source: "main.js",
},
@@ -349,16 +235,6 @@ const bridgeClaims = (
];
};
const asarClaims = (
targets: AgentEvaluationFixtureTargets,
): readonly FixtureClaimExpectation[] =>
bridgeClaims(
{ path: targets.javascript, sha256: targets.javascriptSha256 },
"profileApi",
["read"],
"profile:read",
);
const parserClaims = (
targets: AgentEvaluationFixtureTargets,
): readonly FixtureClaimExpectation[] => {
@@ -523,33 +399,25 @@ export const agentFixtureClaims = (
if (scenarioId === "asar") return asarClaims(targets);
if (scenarioId === "managed") return managedClaims(targets);
if (scenarioId === "javascript-export-shape") return parserClaims(targets);
if (scenarioId === "notes-export") return notesClaims(targets);
return undefined;
};
const bridgeAnswerDescriptions = [
'renderer_api: {"key": string, "members": string[], "world": string, "source": string}, describing the exposed bridge API, its members, world, and source file.',
'renderer_ipc: {"channel": string, "operation": string, "source": string}, describing the renderer-side IPC operation and its source file.',
'main_ipc: {"channel": string, "operation": string, "source": string}, describing the main-side IPC operation and its source file.',
'preload: {"path": string, "resolution_status": string}, describing the BrowserWindow-declared preload resolved within the shipped artifact.',
];
const answerValueDescriptions: Readonly<Record<string, readonly string[]>> = {
managed: [
"public_types: string[], listing the exact public top-level type full names in Unicode code-point order from unfiltered member Evidence (omit type_names).",
'entry_point: {"declaring_type": string, "name": string}, resolving the CLI metadata entry-point token against the member inventory from the same artifact SHA-256 and module MVID. Cite member Evidence; both artifact and member Evidence must be delivered.',
],
asar: bridgeAnswerDescriptions,
asar: [
'renderer_api: {"key": string, "members": string[], "world": string, "source": string}, describing the exposed bridge API, its members, world, and source file.',
'renderer_ipc: {"channel": string, "operation": string, "source": string}, describing the renderer-side IPC operation and its source file.',
'main_ipc: {"channel": string, "operation": string, "source": string}, describing the main-side IPC operation and its source file.',
'preload: {"path": string, "resolution_status": string}, describing the BrowserWindow-declared preload resolved within the shipped artifact.',
],
"javascript-export-shape": [
'heading_change: {"status": string, "path": string, "discriminant": object or null, "left": object, "right": object}, copying the exact heading-change fields from the comparison result.',
'comparison_coverage: {"status": string, "paired_variants": number, "added": number, "removed": number, "changed": number, "unknown": number}, copying comparison coverage and summary counts.',
'runtime_semantics: "established" or "not_established", stating whether the cited comparison establishes runtime behavior.',
],
"notes-export": [
...bridgeAnswerDescriptions,
'feature_modules: {"paths": string[], "coverage": string}, listing the unique source-module observation paths in the feature trace in Unicode code-point order, and its coverage status.',
'semantic_boundary: {"status": string, "coverage": string, "relevant_unknowns": number}, copying the semantic trace status, coverage status, and summary.relevant_unknowns. Static traces do not establish runtime execution.',
],
};
/** Describe the strict answer contract without revealing any golden fixture values. */
@@ -476,10 +476,6 @@ it.each([
{ code: true, runtime: false, browser: false, package: false },
],
["website/src/index.ts", { code: false, website: true, package: false }],
[
"website/public/examples/notes-electron/preload.js",
{ code: true, website: true, package: false },
],
[
"website/locales/zh-CN/messages.po",
{ code: false, website: true, package: false },
@@ -245,334 +245,6 @@ describe("source-produced parser comparison known answers", () => {
});
});
describe("Notes export known-answer evaluation", () => {
it("grades real analysis and traces while rejecting false values and unsupported citations", async () => {
const root = await createTestTempDirectory("rea-known-answer-notes-");
const targets = await createAgentEvaluationFixtures(root, resolve("."));
const claims = agentFixtureClaims("notes-export", targets);
if (claims === undefined) throw new Error("Missing Notes rubric");
expect(claims.map(({ id }) => id)).toEqual([
"renderer_api",
"renderer_ipc",
"main_ipc",
"preload",
"feature_modules",
"semantic_boundary",
]);
const harness = await createApplicationMcpHarness();
try {
const workflow = await notesWorkflow(harness, targets.notesElectron);
assertNotesAnswerFailures(workflow, claims);
assertNotesUnsupportedClaims(workflow, claims);
await assertNotesSeedBinding(harness, workflow, claims);
} finally {
await harness.close();
}
});
it("binds Notes analysis and derived traces to the requested artifact path and bytes", async () => {
const root = await createTestTempDirectory(
"rea-known-answer-notes-identity-",
);
const targets = await createAgentEvaluationFixtures(root, resolve("."));
const claims = agentFixtureClaims("notes-export", targets);
if (claims === undefined) throw new Error("Missing Notes rubric");
const foreign = join(root, "other-notes.asar");
await cp(targets.notesElectron, foreign);
const harness = await createApplicationMcpHarness();
try {
for (const path of [foreign, targets.notesElectron]) {
if (path === targets.notesElectron) {
await appendFile(
join(root, "notes-electron/renderer.js"),
"\n// changed shipped bytes\n",
);
await createPackage(join(root, "notes-electron"), path);
}
const workflow = await notesWorkflow(harness, path);
const digest = notesEvidence(workflow, "analyze_javascript_application")
.subject?.digest.sha256;
expect(digest === targets.notesElectronSha256).toBe(path === foreign);
const assessment = grade(
workflow.events,
claimManifestByOperation(claims, workflow.evidence),
"analyze_javascript_application",
claims,
).factualAssessment;
expect(assessment.status).toBe("failed");
assertClaimFailure(
assessment,
"renderer_api",
"source_subject_mismatch",
);
for (const id of ["feature_modules", "semantic_boundary"])
assertClaimFailure(assessment, id, "source_links_mismatch");
}
} finally {
await harness.close();
}
});
});
type NotesWorkflow = Awaited<ReturnType<typeof notesWorkflow>>;
const notesEvidence = (
workflow: NotesWorkflow,
operation: string,
): Evidence => {
const evidence = workflow.evidence.get(operation);
if (evidence === undefined)
throw new Error(`Missing Notes ${operation} Evidence`);
return evidence;
};
const assertClaimFailure = (
assessment: ReturnType<typeof grade>["factualAssessment"],
id: string,
failure: string,
) => {
expect(assessment.claims).toContainEqual(
expect.objectContaining({
id,
failures: expect.arrayContaining([failure]),
}),
);
};
const assertNotesAnswerFailures = (
workflow: NotesWorkflow,
claims: readonly FixtureClaimExpectation[],
) => {
const manifest = claimManifestByOperation(claims, workflow.evidence);
const assess = (answer: unknown, events = workflow.events) =>
grade(events, answer, "analyze_javascript_application", claims)
.factualAssessment;
expect(assess(manifest).status).toBe("passed");
const cases = [
{
id: "renderer_ipc",
patch: {
value: {
channel: "notes:delete",
operation: "send",
source: "preload.js",
},
},
failure: "value_mismatch",
},
{
id: "semantic_boundary",
patch: {
value: { status: "found", coverage: "complete", relevant_unknowns: 0 },
},
failure: "value_mismatch",
},
{
id: "renderer_api",
patch: {
evidence_id: notesEvidence(workflow, "trace_application_feature")
.evidence_id,
},
failure: "source_operation_mismatch",
},
];
for (const { id, patch, failure } of cases) {
const answer = {
claims: manifest.claims.map((claim) =>
claim.id === id ? { ...claim, ...patch } : claim,
),
};
assertClaimFailure(assess(answer), id, failure);
}
const missingParent = assess(
manifest,
workflow.events.filter((event) => event.item.id !== "notes-analysis"),
);
for (const id of ["feature_modules", "semantic_boundary"])
assertClaimFailure(missingParent, id, "source_links_mismatch");
};
const assertNotesUnsupportedClaims = (
workflow: NotesWorkflow,
claims: readonly FixtureClaimExpectation[],
) => {
const manifest = claimManifestByOperation(claims, workflow.evidence);
const evidenceId = notesEvidence(
workflow,
"trace_application_feature",
).evidence_id;
const unsupportedRuntime = {
claims: [
...manifest.claims,
{
id: "runtime_export",
value: "A click wrote notes.csv to the actual Downloads directory.",
evidence_id: evidenceId,
authority: "controlled-replay",
confidence: "observed",
},
],
};
expect(
grade(
workflow.events,
unsupportedRuntime,
"analyze_javascript_application",
claims,
).factualAssessment.manifestFailures,
).toContain("unknown_claim:runtime_export");
const prose = evaluateCodexEvents(
[
...workflow.events,
{
type: "item.completed",
item: {
type: "agent_message",
text: `Observed Evidence ${evidenceId}: notes:export, preload.js and main.js do not exist. The export certainly executed and wrote notes.csv; runtime behavior is known without further observation.`,
},
},
],
"analyze_javascript_application",
{
requireEvidence: true,
requiredAnswerTermGroups: [["notes:export"], ["preload.js"], ["main.js"]],
fixtureClaims: claims,
},
);
expect(prose).toMatchObject({
finalCitesEvidence: true,
answerTermCoverageMet: true,
factualCorrectness: "failed",
});
expect(prose.factualAssessment.manifestFailures).toContain(
"invalid_final_claim_manifest",
);
};
const assertNotesSeedBinding = async (
harness: Awaited<ReturnType<typeof createApplicationMcpHarness>>,
workflow: NotesWorkflow,
claims: readonly FixtureClaimExpectation[],
) => {
const manifest = claimManifestByOperation(claims, workflow.evidence);
for (const match of ["contains", undefined] as const) {
const feature = await harness.client.callTool({
name: "trace_application_feature",
arguments: {
application: notesEvidence(workflow, "analyze_javascript_application"),
seed: {
kind: "channel",
value: "notes:export",
...(match === undefined ? {} : { match }),
case_sensitive: false,
},
direction: "both",
},
});
expect(feature.isError).not.toBe(true);
const answer = {
claims: manifest.claims.map((claim) =>
claim.id === "feature_modules"
? { ...claim, evidence_id: resultEvidence(feature).evidence_id }
: claim,
),
};
const assessment = grade(
[
...workflow.events,
toolEvent(
`notes-feature-${match ?? "default"}`,
"trace_application_feature",
feature,
),
],
answer,
"analyze_javascript_application",
claims,
).factualAssessment;
expect(assessment.status).toBe(match === undefined ? "passed" : "failed");
if (match === "contains") {
assertClaimFailure(
assessment,
"feature_modules",
"source_assertion_mismatch",
);
expect(
assessment.claims.find(({ id }) => id === "feature_modules")?.failures,
).not.toContain("source_parameters_mismatch");
}
}
};
const notesWorkflow = async (
harness: Awaited<ReturnType<typeof createApplicationMcpHarness>>,
inputPath: string,
) => {
const analysis = await harness.client.callTool({
name: "analyze_javascript_application",
arguments: { input_path: inputPath },
});
expect(analysis.isError).not.toBe(true);
const application = resultEvidence(analysis);
const feature = await harness.client.callTool({
name: "trace_application_feature",
arguments: {
application,
seed: {
kind: "channel",
value: "notes:export",
match: "exact",
case_sensitive: false,
},
direction: "both",
},
});
expect(feature.isError).not.toBe(true);
const semantic = await harness.client.callTool({
name: "trace_javascript_semantics",
arguments: {
application,
query: {
seed: { kind: "property", name: "invoke" },
direction: "forward-influence",
include_ambiguous_dynamic_edges: false,
expected: null,
},
},
});
expect(semantic.isError).not.toBe(true);
return {
events: [
toolEvent("notes-analysis", "analyze_javascript_application", analysis),
toolEvent("notes-feature", "trace_application_feature", feature),
toolEvent("notes-semantic", "trace_javascript_semantics", semantic),
],
evidence: new Map<string, Evidence>([
["analyze_javascript_application", application],
["trace_application_feature", resultEvidence(feature)],
["trace_javascript_semantics", resultEvidence(semantic)],
]),
};
};
const claimManifestByOperation = (
claims: readonly FixtureClaimExpectation[],
evidenceByOperation: ReadonlyMap<string, Evidence>,
) => ({
claims: claims.map(({ id, expectedValue, source }) => {
const evidence = evidenceByOperation.get(source.operation);
if (evidence === undefined)
throw new Error(`Missing producer for claim ${id}`);
return {
id,
value: expectedValue,
evidence_id: evidence.evidence_id,
authority: evidence.authority,
confidence: evidence.confidence,
};
}),
});
const resultEvidence = (result: {
readonly structuredContent?: unknown;
}): Evidence => parseEvidence(result.structuredContent);
@@ -215,97 +215,6 @@ it("advertises exact valid schemas and records inline WASM interface evidence wi
expect(close).toHaveBeenCalled();
});
it("attributes rejected WASM paths to their selected fields before provider work", async () => {
const value = wasmArtifactFixture();
value.glue = [
{
artifact: { path: "/glue.js", sha256: "b".repeat(64), bytes: 0 },
parse_status: "complete",
references: [],
},
];
value.candidates.push({
path: "/other.wasm",
sha256: "c".repeat(64),
bytes: 8,
});
const validInput = {
path: value.artifact.path,
glue_paths: ["/glue.js"],
candidate_paths: ["/other.wasm"],
};
const inspect = vi.fn(() =>
Promise.resolve(ok(wasmArtifactExecution(value))),
);
const harness = await createApplicationMcpHarness({
serverOptions: {
wasmArtifact: new WasmArtifactService({
close: () => Promise.resolve(),
inspect,
}),
},
});
onTestFinished(harness.close);
for (const [input, paths] of [
[{ path: "relative.wasm" }, [["path"]]],
[
{ path: value.artifact.path, glue_paths: ["relative.js"] },
[["glue_paths", 0]],
],
[
{ path: value.artifact.path, candidate_paths: ["relative.wasm"] },
[["candidate_paths", 0]],
],
[
{
path: value.artifact.path,
glue_paths: ["/glue.js", "relative.js"],
candidate_paths: ["/other.wasm", "relative.wasm"],
},
[
["glue_paths", 1],
["candidate_paths", 1],
],
],
[
{
path: "relative.wasm",
glue_paths: ["relative.js"],
candidate_paths: ["relative-candidate.wasm"],
},
[["path"], ["glue_paths", 0], ["candidate_paths", 0]],
],
] as const) {
const response = await harness.client.callTool({
name: "inspect_wasm_artifact",
arguments: input,
});
const failure = parseMcpToolError(response).error;
expect(failure.code).toBe("invalid_request");
const { issues } = z
.object({ issues: z.array(z.unknown()) })
.parse(failure.details);
expect(issues).toEqual(
paths.map((path) => ({
path,
reason: "invalid_format",
message: "Expected an absolute filesystem path on this host.",
})),
);
}
expect(inspect).not.toHaveBeenCalled();
const valid = await harness.client.callTool({
name: "inspect_wasm_artifact",
arguments: validInput,
});
expect(valid.isError).not.toBe(true);
expect(inspect).toHaveBeenCalledExactlyOnceWith(
validInput,
expect.objectContaining({ signal: expect.any(AbortSignal) }),
);
await harness.client.ping();
});
it("rejects a WASM provider response with a non-file subject before recording Evidence", async () => {
const value = wasmArtifactFixture();
const execution = wasmArtifactExecution(value);
@@ -0,0 +1,411 @@
import { execFile } from "node:child_process";
import { randomUUID } from "node:crypto";
import {
chmod,
copyFile,
mkdir,
readFile,
symlink,
unlink,
writeFile,
} from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { assertCleanLinuxHopperRuntime } from "../../../scripts/lib/clean-linux-hopper-runtime.mjs";
import { cleanupOwnedProcessGroup } from "../../../src/process/ProcessOwnership.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const execFileAsync = promisify(execFile);
it.runIf(process.platform === "linux").each([
["Xvfb", "Clean Linux runner retained a Hopper demo process"],
["Hopper", "Clean Linux runner retained a Hopper demo process"],
["python3", "Cannot identify a command-less Python process"],
["unrelated", undefined],
] as const)(
"observes an unreaped %s without signaling or reaping it",
async (name, refusal) => {
const directory = await createTestTempDirectory("rea-runtime-zombie-");
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const executable = join(directory, name);
await copyFile("/usr/bin/true", executable);
await chmod(executable, 0o700);
// WNOWAIT observes exit without reaping. The parent keeps the zombie until
// the test closes its stdin, then waitpid performs the actual cleanup.
const parent = execFileAsync(
"/usr/bin/python3",
[
"-c",
`
import os, sys
pid = os.fork()
if pid == 0:
os.execl(sys.argv[1], sys.argv[1])
try:
os.waitid(os.P_PID, pid, os.WEXITED | os.WNOWAIT)
print(pid, flush=True)
sys.stdin.buffer.read(1)
finally:
os.waitpid(pid, 0)
`,
executable,
],
{ timeout: 10_000 },
);
const ready = new Promise<number>((resolve, reject) => {
parent.child.stdout?.once("data", (data: Buffer) =>
resolve(Number(data.toString().trim())),
);
parent.catch(reject);
});
let zombiePid: number | undefined;
try {
zombiePid = await ready;
expect(Number.isSafeInteger(zombiePid)).toBe(true);
const prefix = `/proc/${String(zombiePid)}`;
const procEntry = join(procRoot, String(zombiePid));
await mkdir(procEntry);
for (const name of ["cmdline", "comm"])
await symlink(`${prefix}/${name}`, join(procEntry, name));
expect(await readFile(`${prefix}/cmdline`, "utf8")).toBe("");
expect((await readFile(`${prefix}/comm`, "utf8")).trim()).toBe(name);
const observation = assertCleanLinuxHopperRuntime([], procRoot);
if (refusal === undefined)
await expect(observation).resolves.toBeUndefined();
else await expect(observation).rejects.toThrow(refusal);
expect(await readFile(`${prefix}/stat`, "utf8")).toContain(
`(${name}) Z `,
);
} finally {
parent.child.stdin?.end("reap");
await parent;
if (zombiePid !== undefined)
await expect(
readFile(`/proc/${String(zombiePid)}/stat`, "utf8"),
).rejects.toMatchObject({ code: "ENOENT" });
}
},
);
it.runIf(process.platform === "linux")(
"does not treat runtime names in another process's arguments as a runtime",
async () => {
const directory = await createTestTempDirectory("rea-runtime-arguments-");
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const observer = new URL(
"../../../scripts/lib/clean-linux-hopper-runtime.mjs",
import.meta.url,
).href;
const code = `
import { mkdir, symlink } from "node:fs/promises";
import { join } from "node:path";
const { assertCleanLinuxHopperRuntime } = await import(${JSON.stringify(observer)});
const entry = join(${JSON.stringify(procRoot)}, String(process.pid));
await mkdir(entry);
await symlink("/proc/" + process.pid + "/cmdline", join(entry, "cmdline"));
await symlink("/proc/" + process.pid + "/comm", join(entry, "comm"));
await assertCleanLinuxHopperRuntime([], ${JSON.stringify(procRoot)});
console.log('clean');
`;
const { stdout } = await execFileAsync(
process.execPath,
[
"--input-type=module",
"-e",
code,
procRoot,
"Xvfb",
"/opt/hopper/bin/Hopper",
"hopper-demo-x11.py",
],
{ timeout: 10_000 },
);
expect(stdout.trim()).toBe("clean");
},
);
it.runIf(process.platform === "linux")(
"distinguishes an ordinary Node script with the helper's filename",
async () => {
const directory = await createTestTempDirectory("rea-runtime-arguments-");
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const script = join(directory, "hopper-demo-x11.py");
const observer = new URL(
"../../../scripts/lib/clean-linux-hopper-runtime.mjs",
import.meta.url,
).href;
const code = `
import { mkdir, symlink } from "node:fs/promises";
import { join } from "node:path";
const entry = join(${JSON.stringify(procRoot)}, String(process.pid));
await mkdir(entry);
await symlink("/proc/" + process.pid + "/cmdline", join(entry, "cmdline"));
await symlink("/proc/" + process.pid + "/comm", join(entry, "comm"));
const { assertCleanLinuxHopperRuntime } = await import(${JSON.stringify(observer)});
await assertCleanLinuxHopperRuntime([], ${JSON.stringify(procRoot)});
console.log('clean');
`;
const { stdout } = await execFileAsync(
process.execPath,
["--input-type=module", "--eval", code, script],
{ timeout: 10_000 },
);
expect(stdout.trim()).toBe("clean");
},
);
const launchReapedOrphan = (runId: string, parentCode: string) => {
// The short-lived Node launcher still exits before observation. A retained
// subreaper adopts its child and waits after owned cleanup, even when the
// container's PID 1 does not reap. It is a sibling of the verifier below.
const reaper = execFileAsync(
"/usr/bin/python3",
[
"-c",
`
import ctypes, json, os, subprocess, sys
libc = ctypes.CDLL(None, use_errno=True)
if libc.prctl(36, 1, 0, 0, 0) != 0: # PR_SET_CHILD_SUBREAPER
raise OSError(ctypes.get_errno(), "PR_SET_CHILD_SUBREAPER")
launcher = subprocess.run(sys.argv[1:], capture_output=True, text=True, check=True)
child = json.loads(launcher.stdout)
try:
print(json.dumps(child), flush=True)
sys.stdin.buffer.read(1)
finally:
os.waitpid(child["pid"], 0)
`,
process.execPath,
"-e",
parentCode,
],
{
env: { ...process.env, REA_PROCESS_RUN_ID: runId },
timeout: 40_000,
},
);
const ready = new Promise<{ pid: number; parentPid: number }>(
(resolve, reject) => {
reaper.child.stdout?.once("data", (data: Buffer) => {
try {
resolve(JSON.parse(data.toString()));
} catch (error) {
reject(error);
}
});
reaper.then(
() => reject(new Error("Fixture reaper exited before readiness")),
reject,
);
},
);
return { reaper, ready };
};
const observeFromSeparateVerifier = async (
runId: string,
launcherPaths: string[],
procRoot: string,
) => {
const observer = new URL(
"../../../scripts/lib/clean-linux-hopper-runtime.mjs",
import.meta.url,
).href;
const verifierRun = new URL(
"../../../scripts/lib/verifier-run.mjs",
import.meta.url,
).href;
const { stdout: observed } = await execFileAsync(
process.execPath,
[
"--input-type=module",
"-e",
`
const { assertCleanLinuxHopperRuntime } = await import(${JSON.stringify(observer)});
const { completeVerifierRun } = await import(${JSON.stringify(verifierRun)});
const { process_lineage } = await completeVerifierRun({
run_id: ${JSON.stringify(runId)}, verifier_pid: process.pid, parent_pid: process.ppid,
});
let refusal;
try { await assertCleanLinuxHopperRuntime(${JSON.stringify(launcherPaths)}, ${JSON.stringify(procRoot)}); }
catch (error) { refusal = error.message; }
console.log(JSON.stringify({ process_lineage, refusal }));
`,
],
{ timeout: 10_000 },
);
return JSON.parse(observed) as {
process_lineage: {
status: string;
descendants: { pid: number }[];
};
refusal?: string;
};
};
it.runIf(process.platform === "linux").each([
["Xvfb-named Node", null, true],
["Python helper", [], true],
["Python helper with -u", ["-u"], true],
[
"Python helper with combined/value options",
["-Bu", "-W", "ignore", "-X", "dev", "--check-hash-based-pycs", "always"],
true,
],
["Python helper after --", ["--"], true],
[
"Python -c script argument",
["-c", "import time; print('ready', flush=True); time.sleep(30)"],
false,
],
["Python -m script argument", ["-m", "unrelated"], false],
["alternate launcher", null, true],
] as const)(
"classifies a reparented %s without signaling it",
async (mode, pythonOptions, retained) => {
const runId = randomUUID();
const childCode =
"process.title = 'Xvfb'; setInterval(() => {}, 1000); process.send('ready');";
const directory = await createTestTempDirectory("rea-clean-runtime-test-");
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const launcher = join(directory, "Hopper");
const alternate = join(directory, "Hopper-alternate");
const helper = join(directory, "hopper-demo-x11.py");
const alternateMode = mode === "alternate launcher";
const pythonMode = pythonOptions !== null;
if (alternateMode) {
await copyFile("/usr/bin/sleep", launcher);
await chmod(launcher, 0o700);
await symlink(launcher, alternate);
}
if (pythonMode) {
await writeFile(
helper,
"import time\nprint('ready', flush=True)\ntime.sleep(30)\n",
);
await copyFile(helper, join(directory, "unrelated.py"));
}
// The fixture must be the cause of a refusal; a retained host runtime
// would otherwise mask a broken classification or executable-identity check.
await assertCleanLinuxHopperRuntime(
alternateMode ? [launcher] : [],
procRoot,
);
const command = alternateMode
? alternate
: pythonMode
? "/usr/bin/python3"
: process.execPath;
const arguments_ = alternateMode
? ["30"]
: pythonMode
? [...pythonOptions, helper]
: ["-e", childCode];
const stdio = alternateMode
? "ignore"
: pythonMode
? ["ignore", "pipe", "ignore"]
: ["ignore", "ignore", "ignore", "ipc"];
const parentCode = `
const { spawn } = require('node:child_process');
const child = spawn(${JSON.stringify(command)}, ${JSON.stringify(arguments_)}, {
detached: true, stdio: ${JSON.stringify(stdio)}, cwd: ${JSON.stringify(directory)}
});
child.once('error', () => process.exit(1));
const ready = () => {
process.stdout.write(JSON.stringify({ pid: child.pid, parentPid: process.pid }));
if (child.connected) child.disconnect();
child.stdout?.destroy();
child.unref();
};
${pythonMode ? "child.stdout.once('data', ready);" : `child.once(${JSON.stringify(alternateMode ? "spawn" : "message")}, ready);`}
`;
const { reaper, ready } = launchReapedOrphan(runId, parentCode);
let orphanPid: number | undefined;
try {
const { pid, parentPid } = await ready;
orphanPid = pid;
const procEntry = join(procRoot, String(pid));
await mkdir(procEntry);
for (const name of ["cmdline", "comm", "exe"])
await symlink(`/proc/${String(pid)}/${name}`, join(procEntry, name));
const { stdout: coordinates } = await execFileAsync("ps", [
"-p",
String(pid),
"-o",
"ppid=,pgid=",
]);
const [currentParent, group] = coordinates
.trim()
.split(/\s+/u)
.map(Number);
expect(currentParent).not.toBe(parentPid);
expect(currentParent).toBe(reaper.child.pid);
expect(group).toBe(pid);
if (alternateMode) {
const commandLine = await readFile(
`/proc/${String(pid)}/cmdline`,
"utf8",
);
expect(commandLine.split("\0")[0]).toBe(alternate);
await unlink(alternate);
}
if (pythonMode) {
const commandLine = await readFile(
`/proc/${String(pid)}/cmdline`,
"utf8",
);
expect(
commandLine.split("\0").slice(0, pythonOptions.length + 2),
).toEqual([command, ...pythonOptions, helper]);
}
const { process_lineage: lineage, refusal } =
await observeFromSeparateVerifier(
runId,
alternateMode ? [launcher] : [],
procRoot,
);
expect(lineage.status).toBe("verified");
expect(lineage.descendants.some((p) => p.pid === pid)).toBe(false);
if (retained)
expect(refusal).toBe(
"Clean Linux runner retained a Hopper demo process",
);
else expect(refusal).toBeUndefined();
// Observation must not terminate the fixture, even though it resembles Xvfb.
const { stdout: alive } = await execFileAsync("ps", [
"-p",
String(pid),
"-o",
"stat=",
]);
expect(alive.trim()).not.toMatch(/^Z/u);
} finally {
try {
if (orphanPid !== undefined) {
const cleanup = await cleanupOwnedProcessGroup({
runId,
leaderPid: orphanPid,
processGroupId: orphanPid,
});
expect(cleanup.cleaned, JSON.stringify(cleanup)).toBe(true);
}
} finally {
reaper.child.stdin?.end("reap");
await reaper;
if (orphanPid !== undefined)
await expect(
readFile(`/proc/${String(orphanPid)}/stat`, "utf8"),
).rejects.toMatchObject({ code: "ENOENT" });
}
}
},
);
@@ -140,8 +140,6 @@ const managedTranscript = () => {
javascriptSha256: "",
javascriptShapeLeft: "",
javascriptShapeRight: "",
notesElectron: "",
notesElectronSha256: "",
managed: target.path,
managedSha256: target.sha256,
});
@@ -74,7 +74,11 @@ it
if (alternateMode) {
const processEntry = join(procRoot, String(pid));
await mkdir(processEntry);
await symlink(`/proc/${String(pid)}/exe`, join(processEntry, "exe"));
for (const name of ["cmdline", "comm", "exe"])
await symlink(
`/proc/${String(pid)}/${name}`,
join(processEntry, name),
);
}
const { stdout: coordinates } = await execFileAsync("ps", [
"-p",