Compare commits

..
168 changed files with 2412 additions and 6189 deletions
-6
View File
@@ -25,12 +25,6 @@ extraction have independent prerequisites; one does not require the other.
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
REA retains at most 8 MiB of combined stdout and stderr per command. Exceeding
that diagnostic budget stops analysis with an explicit truncated-output error;
child process limits do not bound parent-side pipe retention. Completed commands
drain their output streams before recording diagnostics. Failed worker cleanup
retains the worker and workspace for retry before another request or at shutdown.
Unblob needs the external extractor for the selected format on its inherited
`PATH`: for example, `debugfs` for ext2/ext3/ext4. Its default Landlock sandbox
and skip rules stay enabled. Sandbox startup failures remain actionable errors;
-14
View File
@@ -38,20 +38,6 @@ Inspect separately obtained components explicitly and preserve their identities.
A valid implementation marker establishes a candidate native boundary, not
recovered native semantics or runtime behavior.
### Resource admission
The static provider admits regular input files up to 128 MiB before retaining
their bytes. Reads remain bounded if a file grows after admission. Metadata,
CIL decoding, and projected inspection records have a separate conservative
64 MiB representation budget; shared heap strings are cached, but repeated
output occurrences still consume that budget. These are inspection resource
policies, not PE format limits or a hard bound on total V8 heap use.
Input overflow returns a resource constraint. Decode or projection overflow
reports `resource-limit` with unavailable or partial coverage; omitted facts do
not become empty observations. Ambiguous PE section mappings are rejected
rather than interpreted according to section order.
## Analysis objective
REA's managed-code track is intended to answer five different questions without
-4
View File
@@ -187,10 +187,6 @@ Every capture requires `truncation_details`, with separate accounting for:
input, so raw omissions also limit rendered coverage.
- `filesystem_before` and `filesystem_after`: file-count/depth limits,
enumeration failures, whole-file hash budget and bytes successfully hashed.
Directory names are read incrementally within the remaining entry capacity.
When a directory exceeds that capacity, the retained subset follows filesystem
enumeration order and is then sorted; it is explicitly incomplete and cannot
establish that an omitted path is absent.
Each retained regular file without a digest has an aliased path, size,
remaining budget and reason: `file_bytes_budget`,
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is
-24
View File
@@ -187,30 +187,6 @@ The loopback OpenAI completions/SSE custom model declares a one-million-token
context and synthetic usage. Live xAI APIs, native Windows and default OS-home
skill discovery remain unverified.
`verify:commandcode-client` requires installed Command Code (verified with npm
1.79.2); select it with `REA_VERIFY_COMMANDCODE_COMMAND`. Auth, MCP tokens and
sessions use the actual OS home, so provision a disposable POSIX account rather
than overriding `HOME`. Set `REA_VERIFY_COMMANDCODE_ACCOUNT_HOME` to that account's
actual home and create `.rea-client-verification` there with exactly
`Disposable REA client verification account` followed by a newline. The lane
owns that account's `.commandcode` configuration and shared REA skill. Build the
runtime as the checkout owner, then run `node scripts/verify-commandcode-client.mjs`
as the disposable account; the npm shortcut also needs a writable checkout.
The verifier does not create accounts or install clients. It guards setup
targets, preserves an unrelated registration and backups, checks idempotence,
and verifies native default shared-skill discovery and complete activation.
Default deferred-schema delivery stays enabled: the native prompt advertises
every REA tool, exact-name `search_tools` returns every input schema, and a native
shell query recovers the saved catalog's count and digest when the client spills
it. Actual analysis of a Unicode-path fixture delivers full named-schema Evidence
to the next model request. Use `-- chat` for ordinary chat or
`REA_VERIFY_RUNTIME_ROOT` for a production-only installed package. The loopback
OpenAI completions/SSE model is a keyless BYOK endpoint declaring a million-token
context with synthetic usage. A synthetic account-key value satisfies the
client's print-mode gate; native local-only mode refuses hosted API calls.
Updates, telemetry and cron are disabled. Hosted authentication, live models
and Windows remain unverified.
`verify:omp-client` requires installed OMP (verified with the official Linux x64
18.8.7 binary); select it with `REA_VERIFY_OMP_COMMAND`. This optional POSIX lane
isolates the default agent, global config and XDG roots, guards setup targets,
-1
View File
@@ -188,7 +188,6 @@
"verify:gemini-client": "npm run build:cached && node scripts/verify-gemini-client.mjs",
"verify:opencode-client": "npm run build:cached && node scripts/verify-opencode-client.mjs",
"verify:grok-client": "npm run build:cached && node scripts/verify-grok-client.mjs",
"verify:commandcode-client": "npm run build:cached && node scripts/verify-commandcode-client.mjs",
"verify:omp-client": "npm run build:cached && node scripts/verify-omp-client.mjs",
"verify:copilot-client": "npm run build:cached && node scripts/verify-copilot-client.mjs",
"verify:claude-client": "npm run build:cached && node scripts/verify-claude-client.mjs",
-452
View File
@@ -1,452 +0,0 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { homedir } from "node:os";
import { isAbsolute, join, relative } from "node:path";
import { fileURLToPath } from "node:url";
import Ajv2020 from "ajv/dist/2020.js";
import {
TOOL_CONTRACTS,
toolContract,
} from "../dist/contracts/toolContracts.js";
import { exec } from "./lib/verify-package-core.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
assert(
process.platform !== "win32",
"This verifier requires a disposable POSIX account; Windows is unverified",
);
const account = process.env.REA_VERIFY_COMMANDCODE_ACCOUNT_HOME;
assert(
account && isAbsolute(account) && account === homedir(),
"Run as a disposable account whose actual OS home matches REA_VERIFY_COMMANDCODE_ACCOUNT_HOME; do not override HOME",
);
assert.equal(
await readFile(join(account, ".rea-client-verification"), "utf8"),
"Disposable REA client verification account\n",
);
const repo = fileURLToPath(new URL("..", import.meta.url));
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
const command = process.env.REA_VERIFY_COMMANDCODE_COMMAND ?? "cmd";
const mode = process.argv[2] ?? "call";
assert(
["chat", "call"].includes(mode),
"Usage: verify-commandcode-client.mjs [chat|call]",
);
const verifier = createVerifierRun();
const lab = await mkdtemp(join(account, "rea-commandcode-client-"));
const workspace = join(lab, "workspace");
const target = join(workspace, "fixture α测试");
const profile = join(account, ".commandcode");
const configPath = join(profile, "mcp.json");
const skillDirectory = join(
account,
".agents",
"skills",
"reverse-engineer-anything",
);
await Promise.all([
mkdir(target, { recursive: true }),
mkdir(profile, { recursive: true }),
mkdir(join(lab, "tmp")),
]);
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
await writeFile(
join(target, "app.js"),
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
);
const environment = {
PATH: process.env.PATH,
NODE_OPTIONS: process.env.NODE_OPTIONS,
USERPROFILE: account,
TMPDIR: join(lab, "tmp"),
CMD_LOCAL_ONLY: "1",
// Native print mode checks for an account key even with a keyless BYOK model.
// This synthetic value is never a real credential; local-only refuses hosted API calls.
COMMAND_CODE_API_KEY: "local-fixture-only",
COMMANDCODE_SKIP_UPDATES: "1",
COMMANDCODE_DISABLE_CRON: "1",
OTEL_SDK_DISABLED: "true",
REA_PROCESS_RUN_ID: verifier.run_id,
};
const expectedNames = TOOL_CONTRACTS.map(
(tool) => "mcp__rea__" + tool.name,
).sort();
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
const textValues = (value) =>
typeof value === "string"
? [value]
: value !== null && typeof value === "object"
? Object.values(value).flatMap(textValues)
: [];
let clientVersion;
let expectedSkill;
let inventoryChecked = false;
let skillLoaded = false;
let schemasChecked = 0;
let catalogArtifact;
let catalogDigest;
let catalogProjection;
let artifactProjectionSeen = false;
let evidenceSeen = false;
let evidenceBytes;
let finalSeen = false;
let stage = 0;
let failure;
const call = (name, args) => ({
role: "assistant",
content: null,
tool_calls: [
{
id: `call_commandcode_${++stage}`,
type: "function",
function: { name, arguments: JSON.stringify(args) },
},
],
});
const validateCatalog = async (text) => {
const entries = [
...text.matchAll(
/^### (mcp__rea__[A-Za-z0-9_]+)\n[\s\S]*?\nParameters:\n```json\n([\s\S]*?)\n```/gm,
),
];
assert.deepEqual(
entries.map((entry) => entry[1]).sort(),
expectedNames,
"Native exact-name search must return every REA input schema",
);
for (const entry of entries) {
const schema = JSON.parse(entry[2]);
assert(
schemaValidator.validateSchema(schema),
`${entry[1]}: invalid native schema: ${JSON.stringify(schemaValidator.errors)}`,
);
}
schemasChecked = entries.length;
catalogDigest = createHash("sha256").update(text).digest("hex");
await writeFile(join(lab, "native-catalog.md"), text);
};
const fixture = await createOpenAiModelFixture({
directory: lab,
prefix: "commandcode",
onRequest: async ({ body, tools }) => {
assert(tools.some((tool) => tool.name === "search_tools"));
const strings = textValues(body.messages);
if (!inventoryChecked) {
const names = [
...new Set(
strings.flatMap(
(text) => text.match(/mcp__rea__[A-Za-z0-9_]+/g) ?? [],
),
),
].sort();
assert.deepEqual(
names,
expectedNames,
"Default native deferred-tool prompt must advertise the complete REA inventory",
);
assert.equal(
tools.filter((tool) => tool.name.startsWith("mcp__rea__")).length,
0,
"Keep native lazy schema delivery enabled",
);
assert(
tools
.find((tool) => tool.name === "activate_skill")
?.parameters.properties.name.enum.includes(
"reverse-engineer-anything",
),
);
inventoryChecked = true;
}
if (mode === "chat") {
assert.equal(
body.messages.filter((message) => message.role === "tool").length,
0,
);
finalSeen = true;
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
}
if (stage === 0)
return call("activate_skill", { name: "reverse-engineer-anything" });
assert(
strings.some((text) => text.includes(expectedSkill)),
"The model must receive the complete activated skill body",
);
skillLoaded = true;
const last = body.messages
.filter((message) => message.role === "tool")
.at(-1)?.content;
assert.equal(typeof last, "string");
if (stage === 1)
return call("search_tools", {
query: "select:" + expectedNames.join(","),
});
if (stage === 2) {
const spill = last.match(
/\[full output saved to: (.+?) — read it with read_file/,
);
if (spill) {
catalogArtifact = spill[1];
const child = relative(environment.TMPDIR, catalogArtifact);
assert(
isAbsolute(catalogArtifact) &&
child &&
!child.startsWith("..") &&
!isAbsolute(child),
"Native spill must remain inside this run's temporary directory",
);
const text = await readFile(catalogArtifact, "utf8");
await validateCatalog(text);
catalogProjection = { schemas: schemasChecked, sha256: catalogDigest };
const query =
"const fs=require('node:fs'),crypto=require('node:crypto');const text=fs.readFileSync(process.argv[1],'utf8');const schemas=[...text.matchAll(/^### (mcp__rea__[A-Za-z0-9_]+)\\n/gm)].length;console.log(JSON.stringify({rea_catalog:{schemas,sha256:crypto.createHash('sha256').update(text).digest('hex')}}));";
return call("shell_command", {
command: process.execPath,
args: ["-e", query, catalogArtifact],
});
}
await validateCatalog(last);
return call("mcp__rea__analyze_javascript_application", {
input_path: target,
format: "directory",
});
}
if (catalogArtifact && !artifactProjectionSeen) {
const line = last
.split("\n")
.find((item) => item.startsWith('{"rea_catalog":'));
assert(line, "Native shell must return the selected saved-catalog facts");
assert.deepEqual(JSON.parse(line).rea_catalog, catalogProjection);
artifactProjectionSeen = true;
return call("mcp__rea__analyze_javascript_application", {
input_path: target,
format: "directory",
});
}
const evidence = toolContract(
"analyze_javascript_application",
).outputSchema.parse(JSON.parse(last));
assert.equal(evidence.subject.local_path, target);
const shapes = evidence.normalized_result.graph.nodes
.flatMap((node) => node.observations)
.map((observation) => observation.properties)
.filter(
(properties) =>
properties.semantic_role === "export-return-shapes" &&
properties.exported_name === "result",
);
assert(
shapes.length > 0 &&
shapes.every(
(shape) => shape.return_shape_coverage?.projection_complete,
),
);
const fields = shapes.flatMap((shape) =>
shape.static_return_shapes.flatMap((item) => item.fields),
);
assert(
fields.some(
(field) =>
field.path === "/clientCompatibility" &&
field.state === "literal" &&
field.value === "REA_CLIENT_FIXTURE",
),
);
assert(
fields.some(
(field) =>
field.path === "/count" &&
field.state === "literal" &&
field.value === 7,
),
);
evidenceBytes = Buffer.byteLength(last);
await writeFile(join(lab, "native-evidence.json"), last);
evidenceSeen = true;
finalSeen = true;
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
},
});
await writeFile(
join(profile, "providers.json"),
JSON.stringify({
provider: {
"rea-local": {
baseURL: fixture.baseUrl,
apiKey: false,
models: {
"rea-client-fixture": { contextWindow: 1_000_000, maxOutput: 4096 },
},
},
},
}),
);
const original =
JSON.stringify(
{
mcpServers: { other: { command: "unrelated-server", enabled: false } },
preserved: "keep unrelated settings",
},
null,
2,
) + "\n";
await writeFile(configPath, original);
const rea = (args) =>
exec(
process.execPath,
[join(runtimeRoot, "scripts/rea.mjs"), ...args, "--json"],
{ env: environment, cwd: workspace, timeout: 60_000 },
);
try {
clientVersion = (
await exec(command, ["--version"], {
env: environment,
cwd: workspace,
timeout: 30_000,
})
).stdout.trim();
for (const args of [
["setup", "--client", "commandcode", "--dry-run"],
["setup", "--client", "commandcode", "--yes"],
["doctor", "--client", "commandcode", "--skill"],
]) {
const response = await rea(args);
await writeFile(
join(lab, `${args[0]}${args.includes("--dry-run") ? "-plan" : ""}.json`),
response.stdout,
);
const result = JSON.parse(response.stdout);
if (args[0] === "doctor") assert.equal(result.healthy, true);
else
assert.equal(
result.status,
args.includes("--dry-run") ? "planned" : "ready",
);
if (args.includes("--dry-run")) {
assert(
result.plannedActions.some((action) => action.target === configPath),
);
assert(
result.plannedActions.every(
(action) =>
action.target === configPath ||
action.target === skillDirectory ||
action.target.startsWith(skillDirectory + "/"),
),
"Refuse setup writes outside the disposable account's configuration and skill",
);
}
}
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
const configured = await readFile(configPath, "utf8");
assert.equal(JSON.parse(configured).preserved, "keep unrelated settings");
assert.equal(
JSON.parse(configured).mcpServers.other.command,
"unrelated-server",
);
const repeated = JSON.parse(
(await rea(["setup", "--client", "commandcode", "--yes"])).stdout,
);
assert.deepEqual(repeated.appliedActions, []);
assert.equal(await readFile(configPath, "utf8"), configured);
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
expectedSkill = (await readFile(join(skillDirectory, "SKILL.md"), "utf8"))
.replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n/, "")
.trim();
const execution = exec(
command,
[
"-p",
mode === "chat"
? "Say hello briefly."
: `Use the REA skill to analyze ${target}.`,
"-m",
"rea-local/rea-client-fixture",
"--local-only",
"--skip-onboarding",
"--trust",
"--output-format",
"json",
"--max-turns",
"8",
"--yolo",
],
{
env: environment,
cwd: workspace,
timeout: 120_000,
maxBuffer: 10 * 1024 * 1024,
},
);
execution.child.stdin.end();
const response = await execution;
await writeFile(join(lab, `${mode}.stdout`), response.stdout);
await writeFile(join(lab, `${mode}.stderr`), response.stderr);
assert.equal(fixture.failure, undefined);
const events = response.stdout
.trim()
.split("\n")
.map((line) => JSON.parse(line));
assert(
events.some(
(event) =>
event.type === "result" &&
event.subtype === "success" &&
event.finalText === "REA_CLIENT_COMPATIBILITY_OK",
),
);
assert(inventoryChecked && finalSeen);
if (mode === "call")
assert(
skillLoaded &&
schemasChecked === expectedNames.length &&
evidenceSeen &&
(!catalogArtifact || artifactProjectionSeen),
);
} catch (cause) {
failure = cause;
if (typeof cause?.stdout === "string")
await writeFile(join(lab, `${mode}.stdout`), cause.stdout);
if (typeof cause?.stderr === "string")
await writeFile(join(lab, `${mode}.stderr`), cause.stderr);
} finally {
await fixture.close();
const receipt = {
client: "Command Code",
clientVersion,
mode,
passed: failure === undefined,
artifacts: lab,
runtimeRoot,
host: {
platform: process.platform,
architecture: process.arch,
node: process.versions.node,
},
modelFixture:
"loopback OpenAI completions/SSE, keyless BYOK, synthetic account-key gate and usage; no hosted authentication or live-model claim",
discovery:
"actual disposable OS account home; native user MCP configuration and default shared skill discovery; Windows unverified",
interface:
"default deferred schemas, complete exact-name search, native saved-catalog query when needed, actual JavaScript MCP call",
inventoryChecked,
catalogSize: inventoryChecked ? expectedNames.length : 0,
schemasChecked,
skillLoaded,
catalogArtifact,
catalogDigest,
artifactProjectionSeen,
evidenceSeen,
evidenceBytes,
finalSeen,
requests: fixture.requests,
probes: fixture.probes,
fixtureFailure: fixture.failure,
verifier: await completeVerifierRun(verifier),
};
await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
console.log(JSON.stringify(receipt));
}
if (failure !== undefined) throw failure;
-2
View File
@@ -78,14 +78,12 @@ export const parseApktoolVersionOutput = (
export const inspectApktoolClient = async (
selection: ApktoolCommandSelection,
signal?: AbortSignal,
environment: Readonly<Record<string, string | undefined>> = process.env,
): Promise<ApktoolClientIdentity> => {
try {
const { stdout } = await execFileOutput(selection.command, ["--version"], {
timeout: VERSION_TIMEOUT_MS,
stopSignal: "SIGTERM",
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
env: { ...process.env, ...environment },
...(signal === undefined ? {} : { signal }),
});
return parseApktoolVersionOutput(stdout);
+130 -324
View File
@@ -1,20 +1,18 @@
import { createHash, randomUUID } from "node:crypto";
import { createWriteStream, constants as fsConstants } from "node:fs";
import { createHash } from "node:crypto";
import { createReadStream, constants as fsConstants } from "node:fs";
import {
access,
chmod,
copyFile,
mkdir,
mkdtemp,
opendir,
open,
readdir,
rm,
stat,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { Transform, type TransformCallback } from "node:stream";
import { pipeline } from "node:stream/promises";
import type { FileHandle } from "node:fs/promises";
import type {
ExecutionOptions,
@@ -35,19 +33,11 @@ import {
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { err, ok, type Result } from "../domain/result.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
openRegularFile,
sameRegularFileState,
} from "../filesystem/RegularFile.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import {
ApktoolConfigurationFailure,
apktoolConfigurationAvailability,
@@ -74,22 +64,20 @@ const DECODE_MAX_OUTPUT_BYTES = 4 * 1024 * 1024;
const MANIFEST_MAX_BYTES = 512 * 1024;
const STRINGS_XML_MAX_BYTES = 8 * 1024 * 1024;
const MAX_STRINGS = 2_000;
const MAX_DECODED_ENTRIES_VISITED = 100_000;
const MAX_DECODED_FILES_COUNTED = 100_000;
const STDERR_EXCERPT_BYTES = 512;
const throwIfCancelled = (
operation: string,
signal: AbortSignal | undefined,
): void => {
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
/** Stream a local file digest without retaining large APKs in memory. */
const sha256File = async (path: string): Promise<string> => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
};
/** Read at most the byte budget plus one byte, before decoding UTF-8. */
const readBoundedText = async (
path: string,
budget: number,
operation: string,
signal: AbortSignal | undefined,
): Promise<{ text: string | null; truncated: boolean }> => {
const file = await open(path, "r").catch((cause: unknown) => {
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return null;
@@ -97,11 +85,9 @@ const readBoundedText = async (
});
if (file === null) return { text: null, truncated: false };
try {
throwIfCancelled(operation, signal);
const bytes = Buffer.alloc(budget + 1);
let length = 0;
while (length < bytes.length) {
throwIfCancelled(operation, signal);
const result = await file.read(
bytes,
length,
@@ -111,7 +97,6 @@ const readBoundedText = async (
if (result.bytesRead === 0) break;
length += result.bytesRead;
}
throwIfCancelled(operation, signal);
const truncated = length > budget;
// A prefix may end within a UTF-8 character; omit those trailing bytes.
const decoder = new TextDecoder("utf-8");
@@ -140,141 +125,97 @@ const runApktool = async (
arguments_: readonly string[],
limits: DecodeLimits,
signal: AbortSignal | undefined,
environment: Readonly<Record<string, string | undefined>>,
onCleanupOwner: (owner: ProviderProcessSupervisor) => void,
): Promise<Result<{ stdout: string; stderr: string }, AnalysisError>> => {
if (signal?.aborted === true)
return err(new AnalysisCancelledError(operation));
try {
const output = await runOwnedCommand(
{
command,
arguments: arguments_,
runId: randomUUID(),
hostEnvironment: { ...process.env, ...environment },
},
{
timeoutMs: limits.timeoutMs,
diagnosticBytes: limits.maxOutputBytes,
},
signal === undefined ? {} : { signal },
return ok(
await execFileOutput(command, arguments_, {
timeout: limits.timeoutMs,
maxBuffer: limits.maxOutputBytes,
stopSignal: "SIGTERM",
...(signal === undefined ? {} : { signal }),
}),
);
return ok({ stdout: output.stdout.text, stderr: output.stderr.text });
} catch (cause) {
if (!(cause instanceof OwnedCommandFailure))
if (signal?.aborted || execFileOutputFailure(cause)?.code === "ABORT_ERR")
return err(new AnalysisCancelledError(operation, { cause }));
const failure = execFileOutputFailure(cause);
if (failure === undefined)
return err(
new AnalysisProtocolError(`apktool failed for ${operation}`, { cause }),
);
if (cause.cleanupOwner !== undefined) onCleanupOwner(cause.cleanupOwner);
const output = cause.snapshot;
const capturedOutput =
output === null
? undefined
: {
stdout: output.stdout.text,
stderr: output.stderr.text,
truncated: output.diagnosticTruncated === true,
stdout_bytes: output.stdout.bytes,
stderr_bytes: output.stderr.bytes,
exit_code: output.exitCode ?? null,
signal: output.signal ?? null,
};
const cleanup =
cause.cleanupFailure === null
? undefined
: {
reason: cause.cleanupFailure,
resources: cause.resources,
};
if (cause.reason === "cancelled")
return err(
new AnalysisCancelledError(operation, {
cause,
...(cleanup === undefined ? {} : { cleanup }),
...(capturedOutput === undefined ? {} : { capturedOutput }),
}),
);
if (cause.reason === "timeout")
return err(
new AnalysisTimeoutError(operation, limits.timeoutMs, {
cause,
...(cleanup === undefined ? {} : { cleanup }),
...(capturedOutput === undefined ? {} : { capturedOutput }),
}),
);
if (cause.reason === "output-limit")
if (failure.outputTruncated)
return err(
new AnalysisResourceConstraintError(
operation,
"memory",
`apktool output exceeded the ${String(limits.maxOutputBytes)}-byte capture budget`,
{ max_output_bytes: limits.maxOutputBytes },
{
cause,
...(cleanup === undefined ? {} : { cleanup }),
...(capturedOutput === undefined ? {} : { capturedOutput }),
},
{ cause },
),
);
const excerpt = failure.stderr.trim().slice(0, STDERR_EXCERPT_BYTES);
if (failure.killed)
return err(
new AnalysisTimeoutError(operation, limits.timeoutMs, { cause }),
);
return err(
new AnalysisUnsupportedTargetError(
operation,
command,
output?.stderr.text.trim().slice(0, STDERR_EXCERPT_BYTES) ||
`apktool exited with code ${String(output?.exitCode ?? "unknown")} for ${operation}`,
{
cause,
...(cleanup === undefined ? {} : { cleanup }),
...(capturedOutput === undefined ? {} : { capturedOutput }),
},
excerpt === ""
? `apktool exited with code ${String(failure.code)} for ${operation}`
: excerpt,
{ cause },
),
);
}
};
/** Count files while bounding streamed directory-entry traversal. */
/** Count files in the decoded workspace with a hard ceiling. */
const countDecodedFiles = async (
root: string,
signal: AbortSignal | undefined,
operation: string,
): Promise<{
count: number;
capped: boolean;
locales: readonly string[];
}> => {
): Promise<{ count: number; capped: boolean }> => {
let count = 0;
let capped = false;
let visited = 0;
const directories = [root];
const locales: string[] = [];
while (directories.length > 0 && !capped) {
throwIfCancelled(operation, signal);
const directory = directories.pop()!;
for await (const entry of await opendir(directory)) {
throwIfCancelled(operation, signal);
if (visited >= MAX_DECODED_ENTRIES_VISITED) {
const walk = async (directory: string): Promise<void> => {
const entries = await readdir(directory, { withFileTypes: true });
for (const entry of entries) {
if (count >= MAX_DECODED_FILES_COUNTED) {
capped = true;
break;
return;
}
visited += 1;
const path = join(directory, entry.name);
if (entry.isDirectory()) {
directories.push(path);
if (directory === join(root, "res")) {
const match = /^values-(.+)$/u.exec(entry.name);
if (
match !== null &&
(await access(join(path, "strings.xml")).then(
() => true,
() => false,
))
)
locales.push(match[1]!);
}
} else if (entry.isFile()) count += 1;
if (entry.isDirectory()) await walk(path);
else if (entry.isFile()) count += 1;
}
};
await walk(root);
return { count, capped };
};
/** Locale codes present as res/values-<locale>/strings.xml. */
const listStringLocales = async (
workspace: string,
): Promise<readonly string[]> => {
const res = join(workspace, "res");
const entries = await readdir(res, { withFileTypes: true }).catch(() => []);
const locales: string[] = [];
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const match = /^values-(.+)$/u.exec(entry.name);
if (match === null) continue;
const strings = join(res, entry.name, "strings.xml");
if (
await access(strings).then(
() => true,
() => false,
)
)
locales.push(match[1]!);
}
return { count, capped, locales: locales.sort() };
return locales.sort();
};
export interface ApktoolProviderOptions {
@@ -294,14 +235,9 @@ export class ApktoolProvider {
private readonly workspaces = new Set<string>();
private readonly removeWorkspace: (path: string) => Promise<void>;
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
readonly #shutdown = new AbortController();
readonly #active = new Set<Promise<void>>();
#closing = false;
#closePromise: Promise<void> | undefined;
constructor(options: ApktoolProviderOptions = {}) {
this.environment = snapshotEnvironment(options.environment ?? process.env);
this.environment = options.environment ?? process.env;
this.removeWorkspace =
options.removeWorkspace ??
((path) => rm(path, { recursive: true, force: true }));
@@ -312,25 +248,12 @@ export class ApktoolProvider {
}
/** Probe the selected apktool launcher; decodes nothing. */
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability> {
if (this.#closing)
return Promise.reject(
new AnalysisCancelledError("inspect_apktool_client"),
);
const operationSignal =
signal === undefined
? this.#shutdown.signal
: AbortSignal.any([signal, this.#shutdown.signal]);
return this.#track(this.#inspectAvailability(operationSignal));
}
async #inspectAvailability(
signal: AbortSignal,
async inspectAvailability(
signal?: AbortSignal,
): Promise<ProviderAvailability> {
const selection = this.selection();
try {
await inspectApktoolClient(selection, signal, this.environment);
throwIfCancelled("inspect_apktool_client", signal);
await inspectApktoolClient(selection, signal);
return {
status: "available",
code: null,
@@ -349,44 +272,30 @@ export class ApktoolProvider {
/** Retain failed cleanup ownership and retry it on close. */
async close(): Promise<void> {
this.#closing = true;
this.#shutdown.abort();
this.#closePromise ??= this.#drainAndClean().catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
const failures: unknown[] = [];
for (const path of this.workspaces) {
try {
await this.removeWorkspace(path);
this.workspaces.delete(path);
} catch (cause) {
failures.push(cause);
}
}
if (failures.length > 0)
throw new AggregateError(failures, "Apktool workspace cleanup failed");
}
execute(
async execute(
request: ApktoolRequest,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
if (this.#closing)
return Promise.resolve(
err(new AnalysisCancelledError(request.operation)),
);
const signal =
options?.signal === undefined
? this.#shutdown.signal
: AbortSignal.any([options.signal, this.#shutdown.signal]);
return this.#track(this.#execute(request, signal));
}
async #execute(
request: ApktoolRequest,
signal: AbortSignal,
): Promise<Result<AnalysisExecution, AnalysisError>> {
const signal = options?.signal;
if (signal?.aborted === true)
return err(new AnalysisCancelledError(request.operation));
const selection = this.selection();
let identity: ApktoolClientIdentity;
try {
identity = await inspectApktoolClient(
selection,
signal,
this.environment,
);
identity = await inspectApktoolClient(selection, signal);
} catch (cause) {
if (cause instanceof AnalysisCancelledError) return err(cause);
if (cause instanceof ApktoolConfigurationFailure)
@@ -407,8 +316,6 @@ export class ApktoolProvider {
),
);
}
if (signal.aborted)
return err(new AnalysisCancelledError(request.operation));
const client = {
command: selection.command,
command_source: selection.commandSource,
@@ -430,47 +337,6 @@ export class ApktoolProvider {
);
}
#track<T>(operation: Promise<T>): Promise<T> {
const settled = operation.then(
() => undefined,
() => undefined,
);
this.#active.add(settled);
void settled.then(() => this.#active.delete(settled));
return operation;
}
async #drainAndClean(): Promise<void> {
await Promise.all(this.#active);
const failures: unknown[] = [];
for (const [owner, root] of this.#pendingCommandOwners) {
try {
const stopped = await owner.stop();
if (stopped.status === "incomplete") throw new Error(stopped.reason);
this.#pendingCommandOwners.delete(owner);
} catch (cause) {
failures.push(
new Error(
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
{ cause },
),
);
}
}
if (failures.length > 0)
throw new AggregateError(failures, "Apktool process cleanup failed");
for (const path of this.workspaces) {
try {
await this.removeWorkspace(path);
this.workspaces.delete(path);
} catch (cause) {
failures.push(cause);
}
}
if (failures.length > 0)
throw new AggregateError(failures, "Apktool workspace cleanup failed");
}
private async decodeResources(
operation: "decode_android_resources",
selection: ApktoolCommandSelection,
@@ -487,6 +353,28 @@ export class ApktoolProvider {
},
signal: AbortSignal | undefined,
): Promise<Result<AnalysisExecution, AnalysisError>> {
const info = await stat(input.path).then(
(value) => value,
() => null,
);
if (info === null || !info.isFile())
return err(
new AnalysisInputError(operation, {
cause: new Error(
`The selected APK is not a readable file: ${input.path}`,
),
}),
);
const readable = await access(input.path, fsConstants.R_OK).then(
() => true,
() => false,
);
if (!readable)
return err(
new AnalysisInputError(operation, {
cause: new Error(`The selected APK is not readable: ${input.path}`),
}),
);
const root = await mkdtemp(join(tmpdir(), "rea-apktool-"));
this.workspaces.add(root);
const workspace = join(root, "decoded");
@@ -496,10 +384,6 @@ export class ApktoolProvider {
// The workspace is removed before the result is returned, so an
// unremovable workspace is reported instead of silently abandoned.
const removeWorkspace = async (): Promise<void> => {
if ([...this.#pendingCommandOwners.values()].includes(root)) {
cleanupFailure = `The decode workspace is retained until its apktool process stops: ${root}`;
return;
}
try {
await this.removeWorkspace(root);
this.workspaces.delete(root);
@@ -510,78 +394,20 @@ export class ApktoolProvider {
const failureWithCleanup = (failure: AnalysisError): AnalysisError =>
cleanupFailure === null
? failure
: analysisErrorWithCleanupFailure(
failure,
new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
[root],
{ reason: cleanupFailure },
{ operation },
),
operation,
);
let source: FileHandle | undefined;
let sourceAdmissionFailed = false;
: failure instanceof AnalysisCancelledError
? new AnalysisCancelledError(operation, {
cause: failure,
cleanup: { reason: cleanupFailure, resources: [root] },
})
: new AnalysisProtocolError(failure.message, {
cause: failure,
cleanup: { reason: cleanupFailure, resources: [root] },
});
try {
throwIfCancelled(operation, signal);
sourceAdmissionFailed = true;
source = await openRegularFile(input.path, {
symlinks: "follow",
...(signal === undefined ? {} : { signal }),
});
const sourceInfo = await source.stat();
sourceAdmissionFailed = false;
const sourceSize = sourceInfo.size;
let copiedBytes = 0;
const sourceHash = createHash("sha256");
const boundedSource = new Transform({
transform(
chunk: Buffer,
_encoding: BufferEncoding,
callback: TransformCallback,
) {
copiedBytes += chunk.length;
if (copiedBytes > sourceSize) {
callback(
new AnalysisInputError(operation, {
cause: new Error(
`The selected APK changed while being copied: ${input.path}`,
),
}),
);
return;
}
sourceHash.update(chunk);
callback(null, chunk);
},
});
await pipeline(
source.createReadStream({
start: 0,
end: sourceSize,
autoClose: false,
...(signal === undefined ? {} : { signal }),
}),
boundedSource,
createWriteStream(snapshot, { flags: "wx", mode: 0o600 }),
signal === undefined ? {} : { signal },
);
const finalSourceInfo = await source.stat();
if (
copiedBytes !== sourceSize ||
!sameRegularFileState(sourceInfo, finalSourceInfo)
)
throw new AnalysisInputError(operation, {
cause: new Error(
`The selected APK changed while being copied: ${input.path}`,
),
});
const sha256 = sourceHash.digest("hex");
await source.close();
source = undefined;
await copyFile(input.path, snapshot, fsConstants.COPYFILE_EXCL);
await chmod(snapshot, 0o400);
throwIfCancelled(operation, signal);
const snapshotInfo = await stat(snapshot);
const sha256 = await sha256File(snapshot);
await mkdir(workspace);
await mkdir(framework);
const decoded = await runApktool(
@@ -602,19 +428,14 @@ export class ApktoolProvider {
maxOutputBytes: DECODE_MAX_OUTPUT_BYTES,
},
signal,
this.environment,
(owner) => this.#pendingCommandOwners.set(owner, root),
);
if (!decoded.ok) {
await removeWorkspace();
return err(failureWithCleanup(decoded.error));
}
throwIfCancelled(operation, signal);
const yml = await readBoundedText(
join(workspace, "apktool.yml"),
MANIFEST_MAX_BYTES,
operation,
signal,
);
if (yml.text === null || yml.truncated) {
throw new AnalysisProtocolError(
@@ -625,12 +446,11 @@ export class ApktoolProvider {
const manifestRead = await readBoundedText(
join(workspace, "AndroidManifest.xml"),
MANIFEST_MAX_BYTES,
operation,
signal,
);
const manifest = manifestRead.text;
const packageName =
manifest === null ? null : parseManifestPackage(manifest);
const locales = await listStringLocales(workspace);
const stringsDirectory =
input.locale === undefined
? "values"
@@ -639,8 +459,6 @@ export class ApktoolProvider {
? await readBoundedText(
join(workspace, "res", stringsDirectory, "strings.xml"),
STRINGS_XML_MAX_BYTES,
operation,
signal,
)
: { text: null, truncated: false };
// Do not parse an incomplete XML document as though its projection were complete.
@@ -650,8 +468,7 @@ export class ApktoolProvider {
? { entries: [], unparsedCount: 0 }
: parseDecodedStrings(stringsXml);
const bounded = parsedStrings.entries.slice(0, MAX_STRINGS);
const files = await countDecodedFiles(workspace, signal, operation);
throwIfCancelled(operation, signal);
const files = await countDecodedFiles(workspace);
const coverage =
(stringsXml !== null && parsedStrings.entries.length > MAX_STRINGS) ||
manifestRead.truncated ||
@@ -676,7 +493,7 @@ export class ApktoolProvider {
manifest,
strings: bounded,
locale: input.locale ?? null,
locales: files.locales,
locales,
decoded_file_count: files.count,
coverage,
},
@@ -718,7 +535,7 @@ export class ApktoolProvider {
: []),
...(files.capped
? [
`Decoded workspace enumeration stopped after ${String(MAX_DECODED_ENTRIES_VISITED)} entries; decoded_file_count and locales may be partial`,
`decoded_file_count is capped at ${String(MAX_DECODED_FILES_COUNTED)}`,
]
: []),
...(cleanupFailure === null ? [] : [cleanupFailure]),
@@ -730,25 +547,14 @@ export class ApktoolProvider {
await removeWorkspace();
return err(
failureWithCleanup(
signal?.aborted === true
? new AnalysisCancelledError(operation, { cause })
: sourceAdmissionFailed
? new AnalysisInputError(operation, { cause })
: cause instanceof AnalysisInputError
? cause
: cause instanceof AnalysisCancelledError ||
cause instanceof AnalysisProtocolError
? cause
: new AnalysisProtocolError(
"Apktool resource projection failed",
{
cause,
},
),
cause instanceof AnalysisCancelledError ||
cause instanceof AnalysisProtocolError
? cause
: new AnalysisProtocolError("Apktool resource projection failed", {
cause,
}),
),
);
} finally {
await source?.close().catch(() => undefined);
}
}
}
+8 -9
View File
@@ -1,5 +1,5 @@
import { execFile } from "node:child_process";
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
import { lstat, readdir, readFile, realpath, stat } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
@@ -156,14 +156,13 @@ const wrappedBundle = async (bundle: string): Promise<string | undefined> => {
if (isAbsence(cause)) return undefined;
throw cause;
}
let app: string | undefined;
for await (const entry of await opendir(wrapper)) {
if (!entry.isDirectory() || !entry.name.toLowerCase().endsWith(".app"))
continue;
if (app !== undefined) return undefined;
app = join(wrapper, entry.name);
}
return app;
const apps = (await readdir(wrapper, { withFileTypes: true })).filter(
(entry) => entry.isDirectory() && entry.name.toLowerCase().endsWith(".app"),
);
const [app] = apps;
return apps.length === 1 && app !== undefined
? join(wrapper, app.name)
: undefined;
};
const resolveLayoutExecutable = async (
+3 -10
View File
@@ -1,4 +1,4 @@
import { constants, type Stats } from "node:fs";
import { constants } from "node:fs";
import { createHash } from "node:crypto";
import {
access,
@@ -248,7 +248,7 @@ const isArchiveFormat = (
const sha256Handle = async (
handle: FileHandle,
path: string,
initial: Stats,
initial: Awaited<ReturnType<FileHandle["stat"]>>,
signal?: AbortSignal,
): Promise<string> => {
const hash = createHash("sha256");
@@ -256,18 +256,11 @@ const sha256Handle = async (
let position = 0;
while (true) {
throwIfTargetResolutionCancelled(signal);
const observed = await handle.read(
chunk,
0,
Math.min(chunk.length, initial.size - position + 1),
position,
);
const observed = await handle.read(chunk, 0, chunk.length, position);
throwIfTargetResolutionCancelled(signal);
if (observed.bytesRead === 0) break;
hash.update(chunk.subarray(0, observed.bytesRead));
position += observed.bytesRead;
if (position > initial.size)
throw new BinaryTargetError(path, "target grew while being read");
}
throwIfTargetResolutionCancelled(signal);
const [opened, currentPath] = await Promise.all([handle.stat(), lstat(path)]);
@@ -6,7 +6,6 @@ import type { Result } from "../domain/result.js";
/** Provider-neutral application boundary for controlled browser scenarios. */
export interface BrowserScenarioCapturePort {
close?(): Promise<void>;
identity(): ProviderIdentity;
captureScenario(
scenario: BrowserScenario,
+1 -1
View File
@@ -6,7 +6,7 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { err, type Result } from "../domain/result.js";
import type { AnalysisExecution } from "./AnalysisProvider.js";
import type { BinarySession } from "./binary/BinarySession.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { analysisErrorWithCleanupFailure } from "./binary/AnalysisClientCleanup.js";
/** Await one-shot cleanup and retain either the outcome or both failures. */
export const withSessionCleanup = async <Value>(
-43
View File
@@ -1,43 +0,0 @@
/** Maximum admitted vendor response bytes, shared by Hopper setup adapters. */
export const HOPPER_DOWNLOAD_BYTES = 100_000_000;
/** Bound observed response bytes before retaining them, including chunked bodies. */
export const downloadHopperPackage = async (
url: string,
options: { readonly signal?: AbortSignal } = {},
): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }> => {
const deadline = AbortSignal.timeout(120_000);
const response = await fetch(url, {
headers: { accept: "application/json", "user-agent": "rea-installer" },
signal:
options.signal === undefined
? deadline
: AbortSignal.any([options.signal, deadline]),
});
const refused = { ok: false, bytes: new Uint8Array() };
if (
!response.ok ||
Number(response.headers.get("content-length")) > HOPPER_DOWNLOAD_BYTES
) {
await response.body?.cancel();
return refused;
}
if (response.body === null) return { ok: true, bytes: new Uint8Array() };
const reader = response.body.getReader();
const chunks: Uint8Array[] = [];
let bytes = 0;
try {
while (true) {
const chunk = await reader.read();
if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) };
if (chunk.value.byteLength > HOPPER_DOWNLOAD_BYTES - bytes) {
await reader.cancel();
return refused;
}
chunks.push(chunk.value);
bytes += chunk.value.byteLength;
}
} finally {
reader.releaseLock();
}
};
+1 -1
View File
@@ -42,7 +42,7 @@ export const readJsonFile = async (
if (!decoded.ok) {
return err(
new EvidenceFileError("read", "invalid-json", {
cause: decoded.cause ?? new SyntaxError(decoded.error),
cause: decoded.cause,
path: requestedPath,
}),
);
+12 -6
View File
@@ -7,13 +7,10 @@ import { join } from "node:path";
import { promisify } from "node:util";
import { execFileOutput } from "../process/ExecFileOutput.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
const execFileAsync = promisify(execFile);
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const MAX_PACKAGE_BYTES = 100_000_000;
const SUPPORTED_HOPPER_SHA256 = new Set([
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
@@ -234,7 +231,16 @@ export const systemLinuxHopperInstallHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): LinuxHopperInstallHost => ({
distribution: readLinuxDistribution,
download: downloadHopperPackage,
async download(url, options) {
const response = await fetch(url, {
headers: { accept: "application/json", "user-agent": "rea-installer" },
...(options.signal === undefined ? {} : { signal: options.signal }),
});
return {
ok: response.ok,
bytes: new Uint8Array(await response.arrayBuffer()),
};
},
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
@@ -344,7 +350,7 @@ const packageIntegrityMatches = (
release: LinuxHopperRelease,
): boolean =>
bytes.byteLength === Number(release.file_length) &&
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
bytes.byteLength <= MAX_PACKAGE_BYTES &&
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
const packageFamilyFor = (
+167 -213
View File
@@ -15,15 +15,12 @@ import { promisify } from "node:util";
import { z } from "zod";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
const execFileAsync = promisify(execFile);
const RELEASES_URL =
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const MAX_PACKAGE_BYTES = 100_000_000;
const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4";
/** Stable network and command effects disclosed before macOS Hopper setup. */
@@ -39,11 +36,7 @@ export const macHopperInstallDisclosure = {
/** Result of installing the official Hopper macOS application. */
export type MacHopperInstallResult =
| {
readonly status: "installed";
readonly launcherPath: string;
readonly cleanupFailure?: string;
}
| { readonly status: "installed"; readonly launcherPath: string }
| {
readonly status: "failed";
readonly reason:
@@ -55,7 +48,6 @@ export type MacHopperInstallResult =
| "destination_exists"
| "copy"
| "launcher_missing";
readonly cleanupFailure?: string;
};
/** External effects used by the bounded macOS Hopper installer. */
@@ -80,8 +72,6 @@ export interface MacHopperInstallHost {
launcherReady(path: string): Promise<boolean>;
openApplication(path: string): Promise<void>;
cleanup(path: string): Promise<void>;
/** Retain the cleanup capability until the owning setup host retries it. */
retainCleanup?(cleanup: () => Promise<void>): void;
destination(): string;
}
@@ -95,221 +85,187 @@ export const installMacHopper = async (
): Promise<MacHopperInstallResult> => {
let temporary: string | undefined;
let mounted = false;
let result: MacHopperInstallResult;
try {
result = await (async (): Promise<MacHopperInstallResult> => {
const metadata = await host.download(RELEASES_URL);
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
if (!parsed.success)
return { status: "failed", reason: "release_metadata" };
const release = parsed.data["OS X"];
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
return { status: "failed", reason: "release_metadata" };
const archive = await host.download(release.filename);
if (!archive.ok) return { status: "failed", reason: "download" };
if (!integrityMatches(archive.bytes, release))
return { status: "failed", reason: "integrity" };
const metadata = await host.download(RELEASES_URL);
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
if (!parsed.success)
return { status: "failed", reason: "release_metadata" };
const release = parsed.data["OS X"];
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
return { status: "failed", reason: "release_metadata" };
const archive = await host.download(release.filename);
if (!archive.ok) return { status: "failed", reason: "download" };
if (!integrityMatches(archive.bytes, release))
return { status: "failed", reason: "integrity" };
const destination = host.destination();
const destinationExists = await host.destinationExists(destination);
if (destinationExists && options.replaceExisting !== true)
return { status: "failed", reason: "destination_exists" };
temporary = await host.createTemporaryDirectory();
const packagePath = join(temporary, "hopper.dmg");
const mountPath = join(temporary, "mounted");
await host.createMountDirectory(mountPath);
await host.writePackage(packagePath, archive.bytes);
if (!(await host.mount(packagePath, mountPath)))
return { status: "failed", reason: "mount" };
mounted = true;
const source = await host.appBundle(mountPath);
if (source === undefined || !(await host.validBundle(source)))
return { status: "failed", reason: "bundle" };
const stage = `${destination}.rea-stage`;
if (
!(await host.installBundle(
source,
destination,
stage,
destinationExists,
))
)
return { status: "failed", reason: "copy" };
const launcherPath = join(destination, "Contents/MacOS/hopper");
if (!(await host.launcherReady(launcherPath)))
return { status: "failed", reason: "launcher_missing" };
await host.openApplication(destination);
return { status: "installed", launcherPath };
})();
const destination = host.destination();
const destinationExists = await host.destinationExists(destination);
if (destinationExists && options.replaceExisting !== true)
return { status: "failed", reason: "destination_exists" };
temporary = await host.createTemporaryDirectory();
const packagePath = join(temporary, "hopper.dmg");
const mountPath = join(temporary, "mounted");
await host.createMountDirectory(mountPath);
await host.writePackage(packagePath, archive.bytes);
if (!(await host.mount(packagePath, mountPath)))
return { status: "failed", reason: "mount" };
mounted = true;
const source = await host.appBundle(mountPath);
if (source === undefined || !(await host.validBundle(source)))
return { status: "failed", reason: "bundle" };
const stage = `${destination}.rea-stage`;
if (
!(await host.installBundle(source, destination, stage, destinationExists))
)
return { status: "failed", reason: "copy" };
const launcherPath = join(destination, "Contents/MacOS/hopper");
if (!(await host.launcherReady(launcherPath)))
return { status: "failed", reason: "launcher_missing" };
await host.openApplication(destination);
return { status: "installed", launcherPath };
} catch (cause: unknown) {
// Install-stage failures share the download reason; the stage is cleaned below.
void cause;
result = { status: "failed", reason: "download" };
}
let cleanupFailure: string | undefined;
if (temporary !== undefined) {
const temporaryPath = temporary;
const mountPath = join(temporary, "mounted");
const cleanup = async (): Promise<void> => {
if (mounted) {
await host.unmount(mountPath);
mounted = false;
}
await host.cleanup(temporaryPath);
};
try {
await cleanup();
} catch (cause: unknown) {
cleanupFailure = `Hopper DMG cleanup failed: ${failureMessage(cause)}`;
host.retainCleanup?.(cleanup);
return { status: "failed", reason: "download" };
} finally {
if (temporary !== undefined) {
const mountPath = join(temporary, "mounted");
if (mounted) await host.unmount(mountPath);
await host.cleanup(temporary);
}
}
return cleanupFailure === undefined ? result : { ...result, cleanupFailure };
};
/** Compose installer paths and subprocesses from one selected caller context. */
export const systemMacHopperInstallHost = (
homeDirectory: string,
environment: Readonly<NodeJS.ProcessEnv>,
): MacHopperInstallHost & { close(): Promise<string | undefined> } => {
const pendingCleanup = new Set<() => Promise<void>>();
return {
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
createMountDirectory: (path) => mkdir(path),
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
async mount(packagePath, mountPath) {
return commandSucceeds(
"hdiutil",
[
"attach",
"-readonly",
"-nobrowse",
"-mountpoint",
mountPath,
packagePath,
],
): MacHopperInstallHost => ({
download: downloadPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
createMountDirectory: (path) => mkdir(path),
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
async mount(packagePath, mountPath) {
return commandSucceeds(
"hdiutil",
[
"attach",
"-readonly",
"-nobrowse",
"-mountpoint",
mountPath,
packagePath,
],
environment,
);
},
async unmount(mountPath) {
await commandSucceeds(
"hdiutil",
["detach", mountPath, "-force"],
environment,
);
},
async appBundle(mountPath) {
const entries = await readdir(mountPath, { withFileTypes: true });
const app = entries.find(
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
);
return app === undefined ? undefined : join(mountPath, app.name);
},
async validBundle(appPath) {
if (
!(await commandSucceeds(
"codesign",
["--verify", "--deep", "--strict", appPath],
environment,
);
},
async unmount(mountPath) {
if (
!(await commandSucceeds(
"hdiutil",
["detach", mountPath, "-force"],
environment,
))
)
throw new Error(
`hdiutil detach failed for ${JSON.stringify(mountPath)}`,
);
},
async appBundle(mountPath) {
const entries = await readdir(mountPath, { withFileTypes: true });
const app = entries.find(
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
);
return app === undefined ? undefined : join(mountPath, app.name);
},
async validBundle(appPath) {
if (
!(await commandSucceeds(
"codesign",
["--verify", "--deep", "--strict", appPath],
environment,
))
)
return false;
try {
const identifier = (
await execFileAsync(
"/usr/libexec/PlistBuddy",
[
"-c",
"Print :CFBundleIdentifier",
join(appPath, "Contents/Info.plist"),
],
{ env: environment },
)
).stdout.trim();
return identifier === BUNDLE_IDENTIFIER;
} catch (cause: unknown) {
// best-effort cleanup: optional bundle probing; unreadable means invalid.
void cause;
return false;
}
},
async destinationExists(path) {
try {
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional existence probing; absence means missing.
void cause;
return false;
}
},
async installBundle(source, destination, stage, replaceExisting) {
const previous = `${stage}-previous`;
try {
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
await rm(stage, { recursive: true, force: true });
await rm(previous, { recursive: true, force: true });
await execFileAsync("ditto", [source, stage], { env: environment });
await access(join(stage, "Contents/MacOS/hopper"));
if (replaceExisting) await rename(destination, previous);
await rename(stage, destination);
if (replaceExisting)
await rm(previous, { recursive: true, force: true });
return true;
} catch (cause: unknown) {
// Install failure rolls back below; the false return reports it.
void cause;
await rm(stage, { recursive: true, force: true });
if (replaceExisting) {
try {
await rename(previous, destination);
} catch (rollbackCause: unknown) {
// The original destination was not moved, or rollback is impossible.
void rollbackCause;
}
}
return false;
}
},
async launcherReady(path) {
try {
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; absence means not ready.
void cause;
return false;
}
},
async openApplication(path) {
await execFileAsync("open", [path], { env: environment });
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
retainCleanup: (cleanup) => pendingCleanup.add(cleanup),
async close() {
let failure: string | undefined;
for (const cleanup of pendingCleanup) {
))
)
return false;
try {
const identifier = (
await execFileAsync(
"/usr/libexec/PlistBuddy",
[
"-c",
"Print :CFBundleIdentifier",
join(appPath, "Contents/Info.plist"),
],
{ env: environment },
)
).stdout.trim();
return identifier === BUNDLE_IDENTIFIER;
} catch (cause: unknown) {
// best-effort cleanup: optional bundle probing; unreadable means invalid.
void cause;
return false;
}
},
async destinationExists(path) {
try {
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional existence probing; absence means missing.
void cause;
return false;
}
},
async installBundle(source, destination, stage, replaceExisting) {
const previous = `${stage}-previous`;
try {
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
await rm(stage, { recursive: true, force: true });
await rm(previous, { recursive: true, force: true });
await execFileAsync("ditto", [source, stage], { env: environment });
await access(join(stage, "Contents/MacOS/hopper"));
if (replaceExisting) await rename(destination, previous);
await rename(stage, destination);
if (replaceExisting) await rm(previous, { recursive: true, force: true });
return true;
} catch (cause: unknown) {
// Install failure rolls back below; the false return reports it.
void cause;
await rm(stage, { recursive: true, force: true });
if (replaceExisting) {
try {
await cleanup();
pendingCleanup.delete(cleanup);
} catch (cause: unknown) {
failure ??= `Hopper DMG cleanup remains incomplete: ${failureMessage(cause)}`;
await rename(previous, destination);
} catch (rollbackCause: unknown) {
// The original destination was not moved, or rollback is impossible.
void rollbackCause;
}
}
return failure;
},
destination: () =>
join(homeDirectory, "Applications/Hopper Disassembler.app"),
};
return false;
}
},
async launcherReady(path) {
try {
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; absence means not ready.
void cause;
return false;
}
},
async openApplication(path) {
await execFileAsync("open", [path], { env: environment });
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
destination: () =>
join(homeDirectory, "Applications/Hopper Disassembler.app"),
});
const downloadPackage = async (url: string) => {
const response = await fetch(url, {
headers: { accept: "application/json", "user-agent": "rea-installer" },
signal: AbortSignal.timeout(120_000),
});
const length = Number(response.headers.get("content-length") ?? "0");
if (length > MAX_PACKAGE_BYTES) return { ok: false, bytes: new Uint8Array() };
const bytes = new Uint8Array(await response.arrayBuffer());
return { ok: response.ok && bytes.byteLength <= MAX_PACKAGE_BYTES, bytes };
};
const parseJson = (bytes: Uint8Array): unknown =>
@@ -319,7 +275,7 @@ const integrityMatches = (
release: z.infer<typeof releaseSchema>,
): boolean =>
bytes.byteLength === Number(release.file_length) &&
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
bytes.byteLength <= MAX_PACKAGE_BYTES &&
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
const commandSucceeds = async (
command: string,
@@ -335,8 +291,6 @@ const commandSucceeds = async (
return false;
}
};
const failureMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const releaseSchema = z.object({
filename: z.string().url(),
file_length: z.string().regex(/^\d+$/u),
+12 -22
View File
@@ -1,12 +1,10 @@
import { createHash } from "node:crypto";
import { posix } from "node:path";
import { logicalPathEscapesRoot } from "../domain/artifactIdentity.js";
import {
createReferenceSourcePathLookup,
isPortableAbsoluteReferenceTarget,
type ReferenceSourcePathLookup,
} from "../domain/referenceSourcePathIdentity.js";
logicalPathEscapesRoot,
normalizeJoinedLogicalPath,
} from "../domain/artifactIdentity.js";
import {
classifyReferenceSourcePath,
detectReferenceSourceLanguage,
@@ -85,11 +83,11 @@ const failedEntry = (
const resolveInternalSpecifier = (
fromPath: string,
specifier: string,
lookupPath: ReferenceSourcePathLookup,
filePaths: ReadonlySet<string>,
): { to: string; resolution: "internal" | "unresolved" } => {
if (posix.isAbsolute(specifier))
return { to: specifier, resolution: "unresolved" };
const normalized = posix.normalize(
const normalized = normalizeJoinedLogicalPath(
posix.join(posix.dirname(fromPath), specifier),
);
if (logicalPathEscapesRoot(normalized))
@@ -115,7 +113,7 @@ const resolveInternalSpecifier = (
const stem = normalized.slice(0, -posix.extname(normalized).length);
candidates.push(...sourceSuffixes.map((suffix) => `${stem}${suffix}`));
}
const match = lookupPath.resolveCandidates(candidates);
const match = candidates.find((candidate) => filePaths.has(candidate));
return match === undefined
? { to: normalized, resolution: "unresolved" }
: { to: match, resolution: "internal" };
@@ -123,7 +121,7 @@ const resolveInternalSpecifier = (
const appendFile = (
entry: Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>,
lookupPath: ReferenceSourcePathLookup,
filePaths: ReadonlySet<string>,
output: ParsedReferenceSourceEntries,
): void => {
const language = detectReferenceSourceLanguage(entry.path);
@@ -148,7 +146,7 @@ const appendFile = (
const resolved = resolveInternalSpecifier(
relationship.from_path,
relationship.to,
lookupPath,
filePaths,
);
output.relationships.push({
...relationship,
@@ -161,11 +159,11 @@ const appendFile = (
const appendReadEntry = (
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
lookupPath: ReferenceSourcePathLookup,
filePaths: ReadonlySet<string>,
output: ParsedReferenceSourceEntries,
): void => {
if (entry.kind === "file") {
appendFile(entry, lookupPath, output);
appendFile(entry, filePaths, output);
return;
}
const classifications = classifyReferenceSourcePath(entry.path);
@@ -182,14 +180,7 @@ const appendReadEntry = (
output.entries.push({
path: entry.path,
kind: "symlink",
// The reader resolves inventory identities from its root; the persisted
// graph represents relative symlink targets from the link's directory.
target:
entry.targetState === "internal" ||
(entry.targetState === "missing" &&
!isPortableAbsoluteReferenceTarget(entry.target))
? posix.relative(posix.dirname(entry.path), entry.target) || "."
: entry.target,
target: entry.target,
target_state: entry.targetState,
classifications,
limitations: [],
@@ -208,11 +199,10 @@ export const parseReferenceSourceEntries = (
parseFailures: [],
limitations: [...read.limitations],
};
const lookupPath = createReferenceSourcePathLookup(filePaths);
for (const entry of read.entries) {
if (signal?.aborted === true) break;
if (entry.status === "failed") output.entries.push(failedEntry(entry));
else appendReadEntry(entry, lookupPath, output);
else appendReadEntry(entry, filePaths, output);
}
return output;
};
+6 -59
View File
@@ -29,49 +29,6 @@ export const runSetup = async (
options: SetupOptions,
host: SetupHost = systemSetupHost(),
confirm?: SetupConfirmation,
): Promise<SetupResult> => {
let result: SetupResult | undefined;
let primaryFailure: unknown;
let threw = false;
try {
result = await runSetupWorkflow(options, host, confirm);
} catch (cause: unknown) {
threw = true;
primaryFailure = cause;
}
let cleanupFailure: string | undefined;
try {
cleanupFailure = await host.close?.();
} catch (cause: unknown) {
cleanupFailure = `Setup resource cleanup failed: ${failureMessage(cause)}`;
}
if (threw) {
if (cleanupFailure !== undefined)
throw new AggregateError(
[primaryFailure, new Error(cleanupFailure)],
"Setup and resource cleanup failed.",
{ cause: primaryFailure },
);
throw primaryFailure;
}
if (result === undefined) throw new Error("Setup returned no result.");
if (cleanupFailure === undefined) return result;
return {
...result,
status: "needs_human",
code: result.code ?? "cleanup_failed",
remediation: result.remediation
? `${result.remediation} ${cleanupFailure}`
: cleanupFailure,
};
};
const runSetupWorkflow = async (
options: SetupOptions,
host: SetupHost,
confirm?: SetupConfirmation,
): Promise<SetupResult> => {
const appliedActions: string[] = [];
let plannedActions: readonly SetupAction[] = [];
@@ -102,7 +59,6 @@ const runSetupWorkflow = async (
options.readinessScope?.clients === undefined &&
options.allDetectedClients !== true &&
options.installHopper !== true &&
options.skillClientIds === undefined &&
options.installSkill === undefined;
const interactiveSelection =
confirm !== undefined && !options.structured && clientSelectionAllowed;
@@ -111,12 +67,10 @@ const runSetupWorkflow = async (
: (options.clientIds ??
options.readinessScope?.clients ??
discovery.defaultClientIds);
let selectedSkillClientIds = options.skillClientIds ?? selectedClientIds;
let skillNeedsInstall = await host.skillNeedsInstall(selectedSkillClientIds);
let skillNeedsInstall = await host.skillNeedsInstall(selectedClientIds);
let skillSelected =
options.installSkill === true ||
(options.installSkill !== false &&
(selectedClientIds.length > 0 || selectedSkillClientIds.length > 0));
(options.installSkill !== false && selectedClientIds.length > 0);
let installSkill = skillSelected && skillNeedsInstall;
let planDiscovery = discovery;
if (interactiveSelection) {
@@ -184,7 +138,6 @@ const runSetupWorkflow = async (
selectedClientIds.length === 0) ||
(options.installSkill !== false && selectedClientIds.length > 0);
installSkill = skillSelected && skillNeedsInstall;
selectedSkillClientIds = selectedClientIds;
}
const planned = await planSetupActions({
discovery: planDiscovery,
@@ -192,7 +145,7 @@ const runSetupWorkflow = async (
providerEnvironment,
command: host.registrationCommand,
clientIds: selectedClientIds,
skillClientIds: selectedSkillClientIds,
skillClientIds: selectedClientIds,
installSkill,
});
plannedActions = planned.plannedActions;
@@ -303,7 +256,7 @@ const runSetupWorkflow = async (
host,
options,
appliedActions,
selectedSkillClientIds,
selectedClientIds,
))
)
return fail(
@@ -372,11 +325,8 @@ const installHopperAction = async (input: {
emitProgress(input.options, {
actionId: "install_hopper",
label,
state: installed.cleanupFailure === undefined ? "completed" : "warning",
detail:
installed.cleanupFailure === undefined
? installed.launcherPath
: `${installed.launcherPath}; ${installed.cleanupFailure}`,
state: "completed",
detail: installed.launcherPath,
});
const providerEnvironment = {
...input.providerEnvironment,
@@ -542,6 +492,3 @@ const summarizeDoctor = (report: DoctorReport): SetupDoctorSummary => ({
},
}),
});
const failureMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
+9 -32
View File
@@ -94,10 +94,6 @@ export const systemSetupHost = (
const doctorHost = selectedDoctorHost ?? systemDoctorHost({ environment });
const platform = doctorHost.platform;
const { homeDirectory } = doctorHost;
const macHopperHost =
platform === "darwin"
? systemMacHopperInstallHost(homeDirectory, environment)
: undefined;
return {
platform,
homeDirectory,
@@ -110,7 +106,6 @@ export const systemSetupHost = (
nodeVersion: process.versions.node,
macosVersion: () => doctorHost.macosVersion(),
linuxDistribution: readLinuxDistribution,
close: () => macHopperHost?.close() ?? Promise.resolve(undefined),
initialSetupState: async (
scope?: DoctorScope,
): Promise<SetupInitialState> => {
@@ -131,33 +126,15 @@ export const systemSetupHost = (
};
},
installHopper: async (replaceExisting) => {
if (platform === "linux") {
const result = await installLinuxHopper(
systemLinuxHopperInstallHost(environment),
);
return result.status === "installed"
? result
: setupInstallFailure(result.reason);
}
if (macHopperHost === undefined)
return setupInstallFailure("unsupported_host");
const result = await installMacHopper({ replaceExisting }, macHopperHost);
if (result.status === "installed")
return {
status: "installed",
launcherPath: result.launcherPath,
...(result.cleanupFailure === undefined
? {}
: { cleanupFailure: result.cleanupFailure }),
};
const failure = setupInstallFailure(result.reason);
if (result.cleanupFailure === undefined) return failure;
return failure.status === "failed"
? {
...failure,
remediation: `${failure.remediation} ${result.cleanupFailure}`,
}
: failure;
const result =
platform === "linux"
? await installLinuxHopper(systemLinuxHopperInstallHost(environment))
: await installMacHopper(
{ replaceExisting },
systemMacHopperInstallHost(homeDirectory, environment),
);
if (result.status === "installed") return result;
return setupInstallFailure(result.reason);
},
detectedClients: () => detectClients(homeDirectory, platform, environment),
supportedClients: () =>
+2 -7
View File
@@ -10,15 +10,10 @@ export type SetupFailureCode =
| "destination_exists"
| "mount_failed"
| "bundle_invalid"
| "copy_failed"
| "cleanup_failed";
| "copy_failed";
export type SetupHopperInstallResult =
| {
readonly status: "installed";
readonly launcherPath: string;
readonly cleanupFailure?: string;
}
| { readonly status: "installed"; readonly launcherPath: string }
| {
readonly status: "failed";
readonly code: SetupFailureCode;
+5 -65
View File
@@ -1,16 +1,11 @@
import { copyFile, lstat, mkdir, readFile, rm } from "node:fs/promises";
import { constants as fsConstants } from "node:fs";
import { mkdir, readFile, rm } from "node:fs/promises";
import { dirname, join } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
NonRegularFileReadError,
openRegularFile,
} from "../filesystem/RegularFile.js";
import { clientSkillDirectories } from "./SupportedClients.js";
export const MANAGED_SKILL_FILES = [
const SKILL_FILES = [
"SKILL.md",
"references/native-and-artifacts.md",
"references/javascript-applications.md",
@@ -19,19 +14,6 @@ export const MANAGED_SKILL_FILES = [
"references/evidence-workflows.md",
] as const;
export const isManagedSkillManifest = (content: string): boolean =>
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
export const readSkillFile = async (path: string): Promise<string> => {
const handle = await openRegularFile(path, { symlinks: "reject" });
try {
return await handle.readFile("utf8");
} finally {
await handle.close();
}
};
interface CanonicalSkillFile {
readonly content: string;
readonly destination: string;
@@ -65,38 +47,9 @@ export const skillDestinations = (
}),
);
/** Resolve existing REA-owned skill roots using setup's canonical destinations. */
export const existingSkillDestinations = async (
home: string,
environment: Readonly<NodeJS.ProcessEnv> = {},
platform: NodeJS.Platform = process.platform,
): Promise<readonly SkillDestination[]> => {
const existing: SkillDestination[] = [];
for (const destination of skillDestinations(
home,
undefined,
environment,
platform,
)) {
try {
const root = await lstat(destination.path);
if (root.isSymbolicLink() || !root.isDirectory()) continue;
const manifest = join(destination.path, "SKILL.md");
const content = await readSkillFile(manifest);
if (isManagedSkillManifest(content)) existing.push(destination);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
continue;
if (cause instanceof NonRegularFileReadError) continue;
throw cause;
}
}
return existing;
};
const readOptionalText = async (path: string): Promise<string | undefined> => {
try {
return await readSkillFile(path);
return await readFile(path, "utf8");
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return undefined;
@@ -113,7 +66,7 @@ const canonicalSkillFiles = async (
Promise.all(
skillDestinations(home, clientIds, environment, platform).flatMap(
({ path: root }) =>
MANAGED_SKILL_FILES.map(async (relativePath) => {
SKILL_FILES.map(async (relativePath) => {
const destination = join(root, relativePath);
return {
destination,
@@ -151,19 +104,6 @@ export const canonicalSkillNeedsInstall = async (
const writeText = (path: string, content: string): Promise<void> =>
writeFileAtomic(path, content, { encoding: "utf8", mode: 0o600 });
const preserveSkillBackup = async (
source: string,
destination: string,
): Promise<void> => {
try {
await copyFile(source, destination, fsConstants.COPYFILE_EXCL);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "EEXIST")
return;
throw cause;
}
};
const restoreSkillFiles = async (
changed: readonly CanonicalSkillFile[],
): Promise<void> => {
@@ -194,7 +134,7 @@ export const installCanonicalSkill = async (
for (const { destination, original } of changed) {
await mkdir(dirname(destination), { recursive: true });
if (original !== undefined)
await preserveSkillBackup(destination, `${destination}.rea.backup`);
await writeText(`${destination}.rea.backup`, original);
}
for (const { destination, content } of changed)
await writeText(destination, content);
-4
View File
@@ -50,8 +50,6 @@ export interface SetupHost {
linuxDistribution(): Promise<LinuxDistribution | undefined>;
initialSetupState(scope?: DoctorScope): Promise<SetupInitialState>;
installHopper(replaceExisting: boolean): Promise<SetupHopperInstallResult>;
/** Retry and report any cleanup resources still owned by this setup host. */
close?(): Promise<string | undefined>;
detectedClients(): Promise<readonly SetupClient[]>;
supportedClients(): Promise<readonly SetupClient[]>;
configureClient(
@@ -157,8 +155,6 @@ export interface SetupOptions {
readonly structured: boolean;
readonly proposeHopper?: boolean;
readonly clientIds?: readonly string[];
/** Skill roots to update independently of MCP registrations (maintenance). */
readonly skillClientIds?: readonly string[];
readonly installSkill?: boolean;
readonly readinessScope?: DoctorScope;
readonly dryRun?: boolean;
+4 -152
View File
@@ -15,12 +15,7 @@ import writeFileAtomic from "write-file-atomic";
import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
isManagedSkillManifest,
MANAGED_SKILL_FILES,
readSkillFile,
skillDestinations,
} from "./SetupSkill.js";
import { skillDestinations } from "./SetupSkill.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import {
@@ -32,32 +27,29 @@ import {
interface ManagedPathStats {
readonly uid?: number;
isFile?(): boolean;
isDirectory?(): boolean;
isSymbolicLink(): boolean;
}
/** Injectable filesystem operations used to prove uninstall failure recovery. */
export interface UninstallFileSystem {
readText(path: string): Promise<string>;
readRegularText?(path: string): Promise<string>;
/** Preserve the first backup; reject an existing destination with EEXIST. */
copy(source: string, destination: string): Promise<void>;
writeText(path: string, contents: string): Promise<void>;
stat(path: string): Promise<ManagedPathStats>;
realpath?(path: string): Promise<string>;
remove(path: string, recursive?: boolean): Promise<void>;
remove(path: string): Promise<void>;
}
const systemFileSystem: UninstallFileSystem = {
readText: (path) => readFile(path, "utf8"),
readRegularText: readSkillFile,
copy: (source, destination) =>
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
writeText: (path, contents) =>
writeFileAtomic(path, contents, { encoding: "utf8" }),
stat: (path) => lstat(path),
realpath: (path) => realpath(path),
remove: (path, recursive = true) => rm(path, { recursive }),
remove: (path) => rm(path, { recursive: true }),
};
/** One explicitly classified uninstall action. */
@@ -430,7 +422,7 @@ const removeManagedSkills = async (
const results = await Promise.all(
skillDestinations(home, undefined, environment, platform).map(
({ client, path }) =>
removeManagedSkill(
removeManagedPath(
path,
client === "claude_code" ? "Claude Code skill" : "skill",
fileSystem,
@@ -465,146 +457,6 @@ const removeManagedSkills = async (
);
};
const removeManagedSkill = async (
path: string,
name: string,
fileSystem: UninstallFileSystem,
): Promise<UninstallItem> => {
let root: ManagedPathStats;
try {
root = await fileSystem.stat(path);
} catch (cause: unknown) {
return isMissing(cause)
? item(name, "skipped", "Skill directory does not exist.")
: item(name, "failed", "Skill directory could not be inspected.");
}
if (root.isSymbolicLink())
return item(name, "retained", "Skill directory is a symbolic link.");
if (root.isDirectory?.() !== true)
return item(name, "retained", "Skill path is not a directory.");
const manifestPath = join(path, "SKILL.md");
try {
const manifest = await fileSystem.stat(manifestPath);
if (manifest.isSymbolicLink())
return item(name, "retained", "Skill manifest is a symbolic link.");
if (manifest.isFile?.() !== true)
return item(name, "retained", "Skill manifest is not a regular file.");
} catch (cause: unknown) {
return isMissing(cause)
? item(name, "retained", "Skill ownership could not be established.")
: item(name, "failed", "Skill manifest could not be inspected.");
}
let content: string;
try {
content = await (fileSystem.readRegularText?.(manifestPath) ??
fileSystem.readText(manifestPath));
} catch (cause: unknown) {
return isMissing(cause)
? item(name, "retained", "Skill ownership could not be established.")
: item(name, "failed", "Skill manifest could not be read.");
}
if (!isManagedSkillManifest(content))
return item(name, "retained", "Skill manifest is not REA-owned.");
const manifestName = "SKILL.md";
const otherFiles = MANAGED_SKILL_FILES.filter(
(relativePath) => relativePath !== manifestName,
);
const results: UninstallItem[] = [];
for (const relativePath of otherFiles)
results.push(
await removeManagedSkillFile(path, relativePath, name, fileSystem),
);
if (
results.some(({ status }) => status === "failed" || status === "retained")
)
results.push(
item(
name,
"retained",
"REA ownership manifest was retained because a managed file could not be safely removed.",
),
);
else
results.push(
await removeManagedSkillFile(path, manifestName, name, fileSystem),
);
const failed = results.find(({ status }) => status === "failed");
if (failed !== undefined)
return item(name, "failed", results.map(({ detail }) => detail).join(" "));
const removed = results.some(({ status }) => status === "removed");
return item(
name,
removed ? "removed" : "skipped",
"REA-managed skill files were removed; other files were preserved.",
);
};
const removeManagedSkillFile = async (
root: string,
relativePath: string,
name: string,
fileSystem: UninstallFileSystem,
): Promise<UninstallItem> => {
const segments = relativePath.split(/[\\/]/u);
let parent = root;
for (const segment of segments.slice(0, -1)) {
parent = join(parent, segment);
try {
const stats = await fileSystem.stat(parent);
if (stats.isSymbolicLink() || stats.isDirectory?.() !== true)
return item(
name,
"retained",
`Managed path parent was retained: ${parent}.`,
);
} catch (cause: unknown) {
return isMissing(cause)
? item(
name,
"skipped",
`${join(parent, segments.at(-1) ?? "")} does not exist.`,
)
: item(
name,
"failed",
`Managed path parent could not be inspected: ${parent}.`,
);
}
}
const target = join(root, relativePath);
let stats: ManagedPathStats;
try {
stats = await fileSystem.stat(target);
} catch (cause: unknown) {
return isMissing(cause)
? item(name, "skipped", `${target} does not exist.`)
: item(name, "failed", `Managed file could not be inspected: ${target}.`);
}
if (stats.isSymbolicLink())
return item(
name,
"retained",
`Managed file is a symbolic link: ${target}.`,
);
if (stats.isFile?.() !== true)
return item(
name,
"retained",
`Managed path is not a regular file: ${target}.`,
);
try {
await fileSystem.remove(target, false);
return item(name, "removed", `Removed ${target}.`);
} catch (cause: unknown) {
return isMissing(cause)
? item(name, "skipped", `${target} does not exist.`)
: item(name, "failed", `Managed file could not be removed: ${target}.`);
}
};
const item = (
name: string,
status: UninstallItem["status"],
+21 -25
View File
@@ -1,4 +1,5 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
@@ -11,7 +12,6 @@ import {
parseClientConfiguration,
} from "./ClientConfigurationDocument.js";
import { supportedClients } from "./SupportedClients.js";
import { existingSkillDestinations } from "./SetupSkill.js";
import type { SetupAction } from "./SetupTypes.js";
import type { Result } from "../domain/result.js";
@@ -19,10 +19,6 @@ import type { Result } from "../domain/result.js";
export interface MaintenanceScope {
readonly clients: readonly string[];
readonly skill: boolean;
readonly skillDestinations?: readonly {
readonly client: string;
readonly path: string;
}[];
}
/** Unapplied maintenance evidence produced by the updated setup executable. */
@@ -53,7 +49,7 @@ const setupPlanSchema = z.object({
),
});
/** Discover owned registrations and REA skills without writes. */
/** Discover owned registrations and an existing REA skill without writes. */
export const existingMaintenanceScope = async (
home: string,
entryPoint: string,
@@ -91,12 +87,20 @@ export const existingMaintenanceScope = async (
)
clients.push(client.name);
}
const skillDestinations = await existingSkillDestinations(home, environment);
return {
clients,
skill: skillDestinations.length > 0,
skillDestinations,
};
let skill = false;
try {
const content = await readFile(
join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName, "SKILL.md"),
"utf8",
);
skill =
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
} catch (cause: unknown) {
if (!(cause instanceof Error && "code" in cause && cause.code === "ENOENT"))
throw cause;
}
return { clients, skill };
};
/** Ask the verified new executable to plan only existing REA integrations. */
@@ -108,7 +112,6 @@ export const planIntegrationMaintenance = async (
): Promise<IntegrationMaintenance> => {
try {
const scope = await existingMaintenanceScope(home, entryPoint, environment);
const skillDestinations = scope.skillDestinations ?? [];
if (scope.clients.length === 0 && !scope.skill)
return { status: "current", plannedActions: [] };
const command = [
@@ -116,7 +119,6 @@ export const planIntegrationMaintenance = async (
entryPoint,
"setup",
...scope.clients.flatMap((client) => ["--client", client]),
...skillDestinations.flatMap(({ client }) => ["--skill-client", client]),
`--skill=${String(scope.skill)}`,
];
const execution = await execute([...command, "--dry-run", "--json"]);
@@ -130,17 +132,11 @@ export const planIntegrationMaintenance = async (
};
if (plan.data.plannedActions.length === 0)
return { status: "current", plannedActions: [] };
const allowedIds = new Set(
scope.clients.map((client) => `configure_client:${client}`),
);
const skillTargets = new Set(skillDestinations.map(({ path }) => path));
if (
plan.data.plannedActions.some((action) =>
action.kind === "install_skill"
? !skillTargets.has(action.target)
: !allowedIds.has(action.id),
)
)
const allowedIds = new Set([
...scope.clients.map((client) => `configure_client:${client}`),
...(scope.skill ? ["install_skill"] : []),
]);
if (plan.data.plannedActions.some(({ id }) => !allowedIds.has(id)))
return {
status: "unavailable",
remediation:
@@ -28,7 +28,6 @@ export interface WebSourceLocationArtifactPort {
/** Replaceable codec receives map data and one explicit generated point. */
export interface WebSourceMapPort {
close?(): Promise<void>;
trace(
input: {
readonly text: string;
@@ -59,10 +59,6 @@ export class WebSourceLocationService {
readonly decoder: WebSourceMapPort,
) {}
close(): Promise<void> {
return this.decoder.close?.() ?? Promise.resolve();
}
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
async trace(
rawInput: unknown,
@@ -5,8 +5,10 @@ import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { err } from "../../domain/result.js";
import type { AnalysisClient } from "../AnalysisProvider.js";
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import {
analysisErrorWithCleanupFailure,
closeAnalysisClient,
} from "./AnalysisClientCleanup.js";
const clientClosingWith = (cause: unknown): AnalysisClient => ({
execute: () => Promise.resolve(err(new ProviderCleanupError("x", [], {}))),
@@ -1,4 +1,6 @@
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { err, type Result } from "../../domain/result.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { AnalysisClient, ExecutionOptions } from "../AnalysisProvider.js";
@@ -27,3 +29,34 @@ export const closeAnalysisClient = async (
);
}
};
/** Preserve the original lifecycle failure alongside incomplete cleanup. */
export const analysisErrorWithCleanupFailure = (
primary: AnalysisError,
cleanup: AnalysisError,
operation = "close_binary",
): AnalysisError => {
const projected = projectAnalysisError(cleanup);
return new ProviderAdapterError(
cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea",
operation,
{
cause: primary,
userMessage: projectAnalysisError(primary).message,
...(primary.capturedOutput === undefined
? {}
: { capturedOutput: primary.capturedOutput }),
...(primary.partialObservation === undefined
? {}
: { partialObservation: primary.partialObservation }),
cleanup: {
reason: projected.message,
resources: cleanup.cleanupResources,
},
diagnostics: {
primary_error: projectAnalysisError(primary),
cleanup_error: projected,
},
},
);
};
+4 -2
View File
@@ -44,8 +44,10 @@ import {
commitExecutionProfile,
prepareSessionExecution,
} from "./BinarySessionExecution.js";
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import {
analysisErrorWithCleanupFailure,
closeAnalysisClient,
} from "./AnalysisClientCleanup.js";
const OFFICIAL_OPERATIONS: ReadonlySet<string> = new Set(
OFFICIAL_TOOL_CONTRACTS.map(({ name }) => name),
);
-1
View File
@@ -8,7 +8,6 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline interface-recovery engine, independent of wallets and chain clients. */
export interface EvmInterfacePort {
close?(): Promise<void>;
inspect(
input: InspectEvmInterfaceInput,
options?: ExecutionOptions,
@@ -25,10 +25,6 @@ const diagnosticsSchema = z.object({
export class EvmInterfaceService {
constructor(readonly provider: EvmInterfacePort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
/** Preserve selected byte identity while clearly labeling recovered ABI candidates as inferences. */
async inspect(
rawInput: unknown,
@@ -8,7 +8,6 @@ import type { Result } from "../../domain/result.js";
/** File-oriented firmware engines; executable admission remains a separate operation. */
export interface FirmwareAnalysisPort {
close?(): Promise<void>;
execute(
request: FirmwareRequest,
options?: ExecutionOptions,
@@ -18,10 +18,6 @@ import { err, ok, type Result } from "../../domain/result.js";
export class FirmwareAnalysisService {
constructor(readonly provider: FirmwareAnalysisPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
/** Execute an explicit inspection or extraction and preserve its producing identity. */
async execute(
operation: FirmwareOperation,
@@ -8,7 +8,6 @@ import {
} from "./JavaScriptArtifactGraphContext.js";
import { astObservationEvidence } from "./JavaScriptArtifactGraphEvidence.js";
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
import { javascriptDisplayText } from "../../domain/javascript/javascriptAstValues.js";
import {
addElectronInferenceEdge,
electronFindingSourceNode,
@@ -52,27 +51,17 @@ const addNativeBinding = (input: NativeBindingInput): void => {
identity: electronObservationIdentity(
context,
"javascript-requested-native-export",
compositeKey([
file.path,
value.specifier,
value.binding_kind,
value.members,
value.namespace_access ?? null,
value.dynamic_member_access ?? null,
electronRangeKey(value.location),
]),
// Members are a variable-length source-derived list: compositeKey keeps
// ["a\0b"] distinct from ["a", "b"] inside the digested identity.
`${file.path}:${value.specifier}:${value.binding_kind}:${compositeKey(value.members)}:${electronRangeKey(value.location)}`,
),
observations: [
{
label:
value.members.map(javascriptDisplayText).join(", ") ||
value.specifier,
label: value.members.join(", "),
properties: {
specifier: value.specifier,
binding_kind: value.binding_kind,
requested_members: value.members,
namespace_access: value.namespace_access ?? null,
dynamic_member_access: value.dynamic_member_access ?? null,
resolved_path: resolved,
native_export_verification: "not-performed",
},
@@ -101,8 +90,6 @@ const addNativeBinding = (input: NativeBindingInput): void => {
specifier: value.specifier,
binding_kind: value.binding_kind,
requested_members: value.members,
namespace_access: value.namespace_access ?? null,
dynamic_member_access: value.dynamic_member_access ?? null,
},
limitations: [
"Requested JavaScript members are not verified native exports.",
@@ -120,8 +107,6 @@ const addNativeBinding = (input: NativeBindingInput): void => {
properties: {
resolved_path: resolved,
requested_members: value.members,
namespace_access: value.namespace_access ?? null,
dynamic_member_access: value.dynamic_member_access ?? null,
verified: false,
},
confidence: "medium",
@@ -1,7 +1,6 @@
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
import type { JavaScriptBundlerRegistration } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
import {
addAstContainsEdge,
artifactLocalIdentity,
@@ -81,11 +80,7 @@ const createBundlerChunkNode = (
const chunkKey = `${registration.runtime}:${registration.chunk_keys.join(",")}`;
return context.accumulator.addNode({
kind: "javascript-chunk",
identity: artifactLocalIdentity(
file.sha256,
"bundler-chunk",
compositeKey([registration.runtime, registration.chunk_keys]),
),
identity: artifactLocalIdentity(file.sha256, "bundler-chunk", chunkKey),
observations: [
{
label: chunkKey,
@@ -1,5 +1,4 @@
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
import {
completeApplicationCoverage,
partialApplicationCoverage,
@@ -331,7 +330,7 @@ export const chunkLookupKey = (
path: string,
runtime: string,
chunkKey: string,
): string => compositeKey([path, runtime, chunkKey]);
): string => `${path}\0${runtime}\0${chunkKey}`;
/** Resolve a relative static specifier without escaping the artifact root. */
export const resolveArtifactPath = (
@@ -1,9 +1,6 @@
import { posix } from "node:path";
import {
compareUnicodeCodePoints,
compositeKey,
} from "../../domain/unicodeCodePointOrder.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
import {
completeApplicationCoverage,
@@ -180,7 +177,7 @@ const addExportRelationship = (
identity: artifactLocalIdentity(
file.sha256,
"module-export",
compositeKey([file.container_sha256, file.path, link.exportedName]),
`${file.container_sha256}:${file.path}:${link.exportedName}`,
),
observations: [
{
@@ -8,7 +8,6 @@ import type {
/** Typed boundary for upstream source recovery without exposing its protocol. */
export interface JavaScriptRecoveryPort {
close?(): Promise<void>;
recover(
input: JavaScriptRecoveryInput,
options?: ExecutionOptions,
@@ -23,10 +23,6 @@ const OPERATION = "recover_javascript_sources";
export class JavaScriptRecoveryService {
constructor(readonly provider: JavaScriptRecoveryPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
/** Recover sources and bind the result to its original immutable input. */
async recover(
input: unknown,
+26 -20
View File
@@ -1,3 +1,4 @@
import { readFile } from "node:fs/promises";
import { z } from "zod";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
@@ -16,10 +17,9 @@ import { compareProcessCaptures } from "../../domain/process/processComparison.j
import { parseProcessCapture } from "../../domain/process/processCaptureParsing.js";
import { createProgressReporter } from "../ProgressReporter.js";
import { captureProcessScenario } from "../../process/capture/ProcessHarness.js";
import { ProcessCaptureResourceScope } from "../../process/capture/ProcessCaptureLifecycle.js";
import { PROCESS_PROVIDER } from "../../domain/process/processEvidenceProvider.js";
import { createProcessCaptureEvidence } from "./ProcessEvidence.js";
import { readJsonFile } from "../JsonFiles.js";
import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../Utf8JsonInput.js";
/** Safe process-command failure returned to the CLI adapter. */
export interface ProcessCliErrorOutput {
@@ -63,7 +63,6 @@ export const captureProcessScenarioFile = async (
},
{ minimumIntervalMs: 100 },
);
const resourceScope = new ProcessCaptureResourceScope();
const captured = await captureProcessScenario(
parsed.data,
signal,
@@ -72,14 +71,7 @@ export const captureProcessScenarioFile = async (
undefined,
undefined,
progress,
resourceScope,
);
try {
await resourceScope.close();
} catch (cause: unknown) {
if (!captured.ok) return cliAnalysisError(captured.error);
throw cause;
}
if (!captured.ok) return cliAnalysisError(captured.error);
return createProcessCaptureEvidence(parsed.data, captured.value);
} catch (cause: unknown) {
@@ -186,16 +178,30 @@ const invalidCaptureEvidence = (cause: unknown): ProcessCliFailure =>
);
const readJson = async (path: string): Promise<unknown> => {
const loaded = await readJsonFile(path);
if (loaded.ok) return loaded.value;
if (loaded.error._tag !== "EvidenceFileError") throw loaded.error;
const detail = describeValidationFailure(loaded.error.cause);
throw new ProcessCliFailure(
"invalid_input",
loaded.error.reason === "invalid-json"
? `Process input file is not valid UTF-8 JSON: ${path} (${detail}). Repair the file, then try again.`
: `Process input file could not be read: ${path} (${detail}). Check the reported filesystem constraint and retry.`,
);
let bytes;
try {
bytes = await readFile(path);
} catch (cause: unknown) {
throw new ProcessCliFailure(
"invalid_input",
`Process input file could not be read: ${path} (${describeValidationFailure(cause)}). Check the reported filesystem constraint and retry.`,
);
}
try {
const text = new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(bytes);
if (text.startsWith("\uFEFF"))
throw new Error(JSON_BYTE_ORDER_MARK_MESSAGE);
const parsed: unknown = JSON.parse(text);
return parsed;
} catch (cause: unknown) {
throw new ProcessCliFailure(
"invalid_input",
`Process input file is not valid UTF-8 JSON: ${path} (${describeValidationFailure(cause)}). Repair the file, then try again.`,
);
}
};
class ProcessCliFailure extends Error {
-12
View File
@@ -27,7 +27,6 @@ import {
type ArtifactReader,
} from "./ArtifactReader.js";
import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js";
import { admitAsarHeader } from "./AsarHeader.js";
/**
* Official Electron ASAR adapter with range-streamed member reads.
@@ -58,17 +57,6 @@ export class AsarArtifactReader implements ArtifactReader {
const snapshotPath = this.#snapshotPath();
this.#resetArchiveState();
this.#entries.clear();
const headerHandle = await open(
snapshotPath,
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
);
try {
const admitted = await admitAsarHeader(headerHandle);
this.#archiveSize = admitted.archiveBytes;
} finally {
await headerHandle.close();
}
abortIfNeeded(signal);
paths = listPackage(snapshotPath, { isPack: false }).sort((left, right) =>
left.localeCompare(right, "en"),
);
-48
View File
@@ -1,48 +0,0 @@
import type { FileHandle } from "node:fs/promises";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
/** Shared resource budget for the eagerly decoded ASAR header, not member bytes. */
export const MAX_ASAR_HEADER_BYTES = 16 * 1024 * 1024;
/** Admit both Chromium pickle extents before an upstream decoder allocates them. */
export const admitAsarHeader = async (
handle: FileHandle,
): Promise<{
readonly archiveBytes: number;
readonly headerBytes: number;
readonly jsonBytes: number;
}> => {
const observed = await handle.stat();
if (!observed.isFile())
throw new ArtifactReaderFailure(
"format",
"ASAR container is not a regular file",
);
const prefix = Buffer.alloc(16);
const read = await handle.read(prefix, 0, prefix.length, 0);
if (read.bytesRead !== prefix.length || prefix.readUInt32LE(0) !== 4)
throw new ArtifactReaderFailure(
"format",
"Truncated or invalid ASAR size pickle",
);
const headerBytes = prefix.readUInt32LE(4);
const payloadBytes = prefix.readUInt32LE(8);
const jsonBytes = prefix.readUInt32LE(12);
if (
headerBytes < 8 ||
headerBytes > observed.size - 8 ||
payloadBytes !== headerBytes - 4 ||
jsonBytes > payloadBytes - 4
)
throw new ArtifactReaderFailure(
"format",
"ASAR header leaves its container or has invalid pickle extents",
);
if (headerBytes > MAX_ASAR_HEADER_BYTES)
throw new ArtifactReaderFailure(
"limit",
`ASAR header exceeds the ${String(MAX_ASAR_HEADER_BYTES)} byte decode budget`,
);
return { archiveBytes: observed.size, headerBytes, jsonBytes };
};
+44 -42
View File
@@ -3,13 +3,15 @@ import { lstat, open, opendir, realpath } from "node:fs/promises";
import { isAbsolute, join, relative, sep } from "node:path";
import type { Readable } from "node:stream";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import {
ArtifactReaderFailure,
type ArtifactEntry,
type ArtifactReader,
} from "./ArtifactReader.js";
/** Non-symlink-following directory reader. */
/** Deterministic, non-symlink-following directory reader. */
export class DirectoryArtifactReader implements ArtifactReader {
readonly format = "directory" as const;
readonly #rootPromise: Promise<string>;
@@ -36,52 +38,13 @@ export class DirectoryArtifactReader implements ArtifactReader {
await withDirectoryIoContext("inspect directory", directory, () =>
assertContainedDirectory(root, directory),
);
const children: string[] = [];
const handle = await opendir(directory).catch((cause: unknown) => {
throw directoryIoFailure("open directory", directory, cause);
});
const directories: string[] = [];
try {
for await (const child of handle) {
abortIfNeeded(signal);
const absolute = join(directory, child.name);
const metadata = await lstat(absolute).catch((cause: unknown) => {
throw directoryIoFailure("inspect entry", absolute, cause);
});
const path = relative(root, absolute).split(sep).join("/");
const kind = metadata.isSymbolicLink()
? "symlink"
: metadata.isDirectory()
? "directory"
: metadata.isFile()
? "file"
: undefined;
if (kind === undefined) continue;
yield {
path,
kind,
declaredSize: kind === "file" ? metadata.size : null,
compressedSize: null,
executable: (metadata.mode & 0o111) !== 0,
encrypted: false,
byteOffset: null,
declaredSha256: null,
unpacked: false,
limitations:
kind === "symlink"
? ["Symlink target was not followed or disclosed."]
: [],
adapterKey: absolute,
...(kind === "file"
? {
sourceIdentity: { device: metadata.dev, inode: metadata.ino },
}
: {}),
};
if (kind === "directory" && (descendInto?.(path) ?? true))
directories.push(absolute);
}
for await (const child of handle) children.push(child.name);
} catch (cause: unknown) {
if (cause instanceof ArtifactReaderFailure) throw cause;
throw directoryIoFailure("read directory", directory, cause);
} finally {
// best-effort cleanup: directory-handle close must not mask traversal.
@@ -90,6 +53,45 @@ export class DirectoryArtifactReader implements ArtifactReader {
await withDirectoryIoContext("inspect directory", directory, () =>
assertContainedDirectory(root, directory),
);
children.sort(compareUnicodeCodePoints);
const directories: string[] = [];
for (const name of children) {
abortIfNeeded(signal);
const absolute = join(directory, name);
const metadata = await lstat(absolute).catch((cause: unknown) => {
throw directoryIoFailure("inspect entry", absolute, cause);
});
const path = relative(root, absolute).split(sep).join("/");
const kind = metadata.isSymbolicLink()
? "symlink"
: metadata.isDirectory()
? "directory"
: metadata.isFile()
? "file"
: undefined;
if (kind === undefined) continue;
if (kind === "directory" && (descendInto?.(path) ?? true))
directories.push(absolute);
yield {
path,
kind,
declaredSize: kind === "file" ? metadata.size : null,
compressedSize: null,
executable: (metadata.mode & 0o111) !== 0,
encrypted: false,
byteOffset: null,
declaredSha256: null,
unpacked: false,
limitations:
kind === "symlink"
? ["Symlink target was not followed or disclosed."]
: [],
adapterKey: absolute,
...(kind === "file"
? { sourceIdentity: { device: metadata.dev, inode: metadata.ino } }
: {}),
};
}
directories.reverse();
pending.push(...directories);
}
+46 -93
View File
@@ -105,26 +105,7 @@ export class SafeOutputTree {
}
return new SafeOutputTree(canonicalOutput, platform);
} catch (cause: unknown) {
let removalFailure: unknown;
try {
await rm(canonicalOutput, { recursive: true, force: true });
} catch (cleanupCause: unknown) {
removalFailure = cleanupCause;
}
let absent = false;
try {
absent = await isAbsent(canonicalOutput);
} catch (cleanupCause: unknown) {
removalFailure ??= cleanupCause;
}
if (!absent)
throw ArtifactReaderFailure.withCleanup(cause, {
reason:
removalFailure instanceof Error
? removalFailure.message
: "Extraction output root remains after setup failure",
resources: [canonicalOutput],
});
await rm(canonicalOutput, { recursive: true, force: true });
throw cause;
}
}
@@ -141,84 +122,56 @@ export class SafeOutputTree {
async write(
relativePath: string,
source: Readable,
expected: { readonly sha256: string; readonly bytes: number },
expectedSha256: string,
signal?: AbortSignal,
): Promise<SafeOutputFile> {
this.#assertWritable();
const path = normalizeArtifactPath(relativePath);
this.#registry.add(path, "file");
const destination = await this.#prepareParent(path);
const handle = await open(
destination,
constants.O_CREAT |
constants.O_EXCL |
constants.O_WRONLY |
constants.O_NOFOLLOW,
0o600,
).catch(async (cause: unknown) => {
await throwIfDestinationCaseCollision(dirname(destination), path, cause);
throw new ArtifactReaderFailure(
"path",
`Could not exclusively create extraction path: ${path}`,
{ cause },
);
});
const hash = createHash("sha256");
let bytes = 0;
try {
this.#assertWritable();
if (!Number.isSafeInteger(expected.bytes) || expected.bytes < 0)
throw new ArtifactReaderFailure(
"format",
`Invalid expected extraction size for ${relativePath}`,
);
const path = normalizeArtifactPath(relativePath);
this.#registry.add(path, "file");
const destination = await this.#prepareParent(path);
const handle = await open(
destination,
constants.O_CREAT |
constants.O_EXCL |
constants.O_WRONLY |
constants.O_NOFOLLOW,
0o600,
).catch(async (cause: unknown) => {
await throwIfDestinationCaseCollision(
dirname(destination),
path,
cause,
);
throw new ArtifactReaderFailure(
"path",
`Could not exclusively create extraction path: ${path}`,
{ cause },
);
});
const hash = createHash("sha256");
let bytes = 0;
try {
for await (const raw of source) {
abortIfNeeded(signal);
const chunk = streamChunkToBuffer(raw);
if (chunk.length > expected.bytes - bytes)
throw new ArtifactReaderFailure(
"integrity",
`Extracted content exceeds the inventoried size: ${path}`,
);
bytes += chunk.length;
hash.update(chunk);
await writeAll(handle, chunk);
}
if (bytes !== expected.bytes)
throw new ArtifactReaderFailure(
"integrity",
`Extracted content size disagrees with inventory: ${path}`,
);
const sha256 = hash.digest("hex");
if (sha256 !== expected.sha256)
throw new ArtifactReaderFailure(
"integrity",
`Extracted content disagrees with inventory: ${path}`,
);
await handle.sync();
await handle.close();
const readback = await hashFile(destination, bytes, signal);
if (readback.sha256 !== sha256 || readback.bytes !== bytes)
throw new ArtifactReaderFailure(
"integrity",
`Durable readback verification failed: ${path}`,
);
return { relativePath: path, sha256, bytesWritten: bytes };
} catch (cause: unknown) {
// best-effort cleanup: file-handle close must not mask the write failure.
await handle.close().catch(() => undefined);
throw cause;
for await (const raw of source) {
abortIfNeeded(signal);
const chunk = streamChunkToBuffer(raw);
bytes += chunk.length;
hash.update(chunk);
await writeAll(handle, chunk);
}
const sha256 = hash.digest("hex");
if (sha256 !== expectedSha256)
throw new ArtifactReaderFailure(
"integrity",
`Extracted content disagrees with inventory: ${path}`,
);
await handle.sync();
await handle.close();
const readback = await hashFile(destination, bytes, signal);
if (readback.sha256 !== sha256 || readback.bytes !== bytes)
throw new ArtifactReaderFailure(
"integrity",
`Durable readback verification failed: ${path}`,
);
return { relativePath: path, sha256, bytesWritten: bytes };
} catch (cause: unknown) {
try {
source.destroy();
} catch {
// Preserve the write refusal or failure as the caller-visible error.
}
// best-effort cleanup: file-handle close must not mask the write failure.
await handle.close().catch(() => undefined);
throw cause;
}
}
+6 -47
View File
@@ -1,4 +1,4 @@
import type { FileHandle } from "node:fs/promises";
import { open, type FileHandle } from "node:fs/promises";
import { PassThrough, type Readable } from "node:stream";
import { once } from "node:events";
@@ -10,57 +10,20 @@ import {
type ArtifactReader,
} from "./ArtifactReader.js";
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
import {
NonRegularFileReadError,
openRegularFile,
sameRegularFileState,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
class NodeFileReader extends Reader<string> {
#handle: FileHandle | undefined;
#initPromise: Promise<void> | undefined;
readonly #ownsHandle: boolean;
constructor(
private readonly path: string,
private readonly maximumReadBytes?: number,
private readonly admitted?: StableRegularFileDescriptor,
) {
super(path);
this.#ownsHandle = admitted === undefined;
}
override init(): Promise<void> {
this.#initPromise ??= this.#initialize();
return this.#initPromise;
}
async #initialize(): Promise<void> {
try {
this.#handle =
this.admitted?.handle ??
(await openRegularFile(this.path, { symlinks: "reject" }));
} catch (cause: unknown) {
if (cause instanceof NonRegularFileReadError)
throw new ArtifactReaderFailure(
"format",
`ZIP source is not a regular file: ${this.path}`,
{ cause },
);
throw cause;
}
const metadata = await this.#handle.stat();
if (
!metadata.isFile() ||
(this.admitted !== undefined &&
!sameRegularFileState(this.admitted.initial, metadata))
)
throw new ArtifactReaderFailure(
"integrity",
`ZIP source identity changed before reading: ${this.path}`,
);
this.size = metadata.size;
override async init(): Promise<void> {
this.#handle = await open(this.path, "r");
this.size = (await this.#handle.stat()).size;
}
override async readUint8Array(
@@ -84,10 +47,7 @@ class NodeFileReader extends Reader<string> {
}
async closeHandle(): Promise<void> {
if (!this.#ownsHandle) return;
await this.#initPromise?.catch(() => undefined);
if (this.#handle === undefined) return;
await this.#handle.close();
await this.#handle?.close();
this.#handle = undefined;
}
}
@@ -104,10 +64,9 @@ export class ZipArtifactReader implements ArtifactReader {
path: string,
format: ZipPackageFormat,
maximumMetadataReadBytes?: number,
admitted?: StableRegularFileDescriptor,
) {
this.format = format;
this.#source = new NodeFileReader(path, maximumMetadataReadBytes, admitted);
this.#source = new NodeFileReader(path, maximumMetadataReadBytes);
this.#reader = new ZipReader(this.#source, {
checkSignature: true,
checkOverlappingEntry: true,
+10 -65
View File
@@ -25,7 +25,6 @@ import {
type NibHierarchyNode,
} from "./NibViewHierarchy.js";
import { jsonValueSchema, type JsonValue } from "../../domain/jsonValue.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import { decodeXmlPlistText } from "../../domain/propertyListXmlText.js";
@@ -50,6 +49,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
const prototypeKeyOmissions = new Map<string, number>();
let omitted = 0;
let attempted = 0;
let candidates = 0;
let aggregateInputBytes = 0;
const decodeBudget = new InterfaceBuilderDecodeBudget(
MAX_AGGREGATE_DECODE_BYTES,
@@ -60,16 +60,15 @@ export const analyzeInterfaceBuilderBundle = async (input: {
else unreportedOmittedArchives += 1;
};
try {
const selection = await selectInterfaceBuilderCandidates(
reader,
input.signal,
limits.max_documents,
);
omitted = selection.omitted;
for (const entry of selection.omittedSamples)
recordOmittedArchive(entry.path, "max_documents");
unreportedOmittedArchives += selection.unreportedOmitted;
for (const entry of selection.selected) {
for await (const entry of reader.entries(input.signal)) {
if (entry.kind !== "file" || !isInterfaceBuilderArchive(entry.path))
continue;
candidates += 1;
if (candidates > limits.max_documents) {
omitted += 1;
recordOmittedArchive(entry.path, "max_documents");
continue;
}
try {
const remainingInputBytes =
MAX_AGGREGATE_INPUT_BYTES - aggregateInputBytes;
@@ -168,60 +167,6 @@ export const analyzeInterfaceBuilderBundle = async (input: {
});
};
/** Select lexically earliest document paths independent of host traversal order. */
const selectInterfaceBuilderCandidates = async (
reader: DirectoryArtifactReader,
signal: AbortSignal | undefined,
maximumDocuments: number,
): Promise<{
readonly selected: readonly ArtifactEntry[];
readonly omittedSamples: readonly ArtifactEntry[];
readonly omitted: number;
readonly unreportedOmitted: number;
}> => {
const capacity = maximumDocuments + MAX_REPORTED_OMITTED_ARCHIVES;
const candidates: ArtifactEntry[] = [];
let total = 0;
for await (const entry of reader.entries(signal)) {
if (entry.kind !== "file" || !isInterfaceBuilderArchive(entry.path))
continue;
total += 1;
retainCandidate(candidates, entry, capacity);
}
const omittedSamples = candidates.slice(maximumDocuments, capacity);
return {
selected: candidates.slice(0, maximumDocuments),
omittedSamples,
omitted: Math.max(0, total - maximumDocuments),
unreportedOmitted: Math.max(
0,
total - maximumDocuments - omittedSamples.length,
),
};
};
/** Keep the earliest bounded path set; directory iteration order is host-defined. */
const retainCandidate = (
candidates: ArtifactEntry[],
entry: ArtifactEntry,
maximum: number,
): void => {
let low = 0;
let high = candidates.length;
while (low < high) {
const middle = Math.floor((low + high) / 2);
const existing = candidates[middle];
if (
existing !== undefined &&
compareUnicodeCodePoints(existing.path, entry.path) < 0
)
low = middle + 1;
else high = middle;
}
candidates.splice(low, 0, entry);
if (candidates.length > maximum) candidates.pop();
};
const finalizeHierarchyCoverage = (
result: ReturnType<typeof buildInterfaceBuilderAnalysis>,
counts: {
+13 -12
View File
@@ -147,28 +147,29 @@ export const inspectBundleKeyedArchive = async (input: {
try {
// Match inventory spelling while retaining the raw path for I/O and evidence.
const wanted = selected.path.normalize("NFC");
let entry: ArtifactEntry | undefined;
for await (const candidate of reader.entries(input.signal, (directory) =>
const matches: ArtifactEntry[] = [];
for await (const entry of reader.entries(input.signal, (directory) =>
wanted.startsWith(`${directory.normalize("NFC")}/`),
)) {
if (candidate.path.normalize("NFC") !== wanted) continue;
if (candidate.kind !== "file")
if (entry.path.normalize("NFC") !== wanted) continue;
if (entry.kind !== "file")
throw archivePathError(
"invalid_value",
`Archive path selects a ${candidate.kind}, not a regular file: ${selected.path}`,
`Archive path selects a ${entry.kind}, not a regular file: ${selected.path}`,
);
if ((candidate.declaredSize ?? 0) > MAX_BYTES)
if ((entry.declaredSize ?? 0) > MAX_BYTES)
throw new ArtifactReaderFailure(
"limit",
"Keyed archive exceeds 64 MiB",
);
if (entry !== undefined)
throw archivePathError(
"invalid_value",
`Archive path matches multiple Unicode-equivalent bundle entries; selection is ambiguous: ${selected.path}`,
);
entry = candidate;
matches.push(entry);
}
if (matches.length > 1)
throw archivePathError(
"invalid_value",
`Archive path matches ${String(matches.length)} Unicode-equivalent bundle entries; selection is ambiguous: ${selected.path}`,
);
const entry = matches[0];
if (entry === undefined)
throw archivePathError(
"invalid_value",
@@ -215,10 +215,7 @@ const materializeSelection = async ({
const written = await output.write(
path,
stream,
{
sha256: selectedItem.node.sha256,
bytes: selectedItem.node.size,
},
selectedItem.node.sha256,
signal,
);
extracted.push({
+7 -49
View File
@@ -5,7 +5,6 @@ import {
} from "../../filesystem/RegularFile.js";
import type { FileHandle } from "node:fs/promises";
import type { Stats } from "node:fs";
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
import { classifyArtifactContent } from "./ArtifactGraphConstruction.js";
import { ARTIFACT_CLASSIFICATION_PREFIX_BYTES } from "../ArtifactHash.js";
@@ -18,46 +17,18 @@ import { ArtifactReaderFailure } from "../ArtifactReader.js";
import type { HashResult } from "../ArtifactHash.js";
import { hashStableRootArtifactHandle } from "./hashStableRootArtifact.js";
interface RootClassification {
readonly format: ArtifactOccurrence["artifact_format"];
readonly digest: HashResult | null;
}
type RootInventoryClassification = RootClassification & {
readonly zipSource: StableRegularFileDescriptor | undefined;
};
/** Classify and hash one file root through the same stable open descriptor. */
export const classifyAndHashRoot = async (
path: string,
directory: boolean,
expectedMetadata: Stats,
signal?: AbortSignal,
): Promise<RootClassification> => {
const result = await classifyAndHashRootForInventory(
path,
directory,
expectedMetadata,
signal,
);
try {
return { format: result.format, digest: result.digest };
} finally {
await result.zipSource?.handle.close();
}
};
/** Classify a root and retain its admitted ZIP descriptor for child inventory. */
export const classifyAndHashRootForInventory = async (
path: string,
directory: boolean,
expectedMetadata: Stats,
signal?: AbortSignal,
): Promise<RootInventoryClassification> => {
if (directory)
return { format: "directory", digest: null, zipSource: undefined };
): Promise<{
readonly format: ArtifactOccurrence["artifact_format"];
readonly digest: HashResult | null;
}> => {
if (directory) return { format: "directory", digest: null };
const handle = await openRootFile(path, signal);
let transferHandle = false;
try {
const initial = await handle.stat();
if (!sameRegularFileState(expectedMetadata, initial))
@@ -72,25 +43,12 @@ export const classifyAndHashRootForInventory = async (
initial,
signal,
);
const zipSource = isZipFormat(format);
if (zipSource) transferHandle = true;
return {
format,
digest,
zipSource: zipSource ? { handle, initial } : undefined,
};
return { format, digest };
} finally {
if (!transferHandle) await handle.close();
await handle.close();
}
};
const isZipFormat = (format: ArtifactOccurrence["artifact_format"]): boolean =>
format === "zip" ||
format === "ipa" ||
format === "apk" ||
format === "msix" ||
format === "appx";
const openRootFile = async (
path: string,
signal?: AbortSignal,
@@ -1,5 +1,4 @@
import { lstat } from "node:fs/promises";
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
@@ -46,13 +45,11 @@ export const hashStableRootArtifact = async (
export const hashStableRootArtifactHandle = async (
path: string,
handle: FileHandle,
initial: Stats,
initial: Awaited<ReturnType<FileHandle["stat"]>>,
signal?: AbortSignal,
): Promise<HashResult> => {
const digest = await hashReadable(
// Read at most the admitted extent plus one byte to detect growth without
// following an actively appended file indefinitely.
handle.createReadStream({ autoClose: false, start: 0, end: initial.size }),
handle.createReadStream({ autoClose: false }),
signal,
);
abortIfNeeded(signal);
+1 -3
View File
@@ -6,14 +6,12 @@ import { NativeDmgArtifactReader } from "../NativeDmgArtifactReader.js";
import { ZipArtifactReader } from "../ZipArtifactReader.js";
import type { ArtifactOccurrence } from "../../domain/artifactGraph.js";
import { ZIP_NON_ENTRY_TAIL_LIMITATION } from "../../domain/zipPackageFormat.js";
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
export const createReader = async (
path: string,
format: ArtifactOccurrence["artifact_format"],
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
zipSource?: StableRegularFileDescriptor,
): Promise<ArtifactReader | undefined> => {
switch (format) {
case "directory":
@@ -23,7 +21,7 @@ export const createReader = async (
case "apk":
case "msix":
case "appx":
return new ZipArtifactReader(path, format, undefined, zipSource);
return new ZipArtifactReader(path, format);
case "asar":
return new AsarArtifactReader(path);
case "mach-o-universal":
+12 -58
View File
@@ -5,7 +5,6 @@ import { AsarArtifactReader } from "../AsarArtifactReader.js";
import { lstat } from "node:fs/promises";
import type { Stats } from "node:fs";
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
import {
ArtifactReaderFailure,
@@ -32,12 +31,9 @@ import {
artifactGraphDigest,
artifactManifestId,
} from "../../domain/artifactIdentity.js";
import { classifyAndHashRootForInventory } from "./classify.js";
import { classifyAndHashRoot } from "./classify.js";
import { type HashResult } from "../ArtifactHash.js";
import {
hashStableRootArtifact,
hashStableRootArtifactHandle,
} from "./hashStableRootArtifact.js";
import { hashStableRootArtifact } from "./hashStableRootArtifact.js";
import { createReader, inventoryLimitations } from "./reader.js";
import {
scanReader,
@@ -55,29 +51,18 @@ export const scanCanonicalArtifactInventory = async (
): Promise<ArtifactInventorySnapshot> => {
const integrity = options.integrity ?? STRICT_INTEGRITY_POLICY;
const metadata = await lstat(path);
const {
format: rootFormat,
digest: rootDigest,
zipSource,
} = await classifyAndHashRootForInventory(
const { format: rootFormat, digest: rootDigest } = await classifyAndHashRoot(
path,
metadata.isDirectory(),
metadata,
options.signal,
);
let reader: ArtifactReader | undefined;
let readerCreationFailure: { readonly cause: unknown } | undefined;
try {
reader = await readerFactory(
path,
rootFormat,
options.environment ?? {},
options.signal,
zipSource,
);
} catch (cause: unknown) {
readerCreationFailure = { cause };
}
const reader = await readerFactory(
path,
rootFormat,
options.environment ?? {},
options.signal,
);
const ownedReaders: ArtifactReader[] = reader === undefined ? [] : [reader];
let outcome:
| {
@@ -86,7 +71,6 @@ export const scanCanonicalArtifactInventory = async (
}
| { readonly kind: "failed"; readonly cause: unknown };
try {
if (readerCreationFailure !== undefined) throw readerCreationFailure.cause;
if (reader instanceof AsarArtifactReader)
await reader.prepareContainer(rootDigest?.sha256, options.signal);
const {
@@ -102,7 +86,6 @@ export const scanCanonicalArtifactInventory = async (
metadata,
rootFormat,
rootDigest,
zipSource,
reader,
signal: options.signal,
nodes,
@@ -133,24 +116,6 @@ export const scanCanonicalArtifactInventory = async (
);
}
}
if (zipSource !== undefined) {
try {
await zipSource.handle.close();
} catch (cleanupCause: unknown) {
const cleanup = ArtifactReaderFailure.cleanupObservation(
cleanupCause,
`root ZIP descriptor for ${path}`,
);
cleanupFailure = ArtifactReaderFailure.withCleanup(
cleanupFailure ??
(outcome.kind === "failed" ? outcome.cause : cleanupCause),
cleanup,
outcome.kind === "completed"
? { kind: "artifact-inventory", inventory: outcome.snapshot }
: undefined,
);
}
}
if (cleanupFailure !== undefined) throw cleanupFailure;
if (outcome.kind === "failed") throw outcome.cause;
return outcome.snapshot;
@@ -161,7 +126,6 @@ interface SnapshotBuildInput {
readonly metadata: Stats;
readonly rootFormat: ArtifactOccurrence["artifact_format"];
readonly rootDigest: HashResult | null;
readonly zipSource: StableRegularFileDescriptor | undefined;
readonly reader: ArtifactReader | undefined;
readonly signal: AbortSignal | undefined;
readonly nodes: Map<string, ArtifactNode>;
@@ -173,8 +137,7 @@ interface SnapshotBuildInput {
const buildInventorySnapshot = async (
input: SnapshotBuildInput,
): Promise<ArtifactInventorySnapshot> => {
const { path, metadata, rootFormat, rootDigest, zipSource, reader, signal } =
input;
const { path, metadata, rootFormat, rootDigest, reader, signal } = input;
materializeDirectoryNodes(input.occurrences, input.nodes);
const rootNode = createRootNode({
digest: rootDigest,
@@ -218,7 +181,7 @@ const buildInventorySnapshot = async (
ordinal,
}));
await verifyRootDigest(path, rootDigest, zipSource, signal);
await verifyRootDigest(path, rootDigest, signal);
const graphSha256 = artifactGraphDigest({
nodes: orderedNodes,
@@ -293,19 +256,10 @@ const buildIntegrityContradictions = (
const verifyRootDigest = async (
path: string,
rootDigest: HashResult | null,
zipSource: StableRegularFileDescriptor | undefined,
signal: AbortSignal | undefined,
): Promise<void> => {
if (rootDigest === null) return;
const verified =
zipSource === undefined
? await hashStableRootArtifact(path, signal)
: await hashStableRootArtifactHandle(
path,
zipSource.handle,
zipSource.initial,
signal,
);
const verified = await hashStableRootArtifact(path, signal);
if (
verified.sha256 !== rootDigest.sha256 ||
verified.bytes !== rootDigest.bytes
+1 -4
View File
@@ -38,8 +38,5 @@ export interface BrowserScenarioSessionPort {
/** Acquire one scenario session whose lifecycle the caller owns. */
export type BrowserScenarioSessionOpener = (
scenario: BrowserScenario,
options: {
readonly signal?: AbortSignal;
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
},
options: { readonly signal?: AbortSignal },
) => Promise<BrowserScenarioSessionPort>;
+2 -10
View File
@@ -33,7 +33,6 @@ export class CdpConnection {
>();
#nextId = 1;
#closed = false;
#closePromise: Promise<void> | undefined;
#protocolFailed = false;
#transportFailure: "disconnected" | "payload_limit" = "disconnected";
@@ -152,15 +151,8 @@ export class CdpConnection {
}
/** Close only REA's socket; never close the browser or selected page. */
close(): Promise<void> {
this.#closePromise ??= this.#closeTransport().catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
async #closeTransport(): Promise<void> {
async close(): Promise<void> {
if (this.#closed) return;
this.#closed = true;
this.#failPending("disconnected");
if (this.socket.readyState === WebSocket.CLOSED) return;
+46 -32
View File
@@ -50,40 +50,54 @@ export const closeCdpTargetSession = async (
signal?: AbortSignal,
): Promise<void> => {
const { connection, sessionId } = targetSession;
const cleanup = new AbortController();
const timeout = setTimeout(() => cleanup.abort(), 1_000);
timeout.unref();
try {
if (signal?.aborted !== true) {
for (const domain of enabledDomains) {
if (cleanup.signal.aborted) break;
try {
await connection.send(
`${domain}.disable`,
{},
sessionId,
cleanup.signal,
);
} catch {
// Best-effort domain cleanup; transport close is the definitive boundary.
}
}
if (!cleanup.signal.aborted && sessionId !== undefined)
try {
await connection.send(
"Target.detachFromTarget",
{ sessionId },
undefined,
cleanup.signal,
);
} catch {
// Best-effort detach; transport close is the definitive boundary.
}
for (const [index, domain] of enabledDomains.entries()) {
if (signal?.aborted === true) {
await closeCancelledTargetSession(
targetSession,
enabledDomains.slice(index),
);
return;
}
} finally {
clearTimeout(timeout);
cleanup.abort();
try {
await connection.send(`${domain}.disable`, {}, sessionId, signal);
} catch (cause: unknown) {
// best-effort cleanup: domain disable continues to detach/close boundary.
void cause;
}
}
if (signal?.aborted === true) {
await closeCancelledTargetSession(targetSession, []);
return;
}
if (sessionId !== undefined)
try {
await connection.send(
"Target.detachFromTarget",
{ sessionId },
undefined,
signal,
);
} catch (cause: unknown) {
// best-effort cleanup: closing REA's socket is the final cleanup boundary.
void cause;
}
await connection.close();
};
const closeCancelledTargetSession = async (
{ connection, sessionId }: CdpTargetSession,
enabledDomains: readonly string[],
): Promise<void> => {
const commands = enabledDomains.map((domain) =>
connection.send(`${domain}.disable`, {}, sessionId),
);
if (sessionId !== undefined)
commands.push(connection.send("Target.detachFromTarget", { sessionId }));
const settled = Promise.allSettled(commands);
try {
await connection.close();
} finally {
await settled;
}
};
+19 -8
View File
@@ -3,11 +3,6 @@ import { open, realpath, stat } from "node:fs/promises";
import { isAbsolute, join, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
admitAsarHeader,
MAX_ASAR_HEADER_BYTES,
} from "../artifacts/AsarHeader.js";
/** Resolve one local file URL while rejecting remote hosts and encoded separators. */
export const authorizedElectronFile = async (
value: string,
@@ -82,6 +77,7 @@ const authorizedAsarMember = async (
type ArchiveIdentity = Pick<Stats, "dev" | "ino" | "size" | "mtimeMs">;
// This is a retained-memory budget for JSON headers, shared across all archives.
const MAX_HEADER_BYTES = 16 * 1024 * 1024;
const MAX_CACHED_HEADERS = 8;
const headerCache = new Map<
string,
@@ -109,8 +105,23 @@ const asarFiles = async (
const opened = await handle.stat();
if (!opened.isFile() || archiveIdentity(opened) !== identity)
throw new Error("ASAR identity changed before header read");
const admitted = await admitAsarHeader(handle);
jsonBytes = admitted.jsonBytes;
const prefix = Buffer.alloc(16);
const read = await handle.read(prefix, 0, prefix.length, 0);
if (read.bytesRead !== prefix.length || prefix.readUInt32LE(0) !== 4)
throw new Error("Truncated or invalid ASAR size pickle");
const headerBytes = prefix.readUInt32LE(4);
const payloadBytes = prefix.readUInt32LE(8);
jsonBytes = prefix.readUInt32LE(12);
if (
headerBytes < 8 ||
headerBytes > opened.size - 8 ||
payloadBytes !== headerBytes - 4 ||
jsonBytes > payloadBytes - 4 ||
jsonBytes > MAX_HEADER_BYTES
)
throw new Error(
"ASAR JSON header leaves the file or exceeds its memory budget",
);
const json = Buffer.alloc(jsonBytes);
let offset = 0;
while (offset < json.length) {
@@ -140,7 +151,7 @@ const asarFiles = async (
[...headerCache.values()].reduce((sum, entry) => sum + entry.bytes, 0);
while (
headerCache.size > MAX_CACHED_HEADERS ||
retainedBytes() > MAX_ASAR_HEADER_BYTES
retainedBytes() > MAX_HEADER_BYTES
)
headerCache.delete(headerCache.keys().next().value ?? archive);
return files;
@@ -229,15 +229,14 @@ const runScenario = async (
openSession: BrowserScenarioSessionOpener,
scenario: BrowserScenario,
options: ExecutionOptions,
retainCleanup: (close: () => Promise<unknown>) => void,
): Promise<BrowserScenarioCapture> => {
if (options.signal?.aborted === true)
throw new BrowserObservationError(OPERATION, "cancelled");
const startedAt = Date.now();
const session = await openSession(scenario, {
...(options.signal === undefined ? {} : { signal: options.signal }),
retainCleanup,
});
const session = await openSession(
scenario,
options.signal === undefined ? {} : { signal: options.signal },
);
const steps: BrowserScenarioStep[] = [];
let operationOutcome:
| { readonly ok: true }
@@ -280,7 +279,6 @@ const runScenario = async (
cleanupOutcome = { ok: true, cleanup: await session.close() };
} catch (cause: unknown) {
cleanupOutcome = { ok: false, cause };
retainCleanup(() => session.close());
}
if (!cleanupOutcome.ok) {
const cleanup = browserScenarioCleanupObservation(cleanupOutcome.cause);
@@ -339,46 +337,18 @@ const runScenario = async (
/** Controlled Playwright/CDP scenario driver with exact process ownership. */
export class PlaywrightBrowserScenarioProvider implements BrowserScenarioCapturePort {
#tail: Promise<void> = Promise.resolve();
#closing = false;
#closePromise: Promise<void> | undefined;
readonly #pendingCleanup = new Set<() => Promise<unknown>>();
constructor(private readonly openSession: BrowserScenarioSessionOpener) {}
identity(): ProviderIdentity {
return PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY;
}
captureScenario(
async captureScenario(
scenario: BrowserScenario,
options: ExecutionOptions = {},
): Promise<Result<BrowserScenarioCapture, AnalysisError>> {
if (this.#closing) return Promise.resolve(err(this.#closedError()));
const operation = this.#tail.then(() =>
this.#closing
? err(this.#closedError())
: this.#captureScenario(scenario, options),
);
this.#tail = operation.then(
() => undefined,
() => undefined,
);
return operation;
}
async #captureScenario(
scenario: BrowserScenario,
options: ExecutionOptions,
): Promise<Result<BrowserScenarioCapture, AnalysisError>> {
const pending = await this.#retryCleanup();
if (pending !== undefined) return err(pending);
try {
return ok(
await runScenario(this.openSession, scenario, options, (close) => {
this.#pendingCleanup.add(close);
}),
);
return ok(await runScenario(this.openSession, scenario, options));
} catch (cause: unknown) {
if (cause instanceof AnalysisError) return err(cause);
return err(
@@ -390,43 +360,4 @@ export class PlaywrightBrowserScenarioProvider implements BrowserScenarioCapture
);
}
}
async close(): Promise<void> {
this.#closing = true;
this.#closePromise ??= this.#tail
.then(async () => {
const failure = await this.#retryCleanup();
if (failure !== undefined) throw failure;
})
.catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
#closedError(): AnalysisError {
return new BrowserObservationError(OPERATION, "cleanup_failed", {
detail: "Browser scenario provider is closing.",
cleanup: { reason: "Provider is closing", resources: [] },
});
}
async #retryCleanup(): Promise<AnalysisError | undefined> {
for (const pending of this.#pendingCleanup) {
try {
await pending();
this.#pendingCleanup.delete(pending);
} catch (cause: unknown) {
return cause instanceof AnalysisError
? cause
: new ProviderAdapterError(
PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY.id,
OPERATION,
{ cause },
);
}
}
return undefined;
}
}
+8 -32
View File
@@ -42,7 +42,6 @@ export class PlaywrightScenarioBrowserCleanupOwner {
#profileRemoved: boolean;
#closePromise: Promise<void> | undefined;
#eventFinalizationPromise: Promise<void> | undefined;
#eventFinalizationFailureReported = false;
/** Keep the narrow provider actions needed to release one opened browser. */
constructor(private readonly resources: PlaywrightScenarioCleanupResources) {
@@ -98,10 +97,7 @@ export class PlaywrightScenarioBrowserCleanupOwner {
try {
await eventFinalization;
} catch (cause: unknown) {
if (!this.#eventFinalizationFailureReported) {
failures.push(cause);
this.#eventFinalizationFailureReported = true;
}
failures.push(cause);
}
if (!this.#browserClosed) {
try {
@@ -384,7 +380,6 @@ const openAttachedScenarioBrowser = async (
environment: BrowserScenario["environment"],
launcher: ScenarioBrowserLauncher,
signal: AbortSignal | undefined,
retainCleanup: ((close: () => Promise<unknown>) => void) | undefined,
): Promise<OpenedScenarioBrowser> => {
const lease = await acquireAttachedScenario(browserRequest, signal);
let cleanupOwner: PlaywrightScenarioBrowserCleanupOwner | undefined;
@@ -436,13 +431,7 @@ const openAttachedScenarioBrowser = async (
throw cause;
}
const cleanup = cleanupOwner;
const close = () => cleanup.close();
try {
return await failBrowserScenarioOperation(close, cause);
} catch (failure: unknown) {
if (failure !== cause) retainCleanup?.(close);
throw failure;
}
return failBrowserScenarioOperation(() => cleanup.close(), cause);
}
};
@@ -453,7 +442,6 @@ export const openPlaywrightScenarioBrowser = async (
options: {
readonly launcher?: ScenarioBrowserLauncher;
readonly signal?: AbortSignal | undefined;
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
} = {},
): Promise<OpenedScenarioBrowser> => {
const launcher = options.launcher ?? chromium;
@@ -463,7 +451,6 @@ export const openPlaywrightScenarioBrowser = async (
scenario.environment,
launcher,
options.signal,
options.retainCleanup,
);
const profilePath = await mkdtemp(join(tmpdir(), "rea-browser-scenario-"));
@@ -491,16 +478,11 @@ export const openPlaywrightScenarioBrowser = async (
timeout: 0,
});
} catch (cause: unknown) {
const cleanup = () =>
rm(profilePath, { recursive: true, force: true, maxRetries: 3 });
try {
return await failBrowserScenarioOperation(cleanup, cause, [
"browser_profile",
]);
} catch (failure: unknown) {
if (failure !== cause) options.retainCleanup?.(cleanup);
throw failure;
}
return failBrowserScenarioOperation(
() => rm(profilePath, { recursive: true, force: true, maxRetries: 3 }),
cause,
["browser_profile"],
);
}
const cleanup = new PlaywrightScenarioBrowserCleanupOwner({
closeBrowser: () => context.close(),
@@ -520,12 +502,6 @@ export const openPlaywrightScenarioBrowser = async (
cleanup,
};
} catch (cause: unknown) {
const close = () => cleanup.close();
try {
return await failBrowserScenarioOperation(close, cause);
} catch (failure: unknown) {
if (failure !== cause) options.retainCleanup?.(close);
throw failure;
}
return failBrowserScenarioOperation(() => cleanup.close(), cause);
}
};
+18 -30
View File
@@ -127,7 +127,6 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
environment: Readonly<Record<string, string | undefined>>,
options: {
readonly signal?: AbortSignal;
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
},
): Promise<PlaywrightScenarioSession> {
if (options.signal?.aborted === true)
@@ -138,9 +137,6 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
const startedAt = Date.now();
const opening = openPlaywrightScenarioBrowser(scenario, environment, {
signal: options.signal,
...(options.retainCleanup === undefined
? {}
: { retainCleanup: options.retainCleanup }),
});
let opened: OpenedScenarioBrowser;
let events: PlaywrightScenarioEvents | undefined;
@@ -151,18 +147,14 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
options.signal,
);
} catch (cause: unknown) {
const lateCleanup = () =>
opening.then(
(lateOpened) => lateOpened.cleanup.close(undefined, options.signal),
() => undefined,
);
options.retainCleanup?.(() =>
withPlaywrightExecutionBoundary(lateCleanup, 1_000),
);
void lateCleanup().catch((cleanupCause: unknown) => {
// best-effort cleanup: retainCleanup keeps the retry capability.
void cleanupCause;
});
void opening
.then((lateOpened) =>
lateOpened.cleanup.close(undefined, options.signal),
)
.catch((cause: unknown) => {
// best-effort cleanup: late-open cleanup must not mask the boundary failure.
void cause;
});
throw cause;
}
let session: PlaywrightScenarioSession;
@@ -187,19 +179,16 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
eventCapture: events,
});
} catch (cause: unknown) {
const cleanup = () =>
opened.cleanup.close(
events === undefined
? undefined
: () => events?.finish() ?? Promise.resolve(),
options.signal,
);
try {
return await failBrowserScenarioOperation(cleanup, cause);
} catch (failure: unknown) {
if (failure !== cause) options.retainCleanup?.(cleanup);
throw failure;
}
return failBrowserScenarioOperation(
() =>
opened.cleanup.close(
events === undefined
? undefined
: () => events?.finish() ?? Promise.resolve(),
options.signal,
),
cause,
);
}
try {
await withPlaywrightExecutionBoundary(
@@ -223,7 +212,6 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
} catch (failure: unknown) {
cleanup = "incomplete";
cleanupFailure = { cause: failure };
options.retainCleanup?.(() => session.close());
}
throw browserScenarioOperationFailure(
cause,
+2 -7
View File
@@ -76,12 +76,7 @@ export const publishWebScripts = async (
bytes: bytes.length,
},
});
await tree.write(
"manifest.json",
Readable.from([bytes]),
{ sha256: digest, bytes: bytes.byteLength },
signal,
);
await tree.write("manifest.json", Readable.from([bytes]), digest, signal);
signal?.throwIfAborted();
await tree.commit();
return result;
@@ -114,7 +109,7 @@ const publishPlannedScript = async (
tree.write(
`files/${relativePath}`,
Readable.from([content.bytes]),
{ sha256: content.sha256, bytes: content.bytes.byteLength },
content.sha256,
signal,
);
try {
+8 -18
View File
@@ -24,24 +24,14 @@ export const registerEvmCommands = (
.describe("Selected carrier representation"),
}),
run: ({ args }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.inspectEvmInterface,
async () => {
const result = await service.inspect(
{ path: resolve(args.path), encoding: args.encoding },
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectEvmInterface, async () => {
const result = await service.inspect(
{ path: resolve(args.path), encoding: args.encoding },
{ signal },
);
} finally {
await service.close();
}
}),
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
};
+2 -6
View File
@@ -26,12 +26,8 @@ export const registerFirmwareCommands = (
) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
try {
const result = await service.execute(operation, input, { signal });
return result.ok ? result.value : projectAnalysisError(result.error);
} finally {
await service.close();
}
const result = await service.execute(operation, input, { signal });
return result.ok ? result.value : projectAnalysisError(result.error);
}),
);
const path = z.string().describe("Local firmware path; never executed");
+24 -28
View File
@@ -41,33 +41,29 @@ export const registerJavaScriptRecoveryCommands = (
),
}),
run: ({ args, options }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.recoverJavaScriptSources,
async () => {
const result = await service.recover(
{
path: args.path,
output_directory: args.outputDirectory,
...(options.extractionMode === undefined
? {}
: { extraction_mode: options.extractionMode }),
...(options.rewriteLevel === undefined
? {}
: { rewrite_level: options.rewriteLevel }),
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
withCommandCancellation((signal) =>
logCliCommand(
logger,
CLI_COMMANDS.recoverJavaScriptSources,
async () => {
const result = await service.recover(
{
path: args.path,
output_directory: args.outputDirectory,
...(options.extractionMode === undefined
? {}
: { extraction_mode: options.extractionMode }),
...(options.rewriteLevel === undefined
? {}
: { rewrite_level: options.rewriteLevel }),
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
),
),
});
};
+2 -17
View File
@@ -39,7 +39,6 @@ const supportedClientSchema = z
(candidate) => supportedClientIds.some((name) => name === candidate),
"Unsupported agent integration",
);
const skillClientSchema = z.union([z.literal("shared"), supportedClientSchema]);
/** Register setup, doctor, uninstall, and update CLI commands. */
export const registerSetupCommands = (
@@ -73,10 +72,6 @@ const registerSetupCommand = (
.array(supportedClientSchema)
.default([])
.describe("Agent integration to configure; repeat for multiple agents"),
skillClient: z
.array(skillClientSchema)
.default([])
.describe("Existing client skill destination to update; repeatable"),
allDetected: z
.boolean()
.default(false)
@@ -191,7 +186,6 @@ interface SetupCommandOptions {
readonly yes: boolean;
readonly installHopper: boolean;
readonly client: readonly string[];
readonly skillClient: readonly string[];
readonly allDetected: boolean;
readonly skill?: boolean | undefined;
readonly dryRun: boolean;
@@ -230,13 +224,8 @@ const setupRunOptions = (
): SetupOptions => {
const { options } = input;
const agentIntegrationSelected =
options.allDetected ||
options.client.length > 0 ||
options.skillClient.length > 0;
const hasSelectedClients =
hasExplicitScope &&
!options.allDetected &&
(options.client.length > 0 || options.skillClient.length > 0);
options.allDetected || options.client.length > 0;
const hasSelectedClients = hasExplicitScope && !options.allDetected;
return {
approved: options.yes && !options.dryRun,
installHopper: options.installHopper,
@@ -245,9 +234,6 @@ const setupRunOptions = (
allDetectedClients: options.allDetected,
proposeHopper: interactive || options.installHopper,
...(hasSelectedClients ? { clientIds: options.client } : {}),
...(options.skillClient.length > 0
? { skillClientIds: options.skillClient }
: {}),
...(hasExplicitScope
? { installSkill: options.skill ?? agentIntegrationSelected }
: {}),
@@ -278,7 +264,6 @@ const setupIsInteractive = (
const setupHasExplicitScope = (options: SetupCommandOptions): boolean =>
options.allDetected ||
options.client.length > 0 ||
options.skillClient.length > 0 ||
options.skill !== undefined ||
options.installHopper;
+12 -25
View File
@@ -38,32 +38,19 @@ export const registerWebSourceLocationCommands = (
.describe("Zero-based UTF-16 generated column"),
}),
run: ({ args }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.traceWebSourceLocation,
async () => {
const result = await service.trace(
{
manifest_path: args.manifestPath,
script_index: args.scriptIndex,
source_map: {
path: args.sourceMapPath,
url: args.sourceMapUrl,
},
generated_position: { line: args.line, column: args.column },
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.traceWebSourceLocation, async () => {
const result = await service.trace(
{
manifest_path: args.manifestPath,
script_index: args.scriptIndex,
source_map: { path: args.sourceMapPath, url: args.sourceMapUrl },
generated_position: { line: args.line, column: args.column },
},
{ signal },
);
} finally {
await service.close();
}
}),
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
};
-82
View File
@@ -1,82 +0,0 @@
import { expect, it } from "vitest";
import { adbInputSchemas } from "./adbDeviceAnalysis.js";
it("preserves ADB key, action and component admission when making advertised patterns portable", () => {
const serial = "emulator-5554";
const variants = [
{
previous: /^[A-Za-z0-9._-]+$/u,
max: 256,
accepts: (key: string) =>
adbInputSchemas.read_adb_setting.safeParse({
serial,
namespace: "global",
key,
}).success,
},
{
previous: /^[A-Za-z0-9._-]+$/u,
max: 256,
accepts: (key: string) =>
adbInputSchemas.start_adb_activity.safeParse({
serial,
action: "android.intent.action.VIEW",
extras: [{ key, type: "string", value: "payload" }],
}).success,
},
{
previous: /^[A-Za-z][A-Za-z0-9._-]*$/u,
max: 256,
accepts: (action: string) =>
adbInputSchemas.start_adb_activity.safeParse({ serial, action })
.success,
},
{
previous: /^[A-Za-z0-9._-]+(\/[A-Za-z0-9._-]*)?$/u,
max: 512,
accepts: (component: string) =>
adbInputSchemas.start_adb_activity.safeParse({
serial,
action: "android.intent.action.VIEW",
component,
}).success,
},
];
const samples = [
"",
"screen_brightness",
"0",
"_",
".",
"-",
"android.intent.action.VIEW",
"com.example/.MainActivity",
"com.example/",
"com.example/a/b",
"a\n",
"a\r\n",
"é",
"aé",
"a b",
"a\\b",
"a".repeat(256),
"a".repeat(257),
"a".repeat(512),
"a".repeat(513),
];
for (let code = 0; code < 128; code += 1) {
const character = String.fromCharCode(code);
samples.push(character, `a${character}`, `a${character}b`);
}
for (const variant of variants)
for (const sample of samples)
expect(
variant.accepts(sample),
JSON.stringify({ pattern: variant.previous.source, sample }),
).toBe(
sample.length > 0 &&
sample.length <= variant.max &&
variant.previous.test(sample),
);
});
+15 -10
View File
@@ -1,6 +1,5 @@
import { z } from "zod";
import { localPathStringSchema } from "../localPath.js";
import { IDENTIFIER_PATTERN } from "../stringPatterns.js";
/**
* Device serials as adb prints them: emulator-5554, USB identifiers, and
@@ -30,9 +29,6 @@ const packageNameSchema = z
"Package name must be dot-separated Java identifiers",
);
const keyStringSchema = z.string().min(1).max(256);
const keyPattern = /^[A-Za-z0-9._\x2d]+$/u;
/**
* Caller intent for ADB-backed device inspection and acquisition.
*
@@ -178,8 +174,14 @@ export const adbInputSchemas = {
namespace: z
.enum(["system", "secure", "global"])
.describe("Settings namespace to read"),
key: keyStringSchema
.regex(keyPattern, "Setting keys are restricted to identifiers")
key: z
.string()
.min(1)
.max(256)
.regex(
/^[A-Za-z0-9._\-]+$/u,
"Setting keys are restricted to identifiers",
)
.describe("Setting key to read"),
}),
collect_adb_bugreport: z.strictObject({
@@ -239,7 +241,7 @@ export const adbInputSchemas = {
.min(1)
.max(256)
.regex(
IDENTIFIER_PATTERN,
/^[A-Za-z][A-Za-z0-9._\-]*$/u,
"Intent action must be a dotted identifier such as android.intent.action.VIEW",
)
.describe("Intent action to start"),
@@ -255,7 +257,7 @@ export const adbInputSchemas = {
.min(1)
.max(512)
.regex(
/^[A-Za-z0-9._\x2d]+(\/[A-Za-z0-9._\x2d]*)?$/u,
/^[A-Za-z0-9._\-]+(\/[A-Za-z0-9._\-]*)?$/u,
"Component must be package/activity form",
)
.optional()
@@ -263,8 +265,11 @@ export const adbInputSchemas = {
extras: z
.array(
z.strictObject({
key: keyStringSchema
.regex(keyPattern, "Extra keys are identifiers")
key: z
.string()
.min(1)
.max(256)
.regex(/^[A-Za-z0-9._\-]+$/u, "Extra keys are identifiers")
.describe("Extra key"),
type: z
.enum(["string", "boolean", "int", "long", "float"])
+1 -3
View File
@@ -99,7 +99,6 @@ import type { BrowserScenarioPartialObservation } from "./browserScenarioCapture
import type { Evidence } from "./evidence.js";
import type { AndroidPartialObservation } from "./android/androidPartialObservation.js";
import type { FirmwarePartialObservation } from "./firmware/firmwareAnalysis.js";
import type { NativeUiObservationResult } from "./native/nativeUiObservation.js";
/** Provider-neutral evidence collected before a typed analysis failure. */
export type AnalysisPartialObservation =
@@ -112,5 +111,4 @@ export type AnalysisPartialObservation =
| ArtifactInventoryPartialObservation
| Evidence
| AndroidPartialObservation
| FirmwarePartialObservation
| NativeUiObservationResult;
| FirmwarePartialObservation;
-39
View File
@@ -1,39 +0,0 @@
import { ProviderAdapterError } from "./providerAdapterError.js";
import { projectAnalysisError } from "./analysisErrorProjection.js";
import type { AnalysisError } from "./analysisErrorBase.js";
/** Preserve the original lifecycle failure alongside incomplete cleanup. */
export const analysisErrorWithCleanupFailure = (
primary: AnalysisError,
cleanup: AnalysisError,
operation = "close_binary",
): AnalysisError => {
const projected = projectAnalysisError(cleanup);
return new ProviderAdapterError(
cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea",
operation,
{
cause: primary,
userMessage: projectAnalysisError(primary).message,
...(primary.capturedOutput === undefined
? {}
: { capturedOutput: primary.capturedOutput }),
...(primary.partialObservation === undefined
? {}
: { partialObservation: primary.partialObservation }),
cleanup: {
reason: projected.message,
resources: [
...new Set([
...primary.cleanupResources,
...cleanup.cleanupResources,
]),
],
},
diagnostics: {
primary_error: projectAnalysisError(primary),
cleanup_error: projected,
},
},
);
};
+18 -20
View File
@@ -66,9 +66,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
let omittedHierarchyReferences = 0;
for (const document of input.documents.slice(0, input.limits.max_documents)) {
const parsed = parseInterfaceBuilderRecords(document.raw);
const prefix = `ib:${JSON.stringify(document.relativePath)}:`;
const graphId = (kind: string, parts: readonly unknown[] = []): string =>
`${prefix}${kind}:${JSON.stringify(parts)}`;
const prefix = `ib:${document.relativePath}:`;
const rootId = `${prefix}root`;
const objectIds = new Set(parsed.objects.map(({ id }) => id));
const evidenceFor = (description: string) => [
@@ -106,7 +104,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
addNode(rootNode);
let documentObjectCount = 0;
for (const object of parsed.objects.slice(0, input.limits.max_objects)) {
const id = graphId("object", [object.id]);
const id = `${prefix}object:${object.id}`;
const kind = object.kind === "other" ? "unknown" : object.kind;
if (
!addNode({
@@ -131,7 +129,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
let documentOmittedConnections = 0;
let documentOmittedHierarchyEdges = 0;
const objectNodeId = (objectId: string): string =>
graphId("object", [objectId]);
`${prefix}object:${objectId}`;
const addEdge = (
edge: (typeof edges)[number],
isHierarchy: boolean,
@@ -240,7 +238,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
const known = objectIds.has(objectId);
const childId = known
? objectNodeId(objectId)
: graphId("unknown", [objectId]);
: `${prefix}unknown:${objectId}`;
if (
!known &&
!addNode({
@@ -254,7 +252,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
)
hierarchyTruncated = true;
connect(
graphId("hierarchy", [parentId, childId]),
`${prefix}hierarchy:${parentId}:${childId}`,
parentId,
childId,
"contains",
@@ -283,11 +281,11 @@ export const buildInterfaceBuilderAnalysis = (input: {
const destinationId =
item.destination_id === null ? null : objectNodeId(item.destination_id);
if (item.kind === "action") {
const actionId = graphId("action", [item.id]);
const actionId = `${prefix}action:${item.id}`;
const selectorId =
item.label === null
? null
: graphId("selector", [item.destination_id, item.label]);
: `${prefix}selector:${item.destination_id ?? "unknown"}:${item.label}`;
addNode({
id: actionId,
kind: "action",
@@ -301,7 +299,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
evidence: evidenceFor(`action ${item.id}`),
});
connect(
graphId("action-edge", [item.id, "source"]),
`${actionId}:source`,
sourceId,
actionId,
"target_action",
@@ -317,7 +315,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
evidence: evidenceFor(`selector ${item.label}`),
});
connect(
graphId("action-edge", [item.id, "selector"]),
`${actionId}:selector`,
actionId,
selectorId,
"target_action",
@@ -326,7 +324,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
}
if (destinationId === null || !nodes.has(destinationId))
connect(
graphId("action-edge", [item.id, "destination"]),
`${actionId}:destination`,
actionId,
null,
"target_action",
@@ -335,14 +333,14 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
else
connect(
graphId("action-edge", [item.id, "destination"]),
`${actionId}:destination`,
actionId,
destinationId,
"target_action",
`action destination ${item.destination_id}`,
);
} else if (item.kind === "outlet") {
const outletId = graphId("outlet", [item.id]);
const outletId = `${prefix}outlet:${item.id}`;
addNode({
id: outletId,
kind: "outlet",
@@ -352,7 +350,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
evidence: evidenceFor(`outlet ${item.id}`),
});
connect(
graphId("outlet-edge", [item.id, "source"]),
`${outletId}:source`,
sourceId,
outletId,
"contains",
@@ -360,7 +358,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
if (destinationId === null || !nodes.has(destinationId))
connect(
graphId("outlet-edge", [item.id, "destination"]),
`${outletId}:destination`,
outletId,
null,
"outlet_to",
@@ -369,7 +367,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
else
connect(
graphId("outlet-edge", [item.id, "destination"]),
`${outletId}:destination`,
outletId,
destinationId,
"outlet_to",
@@ -378,7 +376,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
} else if (item.kind === "segue") {
if (destinationId === null || !nodes.has(destinationId))
connect(
graphId("segue", [index]),
`${prefix}segue:${index}`,
sourceId,
null,
"segue_to",
@@ -387,7 +385,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
else
connect(
graphId("segue", [index]),
`${prefix}segue:${index}`,
sourceId,
destinationId,
"segue_to",
@@ -395,7 +393,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
} else {
connect(
graphId("unknown-connection", [index]),
`${prefix}unknown-connection:${index}`,
sourceId,
null,
"contains",
@@ -15,7 +15,7 @@ import {
} from "./javascriptStaticAnalysisHelpers.js";
import {
propertyName,
semanticStaticPropertyKey,
semanticStaticPropertyName,
} from "./javascriptAstValues.js";
import type { JavaScriptFindingContext } from "./javascriptStaticAnalysisState.js";
@@ -26,8 +26,6 @@ interface NativeBindingInput {
readonly kind: ElectronNativeAddonBindingFinding["binding_kind"];
readonly moduleKind: ElectronNativeAddonBindingFinding["module_kind"];
readonly members: readonly string[];
readonly namespaceAccess: boolean;
readonly dynamicMemberAccess: boolean;
}
/** Inspect JavaScript-side imports and re-exports of native .node addons. */
@@ -47,10 +45,10 @@ const inspectImport = (
context: JavaScriptFindingContext,
): void => {
if (!isNativeSpecifier(node.source.value, "import")) return;
const members = node.specifiers.flatMap((specifier) => {
if (t.isImportDefaultSpecifier(specifier)) return ["default"];
if (t.isImportNamespaceSpecifier(specifier)) return [];
return [propertyName(specifier.imported)];
const members = node.specifiers.map((specifier) => {
if (t.isImportDefaultSpecifier(specifier)) return "default";
if (t.isImportNamespaceSpecifier(specifier)) return "*";
return propertyName(specifier.imported) || "[dynamic-import]";
});
addBinding({
context,
@@ -59,10 +57,6 @@ const inspectImport = (
kind: "import",
moduleKind: "import",
members,
namespaceAccess: node.specifiers.some((value) =>
t.isImportNamespaceSpecifier(value),
),
dynamicMemberAccess: false,
});
};
@@ -76,10 +70,9 @@ const inspectNamedExport = (
!isNativeSpecifier(node.source.value, "import")
)
return;
const members = node.specifiers.flatMap((specifier) => {
if (t.isExportSpecifier(specifier)) return [propertyName(specifier.local)];
if (t.isExportDefaultSpecifier(specifier)) return ["default"];
return [];
const members = node.specifiers.map((specifier) => {
if (t.isExportSpecifier(specifier)) return propertyName(specifier.local);
return "*";
});
addBinding({
context,
@@ -88,10 +81,6 @@ const inspectNamedExport = (
kind: "re-export",
moduleKind: "import",
members,
namespaceAccess: node.specifiers.some((value) =>
t.isExportNamespaceSpecifier(value),
),
dynamicMemberAccess: false,
});
};
@@ -110,9 +99,7 @@ const inspectExportAll = (
specifier: node.source.value,
kind: "re-export",
moduleKind: "import",
members: [],
namespaceAccess: true,
dynamicMemberAccess: false,
members: ["*"],
});
};
@@ -128,7 +115,8 @@ const inspectRequireBinding = (
specifier: required.specifier,
kind: "require",
moduleKind: "require",
...(required.hasMemberAccess ? required.access : bindingMembers(node.id)),
members:
required.member === null ? bindingMembers(node.id) : [required.member],
});
};
@@ -144,33 +132,27 @@ const inspectReExport = (
specifier: required.specifier,
kind: "re-export",
moduleKind: "require",
...required.access,
members: [required.member ?? exportedMember(node.left) ?? "*"],
});
};
const addBinding = (input: NativeBindingInput): void => {
const { context, node, specifier, kind } = input;
const members = [...new Set(input.members)].sort(compareUnicodeCodePoints);
const unique = [
...new Set(input.members.filter((member) => member !== "")),
].sort(compareUnicodeCodePoints);
const members = unique.length === 0 ? ["*"] : unique;
addLocatedFinding(context, {
collection: context.accumulator.nativeAddonBindings,
// Members are a variable-length source-derived list: compositeKey keeps
// ["a\0b"] distinct from ["a", "b"] instead of collapsing one binding.
key: compositeKey([
"native-addon-binding",
kind,
specifier,
members,
input.namespaceAccess,
input.dynamicMemberAccess,
]),
key: compositeKey(["native-addon-binding", kind, specifier, members]),
node,
value: {
specifier,
binding_kind: kind,
module_kind: input.moduleKind,
members,
namespace_access: input.namespaceAccess,
dynamic_member_access: input.dynamicMemberAccess,
module_key: null,
location: range(node),
},
@@ -180,20 +162,13 @@ const addBinding = (input: NativeBindingInput): void => {
const nativeRequire = (
node: t.Node | null | undefined,
):
| {
readonly specifier: string;
readonly hasMemberAccess: boolean;
readonly access: NativeMemberAccess;
}
| { readonly specifier: string; readonly member: string | null }
| undefined => {
let member: string | null = null;
let hasMemberAccess = false;
while (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) {
if (!t.isNode(node.object)) return undefined;
const property = semanticStaticPropertyKey(node.property, node.computed);
// Walking inward leaves the first member consumed from the addon itself.
member = property;
hasMemberAccess = true;
const property = semanticStaticPropertyName(node.property, node.computed);
if (member === null && property !== "") member = property;
node = node.object;
}
if (!t.isCallExpression(node)) return undefined;
@@ -206,52 +181,34 @@ const nativeRequire = (
return undefined;
const specifier = stringValue(argumentNode(node.arguments[0]));
return specifier !== undefined && isNativeSpecifier(specifier, "require")
? {
specifier,
hasMemberAccess,
access: {
members: member === null ? [] : [member],
namespaceAccess: !hasMemberAccess,
dynamicMemberAccess: hasMemberAccess && member === null,
},
}
? { specifier, member }
: undefined;
};
interface NativeMemberAccess {
readonly members: readonly string[];
readonly namespaceAccess: boolean;
readonly dynamicMemberAccess: boolean;
}
const bindingMembers = (pattern: t.Node): NativeMemberAccess => {
if (!t.isObjectPattern(pattern))
return { members: [], namespaceAccess: true, dynamicMemberAccess: false };
const members: string[] = [];
let namespaceAccess = false;
let dynamicMemberAccess = false;
for (const property of pattern.properties) {
if (t.isRestElement(property)) {
namespaceAccess = true;
continue;
}
const key = semanticStaticPropertyKey(property.key, property.computed);
if (key === null) dynamicMemberAccess = true;
else members.push(key);
}
return { members, namespaceAccess, dynamicMemberAccess };
const bindingMembers = (pattern: t.Node): string[] => {
if (t.isIdentifier(pattern)) return ["*"];
if (!t.isObjectPattern(pattern)) return ["*"];
return pattern.properties.map((property) => {
if (t.isRestElement(property)) return "*";
return propertyName(property.key) || "[dynamic-import]";
});
};
const isModuleExport = (node: t.Node): boolean => {
const keys: (string | null)[] = [];
while (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) {
keys.unshift(semanticStaticPropertyKey(node.property, node.computed));
node = node.object;
}
if (!t.isIdentifier(node)) return false;
return node.name === "exports"
? keys.length > 0
: node.name === "module" && keys[0] === "exports";
const name = calleeName(node);
return (
name === "module.exports" ||
name.startsWith("module.exports.") ||
name.startsWith("exports.")
);
};
const exportedMember = (node: t.Node): string | undefined => {
const name = calleeName(node);
if (name.startsWith("module.exports."))
return name.slice("module.exports.".length);
if (name.startsWith("exports.")) return name.slice("exports.".length);
return undefined;
};
const isNativeSpecifier = (
@@ -163,11 +163,7 @@ export interface ElectronNativeAddonBindingFinding {
readonly specifier: string;
readonly binding_kind: "import" | "require" | "re-export";
readonly module_kind: "import" | "require";
/** Exact requested addon member keys; namespace and dynamic access are separate. */
readonly members: readonly string[];
/** Omitted in legacy findings; omission means unknown, not false. */
readonly namespace_access?: boolean;
readonly dynamic_member_access?: boolean;
readonly module_key: string | null;
readonly location: JavaScriptSourceRange;
}
@@ -257,15 +257,7 @@ const semanticValue = (
return (
joined(
properties.specifier,
compositeKey([
stringList(properties.requested_members),
typeof properties.namespace_access === "boolean"
? properties.namespace_access
: null,
typeof properties.dynamic_member_access === "boolean"
? properties.dynamic_member_access
: null,
]),
stringList(properties.requested_members).join(","),
) ?? firstLabel(node)
);
if (
@@ -103,12 +103,7 @@ const nativeExportNodes = (
const requestedExports = (nodes: readonly ApplicationNode[]): string[] => {
const values = nodes.flatMap((node) =>
node.observations.flatMap((observation) => [
// Display labels can summarize several names or namespace access.
// Only legacy observations without structured member facts need them.
...(observation.label === null ||
Array.isArray(observation.properties.requested_members)
? []
: [observation.label]),
...(observation.label === null ? [] : [observation.label]),
...stringArray(observation.properties.requested_members),
...stringArray(observation.properties.members),
]),
-1
View File
@@ -282,7 +282,6 @@ const managedFieldAccessSchema = z.object({
const managedParseIssueSchema = z.object({
code: z.enum([
"resource-limit",
"invalid-cli-header",
"invalid-directory",
"invalid-metadata-root",
-2
View File
@@ -156,5 +156,3 @@ export const nativeUiResultSchema = z.strictObject({
restore: z.literal("leave-as-is"),
limitations: z.array(z.string()),
});
export type NativeUiObservationResult = z.infer<typeof nativeUiResultSchema>;
+18 -9
View File
@@ -4,10 +4,7 @@ import { z } from "zod";
import { canonicalJson } from "./comparisonSemantics.js";
import { digestCanonicalValue } from "./canonicalDigest.js";
import {
createReferenceSourcePathLookup,
isPortableAbsoluteReferenceTarget,
} from "./referenceSourcePathIdentity.js";
import { normalizeJoinedLogicalPath } from "./artifactIdentity.js";
import { compareUnicodeCodePoints } from "./unicodeCodePointOrder.js";
import { digestSchema } from "./../domain/digests.js";
import { prefixedDigestSchema } from "./../domain/digests.js";
@@ -85,13 +82,18 @@ const symlinkTargetSchema = z
.min(1)
.refine(
(target) =>
isPortableAbsoluteReferenceTarget(target) ||
isPortableAbsoluteSymlinkTarget(target) ||
(!target.startsWith("/") &&
!target.includes("\\") &&
posix.normalize(target) === target),
"Expected a normalized POSIX symlink target",
);
const isPortableAbsoluteSymlinkTarget = (target: string): boolean =>
target.startsWith("/") ||
/^[A-Za-z]:[\\/]/u.test(target) ||
/^\\\\[^\\/]+[\\/][^\\/]+/u.test(target);
const sourceSymlinkSchema = z.union([
z.strictObject({
...entryBaseShape,
@@ -312,11 +314,18 @@ const checkSymlinks = (
paths: ReadonlySet<string>,
context: z.RefinementCtx,
): void => {
const lookupPath = createReferenceSourcePathLookup(paths);
const normalizedPathCounts = new Map<string, number>();
for (const path of paths) {
const identity = normalizeJoinedLogicalPath(path);
normalizedPathCounts.set(
identity,
(normalizedPathCounts.get(identity) ?? 0) + 1,
);
}
for (const [index, entry] of graph.entries.entries()) {
if (entry.kind !== "symlink") continue;
if (entry.target === null) continue;
const absolute = isPortableAbsoluteReferenceTarget(entry.target);
const absolute = isPortableAbsoluteSymlinkTarget(entry.target);
if (
(entry.target_state === "external" && !absolute) ||
(entry.target_state === "internal" && absolute)
@@ -330,11 +339,11 @@ const checkSymlinks = (
const resolved = posix.normalize(
posix.join(posix.dirname(entry.path), entry.target),
);
const normalized = normalizeJoinedLogicalPath(resolved);
if (
resolved.startsWith("../") ||
resolved === ".." ||
// The inventory root is implicit rather than an entry with path ".".
(resolved !== "." && lookupPath.resolve(resolved) === undefined)
(!paths.has(resolved) && normalizedPathCounts.get(normalized) !== 1)
)
context.addIssue({
code: "custom",
-33
View File
@@ -1,33 +0,0 @@
export interface ReferenceSourcePathLookup {
readonly resolve: (path: string) => string | undefined;
readonly resolveCandidates: (paths: readonly string[]) => string | undefined;
}
/**
* Reference inventories retain filesystem spelling. Prefer that exact identity;
* canonical Unicode equivalence is a fallback only when it identifies one entry.
*/
export const createReferenceSourcePathLookup = (
paths: Iterable<string>,
): ReferenceSourcePathLookup => {
const exact = new Set(paths);
const canonical = new Map<string, string | null>();
for (const path of exact) {
const key = path.normalize("NFC");
canonical.set(key, canonical.has(key) ? null : path);
}
const equivalent = (path: string): string | undefined =>
canonical.get(path.normalize("NFC")) ?? undefined;
return {
resolve: (path) => (exact.has(path) ? path : equivalent(path)),
resolveCandidates: (candidates) =>
candidates.find((path) => exact.has(path)) ??
candidates.map(equivalent).find((path) => path !== undefined),
};
};
/** Symlink evidence can originate on a different OS from the graph reader. */
export const isPortableAbsoluteReferenceTarget = (target: string): boolean =>
target.startsWith("/") ||
/^[A-Za-z]:[\\/]/u.test(target) ||
/^\\\\[^\\/]+[\\/][^\\/]+/u.test(target);
+82 -108
View File
@@ -1,7 +1,3 @@
import {
admitManagedProjection,
managedDecodeBudget,
} from "./ManagedDecodeBudget.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
managedArtifactInspectionSchema,
@@ -111,44 +107,40 @@ const unavailableResult = (
: [
"The PE has no CLI data directory; managed metadata and CIL are unavailable.",
];
return managedArtifactInspectionSchema.parse(
admitManagedProjection({
...base(target, bytes, layout),
classification: {
status: malformed ? "malformed" : "not-managed",
container: "pe",
runtime_family: "unknown",
implementation: malformed ? "unknown" : "not-managed",
managed_architecture: "unknown",
evidence: [
{
code: malformed
? "cli-directory-malformed"
: "cli-directory-absent",
detail: limitations[0],
file_offset: layout.cliIssue?.offset ?? null,
},
],
},
metadata: {
status: malformed ? "malformed" : "absent",
version: null,
stream_names: [],
table_row_counts: {},
},
module: null,
assembly: null,
target_frameworks: [],
references: [],
resources: [],
attributes: [],
coverage: {
state: malformed ? "partial" : "unavailable",
issues,
},
limitations,
}),
);
return managedArtifactInspectionSchema.parse({
...base(target, bytes, layout),
classification: {
status: malformed ? "malformed" : "not-managed",
container: "pe",
runtime_family: "unknown",
implementation: malformed ? "unknown" : "not-managed",
managed_architecture: "unknown",
evidence: [
{
code: malformed ? "cli-directory-malformed" : "cli-directory-absent",
detail: limitations[0],
file_offset: layout.cliIssue?.offset ?? null,
},
],
},
metadata: {
status: malformed ? "malformed" : "absent",
version: null,
stream_names: [],
table_row_counts: {},
},
module: null,
assembly: null,
target_frameworks: [],
references: [],
resources: [],
attributes: [],
coverage: {
state: malformed ? "partial" : "unavailable",
issues,
},
limitations,
});
};
interface RuntimeClassificationInput {
@@ -296,7 +288,7 @@ const partialMetadataResult = ({
}: PartialMetadataContext): ManagedArtifactInspection => ({
...base(target, bytes, layout),
classification: {
status: issue.code === "resource-limit" ? "managed" : "malformed",
status: "malformed",
container: "pe",
runtime_family:
(layout.cli?.flags ?? 1) & 1 ? "unknown" : "mixed-clr-native",
@@ -311,7 +303,7 @@ const partialMetadataResult = ({
],
},
metadata: {
status: issue.code === "resource-limit" ? "partial" : "malformed",
status: "malformed",
version: null,
stream_names: [],
table_row_counts: {},
@@ -324,9 +316,7 @@ const partialMetadataResult = ({
attributes: [],
coverage: { state: "partial", issues: [issue] },
limitations: [
issue.code === "resource-limit"
? "CLI metadata inspection stopped at its representation budget; omitted facts are unknown."
: "CLI metadata could not be read because the PE/CLI structure is malformed or unsupported.",
"CLI metadata could not be read because the PE/CLI structure is malformed or unsupported.",
],
});
@@ -353,14 +343,12 @@ export const inspectManagedArtifactBytes = (
} catch (cause: unknown) {
if (!(cause instanceof ManagedReaderFailure)) throw cause;
return managedArtifactInspectionSchema.parse(
admitManagedProjection(
partialMetadataResult({
target,
bytes,
layout,
issue: cause.issue,
}),
),
partialMetadataResult({
target,
bytes,
layout,
issue: cause.issue,
}),
);
}
const resourceIssues: ManagedParseIssue[] = [];
@@ -368,57 +356,43 @@ export const inspectManagedArtifactBytes = (
layout,
resourceIssues,
);
try {
const inventory = readManagedMetadataInventory(
bytes,
metadata,
resourceDirectory,
);
const issues = [...resourceIssues, ...inventory.issues];
const limitations = [
...(issues.length === 0
? []
: [
"At least one metadata or resource item could not be read; coverage is partial.",
]),
"Static inspection did not load the assembly, resolve dependencies through a CLR, decompile C#, or execute target code.",
];
return managedArtifactInspectionSchema.parse(
admitManagedProjection(
{
...base(target, bytes, layout),
classification: {
status: "managed",
container: "pe",
...runtimeClassification({ layout, metadata, inventory }),
},
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: metadata.version,
stream_names: metadata.streamNames,
table_row_counts: managedTableRowCounts(metadata),
},
module: inventory.module,
assembly: inventory.assembly,
target_frameworks: inventory.targetFrameworks,
references: inventory.references,
resources: inventory.resources,
attributes: inventory.attributes,
coverage: {
state: issues.length === 0 ? "complete" : "partial",
issues,
},
limitations,
},
managedDecodeBudget(metadata),
),
);
} catch (cause: unknown) {
if (!(cause instanceof ManagedReaderFailure)) throw cause;
return managedArtifactInspectionSchema.parse(
admitManagedProjection(
partialMetadataResult({ target, bytes, layout, issue: cause.issue }),
),
);
}
const inventory = readManagedMetadataInventory(
bytes,
metadata,
resourceDirectory,
);
const issues = [...resourceIssues, ...inventory.issues];
const limitations = [
...(issues.length === 0
? []
: [
"At least one metadata or resource item could not be read; coverage is partial.",
]),
"Static inspection did not load the assembly, resolve dependencies through a CLR, decompile C#, or execute target code.",
];
return managedArtifactInspectionSchema.parse({
...base(target, bytes, layout),
classification: {
status: "managed",
container: "pe",
...runtimeClassification({ layout, metadata, inventory }),
},
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: metadata.version,
stream_names: metadata.streamNames,
table_row_counts: managedTableRowCounts(metadata),
},
module: inventory.module,
assembly: inventory.assembly,
target_frameworks: inventory.targetFrameworks,
references: inventory.references,
resources: inventory.resources,
attributes: inventory.attributes,
coverage: {
state: issues.length === 0 ? "complete" : "partial",
issues,
},
limitations,
});
};
-106
View File
@@ -1,106 +0,0 @@
import { createHash } from "node:crypto";
import type { ManagedMetadataLayout } from "./ManagedMetadataLayout.js";
import { managedFailure } from "./ManagedReaderFailure.js";
/** Per-inspection capacity for eagerly retained metadata and its JSON projection. */
const MAX_METADATA_REPRESENTATION_BYTES = 64 * 1024 * 1024;
interface CachedString {
readonly value: string;
readonly encodedBytes: number;
}
/** The layout owns decoding capacity and heap identity for all of its consumers. */
export class ManagedDecodeBudget {
#remaining = MAX_METADATA_REPRESENTATION_BYTES;
readonly #strings = new Map<number, CachedString>();
readonly #digests = new WeakMap<ArrayBufferLike, Map<string, string>>();
reserve(bytes: number, offset: number | null): void {
if (!Number.isSafeInteger(bytes) || bytes < 0 || bytes > this.#remaining)
throw managedFailure(
"resource-limit",
"metadata.representation",
`Managed metadata exceeds the ${String(MAX_METADATA_REPRESENTATION_BYTES)} byte representation budget`,
offset,
);
this.#remaining -= bytes;
}
/** UTF-8 cannot decode to more UTF-16 units than bytes; JSON uses at most six bytes per unit. */
reserveString(encodedBytes: number, offset: number | null): void {
this.reserve(encodedBytes * 6 + 2, offset);
}
/** Snapshot buffers are immutable within a layout; cache exact backing-buffer ranges. */
digest(bytes: Buffer, algorithm: "sha256" | "sha1"): string {
let ranges = this.#digests.get(bytes.buffer);
if (ranges === undefined) {
ranges = new Map();
this.#digests.set(bytes.buffer, ranges);
}
const key = `${algorithm}:${String(bytes.byteOffset)}:${String(bytes.byteLength)}`;
const cached = ranges.get(key);
if (cached !== undefined) return cached;
this.reserve(256, null);
const digest = createHash(algorithm).update(bytes).digest("hex");
ranges.set(key, digest);
return digest;
}
cachedString(index: number): CachedString | undefined {
return this.#strings.get(index);
}
cacheString(index: number, value: string, encodedBytes: number): void {
this.#strings.set(index, { value, encodedBytes });
}
}
const budgets = new WeakMap<ManagedMetadataLayout, ManagedDecodeBudget>();
/** Transfer metadata-root admission capacity to its completed immutable layout. */
export const attachManagedDecodeBudget = (
layout: ManagedMetadataLayout,
budget: ManagedDecodeBudget,
): void => {
budgets.set(layout, budget);
};
/** Share one owner for the lifetime of the admitted layout, including all inspection facets. */
export const managedDecodeBudget = (
layout: ManagedMetadataLayout,
): ManagedDecodeBudget => {
let budget = budgets.get(layout);
if (budget === undefined) {
budget = new ManagedDecodeBudget();
budgets.set(layout, budget);
}
return budget;
};
/** Admit repeated values too: shared heap strings still expand at the JSON boundary. */
export const admitManagedProjection = <Value>(
value: Value,
budget: ManagedDecodeBudget = new ManagedDecodeBudget(),
): Value => {
const visit = (item: unknown): void => {
if (typeof item === "string") {
budget.reserveString(item.length, null);
} else if (Array.isArray(item)) {
budget.reserve(item.length + 2, null);
for (const child of item) visit(child);
} else if (item !== null && typeof item === "object") {
budget.reserve(2, null);
for (const key of Object.keys(item)) {
budget.reserve(key.length * 6 + 4, null);
visit(Reflect.get(item, key));
}
} else {
budget.reserve(32, null);
}
};
visit(value);
return value;
};
+69 -87
View File
@@ -1,7 +1,3 @@
import {
admitManagedProjection,
managedDecodeBudget,
} from "./ManagedDecodeBudget.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
managedMemberInspectionSchema,
@@ -35,47 +31,40 @@ const unavailable = (
bytes: Buffer,
issue: ManagedParseIssue | null,
): ManagedMemberInspection =>
managedMemberInspectionSchema.parse(
admitManagedProjection({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: null,
metadata: {
status:
issue === null
? "absent"
: issue.code === "resource-limit"
? "partial"
: "malformed",
version: null,
table_row_counts: {},
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: null,
},
types: [],
fields: [],
methods: [],
member_refs: [],
call_edges: [],
field_accesses: [],
coverage: {
state: "unavailable",
issues: issue === null ? [] : [issue],
},
limitations: [
issue === null
? "The PE has no admitted CLI metadata; managed member inspection is unavailable."
: "The CLI metadata could not be admitted; managed member inspection is unavailable.",
],
}),
);
managedMemberInspectionSchema.parse({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: null,
metadata: {
status: issue === null ? "absent" : "malformed",
version: null,
table_row_counts: {},
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: null,
},
types: [],
fields: [],
methods: [],
member_refs: [],
call_edges: [],
field_accesses: [],
coverage: {
state: "unavailable",
issues: issue === null ? [] : [issue],
},
limitations: [
issue === null
? "The PE has no admitted CLI metadata; managed member inspection is unavailable."
: "The CLI metadata could not be admitted; managed member inspection is unavailable.",
],
});
const readMemberInventory = (
bytes: Buffer,
@@ -143,48 +132,41 @@ export const inspectManagedMembersBytes = (
fields.core,
memberRefs.core,
);
return managedMemberInspectionSchema.parse(
admitManagedProjection(
{
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: inventory.module,
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: layout.version,
table_row_counts: managedTableRowCounts(layout),
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: inventory.module?.mvid ?? null,
},
types: parsedTypes.types,
fields: fields.fields,
methods: methods.methods,
member_refs: memberRefs.refs,
call_edges: related.callEdges,
field_accesses: related.fieldAccesses,
coverage: {
state:
issues.length === 0 && !incompleteMembers
? "complete"
: "partial",
issues: coverageIssues,
},
limitations: [
"Metadata tokens are build-local coordinates and are only meaningful with the reported artifact SHA-256 and MVID.",
"CIL instruction anchors are decoded from file-backed method bodies only; no target assembly is loaded or executed.",
"Signatures are decoded for common ECMA-335 primitive, class, valuetype, pointer, byref, array, and generic variable forms; unsupported forms retain raw signature hashes.",
],
},
managedDecodeBudget(layout),
),
);
return managedMemberInspectionSchema.parse({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: inventory.module,
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: layout.version,
table_row_counts: managedTableRowCounts(layout),
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: inventory.module?.mvid ?? null,
},
types: parsedTypes.types,
fields: fields.fields,
methods: methods.methods,
member_refs: memberRefs.refs,
call_edges: related.callEdges,
field_accesses: related.fieldAccesses,
coverage: {
state:
issues.length === 0 && !incompleteMembers ? "complete" : "partial",
issues: coverageIssues,
},
limitations: [
"Metadata tokens are build-local coordinates and are only meaningful with the reported artifact SHA-256 and MVID.",
"CIL instruction anchors are decoded from file-backed method bodies only; no target assembly is loaded or executed.",
"Signatures are decoded for common ECMA-335 primitive, class, valuetype, pointer, byref, array, and generic variable forms; unsupported forms retain raw signature hashes.",
],
});
} catch (cause: unknown) {
if (cause instanceof ManagedReaderFailure)
return unavailable(target, bytes, cause.issue);
+2 -5
View File
@@ -405,13 +405,10 @@ const callingConvention = (value: number): string => {
return flags.length === 0 ? name : `${name} ${flags.join(" ")}`;
};
export const signature = (
blob: Buffer,
rawSha256?: string,
): ManagedSignature => {
export const signature = (blob: Buffer): ManagedSignature => {
const raw = {
raw_length: blob.length,
raw_sha256: rawSha256 ?? sha256Bytes(blob),
raw_sha256: sha256Bytes(blob),
};
try {
if (blob.length === 0)
+1 -10
View File
@@ -1,4 +1,3 @@
import type { ManagedDecodeBudget } from "./ManagedDecodeBudget.js";
import type {
ManagedExceptionRegion,
ManagedInstructionAnchor,
@@ -435,7 +434,6 @@ export const parseExceptionRegions = (
bytes: Buffer,
offset: number,
methodEnd: number,
budget?: ManagedDecodeBudget,
): ExceptionRegionParseResult => {
const regions: ManagedExceptionRegion[] = [];
let sectionOffset = offset;
@@ -489,14 +487,7 @@ export const parseExceptionRegions = (
);
const count = (size - 4) / clauseSize;
budget?.reserve(count * 256, sectionOffset);
for (const region of readExceptionClauses(
bytes,
sectionOffset + 4,
count,
fat,
))
regions.push(region);
regions.push(...readExceptionClauses(bytes, sectionOffset + 4, count, fat));
hasMoreSections = (kind & 0x80) !== 0;
sectionOffset += size;
+4 -19
View File
@@ -1,4 +1,3 @@
import { managedDecodeBudget } from "./ManagedDecodeBudget.js";
import type { ManagedPeLayout } from "./ManagedPeReader.js";
import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js";
import type { ManagedMetadataLayout } from "./ManagedMetadataLayout.js";
@@ -65,7 +64,7 @@ export const parseFields = (
declaring_type: declared?.fullName ?? null,
name,
flags,
signature: budgetedSignature(sig, layout),
signature: signature(sig),
});
core.set(token, { token, name });
}, issues);
@@ -130,7 +129,7 @@ export const parseMemberRefs = (
layout.rowCounts,
),
name,
signature: budgetedSignature(sig, layout),
signature: signature(sig),
});
core.set(token, { token, name });
}, issues);
@@ -190,14 +189,8 @@ export const parseMethods = ({
rva,
impl_flags: implFlags,
flags,
signature: budgetedSignature(sig, layout),
body: methodBody(
bytes,
pe,
rva,
{ implFlags, flags },
managedDecodeBudget(layout),
),
signature: signature(sig),
body: methodBody(bytes, pe, rva, { implFlags, flags }),
});
core.set(token, {
token,
@@ -265,11 +258,3 @@ export const edges = (
}
return { callEdges, fieldAccesses };
};
const budgetedSignature = (
bytes: Buffer,
layout: ManagedMetadataLayout,
): ReturnType<typeof signature> => {
managedDecodeBudget(layout).reserve(bytes.length * 256, layout.blob.offset);
return signature(bytes, managedDecodeBudget(layout).digest(bytes, "sha256"));
};
+9 -16
View File
@@ -1,7 +1,5 @@
import { createHash } from "node:crypto";
import { managedDecodeBudget } from "./ManagedDecodeBudget.js";
import {
metadataRowOffset,
type ManagedMetadataLayout,
@@ -119,11 +117,6 @@ export const readMetadataString = (
layout.strings.offset + Math.max(index, 0),
);
const start = layout.strings.offset + index;
const cached = managedDecodeBudget(layout).cachedString(index);
if (cached !== undefined && cached.encodedBytes <= heapExtent) {
managedDecodeBudget(layout).reserveString(cached.encodedBytes, start);
return cached.value;
}
const maximum = Math.min(
layout.strings.offset + layout.strings.size,
start + heapExtent + 1,
@@ -137,14 +130,10 @@ export const readMetadataString = (
"String heap item has no terminator within #Strings",
start,
);
managedDecodeBudget(layout).reserveString(end - start, start);
try {
const value = new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(bytes.subarray(start, end));
managedDecodeBudget(layout).cacheString(index, value, end - start);
return value;
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
bytes.subarray(start, end),
);
} catch (cause: unknown) {
void cause;
throw managedFailure(
@@ -229,6 +218,12 @@ export const metadataToken = (table: number, row: number): string =>
export const sha256Bytes = (bytes: Buffer): string =>
createHash("sha256").update(bytes).digest("hex");
/** Derive the conventional strong-name token from a full public key. */
export const strongNameToken = (publicKey: Buffer): string =>
Buffer.from(createHash("sha1").update(publicKey).digest().subarray(-8))
.reverse()
.toString("hex");
/** Create a cursor bounded to one admitted metadata table row. */
export const metadataRowCursor = (
bytes: Buffer,
@@ -245,8 +240,6 @@ export const metadataRowCursor = (
"Metadata table is absent",
start,
);
// Reserve per-row object/index bookkeeping before readers retain decoded facts.
managedDecodeBudget(layout).reserve(256, start);
return new MetadataRowCursor(
bytes,
start,
+9 -26
View File
@@ -1,5 +1,3 @@
import { managedDecodeBudget } from "./ManagedDecodeBudget.js";
import type {
ManagedArtifactInspection,
ManagedParseIssue,
@@ -16,6 +14,8 @@ import {
readMetadataBlob,
readMetadataGuid,
readMetadataString,
sha256Bytes,
strongNameToken,
} from "./ManagedMetadataHeaps.js";
import { managedFailure } from "./ManagedReaderFailure.js";
import type { ManagedResourceDirectory } from "./ManagedMetadataInventory.js";
@@ -29,22 +29,15 @@ type CustomAttribute = ManagedArtifactInspection["attributes"][number];
const publicKeyIdentity = (
bytes: Buffer,
kind: "public-key" | "public-key-token",
layout: ManagedMetadataLayout,
): AssemblyIdentity["public_key"] => ({
kind: bytes.length === 0 ? "none" : kind,
byte_length: bytes.length,
sha256:
bytes.length === 0
? null
: managedDecodeBudget(layout).digest(bytes, "sha256"),
sha256: bytes.length === 0 ? null : sha256Bytes(bytes),
token:
bytes.length === 0
? null
: kind === "public-key"
? Buffer.from(managedDecodeBudget(layout).digest(bytes, "sha1"), "hex")
.subarray(-8)
.reverse()
.toString("hex")
? strongNameToken(bytes)
: bytes.length === 8
? bytes.toString("hex")
: null,
@@ -134,7 +127,6 @@ export const readAssembly = (
public_key: publicKeyIdentity(
key,
(flags & 1) === 0 ? "public-key-token" : "public-key",
layout,
),
token: metadataToken(32, 1),
row_offset: table.offset,
@@ -187,12 +179,8 @@ export const readAssemblyReference = (
public_key_or_token: publicKeyIdentity(
keyOrToken,
(flags & 1) === 0 ? "public-key-token" : "public-key",
layout,
),
hash_value_sha256:
hashValue.length === 0
? null
: managedDecodeBudget(layout).digest(hashValue, "sha256"),
hash_value_sha256: hashValue.length === 0 ? null : sha256Bytes(hashValue),
hash_value_length: hashValue.length,
token: metadataToken(35, row),
row_offset: metadataRowOffset(layout, 35, row),
@@ -350,17 +338,13 @@ const decodeCompressedLength = (
return undefined;
};
const decodeFixedString = (
blob: Buffer,
layout: ManagedMetadataLayout,
): string | null => {
const decodeFixedString = (blob: Buffer): string | null => {
if (blob.length < 3 || blob.readUInt16LE(0) !== 1 || blob[2] === 0xff)
return null;
const length = decodeCompressedLength(blob, 2);
if (length === undefined) return null;
const start = 2 + length.prefix;
if (start > blob.length - length.length) return null;
managedDecodeBudget(layout).reserveString(length.length, layout.blob.offset);
try {
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
blob.subarray(start, start + length.length),
@@ -443,10 +427,10 @@ export const readCustomAttribute = (
),
type_name: typeName,
value_length: value.length,
value_sha256: managedDecodeBudget(layout).digest(value, "sha256"),
value_sha256: sha256Bytes(value),
decoded_fixed_string:
typeName === "System.Runtime.Versioning.TargetFrameworkAttribute"
? decodeFixedString(value, layout)
? decodeFixedString(value)
: null,
token: metadataToken(12, row),
row_offset: cursor.start,
@@ -521,9 +505,8 @@ export const readResource = ({
detail: "Embedded resource length leaves the CLI resources directory",
});
} else {
dataSha256 = managedDecodeBudget(layout).digest(
dataSha256 = sha256Bytes(
bytes.subarray(start + 4, start + 4 + dataLength),
"sha256",
);
}
}
+3 -13
View File
@@ -1,7 +1,3 @@
import {
attachManagedDecodeBudget,
ManagedDecodeBudget,
} from "./ManagedDecodeBudget.js";
import { managedFailure } from "./ManagedReaderFailure.js";
import {
computeRowSize,
@@ -212,7 +208,6 @@ interface ParsedStreams {
const readMetadataStreams = (
bytes: Buffer,
rootOffset: number,
budget: ManagedDecodeBudget,
): ParsedStreams => {
if (bytes.readUInt32LE(0) !== 0x424a_5342)
throw managedFailure(
@@ -225,10 +220,8 @@ const readMetadataStreams = (
requireRange(bytes, 16, versionLength, "metadata.version");
const versionBytes = bytes.subarray(16, 16 + versionLength);
const zero = versionBytes.indexOf(0);
const decodedVersionLength = zero < 0 ? versionBytes.length : zero;
budget.reserveString(decodedVersionLength, rootOffset + 16);
const version = versionBytes
.subarray(0, decodedVersionLength)
.subarray(0, zero < 0 ? versionBytes.length : zero)
.toString("utf8");
let cursor = align4(16 + versionLength);
requireRange(bytes, cursor, 4, "metadata.stream-count");
@@ -397,8 +390,7 @@ export const readManagedMetadataLayout = (
rootOffset,
);
const bytes = artifact.subarray(rootOffset, rootOffset + size);
const budget = new ManagedDecodeBudget();
const { streams, version } = readMetadataStreams(bytes, rootOffset, budget);
const { streams, version } = readMetadataStreams(bytes, rootOffset);
const { tableBytes, tablesOffset, heaps, valid } = readTablesHeader(
artifact,
streams,
@@ -411,7 +403,7 @@ export const readManagedMetadataLayout = (
valid,
tablesOffset,
});
const layout: ManagedMetadataLayout = {
return {
rootOffset,
size,
version,
@@ -428,8 +420,6 @@ export const readManagedMetadataLayout = (
tableIndexSize: (index) => tableSize(index, rowCounts),
codedIndexSize: (name) => codedSize(name, rowCounts),
};
attachManagedDecodeBudget(layout, budget);
return layout;
};
/** Resolve a one-based metadata row to its exact file offset. */
+17 -17
View File
@@ -1,4 +1,3 @@
import type { ManagedDecodeBudget } from "./ManagedDecodeBudget.js";
import type { ManagedPeLayout } from "./ManagedPeReader.js";
import type { ManagedMethodBody } from "./ManagedMemberInspectorCore.js";
import {
@@ -63,6 +62,19 @@ const readMethodBodyHeader = (
};
};
const readExceptionRegions = (
bytes: Buffer,
sectionOffset: number,
methodEnd: number,
ilSize: number,
instructionOffsets: readonly number[],
): ReturnType<typeof parseExceptionRegions> =>
validateExceptionRegionRanges(
parseExceptionRegions(bytes, sectionOffset, methodEnd),
ilSize,
instructionOffsets,
);
/** Decode admitted managed CIL, retaining unavailable implementation metadata as partial. */
export const methodBody = (
bytes: Buffer,
@@ -72,7 +84,6 @@ export const methodBody = (
implFlags: 0,
flags: 0,
},
budget?: ManagedDecodeBudget,
): ManagedMethodBody => {
if (rva === 0) return emptyMethodBody(rva, "absent", null);
if (
@@ -97,9 +108,6 @@ export const methodBody = (
if (ilOffset > bytes.length - header.ilSize)
throw new RangeError("Method IL bytes leave artifact");
const il = bytes.subarray(ilOffset, ilOffset + header.ilSize);
// One byte can produce an instruction record, start index, normalized tuple,
// and anchor. Reserve conservative bookkeeping capacity before decoding.
budget?.reserve(il.length * 256, ilOffset);
const decoded = decodeInstructions(il);
const opcodeCounts: Record<string, number> = {};
for (const instruction of decoded.parsed)
@@ -127,13 +135,10 @@ export const methodBody = (
const sectionOffset = (methodEnd + 3) & ~3;
const exceptionRegions =
header.format === "fat" && (header.flags & 8) !== 0
? validateExceptionRegionRanges(
parseExceptionRegions(
bytes,
sectionOffset,
offset + methodExtent,
budget,
),
? readExceptionRegions(
bytes,
sectionOffset,
offset + methodExtent,
header.ilSize,
decoded.instructionStarts,
)
@@ -174,11 +179,6 @@ export const methodBody = (
: null),
};
} catch (cause: unknown) {
if (
cause instanceof ManagedReaderFailure &&
cause.issue.code === "resource-limit"
)
throw cause;
if (
!(cause instanceof ManagedReaderFailure) &&
!(cause instanceof RangeError)
+37 -46
View File
@@ -1,7 +1,3 @@
import {
admitManagedProjection,
managedDecodeBudget,
} from "./ManagedDecodeBudget.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
managedNativeBoundaryInspectionSchema,
@@ -431,45 +427,40 @@ export const buildNativeBoundaryInspection = ({
native,
issues,
}: BoundaryInspectionContext): ManagedNativeBoundaryInspection =>
managedNativeBoundaryInspectionSchema.parse(
admitManagedProjection(
{
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: inventory.module,
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: layout.version,
table_row_counts: managedTableRowCounts(layout),
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: inventory.module?.mvid ?? null,
},
cli_native: native,
module_refs: moduleRefs,
pinvoke_imports: imports,
native_implementations: implementations,
summary: nativeBoundarySummary(native, {
module_ref_count: moduleRefs.length,
pinvoke_import_count: imports.length,
native_implementation_count: implementations.length,
}),
coverage: {
state: issues.length === 0 ? "complete" : "partial",
issues,
},
limitations: [
"P/Invoke rows prove managed import declarations only; this inspection does not verify that a native library, export, thunk, or provider-qualified function exists.",
"Managed metadata tokens are build-local coordinates and are only meaningful with the reported artifact SHA-256 and MVID.",
"ReadyToRun, NativeAOT, C++/CLI, and IL2CPP native semantics require separately selected native-provider evidence; this tool does not translate managed tokens into native addresses.",
],
},
managedDecodeBudget(layout),
),
);
managedNativeBoundaryInspectionSchema.parse({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: inventory.module,
metadata: {
status: issues.length === 0 ? "complete" : "partial",
version: layout.version,
table_row_counts: managedTableRowCounts(layout),
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: inventory.module?.mvid ?? null,
},
cli_native: native,
module_refs: moduleRefs,
pinvoke_imports: imports,
native_implementations: implementations,
summary: nativeBoundarySummary(native, {
module_ref_count: moduleRefs.length,
pinvoke_import_count: imports.length,
native_implementation_count: implementations.length,
}),
coverage: {
state: issues.length === 0 ? "complete" : "partial",
issues,
},
limitations: [
"P/Invoke rows prove managed import declarations only; this inspection does not verify that a native library, export, thunk, or provider-qualified function exists.",
"Managed metadata tokens are build-local coordinates and are only meaningful with the reported artifact SHA-256 and MVID.",
"ReadyToRun, NativeAOT, C++/CLI, and IL2CPP native semantics require separately selected native-provider evidence; this tool does not translate managed tokens into native addresses.",
],
});
+68 -83
View File
@@ -1,4 +1,3 @@
import { admitManagedProjection } from "./ManagedDecodeBudget.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
managedNativeBoundaryInspectionSchema,
@@ -48,45 +47,39 @@ const emptyInspection = (
readonly issues?: readonly ManagedParseIssue[];
} = {},
): ManagedNativeBoundaryInspection => {
return managedNativeBoundaryInspectionSchema.parse(
admitManagedProjection({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: null,
metadata: {
status: issues.some((issue) => issue.code === "resource-limit")
? "partial"
: classification === "malformed"
? "malformed"
: "absent",
version: null,
table_row_counts: {},
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: null,
},
cli_native: native,
module_refs: [],
pinvoke_imports: [],
native_implementations: [],
summary: nativeBoundarySummary(native, {
module_ref_count: 0,
pinvoke_import_count: 0,
native_implementation_count: 0,
}),
coverage: { state: "unavailable", issues },
limitations: [
"No CLI metadata was admitted; native boundary declarations are unavailable.",
"Static inspection does not load or execute target code, so native export resolution is not performed.",
],
return managedNativeBoundaryInspectionSchema.parse({
artifact: {
path: target.path,
sha256: target.sha256,
byte_length: bytes.length,
format: "pe",
},
module: null,
metadata: {
status: classification === "malformed" ? "malformed" : "absent",
version: null,
table_row_counts: {},
},
identity_scope: {
token_identity: "build-local",
requires_artifact_sha256: target.sha256,
requires_mvid: null,
},
cli_native: native,
module_refs: [],
pinvoke_imports: [],
native_implementations: [],
summary: nativeBoundarySummary(native, {
module_ref_count: 0,
pinvoke_import_count: 0,
native_implementation_count: 0,
}),
);
coverage: { state: "unavailable", issues },
limitations: [
"No CLI metadata was admitted; native boundary declarations are unavailable.",
"Static inspection does not load or execute target code, so native export resolution is not performed.",
],
});
};
const readBoundaryInventory = (
@@ -149,48 +142,40 @@ export const inspectManagedNativeBoundariesBytes = (
issues: [cause.issue],
});
}
try {
const heapExtent = Math.max(layout.strings.size, layout.blob.size);
issues.push(...inventory.issues);
const moduleRefs = parseModuleRefs(bytes, layout, heapExtent, issues);
const members = new Map([
...parseFields(bytes, layout, heapExtent, issues),
...parseMethods(bytes, layout, heapExtent, issues),
]);
const imports = parseImplMaps({
bytes,
layout,
heapExtent,
modules: moduleRefs,
members,
issues,
});
const pinvokeTokens = new Set(
imports
.map(({ member_token }) => member_token)
.filter((token): token is string => token !== null),
);
const implementations = nativeImplementations(
members.values(),
pinvokeTokens,
);
return buildNativeBoundaryInspection({
target,
bytes,
pe,
layout,
inventory,
moduleRefs,
imports,
implementations,
native: cliNative(pe),
issues,
});
} catch (cause: unknown) {
if (!(cause instanceof ManagedReaderFailure)) throw cause;
return emptyInspection(target, bytes, "malformed", {
native: cliNative(pe),
issues: [...issues, cause.issue],
});
}
const heapExtent = Math.max(layout.strings.size, layout.blob.size);
issues.push(...inventory.issues);
const moduleRefs = parseModuleRefs(bytes, layout, heapExtent, issues);
const members = new Map([
...parseFields(bytes, layout, heapExtent, issues),
...parseMethods(bytes, layout, heapExtent, issues),
]);
const imports = parseImplMaps({
bytes,
layout,
heapExtent,
modules: moduleRefs,
members,
issues,
});
const pinvokeTokens = new Set(
imports
.map(({ member_token }) => member_token)
.filter((token): token is string => token !== null),
);
const implementations = nativeImplementations(
members.values(),
pinvokeTokens,
);
return buildNativeBoundaryInspection({
target,
bytes,
pe,
layout,
inventory,
moduleRefs,
imports,
implementations,
native: cliNative(pe),
issues,
});
};
+32 -73
View File
@@ -121,87 +121,31 @@ const architectureFor = (machine: number): ManagedPeLayout["architecture"] => {
}
};
/** Resolve one section identity for both exact reads and available-byte queries. */
const resolveRva = (
mapping: RvaMapping,
): { readonly offset: number; readonly available: number } => {
const mapRva = (mapping: RvaMapping): number => {
const { bytes, sections, rva, size, scope } = mapping;
if (
!Number.isSafeInteger(rva) ||
!Number.isSafeInteger(size) ||
rva < 0 ||
size < 0 ||
rva > 0xffff_ffff ||
(size > 0 && size > 0x1_0000_0000 - rva)
)
throw managedFailure(
"invalid-directory",
scope,
`${scope} has an invalid RVA range`,
);
const selected = sections.find((section) => {
const within = rva - section.virtualAddress;
const mappedSize = Math.max(section.virtualSize, section.rawSize);
return (
mappedSize > 0 &&
within >= 0 &&
within <= mappedSize &&
size <= mappedSize - within
);
});
if (selected === undefined)
throw managedFailure(
"invalid-directory",
scope,
`${scope} RVA is not covered by one PE section`,
);
if (rva === 0 && size === 0) return 0;
for (const section of sections) {
if (
section === selected ||
Math.max(section.virtualSize, section.rawSize) === 0
)
continue;
const end =
section.virtualAddress + Math.max(section.virtualSize, section.rawSize);
const intersects =
size === 0
? rva >= section.virtualAddress && rva < end
: rva < end && rva + size > section.virtualAddress;
if (intersects)
const mappedSize = Math.max(section.virtualSize, section.rawSize);
if (rva < section.virtualAddress) continue;
const within = rva - section.virtualAddress;
if (within > mappedSize || size > mappedSize - within) continue;
if (within > section.rawSize || size > section.rawSize - within)
throw managedFailure(
"invalid-directory",
scope,
`${scope} RVA range has ambiguous overlapping PE section mappings`,
`${scope} occupies virtual bytes without file-backed data`,
);
const offset = add(section.rawOffset, within, scope);
requireRange(bytes, offset, size, scope);
return offset;
}
const within = rva - selected.virtualAddress;
if (within > selected.rawSize || size > selected.rawSize - within)
throw managedFailure(
"invalid-directory",
scope,
`${scope} occupies virtual bytes without file-backed data`,
);
const offset = add(selected.rawOffset, within, scope);
requireRange(bytes, offset, size, scope);
let available = Math.min(
selected.rawSize - within,
bytes.length - offset,
0x1_0000_0000 - rva,
throw managedFailure(
"invalid-directory",
scope,
`${scope} RVA is not covered by a PE section`,
);
// A future overlap ends the uniquely mapped extent even if the current byte is unique.
for (const section of sections)
if (
section !== selected &&
section.virtualAddress > rva &&
Math.max(section.virtualSize, section.rawSize) > 0
)
available = Math.min(available, section.virtualAddress - rva);
return { offset, available };
};
const mapRva = (mapping: RvaMapping): number =>
mapping.rva === 0 && mapping.size === 0 ? 0 : resolveRva(mapping).offset;
const readDirectory = (bytes: Buffer, offset: number): PeDataDirectory => {
requireRange(bytes, offset, 8, "pe.data-directory");
return {
@@ -412,7 +356,22 @@ export const readManagedPeLayout = (bytes: Buffer): ManagedPeLayout => {
cliIssue,
rvaToOffset: (rva, size, scope) =>
mapRva({ bytes, sections, rva, size, scope }),
rvaAvailableBytes: (rva, scope) =>
resolveRva({ bytes, sections, rva, size: 1, scope }).available,
rvaAvailableBytes: (rva, scope) => {
mapRva({ bytes, sections, rva, size: 1, scope });
for (const section of sections) {
if (rva < section.virtualAddress) continue;
const within = rva - section.virtualAddress;
if (within < section.rawSize)
return Math.min(
section.rawSize - within,
bytes.length - section.rawOffset - within,
);
}
throw managedFailure(
"invalid-directory",
scope,
`${scope} RVA is not covered by file-backed PE section data`,
);
},
};
};
+1 -5
View File
@@ -27,11 +27,7 @@ export const readManagedValue = <Value>(
try {
return operation();
} catch (cause: unknown) {
if (
!(cause instanceof ManagedReaderFailure) ||
cause.issue.code === "resource-limit"
)
throw cause;
if (!(cause instanceof ManagedReaderFailure)) throw cause;
issues.push(cause.issue);
return undefined;
}
+28 -74
View File
@@ -1,4 +1,6 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { readFile } from "node:fs/promises";
import {
createAnalysisExecution,
@@ -16,12 +18,11 @@ import {
managedTargetInputSchema,
type ManagedToolName,
} from "../contracts/managed/managedToolContracts.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { EvidenceIntegrityError } from "../domain/evidenceErrors.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
@@ -37,23 +38,6 @@ import { inspectManagedArtifactBytes } from "./ManagedArtifactInspector.js";
import { inspectManagedMembersBytes } from "./ManagedMemberInspector.js";
import { inspectManagedNativeBoundariesBytes } from "./ManagedNativeBoundaryInspector.js";
import { openRegularFile } from "../filesystem/RegularFile.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
// This provider eagerly retains the full PE snapshot. Keep its input resource
// policy separate from decoded metadata/output capacity; it is not a PE format limit.
const MAX_MANAGED_SNAPSHOT_BYTES = 128 * 1024 * 1024;
const snapshotCapacityError = (
operation: string,
): AnalysisResourceConstraintError =>
new AnalysisResourceConstraintError(
operation,
"memory",
"Managed static inspection requires an eagerly retained PE snapshot within its input byte budget.",
{ max_snapshot_bytes: MAX_MANAGED_SNAPSHOT_BYTES },
);
/** Execution-free managed PE/CLI auxiliary provider. */
export class ManagedStaticProvider implements AnalysisProvider {
readonly #capabilities: readonly CapabilityDescriptor[] =
@@ -106,16 +90,8 @@ class ManagedStaticClient implements AnalysisClient {
const snapshot = this.#snapshotBytes;
const observed =
snapshot === undefined
? await readManagedSnapshot(
this.target.path,
operation,
options?.signal,
)
: await hashManagedSource(
this.target.path,
operation,
options?.signal,
);
? await readManagedSnapshot(this.target.path, options?.signal)
: await hashManagedSource(this.target.path, options?.signal);
if (observed.sha256 !== this.target.sha256)
return err(
new EvidenceIntegrityError(
@@ -143,7 +119,6 @@ class ManagedStaticClient implements AnalysisClient {
} catch (cause: unknown) {
if (options?.signal?.aborted === true)
return err(new AnalysisCancelledError(operation));
if (cause instanceof AnalysisError) return err(cause);
return err(
new ProviderAdapterError(MANAGED_STATIC_PROVIDER.id, operation, {
cause,
@@ -166,59 +141,38 @@ interface ManagedSourceObservation {
const readManagedSnapshot = async (
path: string,
operation: string,
signal?: AbortSignal,
): Promise<ManagedSourceObservation> => {
const handle = await openRegularFile(path, { symlinks: "follow", signal });
try {
const metadata = await handle.stat();
if (metadata.size > MAX_MANAGED_SNAPSHOT_BYTES)
throw snapshotCapacityError(operation);
const bytes = await readBoundedFileBytes(
handle,
MAX_MANAGED_SNAPSHOT_BYTES,
signal,
);
if (bytes === undefined) throw snapshotCapacityError(operation);
signal?.throwIfAborted();
return {
byteLength: bytes.length,
sha256: createHash("sha256").update(bytes).digest("hex"),
bytes,
};
} finally {
await handle.close();
}
const bytes = await readFile(
path,
signal === undefined ? undefined : { signal },
);
return {
byteLength: bytes.length,
sha256: createHash("sha256").update(bytes).digest("hex"),
bytes,
};
};
const hashManagedSource = async (
path: string,
operation: string,
signal?: AbortSignal,
): Promise<ManagedSourceObservation> => {
const handle = await openRegularFile(path, { symlinks: "follow", signal });
try {
if ((await handle.stat()).size > MAX_MANAGED_SNAPSHOT_BYTES)
throw snapshotCapacityError(operation);
const digest = createHash("sha256");
let byteLength = 0;
const stream = handle.createReadStream({
autoClose: false,
...(signal === undefined ? {} : { signal }),
});
for await (const chunk of stream) {
signal?.throwIfAborted();
if (!Buffer.isBuffer(chunk))
throw new TypeError("Managed source stream returned non-buffer data");
if (chunk.length > MAX_MANAGED_SNAPSHOT_BYTES - byteLength)
throw snapshotCapacityError(operation);
byteLength += chunk.length;
digest.update(chunk);
}
return { byteLength, sha256: digest.digest("hex") };
} finally {
await handle.close();
const digest = createHash("sha256");
let byteLength = 0;
const stream = createReadStream(
path,
signal === undefined ? undefined : { signal },
);
for await (const chunk of stream) {
if (!Buffer.isBuffer(chunk))
throw new TypeError("Managed source stream returned non-buffer data");
if (chunk.length > Number.MAX_SAFE_INTEGER - byteLength)
throw new RangeError("Managed source byte length overflowed");
byteLength += chunk.length;
digest.update(chunk);
}
return { byteLength, sha256: digest.digest("hex") };
};
const isManagedOperation = (
+2 -116
View File
@@ -31,7 +31,6 @@ import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { runOwnedCommand } from "../process/OwnedCommand.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import {
evmInterfaceFailure,
capturedEvmOutput,
@@ -76,16 +75,6 @@ type Launcher = NonNullable<Parameters<typeof runOwnedCommand>[2]>["launcher"];
/** Run unchanged EVMole/WASM in an owned worker; no target, chain or network execution. */
export class EvmoleInterfaceProvider implements EvmInterfacePort {
#tail: Promise<void> = Promise.resolve();
#closing = false;
#closePromise: Promise<void> | undefined;
#pendingCleanup:
| {
root: Pick<PrivateRuntimeRoot, "path" | "close"> | undefined;
process: ProviderProcessSupervisor | undefined;
}
| undefined;
constructor(
readonly environment: Readonly<NodeJS.ProcessEnv> = process.env,
readonly launcher?: Launcher,
@@ -95,29 +84,11 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort {
) {}
/** Bind recovered interface candidates to both original carrier and decoded byte identity. */
inspect(
async inspect(
input: InspectEvmInterfaceInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
if (this.#closing) return Promise.resolve(err(this.#closedError()));
const operation = this.#tail.then(() =>
this.#closing ? err(this.#closedError()) : this.#inspect(input, options),
);
this.#tail = operation.then(
() => undefined,
() => undefined,
);
return operation;
}
async #inspect(
input: InspectEvmInterfaceInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
const pending = await this.#retire();
if (pending !== undefined) return err(pending);
let root: Pick<PrivateRuntimeRoot, "path" | "close"> | undefined;
let processOwner: ProviderProcessSupervisor | undefined;
let retainedOutput: AnalysisCapturedOutput | undefined;
let phase: "configuration" | "artifact-read" | "worker" = "configuration";
let workerLimits: EvmWorkerLimits | undefined;
@@ -350,8 +321,6 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort {
}),
);
} catch (cause: unknown) {
if (cause instanceof OwnedCommandFailure)
processOwner = cause.cleanupOwner;
let fileFailure: EvmFileSizeFailureEvidence | undefined;
if (
cause instanceof OwnedCommandFailure &&
@@ -399,42 +368,14 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort {
),
);
}
if (processOwner !== undefined) {
this.#pendingCleanup = { root, process: processOwner };
root = undefined;
return err(
new ProviderCleanupError(
EVMOLE_PROVIDER_IDENTITY.id,
[
processOwner.launch.ownership?.runId ?? "owned-worker",
...(this.#pendingCleanup.root === undefined
? []
: [this.#pendingCleanup.root.path]),
],
{
reason: "Owned EVM worker cleanup could not be confirmed",
previous_error: result.ok
? null
: projectAnalysisError(result.error),
...(retainedOutput === undefined
? {}
: { captured_output: { ...retainedOutput } }),
},
{ operation: OPERATION },
),
);
}
if (root !== undefined) {
try {
await root.close();
} catch (cause: unknown) {
const retainedRoot = root;
this.#pendingCleanup = { root: retainedRoot, process: undefined };
root = undefined;
return err(
new ProviderCleanupError(
EVMOLE_PROVIDER_IDENTITY.id,
[retainedRoot.path],
[root.path],
{
reason: cause instanceof Error ? cause.message : String(cause),
previous_error: result.ok
@@ -460,59 +401,4 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort {
)
: result;
}
async close(): Promise<void> {
this.#closing = true;
this.#closePromise ??= this.#tail
.then(async () => {
const failure = await this.#retire();
if (failure !== undefined) throw failure;
})
.catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
#closedError(): ProviderCleanupError {
return new ProviderCleanupError(
EVMOLE_PROVIDER_IDENTITY.id,
[],
{ reason: "Provider is closing" },
{ operation: OPERATION },
);
}
async #retire(): Promise<ProviderCleanupError | undefined> {
const pending = this.#pendingCleanup;
if (pending === undefined) return undefined;
if (pending.process !== undefined) {
const stopped = await pending.process.stop();
if (stopped.status === "incomplete")
return new ProviderCleanupError(
EVMOLE_PROVIDER_IDENTITY.id,
[pending.process.launch.ownership?.runId ?? "owned-worker"],
{ reason: stopped.reason },
{ operation: OPERATION },
);
pending.process = undefined;
}
if (pending.root !== undefined) {
const root = pending.root;
try {
await root.close();
pending.root = undefined;
} catch (cause: unknown) {
return new ProviderCleanupError(
EVMOLE_PROVIDER_IDENTITY.id,
[root.path],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation: OPERATION, cause },
);
}
}
this.#pendingCleanup = undefined;
return undefined;
}
}
-6
View File
@@ -16,12 +16,6 @@ export class NonRegularFileReadError extends Error {
}
}
/** One admitted file descriptor and the metadata that identified its contents. */
export interface StableRegularFileDescriptor {
readonly handle: FileHandle;
readonly initial: Stats;
}
/** Open and verify the selected file descriptor without blocking on pipes. */
export const openRegularFile = async (
path: string,
+9 -39
View File
@@ -113,7 +113,6 @@ export const runFirmwareCommand = async (context: {
launcher?: FirmwareLauncher;
outputBudget?: { root: string; bytes: number; entries: number };
acceptReportedExtractionFailure?: boolean;
retainCleanup(supervisor: ProviderProcessSupervisor): void;
}) => {
const { engine, operation, signal } = context;
signal.throwIfAborted();
@@ -142,43 +141,16 @@ export const runFirmwareCommand = async (context: {
NO_COLOR: "1",
},
});
let diagnosticTruncated = false;
const supervisor = new ProviderProcessSupervisor(
{
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ?? (() => cleanupOwnedProcessGroup(spawned.ownership)),
},
{
maxDiagnosticBytes: FIRMWARE_LIMITS.diagnosticBytes,
onDiagnostic: (event) => {
if (event.type === "output" && event.truncated === true)
diagnosticTruncated = true;
},
},
);
const checkDiagnostics = () => {
if (diagnosticTruncated) {
const snapshot = supervisor.snapshot();
throw new AnalysisOutputError(
operation,
`Tool diagnostics exceeded the ${FIRMWARE_LIMITS.diagnosticBytes}-byte capture budget`,
{
capturedOutput: {
stdout: snapshot.stdout.text,
stderr: snapshot.stderr.text,
truncated: true,
},
},
);
}
};
const supervisor = new ProviderProcessSupervisor({
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ?? (() => cleanupOwnedProcessGroup(spawned.ownership)),
});
let failure: unknown;
try {
const deadline = Date.now() + FIRMWARE_LIMITS.timeoutMs;
while (!(await supervisor.waitForOutputClose(100))) {
checkDiagnostics();
while (!(await supervisor.waitForExit(100))) {
if (signal.aborted) throw new AnalysisCancelledError(operation);
if (Date.now() >= deadline)
throw new AnalysisTimeoutError(operation, FIRMWARE_LIMITS.timeoutMs);
@@ -191,15 +163,14 @@ export const runFirmwareCommand = async (context: {
signal,
);
}
checkDiagnostics();
if (signal.aborted) throw new AnalysisCancelledError(operation);
} catch (cause: unknown) {
failure = cause;
}
const stopped = await supervisor.stop();
const snapshot = supervisor.snapshot();
if (stopped.status === "incomplete") {
context.retainCleanup(supervisor);
supervisor.dispose();
if (stopped.status === "incomplete")
throw new ProviderCleanupError(engine, [runId, context.cwd], {
reason: stopped.reason,
leader_pid: spawned.ownership.leaderPid,
@@ -212,7 +183,6 @@ export const runFirmwareCommand = async (context: {
stderr_bytes: snapshot.stderr.bytes,
previous_error: failure instanceof Error ? failure.message : null,
});
}
if (failure !== undefined) throw failure;
if (
snapshot.exitCode !== 0 &&
+10 -27
View File
@@ -3,8 +3,8 @@ import { constants, createReadStream, createWriteStream } from "node:fs";
import {
chmod,
lstat,
opendir,
open,
readdir,
readlink,
realpath,
} from "node:fs/promises";
@@ -29,13 +29,6 @@ export interface FirmwareEntry {
readonly linkTarget: string | null;
}
const throwIfFirmwareInventoryCancelled = (
signal: AbortSignal | undefined,
operation: string,
): void => {
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
};
/** Hash a regular tool or extracted file without retaining its bytes in memory. */
export const hashFirmwareFile = async (
path: string,
@@ -191,7 +184,7 @@ export const inventoryFirmwareOutput = async (
let bytes = 0;
let count = 0;
while (pending.length > 0) {
throwIfFirmwareInventoryCancelled(signal, operation);
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
const directory = pending.pop();
if (directory === undefined) break;
const directoryStat = await lstat(directory).catch((cause: unknown) => {
@@ -205,28 +198,18 @@ export const inventoryFirmwareOutput = async (
directoryStat.isSymbolicLink()
)
continue;
const handle = await opendir(directory).catch((cause: unknown) => {
const names = await readdir(directory).catch((cause: unknown) => {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return undefined;
return [];
throw cause;
});
if (handle === undefined) continue;
const names: string[] = [];
try {
for await (const entry of handle) {
throwIfFirmwareInventoryCancelled(signal, operation);
count++;
if (count > maxEntries)
throw new AnalysisOutputError(
operation,
`Staging entry budget exceeded (${maxEntries})`,
);
names.push(entry.name);
}
} finally {
await handle.close().catch(() => undefined);
}
for (const name of names.sort()) {
count++;
if (count > maxEntries)
throw new AnalysisOutputError(
operation,
`Staging entry budget exceeded (${maxEntries})`,
);
const path = join(directory, name);
const stat = await lstat(path).catch((cause: unknown) => {
if (
+4 -78
View File
@@ -20,7 +20,6 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok } from "../domain/result.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import { publishFirmwareExtraction } from "./FirmwarePublication.js";
import {
FIRMWARE_LIMITS,
@@ -66,15 +65,6 @@ const limitations = [
/** Queued bring-your-own firmware adapters with private snapshots and owned cleanup. */
export class FirmwareProvider implements FirmwareAnalysisPort {
#tail: Promise<void> = Promise.resolve();
#closed = false;
#pendingCleanup:
| {
supervisor?: ProviderProcessSupervisor;
root: PrivateRuntimeRoot;
engine: string;
operation: string;
}
| undefined;
constructor(
readonly environment: Readonly<
Record<string, string | undefined>
@@ -87,7 +77,6 @@ export class FirmwareProvider implements FirmwareAnalysisPort {
request: FirmwareRequest,
options?: ExecutionOptions,
): Promise<Outcome> {
if (this.#closed) return err(new AnalysisCancelledError(request.operation));
const predecessor = this.#tail;
let release = () => {};
this.#tail = new Promise<void>((resolve) => {
@@ -105,57 +94,12 @@ export class FirmwareProvider implements FirmwareAnalysisPort {
);
}
try {
if (this.#closed)
return err(new AnalysisCancelledError(request.operation));
try {
await this.#retryCleanup();
} catch (cause: unknown) {
return err(
cause instanceof AnalysisError
? cause
: new ProviderCleanupError("firmware", [], {
reason: cause instanceof Error ? cause.message : String(cause),
}),
);
}
return await this.#execute(request, options);
} finally {
release();
}
}
/** Retry identity-bound cleanup before relinquishing private workspaces. */
async close(): Promise<void> {
this.#closed = true;
await this.#tail;
await this.#retryCleanup();
}
async #retryCleanup(): Promise<void> {
const pending = this.#pendingCleanup;
if (pending === undefined) return;
const stopped = await pending.supervisor?.stop();
if (stopped?.status === "incomplete")
throw new ProviderCleanupError(
pending.engine,
[pending.root.path],
{ reason: stopped.reason },
{ operation: pending.operation },
);
delete pending.supervisor;
try {
await pending.root.close();
} catch (cause: unknown) {
throw new ProviderCleanupError(
pending.engine,
[pending.root.path],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation: pending.operation, cause },
);
}
this.#pendingCleanup = undefined;
}
async #execute(
request: FirmwareRequest,
options?: ExecutionOptions,
@@ -177,7 +121,6 @@ export class FirmwareProvider implements FirmwareAnalysisPort {
request.operation,
);
root = await PrivateRuntimeRoot.create({ prefix: "rea-firmware-" });
const ownedRoot = root;
const target = await snapshotFirmware(request, root.path, signal);
const run = (
args: readonly string[],
@@ -191,14 +134,6 @@ export class FirmwareProvider implements FirmwareAnalysisPort {
cwd: root?.path ?? "",
environment: this.environment,
signal,
retainCleanup: (supervisor) => {
this.#pendingCleanup = {
supervisor,
root: ownedRoot,
engine: engineName,
operation: request.operation,
};
},
...(this.launcher === undefined ? {} : { launcher: this.launcher }),
...(outputBudget === undefined
? {}
@@ -380,19 +315,10 @@ export class FirmwareProvider implements FirmwareAnalysisPort {
}),
);
}
const finished = await finishFirmwareWorkspace(
root,
outcome,
engineName,
request,
);
if (!finished.ok && finished.error.cleanupIncomplete && root !== undefined)
this.#pendingCleanup ??= {
root,
engine: engineName,
operation: request.operation,
};
return finished;
// Never remove a workspace while process ownership remains uncertain.
// A leaked root stays recorded on this result and is not deleted. The next
// operation may create a new root; this failure is not replayed for it.
return finishFirmwareWorkspace(root, outcome, engineName, request);
}
}
+1 -1
View File
@@ -60,7 +60,7 @@ export const publishFirmwareExtraction = async (context: {
await tree.write(
entry.relativePath,
createReadStream(entry.path),
{ sha256, bytes: entry.size },
sha256,
context.signal,
);
files.push({
-1
View File
@@ -29,7 +29,6 @@ export const UNBLOB_PROVIDER_IDENTITY = {
export const FIRMWARE_LIMITS = {
inputBytes: 128 * 1024 * 1024,
reportBytes: 8 * 1024 * 1024,
diagnosticBytes: 8 * 1024 * 1024,
addressSpaceBytes: 1024 * 1024 * 1024,
timeoutMs: 120_000,
} as const;

Some files were not shown because too many files have changed in this diff Show More