Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c2e2ed9aa2 | ||
|
|
1ae4bec45b | ||
|
|
058bc0af35 | ||
|
|
dce81ca400 | ||
|
|
bb764d26bd | ||
|
|
acd8d86898 | ||
|
|
e6d0b3993d | ||
|
|
92a901e0db | ||
|
|
cef6d6d3c4 | ||
|
|
bd972bf80b | ||
|
|
ac9709a80a | ||
|
|
d1e6fe2eca | ||
|
|
a147ecd4b4 | ||
|
|
22fdcb8ab7 | ||
|
|
b22d817073 | ||
|
|
3d876c5c1a | ||
|
|
cf346368c4 | ||
|
|
e52c0a72c5 | ||
|
|
999e3e2098 | ||
|
|
c1217e711b | ||
|
|
0d89978333 | ||
|
|
11e3fae697 | ||
|
|
603d04ee3a | ||
|
|
472f313232 | ||
|
|
1f7e279aa2 | ||
|
|
db8ccaa251 | ||
|
|
5daff4d67e | ||
|
|
a9fd1aecae | ||
|
|
abb8e81d3a | ||
|
|
6d8f95e38d | ||
|
|
c1b5ccf0a4 | ||
|
|
5f30ea6ff2 | ||
|
|
d438675677 | ||
|
|
a84dea5500 | ||
|
|
2a66ec9094 | ||
|
|
d1235fadc9 | ||
|
|
a01148553e | ||
|
|
a6f76f5da9 | ||
|
|
2ca3ad2386 | ||
|
|
5d0a883605 | ||
|
|
0f0d6e4d43 | ||
|
|
6d4cefc4f7 | ||
|
|
2a0b58e440 |
@@ -136,6 +136,12 @@ existing analysis families directly, for example
|
||||
- Pull and capture outputs are additive: the destination must not already
|
||||
exist. Re-running a completed pull refuses the existing directory instead
|
||||
of overwriting evidence.
|
||||
- Cancellation during a package pull preserves caller-selected output. The
|
||||
error includes partial observations identifying completed artifacts and the
|
||||
interrupted local path; remaining APKs were not attempted.
|
||||
- Failed package acquisition rollback retains its directory identity and retries
|
||||
cleanup before later work and during shutdown. Cleanup refuses a replacement
|
||||
directory. Terminal shutdown stops admission and drains active operations.
|
||||
- The provider holds no long-lived processes; every invocation is one
|
||||
bounded adb run. The adb server belongs to the caller.
|
||||
|
||||
|
||||
@@ -25,6 +25,12 @@ extraction have independent prerequisites; one does not require the other.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
|
||||
|
||||
REA retains at most 8 MiB of combined stdout and stderr per command. Exceeding
|
||||
that diagnostic budget stops analysis with an explicit truncated-output error;
|
||||
child process limits do not bound parent-side pipe retention. Completed commands
|
||||
drain their output streams before recording diagnostics. Failed worker cleanup
|
||||
retains the worker and workspace for retry before another request or at shutdown.
|
||||
|
||||
Unblob needs the external extractor for the selected format on its inherited
|
||||
`PATH`: for example, `debugfs` for ext2/ext3/ext4. Its default Landlock sandbox
|
||||
and skip rules stay enabled. Sandbox startup failures remain actionable errors;
|
||||
|
||||
@@ -38,6 +38,20 @@ Inspect separately obtained components explicitly and preserve their identities.
|
||||
A valid implementation marker establishes a candidate native boundary, not
|
||||
recovered native semantics or runtime behavior.
|
||||
|
||||
### Resource admission
|
||||
|
||||
The static provider admits regular input files up to 128 MiB before retaining
|
||||
their bytes. Reads remain bounded if a file grows after admission. Metadata,
|
||||
CIL decoding, and projected inspection records have a separate conservative
|
||||
64 MiB representation budget; shared heap strings are cached, but repeated
|
||||
output occurrences still consume that budget. These are inspection resource
|
||||
policies, not PE format limits or a hard bound on total V8 heap use.
|
||||
|
||||
Input overflow returns a resource constraint. Decode or projection overflow
|
||||
reports `resource-limit` with unavailable or partial coverage; omitted facts do
|
||||
not become empty observations. Ambiguous PE section mappings are rejected
|
||||
rather than interpreted according to section order.
|
||||
|
||||
## Analysis objective
|
||||
|
||||
REA's managed-code track is intended to answer five different questions without
|
||||
|
||||
@@ -187,6 +187,10 @@ Every capture requires `truncation_details`, with separate accounting for:
|
||||
input, so raw omissions also limit rendered coverage.
|
||||
- `filesystem_before` and `filesystem_after`: file-count/depth limits,
|
||||
enumeration failures, whole-file hash budget and bytes successfully hashed.
|
||||
Directory names are read incrementally within the remaining entry capacity.
|
||||
When a directory exceeds that capacity, the retained subset follows filesystem
|
||||
enumeration order and is then sorted; it is explicitly incomplete and cannot
|
||||
establish that an omitted path is absent.
|
||||
Each retained regular file without a digest has an aliased path, size,
|
||||
remaining budget and reason: `file_bytes_budget`,
|
||||
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is
|
||||
|
||||
@@ -187,6 +187,30 @@ The loopback OpenAI completions/SSE custom model declares a one-million-token
|
||||
context and synthetic usage. Live xAI APIs, native Windows and default OS-home
|
||||
skill discovery remain unverified.
|
||||
|
||||
`verify:commandcode-client` requires installed Command Code (verified with npm
|
||||
1.79.2); select it with `REA_VERIFY_COMMANDCODE_COMMAND`. Auth, MCP tokens and
|
||||
sessions use the actual OS home, so provision a disposable POSIX account rather
|
||||
than overriding `HOME`. Set `REA_VERIFY_COMMANDCODE_ACCOUNT_HOME` to that account's
|
||||
actual home and create `.rea-client-verification` there with exactly
|
||||
`Disposable REA client verification account` followed by a newline. The lane
|
||||
owns that account's `.commandcode` configuration and shared REA skill. Build the
|
||||
runtime as the checkout owner, then run `node scripts/verify-commandcode-client.mjs`
|
||||
as the disposable account; the npm shortcut also needs a writable checkout.
|
||||
The verifier does not create accounts or install clients. It guards setup
|
||||
targets, preserves an unrelated registration and backups, checks idempotence,
|
||||
and verifies native default shared-skill discovery and complete activation.
|
||||
Default deferred-schema delivery stays enabled: the native prompt advertises
|
||||
every REA tool, exact-name `search_tools` returns every input schema, and a native
|
||||
shell query recovers the saved catalog's count and digest when the client spills
|
||||
it. Actual analysis of a Unicode-path fixture delivers full named-schema Evidence
|
||||
to the next model request. Use `-- chat` for ordinary chat or
|
||||
`REA_VERIFY_RUNTIME_ROOT` for a production-only installed package. The loopback
|
||||
OpenAI completions/SSE model is a keyless BYOK endpoint declaring a million-token
|
||||
context with synthetic usage. A synthetic account-key value satisfies the
|
||||
client's print-mode gate; native local-only mode refuses hosted API calls.
|
||||
Updates, telemetry and cron are disabled. Hosted authentication, live models
|
||||
and Windows remain unverified.
|
||||
|
||||
`verify:omp-client` requires installed OMP (verified with the official Linux x64
|
||||
18.8.7 binary); select it with `REA_VERIFY_OMP_COMMAND`. This optional POSIX lane
|
||||
isolates the default agent, global config and XDG roots, guards setup targets,
|
||||
|
||||
@@ -188,6 +188,7 @@
|
||||
"verify:gemini-client": "npm run build:cached && node scripts/verify-gemini-client.mjs",
|
||||
"verify:opencode-client": "npm run build:cached && node scripts/verify-opencode-client.mjs",
|
||||
"verify:grok-client": "npm run build:cached && node scripts/verify-grok-client.mjs",
|
||||
"verify:commandcode-client": "npm run build:cached && node scripts/verify-commandcode-client.mjs",
|
||||
"verify:omp-client": "npm run build:cached && node scripts/verify-omp-client.mjs",
|
||||
"verify:copilot-client": "npm run build:cached && node scripts/verify-copilot-client.mjs",
|
||||
"verify:claude-client": "npm run build:cached && node scripts/verify-claude-client.mjs",
|
||||
|
||||
@@ -0,0 +1,452 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { homedir } from "node:os";
|
||||
import { isAbsolute, join, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
|
||||
|
||||
assert(
|
||||
process.platform !== "win32",
|
||||
"This verifier requires a disposable POSIX account; Windows is unverified",
|
||||
);
|
||||
const account = process.env.REA_VERIFY_COMMANDCODE_ACCOUNT_HOME;
|
||||
assert(
|
||||
account && isAbsolute(account) && account === homedir(),
|
||||
"Run as a disposable account whose actual OS home matches REA_VERIFY_COMMANDCODE_ACCOUNT_HOME; do not override HOME",
|
||||
);
|
||||
assert.equal(
|
||||
await readFile(join(account, ".rea-client-verification"), "utf8"),
|
||||
"Disposable REA client verification account\n",
|
||||
);
|
||||
const repo = fileURLToPath(new URL("..", import.meta.url));
|
||||
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_COMMANDCODE_COMMAND ?? "cmd";
|
||||
const mode = process.argv[2] ?? "call";
|
||||
assert(
|
||||
["chat", "call"].includes(mode),
|
||||
"Usage: verify-commandcode-client.mjs [chat|call]",
|
||||
);
|
||||
const verifier = createVerifierRun();
|
||||
const lab = await mkdtemp(join(account, "rea-commandcode-client-"));
|
||||
const workspace = join(lab, "workspace");
|
||||
const target = join(workspace, "fixture α测试");
|
||||
const profile = join(account, ".commandcode");
|
||||
const configPath = join(profile, "mcp.json");
|
||||
const skillDirectory = join(
|
||||
account,
|
||||
".agents",
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
);
|
||||
await Promise.all([
|
||||
mkdir(target, { recursive: true }),
|
||||
mkdir(profile, { recursive: true }),
|
||||
mkdir(join(lab, "tmp")),
|
||||
]);
|
||||
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
||||
await writeFile(
|
||||
join(target, "app.js"),
|
||||
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
|
||||
);
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
NODE_OPTIONS: process.env.NODE_OPTIONS,
|
||||
USERPROFILE: account,
|
||||
TMPDIR: join(lab, "tmp"),
|
||||
CMD_LOCAL_ONLY: "1",
|
||||
// Native print mode checks for an account key even with a keyless BYOK model.
|
||||
// This synthetic value is never a real credential; local-only refuses hosted API calls.
|
||||
COMMAND_CODE_API_KEY: "local-fixture-only",
|
||||
COMMANDCODE_SKIP_UPDATES: "1",
|
||||
COMMANDCODE_DISABLE_CRON: "1",
|
||||
OTEL_SDK_DISABLED: "true",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
const expectedNames = TOOL_CONTRACTS.map(
|
||||
(tool) => "mcp__rea__" + tool.name,
|
||||
).sort();
|
||||
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
|
||||
const textValues = (value) =>
|
||||
typeof value === "string"
|
||||
? [value]
|
||||
: value !== null && typeof value === "object"
|
||||
? Object.values(value).flatMap(textValues)
|
||||
: [];
|
||||
let clientVersion;
|
||||
let expectedSkill;
|
||||
let inventoryChecked = false;
|
||||
let skillLoaded = false;
|
||||
let schemasChecked = 0;
|
||||
let catalogArtifact;
|
||||
let catalogDigest;
|
||||
let catalogProjection;
|
||||
let artifactProjectionSeen = false;
|
||||
let evidenceSeen = false;
|
||||
let evidenceBytes;
|
||||
let finalSeen = false;
|
||||
let stage = 0;
|
||||
let failure;
|
||||
const call = (name, args) => ({
|
||||
role: "assistant",
|
||||
content: null,
|
||||
tool_calls: [
|
||||
{
|
||||
id: `call_commandcode_${++stage}`,
|
||||
type: "function",
|
||||
function: { name, arguments: JSON.stringify(args) },
|
||||
},
|
||||
],
|
||||
});
|
||||
const validateCatalog = async (text) => {
|
||||
const entries = [
|
||||
...text.matchAll(
|
||||
/^### (mcp__rea__[A-Za-z0-9_]+)\n[\s\S]*?\nParameters:\n```json\n([\s\S]*?)\n```/gm,
|
||||
),
|
||||
];
|
||||
assert.deepEqual(
|
||||
entries.map((entry) => entry[1]).sort(),
|
||||
expectedNames,
|
||||
"Native exact-name search must return every REA input schema",
|
||||
);
|
||||
for (const entry of entries) {
|
||||
const schema = JSON.parse(entry[2]);
|
||||
assert(
|
||||
schemaValidator.validateSchema(schema),
|
||||
`${entry[1]}: invalid native schema: ${JSON.stringify(schemaValidator.errors)}`,
|
||||
);
|
||||
}
|
||||
schemasChecked = entries.length;
|
||||
catalogDigest = createHash("sha256").update(text).digest("hex");
|
||||
await writeFile(join(lab, "native-catalog.md"), text);
|
||||
};
|
||||
const fixture = await createOpenAiModelFixture({
|
||||
directory: lab,
|
||||
prefix: "commandcode",
|
||||
onRequest: async ({ body, tools }) => {
|
||||
assert(tools.some((tool) => tool.name === "search_tools"));
|
||||
const strings = textValues(body.messages);
|
||||
if (!inventoryChecked) {
|
||||
const names = [
|
||||
...new Set(
|
||||
strings.flatMap(
|
||||
(text) => text.match(/mcp__rea__[A-Za-z0-9_]+/g) ?? [],
|
||||
),
|
||||
),
|
||||
].sort();
|
||||
assert.deepEqual(
|
||||
names,
|
||||
expectedNames,
|
||||
"Default native deferred-tool prompt must advertise the complete REA inventory",
|
||||
);
|
||||
assert.equal(
|
||||
tools.filter((tool) => tool.name.startsWith("mcp__rea__")).length,
|
||||
0,
|
||||
"Keep native lazy schema delivery enabled",
|
||||
);
|
||||
assert(
|
||||
tools
|
||||
.find((tool) => tool.name === "activate_skill")
|
||||
?.parameters.properties.name.enum.includes(
|
||||
"reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
inventoryChecked = true;
|
||||
}
|
||||
if (mode === "chat") {
|
||||
assert.equal(
|
||||
body.messages.filter((message) => message.role === "tool").length,
|
||||
0,
|
||||
);
|
||||
finalSeen = true;
|
||||
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
|
||||
}
|
||||
if (stage === 0)
|
||||
return call("activate_skill", { name: "reverse-engineer-anything" });
|
||||
assert(
|
||||
strings.some((text) => text.includes(expectedSkill)),
|
||||
"The model must receive the complete activated skill body",
|
||||
);
|
||||
skillLoaded = true;
|
||||
const last = body.messages
|
||||
.filter((message) => message.role === "tool")
|
||||
.at(-1)?.content;
|
||||
assert.equal(typeof last, "string");
|
||||
if (stage === 1)
|
||||
return call("search_tools", {
|
||||
query: "select:" + expectedNames.join(","),
|
||||
});
|
||||
if (stage === 2) {
|
||||
const spill = last.match(
|
||||
/\[full output saved to: (.+?) — read it with read_file/,
|
||||
);
|
||||
if (spill) {
|
||||
catalogArtifact = spill[1];
|
||||
const child = relative(environment.TMPDIR, catalogArtifact);
|
||||
assert(
|
||||
isAbsolute(catalogArtifact) &&
|
||||
child &&
|
||||
!child.startsWith("..") &&
|
||||
!isAbsolute(child),
|
||||
"Native spill must remain inside this run's temporary directory",
|
||||
);
|
||||
const text = await readFile(catalogArtifact, "utf8");
|
||||
await validateCatalog(text);
|
||||
catalogProjection = { schemas: schemasChecked, sha256: catalogDigest };
|
||||
const query =
|
||||
"const fs=require('node:fs'),crypto=require('node:crypto');const text=fs.readFileSync(process.argv[1],'utf8');const schemas=[...text.matchAll(/^### (mcp__rea__[A-Za-z0-9_]+)\\n/gm)].length;console.log(JSON.stringify({rea_catalog:{schemas,sha256:crypto.createHash('sha256').update(text).digest('hex')}}));";
|
||||
return call("shell_command", {
|
||||
command: process.execPath,
|
||||
args: ["-e", query, catalogArtifact],
|
||||
});
|
||||
}
|
||||
await validateCatalog(last);
|
||||
return call("mcp__rea__analyze_javascript_application", {
|
||||
input_path: target,
|
||||
format: "directory",
|
||||
});
|
||||
}
|
||||
if (catalogArtifact && !artifactProjectionSeen) {
|
||||
const line = last
|
||||
.split("\n")
|
||||
.find((item) => item.startsWith('{"rea_catalog":'));
|
||||
assert(line, "Native shell must return the selected saved-catalog facts");
|
||||
assert.deepEqual(JSON.parse(line).rea_catalog, catalogProjection);
|
||||
artifactProjectionSeen = true;
|
||||
return call("mcp__rea__analyze_javascript_application", {
|
||||
input_path: target,
|
||||
format: "directory",
|
||||
});
|
||||
}
|
||||
const evidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(JSON.parse(last));
|
||||
assert.equal(evidence.subject.local_path, target);
|
||||
const shapes = evidence.normalized_result.graph.nodes
|
||||
.flatMap((node) => node.observations)
|
||||
.map((observation) => observation.properties)
|
||||
.filter(
|
||||
(properties) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "result",
|
||||
);
|
||||
assert(
|
||||
shapes.length > 0 &&
|
||||
shapes.every(
|
||||
(shape) => shape.return_shape_coverage?.projection_complete,
|
||||
),
|
||||
);
|
||||
const fields = shapes.flatMap((shape) =>
|
||||
shape.static_return_shapes.flatMap((item) => item.fields),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/clientCompatibility" &&
|
||||
field.state === "literal" &&
|
||||
field.value === "REA_CLIENT_FIXTURE",
|
||||
),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/count" &&
|
||||
field.state === "literal" &&
|
||||
field.value === 7,
|
||||
),
|
||||
);
|
||||
evidenceBytes = Buffer.byteLength(last);
|
||||
await writeFile(join(lab, "native-evidence.json"), last);
|
||||
evidenceSeen = true;
|
||||
finalSeen = true;
|
||||
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
|
||||
},
|
||||
});
|
||||
await writeFile(
|
||||
join(profile, "providers.json"),
|
||||
JSON.stringify({
|
||||
provider: {
|
||||
"rea-local": {
|
||||
baseURL: fixture.baseUrl,
|
||||
apiKey: false,
|
||||
models: {
|
||||
"rea-client-fixture": { contextWindow: 1_000_000, maxOutput: 4096 },
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const original =
|
||||
JSON.stringify(
|
||||
{
|
||||
mcpServers: { other: { command: "unrelated-server", enabled: false } },
|
||||
preserved: "keep unrelated settings",
|
||||
},
|
||||
null,
|
||||
2,
|
||||
) + "\n";
|
||||
await writeFile(configPath, original);
|
||||
const rea = (args) =>
|
||||
exec(
|
||||
process.execPath,
|
||||
[join(runtimeRoot, "scripts/rea.mjs"), ...args, "--json"],
|
||||
{ env: environment, cwd: workspace, timeout: 60_000 },
|
||||
);
|
||||
try {
|
||||
clientVersion = (
|
||||
await exec(command, ["--version"], {
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout.trim();
|
||||
for (const args of [
|
||||
["setup", "--client", "commandcode", "--dry-run"],
|
||||
["setup", "--client", "commandcode", "--yes"],
|
||||
["doctor", "--client", "commandcode", "--skill"],
|
||||
]) {
|
||||
const response = await rea(args);
|
||||
await writeFile(
|
||||
join(lab, `${args[0]}${args.includes("--dry-run") ? "-plan" : ""}.json`),
|
||||
response.stdout,
|
||||
);
|
||||
const result = JSON.parse(response.stdout);
|
||||
if (args[0] === "doctor") assert.equal(result.healthy, true);
|
||||
else
|
||||
assert.equal(
|
||||
result.status,
|
||||
args.includes("--dry-run") ? "planned" : "ready",
|
||||
);
|
||||
if (args.includes("--dry-run")) {
|
||||
assert(
|
||||
result.plannedActions.some((action) => action.target === configPath),
|
||||
);
|
||||
assert(
|
||||
result.plannedActions.every(
|
||||
(action) =>
|
||||
action.target === configPath ||
|
||||
action.target === skillDirectory ||
|
||||
action.target.startsWith(skillDirectory + "/"),
|
||||
),
|
||||
"Refuse setup writes outside the disposable account's configuration and skill",
|
||||
);
|
||||
}
|
||||
}
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
const configured = await readFile(configPath, "utf8");
|
||||
assert.equal(JSON.parse(configured).preserved, "keep unrelated settings");
|
||||
assert.equal(
|
||||
JSON.parse(configured).mcpServers.other.command,
|
||||
"unrelated-server",
|
||||
);
|
||||
const repeated = JSON.parse(
|
||||
(await rea(["setup", "--client", "commandcode", "--yes"])).stdout,
|
||||
);
|
||||
assert.deepEqual(repeated.appliedActions, []);
|
||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
expectedSkill = (await readFile(join(skillDirectory, "SKILL.md"), "utf8"))
|
||||
.replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n/, "")
|
||||
.trim();
|
||||
const execution = exec(
|
||||
command,
|
||||
[
|
||||
"-p",
|
||||
mode === "chat"
|
||||
? "Say hello briefly."
|
||||
: `Use the REA skill to analyze ${target}.`,
|
||||
"-m",
|
||||
"rea-local/rea-client-fixture",
|
||||
"--local-only",
|
||||
"--skip-onboarding",
|
||||
"--trust",
|
||||
"--output-format",
|
||||
"json",
|
||||
"--max-turns",
|
||||
"8",
|
||||
"--yolo",
|
||||
],
|
||||
{
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 120_000,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
execution.child.stdin.end();
|
||||
const response = await execution;
|
||||
await writeFile(join(lab, `${mode}.stdout`), response.stdout);
|
||||
await writeFile(join(lab, `${mode}.stderr`), response.stderr);
|
||||
assert.equal(fixture.failure, undefined);
|
||||
const events = response.stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
assert(
|
||||
events.some(
|
||||
(event) =>
|
||||
event.type === "result" &&
|
||||
event.subtype === "success" &&
|
||||
event.finalText === "REA_CLIENT_COMPATIBILITY_OK",
|
||||
),
|
||||
);
|
||||
assert(inventoryChecked && finalSeen);
|
||||
if (mode === "call")
|
||||
assert(
|
||||
skillLoaded &&
|
||||
schemasChecked === expectedNames.length &&
|
||||
evidenceSeen &&
|
||||
(!catalogArtifact || artifactProjectionSeen),
|
||||
);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
await writeFile(join(lab, `${mode}.stdout`), cause.stdout);
|
||||
if (typeof cause?.stderr === "string")
|
||||
await writeFile(join(lab, `${mode}.stderr`), cause.stderr);
|
||||
} finally {
|
||||
await fixture.close();
|
||||
const receipt = {
|
||||
client: "Command Code",
|
||||
clientVersion,
|
||||
mode,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
host: {
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
node: process.versions.node,
|
||||
},
|
||||
modelFixture:
|
||||
"loopback OpenAI completions/SSE, keyless BYOK, synthetic account-key gate and usage; no hosted authentication or live-model claim",
|
||||
discovery:
|
||||
"actual disposable OS account home; native user MCP configuration and default shared skill discovery; Windows unverified",
|
||||
interface:
|
||||
"default deferred schemas, complete exact-name search, native saved-catalog query when needed, actual JavaScript MCP call",
|
||||
inventoryChecked,
|
||||
catalogSize: inventoryChecked ? expectedNames.length : 0,
|
||||
schemasChecked,
|
||||
skillLoaded,
|
||||
catalogArtifact,
|
||||
catalogDigest,
|
||||
artifactProjectionSeen,
|
||||
evidenceSeen,
|
||||
evidenceBytes,
|
||||
finalSeen,
|
||||
requests: fixture.requests,
|
||||
probes: fixture.probes,
|
||||
fixtureFailure: fixture.failure,
|
||||
verifier: await completeVerifierRun(verifier),
|
||||
};
|
||||
await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
|
||||
console.log(JSON.stringify(receipt));
|
||||
}
|
||||
if (failure !== undefined) throw failure;
|
||||
+348
-46
@@ -1,6 +1,6 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { createReadStream, constants as fsConstants } from "node:fs";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
|
||||
import { basename, join, resolve } from "node:path";
|
||||
|
||||
@@ -24,8 +24,11 @@ import {
|
||||
AnalysisTimeoutError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
execFileOutput,
|
||||
execFileOutputFailure,
|
||||
@@ -97,11 +100,18 @@ const STDERR_EXCERPT_BYTES = 512;
|
||||
const UNPARSED_LINES_REPORTED = 10;
|
||||
const STOP_GRACE_MS = 5_000;
|
||||
|
||||
/** Stream a local file digest without retaining large files in memory. */
|
||||
const sha256File = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
const hashAdbFile = async (
|
||||
path: string,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
) => {
|
||||
try {
|
||||
return await hashStableFile(path, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (signal?.aborted === true)
|
||||
throw new AnalysisCancelledError(operation, { cause });
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
/** adb joins shell operands into a remote POSIX command; quote at that boundary. */
|
||||
@@ -335,6 +345,48 @@ export interface AdbProviderOptions {
|
||||
readonly environment?: Readonly<Record<string, string | undefined>>;
|
||||
}
|
||||
|
||||
interface PendingPackageDirectoryCleanup {
|
||||
readonly path: string;
|
||||
readonly identity: PackageDirectoryIdentity | undefined;
|
||||
readonly primary: AnalysisError;
|
||||
readonly remove: () => Promise<void>;
|
||||
}
|
||||
|
||||
interface PackageDirectoryIdentity {
|
||||
readonly dev: number;
|
||||
readonly ino: number;
|
||||
readonly mode: number;
|
||||
}
|
||||
|
||||
const removeOwnedPackageDirectory = async (
|
||||
path: string,
|
||||
identity: PackageDirectoryIdentity | undefined,
|
||||
): Promise<void> => {
|
||||
if (identity === undefined)
|
||||
throw new Error(
|
||||
`Cannot verify the created package directory identity; refusing to remove ${path}`,
|
||||
);
|
||||
let current: Awaited<ReturnType<typeof lstat>>;
|
||||
try {
|
||||
current = await lstat(path);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
|
||||
return;
|
||||
throw cause;
|
||||
}
|
||||
if (
|
||||
!current.isDirectory() ||
|
||||
current.isSymbolicLink() ||
|
||||
current.dev !== identity.dev ||
|
||||
current.ino !== identity.ino ||
|
||||
current.mode !== identity.mode
|
||||
)
|
||||
throw new Error(
|
||||
`The created package directory was replaced or changed; refusing to remove ${path}`,
|
||||
);
|
||||
await rm(path, { recursive: true, force: true });
|
||||
};
|
||||
|
||||
/**
|
||||
* ADB device inspection and acquisition over a caller-selected adb binary.
|
||||
*
|
||||
@@ -346,6 +398,12 @@ export interface AdbProviderOptions {
|
||||
*/
|
||||
export class AdbProvider {
|
||||
private readonly environment: Readonly<Record<string, string | undefined>>;
|
||||
readonly #pendingPackageDirectoryCleanup =
|
||||
new Set<PendingPackageDirectoryCleanup>();
|
||||
readonly #activeOperations = new Set<Promise<void>>();
|
||||
#closing = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#packageCleanupPromise: Promise<void> | undefined;
|
||||
|
||||
constructor(options: AdbProviderOptions = {}) {
|
||||
this.environment = options.environment ?? process.env;
|
||||
@@ -378,12 +436,121 @@ export class AdbProvider {
|
||||
}
|
||||
}
|
||||
|
||||
/** The provider holds no long-lived resources; the adb server is caller-owned. */
|
||||
async close(): Promise<void> {}
|
||||
/** Retry removal of package directories retained after failed rollback. */
|
||||
close(): Promise<void> {
|
||||
this.#closing = true;
|
||||
if (this.#closePromise !== undefined) return this.#closePromise;
|
||||
const closing = (async () => {
|
||||
await Promise.all(this.#activeOperations);
|
||||
await this.#retryPackageDirectoryCleanup();
|
||||
})().finally(() => {
|
||||
if (this.#closePromise === closing) this.#closePromise = undefined;
|
||||
});
|
||||
this.#closePromise = closing;
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
#retryPackageDirectoryCleanup(): Promise<void> {
|
||||
if (this.#packageCleanupPromise !== undefined)
|
||||
return this.#packageCleanupPromise;
|
||||
const cleanup = this.#performPackageDirectoryCleanup().finally(() => {
|
||||
if (this.#packageCleanupPromise === cleanup)
|
||||
this.#packageCleanupPromise = undefined;
|
||||
});
|
||||
this.#packageCleanupPromise = cleanup;
|
||||
return cleanup;
|
||||
}
|
||||
|
||||
async #performPackageDirectoryCleanup(): Promise<void> {
|
||||
const failures: AnalysisError[] = [];
|
||||
for (const owner of this.#pendingPackageDirectoryCleanup) {
|
||||
try {
|
||||
await owner.remove();
|
||||
this.#pendingPackageDirectoryCleanup.delete(owner);
|
||||
} catch (cause: unknown) {
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[owner.path],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: "pull_adb_package", cause },
|
||||
);
|
||||
failures.push(
|
||||
analysisErrorWithCleanupFailure(
|
||||
owner.primary,
|
||||
cleanup,
|
||||
"pull_adb_package",
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (failures.length === 1) throw failures[0];
|
||||
if (failures.length > 1)
|
||||
throw new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[...this.#pendingPackageDirectoryCleanup].map(({ path }) => path),
|
||||
{
|
||||
reason:
|
||||
"Failed package output directories remain after cleanup retry",
|
||||
failures: failures.map(({ message, cleanupResources }) => ({
|
||||
message,
|
||||
resources: [...cleanupResources],
|
||||
})),
|
||||
},
|
||||
{
|
||||
operation: "pull_adb_package",
|
||||
cause: new AggregateError(
|
||||
failures,
|
||||
"Multiple ADB package rollback cleanups failed",
|
||||
),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
async execute(
|
||||
request: AdbRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
if (this.#closing)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
let release!: () => void;
|
||||
const active = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
this.#activeOperations.add(active);
|
||||
try {
|
||||
if (this.#pendingPackageDirectoryCleanup.size > 0) {
|
||||
try {
|
||||
await this.#retryPackageDirectoryCleanup();
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
cause instanceof AnalysisError
|
||||
? cause
|
||||
: new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[...this.#pendingPackageDirectoryCleanup].map(
|
||||
({ path }) => path,
|
||||
),
|
||||
{
|
||||
reason:
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: request.operation, cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
return await this.#executeOperation(request, options);
|
||||
} finally {
|
||||
this.#activeOperations.delete(active);
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
async #executeOperation(
|
||||
request: AdbRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const signal = options?.signal;
|
||||
if (signal?.aborted === true)
|
||||
@@ -1010,9 +1177,11 @@ export class AdbProvider {
|
||||
);
|
||||
if (!directory.ok) return directory;
|
||||
const packageDirectory = join(directory.value, packageName);
|
||||
let packageDirectoryCreated = false;
|
||||
let packageDirectoryIdentity: PackageDirectoryIdentity | undefined;
|
||||
try {
|
||||
if (
|
||||
await stat(packageDirectory).then(
|
||||
await lstat(packageDirectory).then(
|
||||
() => true,
|
||||
() => false,
|
||||
)
|
||||
@@ -1021,14 +1190,44 @@ export class AdbProvider {
|
||||
`The package output directory already exists: ${packageDirectory}`,
|
||||
);
|
||||
await mkdir(packageDirectory);
|
||||
packageDirectoryCreated = true;
|
||||
const created = await lstat(packageDirectory);
|
||||
if (!created.isDirectory() || created.isSymbolicLink())
|
||||
throw new Error(
|
||||
`The created package output is not a regular directory: ${packageDirectory}`,
|
||||
);
|
||||
packageDirectoryIdentity = {
|
||||
dev: created.dev,
|
||||
ino: created.ino,
|
||||
mode: created.mode,
|
||||
};
|
||||
} catch (cause) {
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
{ cause },
|
||||
),
|
||||
const primary = new AnalysisOutputError(
|
||||
operation,
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
{ cause },
|
||||
);
|
||||
if (!packageDirectoryCreated) return err(primary);
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{
|
||||
reason: "Could not establish an owned package directory identity",
|
||||
},
|
||||
{ operation, cause },
|
||||
);
|
||||
return err(analysisErrorWithCleanupFailure(primary, cleanup, operation));
|
||||
}
|
||||
const artifacts: {
|
||||
device_path: string;
|
||||
@@ -1044,6 +1243,37 @@ export class AdbProvider {
|
||||
stage: "resolve_paths" | "pull" | "digest";
|
||||
message: string;
|
||||
}[] = [];
|
||||
const cancelledPackagePull = (
|
||||
error: AnalysisCancelledError,
|
||||
localPath: string,
|
||||
devicePath: string,
|
||||
stage: "pull" | "digest",
|
||||
) => {
|
||||
error.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial,
|
||||
package_name: packageName,
|
||||
output_directory: directory.value,
|
||||
artifacts,
|
||||
failures: [
|
||||
...failures,
|
||||
{
|
||||
device_path: devicePath,
|
||||
stage,
|
||||
message:
|
||||
stage === "pull"
|
||||
? `Pull cancelled; any incomplete file is retained at ${localPath}. Remaining APKs were not attempted.`
|
||||
: `Digesting the pulled file was cancelled; the file is retained at ${localPath}. Remaining APKs were not attempted.`,
|
||||
},
|
||||
],
|
||||
coverage: "partial",
|
||||
},
|
||||
});
|
||||
return err(error);
|
||||
};
|
||||
const seenFileNames = new Set<string>();
|
||||
for (const entry of parsed.paths) {
|
||||
if (seenFileNames.has(entry.file_name)) {
|
||||
@@ -1065,7 +1295,13 @@ export class AdbProvider {
|
||||
signal,
|
||||
);
|
||||
if (!pulled.ok) {
|
||||
if (pulled.error instanceof AnalysisCancelledError) return pulled;
|
||||
if (pulled.error instanceof AnalysisCancelledError)
|
||||
return cancelledPackagePull(
|
||||
pulled.error,
|
||||
localPath,
|
||||
entry.device_path,
|
||||
"pull",
|
||||
);
|
||||
failures.push({
|
||||
device_path: entry.device_path,
|
||||
stage: "pull",
|
||||
@@ -1074,18 +1310,24 @@ export class AdbProvider {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const info = await stat(localPath);
|
||||
const sha256 = await sha256File(localPath);
|
||||
const digest = await hashAdbFile(localPath, operation, signal);
|
||||
artifacts.push({
|
||||
device_path: entry.device_path,
|
||||
file_name: entry.file_name,
|
||||
local_path: localPath,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
role: entry.role,
|
||||
role_basis: "file_name",
|
||||
});
|
||||
} catch {
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError)
|
||||
return cancelledPackagePull(
|
||||
cause,
|
||||
localPath,
|
||||
entry.device_path,
|
||||
"digest",
|
||||
);
|
||||
failures.push({
|
||||
device_path: entry.device_path,
|
||||
stage: "digest",
|
||||
@@ -1094,14 +1336,36 @@ export class AdbProvider {
|
||||
}
|
||||
}
|
||||
if (artifacts.length === 0 && failures.length > 0) {
|
||||
await rm(packageDirectory, { recursive: true, force: true }).catch(
|
||||
() => undefined,
|
||||
);
|
||||
return err(
|
||||
new AnalysisProtocolError(
|
||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||
),
|
||||
const primary = new AnalysisProtocolError(
|
||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||
);
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
try {
|
||||
await owner.remove();
|
||||
} catch (cause: unknown) {
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: operation, cause },
|
||||
);
|
||||
return err(
|
||||
analysisErrorWithCleanupFailure(primary, cleanup, operation),
|
||||
);
|
||||
}
|
||||
return err(primary);
|
||||
}
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
@@ -1156,8 +1420,7 @@ export class AdbProvider {
|
||||
);
|
||||
if (!pulled.ok) return pulled;
|
||||
try {
|
||||
const info = await stat(localPath);
|
||||
const sha256 = await sha256File(localPath);
|
||||
const digest = await hashAdbFile(localPath, operation, signal);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
@@ -1165,8 +1428,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
device_path: input.device_path,
|
||||
local_path: localPath,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
},
|
||||
provider,
|
||||
{
|
||||
@@ -1177,6 +1440,20 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) {
|
||||
cause.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
digest_status: "unknown",
|
||||
},
|
||||
});
|
||||
return err(cause);
|
||||
}
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
@@ -1211,7 +1488,13 @@ export class AdbProvider {
|
||||
cause: new Error(`The local file does not exist: ${localPath}`),
|
||||
}),
|
||||
);
|
||||
const sha256 = await sha256File(localPath);
|
||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
||||
try {
|
||||
digest = await hashAdbFile(localPath, operation, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
throw cause;
|
||||
}
|
||||
const existsProbe = await runAdb(
|
||||
operation,
|
||||
input.device_path,
|
||||
@@ -1257,7 +1540,7 @@ export class AdbProvider {
|
||||
);
|
||||
if (deviceDigest.ok) {
|
||||
const digestMatch = /^([a-f0-9]{64})\s/u.exec(deviceDigest.value.stdout);
|
||||
if (digestMatch !== null && digestMatch[1] !== sha256)
|
||||
if (digestMatch !== null && digestMatch[1] !== digest.sha256)
|
||||
return err(
|
||||
new AnalysisArtifactChangedError(
|
||||
operation,
|
||||
@@ -1272,8 +1555,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
device_sha256: digestMatch?.[1] ?? null,
|
||||
overwritten: deviceExists,
|
||||
device_digest_source: "device_sha256sum",
|
||||
@@ -1294,8 +1577,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
device_sha256: null,
|
||||
overwritten: deviceExists,
|
||||
device_digest_source: "unavailable",
|
||||
@@ -1334,7 +1617,13 @@ export class AdbProvider {
|
||||
cause: new Error(`The local APK does not exist: ${apkPath}`),
|
||||
}),
|
||||
);
|
||||
const sha256 = await sha256File(apkPath);
|
||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
||||
try {
|
||||
digest = await hashAdbFile(apkPath, operation, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
throw cause;
|
||||
}
|
||||
const installed = await runAdb(
|
||||
operation,
|
||||
input.serial,
|
||||
@@ -1363,8 +1652,8 @@ export class AdbProvider {
|
||||
client,
|
||||
serial: input.serial,
|
||||
apk_path: apkPath,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
replaced: input.replace,
|
||||
},
|
||||
provider,
|
||||
@@ -1734,16 +2023,15 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
try {
|
||||
const info = await stat(reportedPath);
|
||||
const sha256 = await sha256File(reportedPath);
|
||||
const digest = await hashAdbFile(reportedPath, operation, signal);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: reportedPath,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
adb_reported_path: reportedPath,
|
||||
},
|
||||
provider,
|
||||
@@ -1755,6 +2043,20 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) {
|
||||
cause.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: reportedPath,
|
||||
adb_reported_path: reportedPath,
|
||||
digest_status: "unknown",
|
||||
},
|
||||
});
|
||||
return err(cause);
|
||||
}
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
|
||||
@@ -1,40 +1,43 @@
|
||||
import { constants, createReadStream } from "node:fs";
|
||||
import { chmod, copyFile } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { ReadStream, Stats, WriteStream } from "node:fs";
|
||||
import { chmod, open, rm, stat, type FileHandle } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { Transform, type TransformCallback } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
||||
|
||||
/** Stream a local file digest without retaining the APK or JAR in memory. */
|
||||
export const hashAndroidFile = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
|
||||
/** Bind decompiler input to the admitted artifact, even if the original changes. */
|
||||
export const snapshotAndroidTarget = async (
|
||||
path: string,
|
||||
sha256: string,
|
||||
root: string,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const snapshot = join(root, "target.apk");
|
||||
await copyFile(path, snapshot, constants.COPYFILE_EXCL);
|
||||
await chmod(snapshot, 0o400);
|
||||
if ((await hashAndroidFile(snapshot)) !== sha256)
|
||||
throw new AnalysisInputError(operation, undefined, [
|
||||
try {
|
||||
await copyAndroidSnapshot(path, snapshot, sha256, signal);
|
||||
signal?.throwIfAborted();
|
||||
await chmod(snapshot, 0o400);
|
||||
return snapshot;
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
||||
throw new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"APK bytes changed after admission; retry against a stable file.",
|
||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
||||
},
|
||||
]);
|
||||
return snapshot;
|
||||
}
|
||||
};
|
||||
|
||||
/** Fingerprint the actual immutable engine bytes executed for this observation. */
|
||||
@@ -43,15 +46,120 @@ export const snapshotAndroidEngine = async (
|
||||
sha256: string,
|
||||
root: string,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ path: string; sha256: string }> => {
|
||||
const path = join(root, "engine.jar");
|
||||
await copyFile(source, path, constants.COPYFILE_EXCL);
|
||||
await chmod(path, 0o400);
|
||||
if ((await hashAndroidFile(path)) !== sha256)
|
||||
try {
|
||||
const observed = await copyAndroidSnapshot(source, path, sha256, signal);
|
||||
signal?.throwIfAborted();
|
||||
await chmod(path, 0o400);
|
||||
return { path, sha256: observed };
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
"REA_JADX_MCP_JAR bytes changed during admission; retry with a stable engine file.",
|
||||
`REA_JADX_MCP_JAR bytes changed during admission at ${source}; retry with a stable engine file.`,
|
||||
{ cause },
|
||||
);
|
||||
return { path, sha256 };
|
||||
}
|
||||
};
|
||||
|
||||
const copyAndroidSnapshot = async (
|
||||
sourcePath: string,
|
||||
snapshotPath: string,
|
||||
expectedSha256: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
signal?.throwIfAborted();
|
||||
const source = await openRegularFile(sourcePath, {
|
||||
symlinks: "follow",
|
||||
signal,
|
||||
});
|
||||
let destination: FileHandle | undefined;
|
||||
let inputStream: ReadStream | undefined;
|
||||
let outputStream: WriteStream | undefined;
|
||||
let removeSnapshot = false;
|
||||
let copiedBytes = 0;
|
||||
const hash = createHash("sha256");
|
||||
try {
|
||||
const initial = await source.stat();
|
||||
signal?.throwIfAborted();
|
||||
const digest = new Transform({
|
||||
transform(
|
||||
chunk: Buffer,
|
||||
_encoding: BufferEncoding,
|
||||
callback: TransformCallback,
|
||||
) {
|
||||
copiedBytes += chunk.length;
|
||||
if (copiedBytes > initial.size) {
|
||||
callback(
|
||||
new AndroidSnapshotChangedError(
|
||||
`Android input grew while being copied: ${sourcePath}`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
hash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
try {
|
||||
destination = await open(snapshotPath, "wx", 0o600);
|
||||
signal?.throwIfAborted();
|
||||
inputStream = source.createReadStream({
|
||||
start: 0,
|
||||
end: initial.size,
|
||||
autoClose: false,
|
||||
});
|
||||
outputStream = destination.createWriteStream({ autoClose: false });
|
||||
await pipeline(inputStream, digest, outputStream, { signal });
|
||||
signal?.throwIfAborted();
|
||||
await verifySourceState(source, sourcePath, initial, copiedBytes);
|
||||
const observedSha256 = hash.digest("hex");
|
||||
if (observedSha256 !== expectedSha256)
|
||||
throw new AndroidSnapshotChangedError(
|
||||
`Android snapshot digest mismatch for ${sourcePath}`,
|
||||
);
|
||||
return observedSha256;
|
||||
} catch (cause: unknown) {
|
||||
removeSnapshot = destination !== undefined;
|
||||
throw cause;
|
||||
}
|
||||
} finally {
|
||||
// Non-auto-closing streams retain a reference to their FileHandle even
|
||||
// after pipeline completion. Release those references before awaiting
|
||||
// descriptor closure, after the admitted source state has been verified.
|
||||
inputStream?.destroy();
|
||||
outputStream?.destroy();
|
||||
try {
|
||||
await Promise.all([source.close(), destination?.close()]);
|
||||
} finally {
|
||||
if (removeSnapshot)
|
||||
await rm(snapshotPath, { force: true }).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const verifySourceState = async (
|
||||
handle: Awaited<ReturnType<typeof openRegularFile>>,
|
||||
path: string,
|
||||
initial: Stats,
|
||||
observedBytes: number,
|
||||
): Promise<void> => {
|
||||
const [opened, currentPath] = await Promise.all([handle.stat(), stat(path)]);
|
||||
if (
|
||||
observedBytes !== initial.size ||
|
||||
!sameRegularFileState(initial, opened) ||
|
||||
!sameRegularFileState(initial, currentPath)
|
||||
)
|
||||
throw new AndroidSnapshotChangedError(
|
||||
`Android input changed while being read: ${path}`,
|
||||
);
|
||||
};
|
||||
|
||||
class AndroidSnapshotChangedError extends Error {
|
||||
constructor(message: string, options?: ErrorOptions) {
|
||||
super(message, options);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,7 +52,11 @@ export const inspectJadxAvailability = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<ProviderAvailability> => {
|
||||
try {
|
||||
const configuration = await readJadxConfiguration(environment, signal);
|
||||
const configuration = await readJadxConfigurationInputs(
|
||||
environment,
|
||||
signal,
|
||||
);
|
||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
||||
return {
|
||||
status: "available",
|
||||
code: null,
|
||||
@@ -77,52 +81,83 @@ export const inspectJadxAvailability = async (
|
||||
}
|
||||
};
|
||||
|
||||
/** Resolve caller-selected tools and preserve the same prerequisite diagnostics. */
|
||||
export const resolveJadxConfiguration = async (
|
||||
/** Resolve caller-selected paths and JVM options without probing their contents. */
|
||||
export const resolveJadxConfigurationInputs = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JadxConfiguration> => {
|
||||
try {
|
||||
return await readJadxConfiguration(environment, signal);
|
||||
return await readJadxConfigurationInputs(environment, signal);
|
||||
} catch (cause) {
|
||||
if (signal?.aborted === true) throw cause;
|
||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
||||
const output = cause.diagnostics;
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
cause.message,
|
||||
{
|
||||
cause,
|
||||
userMessage: cause.message,
|
||||
...(typeof output.stdout === "string" &&
|
||||
typeof output.stderr === "string"
|
||||
? {
|
||||
capturedOutput: {
|
||||
stdout: output.stdout,
|
||||
stderr: output.stderr,
|
||||
truncated: output.output_truncated === true,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
throw projectConfigurationFailure(operation, cause);
|
||||
}
|
||||
};
|
||||
|
||||
const readJadxConfiguration = async (
|
||||
/** Probe Java and archive prerequisites before starting a new engine session. */
|
||||
export const validateJadxConfiguration = async (
|
||||
configuration: JadxConfiguration,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
||||
} catch (cause) {
|
||||
if (signal?.aborted === true) throw cause;
|
||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
||||
throw projectConfigurationFailure(operation, cause);
|
||||
}
|
||||
};
|
||||
|
||||
const readJadxConfigurationInputs = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JadxConfiguration> => {
|
||||
signal?.throwIfAborted();
|
||||
requireSupportedHost();
|
||||
const jar = await resolveJar(environment);
|
||||
const java = await resolveJava(environment, signal);
|
||||
await requireJavaHomeExecutable(environment.JAVA_HOME, java);
|
||||
const jvmArguments = resolveJvmArguments(environment);
|
||||
await inspectJavaRuntime(java, environment, signal);
|
||||
await requireEngineArchive(jar, signal);
|
||||
return { jar, java, jvmArguments };
|
||||
signal?.throwIfAborted();
|
||||
return { jar, java, jvmArguments: resolveJvmArguments(environment) };
|
||||
};
|
||||
|
||||
const validateJadxConfigurationContents = async (
|
||||
configuration: JadxConfiguration,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
signal?.throwIfAborted();
|
||||
await inspectJavaRuntime(configuration.java, environment, signal);
|
||||
await requireEngineArchive(configuration.jar, signal);
|
||||
};
|
||||
|
||||
const projectConfigurationFailure = (
|
||||
operation: AndroidOperation,
|
||||
cause: JadxConfigurationFailure,
|
||||
): AnalysisCapabilityUnavailableError => {
|
||||
const output = cause.diagnostics;
|
||||
return new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
cause.message,
|
||||
{
|
||||
cause,
|
||||
userMessage: cause.message,
|
||||
...(typeof output.stdout === "string" && typeof output.stderr === "string"
|
||||
? {
|
||||
capturedOutput: {
|
||||
stdout: output.stdout,
|
||||
stderr: output.stderr,
|
||||
truncated: output.output_truncated === true,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
const requireEngineArchive = async (
|
||||
|
||||
+98
-28
@@ -21,12 +21,14 @@ import { err, ok, type Result } from "../domain/result.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
snapshotAndroidEngine,
|
||||
hashAndroidFile,
|
||||
snapshotAndroidTarget,
|
||||
} from "./AndroidTargetSnapshot.js";
|
||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
||||
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
inspectJadxAvailability,
|
||||
resolveJadxConfiguration,
|
||||
resolveJadxConfigurationInputs,
|
||||
validateJadxConfiguration,
|
||||
} from "./JadxConfiguration.js";
|
||||
import type { JadxLauncher } from "./JadxMcpTransport.js";
|
||||
|
||||
@@ -104,6 +106,50 @@ const executionError = (context: {
|
||||
});
|
||||
};
|
||||
|
||||
const androidTargetChanged = (
|
||||
operation: string,
|
||||
path: string,
|
||||
cause?: unknown,
|
||||
): AnalysisInputError =>
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
cause === undefined ? undefined : { cause },
|
||||
[
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
const androidSourceChanged = (
|
||||
operation: string,
|
||||
source: string,
|
||||
cause?: unknown,
|
||||
): AnalysisCapabilityUnavailableError =>
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
`Selected JADX input bytes changed during admission at ${source}; retry with a stable installation.`,
|
||||
cause === undefined ? undefined : { cause },
|
||||
);
|
||||
|
||||
const hashAndroidSource = async (
|
||||
path: string,
|
||||
operation: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<string> => {
|
||||
try {
|
||||
return (await hashStableFile(path, signal)).sha256;
|
||||
} catch (cause: unknown) {
|
||||
if (signal.aborted) throw cause;
|
||||
if (cause instanceof RegularFileChangedError)
|
||||
throw androidSourceChanged(operation, path, cause);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
interface PendingCleanup {
|
||||
session: JadxSession | undefined;
|
||||
root: PrivateRuntimeRoot | undefined;
|
||||
@@ -147,17 +193,18 @@ const cleanup = async (
|
||||
await root?.close();
|
||||
resources.root = undefined;
|
||||
} catch (cause) {
|
||||
return err(
|
||||
new ProviderCleanupError(
|
||||
"jadx",
|
||||
[root?.path ?? "unknown"],
|
||||
{
|
||||
...diagnostics,
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause: previousError ?? cause },
|
||||
),
|
||||
const error = new ProviderCleanupError(
|
||||
"jadx",
|
||||
[root?.path ?? "unknown"],
|
||||
{
|
||||
...diagnostics,
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause: previousError ?? cause },
|
||||
);
|
||||
if (previousError?.partialObservation !== undefined)
|
||||
error.retainPartialObservation(previousError.partialObservation);
|
||||
return err(error);
|
||||
}
|
||||
return ok(undefined);
|
||||
};
|
||||
@@ -287,23 +334,33 @@ export class JadxProvider implements AndroidAnalysisPort {
|
||||
request.operation,
|
||||
`Target ${target.path} is ${target.format}; provide one standalone APK.`,
|
||||
);
|
||||
const configuration = await resolveJadxConfiguration(
|
||||
const configuration = await resolveJadxConfigurationInputs(
|
||||
this.environment,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
signal.throwIfAborted();
|
||||
const jarHash = await hashAndroidFile(configuration.jar);
|
||||
const bridgeHash = await hashAndroidFile(BRIDGE_SOURCE);
|
||||
if ((await hashAndroidFile(target.path)) !== target.sha256)
|
||||
throw new AnalysisInputError(request.operation, undefined, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"APK bytes changed after admission; retry against a stable file.",
|
||||
},
|
||||
]);
|
||||
const jarHash = await hashAndroidSource(
|
||||
configuration.jar,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const bridgeHash = await hashAndroidSource(
|
||||
BRIDGE_SOURCE,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
let targetHash: string;
|
||||
try {
|
||||
targetHash = (await hashStableFile(target.path, signal)).sha256;
|
||||
} catch (cause: unknown) {
|
||||
if (signal.aborted) throw cause;
|
||||
if (cause instanceof RegularFileChangedError)
|
||||
throw androidTargetChanged(request.operation, target.path, cause);
|
||||
throw cause;
|
||||
}
|
||||
if (targetHash !== target.sha256)
|
||||
throw androidTargetChanged(request.operation, target.path);
|
||||
const key = JSON.stringify({
|
||||
path: target.path,
|
||||
sha256: target.sha256,
|
||||
@@ -326,27 +383,40 @@ export class JadxProvider implements AndroidAnalysisPort {
|
||||
}
|
||||
let retained = this.#retained;
|
||||
if (retained === undefined) {
|
||||
await validateJadxConfiguration(
|
||||
configuration,
|
||||
this.environment,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" });
|
||||
const engine = await snapshotAndroidEngine(
|
||||
configuration.jar,
|
||||
jarHash,
|
||||
root.path,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const snapshot = await snapshotAndroidTarget(
|
||||
target.path,
|
||||
target.sha256,
|
||||
root.path,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const bridge = join(root.path, "ReaJadxBridge.java");
|
||||
await copyFile(BRIDGE_SOURCE, bridge);
|
||||
await chmod(bridge, 0o400);
|
||||
if ((await hashAndroidFile(bridge)) !== bridgeHash)
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
const copiedBridgeHash = await hashAndroidSource(
|
||||
bridge,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
if (copiedBridgeHash !== bridgeHash)
|
||||
throw androidSourceChanged(
|
||||
request.operation,
|
||||
"REA Android bridge bytes changed during admission; retry with a stable installation.",
|
||||
BRIDGE_SOURCE,
|
||||
undefined,
|
||||
);
|
||||
signal.throwIfAborted();
|
||||
session = new JadxSession(
|
||||
|
||||
@@ -78,12 +78,14 @@ export const parseApktoolVersionOutput = (
|
||||
export const inspectApktoolClient = async (
|
||||
selection: ApktoolCommandSelection,
|
||||
signal?: AbortSignal,
|
||||
environment: Readonly<Record<string, string | undefined>> = process.env,
|
||||
): Promise<ApktoolClientIdentity> => {
|
||||
try {
|
||||
const { stdout } = await execFileOutput(selection.command, ["--version"], {
|
||||
timeout: VERSION_TIMEOUT_MS,
|
||||
stopSignal: "SIGTERM",
|
||||
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
|
||||
env: { ...process.env, ...environment },
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
return parseApktoolVersionOutput(stdout);
|
||||
|
||||
+327
-133
@@ -1,18 +1,20 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { createReadStream, constants as fsConstants } from "node:fs";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { createWriteStream, constants as fsConstants } from "node:fs";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
opendir,
|
||||
open,
|
||||
readdir,
|
||||
rm,
|
||||
stat,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { Transform, type TransformCallback } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
import type {
|
||||
ExecutionOptions,
|
||||
@@ -33,11 +35,19 @@ import {
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
execFileOutput,
|
||||
execFileOutputFailure,
|
||||
} from "../process/ExecFileOutput.js";
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../process/OwnedCommand.js";
|
||||
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
|
||||
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
|
||||
import {
|
||||
ApktoolConfigurationFailure,
|
||||
apktoolConfigurationAvailability,
|
||||
@@ -64,20 +74,22 @@ const DECODE_MAX_OUTPUT_BYTES = 4 * 1024 * 1024;
|
||||
const MANIFEST_MAX_BYTES = 512 * 1024;
|
||||
const STRINGS_XML_MAX_BYTES = 8 * 1024 * 1024;
|
||||
const MAX_STRINGS = 2_000;
|
||||
const MAX_DECODED_FILES_COUNTED = 100_000;
|
||||
const MAX_DECODED_ENTRIES_VISITED = 100_000;
|
||||
const STDERR_EXCERPT_BYTES = 512;
|
||||
|
||||
/** Stream a local file digest without retaining large APKs in memory. */
|
||||
const sha256File = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
const throwIfCancelled = (
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): void => {
|
||||
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
|
||||
};
|
||||
|
||||
/** Read at most the byte budget plus one byte, before decoding UTF-8. */
|
||||
const readBoundedText = async (
|
||||
path: string,
|
||||
budget: number,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<{ text: string | null; truncated: boolean }> => {
|
||||
const file = await open(path, "r").catch((cause: unknown) => {
|
||||
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return null;
|
||||
@@ -85,9 +97,11 @@ const readBoundedText = async (
|
||||
});
|
||||
if (file === null) return { text: null, truncated: false };
|
||||
try {
|
||||
throwIfCancelled(operation, signal);
|
||||
const bytes = Buffer.alloc(budget + 1);
|
||||
let length = 0;
|
||||
while (length < bytes.length) {
|
||||
throwIfCancelled(operation, signal);
|
||||
const result = await file.read(
|
||||
bytes,
|
||||
length,
|
||||
@@ -97,6 +111,7 @@ const readBoundedText = async (
|
||||
if (result.bytesRead === 0) break;
|
||||
length += result.bytesRead;
|
||||
}
|
||||
throwIfCancelled(operation, signal);
|
||||
const truncated = length > budget;
|
||||
// A prefix may end within a UTF-8 character; omit those trailing bytes.
|
||||
const decoder = new TextDecoder("utf-8");
|
||||
@@ -125,97 +140,141 @@ const runApktool = async (
|
||||
arguments_: readonly string[],
|
||||
limits: DecodeLimits,
|
||||
signal: AbortSignal | undefined,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
onCleanupOwner: (owner: ProviderProcessSupervisor) => void,
|
||||
): Promise<Result<{ stdout: string; stderr: string }, AnalysisError>> => {
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
try {
|
||||
return ok(
|
||||
await execFileOutput(command, arguments_, {
|
||||
timeout: limits.timeoutMs,
|
||||
maxBuffer: limits.maxOutputBytes,
|
||||
stopSignal: "SIGTERM",
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
}),
|
||||
const output = await runOwnedCommand(
|
||||
{
|
||||
command,
|
||||
arguments: arguments_,
|
||||
runId: randomUUID(),
|
||||
hostEnvironment: { ...process.env, ...environment },
|
||||
},
|
||||
{
|
||||
timeoutMs: limits.timeoutMs,
|
||||
diagnosticBytes: limits.maxOutputBytes,
|
||||
},
|
||||
signal === undefined ? {} : { signal },
|
||||
);
|
||||
return ok({ stdout: output.stdout.text, stderr: output.stderr.text });
|
||||
} catch (cause) {
|
||||
if (signal?.aborted || execFileOutputFailure(cause)?.code === "ABORT_ERR")
|
||||
return err(new AnalysisCancelledError(operation, { cause }));
|
||||
const failure = execFileOutputFailure(cause);
|
||||
if (failure === undefined)
|
||||
if (!(cause instanceof OwnedCommandFailure))
|
||||
return err(
|
||||
new AnalysisProtocolError(`apktool failed for ${operation}`, { cause }),
|
||||
);
|
||||
if (failure.outputTruncated)
|
||||
if (cause.cleanupOwner !== undefined) onCleanupOwner(cause.cleanupOwner);
|
||||
const output = cause.snapshot;
|
||||
const capturedOutput =
|
||||
output === null
|
||||
? undefined
|
||||
: {
|
||||
stdout: output.stdout.text,
|
||||
stderr: output.stderr.text,
|
||||
truncated: output.diagnosticTruncated === true,
|
||||
stdout_bytes: output.stdout.bytes,
|
||||
stderr_bytes: output.stderr.bytes,
|
||||
exit_code: output.exitCode ?? null,
|
||||
signal: output.signal ?? null,
|
||||
};
|
||||
const cleanup =
|
||||
cause.cleanupFailure === null
|
||||
? undefined
|
||||
: {
|
||||
reason: cause.cleanupFailure,
|
||||
resources: cause.resources,
|
||||
};
|
||||
if (cause.reason === "cancelled")
|
||||
return err(
|
||||
new AnalysisCancelledError(operation, {
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
}),
|
||||
);
|
||||
if (cause.reason === "timeout")
|
||||
return err(
|
||||
new AnalysisTimeoutError(operation, limits.timeoutMs, {
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
}),
|
||||
);
|
||||
if (cause.reason === "output-limit")
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
`apktool output exceeded the ${String(limits.maxOutputBytes)}-byte capture budget`,
|
||||
{ max_output_bytes: limits.maxOutputBytes },
|
||||
{ cause },
|
||||
{
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
},
|
||||
),
|
||||
);
|
||||
const excerpt = failure.stderr.trim().slice(0, STDERR_EXCERPT_BYTES);
|
||||
if (failure.killed)
|
||||
return err(
|
||||
new AnalysisTimeoutError(operation, limits.timeoutMs, { cause }),
|
||||
);
|
||||
return err(
|
||||
new AnalysisUnsupportedTargetError(
|
||||
operation,
|
||||
command,
|
||||
excerpt === ""
|
||||
? `apktool exited with code ${String(failure.code)} for ${operation}`
|
||||
: excerpt,
|
||||
{ cause },
|
||||
output?.stderr.text.trim().slice(0, STDERR_EXCERPT_BYTES) ||
|
||||
`apktool exited with code ${String(output?.exitCode ?? "unknown")} for ${operation}`,
|
||||
{
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** Count files in the decoded workspace with a hard ceiling. */
|
||||
/** Count files while bounding streamed directory-entry traversal. */
|
||||
const countDecodedFiles = async (
|
||||
root: string,
|
||||
): Promise<{ count: number; capped: boolean }> => {
|
||||
signal: AbortSignal | undefined,
|
||||
operation: string,
|
||||
): Promise<{
|
||||
count: number;
|
||||
capped: boolean;
|
||||
locales: readonly string[];
|
||||
}> => {
|
||||
let count = 0;
|
||||
let capped = false;
|
||||
const walk = async (directory: string): Promise<void> => {
|
||||
const entries = await readdir(directory, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
if (count >= MAX_DECODED_FILES_COUNTED) {
|
||||
capped = true;
|
||||
return;
|
||||
}
|
||||
const path = join(directory, entry.name);
|
||||
if (entry.isDirectory()) await walk(path);
|
||||
else if (entry.isFile()) count += 1;
|
||||
}
|
||||
};
|
||||
await walk(root);
|
||||
return { count, capped };
|
||||
};
|
||||
|
||||
/** Locale codes present as res/values-<locale>/strings.xml. */
|
||||
const listStringLocales = async (
|
||||
workspace: string,
|
||||
): Promise<readonly string[]> => {
|
||||
const res = join(workspace, "res");
|
||||
const entries = await readdir(res, { withFileTypes: true }).catch(() => []);
|
||||
let visited = 0;
|
||||
const directories = [root];
|
||||
const locales: string[] = [];
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const match = /^values-(.+)$/u.exec(entry.name);
|
||||
if (match === null) continue;
|
||||
const strings = join(res, entry.name, "strings.xml");
|
||||
if (
|
||||
await access(strings).then(
|
||||
() => true,
|
||||
() => false,
|
||||
)
|
||||
)
|
||||
locales.push(match[1]!);
|
||||
while (directories.length > 0 && !capped) {
|
||||
throwIfCancelled(operation, signal);
|
||||
const directory = directories.pop()!;
|
||||
for await (const entry of await opendir(directory)) {
|
||||
throwIfCancelled(operation, signal);
|
||||
if (visited >= MAX_DECODED_ENTRIES_VISITED) {
|
||||
capped = true;
|
||||
break;
|
||||
}
|
||||
visited += 1;
|
||||
const path = join(directory, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
directories.push(path);
|
||||
if (directory === join(root, "res")) {
|
||||
const match = /^values-(.+)$/u.exec(entry.name);
|
||||
if (
|
||||
match !== null &&
|
||||
(await access(join(path, "strings.xml")).then(
|
||||
() => true,
|
||||
() => false,
|
||||
))
|
||||
)
|
||||
locales.push(match[1]!);
|
||||
}
|
||||
} else if (entry.isFile()) count += 1;
|
||||
}
|
||||
}
|
||||
return locales.sort();
|
||||
return { count, capped, locales: locales.sort() };
|
||||
};
|
||||
|
||||
export interface ApktoolProviderOptions {
|
||||
@@ -235,9 +294,14 @@ export class ApktoolProvider {
|
||||
|
||||
private readonly workspaces = new Set<string>();
|
||||
private readonly removeWorkspace: (path: string) => Promise<void>;
|
||||
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
|
||||
readonly #shutdown = new AbortController();
|
||||
readonly #active = new Set<Promise<void>>();
|
||||
#closing = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
|
||||
constructor(options: ApktoolProviderOptions = {}) {
|
||||
this.environment = options.environment ?? process.env;
|
||||
this.environment = snapshotEnvironment(options.environment ?? process.env);
|
||||
this.removeWorkspace =
|
||||
options.removeWorkspace ??
|
||||
((path) => rm(path, { recursive: true, force: true }));
|
||||
@@ -248,12 +312,25 @@ export class ApktoolProvider {
|
||||
}
|
||||
|
||||
/** Probe the selected apktool launcher; decodes nothing. */
|
||||
async inspectAvailability(
|
||||
signal?: AbortSignal,
|
||||
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability> {
|
||||
if (this.#closing)
|
||||
return Promise.reject(
|
||||
new AnalysisCancelledError("inspect_apktool_client"),
|
||||
);
|
||||
const operationSignal =
|
||||
signal === undefined
|
||||
? this.#shutdown.signal
|
||||
: AbortSignal.any([signal, this.#shutdown.signal]);
|
||||
return this.#track(this.#inspectAvailability(operationSignal));
|
||||
}
|
||||
|
||||
async #inspectAvailability(
|
||||
signal: AbortSignal,
|
||||
): Promise<ProviderAvailability> {
|
||||
const selection = this.selection();
|
||||
try {
|
||||
await inspectApktoolClient(selection, signal);
|
||||
await inspectApktoolClient(selection, signal, this.environment);
|
||||
throwIfCancelled("inspect_apktool_client", signal);
|
||||
return {
|
||||
status: "available",
|
||||
code: null,
|
||||
@@ -272,30 +349,44 @@ export class ApktoolProvider {
|
||||
|
||||
/** Retain failed cleanup ownership and retry it on close. */
|
||||
async close(): Promise<void> {
|
||||
const failures: unknown[] = [];
|
||||
for (const path of this.workspaces) {
|
||||
try {
|
||||
await this.removeWorkspace(path);
|
||||
this.workspaces.delete(path);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool workspace cleanup failed");
|
||||
this.#closing = true;
|
||||
this.#shutdown.abort();
|
||||
this.#closePromise ??= this.#drainAndClean().catch((cause: unknown) => {
|
||||
this.#closePromise = undefined;
|
||||
throw cause;
|
||||
});
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
async execute(
|
||||
execute(
|
||||
request: ApktoolRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const signal = options?.signal;
|
||||
if (this.#closing)
|
||||
return Promise.resolve(
|
||||
err(new AnalysisCancelledError(request.operation)),
|
||||
);
|
||||
const signal =
|
||||
options?.signal === undefined
|
||||
? this.#shutdown.signal
|
||||
: AbortSignal.any([options.signal, this.#shutdown.signal]);
|
||||
return this.#track(this.#execute(request, signal));
|
||||
}
|
||||
|
||||
async #execute(
|
||||
request: ApktoolRequest,
|
||||
signal: AbortSignal,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
const selection = this.selection();
|
||||
let identity: ApktoolClientIdentity;
|
||||
try {
|
||||
identity = await inspectApktoolClient(selection, signal);
|
||||
identity = await inspectApktoolClient(
|
||||
selection,
|
||||
signal,
|
||||
this.environment,
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
if (cause instanceof ApktoolConfigurationFailure)
|
||||
@@ -316,6 +407,8 @@ export class ApktoolProvider {
|
||||
),
|
||||
);
|
||||
}
|
||||
if (signal.aborted)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
const client = {
|
||||
command: selection.command,
|
||||
command_source: selection.commandSource,
|
||||
@@ -337,6 +430,47 @@ export class ApktoolProvider {
|
||||
);
|
||||
}
|
||||
|
||||
#track<T>(operation: Promise<T>): Promise<T> {
|
||||
const settled = operation.then(
|
||||
() => undefined,
|
||||
() => undefined,
|
||||
);
|
||||
this.#active.add(settled);
|
||||
void settled.then(() => this.#active.delete(settled));
|
||||
return operation;
|
||||
}
|
||||
|
||||
async #drainAndClean(): Promise<void> {
|
||||
await Promise.all(this.#active);
|
||||
const failures: unknown[] = [];
|
||||
for (const [owner, root] of this.#pendingCommandOwners) {
|
||||
try {
|
||||
const stopped = await owner.stop();
|
||||
if (stopped.status === "incomplete") throw new Error(stopped.reason);
|
||||
this.#pendingCommandOwners.delete(owner);
|
||||
} catch (cause) {
|
||||
failures.push(
|
||||
new Error(
|
||||
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool process cleanup failed");
|
||||
for (const path of this.workspaces) {
|
||||
try {
|
||||
await this.removeWorkspace(path);
|
||||
this.workspaces.delete(path);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool workspace cleanup failed");
|
||||
}
|
||||
|
||||
private async decodeResources(
|
||||
operation: "decode_android_resources",
|
||||
selection: ApktoolCommandSelection,
|
||||
@@ -353,28 +487,6 @@ export class ApktoolProvider {
|
||||
},
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const info = await stat(input.path).then(
|
||||
(value) => value,
|
||||
() => null,
|
||||
);
|
||||
if (info === null || !info.isFile())
|
||||
return err(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK is not a readable file: ${input.path}`,
|
||||
),
|
||||
}),
|
||||
);
|
||||
const readable = await access(input.path, fsConstants.R_OK).then(
|
||||
() => true,
|
||||
() => false,
|
||||
);
|
||||
if (!readable)
|
||||
return err(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(`The selected APK is not readable: ${input.path}`),
|
||||
}),
|
||||
);
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-apktool-"));
|
||||
this.workspaces.add(root);
|
||||
const workspace = join(root, "decoded");
|
||||
@@ -384,6 +496,10 @@ export class ApktoolProvider {
|
||||
// The workspace is removed before the result is returned, so an
|
||||
// unremovable workspace is reported instead of silently abandoned.
|
||||
const removeWorkspace = async (): Promise<void> => {
|
||||
if ([...this.#pendingCommandOwners.values()].includes(root)) {
|
||||
cleanupFailure = `The decode workspace is retained until its apktool process stops: ${root}`;
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await this.removeWorkspace(root);
|
||||
this.workspaces.delete(root);
|
||||
@@ -394,20 +510,78 @@ export class ApktoolProvider {
|
||||
const failureWithCleanup = (failure: AnalysisError): AnalysisError =>
|
||||
cleanupFailure === null
|
||||
? failure
|
||||
: failure instanceof AnalysisCancelledError
|
||||
? new AnalysisCancelledError(operation, {
|
||||
cause: failure,
|
||||
cleanup: { reason: cleanupFailure, resources: [root] },
|
||||
})
|
||||
: new AnalysisProtocolError(failure.message, {
|
||||
cause: failure,
|
||||
cleanup: { reason: cleanupFailure, resources: [root] },
|
||||
});
|
||||
: analysisErrorWithCleanupFailure(
|
||||
failure,
|
||||
new ProviderCleanupError(
|
||||
APKTOOL_PROVIDER_IDENTITY.id,
|
||||
[root],
|
||||
{ reason: cleanupFailure },
|
||||
{ operation },
|
||||
),
|
||||
operation,
|
||||
);
|
||||
let source: FileHandle | undefined;
|
||||
let sourceAdmissionFailed = false;
|
||||
try {
|
||||
await copyFile(input.path, snapshot, fsConstants.COPYFILE_EXCL);
|
||||
throwIfCancelled(operation, signal);
|
||||
sourceAdmissionFailed = true;
|
||||
source = await openRegularFile(input.path, {
|
||||
symlinks: "follow",
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
const sourceInfo = await source.stat();
|
||||
sourceAdmissionFailed = false;
|
||||
const sourceSize = sourceInfo.size;
|
||||
let copiedBytes = 0;
|
||||
const sourceHash = createHash("sha256");
|
||||
const boundedSource = new Transform({
|
||||
transform(
|
||||
chunk: Buffer,
|
||||
_encoding: BufferEncoding,
|
||||
callback: TransformCallback,
|
||||
) {
|
||||
copiedBytes += chunk.length;
|
||||
if (copiedBytes > sourceSize) {
|
||||
callback(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK changed while being copied: ${input.path}`,
|
||||
),
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
sourceHash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
await pipeline(
|
||||
source.createReadStream({
|
||||
start: 0,
|
||||
end: sourceSize,
|
||||
autoClose: false,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
}),
|
||||
boundedSource,
|
||||
createWriteStream(snapshot, { flags: "wx", mode: 0o600 }),
|
||||
signal === undefined ? {} : { signal },
|
||||
);
|
||||
const finalSourceInfo = await source.stat();
|
||||
if (
|
||||
copiedBytes !== sourceSize ||
|
||||
!sameRegularFileState(sourceInfo, finalSourceInfo)
|
||||
)
|
||||
throw new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK changed while being copied: ${input.path}`,
|
||||
),
|
||||
});
|
||||
const sha256 = sourceHash.digest("hex");
|
||||
await source.close();
|
||||
source = undefined;
|
||||
await chmod(snapshot, 0o400);
|
||||
throwIfCancelled(operation, signal);
|
||||
const snapshotInfo = await stat(snapshot);
|
||||
const sha256 = await sha256File(snapshot);
|
||||
await mkdir(workspace);
|
||||
await mkdir(framework);
|
||||
const decoded = await runApktool(
|
||||
@@ -428,14 +602,19 @@ export class ApktoolProvider {
|
||||
maxOutputBytes: DECODE_MAX_OUTPUT_BYTES,
|
||||
},
|
||||
signal,
|
||||
this.environment,
|
||||
(owner) => this.#pendingCommandOwners.set(owner, root),
|
||||
);
|
||||
if (!decoded.ok) {
|
||||
await removeWorkspace();
|
||||
return err(failureWithCleanup(decoded.error));
|
||||
}
|
||||
throwIfCancelled(operation, signal);
|
||||
const yml = await readBoundedText(
|
||||
join(workspace, "apktool.yml"),
|
||||
MANIFEST_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
if (yml.text === null || yml.truncated) {
|
||||
throw new AnalysisProtocolError(
|
||||
@@ -446,11 +625,12 @@ export class ApktoolProvider {
|
||||
const manifestRead = await readBoundedText(
|
||||
join(workspace, "AndroidManifest.xml"),
|
||||
MANIFEST_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
const manifest = manifestRead.text;
|
||||
const packageName =
|
||||
manifest === null ? null : parseManifestPackage(manifest);
|
||||
const locales = await listStringLocales(workspace);
|
||||
const stringsDirectory =
|
||||
input.locale === undefined
|
||||
? "values"
|
||||
@@ -459,6 +639,8 @@ export class ApktoolProvider {
|
||||
? await readBoundedText(
|
||||
join(workspace, "res", stringsDirectory, "strings.xml"),
|
||||
STRINGS_XML_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
)
|
||||
: { text: null, truncated: false };
|
||||
// Do not parse an incomplete XML document as though its projection were complete.
|
||||
@@ -468,7 +650,8 @@ export class ApktoolProvider {
|
||||
? { entries: [], unparsedCount: 0 }
|
||||
: parseDecodedStrings(stringsXml);
|
||||
const bounded = parsedStrings.entries.slice(0, MAX_STRINGS);
|
||||
const files = await countDecodedFiles(workspace);
|
||||
const files = await countDecodedFiles(workspace, signal, operation);
|
||||
throwIfCancelled(operation, signal);
|
||||
const coverage =
|
||||
(stringsXml !== null && parsedStrings.entries.length > MAX_STRINGS) ||
|
||||
manifestRead.truncated ||
|
||||
@@ -493,7 +676,7 @@ export class ApktoolProvider {
|
||||
manifest,
|
||||
strings: bounded,
|
||||
locale: input.locale ?? null,
|
||||
locales,
|
||||
locales: files.locales,
|
||||
decoded_file_count: files.count,
|
||||
coverage,
|
||||
},
|
||||
@@ -535,7 +718,7 @@ export class ApktoolProvider {
|
||||
: []),
|
||||
...(files.capped
|
||||
? [
|
||||
`decoded_file_count is capped at ${String(MAX_DECODED_FILES_COUNTED)}`,
|
||||
`Decoded workspace enumeration stopped after ${String(MAX_DECODED_ENTRIES_VISITED)} entries; decoded_file_count and locales may be partial`,
|
||||
]
|
||||
: []),
|
||||
...(cleanupFailure === null ? [] : [cleanupFailure]),
|
||||
@@ -547,14 +730,25 @@ export class ApktoolProvider {
|
||||
await removeWorkspace();
|
||||
return err(
|
||||
failureWithCleanup(
|
||||
cause instanceof AnalysisCancelledError ||
|
||||
cause instanceof AnalysisProtocolError
|
||||
? cause
|
||||
: new AnalysisProtocolError("Apktool resource projection failed", {
|
||||
cause,
|
||||
}),
|
||||
signal?.aborted === true
|
||||
? new AnalysisCancelledError(operation, { cause })
|
||||
: sourceAdmissionFailed
|
||||
? new AnalysisInputError(operation, { cause })
|
||||
: cause instanceof AnalysisInputError
|
||||
? cause
|
||||
: cause instanceof AnalysisCancelledError ||
|
||||
cause instanceof AnalysisProtocolError
|
||||
? cause
|
||||
: new AnalysisProtocolError(
|
||||
"Apktool resource projection failed",
|
||||
{
|
||||
cause,
|
||||
},
|
||||
),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
await source?.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { lstat, readdir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -156,13 +156,14 @@ const wrappedBundle = async (bundle: string): Promise<string | undefined> => {
|
||||
if (isAbsence(cause)) return undefined;
|
||||
throw cause;
|
||||
}
|
||||
const apps = (await readdir(wrapper, { withFileTypes: true })).filter(
|
||||
(entry) => entry.isDirectory() && entry.name.toLowerCase().endsWith(".app"),
|
||||
);
|
||||
const [app] = apps;
|
||||
return apps.length === 1 && app !== undefined
|
||||
? join(wrapper, app.name)
|
||||
: undefined;
|
||||
let app: string | undefined;
|
||||
for await (const entry of await opendir(wrapper)) {
|
||||
if (!entry.isDirectory() || !entry.name.toLowerCase().endsWith(".app"))
|
||||
continue;
|
||||
if (app !== undefined) return undefined;
|
||||
app = join(wrapper, entry.name);
|
||||
}
|
||||
return app;
|
||||
};
|
||||
|
||||
const resolveLayoutExecutable = async (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { constants } from "node:fs";
|
||||
import { constants, type Stats } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
access,
|
||||
@@ -248,7 +248,7 @@ const isArchiveFormat = (
|
||||
const sha256Handle = async (
|
||||
handle: FileHandle,
|
||||
path: string,
|
||||
initial: Awaited<ReturnType<FileHandle["stat"]>>,
|
||||
initial: Stats,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
@@ -256,11 +256,18 @@ const sha256Handle = async (
|
||||
let position = 0;
|
||||
while (true) {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const observed = await handle.read(chunk, 0, chunk.length, position);
|
||||
const observed = await handle.read(
|
||||
chunk,
|
||||
0,
|
||||
Math.min(chunk.length, initial.size - position + 1),
|
||||
position,
|
||||
);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (observed.bytesRead === 0) break;
|
||||
hash.update(chunk.subarray(0, observed.bytesRead));
|
||||
position += observed.bytesRead;
|
||||
if (position > initial.size)
|
||||
throw new BinaryTargetError(path, "target grew while being read");
|
||||
}
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const [opened, currentPath] = await Promise.all([handle.stat(), lstat(path)]);
|
||||
|
||||
@@ -6,6 +6,7 @@ import type { Result } from "../domain/result.js";
|
||||
|
||||
/** Provider-neutral application boundary for controlled browser scenarios. */
|
||||
export interface BrowserScenarioCapturePort {
|
||||
close?(): Promise<void>;
|
||||
identity(): ProviderIdentity;
|
||||
captureScenario(
|
||||
scenario: BrowserScenario,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import {
|
||||
effectiveClientServer,
|
||||
parseClientConfiguration,
|
||||
@@ -19,7 +19,7 @@ export const readConfigurationText = async (
|
||||
path: string,
|
||||
): Promise<Result<string | undefined, ClientConfigurationFileError>> => {
|
||||
try {
|
||||
return ok(await readFile(path, "utf8"));
|
||||
return ok(await readRegularFileText(path));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return ok(undefined);
|
||||
|
||||
@@ -6,7 +6,7 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
import type { AnalysisExecution } from "./AnalysisProvider.js";
|
||||
import type { BinarySession } from "./binary/BinarySession.js";
|
||||
import { analysisErrorWithCleanupFailure } from "./binary/AnalysisClientCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
|
||||
/** Await one-shot cleanup and retain either the outcome or both failures. */
|
||||
export const withSessionCleanup = async <Value>(
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
/** Maximum admitted vendor response bytes, shared by Hopper setup adapters. */
|
||||
export const HOPPER_DOWNLOAD_BYTES = 100_000_000;
|
||||
|
||||
/** Bound observed response bytes before retaining them, including chunked bodies. */
|
||||
export const downloadHopperPackage = async (
|
||||
url: string,
|
||||
options: { readonly signal?: AbortSignal } = {},
|
||||
): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }> => {
|
||||
const deadline = AbortSignal.timeout(120_000);
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
signal:
|
||||
options.signal === undefined
|
||||
? deadline
|
||||
: AbortSignal.any([options.signal, deadline]),
|
||||
});
|
||||
const refused = { ok: false, bytes: new Uint8Array() };
|
||||
if (
|
||||
!response.ok ||
|
||||
Number(response.headers.get("content-length")) > HOPPER_DOWNLOAD_BYTES
|
||||
) {
|
||||
await response.body?.cancel();
|
||||
return refused;
|
||||
}
|
||||
if (response.body === null) return { ok: true, bytes: new Uint8Array() };
|
||||
const reader = response.body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let bytes = 0;
|
||||
try {
|
||||
while (true) {
|
||||
const chunk = await reader.read();
|
||||
if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) };
|
||||
if (chunk.value.byteLength > HOPPER_DOWNLOAD_BYTES - bytes) {
|
||||
await reader.cancel();
|
||||
return refused;
|
||||
}
|
||||
chunks.push(chunk.value);
|
||||
bytes += chunk.value.byteLength;
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
};
|
||||
@@ -16,8 +16,7 @@ import {
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { parseUtf8Json } from "./Utf8JsonInput.js";
|
||||
import { readRegularFile } from "./RegularFileRead.js";
|
||||
import { readJsonInputFile } from "./JsonInputFile.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
|
||||
/** Request control and its owning operation for an interruptible atomic write. */
|
||||
@@ -37,12 +36,20 @@ export const readJsonFile = async (
|
||||
> => {
|
||||
const requestedPath = resolve(path);
|
||||
try {
|
||||
const encoded = await readRegularFile(requestedPath);
|
||||
const decoded = parseUtf8Json(encoded, "read_evidence_file", requestedPath);
|
||||
const decoded = await readJsonInputFile(
|
||||
requestedPath,
|
||||
"read_evidence_file",
|
||||
);
|
||||
if (!decoded.ok) {
|
||||
if (decoded.error instanceof AnalysisResourceConstraintError)
|
||||
return err(decoded.error);
|
||||
return err(
|
||||
new EvidenceFileError("read", "invalid-json", {
|
||||
cause: decoded.cause,
|
||||
cause:
|
||||
decoded.error.cause ??
|
||||
new SyntaxError(
|
||||
decoded.error.issues[0]?.message ?? decoded.error.message,
|
||||
),
|
||||
path: requestedPath,
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -6,61 +6,76 @@ import { getHeapStatistics } from "node:v8";
|
||||
import parser from "stream-json/parser.js";
|
||||
import streamValues from "stream-json/streamers/stream-values.js";
|
||||
|
||||
import { withRegularFile } from "./application/RegularFileRead.js";
|
||||
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
|
||||
import { withRegularFile } from "./RegularFileRead.js";
|
||||
import {
|
||||
JSON_BYTE_ORDER_MARK_MESSAGE,
|
||||
JSON_INPUT_RESOURCE_REMEDIATION,
|
||||
parseUtf8Json,
|
||||
} from "./application/Utf8JsonInput.js";
|
||||
} from "./Utf8JsonInput.js";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "./domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "./domain/result.js";
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
type JsonFileFailure = AnalysisInputError | AnalysisResourceConstraintError;
|
||||
type JsonInputFileFailure =
|
||||
| AnalysisInputError
|
||||
| AnalysisResourceConstraintError;
|
||||
const READ_CHUNK_BYTES = 64 * 1024;
|
||||
const PACKED_STRING_PART_CODE_UNITS = 64 * 1024;
|
||||
const PACKED_TOKEN_PART_CODE_UNITS = 64 * 1024;
|
||||
// Retain native-parser speed for modest inputs while bounding its extra copies.
|
||||
// This selects an implementation; larger files remain accepted through streaming.
|
||||
const NATIVE_PARSE_READ_BUDGET = 8 * 1024 * 1024;
|
||||
|
||||
/** Keep native parsing only when input copies and value expansion fit live headroom. */
|
||||
const nativeParseFits = (inputBytes: number): boolean =>
|
||||
inputBytes <= NATIVE_PARSE_READ_BUDGET &&
|
||||
// The assembler estimate reserves up to 128 bytes per two-byte container.
|
||||
// Allow that 64x expansion plus byte/text copies before native JSON.parse,
|
||||
// whose individual container allocations cannot be intercepted.
|
||||
inputBytes * 72 + READ_CHUNK_BYTES <=
|
||||
getHeapStatistics().total_available_size;
|
||||
|
||||
/** Parse strict UTF-8 JSON without requiring a whole-document string. */
|
||||
export const readCliJsonFile = (
|
||||
export const readJsonInputFile = (
|
||||
path: string,
|
||||
operation: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<unknown, JsonFileFailure>> =>
|
||||
): Promise<Result<unknown, JsonInputFileFailure>> =>
|
||||
withRegularFile(
|
||||
path,
|
||||
async (handle, stats) => {
|
||||
let found = false;
|
||||
let value: unknown;
|
||||
try {
|
||||
const prefix =
|
||||
stats.size <= NATIVE_PARSE_READ_BUDGET
|
||||
? await readPrefix(handle, stats.size + 1, signal)
|
||||
: undefined;
|
||||
if (prefix?.complete === true) {
|
||||
const prefix = nativeParseFits(stats.size)
|
||||
? await readPrefix(handle, stats.size + 1, signal)
|
||||
: undefined;
|
||||
if (prefix !== undefined) {
|
||||
if (!prefix.complete || prefix.bytes.length !== stats.size)
|
||||
throw new RegularFileChangedError(path);
|
||||
const parsed = parseUtf8Json(
|
||||
prefix.bytes,
|
||||
operation,
|
||||
path,
|
||||
"cli-json-input",
|
||||
"json-file-input",
|
||||
);
|
||||
return parsed.ok
|
||||
? ok(parsed.value)
|
||||
: err(invalidJson(operation, parsed.error, parsed.cause));
|
||||
}
|
||||
await pipeline(
|
||||
decodedChunks(handle, signal, prefix?.bytes),
|
||||
decodedChunks(handle, stats.size, path, signal),
|
||||
parser.asStream({
|
||||
jsonStreaming: false,
|
||||
streamValues: false,
|
||||
packKeys: false,
|
||||
packStrings: false,
|
||||
packNumbers: false,
|
||||
}),
|
||||
packJsonStrings(operation, stats.size),
|
||||
packJsonScalars(operation, stats.size, path),
|
||||
admitJsonAssembly(operation, stats.size, path),
|
||||
streamValues.asStream(),
|
||||
async (rows: AsyncIterable<unknown>) => {
|
||||
for await (const row of rows) {
|
||||
@@ -107,8 +122,9 @@ export const readCliJsonFile = (
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
"An individual JSON string exceeds the runtime string limit",
|
||||
"An individual JSON string or number token exceeds the runtime string limit",
|
||||
{
|
||||
input_path: path,
|
||||
input_file_bytes: stats.size,
|
||||
max_string_code_units: constants.MAX_STRING_LENGTH,
|
||||
},
|
||||
@@ -121,12 +137,16 @@ export const readCliJsonFile = (
|
||||
throw cause;
|
||||
}
|
||||
},
|
||||
signal,
|
||||
{ signal },
|
||||
);
|
||||
|
||||
const packJsonStrings = (operation: string, inputFileBytes: number) =>
|
||||
const packJsonScalars = (
|
||||
operation: string,
|
||||
inputFileBytes: number,
|
||||
path: string,
|
||||
) =>
|
||||
async function* (tokens: AsyncIterable<unknown>): AsyncGenerator<unknown> {
|
||||
let kind: "keyValue" | "stringValue" | undefined;
|
||||
let kind: "keyValue" | "stringValue" | "numberValue" | undefined;
|
||||
let length = 0;
|
||||
let value = "";
|
||||
let parts: string[] = [];
|
||||
@@ -139,23 +159,32 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
|
||||
case "startString":
|
||||
kind = token.name === "startKey" ? "keyValue" : "stringValue";
|
||||
break;
|
||||
case "startNumber":
|
||||
kind = "numberValue";
|
||||
break;
|
||||
case "numberChunk":
|
||||
case "stringChunk":
|
||||
if (
|
||||
kind === undefined ||
|
||||
!("value" in token) ||
|
||||
typeof token.value !== "string"
|
||||
)
|
||||
throw new Error("Unexpected JSON string fragment");
|
||||
throw new Error("Unexpected JSON scalar fragment");
|
||||
length += token.value.length;
|
||||
if (length > constants.MAX_STRING_LENGTH)
|
||||
throw new RangeError("Invalid string length");
|
||||
parts.push(token.value);
|
||||
partLength += token.value.length;
|
||||
if (partLength >= PACKED_STRING_PART_CODE_UNITS) {
|
||||
if (partLength >= PACKED_TOKEN_PART_CODE_UNITS) {
|
||||
// The tokenizer emits short fragments. Coalesce them before retaining
|
||||
// a string so millions of substring/rope nodes cannot exhaust the heap
|
||||
// a scalar so millions of substring/rope nodes cannot exhaust the heap
|
||||
// before the native string-length constraint can be reported.
|
||||
requireStringAssemblyHeadroom(operation, inputFileBytes, length);
|
||||
requireStringAssemblyHeadroom(
|
||||
operation,
|
||||
inputFileBytes,
|
||||
path,
|
||||
length,
|
||||
);
|
||||
value += parts.join("");
|
||||
parts = [];
|
||||
partLength = 0;
|
||||
@@ -163,8 +192,14 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
|
||||
break;
|
||||
case "endKey":
|
||||
case "endString":
|
||||
if (kind === undefined) throw new Error("Unexpected JSON string end");
|
||||
requireStringAssemblyHeadroom(operation, inputFileBytes, length);
|
||||
case "endNumber":
|
||||
if (kind === undefined) throw new Error("Unexpected JSON scalar end");
|
||||
requireStringAssemblyHeadroom(
|
||||
operation,
|
||||
inputFileBytes,
|
||||
path,
|
||||
length,
|
||||
);
|
||||
yield { name: kind, value: value + parts.join("") };
|
||||
kind = undefined;
|
||||
length = 0;
|
||||
@@ -178,22 +213,78 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
|
||||
}
|
||||
};
|
||||
|
||||
/** Admit the retained value graph before the upstream assembler grows it. */
|
||||
const admitJsonAssembly = (
|
||||
operation: string,
|
||||
inputFileBytes: number,
|
||||
path: string,
|
||||
) =>
|
||||
async function* (tokens: AsyncIterable<unknown>): AsyncGenerator<unknown> {
|
||||
let projectedBytes = 0;
|
||||
let nextCheck = 0;
|
||||
for await (const token of tokens) {
|
||||
if (typeof token !== "object" || token === null || !("name" in token))
|
||||
throw new Error("Unexpected JSON assembly token");
|
||||
if (token.name === "startObject" || token.name === "startArray") {
|
||||
projectedBytes += 128;
|
||||
} else if (token.name === "keyValue" || token.name === "stringValue") {
|
||||
if (!("value" in token) || typeof token.value !== "string")
|
||||
throw new Error("Unexpected packed JSON string");
|
||||
projectedBytes += 64 + token.value.length * 2;
|
||||
} else if (
|
||||
token.name === "numberValue" ||
|
||||
token.name === "nullValue" ||
|
||||
token.name === "trueValue" ||
|
||||
token.name === "falseValue"
|
||||
) {
|
||||
projectedBytes += 64;
|
||||
}
|
||||
if (projectedBytes >= nextCheck) {
|
||||
const heap = getHeapStatistics();
|
||||
// Conservative capacity for containers, property/array slots and UTF-16
|
||||
// values. Keep another whole projection available for container growth,
|
||||
// plus the next chunk of estimates between samples. This is admission
|
||||
// against live runtime headroom, not a fixed document-size policy.
|
||||
const requiredBytes = projectedBytes + READ_CHUNK_BYTES;
|
||||
if (requiredBytes > heap.total_available_size)
|
||||
throw new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
"Insufficient heap headroom to materialize the complete JSON value",
|
||||
{
|
||||
input_path: path,
|
||||
input_file_bytes: inputFileBytes,
|
||||
projected_value_bytes: projectedBytes,
|
||||
assembly_headroom_bytes: requiredBytes,
|
||||
available_heap_bytes: heap.total_available_size,
|
||||
heap_size_limit_bytes: heap.heap_size_limit,
|
||||
},
|
||||
{ remediationAction: JSON_INPUT_RESOURCE_REMEDIATION },
|
||||
);
|
||||
nextCheck = projectedBytes + READ_CHUNK_BYTES;
|
||||
}
|
||||
yield token;
|
||||
}
|
||||
};
|
||||
|
||||
const requireStringAssemblyHeadroom = (
|
||||
operation: string,
|
||||
inputFileBytes: number,
|
||||
path: string,
|
||||
stringCodeUnits: number,
|
||||
): void => {
|
||||
if (stringCodeUnits < PACKED_STRING_PART_CODE_UNITS) return;
|
||||
if (stringCodeUnits < PACKED_TOKEN_PART_CODE_UNITS) return;
|
||||
const heap = getHeapStatistics();
|
||||
// Concatenation or key interning may flatten the retained rope. Reserve its
|
||||
// UTF-16 representation and the next coalesced part before that allocation.
|
||||
const requiredBytes = 2 * (stringCodeUnits + PACKED_STRING_PART_CODE_UNITS);
|
||||
const requiredBytes = 2 * (stringCodeUnits + PACKED_TOKEN_PART_CODE_UNITS);
|
||||
if (requiredBytes > heap.total_available_size)
|
||||
throw new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
"Insufficient heap headroom to assemble a complete JSON string",
|
||||
{
|
||||
input_path: path,
|
||||
input_file_bytes: inputFileBytes,
|
||||
string_code_units: stringCodeUnits,
|
||||
string_assembly_headroom_bytes: requiredBytes,
|
||||
@@ -208,8 +299,9 @@ const requireStringAssemblyHeadroom = (
|
||||
|
||||
async function* decodedChunks(
|
||||
handle: FileHandle,
|
||||
admittedBytes: number,
|
||||
path: string,
|
||||
signal: AbortSignal | undefined,
|
||||
prefix?: Uint8Array,
|
||||
): AsyncGenerator<string> {
|
||||
const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
|
||||
let leading = true;
|
||||
@@ -220,25 +312,25 @@ async function* decodedChunks(
|
||||
}
|
||||
return text;
|
||||
};
|
||||
if (prefix !== undefined)
|
||||
for (let offset = 0; offset < prefix.length; offset += READ_CHUNK_BYTES) {
|
||||
signal?.throwIfAborted();
|
||||
yield checked(
|
||||
decoder.decode(prefix.subarray(offset, offset + READ_CHUNK_BYTES), {
|
||||
stream: true,
|
||||
}),
|
||||
);
|
||||
}
|
||||
const bytes = Buffer.allocUnsafe(READ_CHUNK_BYTES);
|
||||
let observedBytes = 0;
|
||||
while (true) {
|
||||
signal?.throwIfAborted();
|
||||
const read = await handle.read(bytes, 0, bytes.length, null);
|
||||
const read = await handle.read(
|
||||
bytes,
|
||||
0,
|
||||
Math.min(bytes.length, admittedBytes - observedBytes + 1),
|
||||
null,
|
||||
);
|
||||
signal?.throwIfAborted();
|
||||
if (read.bytesRead === 0) break;
|
||||
observedBytes += read.bytesRead;
|
||||
if (observedBytes > admittedBytes) throw new RegularFileChangedError(path);
|
||||
yield checked(
|
||||
decoder.decode(bytes.subarray(0, read.bytesRead), { stream: true }),
|
||||
);
|
||||
}
|
||||
if (observedBytes !== admittedBytes) throw new RegularFileChangedError(path);
|
||||
yield checked(decoder.decode());
|
||||
}
|
||||
|
||||
@@ -264,8 +356,7 @@ const readPrefix = async (
|
||||
return { bytes: bytes.subarray(0, offset), complete: true };
|
||||
offset += read.bytesRead;
|
||||
}
|
||||
// The file grew after admission. Continue from this same handle, preserving
|
||||
// the prefix already read rather than reopening or dropping its bytes.
|
||||
// One byte beyond the admitted extent proves the selected file changed.
|
||||
return { bytes, complete: false };
|
||||
};
|
||||
|
||||
@@ -7,10 +7,13 @@ import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { execFileOutput } from "../process/ExecFileOutput.js";
|
||||
import {
|
||||
downloadHopperPackage,
|
||||
HOPPER_DOWNLOAD_BYTES,
|
||||
} from "./HopperDownload.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
|
||||
const MAX_PACKAGE_BYTES = 100_000_000;
|
||||
const SUPPORTED_HOPPER_SHA256 = new Set([
|
||||
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
|
||||
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
|
||||
@@ -231,16 +234,7 @@ export const systemLinuxHopperInstallHost = (
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): LinuxHopperInstallHost => ({
|
||||
distribution: readLinuxDistribution,
|
||||
async download(url, options) {
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
return {
|
||||
ok: response.ok,
|
||||
bytes: new Uint8Array(await response.arrayBuffer()),
|
||||
};
|
||||
},
|
||||
download: downloadHopperPackage,
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
|
||||
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
installPackage: (family, archive) =>
|
||||
@@ -350,7 +344,7 @@ const packageIntegrityMatches = (
|
||||
release: LinuxHopperRelease,
|
||||
): boolean =>
|
||||
bytes.byteLength === Number(release.file_length) &&
|
||||
bytes.byteLength <= MAX_PACKAGE_BYTES &&
|
||||
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
|
||||
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
|
||||
|
||||
const packageFamilyFor = (
|
||||
|
||||
+213
-167
@@ -15,12 +15,15 @@ import { promisify } from "node:util";
|
||||
|
||||
import { z } from "zod";
|
||||
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
||||
import {
|
||||
downloadHopperPackage,
|
||||
HOPPER_DOWNLOAD_BYTES,
|
||||
} from "./HopperDownload.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const RELEASES_URL =
|
||||
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
|
||||
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
|
||||
const MAX_PACKAGE_BYTES = 100_000_000;
|
||||
const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4";
|
||||
|
||||
/** Stable network and command effects disclosed before macOS Hopper setup. */
|
||||
@@ -36,7 +39,11 @@ export const macHopperInstallDisclosure = {
|
||||
|
||||
/** Result of installing the official Hopper macOS application. */
|
||||
export type MacHopperInstallResult =
|
||||
| { readonly status: "installed"; readonly launcherPath: string }
|
||||
| {
|
||||
readonly status: "installed";
|
||||
readonly launcherPath: string;
|
||||
readonly cleanupFailure?: string;
|
||||
}
|
||||
| {
|
||||
readonly status: "failed";
|
||||
readonly reason:
|
||||
@@ -48,6 +55,7 @@ export type MacHopperInstallResult =
|
||||
| "destination_exists"
|
||||
| "copy"
|
||||
| "launcher_missing";
|
||||
readonly cleanupFailure?: string;
|
||||
};
|
||||
|
||||
/** External effects used by the bounded macOS Hopper installer. */
|
||||
@@ -72,6 +80,8 @@ export interface MacHopperInstallHost {
|
||||
launcherReady(path: string): Promise<boolean>;
|
||||
openApplication(path: string): Promise<void>;
|
||||
cleanup(path: string): Promise<void>;
|
||||
/** Retain the cleanup capability until the owning setup host retries it. */
|
||||
retainCleanup?(cleanup: () => Promise<void>): void;
|
||||
destination(): string;
|
||||
}
|
||||
|
||||
@@ -85,187 +95,221 @@ export const installMacHopper = async (
|
||||
): Promise<MacHopperInstallResult> => {
|
||||
let temporary: string | undefined;
|
||||
let mounted = false;
|
||||
let result: MacHopperInstallResult;
|
||||
try {
|
||||
const metadata = await host.download(RELEASES_URL);
|
||||
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
|
||||
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
|
||||
if (!parsed.success)
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const release = parsed.data["OS X"];
|
||||
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const archive = await host.download(release.filename);
|
||||
if (!archive.ok) return { status: "failed", reason: "download" };
|
||||
if (!integrityMatches(archive.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
result = await (async (): Promise<MacHopperInstallResult> => {
|
||||
const metadata = await host.download(RELEASES_URL);
|
||||
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
|
||||
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
|
||||
if (!parsed.success)
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const release = parsed.data["OS X"];
|
||||
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const archive = await host.download(release.filename);
|
||||
if (!archive.ok) return { status: "failed", reason: "download" };
|
||||
if (!integrityMatches(archive.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
|
||||
const destination = host.destination();
|
||||
const destinationExists = await host.destinationExists(destination);
|
||||
if (destinationExists && options.replaceExisting !== true)
|
||||
return { status: "failed", reason: "destination_exists" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const packagePath = join(temporary, "hopper.dmg");
|
||||
const mountPath = join(temporary, "mounted");
|
||||
await host.createMountDirectory(mountPath);
|
||||
await host.writePackage(packagePath, archive.bytes);
|
||||
if (!(await host.mount(packagePath, mountPath)))
|
||||
return { status: "failed", reason: "mount" };
|
||||
mounted = true;
|
||||
const source = await host.appBundle(mountPath);
|
||||
if (source === undefined || !(await host.validBundle(source)))
|
||||
return { status: "failed", reason: "bundle" };
|
||||
const stage = `${destination}.rea-stage`;
|
||||
if (
|
||||
!(await host.installBundle(source, destination, stage, destinationExists))
|
||||
)
|
||||
return { status: "failed", reason: "copy" };
|
||||
const launcherPath = join(destination, "Contents/MacOS/hopper");
|
||||
if (!(await host.launcherReady(launcherPath)))
|
||||
return { status: "failed", reason: "launcher_missing" };
|
||||
await host.openApplication(destination);
|
||||
return { status: "installed", launcherPath };
|
||||
const destination = host.destination();
|
||||
const destinationExists = await host.destinationExists(destination);
|
||||
if (destinationExists && options.replaceExisting !== true)
|
||||
return { status: "failed", reason: "destination_exists" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const packagePath = join(temporary, "hopper.dmg");
|
||||
const mountPath = join(temporary, "mounted");
|
||||
await host.createMountDirectory(mountPath);
|
||||
await host.writePackage(packagePath, archive.bytes);
|
||||
if (!(await host.mount(packagePath, mountPath)))
|
||||
return { status: "failed", reason: "mount" };
|
||||
mounted = true;
|
||||
const source = await host.appBundle(mountPath);
|
||||
if (source === undefined || !(await host.validBundle(source)))
|
||||
return { status: "failed", reason: "bundle" };
|
||||
const stage = `${destination}.rea-stage`;
|
||||
if (
|
||||
!(await host.installBundle(
|
||||
source,
|
||||
destination,
|
||||
stage,
|
||||
destinationExists,
|
||||
))
|
||||
)
|
||||
return { status: "failed", reason: "copy" };
|
||||
const launcherPath = join(destination, "Contents/MacOS/hopper");
|
||||
if (!(await host.launcherReady(launcherPath)))
|
||||
return { status: "failed", reason: "launcher_missing" };
|
||||
await host.openApplication(destination);
|
||||
return { status: "installed", launcherPath };
|
||||
})();
|
||||
} catch (cause: unknown) {
|
||||
// Install-stage failures share the download reason; the stage is cleaned below.
|
||||
void cause;
|
||||
return { status: "failed", reason: "download" };
|
||||
} finally {
|
||||
if (temporary !== undefined) {
|
||||
const mountPath = join(temporary, "mounted");
|
||||
if (mounted) await host.unmount(mountPath);
|
||||
await host.cleanup(temporary);
|
||||
result = { status: "failed", reason: "download" };
|
||||
}
|
||||
|
||||
let cleanupFailure: string | undefined;
|
||||
if (temporary !== undefined) {
|
||||
const temporaryPath = temporary;
|
||||
const mountPath = join(temporary, "mounted");
|
||||
const cleanup = async (): Promise<void> => {
|
||||
if (mounted) {
|
||||
await host.unmount(mountPath);
|
||||
mounted = false;
|
||||
}
|
||||
await host.cleanup(temporaryPath);
|
||||
};
|
||||
try {
|
||||
await cleanup();
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = `Hopper DMG cleanup failed: ${failureMessage(cause)}`;
|
||||
host.retainCleanup?.(cleanup);
|
||||
}
|
||||
}
|
||||
|
||||
return cleanupFailure === undefined ? result : { ...result, cleanupFailure };
|
||||
};
|
||||
|
||||
/** Compose installer paths and subprocesses from one selected caller context. */
|
||||
export const systemMacHopperInstallHost = (
|
||||
homeDirectory: string,
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): MacHopperInstallHost => ({
|
||||
download: downloadPackage,
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
|
||||
createMountDirectory: (path) => mkdir(path),
|
||||
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
async mount(packagePath, mountPath) {
|
||||
return commandSucceeds(
|
||||
"hdiutil",
|
||||
[
|
||||
"attach",
|
||||
"-readonly",
|
||||
"-nobrowse",
|
||||
"-mountpoint",
|
||||
mountPath,
|
||||
packagePath,
|
||||
],
|
||||
environment,
|
||||
);
|
||||
},
|
||||
async unmount(mountPath) {
|
||||
await commandSucceeds(
|
||||
"hdiutil",
|
||||
["detach", mountPath, "-force"],
|
||||
environment,
|
||||
);
|
||||
},
|
||||
async appBundle(mountPath) {
|
||||
const entries = await readdir(mountPath, { withFileTypes: true });
|
||||
const app = entries.find(
|
||||
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
|
||||
);
|
||||
return app === undefined ? undefined : join(mountPath, app.name);
|
||||
},
|
||||
async validBundle(appPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"codesign",
|
||||
["--verify", "--deep", "--strict", appPath],
|
||||
): MacHopperInstallHost & { close(): Promise<string | undefined> } => {
|
||||
const pendingCleanup = new Set<() => Promise<void>>();
|
||||
return {
|
||||
download: downloadHopperPackage,
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
|
||||
createMountDirectory: (path) => mkdir(path),
|
||||
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
async mount(packagePath, mountPath) {
|
||||
return commandSucceeds(
|
||||
"hdiutil",
|
||||
[
|
||||
"attach",
|
||||
"-readonly",
|
||||
"-nobrowse",
|
||||
"-mountpoint",
|
||||
mountPath,
|
||||
packagePath,
|
||||
],
|
||||
environment,
|
||||
))
|
||||
)
|
||||
return false;
|
||||
try {
|
||||
const identifier = (
|
||||
await execFileAsync(
|
||||
"/usr/libexec/PlistBuddy",
|
||||
[
|
||||
"-c",
|
||||
"Print :CFBundleIdentifier",
|
||||
join(appPath, "Contents/Info.plist"),
|
||||
],
|
||||
{ env: environment },
|
||||
)
|
||||
).stdout.trim();
|
||||
return identifier === BUNDLE_IDENTIFIER;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional bundle probing; unreadable means invalid.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async destinationExists(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional existence probing; absence means missing.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async installBundle(source, destination, stage, replaceExisting) {
|
||||
const previous = `${stage}-previous`;
|
||||
try {
|
||||
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
await execFileAsync("ditto", [source, stage], { env: environment });
|
||||
await access(join(stage, "Contents/MacOS/hopper"));
|
||||
if (replaceExisting) await rename(destination, previous);
|
||||
await rename(stage, destination);
|
||||
if (replaceExisting) await rm(previous, { recursive: true, force: true });
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// Install failure rolls back below; the false return reports it.
|
||||
void cause;
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
if (replaceExisting) {
|
||||
);
|
||||
},
|
||||
async unmount(mountPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"hdiutil",
|
||||
["detach", mountPath, "-force"],
|
||||
environment,
|
||||
))
|
||||
)
|
||||
throw new Error(
|
||||
`hdiutil detach failed for ${JSON.stringify(mountPath)}`,
|
||||
);
|
||||
},
|
||||
async appBundle(mountPath) {
|
||||
const entries = await readdir(mountPath, { withFileTypes: true });
|
||||
const app = entries.find(
|
||||
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
|
||||
);
|
||||
return app === undefined ? undefined : join(mountPath, app.name);
|
||||
},
|
||||
async validBundle(appPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"codesign",
|
||||
["--verify", "--deep", "--strict", appPath],
|
||||
environment,
|
||||
))
|
||||
)
|
||||
return false;
|
||||
try {
|
||||
const identifier = (
|
||||
await execFileAsync(
|
||||
"/usr/libexec/PlistBuddy",
|
||||
[
|
||||
"-c",
|
||||
"Print :CFBundleIdentifier",
|
||||
join(appPath, "Contents/Info.plist"),
|
||||
],
|
||||
{ env: environment },
|
||||
)
|
||||
).stdout.trim();
|
||||
return identifier === BUNDLE_IDENTIFIER;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional bundle probing; unreadable means invalid.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async destinationExists(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional existence probing; absence means missing.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async installBundle(source, destination, stage, replaceExisting) {
|
||||
const previous = `${stage}-previous`;
|
||||
try {
|
||||
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
await execFileAsync("ditto", [source, stage], { env: environment });
|
||||
await access(join(stage, "Contents/MacOS/hopper"));
|
||||
if (replaceExisting) await rename(destination, previous);
|
||||
await rename(stage, destination);
|
||||
if (replaceExisting)
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// Install failure rolls back below; the false return reports it.
|
||||
void cause;
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
if (replaceExisting) {
|
||||
try {
|
||||
await rename(previous, destination);
|
||||
} catch (rollbackCause: unknown) {
|
||||
// The original destination was not moved, or rollback is impossible.
|
||||
void rollbackCause;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async launcherReady(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional launcher probing; absence means not ready.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async openApplication(path) {
|
||||
await execFileAsync("open", [path], { env: environment });
|
||||
},
|
||||
cleanup: (path) => rm(path, { recursive: true, force: true }),
|
||||
retainCleanup: (cleanup) => pendingCleanup.add(cleanup),
|
||||
async close() {
|
||||
let failure: string | undefined;
|
||||
for (const cleanup of pendingCleanup) {
|
||||
try {
|
||||
await rename(previous, destination);
|
||||
} catch (rollbackCause: unknown) {
|
||||
// The original destination was not moved, or rollback is impossible.
|
||||
void rollbackCause;
|
||||
await cleanup();
|
||||
pendingCleanup.delete(cleanup);
|
||||
} catch (cause: unknown) {
|
||||
failure ??= `Hopper DMG cleanup remains incomplete: ${failureMessage(cause)}`;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async launcherReady(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional launcher probing; absence means not ready.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async openApplication(path) {
|
||||
await execFileAsync("open", [path], { env: environment });
|
||||
},
|
||||
cleanup: (path) => rm(path, { recursive: true, force: true }),
|
||||
destination: () =>
|
||||
join(homeDirectory, "Applications/Hopper Disassembler.app"),
|
||||
});
|
||||
|
||||
const downloadPackage = async (url: string) => {
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
signal: AbortSignal.timeout(120_000),
|
||||
});
|
||||
const length = Number(response.headers.get("content-length") ?? "0");
|
||||
if (length > MAX_PACKAGE_BYTES) return { ok: false, bytes: new Uint8Array() };
|
||||
const bytes = new Uint8Array(await response.arrayBuffer());
|
||||
return { ok: response.ok && bytes.byteLength <= MAX_PACKAGE_BYTES, bytes };
|
||||
return failure;
|
||||
},
|
||||
destination: () =>
|
||||
join(homeDirectory, "Applications/Hopper Disassembler.app"),
|
||||
};
|
||||
};
|
||||
|
||||
const parseJson = (bytes: Uint8Array): unknown =>
|
||||
@@ -275,7 +319,7 @@ const integrityMatches = (
|
||||
release: z.infer<typeof releaseSchema>,
|
||||
): boolean =>
|
||||
bytes.byteLength === Number(release.file_length) &&
|
||||
bytes.byteLength <= MAX_PACKAGE_BYTES &&
|
||||
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
|
||||
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
|
||||
const commandSucceeds = async (
|
||||
command: string,
|
||||
@@ -291,6 +335,8 @@ const commandSucceeds = async (
|
||||
return false;
|
||||
}
|
||||
};
|
||||
const failureMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
const releaseSchema = z.object({
|
||||
filename: z.string().url(),
|
||||
file_length: z.string().regex(/^\d+$/u),
|
||||
|
||||
@@ -1,3 +1,11 @@
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
import {
|
||||
createJavaScriptSemanticGraph,
|
||||
createJavaScriptSemanticGraphNode,
|
||||
JavaScriptSemanticEvidenceContextRegistry,
|
||||
} from "../domain/javascript/javascriptSemanticGraph.js";
|
||||
import { proofEvidence } from "./ReconstructionObligationLedger.fixture.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { createEvidence } from "../domain/evidence.js";
|
||||
@@ -171,3 +179,115 @@ describe("reconstruction obligation candidates", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps each semantic candidate bound to its own file and authority context", () => {
|
||||
const contexts = new JavaScriptSemanticEvidenceContextRegistry();
|
||||
const base = javascriptApplicationAnalysisResultSchema.parse(
|
||||
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
|
||||
);
|
||||
const sourceRecords = [
|
||||
proofEvidence("first-file"),
|
||||
proofEvidence("second-file"),
|
||||
];
|
||||
const module = createJavaScriptSemanticGraphNode(
|
||||
{
|
||||
kind: "module",
|
||||
identity: {
|
||||
artifact_sha256: base.root_artifact_sha256,
|
||||
module_path: "app.js",
|
||||
source_range: null,
|
||||
role_key: "module",
|
||||
},
|
||||
function_node_id: null,
|
||||
application_node_ids: [],
|
||||
label: "app.js",
|
||||
properties: {},
|
||||
evidence: artifactEvidence(base.root_artifact_sha256, "app.js"),
|
||||
},
|
||||
contexts,
|
||||
);
|
||||
const requests = sourceRecords.flatMap((source, fileIndex) =>
|
||||
[1, 2].map((line) => {
|
||||
const sha256 = String(fileIndex + 4).repeat(64);
|
||||
const path = `file-${String(fileIndex)}.js`;
|
||||
return createJavaScriptSemanticGraphNode(
|
||||
{
|
||||
kind: "request",
|
||||
identity: {
|
||||
artifact_sha256: sha256,
|
||||
module_path: path,
|
||||
source_range: {
|
||||
start: { line, column: 0 },
|
||||
end: { line, column: 8 },
|
||||
},
|
||||
role_key: `request-${String(line)}`,
|
||||
},
|
||||
function_node_id: null,
|
||||
application_node_ids: [],
|
||||
label: `${path}:${String(line)}`,
|
||||
properties: {},
|
||||
evidence: {
|
||||
...artifactEvidence(sha256, path, "ast-static-analysis"),
|
||||
evidence_ids: [source.evidence_id],
|
||||
},
|
||||
},
|
||||
contexts,
|
||||
);
|
||||
}),
|
||||
);
|
||||
const { graph_id: _graphId, ...semantic } = base.semantic_graph;
|
||||
const semanticGraph = createJavaScriptSemanticGraph({
|
||||
...semantic,
|
||||
root_node_ids: [module.node_id],
|
||||
nodes: [module, ...requests],
|
||||
evidence_contexts: contexts.contexts,
|
||||
});
|
||||
const example = JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE;
|
||||
const application = createEvidence(
|
||||
{
|
||||
path: base.input_path,
|
||||
sha256: base.root_artifact_sha256,
|
||||
format: base.format,
|
||||
},
|
||||
example.provider,
|
||||
{
|
||||
predicateType: example.predicate_type,
|
||||
operation: example.operation,
|
||||
parameters: example.parameters,
|
||||
result: jsonValueSchema.parse({ ...base, semantic_graph: semanticGraph }),
|
||||
confidence: "derived",
|
||||
authority: "shipped-artifact",
|
||||
evidenceLinks: sourceRecords.map(({ evidence_id }) => evidence_id),
|
||||
},
|
||||
);
|
||||
const generated = deriveReconstructionObligationCandidates(
|
||||
createEvidenceBundle([...sourceRecords, application]),
|
||||
[],
|
||||
);
|
||||
expect(generated.limitations).toEqual([]);
|
||||
expect(generated.candidates).toHaveLength(requests.length);
|
||||
for (const [index, node] of requests.entries()) {
|
||||
const source = sourceRecords[Math.floor(index / 2)];
|
||||
expect(
|
||||
generated.candidates.find(
|
||||
({ target }) => target.semantic_node_id === node.node_id,
|
||||
),
|
||||
).toMatchObject({
|
||||
title: node.label,
|
||||
family: "request",
|
||||
source_state: "candidate",
|
||||
target: {
|
||||
artifact_sha256: node.identity.artifact_sha256,
|
||||
semantic_node_id: node.node_id,
|
||||
},
|
||||
authority_references: [
|
||||
{
|
||||
evidence_id: source?.evidence_id,
|
||||
authority: "controlled-replay",
|
||||
state: "candidate",
|
||||
location: `${semanticGraph.graph_id}/node/${node.node_id}`,
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
@@ -12,7 +12,6 @@ import type {
|
||||
ReconstructionObligation,
|
||||
ReviewedReconstructionObligation,
|
||||
} from "../domain/reconstructionObligationLedgerSchemas.js";
|
||||
import { resolveJavaScriptSemanticEvidence } from "../domain/javascript/javascriptSemanticGraph.js";
|
||||
import { parseApplicationGraphEvidence } from "./javascript/JavaScriptApplicationEvidenceGraph.js";
|
||||
import {
|
||||
applicationObligationPolicy,
|
||||
@@ -115,14 +114,21 @@ const deriveApplicationCandidates = (
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (source.semanticGraph !== null)
|
||||
if (source.semanticGraph !== null) {
|
||||
const evidenceContexts = new Map(
|
||||
source.semanticGraph.evidence_contexts.map((context) => [
|
||||
context.context_id,
|
||||
context,
|
||||
]),
|
||||
);
|
||||
for (const node of source.semanticGraph.nodes) {
|
||||
const candidatePolicy = semanticObligationPolicy(node.kind);
|
||||
if (candidatePolicy === undefined) continue;
|
||||
const evidenceContext = resolveJavaScriptSemanticEvidence(
|
||||
source.semanticGraph,
|
||||
node.evidence,
|
||||
);
|
||||
const evidenceContext = evidenceContexts.get(node.evidence.context_id);
|
||||
if (evidenceContext === undefined)
|
||||
throw new TypeError(
|
||||
"Semantic evidence reference names an absent context",
|
||||
);
|
||||
addCandidate(
|
||||
candidates,
|
||||
generatedCandidate({
|
||||
@@ -144,6 +150,7 @@ const deriveApplicationCandidates = (
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (source.graph.coverage.status !== "complete")
|
||||
limitations.add(
|
||||
`Application graph ${source.graph.graph_id} coverage is ${source.graph.coverage.status}; omitted candidates remain unknown.`,
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { posix } from "node:path";
|
||||
|
||||
import { logicalPathEscapesRoot } from "../domain/artifactIdentity.js";
|
||||
import {
|
||||
logicalPathEscapesRoot,
|
||||
normalizeJoinedLogicalPath,
|
||||
} from "../domain/artifactIdentity.js";
|
||||
createReferenceSourcePathLookup,
|
||||
isPortableAbsoluteReferenceTarget,
|
||||
type ReferenceSourcePathLookup,
|
||||
} from "../domain/referenceSourcePathIdentity.js";
|
||||
import {
|
||||
classifyReferenceSourcePath,
|
||||
detectReferenceSourceLanguage,
|
||||
@@ -83,11 +85,11 @@ const failedEntry = (
|
||||
const resolveInternalSpecifier = (
|
||||
fromPath: string,
|
||||
specifier: string,
|
||||
filePaths: ReadonlySet<string>,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
): { to: string; resolution: "internal" | "unresolved" } => {
|
||||
if (posix.isAbsolute(specifier))
|
||||
return { to: specifier, resolution: "unresolved" };
|
||||
const normalized = normalizeJoinedLogicalPath(
|
||||
const normalized = posix.normalize(
|
||||
posix.join(posix.dirname(fromPath), specifier),
|
||||
);
|
||||
if (logicalPathEscapesRoot(normalized))
|
||||
@@ -113,7 +115,7 @@ const resolveInternalSpecifier = (
|
||||
const stem = normalized.slice(0, -posix.extname(normalized).length);
|
||||
candidates.push(...sourceSuffixes.map((suffix) => `${stem}${suffix}`));
|
||||
}
|
||||
const match = candidates.find((candidate) => filePaths.has(candidate));
|
||||
const match = lookupPath.resolveCandidates(candidates);
|
||||
return match === undefined
|
||||
? { to: normalized, resolution: "unresolved" }
|
||||
: { to: match, resolution: "internal" };
|
||||
@@ -121,7 +123,7 @@ const resolveInternalSpecifier = (
|
||||
|
||||
const appendFile = (
|
||||
entry: Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>,
|
||||
filePaths: ReadonlySet<string>,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
output: ParsedReferenceSourceEntries,
|
||||
): void => {
|
||||
const language = detectReferenceSourceLanguage(entry.path);
|
||||
@@ -146,7 +148,7 @@ const appendFile = (
|
||||
const resolved = resolveInternalSpecifier(
|
||||
relationship.from_path,
|
||||
relationship.to,
|
||||
filePaths,
|
||||
lookupPath,
|
||||
);
|
||||
output.relationships.push({
|
||||
...relationship,
|
||||
@@ -159,11 +161,11 @@ const appendFile = (
|
||||
|
||||
const appendReadEntry = (
|
||||
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
|
||||
filePaths: ReadonlySet<string>,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
output: ParsedReferenceSourceEntries,
|
||||
): void => {
|
||||
if (entry.kind === "file") {
|
||||
appendFile(entry, filePaths, output);
|
||||
appendFile(entry, lookupPath, output);
|
||||
return;
|
||||
}
|
||||
const classifications = classifyReferenceSourcePath(entry.path);
|
||||
@@ -180,7 +182,14 @@ const appendReadEntry = (
|
||||
output.entries.push({
|
||||
path: entry.path,
|
||||
kind: "symlink",
|
||||
target: entry.target,
|
||||
// The reader resolves inventory identities from its root; the persisted
|
||||
// graph represents relative symlink targets from the link's directory.
|
||||
target:
|
||||
entry.targetState === "internal" ||
|
||||
(entry.targetState === "missing" &&
|
||||
!isPortableAbsoluteReferenceTarget(entry.target))
|
||||
? posix.relative(posix.dirname(entry.path), entry.target) || "."
|
||||
: entry.target,
|
||||
target_state: entry.targetState,
|
||||
classifications,
|
||||
limitations: [],
|
||||
@@ -199,10 +208,11 @@ export const parseReferenceSourceEntries = (
|
||||
parseFailures: [],
|
||||
limitations: [...read.limitations],
|
||||
};
|
||||
const lookupPath = createReferenceSourcePathLookup(filePaths);
|
||||
for (const entry of read.entries) {
|
||||
if (signal?.aborted === true) break;
|
||||
if (entry.status === "failed") output.entries.push(failedEntry(entry));
|
||||
else appendReadEntry(entry, filePaths, output);
|
||||
else appendReadEntry(entry, lookupPath, output);
|
||||
}
|
||||
return output;
|
||||
};
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { readFile, realpath, stat } from "node:fs/promises";
|
||||
import { lstat, realpath, stat } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
import ignore from "ignore";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
@@ -47,14 +48,27 @@ const resolveRoot = async (
|
||||
const buildIgnored = async (
|
||||
root: string,
|
||||
excludePaths: readonly string[],
|
||||
signal?: AbortSignal,
|
||||
): Promise<ReturnType<typeof ignore>> => {
|
||||
const ignored = ignore();
|
||||
const policyPath = join(root, ".gitignore");
|
||||
let present = true;
|
||||
try {
|
||||
ignored.add(await readFile(join(root, ".gitignore"), "utf8"));
|
||||
await lstat(policyPath);
|
||||
} catch (cause: unknown) {
|
||||
// Missing or unreadable ignore file does not authorize broader access.
|
||||
void cause;
|
||||
if (
|
||||
!(
|
||||
typeof cause === "object" &&
|
||||
cause !== null &&
|
||||
"code" in cause &&
|
||||
cause.code === "ENOENT"
|
||||
)
|
||||
)
|
||||
throw cause;
|
||||
present = false;
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
if (present) ignored.add(await readRegularFileText(policyPath, { signal }));
|
||||
ignored.add([...DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS]);
|
||||
for (const path of excludePaths) {
|
||||
ignored.add(path);
|
||||
@@ -71,9 +85,22 @@ export const prepareReferenceSourceImport = async (
|
||||
> => {
|
||||
const root = await resolveRoot(options.root);
|
||||
if (!root.ok) return root;
|
||||
return ok({
|
||||
root: root.value,
|
||||
ignored: await buildIgnored(root.value, options.excludePaths ?? []),
|
||||
secrets: ignore().add([...options.policy.secretPatterns]),
|
||||
});
|
||||
try {
|
||||
return ok({
|
||||
root: root.value,
|
||||
ignored: await buildIgnored(
|
||||
root.value,
|
||||
options.excludePaths ?? [],
|
||||
options.signal,
|
||||
),
|
||||
secrets: ignore().add([...options.policy.secretPatterns]),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
failure(
|
||||
options.signal?.aborted === true ? "cancelled" : "io",
|
||||
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import fs from "node:fs";
|
||||
import { lstat, readFile } from "node:fs/promises";
|
||||
import { lstat } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { readRegularFile, readRegularFileText } from "./RegularFileRead.js";
|
||||
|
||||
export type ReferenceSourceVcsInfo =
|
||||
| {
|
||||
@@ -29,7 +30,7 @@ export const readReferenceSourceVcs = async (
|
||||
: { kind: "unknown", head: null, dirty: null };
|
||||
}
|
||||
try {
|
||||
const head = await resolveSourceHead(root);
|
||||
const head = await resolveSourceHead(root, signal);
|
||||
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
|
||||
return { kind: "git", head, dirty: null };
|
||||
} catch (cause: unknown) {
|
||||
@@ -39,13 +40,59 @@ export const readReferenceSourceVcs = async (
|
||||
}
|
||||
};
|
||||
|
||||
const resolveSourceHead = async (root: string): Promise<string> => {
|
||||
const resolveSourceHead = async (
|
||||
root: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
|
||||
const { resolveRef } = await import("isomorphic-git");
|
||||
let failedRead: { readonly cause: unknown } | undefined;
|
||||
const referenceFs = {
|
||||
...fs,
|
||||
promises: {
|
||||
...fs.promises,
|
||||
// isomorphic-git treats every read failure as absence. Preserve real
|
||||
// failures so a readable packed ref cannot hide an unreadable loose ref.
|
||||
readFile: async (
|
||||
path: string | undefined,
|
||||
options:
|
||||
| BufferEncoding
|
||||
| { readonly encoding?: BufferEncoding | null }
|
||||
| null = null,
|
||||
) => {
|
||||
// The library probes readFile() without a path to detect promise APIs.
|
||||
// Reject that invocation without attributing it to a source read.
|
||||
if (path === undefined)
|
||||
throw new TypeError("Git filesystem read requires a path");
|
||||
try {
|
||||
const bytes = await readRegularFile(path, { signal });
|
||||
const encoding =
|
||||
typeof options === "string" ? options : options?.encoding;
|
||||
return encoding === undefined || encoding === null
|
||||
? bytes
|
||||
: bytes.toString(encoding);
|
||||
} catch (cause: unknown) {
|
||||
if (errorCode(cause) !== "ENOENT") failedRead ??= { cause };
|
||||
throw cause;
|
||||
}
|
||||
},
|
||||
},
|
||||
};
|
||||
const readRef = async (options: {
|
||||
readonly gitdir?: string;
|
||||
readonly ref: string;
|
||||
}) => {
|
||||
const head = await resolveRef({ fs: referenceFs, dir: root, ...options });
|
||||
signal?.throwIfAborted();
|
||||
if (failedRead !== undefined) throw failedRead.cause;
|
||||
return head;
|
||||
};
|
||||
const dotgit = join(root, ".git");
|
||||
if (!(await lstat(dotgit)).isFile())
|
||||
return resolveRef({ fs, dir: root, ref: "HEAD" });
|
||||
const pointer = (await readFile(dotgit, "utf8")).replace(/\r?\n$/u, "");
|
||||
if (!(await lstat(dotgit)).isFile()) return readRef({ ref: "HEAD" });
|
||||
const pointer = (await readRegularFileText(dotgit, { signal })).replace(
|
||||
/\r?\n$/u,
|
||||
"",
|
||||
);
|
||||
if (!pointer.startsWith("gitdir: ") || pointer.length === 8)
|
||||
throw new Error("Invalid Git directory pointer");
|
||||
const gitdir = resolve(root, pointer.slice(8));
|
||||
@@ -53,25 +100,23 @@ const resolveSourceHead = async (root: string): Promise<string> => {
|
||||
try {
|
||||
commonDirectory = resolve(
|
||||
gitdir,
|
||||
(await readFile(join(gitdir, "commondir"), "utf8")).replace(
|
||||
/\r?\n$/u,
|
||||
"",
|
||||
),
|
||||
(
|
||||
await readRegularFileText(join(gitdir, "commondir"), { signal })
|
||||
).replace(/\r?\n$/u, ""),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
if (errorCode(cause) !== "ENOENT") throw cause;
|
||||
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||
return readRef({ gitdir, ref: "HEAD" });
|
||||
}
|
||||
const head = (await readFile(join(gitdir, "HEAD"), "utf8")).trim();
|
||||
if (!head.startsWith("ref: "))
|
||||
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||
const head = (
|
||||
await readRegularFileText(join(gitdir, "HEAD"), { signal })
|
||||
).trim();
|
||||
if (!head.startsWith("ref: ")) return readRef({ gitdir, ref: "HEAD" });
|
||||
const ref = head.slice(5);
|
||||
const privateRef = ["refs/bisect/", "refs/rewritten/", "refs/worktree/"].some(
|
||||
(prefix) => ref.startsWith(prefix),
|
||||
);
|
||||
return resolveRef({
|
||||
fs,
|
||||
dir: root,
|
||||
return readRef({
|
||||
gitdir: privateRef ? gitdir : commonDirectory,
|
||||
ref,
|
||||
});
|
||||
|
||||
@@ -1,20 +1,44 @@
|
||||
import type { Stats } from "node:fs";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
import { lstat, stat, type FileHandle } from "node:fs/promises";
|
||||
|
||||
import { openRegularFile } from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
RegularFileChangedError,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
||||
|
||||
/** Filesystem admission policy for one selected regular input. */
|
||||
export interface RegularFileReadOptions {
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
readonly symlinks?: "follow" | "reject";
|
||||
}
|
||||
|
||||
/** Admit one regular-file handle, retain it through the consumer, and always close it. */
|
||||
export const withRegularFile = async <Value>(
|
||||
path: string,
|
||||
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
|
||||
signal?: AbortSignal,
|
||||
options: RegularFileReadOptions = {},
|
||||
): Promise<Value> => {
|
||||
const handle = await openRegularFile(path, { symlinks: "follow", signal });
|
||||
const { signal } = options;
|
||||
const handle = await openRegularFile(path, {
|
||||
symlinks: options.symlinks ?? "follow",
|
||||
signal,
|
||||
});
|
||||
try {
|
||||
const stats = await handle.stat();
|
||||
signal?.throwIfAborted();
|
||||
const value = await read(handle, stats);
|
||||
signal?.throwIfAborted();
|
||||
const [opened, currentPath] = await Promise.all([
|
||||
handle.stat(),
|
||||
options.symlinks === "reject" ? lstat(path) : stat(path),
|
||||
]);
|
||||
if (
|
||||
!sameRegularFileState(stats, opened) ||
|
||||
!sameRegularFileState(stats, currentPath)
|
||||
)
|
||||
throw new RegularFileChangedError(path);
|
||||
return value;
|
||||
} finally {
|
||||
await handle.close();
|
||||
@@ -24,6 +48,25 @@ export const withRegularFile = async <Value>(
|
||||
/** Admit a regular file without waiting for a pipe, then read its verified handle. */
|
||||
export const readRegularFile = (
|
||||
path: string,
|
||||
signal?: AbortSignal,
|
||||
options: RegularFileReadOptions = {},
|
||||
): Promise<Buffer> =>
|
||||
withRegularFile(path, (handle) => handle.readFile({ signal }), signal);
|
||||
withRegularFile(
|
||||
path,
|
||||
async (handle, stats) => {
|
||||
const bytes = await readBoundedFileBytes(
|
||||
handle,
|
||||
stats.size,
|
||||
options.signal,
|
||||
);
|
||||
if (bytes === undefined || bytes.length !== stats.size)
|
||||
throw new RegularFileChangedError(path);
|
||||
return bytes;
|
||||
},
|
||||
options,
|
||||
);
|
||||
|
||||
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
|
||||
export const readRegularFileText = async (
|
||||
path: string,
|
||||
options: RegularFileReadOptions = {},
|
||||
): Promise<string> => (await readRegularFile(path, options)).toString("utf8");
|
||||
|
||||
@@ -29,6 +29,49 @@ export const runSetup = async (
|
||||
options: SetupOptions,
|
||||
host: SetupHost = systemSetupHost(),
|
||||
confirm?: SetupConfirmation,
|
||||
): Promise<SetupResult> => {
|
||||
let result: SetupResult | undefined;
|
||||
let primaryFailure: unknown;
|
||||
let threw = false;
|
||||
try {
|
||||
result = await runSetupWorkflow(options, host, confirm);
|
||||
} catch (cause: unknown) {
|
||||
threw = true;
|
||||
primaryFailure = cause;
|
||||
}
|
||||
|
||||
let cleanupFailure: string | undefined;
|
||||
try {
|
||||
cleanupFailure = await host.close?.();
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = `Setup resource cleanup failed: ${failureMessage(cause)}`;
|
||||
}
|
||||
|
||||
if (threw) {
|
||||
if (cleanupFailure !== undefined)
|
||||
throw new AggregateError(
|
||||
[primaryFailure, new Error(cleanupFailure)],
|
||||
"Setup and resource cleanup failed.",
|
||||
{ cause: primaryFailure },
|
||||
);
|
||||
throw primaryFailure;
|
||||
}
|
||||
if (result === undefined) throw new Error("Setup returned no result.");
|
||||
if (cleanupFailure === undefined) return result;
|
||||
return {
|
||||
...result,
|
||||
status: "needs_human",
|
||||
code: result.code ?? "cleanup_failed",
|
||||
remediation: result.remediation
|
||||
? `${result.remediation} ${cleanupFailure}`
|
||||
: cleanupFailure,
|
||||
};
|
||||
};
|
||||
|
||||
const runSetupWorkflow = async (
|
||||
options: SetupOptions,
|
||||
host: SetupHost,
|
||||
confirm?: SetupConfirmation,
|
||||
): Promise<SetupResult> => {
|
||||
const appliedActions: string[] = [];
|
||||
let plannedActions: readonly SetupAction[] = [];
|
||||
@@ -59,6 +102,7 @@ export const runSetup = async (
|
||||
options.readinessScope?.clients === undefined &&
|
||||
options.allDetectedClients !== true &&
|
||||
options.installHopper !== true &&
|
||||
options.skillClientIds === undefined &&
|
||||
options.installSkill === undefined;
|
||||
const interactiveSelection =
|
||||
confirm !== undefined && !options.structured && clientSelectionAllowed;
|
||||
@@ -67,10 +111,12 @@ export const runSetup = async (
|
||||
: (options.clientIds ??
|
||||
options.readinessScope?.clients ??
|
||||
discovery.defaultClientIds);
|
||||
let skillNeedsInstall = await host.skillNeedsInstall(selectedClientIds);
|
||||
let selectedSkillClientIds = options.skillClientIds ?? selectedClientIds;
|
||||
let skillNeedsInstall = await host.skillNeedsInstall(selectedSkillClientIds);
|
||||
let skillSelected =
|
||||
options.installSkill === true ||
|
||||
(options.installSkill !== false && selectedClientIds.length > 0);
|
||||
(options.installSkill !== false &&
|
||||
(selectedClientIds.length > 0 || selectedSkillClientIds.length > 0));
|
||||
let installSkill = skillSelected && skillNeedsInstall;
|
||||
let planDiscovery = discovery;
|
||||
if (interactiveSelection) {
|
||||
@@ -138,6 +184,7 @@ export const runSetup = async (
|
||||
selectedClientIds.length === 0) ||
|
||||
(options.installSkill !== false && selectedClientIds.length > 0);
|
||||
installSkill = skillSelected && skillNeedsInstall;
|
||||
selectedSkillClientIds = selectedClientIds;
|
||||
}
|
||||
const planned = await planSetupActions({
|
||||
discovery: planDiscovery,
|
||||
@@ -145,7 +192,7 @@ export const runSetup = async (
|
||||
providerEnvironment,
|
||||
command: host.registrationCommand,
|
||||
clientIds: selectedClientIds,
|
||||
skillClientIds: selectedClientIds,
|
||||
skillClientIds: selectedSkillClientIds,
|
||||
installSkill,
|
||||
});
|
||||
plannedActions = planned.plannedActions;
|
||||
@@ -256,7 +303,7 @@ export const runSetup = async (
|
||||
host,
|
||||
options,
|
||||
appliedActions,
|
||||
selectedClientIds,
|
||||
selectedSkillClientIds,
|
||||
))
|
||||
)
|
||||
return fail(
|
||||
@@ -325,8 +372,11 @@ const installHopperAction = async (input: {
|
||||
emitProgress(input.options, {
|
||||
actionId: "install_hopper",
|
||||
label,
|
||||
state: "completed",
|
||||
detail: installed.launcherPath,
|
||||
state: installed.cleanupFailure === undefined ? "completed" : "warning",
|
||||
detail:
|
||||
installed.cleanupFailure === undefined
|
||||
? installed.launcherPath
|
||||
: `${installed.launcherPath}; ${installed.cleanupFailure}`,
|
||||
});
|
||||
const providerEnvironment = {
|
||||
...input.providerEnvironment,
|
||||
@@ -492,3 +542,6 @@ const summarizeDoctor = (report: DoctorReport): SetupDoctorSummary => ({
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
const failureMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
@@ -14,7 +14,7 @@ import {
|
||||
type ClientRegistrationDialect,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import writeFileAtomic from "write-file-atomic";
|
||||
|
||||
@@ -22,6 +22,7 @@ import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import type {
|
||||
ClientConfigurationInspection,
|
||||
ClientConfigurationResult,
|
||||
@@ -64,7 +65,7 @@ const configureClientDocument = async (
|
||||
return { status: "failed", reason: "path" };
|
||||
let original: string | undefined;
|
||||
try {
|
||||
original = await readFile(transactionPath, "utf8");
|
||||
original = await readRegularFileText(transactionPath);
|
||||
} catch (cause: unknown) {
|
||||
if (!isMissing(cause)) return { status: "failed", reason: "readback" };
|
||||
}
|
||||
@@ -149,7 +150,7 @@ const configureClientDocument = async (
|
||||
}
|
||||
try {
|
||||
const readback = parseClientConfiguration(
|
||||
await readFile(transactionPath, "utf8"),
|
||||
await readRegularFileText(transactionPath),
|
||||
format,
|
||||
);
|
||||
if (!registrationCurrent(readback, desired)) {
|
||||
@@ -178,7 +179,7 @@ export const clientConfigurationAligned = async (
|
||||
try {
|
||||
const files = await readClientConfigurationFiles(client);
|
||||
if (!files.ok) return false;
|
||||
const original = await readFile(client.configPath, "utf8");
|
||||
const original = await readRegularFileText(client.configPath);
|
||||
const parsed = parseClientConfiguration(original, client.format);
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
@@ -246,7 +247,7 @@ export const inspectClientConfiguration = async (
|
||||
};
|
||||
let original: string;
|
||||
try {
|
||||
original = await readFile(transactionPath, "utf8");
|
||||
original = await readRegularFileText(transactionPath);
|
||||
} catch (cause: unknown) {
|
||||
if (isMissing(cause)) return { status: "create" };
|
||||
return {
|
||||
|
||||
@@ -94,6 +94,10 @@ export const systemSetupHost = (
|
||||
const doctorHost = selectedDoctorHost ?? systemDoctorHost({ environment });
|
||||
const platform = doctorHost.platform;
|
||||
const { homeDirectory } = doctorHost;
|
||||
const macHopperHost =
|
||||
platform === "darwin"
|
||||
? systemMacHopperInstallHost(homeDirectory, environment)
|
||||
: undefined;
|
||||
return {
|
||||
platform,
|
||||
homeDirectory,
|
||||
@@ -106,6 +110,7 @@ export const systemSetupHost = (
|
||||
nodeVersion: process.versions.node,
|
||||
macosVersion: () => doctorHost.macosVersion(),
|
||||
linuxDistribution: readLinuxDistribution,
|
||||
close: () => macHopperHost?.close() ?? Promise.resolve(undefined),
|
||||
initialSetupState: async (
|
||||
scope?: DoctorScope,
|
||||
): Promise<SetupInitialState> => {
|
||||
@@ -126,15 +131,33 @@ export const systemSetupHost = (
|
||||
};
|
||||
},
|
||||
installHopper: async (replaceExisting) => {
|
||||
const result =
|
||||
platform === "linux"
|
||||
? await installLinuxHopper(systemLinuxHopperInstallHost(environment))
|
||||
: await installMacHopper(
|
||||
{ replaceExisting },
|
||||
systemMacHopperInstallHost(homeDirectory, environment),
|
||||
);
|
||||
if (result.status === "installed") return result;
|
||||
return setupInstallFailure(result.reason);
|
||||
if (platform === "linux") {
|
||||
const result = await installLinuxHopper(
|
||||
systemLinuxHopperInstallHost(environment),
|
||||
);
|
||||
return result.status === "installed"
|
||||
? result
|
||||
: setupInstallFailure(result.reason);
|
||||
}
|
||||
if (macHopperHost === undefined)
|
||||
return setupInstallFailure("unsupported_host");
|
||||
const result = await installMacHopper({ replaceExisting }, macHopperHost);
|
||||
if (result.status === "installed")
|
||||
return {
|
||||
status: "installed",
|
||||
launcherPath: result.launcherPath,
|
||||
...(result.cleanupFailure === undefined
|
||||
? {}
|
||||
: { cleanupFailure: result.cleanupFailure }),
|
||||
};
|
||||
const failure = setupInstallFailure(result.reason);
|
||||
if (result.cleanupFailure === undefined) return failure;
|
||||
return failure.status === "failed"
|
||||
? {
|
||||
...failure,
|
||||
remediation: `${failure.remediation} ${result.cleanupFailure}`,
|
||||
}
|
||||
: failure;
|
||||
},
|
||||
detectedClients: () => detectClients(homeDirectory, platform, environment),
|
||||
supportedClients: () =>
|
||||
|
||||
@@ -10,10 +10,15 @@ export type SetupFailureCode =
|
||||
| "destination_exists"
|
||||
| "mount_failed"
|
||||
| "bundle_invalid"
|
||||
| "copy_failed";
|
||||
| "copy_failed"
|
||||
| "cleanup_failed";
|
||||
|
||||
export type SetupHopperInstallResult =
|
||||
| { readonly status: "installed"; readonly launcherPath: string }
|
||||
| {
|
||||
readonly status: "installed";
|
||||
readonly launcherPath: string;
|
||||
readonly cleanupFailure?: string;
|
||||
}
|
||||
| {
|
||||
readonly status: "failed";
|
||||
readonly code: SetupFailureCode;
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
import { mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { copyFile, lstat, mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import writeFileAtomic from "write-file-atomic";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import { clientSkillDirectories } from "./SupportedClients.js";
|
||||
|
||||
const SKILL_FILES = [
|
||||
export const MANAGED_SKILL_FILES = [
|
||||
"SKILL.md",
|
||||
"references/native-and-artifacts.md",
|
||||
"references/javascript-applications.md",
|
||||
@@ -14,6 +17,13 @@ const SKILL_FILES = [
|
||||
"references/evidence-workflows.md",
|
||||
] as const;
|
||||
|
||||
export const isManagedSkillManifest = (content: string): boolean =>
|
||||
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
||||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
||||
|
||||
export const readSkillFile = (path: string): Promise<string> =>
|
||||
readRegularFileText(path, { symlinks: "reject" });
|
||||
|
||||
interface CanonicalSkillFile {
|
||||
readonly content: string;
|
||||
readonly destination: string;
|
||||
@@ -47,9 +57,38 @@ export const skillDestinations = (
|
||||
}),
|
||||
);
|
||||
|
||||
/** Resolve existing REA-owned skill roots using setup's canonical destinations. */
|
||||
export const existingSkillDestinations = async (
|
||||
home: string,
|
||||
environment: Readonly<NodeJS.ProcessEnv> = {},
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
): Promise<readonly SkillDestination[]> => {
|
||||
const existing: SkillDestination[] = [];
|
||||
for (const destination of skillDestinations(
|
||||
home,
|
||||
undefined,
|
||||
environment,
|
||||
platform,
|
||||
)) {
|
||||
try {
|
||||
const root = await lstat(destination.path);
|
||||
if (root.isSymbolicLink() || !root.isDirectory()) continue;
|
||||
const manifest = join(destination.path, "SKILL.md");
|
||||
const content = await readSkillFile(manifest);
|
||||
if (isManagedSkillManifest(content)) existing.push(destination);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
continue;
|
||||
if (cause instanceof NonRegularFileReadError) continue;
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
return existing;
|
||||
};
|
||||
|
||||
const readOptionalText = async (path: string): Promise<string | undefined> => {
|
||||
try {
|
||||
return await readFile(path, "utf8");
|
||||
return await readSkillFile(path);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return undefined;
|
||||
@@ -66,7 +105,7 @@ const canonicalSkillFiles = async (
|
||||
Promise.all(
|
||||
skillDestinations(home, clientIds, environment, platform).flatMap(
|
||||
({ path: root }) =>
|
||||
SKILL_FILES.map(async (relativePath) => {
|
||||
MANAGED_SKILL_FILES.map(async (relativePath) => {
|
||||
const destination = join(root, relativePath);
|
||||
return {
|
||||
destination,
|
||||
@@ -104,6 +143,19 @@ export const canonicalSkillNeedsInstall = async (
|
||||
const writeText = (path: string, content: string): Promise<void> =>
|
||||
writeFileAtomic(path, content, { encoding: "utf8", mode: 0o600 });
|
||||
|
||||
const preserveSkillBackup = async (
|
||||
source: string,
|
||||
destination: string,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
await copyFile(source, destination, fsConstants.COPYFILE_EXCL);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "EEXIST")
|
||||
return;
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const restoreSkillFiles = async (
|
||||
changed: readonly CanonicalSkillFile[],
|
||||
): Promise<void> => {
|
||||
@@ -134,12 +186,12 @@ export const installCanonicalSkill = async (
|
||||
for (const { destination, original } of changed) {
|
||||
await mkdir(dirname(destination), { recursive: true });
|
||||
if (original !== undefined)
|
||||
await writeText(`${destination}.rea.backup`, original);
|
||||
await preserveSkillBackup(destination, `${destination}.rea.backup`);
|
||||
}
|
||||
for (const { destination, content } of changed)
|
||||
await writeText(destination, content);
|
||||
for (const { destination, content } of changed)
|
||||
if ((await readFile(destination, "utf8")) !== content)
|
||||
if ((await readSkillFile(destination)) !== content)
|
||||
throw new Error(`skill readback mismatch: ${destination}`);
|
||||
return "installed";
|
||||
} catch (cause: unknown) {
|
||||
@@ -171,7 +223,7 @@ export const readInstalledSkillIdentity = async (
|
||||
let content: string | undefined;
|
||||
for (const destination of destinations) {
|
||||
try {
|
||||
content = await readFile(join(destination.path, "SKILL.md"), "utf8");
|
||||
content = await readSkillFile(join(destination.path, "SKILL.md"));
|
||||
break;
|
||||
} catch {
|
||||
// Another selected copy may provide metadata; canonical validation below
|
||||
|
||||
@@ -50,6 +50,8 @@ export interface SetupHost {
|
||||
linuxDistribution(): Promise<LinuxDistribution | undefined>;
|
||||
initialSetupState(scope?: DoctorScope): Promise<SetupInitialState>;
|
||||
installHopper(replaceExisting: boolean): Promise<SetupHopperInstallResult>;
|
||||
/** Retry and report any cleanup resources still owned by this setup host. */
|
||||
close?(): Promise<string | undefined>;
|
||||
detectedClients(): Promise<readonly SetupClient[]>;
|
||||
supportedClients(): Promise<readonly SetupClient[]>;
|
||||
configureClient(
|
||||
@@ -155,6 +157,8 @@ export interface SetupOptions {
|
||||
readonly structured: boolean;
|
||||
readonly proposeHopper?: boolean;
|
||||
readonly clientIds?: readonly string[];
|
||||
/** Skill roots to update independently of MCP registrations (maintenance). */
|
||||
readonly skillClientIds?: readonly string[];
|
||||
readonly installSkill?: boolean;
|
||||
readonly readinessScope?: DoctorScope;
|
||||
readonly dryRun?: boolean;
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
withClientServers,
|
||||
type ClientConfigurationDocument,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { copyFile, lstat, readFile, realpath, rm } from "node:fs/promises";
|
||||
import { copyFile, lstat, realpath, rm } from "node:fs/promises";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
||||
import { join } from "node:path";
|
||||
@@ -15,9 +15,15 @@ import writeFileAtomic from "write-file-atomic";
|
||||
import { z } from "zod";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { skillDestinations } from "./SetupSkill.js";
|
||||
import {
|
||||
isManagedSkillManifest,
|
||||
MANAGED_SKILL_FILES,
|
||||
readSkillFile,
|
||||
skillDestinations,
|
||||
} from "./SetupSkill.js";
|
||||
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import {
|
||||
manualRegistrationRemediation,
|
||||
supportedClients,
|
||||
@@ -27,29 +33,32 @@ import {
|
||||
interface ManagedPathStats {
|
||||
readonly uid?: number;
|
||||
isFile?(): boolean;
|
||||
isDirectory?(): boolean;
|
||||
isSymbolicLink(): boolean;
|
||||
}
|
||||
|
||||
/** Injectable filesystem operations used to prove uninstall failure recovery. */
|
||||
export interface UninstallFileSystem {
|
||||
readText(path: string): Promise<string>;
|
||||
readRegularText?(path: string): Promise<string>;
|
||||
/** Preserve the first backup; reject an existing destination with EEXIST. */
|
||||
copy(source: string, destination: string): Promise<void>;
|
||||
writeText(path: string, contents: string): Promise<void>;
|
||||
stat(path: string): Promise<ManagedPathStats>;
|
||||
realpath?(path: string): Promise<string>;
|
||||
remove(path: string): Promise<void>;
|
||||
remove(path: string, recursive?: boolean): Promise<void>;
|
||||
}
|
||||
|
||||
const systemFileSystem: UninstallFileSystem = {
|
||||
readText: (path) => readFile(path, "utf8"),
|
||||
readText: (path) => readRegularFileText(path),
|
||||
readRegularText: readSkillFile,
|
||||
copy: (source, destination) =>
|
||||
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
|
||||
writeText: (path, contents) =>
|
||||
writeFileAtomic(path, contents, { encoding: "utf8" }),
|
||||
stat: (path) => lstat(path),
|
||||
realpath: (path) => realpath(path),
|
||||
remove: (path) => rm(path, { recursive: true }),
|
||||
remove: (path, recursive = true) => rm(path, { recursive }),
|
||||
};
|
||||
|
||||
/** One explicitly classified uninstall action. */
|
||||
@@ -422,7 +431,7 @@ const removeManagedSkills = async (
|
||||
const results = await Promise.all(
|
||||
skillDestinations(home, undefined, environment, platform).map(
|
||||
({ client, path }) =>
|
||||
removeManagedPath(
|
||||
removeManagedSkill(
|
||||
path,
|
||||
client === "claude_code" ? "Claude Code skill" : "skill",
|
||||
fileSystem,
|
||||
@@ -457,6 +466,146 @@ const removeManagedSkills = async (
|
||||
);
|
||||
};
|
||||
|
||||
const removeManagedSkill = async (
|
||||
path: string,
|
||||
name: string,
|
||||
fileSystem: UninstallFileSystem,
|
||||
): Promise<UninstallItem> => {
|
||||
let root: ManagedPathStats;
|
||||
try {
|
||||
root = await fileSystem.stat(path);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", "Skill directory does not exist.")
|
||||
: item(name, "failed", "Skill directory could not be inspected.");
|
||||
}
|
||||
if (root.isSymbolicLink())
|
||||
return item(name, "retained", "Skill directory is a symbolic link.");
|
||||
if (root.isDirectory?.() !== true)
|
||||
return item(name, "retained", "Skill path is not a directory.");
|
||||
|
||||
const manifestPath = join(path, "SKILL.md");
|
||||
try {
|
||||
const manifest = await fileSystem.stat(manifestPath);
|
||||
if (manifest.isSymbolicLink())
|
||||
return item(name, "retained", "Skill manifest is a symbolic link.");
|
||||
if (manifest.isFile?.() !== true)
|
||||
return item(name, "retained", "Skill manifest is not a regular file.");
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "retained", "Skill ownership could not be established.")
|
||||
: item(name, "failed", "Skill manifest could not be inspected.");
|
||||
}
|
||||
let content: string;
|
||||
try {
|
||||
content = await (fileSystem.readRegularText?.(manifestPath) ??
|
||||
fileSystem.readText(manifestPath));
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "retained", "Skill ownership could not be established.")
|
||||
: item(name, "failed", "Skill manifest could not be read.");
|
||||
}
|
||||
if (!isManagedSkillManifest(content))
|
||||
return item(name, "retained", "Skill manifest is not REA-owned.");
|
||||
|
||||
const manifestName = "SKILL.md";
|
||||
const otherFiles = MANAGED_SKILL_FILES.filter(
|
||||
(relativePath) => relativePath !== manifestName,
|
||||
);
|
||||
const results: UninstallItem[] = [];
|
||||
for (const relativePath of otherFiles)
|
||||
results.push(
|
||||
await removeManagedSkillFile(path, relativePath, name, fileSystem),
|
||||
);
|
||||
if (
|
||||
results.some(({ status }) => status === "failed" || status === "retained")
|
||||
)
|
||||
results.push(
|
||||
item(
|
||||
name,
|
||||
"retained",
|
||||
"REA ownership manifest was retained because a managed file could not be safely removed.",
|
||||
),
|
||||
);
|
||||
else
|
||||
results.push(
|
||||
await removeManagedSkillFile(path, manifestName, name, fileSystem),
|
||||
);
|
||||
const failed = results.find(({ status }) => status === "failed");
|
||||
if (failed !== undefined)
|
||||
return item(name, "failed", results.map(({ detail }) => detail).join(" "));
|
||||
const removed = results.some(({ status }) => status === "removed");
|
||||
return item(
|
||||
name,
|
||||
removed ? "removed" : "skipped",
|
||||
"REA-managed skill files were removed; other files were preserved.",
|
||||
);
|
||||
};
|
||||
|
||||
const removeManagedSkillFile = async (
|
||||
root: string,
|
||||
relativePath: string,
|
||||
name: string,
|
||||
fileSystem: UninstallFileSystem,
|
||||
): Promise<UninstallItem> => {
|
||||
const segments = relativePath.split(/[\\/]/u);
|
||||
let parent = root;
|
||||
for (const segment of segments.slice(0, -1)) {
|
||||
parent = join(parent, segment);
|
||||
try {
|
||||
const stats = await fileSystem.stat(parent);
|
||||
if (stats.isSymbolicLink() || stats.isDirectory?.() !== true)
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed path parent was retained: ${parent}.`,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(
|
||||
name,
|
||||
"skipped",
|
||||
`${join(parent, segments.at(-1) ?? "")} does not exist.`,
|
||||
)
|
||||
: item(
|
||||
name,
|
||||
"failed",
|
||||
`Managed path parent could not be inspected: ${parent}.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const target = join(root, relativePath);
|
||||
let stats: ManagedPathStats;
|
||||
try {
|
||||
stats = await fileSystem.stat(target);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", `${target} does not exist.`)
|
||||
: item(name, "failed", `Managed file could not be inspected: ${target}.`);
|
||||
}
|
||||
if (stats.isSymbolicLink())
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed file is a symbolic link: ${target}.`,
|
||||
);
|
||||
if (stats.isFile?.() !== true)
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed path is not a regular file: ${target}.`,
|
||||
);
|
||||
try {
|
||||
await fileSystem.remove(target, false);
|
||||
return item(name, "removed", `Removed ${target}.`);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", `${target} does not exist.`)
|
||||
: item(name, "failed", `Managed file could not be removed: ${target}.`);
|
||||
}
|
||||
};
|
||||
|
||||
const item = (
|
||||
name: string,
|
||||
status: UninstallItem["status"],
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { z } from "zod";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
@@ -12,13 +10,19 @@ import {
|
||||
parseClientConfiguration,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { supportedClients } from "./SupportedClients.js";
|
||||
import { existingSkillDestinations } from "./SetupSkill.js";
|
||||
import type { SetupAction } from "./SetupTypes.js";
|
||||
import type { Result } from "../domain/result.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
|
||||
/** Existing REA integrations selected for maintenance, without discovering new targets. */
|
||||
export interface MaintenanceScope {
|
||||
readonly clients: readonly string[];
|
||||
readonly skill: boolean;
|
||||
readonly skillDestinations?: readonly {
|
||||
readonly client: string;
|
||||
readonly path: string;
|
||||
}[];
|
||||
}
|
||||
|
||||
/** Unapplied maintenance evidence produced by the updated setup executable. */
|
||||
@@ -49,7 +53,7 @@ const setupPlanSchema = z.object({
|
||||
),
|
||||
});
|
||||
|
||||
/** Discover owned registrations and an existing REA skill without writes. */
|
||||
/** Discover owned registrations and REA skills without writes. */
|
||||
export const existingMaintenanceScope = async (
|
||||
home: string,
|
||||
entryPoint: string,
|
||||
@@ -69,7 +73,7 @@ export const existingMaintenanceScope = async (
|
||||
const client = supported.find(({ name }) => name === registration.client);
|
||||
if (client === undefined) continue;
|
||||
const parsed = parseClientConfiguration(
|
||||
await readFile(client.configPath, "utf8"),
|
||||
await readRegularFileText(client.configPath),
|
||||
client.format,
|
||||
);
|
||||
const enabled = z
|
||||
@@ -87,20 +91,12 @@ export const existingMaintenanceScope = async (
|
||||
)
|
||||
clients.push(client.name);
|
||||
}
|
||||
let skill = false;
|
||||
try {
|
||||
const content = await readFile(
|
||||
join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName, "SKILL.md"),
|
||||
"utf8",
|
||||
);
|
||||
skill =
|
||||
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
||||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof Error && "code" in cause && cause.code === "ENOENT"))
|
||||
throw cause;
|
||||
}
|
||||
return { clients, skill };
|
||||
const skillDestinations = await existingSkillDestinations(home, environment);
|
||||
return {
|
||||
clients,
|
||||
skill: skillDestinations.length > 0,
|
||||
skillDestinations,
|
||||
};
|
||||
};
|
||||
|
||||
/** Ask the verified new executable to plan only existing REA integrations. */
|
||||
@@ -112,6 +108,7 @@ export const planIntegrationMaintenance = async (
|
||||
): Promise<IntegrationMaintenance> => {
|
||||
try {
|
||||
const scope = await existingMaintenanceScope(home, entryPoint, environment);
|
||||
const skillDestinations = scope.skillDestinations ?? [];
|
||||
if (scope.clients.length === 0 && !scope.skill)
|
||||
return { status: "current", plannedActions: [] };
|
||||
const command = [
|
||||
@@ -119,6 +116,7 @@ export const planIntegrationMaintenance = async (
|
||||
entryPoint,
|
||||
"setup",
|
||||
...scope.clients.flatMap((client) => ["--client", client]),
|
||||
...skillDestinations.flatMap(({ client }) => ["--skill-client", client]),
|
||||
`--skill=${String(scope.skill)}`,
|
||||
];
|
||||
const execution = await execute([...command, "--dry-run", "--json"]);
|
||||
@@ -132,11 +130,17 @@ export const planIntegrationMaintenance = async (
|
||||
};
|
||||
if (plan.data.plannedActions.length === 0)
|
||||
return { status: "current", plannedActions: [] };
|
||||
const allowedIds = new Set([
|
||||
...scope.clients.map((client) => `configure_client:${client}`),
|
||||
...(scope.skill ? ["install_skill"] : []),
|
||||
]);
|
||||
if (plan.data.plannedActions.some(({ id }) => !allowedIds.has(id)))
|
||||
const allowedIds = new Set(
|
||||
scope.clients.map((client) => `configure_client:${client}`),
|
||||
);
|
||||
const skillTargets = new Set(skillDestinations.map(({ path }) => path));
|
||||
if (
|
||||
plan.data.plannedActions.some((action) =>
|
||||
action.kind === "install_skill"
|
||||
? !skillTargets.has(action.target)
|
||||
: !allowedIds.has(action.id),
|
||||
)
|
||||
)
|
||||
return {
|
||||
status: "unavailable",
|
||||
remediation:
|
||||
|
||||
@@ -27,7 +27,10 @@ import { WebScriptExportError } from "../domain/webScriptExportError.js";
|
||||
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
||||
import { WEB_SCRIPT_EXPORT_PROVIDER as PROVIDER } from "./InvestigationProviders.js";
|
||||
import { readRegularFile } from "./RegularFileRead.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
NonRegularFileReadError,
|
||||
RegularFileChangedError,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
|
||||
const OPERATION = "export_web_scripts";
|
||||
|
||||
@@ -153,7 +156,7 @@ const readCapture = async (
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<Buffer, AnalysisError>> => {
|
||||
try {
|
||||
return ok(await readRegularFile(path, signal));
|
||||
return ok(await readRegularFile(path, { signal }));
|
||||
} catch (cause: unknown) {
|
||||
const code =
|
||||
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
||||
@@ -163,6 +166,7 @@ const readCapture = async (
|
||||
);
|
||||
if (
|
||||
cause instanceof NonRegularFileReadError ||
|
||||
cause instanceof RegularFileChangedError ||
|
||||
code === "ENOENT" ||
|
||||
code === "ENOTDIR" ||
|
||||
code === "EISDIR" ||
|
||||
|
||||
@@ -28,6 +28,7 @@ export interface WebSourceLocationArtifactPort {
|
||||
|
||||
/** Replaceable codec receives map data and one explicit generated point. */
|
||||
export interface WebSourceMapPort {
|
||||
close?(): Promise<void>;
|
||||
trace(
|
||||
input: {
|
||||
readonly text: string;
|
||||
|
||||
@@ -59,6 +59,10 @@ export class WebSourceLocationService {
|
||||
readonly decoder: WebSourceMapPort,
|
||||
) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.decoder.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
|
||||
async trace(
|
||||
rawInput: unknown,
|
||||
|
||||
@@ -14,7 +14,7 @@ import type { Result } from "../../domain/result.js";
|
||||
export interface AdbDeviceAnalysisPort {
|
||||
/** Probe the selected adb binary without contacting devices. */
|
||||
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
|
||||
/** The provider holds no long-lived resources; safe to call always. */
|
||||
/** Terminally stop admission, drain work, and retry retained package rollback. */
|
||||
close(): Promise<void>;
|
||||
execute(
|
||||
request: AdbRequest,
|
||||
|
||||
@@ -5,10 +5,8 @@ import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
|
||||
import { err } from "../../domain/result.js";
|
||||
import type { AnalysisClient } from "../AnalysisProvider.js";
|
||||
import {
|
||||
analysisErrorWithCleanupFailure,
|
||||
closeAnalysisClient,
|
||||
} from "./AnalysisClientCleanup.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
|
||||
const clientClosingWith = (cause: unknown): AnalysisClient => ({
|
||||
execute: () => Promise.resolve(err(new ProviderCleanupError("x", [], {}))),
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
|
||||
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { err, type Result } from "../../domain/result.js";
|
||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { AnalysisClient, ExecutionOptions } from "../AnalysisProvider.js";
|
||||
@@ -29,34 +27,3 @@ export const closeAnalysisClient = async (
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** Preserve the original lifecycle failure alongside incomplete cleanup. */
|
||||
export const analysisErrorWithCleanupFailure = (
|
||||
primary: AnalysisError,
|
||||
cleanup: AnalysisError,
|
||||
operation = "close_binary",
|
||||
): AnalysisError => {
|
||||
const projected = projectAnalysisError(cleanup);
|
||||
return new ProviderAdapterError(
|
||||
cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea",
|
||||
operation,
|
||||
{
|
||||
cause: primary,
|
||||
userMessage: projectAnalysisError(primary).message,
|
||||
...(primary.capturedOutput === undefined
|
||||
? {}
|
||||
: { capturedOutput: primary.capturedOutput }),
|
||||
...(primary.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: primary.partialObservation }),
|
||||
cleanup: {
|
||||
reason: projected.message,
|
||||
resources: cleanup.cleanupResources,
|
||||
},
|
||||
diagnostics: {
|
||||
primary_error: projectAnalysisError(primary),
|
||||
cleanup_error: projected,
|
||||
},
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
@@ -44,10 +44,8 @@ import {
|
||||
commitExecutionProfile,
|
||||
prepareSessionExecution,
|
||||
} from "./BinarySessionExecution.js";
|
||||
import {
|
||||
analysisErrorWithCleanupFailure,
|
||||
closeAnalysisClient,
|
||||
} from "./AnalysisClientCleanup.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
const OFFICIAL_OPERATIONS: ReadonlySet<string> = new Set(
|
||||
OFFICIAL_TOOL_CONTRACTS.map(({ name }) => name),
|
||||
);
|
||||
|
||||
@@ -8,6 +8,7 @@ import type { Result } from "../../domain/result.js";
|
||||
|
||||
/** Replaceable offline interface-recovery engine, independent of wallets and chain clients. */
|
||||
export interface EvmInterfacePort {
|
||||
close?(): Promise<void>;
|
||||
inspect(
|
||||
input: InspectEvmInterfaceInput,
|
||||
options?: ExecutionOptions,
|
||||
|
||||
@@ -25,6 +25,10 @@ const diagnosticsSchema = z.object({
|
||||
export class EvmInterfaceService {
|
||||
constructor(readonly provider: EvmInterfacePort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Preserve selected byte identity while clearly labeling recovered ABI candidates as inferences. */
|
||||
async inspect(
|
||||
rawInput: unknown,
|
||||
|
||||
@@ -8,6 +8,7 @@ import type { Result } from "../../domain/result.js";
|
||||
|
||||
/** File-oriented firmware engines; executable admission remains a separate operation. */
|
||||
export interface FirmwareAnalysisPort {
|
||||
close?(): Promise<void>;
|
||||
execute(
|
||||
request: FirmwareRequest,
|
||||
options?: ExecutionOptions,
|
||||
|
||||
@@ -18,6 +18,10 @@ import { err, ok, type Result } from "../../domain/result.js";
|
||||
export class FirmwareAnalysisService {
|
||||
constructor(readonly provider: FirmwareAnalysisPort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Execute an explicit inspection or extraction and preserve its producing identity. */
|
||||
async execute(
|
||||
operation: FirmwareOperation,
|
||||
|
||||
@@ -2,9 +2,10 @@ import type { ApplicationNode } from "../../domain/javascript/javascriptApplicat
|
||||
import { sha256Text } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
||||
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import type {
|
||||
JavaScriptArtifactGraphContext,
|
||||
JavaScriptArtifactGraphCoverage,
|
||||
import {
|
||||
sourceNodeFor,
|
||||
type JavaScriptArtifactGraphContext,
|
||||
type JavaScriptArtifactGraphCoverage,
|
||||
} from "./JavaScriptArtifactGraphContext.js";
|
||||
import {
|
||||
astObservationEvidence,
|
||||
@@ -38,10 +39,11 @@ export const electronFindingSourceNode = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
moduleKey: string | null,
|
||||
runtime?: string,
|
||||
): ApplicationNode | undefined =>
|
||||
moduleKey === null
|
||||
? (context.assetNodes.get(file.path) ?? context.fileNodes.get(file.path))
|
||||
: context.moduleNodes.get(`${file.path}\0${moduleKey}`);
|
||||
: sourceNodeFor(context, file.path, moduleKey, runtime);
|
||||
|
||||
/** Observation-scoped identity shared safely across different artifact files. */
|
||||
export const electronObservationIdentity = (
|
||||
|
||||
@@ -73,6 +73,7 @@ const addIpcRecord = (
|
||||
context,
|
||||
record.file,
|
||||
record.finding.module_key,
|
||||
record.finding.module_runtime,
|
||||
);
|
||||
if (source === undefined) return;
|
||||
state.sources.set(record.key, source);
|
||||
@@ -255,6 +256,7 @@ const addValidationObservations = (state: IpcGraphState): void => {
|
||||
state.context,
|
||||
file,
|
||||
validation.module_key,
|
||||
validation.module_runtime,
|
||||
)
|
||||
: state.handlers.get(handlerRecord.key);
|
||||
if (target !== undefined)
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
} from "./JavaScriptArtifactGraphContext.js";
|
||||
import { astObservationEvidence } from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import { javascriptDisplayText } from "../../domain/javascript/javascriptAstValues.js";
|
||||
import {
|
||||
addElectronInferenceEdge,
|
||||
electronFindingSourceNode,
|
||||
@@ -37,7 +38,12 @@ export const addElectronNativeBoundaries = (
|
||||
|
||||
const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
const { context, file, value, coverage } = input;
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
if (source === undefined) return;
|
||||
const resolved = resolveArtifactPath(
|
||||
value.specifier,
|
||||
@@ -51,17 +57,27 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
identity: electronObservationIdentity(
|
||||
context,
|
||||
"javascript-requested-native-export",
|
||||
// Members are a variable-length source-derived list: compositeKey keeps
|
||||
// ["a\0b"] distinct from ["a", "b"] inside the digested identity.
|
||||
`${file.path}:${value.specifier}:${value.binding_kind}:${compositeKey(value.members)}:${electronRangeKey(value.location)}`,
|
||||
compositeKey([
|
||||
file.path,
|
||||
value.specifier,
|
||||
value.binding_kind,
|
||||
value.members,
|
||||
value.namespace_access ?? null,
|
||||
value.dynamic_member_access ?? null,
|
||||
electronRangeKey(value.location),
|
||||
]),
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
label: value.members.join(", "),
|
||||
label:
|
||||
value.members.map(javascriptDisplayText).join(", ") ||
|
||||
value.specifier,
|
||||
properties: {
|
||||
specifier: value.specifier,
|
||||
binding_kind: value.binding_kind,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
resolved_path: resolved,
|
||||
native_export_verification: "not-performed",
|
||||
},
|
||||
@@ -90,6 +106,8 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
specifier: value.specifier,
|
||||
binding_kind: value.binding_kind,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
},
|
||||
limitations: [
|
||||
"Requested JavaScript members are not verified native exports.",
|
||||
@@ -107,6 +125,8 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
properties: {
|
||||
resolved_path: resolved,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
verified: false,
|
||||
},
|
||||
confidence: "medium",
|
||||
|
||||
@@ -39,17 +39,32 @@ export const addElectronWindowBoundaries = (
|
||||
if (javascript === null) continue;
|
||||
const coverage = javascriptAnalysisCoverage(javascript);
|
||||
for (const value of javascript.electron.browser_windows) {
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
if (source !== undefined)
|
||||
addBrowserWindow({ context, file, source, value, coverage });
|
||||
}
|
||||
for (const value of javascript.electron.context_bridge_apis) {
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
if (source !== undefined)
|
||||
addContextBridge({ context, file, source, value, coverage });
|
||||
}
|
||||
for (const value of javascript.electron.utility_processes) {
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
if (source !== undefined)
|
||||
addUtilityProcess({ context, file, source, value, coverage });
|
||||
}
|
||||
|
||||
@@ -31,7 +31,16 @@ const graphIdentities = async (source: string) => {
|
||||
...(identity.strategy === "artifact-local-key"
|
||||
? {
|
||||
namespace: identity.namespace,
|
||||
key: identity.key.replace(/^[0-9a-f]{64}:/u, ""),
|
||||
// Module-export identity is a JSON tuple whose first entry
|
||||
// binds the whole container; only that digest changes here.
|
||||
key:
|
||||
identity.namespace === "module-export"
|
||||
? JSON.stringify(
|
||||
(JSON.parse(identity.key) as unknown[]).slice(1),
|
||||
)
|
||||
: identity.namespace === "source-module"
|
||||
? identity.key.replace(/^[0-9a-f]{64}:/u, "")
|
||||
: identity.key,
|
||||
}
|
||||
: {}),
|
||||
}))
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import type { JavaScriptBundlerRegistration } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import {
|
||||
addAstContainsEdge,
|
||||
artifactLocalIdentity,
|
||||
@@ -80,7 +81,11 @@ const createBundlerChunkNode = (
|
||||
const chunkKey = `${registration.runtime}:${registration.chunk_keys.join(",")}`;
|
||||
return context.accumulator.addNode({
|
||||
kind: "javascript-chunk",
|
||||
identity: artifactLocalIdentity(file.sha256, "bundler-chunk", chunkKey),
|
||||
identity: artifactLocalIdentity(
|
||||
file.sha256,
|
||||
"bundler-chunk",
|
||||
compositeKey([registration.runtime, registration.chunk_keys]),
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
label: chunkKey,
|
||||
@@ -156,7 +161,7 @@ const addBundlerModuleNodes = (
|
||||
],
|
||||
});
|
||||
context.moduleNodes.set(
|
||||
moduleLookupKey(file.path, moduleValue.module_key),
|
||||
moduleLookupKey(file.path, registration.runtime, moduleValue.module_key),
|
||||
module,
|
||||
);
|
||||
addAstContainsEdge(context, {
|
||||
@@ -190,7 +195,7 @@ const addBundlerRuntimeEdges = (
|
||||
const { file, registration, chunk, coverage } = record;
|
||||
for (const moduleKey of registration.entry_module_keys) {
|
||||
const resolved = context.moduleNodes.get(
|
||||
moduleLookupKey(file.path, moduleKey),
|
||||
moduleLookupKey(file.path, registration.runtime, moduleKey),
|
||||
);
|
||||
const target =
|
||||
resolved ?? unresolvedBundlerModuleNode(context, record, moduleKey);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import {
|
||||
completeApplicationCoverage,
|
||||
partialApplicationCoverage,
|
||||
@@ -322,15 +323,18 @@ export const selfReferenceOmissions = (
|
||||
];
|
||||
|
||||
/** Deterministic key for one recovered module inside a bundle asset. */
|
||||
export const moduleLookupKey = (path: string, moduleKey: string): string =>
|
||||
`${path}\0${moduleKey}`;
|
||||
export const moduleLookupKey = (
|
||||
path: string,
|
||||
runtime: string,
|
||||
moduleKey: string,
|
||||
): string => compositeKey([path, runtime, moduleKey]);
|
||||
|
||||
/** Deterministic key for one recovered bundler chunk inside a bundle asset. */
|
||||
export const chunkLookupKey = (
|
||||
path: string,
|
||||
runtime: string,
|
||||
chunkKey: string,
|
||||
): string => `${path}\0${runtime}\0${chunkKey}`;
|
||||
): string => compositeKey([path, runtime, chunkKey]);
|
||||
|
||||
/** Resolve a relative static specifier without escaping the artifact root. */
|
||||
export const resolveArtifactPath = (
|
||||
@@ -352,7 +356,10 @@ export const sourceNodeFor = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
path: string,
|
||||
moduleKey: string | null,
|
||||
runtime?: string,
|
||||
): ApplicationNode | undefined =>
|
||||
moduleKey === null
|
||||
? context.sourceModuleNodes.get(path)
|
||||
: context.moduleNodes.get(moduleLookupKey(path, moduleKey));
|
||||
: runtime === undefined
|
||||
? undefined
|
||||
: context.moduleNodes.get(moduleLookupKey(path, runtime, moduleKey));
|
||||
|
||||
@@ -78,8 +78,12 @@ const addReference = (
|
||||
omissions: { selfReferences: number },
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
const resolved =
|
||||
input.value.specifier === null
|
||||
? null
|
||||
@@ -162,8 +166,12 @@ const addEndpoint = (
|
||||
input: FindingInput<StaticEndpoint>,
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
const key = compactStringIdentityKey(input.value.value);
|
||||
const label = `${input.value.kind} endpoint`;
|
||||
const node = context.accumulator.addNode({
|
||||
@@ -204,8 +212,12 @@ const addStorage = (
|
||||
input: FindingInput<StaticStorage>,
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
const node = context.accumulator.addNode({
|
||||
kind: "storage",
|
||||
identity: artifactLocalIdentity(
|
||||
@@ -262,8 +274,12 @@ const addDiscoveredRole = (
|
||||
coverage: input.coverage,
|
||||
});
|
||||
const source =
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
addStaticInferenceEdge(context, {
|
||||
source,
|
||||
target: role,
|
||||
@@ -359,9 +375,12 @@ const resolveReference = (
|
||||
if (specifier === null) return null;
|
||||
const chunk = resolveBundlerChunkReference(context, sourcePath, reference);
|
||||
if (chunk !== null) return chunk;
|
||||
const module = context.moduleNodes.get(
|
||||
moduleLookupKey(sourcePath, specifier),
|
||||
);
|
||||
const module =
|
||||
reference.module_runtime === undefined
|
||||
? undefined
|
||||
: context.moduleNodes.get(
|
||||
moduleLookupKey(sourcePath, reference.module_runtime, specifier),
|
||||
);
|
||||
if (module !== undefined) {
|
||||
const file = context.filesByPath.get(sourcePath);
|
||||
return file === undefined
|
||||
@@ -407,10 +426,11 @@ const resolveBundlerChunkReference = (
|
||||
if (file === undefined) return null;
|
||||
const registrations = context.analysis.files
|
||||
.find(({ file: analyzedFile }) => analyzedFile.path === sourcePath)
|
||||
?.javascript?.bundler_registrations.filter(({ modules }) =>
|
||||
reference.module_key === null
|
||||
? true
|
||||
: modules.some(
|
||||
?.javascript?.bundler_registrations.filter(({ runtime, modules }) =>
|
||||
reference.module_runtime === undefined
|
||||
? reference.module_key === null
|
||||
: runtime === reference.module_runtime &&
|
||||
modules.some(
|
||||
({ module_key: moduleKey }) => moduleKey === reference.module_key,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import { posix } from "node:path";
|
||||
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
import {
|
||||
compareUnicodeCodePoints,
|
||||
compositeKey,
|
||||
} from "../../domain/unicodeCodePointOrder.js";
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import {
|
||||
completeApplicationCoverage,
|
||||
@@ -177,7 +180,7 @@ const addExportRelationship = (
|
||||
identity: artifactLocalIdentity(
|
||||
file.sha256,
|
||||
"module-export",
|
||||
`${file.container_sha256}:${file.path}:${link.exportedName}`,
|
||||
compositeKey([file.container_sha256, file.path, link.exportedName]),
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
|
||||
@@ -8,6 +8,7 @@ import type {
|
||||
|
||||
/** Typed boundary for upstream source recovery without exposing its protocol. */
|
||||
export interface JavaScriptRecoveryPort {
|
||||
close?(): Promise<void>;
|
||||
recover(
|
||||
input: JavaScriptRecoveryInput,
|
||||
options?: ExecutionOptions,
|
||||
|
||||
@@ -23,6 +23,10 @@ const OPERATION = "recover_javascript_sources";
|
||||
export class JavaScriptRecoveryService {
|
||||
constructor(readonly provider: JavaScriptRecoveryPort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Recover sources and bind the result to its original immutable input. */
|
||||
async recover(
|
||||
input: unknown,
|
||||
|
||||
@@ -325,8 +325,7 @@ const resolveUiAction = (
|
||||
(node) =>
|
||||
["control", "view", "view_controller", "scene"].includes(node.kind) &&
|
||||
(node.id === query ||
|
||||
node.attributes.interface_builder_object_id === query ||
|
||||
node.id.endsWith(`:object:${query}`)),
|
||||
node.attributes.interface_builder_object_id === query),
|
||||
);
|
||||
const actionIds = new Set(
|
||||
graph.edges.flatMap((edge) =>
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { z } from "zod";
|
||||
|
||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
@@ -17,9 +16,10 @@ import { compareProcessCaptures } from "../../domain/process/processComparison.j
|
||||
import { parseProcessCapture } from "../../domain/process/processCaptureParsing.js";
|
||||
import { createProgressReporter } from "../ProgressReporter.js";
|
||||
import { captureProcessScenario } from "../../process/capture/ProcessHarness.js";
|
||||
import { ProcessCaptureResourceScope } from "../../process/capture/ProcessCaptureLifecycle.js";
|
||||
import { PROCESS_PROVIDER } from "../../domain/process/processEvidenceProvider.js";
|
||||
import { createProcessCaptureEvidence } from "./ProcessEvidence.js";
|
||||
import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../Utf8JsonInput.js";
|
||||
import { readJsonFile } from "../JsonFiles.js";
|
||||
|
||||
/** Safe process-command failure returned to the CLI adapter. */
|
||||
export interface ProcessCliErrorOutput {
|
||||
@@ -63,6 +63,7 @@ export const captureProcessScenarioFile = async (
|
||||
},
|
||||
{ minimumIntervalMs: 100 },
|
||||
);
|
||||
const resourceScope = new ProcessCaptureResourceScope();
|
||||
const captured = await captureProcessScenario(
|
||||
parsed.data,
|
||||
signal,
|
||||
@@ -71,7 +72,14 @@ export const captureProcessScenarioFile = async (
|
||||
undefined,
|
||||
undefined,
|
||||
progress,
|
||||
resourceScope,
|
||||
);
|
||||
try {
|
||||
await resourceScope.close();
|
||||
} catch (cause: unknown) {
|
||||
if (!captured.ok) return cliAnalysisError(captured.error);
|
||||
throw cause;
|
||||
}
|
||||
if (!captured.ok) return cliAnalysisError(captured.error);
|
||||
return createProcessCaptureEvidence(parsed.data, captured.value);
|
||||
} catch (cause: unknown) {
|
||||
@@ -178,30 +186,16 @@ const invalidCaptureEvidence = (cause: unknown): ProcessCliFailure =>
|
||||
);
|
||||
|
||||
const readJson = async (path: string): Promise<unknown> => {
|
||||
let bytes;
|
||||
try {
|
||||
bytes = await readFile(path);
|
||||
} catch (cause: unknown) {
|
||||
throw new ProcessCliFailure(
|
||||
"invalid_input",
|
||||
`Process input file could not be read: ${path} (${describeValidationFailure(cause)}). Check the reported filesystem constraint and retry.`,
|
||||
);
|
||||
}
|
||||
try {
|
||||
const text = new TextDecoder("utf-8", {
|
||||
fatal: true,
|
||||
ignoreBOM: true,
|
||||
}).decode(bytes);
|
||||
if (text.startsWith("\uFEFF"))
|
||||
throw new Error(JSON_BYTE_ORDER_MARK_MESSAGE);
|
||||
const parsed: unknown = JSON.parse(text);
|
||||
return parsed;
|
||||
} catch (cause: unknown) {
|
||||
throw new ProcessCliFailure(
|
||||
"invalid_input",
|
||||
`Process input file is not valid UTF-8 JSON: ${path} (${describeValidationFailure(cause)}). Repair the file, then try again.`,
|
||||
);
|
||||
}
|
||||
const loaded = await readJsonFile(path);
|
||||
if (loaded.ok) return loaded.value;
|
||||
if (loaded.error._tag !== "EvidenceFileError") throw loaded.error;
|
||||
const detail = describeValidationFailure(loaded.error.cause);
|
||||
throw new ProcessCliFailure(
|
||||
"invalid_input",
|
||||
loaded.error.reason === "invalid-json"
|
||||
? `Process input file is not valid UTF-8 JSON: ${path} (${detail}). Repair the file, then try again.`
|
||||
: `Process input file could not be read: ${path} (${detail}). Check the reported filesystem constraint and retry.`,
|
||||
);
|
||||
};
|
||||
|
||||
class ProcessCliFailure extends Error {
|
||||
|
||||
@@ -60,6 +60,30 @@ describe("English Unicode case collisions", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("retains every spelling in a high-cardinality folded sibling group", () => {
|
||||
const count = 8_192;
|
||||
const variants = caseVariants("abcdefghijklm", count);
|
||||
const { limitation, byPath } = annotate(variants);
|
||||
const ordered = [...variants].sort();
|
||||
const first = ordered[0];
|
||||
const second = ordered[1];
|
||||
const original = variants[0];
|
||||
if (first === undefined || second === undefined || original === undefined) {
|
||||
throw new Error("Expected at least two case variants");
|
||||
}
|
||||
expect(limitation).toBe(ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION);
|
||||
expect(byPath.size).toBe(count);
|
||||
expect(byPath.get(first)).toEqual([
|
||||
`Logical path ${first} collides under English (en-US) Unicode case folding with ${second} and ${String(count - 2)} other spellings; ${NOTE}`,
|
||||
]);
|
||||
expect(byPath.get(original)?.[0]).toContain(
|
||||
`with ${first} and ${String(count - 2)} other spellings;`,
|
||||
);
|
||||
expect(
|
||||
[...byPath.values()].every((limitations) => limitations.length === 1),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("stays linear for large case-variant subtrees", () => {
|
||||
const count = 5_000;
|
||||
const paths = Array.from({ length: count }, (_, index) => [
|
||||
@@ -81,3 +105,12 @@ describe("English Unicode case collisions", () => {
|
||||
expect(byPath.get(deep)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
const caseVariants = (lowercase: string, count: number): string[] =>
|
||||
Array.from({ length: count }, (_, mask) =>
|
||||
[...lowercase]
|
||||
.map((character, index) =>
|
||||
(mask & (1 << index)) === 0 ? character : character.toUpperCase(),
|
||||
)
|
||||
.join(""),
|
||||
);
|
||||
|
||||
@@ -177,12 +177,12 @@ const englishUnicodeCaseCollisions = (
|
||||
if (path.length === 0 || path === ".") continue;
|
||||
let node = root;
|
||||
for (const part of path.split("/")) {
|
||||
const folded = englishCaseFold(part);
|
||||
const spellings = node.spellingsByFold.get(folded);
|
||||
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
|
||||
else if (!spellings.includes(part)) spellings.push(part);
|
||||
let child = node.bySpelling.get(part);
|
||||
if (child === undefined) {
|
||||
const folded = englishCaseFold(part);
|
||||
const spellings = node.spellingsByFold.get(folded);
|
||||
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
|
||||
else spellings.push(part);
|
||||
child = emptyCaseNode();
|
||||
node.bySpelling.set(part, child);
|
||||
}
|
||||
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js";
|
||||
import { admitAsarHeader } from "./AsarHeader.js";
|
||||
|
||||
/**
|
||||
* Official Electron ASAR adapter with range-streamed member reads.
|
||||
@@ -57,6 +58,17 @@ export class AsarArtifactReader implements ArtifactReader {
|
||||
const snapshotPath = this.#snapshotPath();
|
||||
this.#resetArchiveState();
|
||||
this.#entries.clear();
|
||||
const headerHandle = await open(
|
||||
snapshotPath,
|
||||
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
||||
);
|
||||
try {
|
||||
const admitted = await admitAsarHeader(headerHandle);
|
||||
this.#archiveSize = admitted.archiveBytes;
|
||||
} finally {
|
||||
await headerHandle.close();
|
||||
}
|
||||
abortIfNeeded(signal);
|
||||
paths = listPackage(snapshotPath, { isPack: false }).sort((left, right) =>
|
||||
left.localeCompare(right, "en"),
|
||||
);
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||
|
||||
/** Shared resource budget for the eagerly decoded ASAR header, not member bytes. */
|
||||
export const MAX_ASAR_HEADER_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
/** Admit both Chromium pickle extents before an upstream decoder allocates them. */
|
||||
export const admitAsarHeader = async (
|
||||
handle: FileHandle,
|
||||
): Promise<{
|
||||
readonly archiveBytes: number;
|
||||
readonly headerBytes: number;
|
||||
readonly jsonBytes: number;
|
||||
}> => {
|
||||
const observed = await handle.stat();
|
||||
if (!observed.isFile())
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"ASAR container is not a regular file",
|
||||
);
|
||||
const prefix = Buffer.alloc(16);
|
||||
const read = await handle.read(prefix, 0, prefix.length, 0);
|
||||
if (read.bytesRead !== prefix.length || prefix.readUInt32LE(0) !== 4)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"Truncated or invalid ASAR size pickle",
|
||||
);
|
||||
const headerBytes = prefix.readUInt32LE(4);
|
||||
const payloadBytes = prefix.readUInt32LE(8);
|
||||
const jsonBytes = prefix.readUInt32LE(12);
|
||||
if (
|
||||
headerBytes < 8 ||
|
||||
headerBytes > observed.size - 8 ||
|
||||
payloadBytes !== headerBytes - 4 ||
|
||||
jsonBytes > payloadBytes - 4
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
"ASAR header leaves its container or has invalid pickle extents",
|
||||
);
|
||||
if (headerBytes > MAX_ASAR_HEADER_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
`ASAR header exceeds the ${String(MAX_ASAR_HEADER_BYTES)} byte decode budget`,
|
||||
);
|
||||
return { archiveBytes: observed.size, headerBytes, jsonBytes };
|
||||
};
|
||||
@@ -3,15 +3,13 @@ import { lstat, open, opendir, realpath } from "node:fs/promises";
|
||||
import { isAbsolute, join, relative, sep } from "node:path";
|
||||
import type { Readable } from "node:stream";
|
||||
|
||||
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
|
||||
|
||||
import {
|
||||
ArtifactReaderFailure,
|
||||
type ArtifactEntry,
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
|
||||
/** Deterministic, non-symlink-following directory reader. */
|
||||
/** Non-symlink-following directory reader. */
|
||||
export class DirectoryArtifactReader implements ArtifactReader {
|
||||
readonly format = "directory" as const;
|
||||
readonly #rootPromise: Promise<string>;
|
||||
@@ -38,13 +36,52 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
||||
await withDirectoryIoContext("inspect directory", directory, () =>
|
||||
assertContainedDirectory(root, directory),
|
||||
);
|
||||
const children: string[] = [];
|
||||
const handle = await opendir(directory).catch((cause: unknown) => {
|
||||
throw directoryIoFailure("open directory", directory, cause);
|
||||
});
|
||||
const directories: string[] = [];
|
||||
try {
|
||||
for await (const child of handle) children.push(child.name);
|
||||
for await (const child of handle) {
|
||||
abortIfNeeded(signal);
|
||||
const absolute = join(directory, child.name);
|
||||
const metadata = await lstat(absolute).catch((cause: unknown) => {
|
||||
throw directoryIoFailure("inspect entry", absolute, cause);
|
||||
});
|
||||
const path = relative(root, absolute).split(sep).join("/");
|
||||
const kind = metadata.isSymbolicLink()
|
||||
? "symlink"
|
||||
: metadata.isDirectory()
|
||||
? "directory"
|
||||
: metadata.isFile()
|
||||
? "file"
|
||||
: undefined;
|
||||
if (kind === undefined) continue;
|
||||
yield {
|
||||
path,
|
||||
kind,
|
||||
declaredSize: kind === "file" ? metadata.size : null,
|
||||
compressedSize: null,
|
||||
executable: (metadata.mode & 0o111) !== 0,
|
||||
encrypted: false,
|
||||
byteOffset: null,
|
||||
declaredSha256: null,
|
||||
unpacked: false,
|
||||
limitations:
|
||||
kind === "symlink"
|
||||
? ["Symlink target was not followed or disclosed."]
|
||||
: [],
|
||||
adapterKey: absolute,
|
||||
...(kind === "file"
|
||||
? {
|
||||
sourceIdentity: { device: metadata.dev, inode: metadata.ino },
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
if (kind === "directory" && (descendInto?.(path) ?? true))
|
||||
directories.push(absolute);
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof ArtifactReaderFailure) throw cause;
|
||||
throw directoryIoFailure("read directory", directory, cause);
|
||||
} finally {
|
||||
// best-effort cleanup: directory-handle close must not mask traversal.
|
||||
@@ -53,45 +90,6 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
||||
await withDirectoryIoContext("inspect directory", directory, () =>
|
||||
assertContainedDirectory(root, directory),
|
||||
);
|
||||
children.sort(compareUnicodeCodePoints);
|
||||
const directories: string[] = [];
|
||||
for (const name of children) {
|
||||
abortIfNeeded(signal);
|
||||
const absolute = join(directory, name);
|
||||
const metadata = await lstat(absolute).catch((cause: unknown) => {
|
||||
throw directoryIoFailure("inspect entry", absolute, cause);
|
||||
});
|
||||
const path = relative(root, absolute).split(sep).join("/");
|
||||
const kind = metadata.isSymbolicLink()
|
||||
? "symlink"
|
||||
: metadata.isDirectory()
|
||||
? "directory"
|
||||
: metadata.isFile()
|
||||
? "file"
|
||||
: undefined;
|
||||
if (kind === undefined) continue;
|
||||
if (kind === "directory" && (descendInto?.(path) ?? true))
|
||||
directories.push(absolute);
|
||||
yield {
|
||||
path,
|
||||
kind,
|
||||
declaredSize: kind === "file" ? metadata.size : null,
|
||||
compressedSize: null,
|
||||
executable: (metadata.mode & 0o111) !== 0,
|
||||
encrypted: false,
|
||||
byteOffset: null,
|
||||
declaredSha256: null,
|
||||
unpacked: false,
|
||||
limitations:
|
||||
kind === "symlink"
|
||||
? ["Symlink target was not followed or disclosed."]
|
||||
: [],
|
||||
adapterKey: absolute,
|
||||
...(kind === "file"
|
||||
? { sourceIdentity: { device: metadata.dev, inode: metadata.ino } }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
directories.reverse();
|
||||
pending.push(...directories);
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { createReadStream } from "node:fs";
|
||||
import { open as openFile, stat } from "node:fs/promises";
|
||||
import { lstat } from "node:fs/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
import { Readable } from "node:stream";
|
||||
|
||||
import type { ArtifactCommand } from "../domain/artifactGraph.js";
|
||||
@@ -19,11 +19,21 @@ import {
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
import { streamChunkToBuffer } from "./StreamBytes.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
type StableRegularFileDescriptor,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
|
||||
/** Read-only universal Mach-O slice reader backed by native lipo metadata. */
|
||||
export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
readonly format = "file" as const;
|
||||
#command: ArtifactCommand | undefined;
|
||||
#source: StableRegularFileDescriptor | undefined;
|
||||
#pendingSource: Promise<StableRegularFileDescriptor> | undefined;
|
||||
readonly #ownsSource: boolean;
|
||||
#closed = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
|
||||
constructor(
|
||||
private readonly path: string,
|
||||
@@ -31,9 +41,15 @@ export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
private readonly runner: NativeCommandRunner = new XcrunCommandRunner(
|
||||
environment,
|
||||
),
|
||||
) {}
|
||||
source?: StableRegularFileDescriptor,
|
||||
) {
|
||||
this.#source = source;
|
||||
this.#ownsSource = source === undefined;
|
||||
}
|
||||
|
||||
async *entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry> {
|
||||
const source = await this.#ensureSource();
|
||||
await this.#verifySource(source);
|
||||
const captured = await this.runner.run(
|
||||
"lipo",
|
||||
["-detailed_info", this.path],
|
||||
@@ -53,9 +69,11 @@ export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
exit_code: captured.value.exitCode,
|
||||
effects: ["read"],
|
||||
};
|
||||
const fileSize = (await stat(this.path)).size;
|
||||
await this.#verifySource(source);
|
||||
const fileSize = source.initial.size;
|
||||
const architectures = parseLipoArchitectures(captured.value.stdout);
|
||||
const structural = await readStructuralSlices(this.path, fileSize);
|
||||
const structural = await readStructuralSlices(source.handle, fileSize);
|
||||
await this.#verifySource(source);
|
||||
if (structural.status === "malformed")
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
@@ -168,22 +186,28 @@ export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
"integrity",
|
||||
"Invalid Mach-O slice byte range",
|
||||
);
|
||||
const fileSize = (await stat(this.path)).size;
|
||||
if (!Number.isSafeInteger(offset + size) || offset + size > fileSize)
|
||||
const source = await this.#ensureSource();
|
||||
await this.#verifySource(source);
|
||||
if (
|
||||
!Number.isSafeInteger(offset + size) ||
|
||||
offset + size > source.initial.size
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Mach-O slice range is outside the artifact: ${entry.path}`,
|
||||
);
|
||||
const source = createReadStream(this.path, {
|
||||
const sourceStream = source.handle.createReadStream({
|
||||
start: offset,
|
||||
end: offset + size - 1,
|
||||
autoClose: false,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
const verifySource = () => this.#verifySource(source);
|
||||
return Readable.from(
|
||||
(async function* () {
|
||||
let observedBytes = 0;
|
||||
try {
|
||||
for await (const raw of source) {
|
||||
for await (const raw of sourceStream) {
|
||||
const chunk = streamChunkToBuffer(raw);
|
||||
observedBytes += chunk.byteLength;
|
||||
if (observedBytes > size)
|
||||
@@ -212,6 +236,7 @@ export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
"integrity",
|
||||
`Mach-O slice size disagrees with lipo metadata: ${entry.path}`,
|
||||
);
|
||||
await verifySource();
|
||||
})(),
|
||||
);
|
||||
}
|
||||
@@ -220,28 +245,80 @@ export class MachOSliceArtifactReader implements ArtifactReader {
|
||||
return this.#command === undefined ? [] : [structuredClone(this.#command)];
|
||||
}
|
||||
|
||||
close(): Promise<void> {
|
||||
return Promise.resolve();
|
||||
async close(): Promise<void> {
|
||||
this.#closed = true;
|
||||
this.#closePromise ??= this.#closeSource().catch((cause: unknown) => {
|
||||
this.#closePromise = undefined;
|
||||
throw cause;
|
||||
});
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
async #ensureSource(): Promise<StableRegularFileDescriptor> {
|
||||
if (this.#closed)
|
||||
throw new ArtifactReaderFailure(
|
||||
"unavailable",
|
||||
"Universal Mach-O reader is closed",
|
||||
);
|
||||
if (this.#source !== undefined) return this.#source;
|
||||
this.#pendingSource ??= this.#acquireSource();
|
||||
const source = await this.#pendingSource;
|
||||
if (this.#closed)
|
||||
throw new ArtifactReaderFailure(
|
||||
"unavailable",
|
||||
"Universal Mach-O reader closed during source acquisition",
|
||||
);
|
||||
return source;
|
||||
}
|
||||
|
||||
async #acquireSource(): Promise<StableRegularFileDescriptor> {
|
||||
const handle = await openRegularFile(this.path, { symlinks: "reject" });
|
||||
try {
|
||||
const initial = await handle.stat();
|
||||
const source = { handle, initial };
|
||||
this.#source = source;
|
||||
return source;
|
||||
} catch (cause: unknown) {
|
||||
await handle.close().catch(() => undefined);
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
async #closeSource(): Promise<void> {
|
||||
await this.#pendingSource?.catch(() => undefined);
|
||||
if (!this.#ownsSource || this.#source === undefined) return;
|
||||
const source = this.#source;
|
||||
await source.handle.close();
|
||||
this.#source = undefined;
|
||||
}
|
||||
|
||||
async #verifySource(source: StableRegularFileDescriptor): Promise<void> {
|
||||
const [opened, currentPath] = await Promise.all([
|
||||
source.handle.stat(),
|
||||
lstat(this.path),
|
||||
]);
|
||||
if (
|
||||
!sameRegularFileState(source.initial, opened) ||
|
||||
!sameRegularFileState(source.initial, currentPath)
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Universal Mach-O source changed during inventory: ${this.path}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Compare lipo's independent ranges with structurally parsed slice identity. */
|
||||
const readStructuralSlices = async (
|
||||
path: string,
|
||||
file: FileHandle,
|
||||
size: number,
|
||||
): Promise<Awaited<ReturnType<typeof readMachoImage>>> => {
|
||||
const file = await openFile(path, "r");
|
||||
try {
|
||||
const readAt: ReadAt = async (offset, length) => {
|
||||
const bytes = Buffer.alloc(length);
|
||||
const { bytesRead } = await file.read(bytes, 0, length, offset);
|
||||
return bytes.subarray(0, bytesRead);
|
||||
};
|
||||
const facts = await readMachoImage(readAt, size);
|
||||
return facts;
|
||||
} finally {
|
||||
await file.close();
|
||||
}
|
||||
const readAt: ReadAt = async (offset, length) => {
|
||||
const bytes = Buffer.alloc(length);
|
||||
const { bytesRead } = await file.read(bytes, 0, length, offset);
|
||||
return bytes.subarray(0, bytesRead);
|
||||
};
|
||||
return readMachoImage(readAt, size);
|
||||
};
|
||||
|
||||
const lipoMatchesSlice = (
|
||||
|
||||
@@ -105,7 +105,26 @@ export class SafeOutputTree {
|
||||
}
|
||||
return new SafeOutputTree(canonicalOutput, platform);
|
||||
} catch (cause: unknown) {
|
||||
await rm(canonicalOutput, { recursive: true, force: true });
|
||||
let removalFailure: unknown;
|
||||
try {
|
||||
await rm(canonicalOutput, { recursive: true, force: true });
|
||||
} catch (cleanupCause: unknown) {
|
||||
removalFailure = cleanupCause;
|
||||
}
|
||||
let absent = false;
|
||||
try {
|
||||
absent = await isAbsent(canonicalOutput);
|
||||
} catch (cleanupCause: unknown) {
|
||||
removalFailure ??= cleanupCause;
|
||||
}
|
||||
if (!absent)
|
||||
throw ArtifactReaderFailure.withCleanup(cause, {
|
||||
reason:
|
||||
removalFailure instanceof Error
|
||||
? removalFailure.message
|
||||
: "Extraction output root remains after setup failure",
|
||||
resources: [canonicalOutput],
|
||||
});
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
@@ -122,56 +141,84 @@ export class SafeOutputTree {
|
||||
async write(
|
||||
relativePath: string,
|
||||
source: Readable,
|
||||
expectedSha256: string,
|
||||
expected: { readonly sha256: string; readonly bytes: number },
|
||||
signal?: AbortSignal,
|
||||
): Promise<SafeOutputFile> {
|
||||
this.#assertWritable();
|
||||
const path = normalizeArtifactPath(relativePath);
|
||||
this.#registry.add(path, "file");
|
||||
const destination = await this.#prepareParent(path);
|
||||
const handle = await open(
|
||||
destination,
|
||||
constants.O_CREAT |
|
||||
constants.O_EXCL |
|
||||
constants.O_WRONLY |
|
||||
constants.O_NOFOLLOW,
|
||||
0o600,
|
||||
).catch(async (cause: unknown) => {
|
||||
await throwIfDestinationCaseCollision(dirname(destination), path, cause);
|
||||
throw new ArtifactReaderFailure(
|
||||
"path",
|
||||
`Could not exclusively create extraction path: ${path}`,
|
||||
{ cause },
|
||||
);
|
||||
});
|
||||
const hash = createHash("sha256");
|
||||
let bytes = 0;
|
||||
try {
|
||||
for await (const raw of source) {
|
||||
abortIfNeeded(signal);
|
||||
const chunk = streamChunkToBuffer(raw);
|
||||
bytes += chunk.length;
|
||||
hash.update(chunk);
|
||||
await writeAll(handle, chunk);
|
||||
this.#assertWritable();
|
||||
if (!Number.isSafeInteger(expected.bytes) || expected.bytes < 0)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`Invalid expected extraction size for ${relativePath}`,
|
||||
);
|
||||
const path = normalizeArtifactPath(relativePath);
|
||||
this.#registry.add(path, "file");
|
||||
const destination = await this.#prepareParent(path);
|
||||
const handle = await open(
|
||||
destination,
|
||||
constants.O_CREAT |
|
||||
constants.O_EXCL |
|
||||
constants.O_WRONLY |
|
||||
constants.O_NOFOLLOW,
|
||||
0o600,
|
||||
).catch(async (cause: unknown) => {
|
||||
await throwIfDestinationCaseCollision(
|
||||
dirname(destination),
|
||||
path,
|
||||
cause,
|
||||
);
|
||||
throw new ArtifactReaderFailure(
|
||||
"path",
|
||||
`Could not exclusively create extraction path: ${path}`,
|
||||
{ cause },
|
||||
);
|
||||
});
|
||||
const hash = createHash("sha256");
|
||||
let bytes = 0;
|
||||
try {
|
||||
for await (const raw of source) {
|
||||
abortIfNeeded(signal);
|
||||
const chunk = streamChunkToBuffer(raw);
|
||||
if (chunk.length > expected.bytes - bytes)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Extracted content exceeds the inventoried size: ${path}`,
|
||||
);
|
||||
bytes += chunk.length;
|
||||
hash.update(chunk);
|
||||
await writeAll(handle, chunk);
|
||||
}
|
||||
if (bytes !== expected.bytes)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Extracted content size disagrees with inventory: ${path}`,
|
||||
);
|
||||
const sha256 = hash.digest("hex");
|
||||
if (sha256 !== expected.sha256)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Extracted content disagrees with inventory: ${path}`,
|
||||
);
|
||||
await handle.sync();
|
||||
await handle.close();
|
||||
const readback = await hashFile(destination, bytes, signal);
|
||||
if (readback.sha256 !== sha256 || readback.bytes !== bytes)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Durable readback verification failed: ${path}`,
|
||||
);
|
||||
return { relativePath: path, sha256, bytesWritten: bytes };
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: file-handle close must not mask the write failure.
|
||||
await handle.close().catch(() => undefined);
|
||||
throw cause;
|
||||
}
|
||||
const sha256 = hash.digest("hex");
|
||||
if (sha256 !== expectedSha256)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Extracted content disagrees with inventory: ${path}`,
|
||||
);
|
||||
await handle.sync();
|
||||
await handle.close();
|
||||
const readback = await hashFile(destination, bytes, signal);
|
||||
if (readback.sha256 !== sha256 || readback.bytes !== bytes)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`Durable readback verification failed: ${path}`,
|
||||
);
|
||||
return { relativePath: path, sha256, bytesWritten: bytes };
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: file-handle close must not mask the write failure.
|
||||
await handle.close().catch(() => undefined);
|
||||
try {
|
||||
source.destroy();
|
||||
} catch {
|
||||
// Preserve the write refusal or failure as the caller-visible error.
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { open, type FileHandle } from "node:fs/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
import { PassThrough, type Readable } from "node:stream";
|
||||
import { once } from "node:events";
|
||||
|
||||
@@ -10,20 +10,57 @@ import {
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
|
||||
import {
|
||||
NonRegularFileReadError,
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
type StableRegularFileDescriptor,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
|
||||
class NodeFileReader extends Reader<string> {
|
||||
#handle: FileHandle | undefined;
|
||||
#initPromise: Promise<void> | undefined;
|
||||
readonly #ownsHandle: boolean;
|
||||
|
||||
constructor(
|
||||
private readonly path: string,
|
||||
private readonly maximumReadBytes?: number,
|
||||
private readonly admitted?: StableRegularFileDescriptor,
|
||||
) {
|
||||
super(path);
|
||||
this.#ownsHandle = admitted === undefined;
|
||||
}
|
||||
|
||||
override async init(): Promise<void> {
|
||||
this.#handle = await open(this.path, "r");
|
||||
this.size = (await this.#handle.stat()).size;
|
||||
override init(): Promise<void> {
|
||||
this.#initPromise ??= this.#initialize();
|
||||
return this.#initPromise;
|
||||
}
|
||||
|
||||
async #initialize(): Promise<void> {
|
||||
try {
|
||||
this.#handle =
|
||||
this.admitted?.handle ??
|
||||
(await openRegularFile(this.path, { symlinks: "reject" }));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof NonRegularFileReadError)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`ZIP source is not a regular file: ${this.path}`,
|
||||
{ cause },
|
||||
);
|
||||
throw cause;
|
||||
}
|
||||
const metadata = await this.#handle.stat();
|
||||
if (
|
||||
!metadata.isFile() ||
|
||||
(this.admitted !== undefined &&
|
||||
!sameRegularFileState(this.admitted.initial, metadata))
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`ZIP source identity changed before reading: ${this.path}`,
|
||||
);
|
||||
this.size = metadata.size;
|
||||
}
|
||||
|
||||
override async readUint8Array(
|
||||
@@ -47,7 +84,10 @@ class NodeFileReader extends Reader<string> {
|
||||
}
|
||||
|
||||
async closeHandle(): Promise<void> {
|
||||
await this.#handle?.close();
|
||||
if (!this.#ownsHandle) return;
|
||||
await this.#initPromise?.catch(() => undefined);
|
||||
if (this.#handle === undefined) return;
|
||||
await this.#handle.close();
|
||||
this.#handle = undefined;
|
||||
}
|
||||
}
|
||||
@@ -64,9 +104,10 @@ export class ZipArtifactReader implements ArtifactReader {
|
||||
path: string,
|
||||
format: ZipPackageFormat,
|
||||
maximumMetadataReadBytes?: number,
|
||||
admitted?: StableRegularFileDescriptor,
|
||||
) {
|
||||
this.format = format;
|
||||
this.#source = new NodeFileReader(path, maximumMetadataReadBytes);
|
||||
this.#source = new NodeFileReader(path, maximumMetadataReadBytes, admitted);
|
||||
this.#reader = new ZipReader(this.#source, {
|
||||
checkSignature: true,
|
||||
checkOverlappingEntry: true,
|
||||
|
||||
@@ -25,6 +25,7 @@ import {
|
||||
type NibHierarchyNode,
|
||||
} from "./NibViewHierarchy.js";
|
||||
import { jsonValueSchema, type JsonValue } from "../../domain/jsonValue.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
|
||||
import { decodeXmlPlistText } from "../../domain/propertyListXmlText.js";
|
||||
|
||||
@@ -49,7 +50,6 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
const prototypeKeyOmissions = new Map<string, number>();
|
||||
let omitted = 0;
|
||||
let attempted = 0;
|
||||
let candidates = 0;
|
||||
let aggregateInputBytes = 0;
|
||||
const decodeBudget = new InterfaceBuilderDecodeBudget(
|
||||
MAX_AGGREGATE_DECODE_BYTES,
|
||||
@@ -60,15 +60,16 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
else unreportedOmittedArchives += 1;
|
||||
};
|
||||
try {
|
||||
for await (const entry of reader.entries(input.signal)) {
|
||||
if (entry.kind !== "file" || !isInterfaceBuilderArchive(entry.path))
|
||||
continue;
|
||||
candidates += 1;
|
||||
if (candidates > limits.max_documents) {
|
||||
omitted += 1;
|
||||
recordOmittedArchive(entry.path, "max_documents");
|
||||
continue;
|
||||
}
|
||||
const selection = await selectInterfaceBuilderCandidates(
|
||||
reader,
|
||||
input.signal,
|
||||
limits.max_documents,
|
||||
);
|
||||
omitted = selection.omitted;
|
||||
for (const entry of selection.omittedSamples)
|
||||
recordOmittedArchive(entry.path, "max_documents");
|
||||
unreportedOmittedArchives += selection.unreportedOmitted;
|
||||
for (const entry of selection.selected) {
|
||||
try {
|
||||
const remainingInputBytes =
|
||||
MAX_AGGREGATE_INPUT_BYTES - aggregateInputBytes;
|
||||
@@ -167,6 +168,60 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
});
|
||||
};
|
||||
|
||||
/** Select lexically earliest document paths independent of host traversal order. */
|
||||
const selectInterfaceBuilderCandidates = async (
|
||||
reader: DirectoryArtifactReader,
|
||||
signal: AbortSignal | undefined,
|
||||
maximumDocuments: number,
|
||||
): Promise<{
|
||||
readonly selected: readonly ArtifactEntry[];
|
||||
readonly omittedSamples: readonly ArtifactEntry[];
|
||||
readonly omitted: number;
|
||||
readonly unreportedOmitted: number;
|
||||
}> => {
|
||||
const capacity = maximumDocuments + MAX_REPORTED_OMITTED_ARCHIVES;
|
||||
const candidates: ArtifactEntry[] = [];
|
||||
let total = 0;
|
||||
for await (const entry of reader.entries(signal)) {
|
||||
if (entry.kind !== "file" || !isInterfaceBuilderArchive(entry.path))
|
||||
continue;
|
||||
total += 1;
|
||||
retainCandidate(candidates, entry, capacity);
|
||||
}
|
||||
const omittedSamples = candidates.slice(maximumDocuments, capacity);
|
||||
return {
|
||||
selected: candidates.slice(0, maximumDocuments),
|
||||
omittedSamples,
|
||||
omitted: Math.max(0, total - maximumDocuments),
|
||||
unreportedOmitted: Math.max(
|
||||
0,
|
||||
total - maximumDocuments - omittedSamples.length,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
/** Keep the earliest bounded path set; directory iteration order is host-defined. */
|
||||
const retainCandidate = (
|
||||
candidates: ArtifactEntry[],
|
||||
entry: ArtifactEntry,
|
||||
maximum: number,
|
||||
): void => {
|
||||
let low = 0;
|
||||
let high = candidates.length;
|
||||
while (low < high) {
|
||||
const middle = Math.floor((low + high) / 2);
|
||||
const existing = candidates[middle];
|
||||
if (
|
||||
existing !== undefined &&
|
||||
compareUnicodeCodePoints(existing.path, entry.path) < 0
|
||||
)
|
||||
low = middle + 1;
|
||||
else high = middle;
|
||||
}
|
||||
candidates.splice(low, 0, entry);
|
||||
if (candidates.length > maximum) candidates.pop();
|
||||
};
|
||||
|
||||
const finalizeHierarchyCoverage = (
|
||||
result: ReturnType<typeof buildInterfaceBuilderAnalysis>,
|
||||
counts: {
|
||||
|
||||
@@ -147,29 +147,28 @@ export const inspectBundleKeyedArchive = async (input: {
|
||||
try {
|
||||
// Match inventory spelling while retaining the raw path for I/O and evidence.
|
||||
const wanted = selected.path.normalize("NFC");
|
||||
const matches: ArtifactEntry[] = [];
|
||||
for await (const entry of reader.entries(input.signal, (directory) =>
|
||||
let entry: ArtifactEntry | undefined;
|
||||
for await (const candidate of reader.entries(input.signal, (directory) =>
|
||||
wanted.startsWith(`${directory.normalize("NFC")}/`),
|
||||
)) {
|
||||
if (entry.path.normalize("NFC") !== wanted) continue;
|
||||
if (entry.kind !== "file")
|
||||
if (candidate.path.normalize("NFC") !== wanted) continue;
|
||||
if (candidate.kind !== "file")
|
||||
throw archivePathError(
|
||||
"invalid_value",
|
||||
`Archive path selects a ${entry.kind}, not a regular file: ${selected.path}`,
|
||||
`Archive path selects a ${candidate.kind}, not a regular file: ${selected.path}`,
|
||||
);
|
||||
if ((entry.declaredSize ?? 0) > MAX_BYTES)
|
||||
if ((candidate.declaredSize ?? 0) > MAX_BYTES)
|
||||
throw new ArtifactReaderFailure(
|
||||
"limit",
|
||||
"Keyed archive exceeds 64 MiB",
|
||||
);
|
||||
matches.push(entry);
|
||||
if (entry !== undefined)
|
||||
throw archivePathError(
|
||||
"invalid_value",
|
||||
`Archive path matches multiple Unicode-equivalent bundle entries; selection is ambiguous: ${selected.path}`,
|
||||
);
|
||||
entry = candidate;
|
||||
}
|
||||
if (matches.length > 1)
|
||||
throw archivePathError(
|
||||
"invalid_value",
|
||||
`Archive path matches ${String(matches.length)} Unicode-equivalent bundle entries; selection is ambiguous: ${selected.path}`,
|
||||
);
|
||||
const entry = matches[0];
|
||||
if (entry === undefined)
|
||||
throw archivePathError(
|
||||
"invalid_value",
|
||||
|
||||
@@ -215,7 +215,10 @@ const materializeSelection = async ({
|
||||
const written = await output.write(
|
||||
path,
|
||||
stream,
|
||||
selectedItem.node.sha256,
|
||||
{
|
||||
sha256: selectedItem.node.sha256,
|
||||
bytes: selectedItem.node.size,
|
||||
},
|
||||
signal,
|
||||
);
|
||||
extracted.push({
|
||||
|
||||
@@ -295,23 +295,53 @@ export const createArtifactEdges = (
|
||||
return edges;
|
||||
};
|
||||
|
||||
export const nearestParent = (
|
||||
path: string,
|
||||
occurrences: ReadonlyMap<string, MutableOccurrence>,
|
||||
interface ArtifactParentIndex {
|
||||
readonly children: Map<string, ArtifactParentIndex>;
|
||||
occurrence?: MutableOccurrence;
|
||||
}
|
||||
|
||||
/** Resolve strict observed ancestors without rebuilding every path prefix. */
|
||||
export const resolveOccurrenceParents = (
|
||||
occurrences: readonly MutableOccurrence[],
|
||||
byPath: ReadonlyMap<string, MutableOccurrence>,
|
||||
expandedContainerIds: ReadonlySet<string>,
|
||||
): MutableOccurrence | undefined => {
|
||||
const parts = path.split("/");
|
||||
while (parts.length > 1) {
|
||||
parts.pop();
|
||||
const candidate = occurrences.get(parts.join("/"));
|
||||
): void => {
|
||||
const root: ArtifactParentIndex = { children: new Map() };
|
||||
for (const occurrence of byPath.values()) {
|
||||
if (
|
||||
candidate !== undefined &&
|
||||
(candidate.entry_kind === "directory" ||
|
||||
expandedContainerIds.has(candidate.occurrence_id))
|
||||
occurrence.entry_kind !== "directory" &&
|
||||
!expandedContainerIds.has(occurrence.occurrence_id)
|
||||
)
|
||||
return candidate;
|
||||
continue;
|
||||
let branch = root;
|
||||
for (const segment of occurrence.logical_path.split("/")) {
|
||||
let child = branch.children.get(segment);
|
||||
if (child === undefined) {
|
||||
child = { children: new Map() };
|
||||
branch.children.set(segment, child);
|
||||
}
|
||||
branch = child;
|
||||
}
|
||||
branch.occurrence = occurrence;
|
||||
}
|
||||
for (const occurrence of occurrences) {
|
||||
const path = occurrence.logical_path;
|
||||
let branch = root;
|
||||
let parent: MutableOccurrence | undefined;
|
||||
// Walk parent segments only. Stop scanning the path once no observed
|
||||
// directory or expanded container can be an ancestor.
|
||||
for (let start = 0; ;) {
|
||||
const slash = path.indexOf("/", start);
|
||||
if (slash < 0) break;
|
||||
const segment = path.slice(start, slash);
|
||||
const child = branch.children.get(segment);
|
||||
if (child === undefined) break;
|
||||
branch = child;
|
||||
parent = branch.occurrence ?? parent;
|
||||
start = slash + 1;
|
||||
}
|
||||
occurrence.parent_occurrence_id = parent?.occurrence_id ?? null;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
} from "../../filesystem/RegularFile.js";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
import type { Stats } from "node:fs";
|
||||
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
|
||||
|
||||
import { classifyArtifactContent } from "./ArtifactGraphConstruction.js";
|
||||
import { ARTIFACT_CLASSIFICATION_PREFIX_BYTES } from "../ArtifactHash.js";
|
||||
@@ -17,18 +18,46 @@ import { ArtifactReaderFailure } from "../ArtifactReader.js";
|
||||
import type { HashResult } from "../ArtifactHash.js";
|
||||
import { hashStableRootArtifactHandle } from "./hashStableRootArtifact.js";
|
||||
|
||||
interface RootClassification {
|
||||
readonly format: ArtifactOccurrence["artifact_format"];
|
||||
readonly digest: HashResult | null;
|
||||
}
|
||||
|
||||
type RootInventoryClassification = RootClassification & {
|
||||
readonly rootSource: StableRegularFileDescriptor | undefined;
|
||||
};
|
||||
|
||||
/** Classify and hash one file root through the same stable open descriptor. */
|
||||
export const classifyAndHashRoot = async (
|
||||
path: string,
|
||||
directory: boolean,
|
||||
expectedMetadata: Stats,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{
|
||||
readonly format: ArtifactOccurrence["artifact_format"];
|
||||
readonly digest: HashResult | null;
|
||||
}> => {
|
||||
if (directory) return { format: "directory", digest: null };
|
||||
): Promise<RootClassification> => {
|
||||
const result = await classifyAndHashRootForInventory(
|
||||
path,
|
||||
directory,
|
||||
expectedMetadata,
|
||||
signal,
|
||||
);
|
||||
try {
|
||||
return { format: result.format, digest: result.digest };
|
||||
} finally {
|
||||
await result.rootSource?.handle.close();
|
||||
}
|
||||
};
|
||||
|
||||
/** Classify a root and retain its admitted ZIP descriptor for child inventory. */
|
||||
export const classifyAndHashRootForInventory = async (
|
||||
path: string,
|
||||
directory: boolean,
|
||||
expectedMetadata: Stats,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RootInventoryClassification> => {
|
||||
if (directory)
|
||||
return { format: "directory", digest: null, rootSource: undefined };
|
||||
const handle = await openRootFile(path, signal);
|
||||
let transferHandle = false;
|
||||
try {
|
||||
const initial = await handle.stat();
|
||||
if (!sameRegularFileState(expectedMetadata, initial))
|
||||
@@ -43,12 +72,25 @@ export const classifyAndHashRoot = async (
|
||||
initial,
|
||||
signal,
|
||||
);
|
||||
return { format, digest };
|
||||
const retainSource = isZipFormat(format) || format === "mach-o-universal";
|
||||
if (retainSource) transferHandle = true;
|
||||
return {
|
||||
format,
|
||||
digest,
|
||||
rootSource: retainSource ? { handle, initial } : undefined,
|
||||
};
|
||||
} finally {
|
||||
await handle.close();
|
||||
if (!transferHandle) await handle.close();
|
||||
}
|
||||
};
|
||||
|
||||
const isZipFormat = (format: ArtifactOccurrence["artifact_format"]): boolean =>
|
||||
format === "zip" ||
|
||||
format === "ipa" ||
|
||||
format === "apk" ||
|
||||
format === "msix" ||
|
||||
format === "appx";
|
||||
|
||||
const openRootFile = async (
|
||||
path: string,
|
||||
signal?: AbortSignal,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { lstat } from "node:fs/promises";
|
||||
import type { Stats } from "node:fs";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
import { ArtifactReaderFailure } from "../ArtifactReader.js";
|
||||
@@ -45,11 +46,13 @@ export const hashStableRootArtifact = async (
|
||||
export const hashStableRootArtifactHandle = async (
|
||||
path: string,
|
||||
handle: FileHandle,
|
||||
initial: Awaited<ReturnType<FileHandle["stat"]>>,
|
||||
initial: Stats,
|
||||
signal?: AbortSignal,
|
||||
): Promise<HashResult> => {
|
||||
const digest = await hashReadable(
|
||||
handle.createReadStream({ autoClose: false }),
|
||||
// Read at most the admitted extent plus one byte to detect growth without
|
||||
// following an actively appended file indefinitely.
|
||||
handle.createReadStream({ autoClose: false, start: 0, end: initial.size }),
|
||||
signal,
|
||||
);
|
||||
abortIfNeeded(signal);
|
||||
|
||||
@@ -6,12 +6,14 @@ import { NativeDmgArtifactReader } from "../NativeDmgArtifactReader.js";
|
||||
import { ZipArtifactReader } from "../ZipArtifactReader.js";
|
||||
import type { ArtifactOccurrence } from "../../domain/artifactGraph.js";
|
||||
import { ZIP_NON_ENTRY_TAIL_LIMITATION } from "../../domain/zipPackageFormat.js";
|
||||
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
|
||||
|
||||
export const createReader = async (
|
||||
path: string,
|
||||
format: ArtifactOccurrence["artifact_format"],
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
signal?: AbortSignal,
|
||||
rootSource?: StableRegularFileDescriptor,
|
||||
): Promise<ArtifactReader | undefined> => {
|
||||
switch (format) {
|
||||
case "directory":
|
||||
@@ -21,12 +23,12 @@ export const createReader = async (
|
||||
case "apk":
|
||||
case "msix":
|
||||
case "appx":
|
||||
return new ZipArtifactReader(path, format);
|
||||
return new ZipArtifactReader(path, format, undefined, rootSource);
|
||||
case "asar":
|
||||
return new AsarArtifactReader(path);
|
||||
case "mach-o-universal":
|
||||
return process.platform === "darwin"
|
||||
? new MachOSliceArtifactReader(path, environment)
|
||||
? new MachOSliceArtifactReader(path, environment, undefined, rootSource)
|
||||
: undefined;
|
||||
case "dmg":
|
||||
if (process.platform !== "darwin") return undefined;
|
||||
|
||||
@@ -5,6 +5,7 @@ import { AsarArtifactReader } from "../AsarArtifactReader.js";
|
||||
|
||||
import { lstat } from "node:fs/promises";
|
||||
import type { Stats } from "node:fs";
|
||||
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
|
||||
|
||||
import {
|
||||
ArtifactReaderFailure,
|
||||
@@ -31,9 +32,12 @@ import {
|
||||
artifactGraphDigest,
|
||||
artifactManifestId,
|
||||
} from "../../domain/artifactIdentity.js";
|
||||
import { classifyAndHashRoot } from "./classify.js";
|
||||
import { classifyAndHashRootForInventory } from "./classify.js";
|
||||
import { type HashResult } from "../ArtifactHash.js";
|
||||
import { hashStableRootArtifact } from "./hashStableRootArtifact.js";
|
||||
import {
|
||||
hashStableRootArtifact,
|
||||
hashStableRootArtifactHandle,
|
||||
} from "./hashStableRootArtifact.js";
|
||||
import { createReader, inventoryLimitations } from "./reader.js";
|
||||
import {
|
||||
scanReader,
|
||||
@@ -51,18 +55,29 @@ export const scanCanonicalArtifactInventory = async (
|
||||
): Promise<ArtifactInventorySnapshot> => {
|
||||
const integrity = options.integrity ?? STRICT_INTEGRITY_POLICY;
|
||||
const metadata = await lstat(path);
|
||||
const { format: rootFormat, digest: rootDigest } = await classifyAndHashRoot(
|
||||
const {
|
||||
format: rootFormat,
|
||||
digest: rootDigest,
|
||||
rootSource,
|
||||
} = await classifyAndHashRootForInventory(
|
||||
path,
|
||||
metadata.isDirectory(),
|
||||
metadata,
|
||||
options.signal,
|
||||
);
|
||||
const reader = await readerFactory(
|
||||
path,
|
||||
rootFormat,
|
||||
options.environment ?? {},
|
||||
options.signal,
|
||||
);
|
||||
let reader: ArtifactReader | undefined;
|
||||
let readerCreationFailure: { readonly cause: unknown } | undefined;
|
||||
try {
|
||||
reader = await readerFactory(
|
||||
path,
|
||||
rootFormat,
|
||||
options.environment ?? {},
|
||||
options.signal,
|
||||
rootSource,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
readerCreationFailure = { cause };
|
||||
}
|
||||
const ownedReaders: ArtifactReader[] = reader === undefined ? [] : [reader];
|
||||
let outcome:
|
||||
| {
|
||||
@@ -71,6 +86,7 @@ export const scanCanonicalArtifactInventory = async (
|
||||
}
|
||||
| { readonly kind: "failed"; readonly cause: unknown };
|
||||
try {
|
||||
if (readerCreationFailure !== undefined) throw readerCreationFailure.cause;
|
||||
if (reader instanceof AsarArtifactReader)
|
||||
await reader.prepareContainer(rootDigest?.sha256, options.signal);
|
||||
const {
|
||||
@@ -86,6 +102,7 @@ export const scanCanonicalArtifactInventory = async (
|
||||
metadata,
|
||||
rootFormat,
|
||||
rootDigest,
|
||||
rootSource,
|
||||
reader,
|
||||
signal: options.signal,
|
||||
nodes,
|
||||
@@ -116,6 +133,24 @@ export const scanCanonicalArtifactInventory = async (
|
||||
);
|
||||
}
|
||||
}
|
||||
if (rootSource !== undefined) {
|
||||
try {
|
||||
await rootSource.handle.close();
|
||||
} catch (cleanupCause: unknown) {
|
||||
const cleanup = ArtifactReaderFailure.cleanupObservation(
|
||||
cleanupCause,
|
||||
`root artifact descriptor for ${path}`,
|
||||
);
|
||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
||||
cleanupFailure ??
|
||||
(outcome.kind === "failed" ? outcome.cause : cleanupCause),
|
||||
cleanup,
|
||||
outcome.kind === "completed"
|
||||
? { kind: "artifact-inventory", inventory: outcome.snapshot }
|
||||
: undefined,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (cleanupFailure !== undefined) throw cleanupFailure;
|
||||
if (outcome.kind === "failed") throw outcome.cause;
|
||||
return outcome.snapshot;
|
||||
@@ -126,6 +161,7 @@ interface SnapshotBuildInput {
|
||||
readonly metadata: Stats;
|
||||
readonly rootFormat: ArtifactOccurrence["artifact_format"];
|
||||
readonly rootDigest: HashResult | null;
|
||||
readonly rootSource: StableRegularFileDescriptor | undefined;
|
||||
readonly reader: ArtifactReader | undefined;
|
||||
readonly signal: AbortSignal | undefined;
|
||||
readonly nodes: Map<string, ArtifactNode>;
|
||||
@@ -137,7 +173,8 @@ interface SnapshotBuildInput {
|
||||
const buildInventorySnapshot = async (
|
||||
input: SnapshotBuildInput,
|
||||
): Promise<ArtifactInventorySnapshot> => {
|
||||
const { path, metadata, rootFormat, rootDigest, reader, signal } = input;
|
||||
const { path, metadata, rootFormat, rootDigest, rootSource, reader, signal } =
|
||||
input;
|
||||
materializeDirectoryNodes(input.occurrences, input.nodes);
|
||||
const rootNode = createRootNode({
|
||||
digest: rootDigest,
|
||||
@@ -181,7 +218,7 @@ const buildInventorySnapshot = async (
|
||||
ordinal,
|
||||
}));
|
||||
|
||||
await verifyRootDigest(path, rootDigest, signal);
|
||||
await verifyRootDigest(path, rootDigest, rootSource, signal);
|
||||
|
||||
const graphSha256 = artifactGraphDigest({
|
||||
nodes: orderedNodes,
|
||||
@@ -256,10 +293,19 @@ const buildIntegrityContradictions = (
|
||||
const verifyRootDigest = async (
|
||||
path: string,
|
||||
rootDigest: HashResult | null,
|
||||
rootSource: StableRegularFileDescriptor | undefined,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<void> => {
|
||||
if (rootDigest === null) return;
|
||||
const verified = await hashStableRootArtifact(path, signal);
|
||||
const verified =
|
||||
rootSource === undefined
|
||||
? await hashStableRootArtifact(path, signal)
|
||||
: await hashStableRootArtifactHandle(
|
||||
path,
|
||||
rootSource.handle,
|
||||
rootSource.initial,
|
||||
signal,
|
||||
);
|
||||
if (
|
||||
verified.sha256 !== rootDigest.sha256 ||
|
||||
verified.bytes !== rootDigest.bytes
|
||||
|
||||
@@ -18,7 +18,7 @@ import {
|
||||
createArtifactNode,
|
||||
createOccurrence,
|
||||
noteUnreadContainerSuffix,
|
||||
nearestParent,
|
||||
resolveOccurrenceParents,
|
||||
type MutableOccurrence,
|
||||
} from "./ArtifactGraphConstruction.js";
|
||||
import {
|
||||
@@ -91,13 +91,11 @@ export const scanReader = async (
|
||||
await visitArtifactEntries(context, reader, "");
|
||||
// Archive directories may appear after their children. Resolve containment
|
||||
// against the complete index before directory identities are materialized.
|
||||
for (const occurrence of occurrences)
|
||||
occurrence.parent_occurrence_id =
|
||||
nearestParent(
|
||||
occurrence.logical_path,
|
||||
context.occurrenceByPath,
|
||||
context.expandedContainerIds,
|
||||
)?.occurrence_id ?? null;
|
||||
resolveOccurrenceParents(
|
||||
occurrences,
|
||||
context.occurrenceByPath,
|
||||
context.expandedContainerIds,
|
||||
);
|
||||
return {
|
||||
nodes,
|
||||
occurrences,
|
||||
|
||||
@@ -38,5 +38,8 @@ export interface BrowserScenarioSessionPort {
|
||||
/** Acquire one scenario session whose lifecycle the caller owns. */
|
||||
export type BrowserScenarioSessionOpener = (
|
||||
scenario: BrowserScenario,
|
||||
options: { readonly signal?: AbortSignal },
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
},
|
||||
) => Promise<BrowserScenarioSessionPort>;
|
||||
|
||||
@@ -33,6 +33,7 @@ export class CdpConnection {
|
||||
>();
|
||||
#nextId = 1;
|
||||
#closed = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#protocolFailed = false;
|
||||
#transportFailure: "disconnected" | "payload_limit" = "disconnected";
|
||||
|
||||
@@ -151,8 +152,15 @@ export class CdpConnection {
|
||||
}
|
||||
|
||||
/** Close only REA's socket; never close the browser or selected page. */
|
||||
async close(): Promise<void> {
|
||||
if (this.#closed) return;
|
||||
close(): Promise<void> {
|
||||
this.#closePromise ??= this.#closeTransport().catch((cause: unknown) => {
|
||||
this.#closePromise = undefined;
|
||||
throw cause;
|
||||
});
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
async #closeTransport(): Promise<void> {
|
||||
this.#closed = true;
|
||||
this.#failPending("disconnected");
|
||||
if (this.socket.readyState === WebSocket.CLOSED) return;
|
||||
|
||||
@@ -50,54 +50,42 @@ export const closeCdpTargetSession = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
const { connection, sessionId } = targetSession;
|
||||
for (const [index, domain] of enabledDomains.entries()) {
|
||||
if (signal?.aborted === true) {
|
||||
await closeCancelledTargetSession(
|
||||
targetSession,
|
||||
enabledDomains.slice(index),
|
||||
);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await connection.send(`${domain}.disable`, {}, sessionId, signal);
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: domain disable continues to detach/close boundary.
|
||||
void cause;
|
||||
}
|
||||
}
|
||||
if (signal?.aborted === true) {
|
||||
await closeCancelledTargetSession(targetSession, []);
|
||||
return;
|
||||
}
|
||||
if (sessionId !== undefined)
|
||||
try {
|
||||
await connection.send(
|
||||
"Target.detachFromTarget",
|
||||
{ sessionId },
|
||||
undefined,
|
||||
signal,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: closing REA's socket is the final cleanup boundary.
|
||||
void cause;
|
||||
}
|
||||
await connection.close();
|
||||
};
|
||||
|
||||
const closeCancelledTargetSession = async (
|
||||
{ connection, sessionId }: CdpTargetSession,
|
||||
enabledDomains: readonly string[],
|
||||
): Promise<void> => {
|
||||
const commands = enabledDomains.map((domain) =>
|
||||
connection.send(`${domain}.disable`, {}, sessionId),
|
||||
);
|
||||
if (sessionId !== undefined)
|
||||
commands.push(connection.send("Target.detachFromTarget", { sessionId }));
|
||||
const settled = Promise.allSettled(commands);
|
||||
const cleanup = new AbortController();
|
||||
const timeout = setTimeout(() => cleanup.abort(), 1_000);
|
||||
timeout.unref();
|
||||
try {
|
||||
await connection.close();
|
||||
if (signal?.aborted !== true) {
|
||||
for (const domain of enabledDomains) {
|
||||
if (cleanup.signal.aborted) break;
|
||||
try {
|
||||
await connection.send(
|
||||
`${domain}.disable`,
|
||||
{},
|
||||
sessionId,
|
||||
cleanup.signal,
|
||||
);
|
||||
} catch {
|
||||
// Best-effort domain cleanup; transport close is the definitive boundary.
|
||||
}
|
||||
}
|
||||
}
|
||||
// Cancellation skips optional domain shutdown but still releases the
|
||||
// attached session using its independent, bounded cleanup signal.
|
||||
if (!cleanup.signal.aborted && sessionId !== undefined)
|
||||
try {
|
||||
await connection.send(
|
||||
"Target.detachFromTarget",
|
||||
{ sessionId },
|
||||
undefined,
|
||||
cleanup.signal,
|
||||
);
|
||||
} catch {
|
||||
// Best-effort detach; transport close is the definitive boundary.
|
||||
}
|
||||
} finally {
|
||||
await settled;
|
||||
clearTimeout(timeout);
|
||||
cleanup.abort();
|
||||
await connection.close();
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -365,20 +365,30 @@ const ingestWebMcpEvent = (options: WebMcpIngestOptions): void => {
|
||||
transientFrames,
|
||||
resolvedTransientFrames,
|
||||
} = options;
|
||||
if (
|
||||
event.method !== "WebMCP.toolsAdded" &&
|
||||
event.method !== "WebMCP.toolsRemoved"
|
||||
)
|
||||
return;
|
||||
const params = recordValue(event.params);
|
||||
if (params === undefined) return;
|
||||
if (params === undefined) {
|
||||
completeness.exclude("webmcp_tools", "invalid_protocol_value");
|
||||
return;
|
||||
}
|
||||
if (event.method === "WebMCP.toolsRemoved") {
|
||||
for (const removed of recordsValue(params.tools)) {
|
||||
for (const removed of webMcpEventTools(params.tools, completeness)) {
|
||||
const name = cdpStringValue(removed.name);
|
||||
const frameId = cdpStringValue(removed.frameId);
|
||||
if (name === undefined || frameId === undefined) continue;
|
||||
if (name === undefined || frameId === undefined) {
|
||||
completeness.exclude("webmcp_tools", "invalid_protocol_value");
|
||||
continue;
|
||||
}
|
||||
for (const [key, tool] of tools)
|
||||
if (tool.name === name && tool.frame_id === frameId) tools.delete(key);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (event.method !== "WebMCP.toolsAdded") return;
|
||||
for (const declared of recordsValue(params.tools)) {
|
||||
for (const declared of webMcpEventTools(params.tools, completeness)) {
|
||||
const frameId = cdpStringValue(declared.frameId);
|
||||
if (frameId !== undefined && transientFrames.has(frameId)) {
|
||||
if (!resolvedTransientFrames.has(frameId)) continue;
|
||||
@@ -391,6 +401,26 @@ const ingestWebMcpEvent = (options: WebMcpIngestOptions): void => {
|
||||
}
|
||||
};
|
||||
|
||||
const webMcpEventTools = (
|
||||
value: unknown,
|
||||
completeness: CdpCaptureCompleteness,
|
||||
): UnknownRecord[] => {
|
||||
if (!Array.isArray(value)) {
|
||||
completeness.exclude("webmcp_tools", "invalid_protocol_value");
|
||||
return [];
|
||||
}
|
||||
const records: UnknownRecord[] = [];
|
||||
for (const item of value) {
|
||||
const record = recordValue(item);
|
||||
if (record === undefined) {
|
||||
completeness.exclude("webmcp_tools", "invalid_protocol_value");
|
||||
continue;
|
||||
}
|
||||
records.push(record);
|
||||
}
|
||||
return records;
|
||||
};
|
||||
|
||||
const normalizeTool = (
|
||||
value: UnknownRecord,
|
||||
input: DiscoverWebMcpToolsInput,
|
||||
@@ -403,12 +433,11 @@ const normalizeTool = (
|
||||
const frameId = cdpStringValue(value.frameId);
|
||||
const name = cdpStringValue(value.name);
|
||||
const frame = frameId === undefined ? undefined : frames.get(frameId);
|
||||
if (
|
||||
frameId === undefined ||
|
||||
name === undefined ||
|
||||
frame?.origin === null ||
|
||||
frame === undefined
|
||||
) {
|
||||
if (frameId === undefined || name === undefined) {
|
||||
completeness.exclude("webmcp_tools", "invalid_protocol_value");
|
||||
return undefined;
|
||||
}
|
||||
if (frame?.origin === null || frame === undefined) {
|
||||
completeness.exclude("webmcp_tools", "out_of_target_scope");
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -3,6 +3,11 @@ import { open, realpath, stat } from "node:fs/promises";
|
||||
import { isAbsolute, join, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import {
|
||||
admitAsarHeader,
|
||||
MAX_ASAR_HEADER_BYTES,
|
||||
} from "../artifacts/AsarHeader.js";
|
||||
|
||||
/** Resolve one local file URL while rejecting remote hosts and encoded separators. */
|
||||
export const authorizedElectronFile = async (
|
||||
value: string,
|
||||
@@ -77,7 +82,6 @@ const authorizedAsarMember = async (
|
||||
type ArchiveIdentity = Pick<Stats, "dev" | "ino" | "size" | "mtimeMs">;
|
||||
|
||||
// This is a retained-memory budget for JSON headers, shared across all archives.
|
||||
const MAX_HEADER_BYTES = 16 * 1024 * 1024;
|
||||
const MAX_CACHED_HEADERS = 8;
|
||||
const headerCache = new Map<
|
||||
string,
|
||||
@@ -105,23 +109,8 @@ const asarFiles = async (
|
||||
const opened = await handle.stat();
|
||||
if (!opened.isFile() || archiveIdentity(opened) !== identity)
|
||||
throw new Error("ASAR identity changed before header read");
|
||||
const prefix = Buffer.alloc(16);
|
||||
const read = await handle.read(prefix, 0, prefix.length, 0);
|
||||
if (read.bytesRead !== prefix.length || prefix.readUInt32LE(0) !== 4)
|
||||
throw new Error("Truncated or invalid ASAR size pickle");
|
||||
const headerBytes = prefix.readUInt32LE(4);
|
||||
const payloadBytes = prefix.readUInt32LE(8);
|
||||
jsonBytes = prefix.readUInt32LE(12);
|
||||
if (
|
||||
headerBytes < 8 ||
|
||||
headerBytes > opened.size - 8 ||
|
||||
payloadBytes !== headerBytes - 4 ||
|
||||
jsonBytes > payloadBytes - 4 ||
|
||||
jsonBytes > MAX_HEADER_BYTES
|
||||
)
|
||||
throw new Error(
|
||||
"ASAR JSON header leaves the file or exceeds its memory budget",
|
||||
);
|
||||
const admitted = await admitAsarHeader(handle);
|
||||
jsonBytes = admitted.jsonBytes;
|
||||
const json = Buffer.alloc(jsonBytes);
|
||||
let offset = 0;
|
||||
while (offset < json.length) {
|
||||
@@ -151,7 +140,7 @@ const asarFiles = async (
|
||||
[...headerCache.values()].reduce((sum, entry) => sum + entry.bytes, 0);
|
||||
while (
|
||||
headerCache.size > MAX_CACHED_HEADERS ||
|
||||
retainedBytes() > MAX_HEADER_BYTES
|
||||
retainedBytes() > MAX_ASAR_HEADER_BYTES
|
||||
)
|
||||
headerCache.delete(headerCache.keys().next().value ?? archive);
|
||||
return files;
|
||||
|
||||
@@ -229,14 +229,15 @@ const runScenario = async (
|
||||
openSession: BrowserScenarioSessionOpener,
|
||||
scenario: BrowserScenario,
|
||||
options: ExecutionOptions,
|
||||
retainCleanup: (close: () => Promise<unknown>) => void,
|
||||
): Promise<BrowserScenarioCapture> => {
|
||||
if (options.signal?.aborted === true)
|
||||
throw new BrowserObservationError(OPERATION, "cancelled");
|
||||
const startedAt = Date.now();
|
||||
const session = await openSession(
|
||||
scenario,
|
||||
options.signal === undefined ? {} : { signal: options.signal },
|
||||
);
|
||||
const session = await openSession(scenario, {
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
retainCleanup,
|
||||
});
|
||||
const steps: BrowserScenarioStep[] = [];
|
||||
let operationOutcome:
|
||||
| { readonly ok: true }
|
||||
@@ -279,6 +280,7 @@ const runScenario = async (
|
||||
cleanupOutcome = { ok: true, cleanup: await session.close() };
|
||||
} catch (cause: unknown) {
|
||||
cleanupOutcome = { ok: false, cause };
|
||||
retainCleanup(() => session.close());
|
||||
}
|
||||
if (!cleanupOutcome.ok) {
|
||||
const cleanup = browserScenarioCleanupObservation(cleanupOutcome.cause);
|
||||
@@ -337,18 +339,46 @@ const runScenario = async (
|
||||
|
||||
/** Controlled Playwright/CDP scenario driver with exact process ownership. */
|
||||
export class PlaywrightBrowserScenarioProvider implements BrowserScenarioCapturePort {
|
||||
#tail: Promise<void> = Promise.resolve();
|
||||
#closing = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
readonly #pendingCleanup = new Set<() => Promise<unknown>>();
|
||||
|
||||
constructor(private readonly openSession: BrowserScenarioSessionOpener) {}
|
||||
|
||||
identity(): ProviderIdentity {
|
||||
return PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY;
|
||||
}
|
||||
|
||||
async captureScenario(
|
||||
captureScenario(
|
||||
scenario: BrowserScenario,
|
||||
options: ExecutionOptions = {},
|
||||
): Promise<Result<BrowserScenarioCapture, AnalysisError>> {
|
||||
if (this.#closing) return Promise.resolve(err(this.#closedError()));
|
||||
const operation = this.#tail.then(() =>
|
||||
this.#closing
|
||||
? err(this.#closedError())
|
||||
: this.#captureScenario(scenario, options),
|
||||
);
|
||||
this.#tail = operation.then(
|
||||
() => undefined,
|
||||
() => undefined,
|
||||
);
|
||||
return operation;
|
||||
}
|
||||
|
||||
async #captureScenario(
|
||||
scenario: BrowserScenario,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<BrowserScenarioCapture, AnalysisError>> {
|
||||
const pending = await this.#retryCleanup();
|
||||
if (pending !== undefined) return err(pending);
|
||||
try {
|
||||
return ok(await runScenario(this.openSession, scenario, options));
|
||||
return ok(
|
||||
await runScenario(this.openSession, scenario, options, (close) => {
|
||||
this.#pendingCleanup.add(close);
|
||||
}),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisError) return err(cause);
|
||||
return err(
|
||||
@@ -360,4 +390,43 @@ export class PlaywrightBrowserScenarioProvider implements BrowserScenarioCapture
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.#closing = true;
|
||||
this.#closePromise ??= this.#tail
|
||||
.then(async () => {
|
||||
const failure = await this.#retryCleanup();
|
||||
if (failure !== undefined) throw failure;
|
||||
})
|
||||
.catch((cause: unknown) => {
|
||||
this.#closePromise = undefined;
|
||||
throw cause;
|
||||
});
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
#closedError(): AnalysisError {
|
||||
return new BrowserObservationError(OPERATION, "cleanup_failed", {
|
||||
detail: "Browser scenario provider is closing.",
|
||||
cleanup: { reason: "Provider is closing", resources: [] },
|
||||
});
|
||||
}
|
||||
|
||||
async #retryCleanup(): Promise<AnalysisError | undefined> {
|
||||
for (const pending of this.#pendingCleanup) {
|
||||
try {
|
||||
await pending();
|
||||
this.#pendingCleanup.delete(pending);
|
||||
} catch (cause: unknown) {
|
||||
return cause instanceof AnalysisError
|
||||
? cause
|
||||
: new ProviderAdapterError(
|
||||
PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY.id,
|
||||
OPERATION,
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -97,6 +97,9 @@ export class PlaywrightScenarioBrowserCleanupOwner {
|
||||
try {
|
||||
await eventFinalization;
|
||||
} catch (cause: unknown) {
|
||||
// A failed finalization stays failed: the memoized attempt rejects on
|
||||
// every retry so cleanup can never report success for unfinalized
|
||||
// events.
|
||||
failures.push(cause);
|
||||
}
|
||||
if (!this.#browserClosed) {
|
||||
@@ -380,6 +383,7 @@ const openAttachedScenarioBrowser = async (
|
||||
environment: BrowserScenario["environment"],
|
||||
launcher: ScenarioBrowserLauncher,
|
||||
signal: AbortSignal | undefined,
|
||||
retainCleanup: ((close: () => Promise<unknown>) => void) | undefined,
|
||||
): Promise<OpenedScenarioBrowser> => {
|
||||
const lease = await acquireAttachedScenario(browserRequest, signal);
|
||||
let cleanupOwner: PlaywrightScenarioBrowserCleanupOwner | undefined;
|
||||
@@ -431,7 +435,13 @@ const openAttachedScenarioBrowser = async (
|
||||
throw cause;
|
||||
}
|
||||
const cleanup = cleanupOwner;
|
||||
return failBrowserScenarioOperation(() => cleanup.close(), cause);
|
||||
const close = () => cleanup.close();
|
||||
try {
|
||||
return await failBrowserScenarioOperation(close, cause);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) retainCleanup?.(close);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@@ -442,6 +452,7 @@ export const openPlaywrightScenarioBrowser = async (
|
||||
options: {
|
||||
readonly launcher?: ScenarioBrowserLauncher;
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
} = {},
|
||||
): Promise<OpenedScenarioBrowser> => {
|
||||
const launcher = options.launcher ?? chromium;
|
||||
@@ -451,6 +462,7 @@ export const openPlaywrightScenarioBrowser = async (
|
||||
scenario.environment,
|
||||
launcher,
|
||||
options.signal,
|
||||
options.retainCleanup,
|
||||
);
|
||||
|
||||
const profilePath = await mkdtemp(join(tmpdir(), "rea-browser-scenario-"));
|
||||
@@ -478,11 +490,16 @@ export const openPlaywrightScenarioBrowser = async (
|
||||
timeout: 0,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return failBrowserScenarioOperation(
|
||||
() => rm(profilePath, { recursive: true, force: true, maxRetries: 3 }),
|
||||
cause,
|
||||
["browser_profile"],
|
||||
);
|
||||
const cleanup = () =>
|
||||
rm(profilePath, { recursive: true, force: true, maxRetries: 3 });
|
||||
try {
|
||||
return await failBrowserScenarioOperation(cleanup, cause, [
|
||||
"browser_profile",
|
||||
]);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) options.retainCleanup?.(cleanup);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
const cleanup = new PlaywrightScenarioBrowserCleanupOwner({
|
||||
closeBrowser: () => context.close(),
|
||||
@@ -502,6 +519,12 @@ export const openPlaywrightScenarioBrowser = async (
|
||||
cleanup,
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
return failBrowserScenarioOperation(() => cleanup.close(), cause);
|
||||
const close = () => cleanup.close();
|
||||
try {
|
||||
return await failBrowserScenarioOperation(close, cause);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) options.retainCleanup?.(close);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@@ -127,6 +127,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
},
|
||||
): Promise<PlaywrightScenarioSession> {
|
||||
if (options.signal?.aborted === true)
|
||||
@@ -137,6 +138,9 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
const startedAt = Date.now();
|
||||
const opening = openPlaywrightScenarioBrowser(scenario, environment, {
|
||||
signal: options.signal,
|
||||
...(options.retainCleanup === undefined
|
||||
? {}
|
||||
: { retainCleanup: options.retainCleanup }),
|
||||
});
|
||||
let opened: OpenedScenarioBrowser;
|
||||
let events: PlaywrightScenarioEvents | undefined;
|
||||
@@ -147,14 +151,18 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
options.signal,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
void opening
|
||||
.then((lateOpened) =>
|
||||
lateOpened.cleanup.close(undefined, options.signal),
|
||||
)
|
||||
.catch((cause: unknown) => {
|
||||
// best-effort cleanup: late-open cleanup must not mask the boundary failure.
|
||||
void cause;
|
||||
});
|
||||
const lateCleanup = () =>
|
||||
opening.then(
|
||||
(lateOpened) => lateOpened.cleanup.close(undefined, options.signal),
|
||||
() => undefined,
|
||||
);
|
||||
options.retainCleanup?.(() =>
|
||||
withPlaywrightExecutionBoundary(lateCleanup, 1_000),
|
||||
);
|
||||
void lateCleanup().catch((cleanupCause: unknown) => {
|
||||
// best-effort cleanup: retainCleanup keeps the retry capability.
|
||||
void cleanupCause;
|
||||
});
|
||||
throw cause;
|
||||
}
|
||||
let session: PlaywrightScenarioSession;
|
||||
@@ -179,16 +187,19 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
eventCapture: events,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return failBrowserScenarioOperation(
|
||||
() =>
|
||||
opened.cleanup.close(
|
||||
events === undefined
|
||||
? undefined
|
||||
: () => events?.finish() ?? Promise.resolve(),
|
||||
options.signal,
|
||||
),
|
||||
cause,
|
||||
);
|
||||
const cleanup = () =>
|
||||
opened.cleanup.close(
|
||||
events === undefined
|
||||
? undefined
|
||||
: () => events?.finish() ?? Promise.resolve(),
|
||||
options.signal,
|
||||
);
|
||||
try {
|
||||
return await failBrowserScenarioOperation(cleanup, cause);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) options.retainCleanup?.(cleanup);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
@@ -212,6 +223,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
} catch (failure: unknown) {
|
||||
cleanup = "incomplete";
|
||||
cleanupFailure = { cause: failure };
|
||||
options.retainCleanup?.(() => session.close());
|
||||
}
|
||||
throw browserScenarioOperationFailure(
|
||||
cause,
|
||||
|
||||
@@ -76,7 +76,12 @@ export const publishWebScripts = async (
|
||||
bytes: bytes.length,
|
||||
},
|
||||
});
|
||||
await tree.write("manifest.json", Readable.from([bytes]), digest, signal);
|
||||
await tree.write(
|
||||
"manifest.json",
|
||||
Readable.from([bytes]),
|
||||
{ sha256: digest, bytes: bytes.byteLength },
|
||||
signal,
|
||||
);
|
||||
signal?.throwIfAborted();
|
||||
await tree.commit();
|
||||
return result;
|
||||
@@ -109,7 +114,7 @@ const publishPlannedScript = async (
|
||||
tree.write(
|
||||
`files/${relativePath}`,
|
||||
Readable.from([content.bytes]),
|
||||
content.sha256,
|
||||
{ sha256: content.sha256, bytes: content.bytes.byteLength },
|
||||
signal,
|
||||
);
|
||||
try {
|
||||
|
||||
+18
-8
@@ -24,14 +24,24 @@ export const registerEvmCommands = (
|
||||
.describe("Selected carrier representation"),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectEvmInterface, async () => {
|
||||
const result = await service.inspect(
|
||||
{ path: resolve(args.path), encoding: args.encoding },
|
||||
{ signal },
|
||||
withCommandCancellation(async (signal) => {
|
||||
try {
|
||||
return await logCliCommand(
|
||||
logger,
|
||||
CLI_COMMANDS.inspectEvmInterface,
|
||||
async () => {
|
||||
const result = await service.inspect(
|
||||
{ path: resolve(args.path), encoding: args.encoding },
|
||||
{ signal },
|
||||
);
|
||||
return result.ok
|
||||
? result.value
|
||||
: projectAnalysisError(result.error);
|
||||
},
|
||||
);
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
}),
|
||||
),
|
||||
} finally {
|
||||
await service.close();
|
||||
}
|
||||
}),
|
||||
});
|
||||
};
|
||||
|
||||
@@ -26,8 +26,12 @@ export const registerFirmwareCommands = (
|
||||
) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, name, async () => {
|
||||
const result = await service.execute(operation, input, { signal });
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
try {
|
||||
const result = await service.execute(operation, input, { signal });
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
} finally {
|
||||
await service.close();
|
||||
}
|
||||
}),
|
||||
);
|
||||
const path = z.string().describe("Local firmware path; never executed");
|
||||
|
||||
@@ -41,29 +41,33 @@ export const registerJavaScriptRecoveryCommands = (
|
||||
),
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(
|
||||
logger,
|
||||
CLI_COMMANDS.recoverJavaScriptSources,
|
||||
async () => {
|
||||
const result = await service.recover(
|
||||
{
|
||||
path: args.path,
|
||||
output_directory: args.outputDirectory,
|
||||
...(options.extractionMode === undefined
|
||||
? {}
|
||||
: { extraction_mode: options.extractionMode }),
|
||||
...(options.rewriteLevel === undefined
|
||||
? {}
|
||||
: { rewrite_level: options.rewriteLevel }),
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
return result.ok
|
||||
? result.value
|
||||
: projectAnalysisError(result.error);
|
||||
},
|
||||
),
|
||||
),
|
||||
withCommandCancellation(async (signal) => {
|
||||
try {
|
||||
return await logCliCommand(
|
||||
logger,
|
||||
CLI_COMMANDS.recoverJavaScriptSources,
|
||||
async () => {
|
||||
const result = await service.recover(
|
||||
{
|
||||
path: args.path,
|
||||
output_directory: args.outputDirectory,
|
||||
...(options.extractionMode === undefined
|
||||
? {}
|
||||
: { extraction_mode: options.extractionMode }),
|
||||
...(options.rewriteLevel === undefined
|
||||
? {}
|
||||
: { rewrite_level: options.rewriteLevel }),
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
return result.ok
|
||||
? result.value
|
||||
: projectAnalysisError(result.error);
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
await service.close();
|
||||
}
|
||||
}),
|
||||
});
|
||||
};
|
||||
|
||||
@@ -39,6 +39,7 @@ const supportedClientSchema = z
|
||||
(candidate) => supportedClientIds.some((name) => name === candidate),
|
||||
"Unsupported agent integration",
|
||||
);
|
||||
const skillClientSchema = z.union([z.literal("shared"), supportedClientSchema]);
|
||||
|
||||
/** Register setup, doctor, uninstall, and update CLI commands. */
|
||||
export const registerSetupCommands = (
|
||||
@@ -72,6 +73,10 @@ const registerSetupCommand = (
|
||||
.array(supportedClientSchema)
|
||||
.default([])
|
||||
.describe("Agent integration to configure; repeat for multiple agents"),
|
||||
skillClient: z
|
||||
.array(skillClientSchema)
|
||||
.default([])
|
||||
.describe("Existing client skill destination to update; repeatable"),
|
||||
allDetected: z
|
||||
.boolean()
|
||||
.default(false)
|
||||
@@ -186,6 +191,7 @@ interface SetupCommandOptions {
|
||||
readonly yes: boolean;
|
||||
readonly installHopper: boolean;
|
||||
readonly client: readonly string[];
|
||||
readonly skillClient: readonly string[];
|
||||
readonly allDetected: boolean;
|
||||
readonly skill?: boolean | undefined;
|
||||
readonly dryRun: boolean;
|
||||
@@ -224,8 +230,13 @@ const setupRunOptions = (
|
||||
): SetupOptions => {
|
||||
const { options } = input;
|
||||
const agentIntegrationSelected =
|
||||
options.allDetected || options.client.length > 0;
|
||||
const hasSelectedClients = hasExplicitScope && !options.allDetected;
|
||||
options.allDetected ||
|
||||
options.client.length > 0 ||
|
||||
options.skillClient.length > 0;
|
||||
const hasSelectedClients =
|
||||
hasExplicitScope &&
|
||||
!options.allDetected &&
|
||||
(options.client.length > 0 || options.skillClient.length > 0);
|
||||
return {
|
||||
approved: options.yes && !options.dryRun,
|
||||
installHopper: options.installHopper,
|
||||
@@ -234,6 +245,9 @@ const setupRunOptions = (
|
||||
allDetectedClients: options.allDetected,
|
||||
proposeHopper: interactive || options.installHopper,
|
||||
...(hasSelectedClients ? { clientIds: options.client } : {}),
|
||||
...(options.skillClient.length > 0
|
||||
? { skillClientIds: options.skillClient }
|
||||
: {}),
|
||||
...(hasExplicitScope
|
||||
? { installSkill: options.skill ?? agentIntegrationSelected }
|
||||
: {}),
|
||||
@@ -264,6 +278,7 @@ const setupIsInteractive = (
|
||||
const setupHasExplicitScope = (options: SetupCommandOptions): boolean =>
|
||||
options.allDetected ||
|
||||
options.client.length > 0 ||
|
||||
options.skillClient.length > 0 ||
|
||||
options.skill !== undefined ||
|
||||
options.installHopper;
|
||||
|
||||
|
||||
@@ -38,19 +38,32 @@ export const registerWebSourceLocationCommands = (
|
||||
.describe("Zero-based UTF-16 generated column"),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.traceWebSourceLocation, async () => {
|
||||
const result = await service.trace(
|
||||
{
|
||||
manifest_path: args.manifestPath,
|
||||
script_index: args.scriptIndex,
|
||||
source_map: { path: args.sourceMapPath, url: args.sourceMapUrl },
|
||||
generated_position: { line: args.line, column: args.column },
|
||||
withCommandCancellation(async (signal) => {
|
||||
try {
|
||||
return await logCliCommand(
|
||||
logger,
|
||||
CLI_COMMANDS.traceWebSourceLocation,
|
||||
async () => {
|
||||
const result = await service.trace(
|
||||
{
|
||||
manifest_path: args.manifestPath,
|
||||
script_index: args.scriptIndex,
|
||||
source_map: {
|
||||
path: args.sourceMapPath,
|
||||
url: args.sourceMapUrl,
|
||||
},
|
||||
generated_position: { line: args.line, column: args.column },
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
return result.ok
|
||||
? result.value
|
||||
: projectAnalysisError(result.error);
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
}),
|
||||
),
|
||||
} finally {
|
||||
await service.close();
|
||||
}
|
||||
}),
|
||||
});
|
||||
};
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import { readCliJsonFile } from "./cliJsonFile.js";
|
||||
import { readJsonInputFile } from "./application/JsonInputFile.js";
|
||||
import { NonRegularFileReadError } from "./filesystem/RegularFile.js";
|
||||
import {
|
||||
AnalysisAccessDeniedError,
|
||||
@@ -21,7 +21,7 @@ export const parseCliJsonInput = async (
|
||||
const inline = safeParseJson(value);
|
||||
if (inline.ok) return { ok: true, value: inline.value };
|
||||
try {
|
||||
const parsed = await readCliJsonFile(value, operation);
|
||||
const parsed = await readJsonInputFile(value, operation);
|
||||
return parsed.ok
|
||||
? { ok: true, value: parsed.value }
|
||||
: {
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { adbInputSchemas } from "./adbDeviceAnalysis.js";
|
||||
|
||||
it("preserves ADB key, action and component admission when making advertised patterns portable", () => {
|
||||
const serial = "emulator-5554";
|
||||
const variants = [
|
||||
{
|
||||
previous: /^[A-Za-z0-9._-]+$/u,
|
||||
max: 256,
|
||||
accepts: (key: string) =>
|
||||
adbInputSchemas.read_adb_setting.safeParse({
|
||||
serial,
|
||||
namespace: "global",
|
||||
key,
|
||||
}).success,
|
||||
},
|
||||
{
|
||||
previous: /^[A-Za-z0-9._-]+$/u,
|
||||
max: 256,
|
||||
accepts: (key: string) =>
|
||||
adbInputSchemas.start_adb_activity.safeParse({
|
||||
serial,
|
||||
action: "android.intent.action.VIEW",
|
||||
extras: [{ key, type: "string", value: "payload" }],
|
||||
}).success,
|
||||
},
|
||||
{
|
||||
previous: /^[A-Za-z][A-Za-z0-9._-]*$/u,
|
||||
max: 256,
|
||||
accepts: (action: string) =>
|
||||
adbInputSchemas.start_adb_activity.safeParse({ serial, action })
|
||||
.success,
|
||||
},
|
||||
{
|
||||
previous: /^[A-Za-z0-9._-]+(\/[A-Za-z0-9._-]*)?$/u,
|
||||
max: 512,
|
||||
accepts: (component: string) =>
|
||||
adbInputSchemas.start_adb_activity.safeParse({
|
||||
serial,
|
||||
action: "android.intent.action.VIEW",
|
||||
component,
|
||||
}).success,
|
||||
},
|
||||
];
|
||||
const samples = [
|
||||
"",
|
||||
"screen_brightness",
|
||||
"0",
|
||||
"_",
|
||||
".",
|
||||
"-",
|
||||
"android.intent.action.VIEW",
|
||||
"com.example/.MainActivity",
|
||||
"com.example/",
|
||||
"com.example/a/b",
|
||||
"a\n",
|
||||
"a\r\n",
|
||||
"é",
|
||||
"aé",
|
||||
"a b",
|
||||
"a\\b",
|
||||
"a".repeat(256),
|
||||
"a".repeat(257),
|
||||
"a".repeat(512),
|
||||
"a".repeat(513),
|
||||
];
|
||||
for (let code = 0; code < 128; code += 1) {
|
||||
const character = String.fromCharCode(code);
|
||||
samples.push(character, `a${character}`, `a${character}b`);
|
||||
}
|
||||
for (const variant of variants)
|
||||
for (const sample of samples)
|
||||
expect(
|
||||
variant.accepts(sample),
|
||||
JSON.stringify({ pattern: variant.previous.source, sample }),
|
||||
).toBe(
|
||||
sample.length > 0 &&
|
||||
sample.length <= variant.max &&
|
||||
variant.previous.test(sample),
|
||||
);
|
||||
});
|
||||
@@ -1,5 +1,6 @@
|
||||
import { z } from "zod";
|
||||
import { localPathStringSchema } from "../localPath.js";
|
||||
import { IDENTIFIER_PATTERN } from "../stringPatterns.js";
|
||||
|
||||
/**
|
||||
* Device serials as adb prints them: emulator-5554, USB identifiers, and
|
||||
@@ -29,6 +30,9 @@ const packageNameSchema = z
|
||||
"Package name must be dot-separated Java identifiers",
|
||||
);
|
||||
|
||||
const keyStringSchema = z.string().min(1).max(256);
|
||||
const keyPattern = /^[A-Za-z0-9._\x2d]+$/u;
|
||||
|
||||
/**
|
||||
* Caller intent for ADB-backed device inspection and acquisition.
|
||||
*
|
||||
@@ -174,14 +178,8 @@ export const adbInputSchemas = {
|
||||
namespace: z
|
||||
.enum(["system", "secure", "global"])
|
||||
.describe("Settings namespace to read"),
|
||||
key: z
|
||||
.string()
|
||||
.min(1)
|
||||
.max(256)
|
||||
.regex(
|
||||
/^[A-Za-z0-9._\-]+$/u,
|
||||
"Setting keys are restricted to identifiers",
|
||||
)
|
||||
key: keyStringSchema
|
||||
.regex(keyPattern, "Setting keys are restricted to identifiers")
|
||||
.describe("Setting key to read"),
|
||||
}),
|
||||
collect_adb_bugreport: z.strictObject({
|
||||
@@ -241,7 +239,7 @@ export const adbInputSchemas = {
|
||||
.min(1)
|
||||
.max(256)
|
||||
.regex(
|
||||
/^[A-Za-z][A-Za-z0-9._\-]*$/u,
|
||||
IDENTIFIER_PATTERN,
|
||||
"Intent action must be a dotted identifier such as android.intent.action.VIEW",
|
||||
)
|
||||
.describe("Intent action to start"),
|
||||
@@ -257,7 +255,7 @@ export const adbInputSchemas = {
|
||||
.min(1)
|
||||
.max(512)
|
||||
.regex(
|
||||
/^[A-Za-z0-9._\-]+(\/[A-Za-z0-9._\-]*)?$/u,
|
||||
/^[A-Za-z0-9._\x2d]+(\/[A-Za-z0-9._\x2d]*)?$/u,
|
||||
"Component must be package/activity form",
|
||||
)
|
||||
.optional()
|
||||
@@ -265,11 +263,8 @@ export const adbInputSchemas = {
|
||||
extras: z
|
||||
.array(
|
||||
z.strictObject({
|
||||
key: z
|
||||
.string()
|
||||
.min(1)
|
||||
.max(256)
|
||||
.regex(/^[A-Za-z0-9._\-]+$/u, "Extra keys are identifiers")
|
||||
key: keyStringSchema
|
||||
.regex(keyPattern, "Extra keys are identifiers")
|
||||
.describe("Extra key"),
|
||||
type: z
|
||||
.enum(["string", "boolean", "int", "long", "float"])
|
||||
@@ -704,3 +699,39 @@ export const adbResultSchemas = {
|
||||
package_name: packageNameSchema,
|
||||
}),
|
||||
} as const;
|
||||
|
||||
/** Acquired package facts retained when cancellation leaves caller-owned output. */
|
||||
export type AdbPackagePullPartialObservation = {
|
||||
readonly provider_id: "adb";
|
||||
readonly operation: "pull_adb_package";
|
||||
readonly result: z.infer<typeof adbResultSchemas.pull_adb_package>;
|
||||
};
|
||||
|
||||
/** A completed acquisition whose digest read was cancelled before verification. */
|
||||
export type AdbAcquiredFilePartialObservation =
|
||||
| {
|
||||
readonly provider_id: "adb";
|
||||
readonly operation: "pull_adb_file";
|
||||
readonly result: {
|
||||
readonly client: z.infer<
|
||||
typeof adbResultSchemas.pull_adb_file
|
||||
>["client"];
|
||||
readonly serial: string;
|
||||
readonly local_path: string;
|
||||
readonly device_path: string;
|
||||
readonly digest_status: "unknown";
|
||||
};
|
||||
}
|
||||
| {
|
||||
readonly provider_id: "adb";
|
||||
readonly operation: "collect_adb_bugreport";
|
||||
readonly result: {
|
||||
readonly client: z.infer<
|
||||
typeof adbResultSchemas.collect_adb_bugreport
|
||||
>["client"];
|
||||
readonly serial: string;
|
||||
readonly local_path: string;
|
||||
readonly adb_reported_path: string;
|
||||
readonly digest_status: "unknown";
|
||||
};
|
||||
};
|
||||
|
||||
@@ -99,6 +99,11 @@ import type { BrowserScenarioPartialObservation } from "./browserScenarioCapture
|
||||
import type { Evidence } from "./evidence.js";
|
||||
import type { AndroidPartialObservation } from "./android/androidPartialObservation.js";
|
||||
import type { FirmwarePartialObservation } from "./firmware/firmwareAnalysis.js";
|
||||
import type {
|
||||
AdbAcquiredFilePartialObservation,
|
||||
AdbPackagePullPartialObservation,
|
||||
} from "./adb/adbDeviceAnalysis.js";
|
||||
import type { NativeUiObservationResult } from "./native/nativeUiObservation.js";
|
||||
|
||||
/** Provider-neutral evidence collected before a typed analysis failure. */
|
||||
export type AnalysisPartialObservation =
|
||||
@@ -111,4 +116,7 @@ export type AnalysisPartialObservation =
|
||||
| ArtifactInventoryPartialObservation
|
||||
| Evidence
|
||||
| AndroidPartialObservation
|
||||
| FirmwarePartialObservation;
|
||||
| FirmwarePartialObservation
|
||||
| NativeUiObservationResult
|
||||
| AdbAcquiredFilePartialObservation
|
||||
| AdbPackagePullPartialObservation;
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import { ProviderAdapterError } from "./providerAdapterError.js";
|
||||
import { projectAnalysisError } from "./analysisErrorProjection.js";
|
||||
import type { AnalysisError } from "./analysisErrorBase.js";
|
||||
|
||||
/** Preserve the original lifecycle failure alongside incomplete cleanup. */
|
||||
export const analysisErrorWithCleanupFailure = (
|
||||
primary: AnalysisError,
|
||||
cleanup: AnalysisError,
|
||||
operation = "close_binary",
|
||||
): AnalysisError => {
|
||||
const projected = projectAnalysisError(cleanup);
|
||||
return new ProviderAdapterError(
|
||||
cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea",
|
||||
operation,
|
||||
{
|
||||
cause: primary,
|
||||
userMessage: projectAnalysisError(primary).message,
|
||||
...(primary.capturedOutput === undefined
|
||||
? {}
|
||||
: { capturedOutput: primary.capturedOutput }),
|
||||
...(primary.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: primary.partialObservation }),
|
||||
cleanup: {
|
||||
reason: projected.message,
|
||||
resources: [
|
||||
...new Set([
|
||||
...primary.cleanupResources,
|
||||
...cleanup.cleanupResources,
|
||||
]),
|
||||
],
|
||||
},
|
||||
diagnostics: {
|
||||
primary_error: projectAnalysisError(primary),
|
||||
cleanup_error: projected,
|
||||
},
|
||||
},
|
||||
);
|
||||
};
|
||||
@@ -66,7 +66,9 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
let omittedHierarchyReferences = 0;
|
||||
for (const document of input.documents.slice(0, input.limits.max_documents)) {
|
||||
const parsed = parseInterfaceBuilderRecords(document.raw);
|
||||
const prefix = `ib:${document.relativePath}:`;
|
||||
const prefix = `ib:${JSON.stringify(document.relativePath)}:`;
|
||||
const graphId = (kind: string, parts: readonly unknown[] = []): string =>
|
||||
`${prefix}${kind}:${JSON.stringify(parts)}`;
|
||||
const rootId = `${prefix}root`;
|
||||
const objectIds = new Set(parsed.objects.map(({ id }) => id));
|
||||
const evidenceFor = (description: string) => [
|
||||
@@ -104,7 +106,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
addNode(rootNode);
|
||||
let documentObjectCount = 0;
|
||||
for (const object of parsed.objects.slice(0, input.limits.max_objects)) {
|
||||
const id = `${prefix}object:${object.id}`;
|
||||
const id = graphId("object", [object.id]);
|
||||
const kind = object.kind === "other" ? "unknown" : object.kind;
|
||||
if (
|
||||
!addNode({
|
||||
@@ -129,7 +131,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
let documentOmittedConnections = 0;
|
||||
let documentOmittedHierarchyEdges = 0;
|
||||
const objectNodeId = (objectId: string): string =>
|
||||
`${prefix}object:${objectId}`;
|
||||
graphId("object", [objectId]);
|
||||
const addEdge = (
|
||||
edge: (typeof edges)[number],
|
||||
isHierarchy: boolean,
|
||||
@@ -238,7 +240,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
const known = objectIds.has(objectId);
|
||||
const childId = known
|
||||
? objectNodeId(objectId)
|
||||
: `${prefix}unknown:${objectId}`;
|
||||
: graphId("unknown", [objectId]);
|
||||
if (
|
||||
!known &&
|
||||
!addNode({
|
||||
@@ -252,7 +254,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
)
|
||||
hierarchyTruncated = true;
|
||||
connect(
|
||||
`${prefix}hierarchy:${parentId}:${childId}`,
|
||||
graphId("hierarchy", [parentId, childId]),
|
||||
parentId,
|
||||
childId,
|
||||
"contains",
|
||||
@@ -281,11 +283,11 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
const destinationId =
|
||||
item.destination_id === null ? null : objectNodeId(item.destination_id);
|
||||
if (item.kind === "action") {
|
||||
const actionId = `${prefix}action:${item.id}`;
|
||||
const actionId = graphId("action", [item.id]);
|
||||
const selectorId =
|
||||
item.label === null
|
||||
? null
|
||||
: `${prefix}selector:${item.destination_id ?? "unknown"}:${item.label}`;
|
||||
: graphId("selector", [item.destination_id, item.label]);
|
||||
addNode({
|
||||
id: actionId,
|
||||
kind: "action",
|
||||
@@ -299,7 +301,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
evidence: evidenceFor(`action ${item.id}`),
|
||||
});
|
||||
connect(
|
||||
`${actionId}:source`,
|
||||
graphId("action-edge", [item.id, "source"]),
|
||||
sourceId,
|
||||
actionId,
|
||||
"target_action",
|
||||
@@ -315,7 +317,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
evidence: evidenceFor(`selector ${item.label}`),
|
||||
});
|
||||
connect(
|
||||
`${actionId}:selector`,
|
||||
graphId("action-edge", [item.id, "selector"]),
|
||||
actionId,
|
||||
selectorId,
|
||||
"target_action",
|
||||
@@ -324,7 +326,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
}
|
||||
if (destinationId === null || !nodes.has(destinationId))
|
||||
connect(
|
||||
`${actionId}:destination`,
|
||||
graphId("action-edge", [item.id, "destination"]),
|
||||
actionId,
|
||||
null,
|
||||
"target_action",
|
||||
@@ -333,14 +335,14 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
);
|
||||
else
|
||||
connect(
|
||||
`${actionId}:destination`,
|
||||
graphId("action-edge", [item.id, "destination"]),
|
||||
actionId,
|
||||
destinationId,
|
||||
"target_action",
|
||||
`action destination ${item.destination_id}`,
|
||||
);
|
||||
} else if (item.kind === "outlet") {
|
||||
const outletId = `${prefix}outlet:${item.id}`;
|
||||
const outletId = graphId("outlet", [item.id]);
|
||||
addNode({
|
||||
id: outletId,
|
||||
kind: "outlet",
|
||||
@@ -350,7 +352,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
evidence: evidenceFor(`outlet ${item.id}`),
|
||||
});
|
||||
connect(
|
||||
`${outletId}:source`,
|
||||
graphId("outlet-edge", [item.id, "source"]),
|
||||
sourceId,
|
||||
outletId,
|
||||
"contains",
|
||||
@@ -358,7 +360,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
);
|
||||
if (destinationId === null || !nodes.has(destinationId))
|
||||
connect(
|
||||
`${outletId}:destination`,
|
||||
graphId("outlet-edge", [item.id, "destination"]),
|
||||
outletId,
|
||||
null,
|
||||
"outlet_to",
|
||||
@@ -367,7 +369,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
);
|
||||
else
|
||||
connect(
|
||||
`${outletId}:destination`,
|
||||
graphId("outlet-edge", [item.id, "destination"]),
|
||||
outletId,
|
||||
destinationId,
|
||||
"outlet_to",
|
||||
@@ -376,7 +378,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
} else if (item.kind === "segue") {
|
||||
if (destinationId === null || !nodes.has(destinationId))
|
||||
connect(
|
||||
`${prefix}segue:${index}`,
|
||||
graphId("segue", [index]),
|
||||
sourceId,
|
||||
null,
|
||||
"segue_to",
|
||||
@@ -385,7 +387,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
);
|
||||
else
|
||||
connect(
|
||||
`${prefix}segue:${index}`,
|
||||
graphId("segue", [index]),
|
||||
sourceId,
|
||||
destinationId,
|
||||
"segue_to",
|
||||
@@ -393,7 +395,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
|
||||
);
|
||||
} else {
|
||||
connect(
|
||||
`${prefix}unknown-connection:${index}`,
|
||||
graphId("unknown-connection", [index]),
|
||||
sourceId,
|
||||
null,
|
||||
"contains",
|
||||
|
||||
@@ -15,7 +15,7 @@ import {
|
||||
} from "./javascriptStaticAnalysisHelpers.js";
|
||||
import {
|
||||
propertyName,
|
||||
semanticStaticPropertyName,
|
||||
semanticStaticPropertyKey,
|
||||
} from "./javascriptAstValues.js";
|
||||
import type { JavaScriptFindingContext } from "./javascriptStaticAnalysisState.js";
|
||||
|
||||
@@ -26,6 +26,8 @@ interface NativeBindingInput {
|
||||
readonly kind: ElectronNativeAddonBindingFinding["binding_kind"];
|
||||
readonly moduleKind: ElectronNativeAddonBindingFinding["module_kind"];
|
||||
readonly members: readonly string[];
|
||||
readonly namespaceAccess: boolean;
|
||||
readonly dynamicMemberAccess: boolean;
|
||||
}
|
||||
|
||||
/** Inspect JavaScript-side imports and re-exports of native .node addons. */
|
||||
@@ -45,10 +47,10 @@ const inspectImport = (
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
if (!isNativeSpecifier(node.source.value, "import")) return;
|
||||
const members = node.specifiers.map((specifier) => {
|
||||
if (t.isImportDefaultSpecifier(specifier)) return "default";
|
||||
if (t.isImportNamespaceSpecifier(specifier)) return "*";
|
||||
return propertyName(specifier.imported) || "[dynamic-import]";
|
||||
const members = node.specifiers.flatMap((specifier) => {
|
||||
if (t.isImportDefaultSpecifier(specifier)) return ["default"];
|
||||
if (t.isImportNamespaceSpecifier(specifier)) return [];
|
||||
return [propertyName(specifier.imported)];
|
||||
});
|
||||
addBinding({
|
||||
context,
|
||||
@@ -57,6 +59,10 @@ const inspectImport = (
|
||||
kind: "import",
|
||||
moduleKind: "import",
|
||||
members,
|
||||
namespaceAccess: node.specifiers.some((value) =>
|
||||
t.isImportNamespaceSpecifier(value),
|
||||
),
|
||||
dynamicMemberAccess: false,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -70,9 +76,10 @@ const inspectNamedExport = (
|
||||
!isNativeSpecifier(node.source.value, "import")
|
||||
)
|
||||
return;
|
||||
const members = node.specifiers.map((specifier) => {
|
||||
if (t.isExportSpecifier(specifier)) return propertyName(specifier.local);
|
||||
return "*";
|
||||
const members = node.specifiers.flatMap((specifier) => {
|
||||
if (t.isExportSpecifier(specifier)) return [propertyName(specifier.local)];
|
||||
if (t.isExportDefaultSpecifier(specifier)) return ["default"];
|
||||
return [];
|
||||
});
|
||||
addBinding({
|
||||
context,
|
||||
@@ -81,6 +88,10 @@ const inspectNamedExport = (
|
||||
kind: "re-export",
|
||||
moduleKind: "import",
|
||||
members,
|
||||
namespaceAccess: node.specifiers.some((value) =>
|
||||
t.isExportNamespaceSpecifier(value),
|
||||
),
|
||||
dynamicMemberAccess: false,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -99,7 +110,9 @@ const inspectExportAll = (
|
||||
specifier: node.source.value,
|
||||
kind: "re-export",
|
||||
moduleKind: "import",
|
||||
members: ["*"],
|
||||
members: [],
|
||||
namespaceAccess: true,
|
||||
dynamicMemberAccess: false,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -115,8 +128,7 @@ const inspectRequireBinding = (
|
||||
specifier: required.specifier,
|
||||
kind: "require",
|
||||
moduleKind: "require",
|
||||
members:
|
||||
required.member === null ? bindingMembers(node.id) : [required.member],
|
||||
...(required.hasMemberAccess ? required.access : bindingMembers(node.id)),
|
||||
});
|
||||
};
|
||||
|
||||
@@ -132,27 +144,33 @@ const inspectReExport = (
|
||||
specifier: required.specifier,
|
||||
kind: "re-export",
|
||||
moduleKind: "require",
|
||||
members: [required.member ?? exportedMember(node.left) ?? "*"],
|
||||
...required.access,
|
||||
});
|
||||
};
|
||||
|
||||
const addBinding = (input: NativeBindingInput): void => {
|
||||
const { context, node, specifier, kind } = input;
|
||||
const unique = [
|
||||
...new Set(input.members.filter((member) => member !== "")),
|
||||
].sort(compareUnicodeCodePoints);
|
||||
const members = unique.length === 0 ? ["*"] : unique;
|
||||
const members = [...new Set(input.members)].sort(compareUnicodeCodePoints);
|
||||
addLocatedFinding(context, {
|
||||
collection: context.accumulator.nativeAddonBindings,
|
||||
// Members are a variable-length source-derived list: compositeKey keeps
|
||||
// ["a\0b"] distinct from ["a", "b"] instead of collapsing one binding.
|
||||
key: compositeKey(["native-addon-binding", kind, specifier, members]),
|
||||
key: compositeKey([
|
||||
"native-addon-binding",
|
||||
kind,
|
||||
specifier,
|
||||
members,
|
||||
input.namespaceAccess,
|
||||
input.dynamicMemberAccess,
|
||||
]),
|
||||
node,
|
||||
value: {
|
||||
specifier,
|
||||
binding_kind: kind,
|
||||
module_kind: input.moduleKind,
|
||||
members,
|
||||
namespace_access: input.namespaceAccess,
|
||||
dynamic_member_access: input.dynamicMemberAccess,
|
||||
module_key: null,
|
||||
location: range(node),
|
||||
},
|
||||
@@ -162,13 +180,20 @@ const addBinding = (input: NativeBindingInput): void => {
|
||||
const nativeRequire = (
|
||||
node: t.Node | null | undefined,
|
||||
):
|
||||
| { readonly specifier: string; readonly member: string | null }
|
||||
| {
|
||||
readonly specifier: string;
|
||||
readonly hasMemberAccess: boolean;
|
||||
readonly access: NativeMemberAccess;
|
||||
}
|
||||
| undefined => {
|
||||
let member: string | null = null;
|
||||
let hasMemberAccess = false;
|
||||
while (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) {
|
||||
if (!t.isNode(node.object)) return undefined;
|
||||
const property = semanticStaticPropertyName(node.property, node.computed);
|
||||
if (member === null && property !== "") member = property;
|
||||
const property = semanticStaticPropertyKey(node.property, node.computed);
|
||||
// Walking inward leaves the first member consumed from the addon itself.
|
||||
member = property;
|
||||
hasMemberAccess = true;
|
||||
node = node.object;
|
||||
}
|
||||
if (!t.isCallExpression(node)) return undefined;
|
||||
@@ -181,34 +206,52 @@ const nativeRequire = (
|
||||
return undefined;
|
||||
const specifier = stringValue(argumentNode(node.arguments[0]));
|
||||
return specifier !== undefined && isNativeSpecifier(specifier, "require")
|
||||
? { specifier, member }
|
||||
? {
|
||||
specifier,
|
||||
hasMemberAccess,
|
||||
access: {
|
||||
members: member === null ? [] : [member],
|
||||
namespaceAccess: !hasMemberAccess,
|
||||
dynamicMemberAccess: hasMemberAccess && member === null,
|
||||
},
|
||||
}
|
||||
: undefined;
|
||||
};
|
||||
|
||||
const bindingMembers = (pattern: t.Node): string[] => {
|
||||
if (t.isIdentifier(pattern)) return ["*"];
|
||||
if (!t.isObjectPattern(pattern)) return ["*"];
|
||||
return pattern.properties.map((property) => {
|
||||
if (t.isRestElement(property)) return "*";
|
||||
return propertyName(property.key) || "[dynamic-import]";
|
||||
});
|
||||
interface NativeMemberAccess {
|
||||
readonly members: readonly string[];
|
||||
readonly namespaceAccess: boolean;
|
||||
readonly dynamicMemberAccess: boolean;
|
||||
}
|
||||
|
||||
const bindingMembers = (pattern: t.Node): NativeMemberAccess => {
|
||||
if (!t.isObjectPattern(pattern))
|
||||
return { members: [], namespaceAccess: true, dynamicMemberAccess: false };
|
||||
const members: string[] = [];
|
||||
let namespaceAccess = false;
|
||||
let dynamicMemberAccess = false;
|
||||
for (const property of pattern.properties) {
|
||||
if (t.isRestElement(property)) {
|
||||
namespaceAccess = true;
|
||||
continue;
|
||||
}
|
||||
const key = semanticStaticPropertyKey(property.key, property.computed);
|
||||
if (key === null) dynamicMemberAccess = true;
|
||||
else members.push(key);
|
||||
}
|
||||
return { members, namespaceAccess, dynamicMemberAccess };
|
||||
};
|
||||
|
||||
const isModuleExport = (node: t.Node): boolean => {
|
||||
const name = calleeName(node);
|
||||
return (
|
||||
name === "module.exports" ||
|
||||
name.startsWith("module.exports.") ||
|
||||
name.startsWith("exports.")
|
||||
);
|
||||
};
|
||||
|
||||
const exportedMember = (node: t.Node): string | undefined => {
|
||||
const name = calleeName(node);
|
||||
if (name.startsWith("module.exports."))
|
||||
return name.slice("module.exports.".length);
|
||||
if (name.startsWith("exports.")) return name.slice("exports.".length);
|
||||
return undefined;
|
||||
const keys: (string | null)[] = [];
|
||||
while (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) {
|
||||
keys.unshift(semanticStaticPropertyKey(node.property, node.computed));
|
||||
node = node.object;
|
||||
}
|
||||
if (!t.isIdentifier(node)) return false;
|
||||
return node.name === "exports"
|
||||
? keys.length > 0
|
||||
: node.name === "module" && keys[0] === "exports";
|
||||
};
|
||||
|
||||
const isNativeSpecifier = (
|
||||
|
||||
@@ -39,6 +39,8 @@ export type ElectronBrowserWindowFinding = ElectronBrowserWindowPreload & {
|
||||
readonly web_preferences_status: "object-literal" | "dynamic" | "missing";
|
||||
readonly web_preferences: readonly ElectronWebPreference[];
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
};
|
||||
|
||||
@@ -56,6 +58,8 @@ export type ElectronContextBridgeFinding = ElectronContextBridgeApiKey & {
|
||||
readonly members: readonly string[];
|
||||
readonly unknown_members: number;
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
};
|
||||
|
||||
@@ -109,6 +113,8 @@ type ElectronIpcHandler =
|
||||
|
||||
interface ElectronIpcFindingContext {
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
}
|
||||
|
||||
@@ -134,12 +140,16 @@ export interface ElectronSenderValidationFinding {
|
||||
readonly expected: ElectronStaticValue;
|
||||
readonly enforcement: "unknown";
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
}
|
||||
|
||||
interface ElectronUtilityProcessFindingState {
|
||||
readonly service_name: string | null;
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
}
|
||||
|
||||
@@ -163,8 +173,14 @@ export interface ElectronNativeAddonBindingFinding {
|
||||
readonly specifier: string;
|
||||
readonly binding_kind: "import" | "require" | "re-export";
|
||||
readonly module_kind: "import" | "require";
|
||||
/** Exact requested addon member keys; namespace and dynamic access are separate. */
|
||||
readonly members: readonly string[];
|
||||
/** Omitted in legacy findings; omission means unknown, not false. */
|
||||
readonly namespace_access?: boolean;
|
||||
readonly dynamic_member_access?: boolean;
|
||||
readonly module_key: string | null;
|
||||
/** Recovered factory runtime; absent in older or unbundled findings. */
|
||||
readonly module_runtime?: string;
|
||||
readonly location: JavaScriptSourceRange;
|
||||
}
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user