Compare commits

...
Author SHA1 Message Date
N0zoM1z0 c2e2ed9aa2 test(firmware): retain unverified workers before further launches 2026-10-11 02:38:48 +08:00
N0zoM1z0 1ae4bec45b Merge remote-tracking branch 'origin/main' into test/maintenance-boundary-contracts-20261011 2026-10-11 02:38:48 +08:00
N0zoM1z0 058bc0af35 test: verify current maintenance boundary behavior 2026-10-11 02:34:45 +08:00
morluto dce81ca400 perf(javascript): count candidate relevance without a second adjacency
Scan eligible candidate relations directly for query ambiguity instead of allocating and sorting another traversal index. Preserve direction, relation filters and cycle handling.

Semantic graph and builder suites plus public application workflow CLI tests passed. Full query outputs matched on a generated 20000-node graph; the single repeated-query timing run is indicative only.
2026-10-11 02:32:11 +08:00
morluto bb764d26bd perf(artifacts): stop parsing parent paths after a missing branch
Walk delimiter-terminated parent segments lazily instead of splitting full paths before discovering that no deeper observed parent exists. Preserve canonical paths and observed ancestry.

Verified real ZIP inventory with multiple 30000-component siblings and the full entry-order suite. A 96-member deep ZIP benchmark retained an identical full inventory digest and reduced median scan time from 517.5 to 464.5 ms across five measured inventories.
2026-10-11 02:32:04 +08:00
morluto acd8d86898 perf(android): validate prerequisites once per retained JADX session
Resolve current paths and JVM options and rehash source identities for each request. Probe Java modules and engine capabilities only when creating a new session; remove the superseded full configuration resolver.

Focused configuration, session retirement, cancellation, cleanup and Android MCP boundaries passed. The 34-operation retained session now performs one prerequisite probe. Protocol fixtures do not establish real JADX or JDK performance.
2026-10-11 02:31:57 +08:00
morluto e6d0b3993d perf(process): batch liveness rechecks within ownership phases
Recheck unreadable token and identity observations against one fresh live process snapshot per phase. Preserve token mismatches, fail closed on inspection errors, and retain liveness failure diagnostics.

Validated with focused process ownership, native capture, CLI and MCP boundaries. An eight-member injected-unreadable workload using native ps reduced snapshots from nine to two with equivalent failures; real permission-denied provider behavior was not exercised.
2026-10-11 02:31:07 +08:00
morluto 92a901e0db perf(process): share settlement probes across captured groups 2026-10-11 02:21:50 +08:00
morluto cef6d6d3c4 perf(artifacts): reuse trie membership for case-folded siblings 2026-10-11 02:21:40 +08:00
morluto bd972bf80b perf(javascript): reuse capture completeness during load classification 2026-10-11 02:21:35 +08:00
morluto ac9709a80a perf(reconstruction): index semantic evidence contexts per derivation 2026-10-11 02:21:28 +08:00
morluto d1e6fe2eca test(artifacts): close owned Mach-O readers after each case 2026-10-11 02:11:13 +08:00
morluto a147ecd4b4 perf(artifacts): index observed parents for deep archive paths 2026-10-11 02:10:50 +08:00
morluto 22fdcb8ab7 perf(process): share batched token observation across ownership checks 2026-10-11 02:10:44 +08:00
morluto b22d817073 perf(process): batch ownership tokens during rooted cleanup
Use the host bulk token reader for each cleanup validation while preserving identity checks before and after token reads. Fall back for unsupported readers or omitted rows, retain explicit unavailable observations, and preserve batch and individual failures. Source ownership regressions and real owned-process boundary tests pass; a warm Darwin validation probe reduced median time from 90 ms to 52 ms.
2026-10-11 01:59:30 +08:00
morluto 3d876c5c1a perf(javascript): validate semantic factory inputs once
Construct typed semantic records directly from cloned, normalized input and derived commitments instead of reparsing the same fields. Keep full graph integrity validation and verify caller mutation isolation. Four alternating cold runs preserved complete result digests while median projection time fell about 8 percent and 17 percent for the two 20,000-node capacity workloads.
2026-10-11 01:54:46 +08:00
morluto cf346368c4 perf(javascript): reuse graph buckets and query indexes
Append overlapping observations to node buckets instead of recopying groups, and share one node index across semantic seed admission and traversal. Preserve merged provenance and absent-seed behavior.
2026-10-11 01:54:36 +08:00
morluto e52c0a72c5 fix(browser): retain incomplete WebMCP replay coverage
Mark malformed WebMCP event envelopes, tool lists and entries as invalid protocol values while preserving valid siblings and out-of-scope classifications. Verify toolsAdded and toolsRemoved replay uncertainty through CDP discovery and capture comparison.
2026-10-11 01:54:26 +08:00
morluto 999e3e2098 fix(android): cancel admitted snapshot copies with their operation
Forward caller and timeout cancellation through engine and APK snapshots, release stream handles before removing partial output, and retain the existing root cleanup authority. Verify cancellation after destination bytes appear with a sparse filesystem fixture.
2026-10-11 01:54:20 +08:00
dakjdakdandYsopwnable c1217e711b fix(javascript): retain runtime scope in bundled module relationships 2026-10-11 01:36:12 +08:00
morluto 0d89978333 fix(android): release snapshot streams before closing file handles
Retain both non-auto-closing copy streams and destroy them after verification, before awaiting their FileHandle closure. This releases stream-held descriptor references on success and failure instead of deadlocking Android snapshot cleanup.

Validation: all 33 JADX subprocess integration tests passed on macOS with Node 24.18.0; check:fast and formatting passed. No real JADX provider execution or Linux/Windows verification was performed locally.
2026-10-11 01:12:48 +08:00
morluto 11e3fae697 fix(ci): repair boundary regressions and evidence assertions
Detach cancelled CDP sessions within the cleanup budget and retain event-finalization failures across retries. Honor Git filesystem encodings and exclude the promise-API probe from source-read failures.

Align existing integration regressions with retained process resources, filesystem-order-independent truncation, owned skill files, explicit native access facts and streamed JSON errors.

Validation: 132 acceptance tests passed (2 skipped), 83 affected boundary tests passed, Git/JSON/CLI boundary regressions passed, check:ci, docs:check and test discovery passed. Local verification used Node 24.18.0 on macOS; cross-platform CI remains pending.
2026-10-11 01:00:30 +08:00
morluto 603d04ee3a fix(ci): repair format gate and pin maintenance skill destinations
format:check failed on ApktoolProvider; existingMaintenanceScope now
returns skillDestinations (consumed by maintenance planning), so pin
the field in the exact-shape scope assertions.
2026-10-11 00:33:36 +08:00
morluto 472f313232 refactor(native): match UI controls by explicit object identity 2026-10-11 00:28:39 +08:00
morluto 1f7e279aa2 fix(adb): retain rollback ownership and cancelled acquisition facts 2026-10-11 00:28:39 +08:00
morluto db8ccaa251 fix(android): bind JADX snapshots and hashes to admitted sources 2026-10-11 00:28:25 +08:00
morluto 5daff4d67e fix(artifacts): retain Mach-O source identity through inventory 2026-10-11 00:28:25 +08:00
morluto a9fd1aecae fix(application): verify configuration and reference input reads 2026-10-11 00:28:24 +08:00
morluto abb8e81d3a fix(filesystem): bound and verify admitted file and JSON reads 2026-10-11 00:28:24 +08:00
94 changed files with 3491 additions and 719 deletions
+6
View File
@@ -136,6 +136,12 @@ existing analysis families directly, for example
- Pull and capture outputs are additive: the destination must not already
exist. Re-running a completed pull refuses the existing directory instead
of overwriting evidence.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Failed package acquisition rollback retains its directory identity and retries
cleanup before later work and during shutdown. Cleanup refuses a replacement
directory. Terminal shutdown stops admission and drains active operations.
- The provider holds no long-lived processes; every invocation is one
bounded adb run. The adb server belongs to the caller.
+348 -46
View File
@@ -1,6 +1,6 @@
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { createReadStream, constants as fsConstants } from "node:fs";
import { constants as fsConstants } from "node:fs";
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
import { basename, join, resolve } from "node:path";
@@ -24,8 +24,11 @@ import {
AnalysisTimeoutError,
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
execFileOutput,
execFileOutputFailure,
@@ -97,11 +100,18 @@ const STDERR_EXCERPT_BYTES = 512;
const UNPARSED_LINES_REPORTED = 10;
const STOP_GRACE_MS = 5_000;
/** Stream a local file digest without retaining large files in memory. */
const sha256File = async (path: string): Promise<string> => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
const hashAdbFile = async (
path: string,
operation: string,
signal: AbortSignal | undefined,
) => {
try {
return await hashStableFile(path, signal);
} catch (cause: unknown) {
if (signal?.aborted === true)
throw new AnalysisCancelledError(operation, { cause });
throw cause;
}
};
/** adb joins shell operands into a remote POSIX command; quote at that boundary. */
@@ -335,6 +345,48 @@ export interface AdbProviderOptions {
readonly environment?: Readonly<Record<string, string | undefined>>;
}
interface PendingPackageDirectoryCleanup {
readonly path: string;
readonly identity: PackageDirectoryIdentity | undefined;
readonly primary: AnalysisError;
readonly remove: () => Promise<void>;
}
interface PackageDirectoryIdentity {
readonly dev: number;
readonly ino: number;
readonly mode: number;
}
const removeOwnedPackageDirectory = async (
path: string,
identity: PackageDirectoryIdentity | undefined,
): Promise<void> => {
if (identity === undefined)
throw new Error(
`Cannot verify the created package directory identity; refusing to remove ${path}`,
);
let current: Awaited<ReturnType<typeof lstat>>;
try {
current = await lstat(path);
} catch (cause: unknown) {
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
return;
throw cause;
}
if (
!current.isDirectory() ||
current.isSymbolicLink() ||
current.dev !== identity.dev ||
current.ino !== identity.ino ||
current.mode !== identity.mode
)
throw new Error(
`The created package directory was replaced or changed; refusing to remove ${path}`,
);
await rm(path, { recursive: true, force: true });
};
/**
* ADB device inspection and acquisition over a caller-selected adb binary.
*
@@ -346,6 +398,12 @@ export interface AdbProviderOptions {
*/
export class AdbProvider {
private readonly environment: Readonly<Record<string, string | undefined>>;
readonly #pendingPackageDirectoryCleanup =
new Set<PendingPackageDirectoryCleanup>();
readonly #activeOperations = new Set<Promise<void>>();
#closing = false;
#closePromise: Promise<void> | undefined;
#packageCleanupPromise: Promise<void> | undefined;
constructor(options: AdbProviderOptions = {}) {
this.environment = options.environment ?? process.env;
@@ -378,12 +436,121 @@ export class AdbProvider {
}
}
/** The provider holds no long-lived resources; the adb server is caller-owned. */
async close(): Promise<void> {}
/** Retry removal of package directories retained after failed rollback. */
close(): Promise<void> {
this.#closing = true;
if (this.#closePromise !== undefined) return this.#closePromise;
const closing = (async () => {
await Promise.all(this.#activeOperations);
await this.#retryPackageDirectoryCleanup();
})().finally(() => {
if (this.#closePromise === closing) this.#closePromise = undefined;
});
this.#closePromise = closing;
return this.#closePromise;
}
#retryPackageDirectoryCleanup(): Promise<void> {
if (this.#packageCleanupPromise !== undefined)
return this.#packageCleanupPromise;
const cleanup = this.#performPackageDirectoryCleanup().finally(() => {
if (this.#packageCleanupPromise === cleanup)
this.#packageCleanupPromise = undefined;
});
this.#packageCleanupPromise = cleanup;
return cleanup;
}
async #performPackageDirectoryCleanup(): Promise<void> {
const failures: AnalysisError[] = [];
for (const owner of this.#pendingPackageDirectoryCleanup) {
try {
await owner.remove();
this.#pendingPackageDirectoryCleanup.delete(owner);
} catch (cause: unknown) {
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[owner.path],
{
reason: cause instanceof Error ? cause.message : String(cause),
},
{ operation: "pull_adb_package", cause },
);
failures.push(
analysisErrorWithCleanupFailure(
owner.primary,
cleanup,
"pull_adb_package",
),
);
}
}
if (failures.length === 1) throw failures[0];
if (failures.length > 1)
throw new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[...this.#pendingPackageDirectoryCleanup].map(({ path }) => path),
{
reason:
"Failed package output directories remain after cleanup retry",
failures: failures.map(({ message, cleanupResources }) => ({
message,
resources: [...cleanupResources],
})),
},
{
operation: "pull_adb_package",
cause: new AggregateError(
failures,
"Multiple ADB package rollback cleanups failed",
),
},
);
}
async execute(
request: AdbRequest,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
if (this.#closing)
return err(new AnalysisCancelledError(request.operation));
let release!: () => void;
const active = new Promise<void>((resolve) => {
release = resolve;
});
this.#activeOperations.add(active);
try {
if (this.#pendingPackageDirectoryCleanup.size > 0) {
try {
await this.#retryPackageDirectoryCleanup();
} catch (cause: unknown) {
return err(
cause instanceof AnalysisError
? cause
: new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[...this.#pendingPackageDirectoryCleanup].map(
({ path }) => path,
),
{
reason:
cause instanceof Error ? cause.message : String(cause),
},
{ operation: request.operation, cause },
),
);
}
}
return await this.#executeOperation(request, options);
} finally {
this.#activeOperations.delete(active);
release();
}
}
async #executeOperation(
request: AdbRequest,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
const signal = options?.signal;
if (signal?.aborted === true)
@@ -1010,9 +1177,11 @@ export class AdbProvider {
);
if (!directory.ok) return directory;
const packageDirectory = join(directory.value, packageName);
let packageDirectoryCreated = false;
let packageDirectoryIdentity: PackageDirectoryIdentity | undefined;
try {
if (
await stat(packageDirectory).then(
await lstat(packageDirectory).then(
() => true,
() => false,
)
@@ -1021,14 +1190,44 @@ export class AdbProvider {
`The package output directory already exists: ${packageDirectory}`,
);
await mkdir(packageDirectory);
packageDirectoryCreated = true;
const created = await lstat(packageDirectory);
if (!created.isDirectory() || created.isSymbolicLink())
throw new Error(
`The created package output is not a regular directory: ${packageDirectory}`,
);
packageDirectoryIdentity = {
dev: created.dev,
ino: created.ino,
mode: created.mode,
};
} catch (cause) {
return err(
new AnalysisOutputError(
operation,
cause instanceof Error ? cause.message : String(cause),
{ cause },
),
const primary = new AnalysisOutputError(
operation,
cause instanceof Error ? cause.message : String(cause),
{ cause },
);
if (!packageDirectoryCreated) return err(primary);
const owner: PendingPackageDirectoryCleanup = {
path: packageDirectory,
identity: packageDirectoryIdentity,
primary,
remove: () =>
removeOwnedPackageDirectory(
packageDirectory,
packageDirectoryIdentity,
),
};
this.#pendingPackageDirectoryCleanup.add(owner);
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[packageDirectory],
{
reason: "Could not establish an owned package directory identity",
},
{ operation, cause },
);
return err(analysisErrorWithCleanupFailure(primary, cleanup, operation));
}
const artifacts: {
device_path: string;
@@ -1044,6 +1243,37 @@ export class AdbProvider {
stage: "resolve_paths" | "pull" | "digest";
message: string;
}[] = [];
const cancelledPackagePull = (
error: AnalysisCancelledError,
localPath: string,
devicePath: string,
stage: "pull" | "digest",
) => {
error.retainPartialObservation({
provider_id: "adb",
operation,
result: {
client,
serial,
package_name: packageName,
output_directory: directory.value,
artifacts,
failures: [
...failures,
{
device_path: devicePath,
stage,
message:
stage === "pull"
? `Pull cancelled; any incomplete file is retained at ${localPath}. Remaining APKs were not attempted.`
: `Digesting the pulled file was cancelled; the file is retained at ${localPath}. Remaining APKs were not attempted.`,
},
],
coverage: "partial",
},
});
return err(error);
};
const seenFileNames = new Set<string>();
for (const entry of parsed.paths) {
if (seenFileNames.has(entry.file_name)) {
@@ -1065,7 +1295,13 @@ export class AdbProvider {
signal,
);
if (!pulled.ok) {
if (pulled.error instanceof AnalysisCancelledError) return pulled;
if (pulled.error instanceof AnalysisCancelledError)
return cancelledPackagePull(
pulled.error,
localPath,
entry.device_path,
"pull",
);
failures.push({
device_path: entry.device_path,
stage: "pull",
@@ -1074,18 +1310,24 @@ export class AdbProvider {
continue;
}
try {
const info = await stat(localPath);
const sha256 = await sha256File(localPath);
const digest = await hashAdbFile(localPath, operation, signal);
artifacts.push({
device_path: entry.device_path,
file_name: entry.file_name,
local_path: localPath,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
role: entry.role,
role_basis: "file_name",
});
} catch {
} catch (cause: unknown) {
if (cause instanceof AnalysisCancelledError)
return cancelledPackagePull(
cause,
localPath,
entry.device_path,
"digest",
);
failures.push({
device_path: entry.device_path,
stage: "digest",
@@ -1094,14 +1336,36 @@ export class AdbProvider {
}
}
if (artifacts.length === 0 && failures.length > 0) {
await rm(packageDirectory, { recursive: true, force: true }).catch(
() => undefined,
);
return err(
new AnalysisProtocolError(
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
),
const primary = new AnalysisProtocolError(
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
);
const owner: PendingPackageDirectoryCleanup = {
path: packageDirectory,
identity: packageDirectoryIdentity,
primary,
remove: () =>
removeOwnedPackageDirectory(
packageDirectory,
packageDirectoryIdentity,
),
};
try {
await owner.remove();
} catch (cause: unknown) {
this.#pendingPackageDirectoryCleanup.add(owner);
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[packageDirectory],
{
reason: cause instanceof Error ? cause.message : String(cause),
},
{ operation: operation, cause },
);
return err(
analysisErrorWithCleanupFailure(primary, cleanup, operation),
);
}
return err(primary);
}
return ok(
createAnalysisExecution(
@@ -1156,8 +1420,7 @@ export class AdbProvider {
);
if (!pulled.ok) return pulled;
try {
const info = await stat(localPath);
const sha256 = await sha256File(localPath);
const digest = await hashAdbFile(localPath, operation, signal);
return ok(
createAnalysisExecution(
{
@@ -1165,8 +1428,8 @@ export class AdbProvider {
serial: input.serial,
device_path: input.device_path,
local_path: localPath,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
},
provider,
{
@@ -1177,6 +1440,20 @@ export class AdbProvider {
),
);
} catch (cause) {
if (cause instanceof AnalysisCancelledError) {
cause.retainPartialObservation({
provider_id: "adb",
operation,
result: {
client,
serial: input.serial,
local_path: localPath,
device_path: input.device_path,
digest_status: "unknown",
},
});
return err(cause);
}
return err(
new AnalysisOutputError(
operation,
@@ -1211,7 +1488,13 @@ export class AdbProvider {
cause: new Error(`The local file does not exist: ${localPath}`),
}),
);
const sha256 = await sha256File(localPath);
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
try {
digest = await hashAdbFile(localPath, operation, signal);
} catch (cause: unknown) {
if (cause instanceof AnalysisCancelledError) return err(cause);
throw cause;
}
const existsProbe = await runAdb(
operation,
input.device_path,
@@ -1257,7 +1540,7 @@ export class AdbProvider {
);
if (deviceDigest.ok) {
const digestMatch = /^([a-f0-9]{64})\s/u.exec(deviceDigest.value.stdout);
if (digestMatch !== null && digestMatch[1] !== sha256)
if (digestMatch !== null && digestMatch[1] !== digest.sha256)
return err(
new AnalysisArtifactChangedError(
operation,
@@ -1272,8 +1555,8 @@ export class AdbProvider {
serial: input.serial,
local_path: localPath,
device_path: input.device_path,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
device_sha256: digestMatch?.[1] ?? null,
overwritten: deviceExists,
device_digest_source: "device_sha256sum",
@@ -1294,8 +1577,8 @@ export class AdbProvider {
serial: input.serial,
local_path: localPath,
device_path: input.device_path,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
device_sha256: null,
overwritten: deviceExists,
device_digest_source: "unavailable",
@@ -1334,7 +1617,13 @@ export class AdbProvider {
cause: new Error(`The local APK does not exist: ${apkPath}`),
}),
);
const sha256 = await sha256File(apkPath);
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
try {
digest = await hashAdbFile(apkPath, operation, signal);
} catch (cause: unknown) {
if (cause instanceof AnalysisCancelledError) return err(cause);
throw cause;
}
const installed = await runAdb(
operation,
input.serial,
@@ -1363,8 +1652,8 @@ export class AdbProvider {
client,
serial: input.serial,
apk_path: apkPath,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
replaced: input.replace,
},
provider,
@@ -1734,16 +2023,15 @@ export class AdbProvider {
),
);
try {
const info = await stat(reportedPath);
const sha256 = await sha256File(reportedPath);
const digest = await hashAdbFile(reportedPath, operation, signal);
return ok(
createAnalysisExecution(
{
client,
serial: input.serial,
local_path: reportedPath,
bytes: info.size,
sha256,
bytes: digest.bytes,
sha256: digest.sha256,
adb_reported_path: reportedPath,
},
provider,
@@ -1755,6 +2043,20 @@ export class AdbProvider {
),
);
} catch (cause) {
if (cause instanceof AnalysisCancelledError) {
cause.retainPartialObservation({
provider_id: "adb",
operation,
result: {
client,
serial: input.serial,
local_path: reportedPath,
adb_reported_path: reportedPath,
digest_status: "unknown",
},
});
return err(cause);
}
return err(
new AnalysisOutputError(
operation,
+129 -21
View File
@@ -1,40 +1,43 @@
import { constants, createReadStream } from "node:fs";
import { chmod, copyFile } from "node:fs/promises";
import { createHash } from "node:crypto";
import type { ReadStream, Stats, WriteStream } from "node:fs";
import { chmod, open, rm, stat, type FileHandle } from "node:fs/promises";
import { join } from "node:path";
import { Transform, type TransformCallback } from "node:stream";
import { pipeline } from "node:stream/promises";
import {
AnalysisInputError,
AnalysisCapabilityUnavailableError,
} from "../domain/analysisErrorCore.js";
import {
openRegularFile,
sameRegularFileState,
} from "../filesystem/RegularFile.js";
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
/** Stream a local file digest without retaining the APK or JAR in memory. */
export const hashAndroidFile = async (path: string): Promise<string> => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
};
/** Bind decompiler input to the admitted artifact, even if the original changes. */
export const snapshotAndroidTarget = async (
path: string,
sha256: string,
root: string,
operation: AndroidOperation,
signal?: AbortSignal,
): Promise<string> => {
const snapshot = join(root, "target.apk");
await copyFile(path, snapshot, constants.COPYFILE_EXCL);
await chmod(snapshot, 0o400);
if ((await hashAndroidFile(snapshot)) !== sha256)
throw new AnalysisInputError(operation, undefined, [
try {
await copyAndroidSnapshot(path, snapshot, sha256, signal);
signal?.throwIfAborted();
await chmod(snapshot, 0o400);
return snapshot;
} catch (cause: unknown) {
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
throw new AnalysisInputError(operation, { cause }, [
{
path: ["path"],
reason: "invalid_value",
message:
"APK bytes changed after admission; retry against a stable file.",
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
},
]);
return snapshot;
}
};
/** Fingerprint the actual immutable engine bytes executed for this observation. */
@@ -43,15 +46,120 @@ export const snapshotAndroidEngine = async (
sha256: string,
root: string,
operation: AndroidOperation,
signal?: AbortSignal,
): Promise<{ path: string; sha256: string }> => {
const path = join(root, "engine.jar");
await copyFile(source, path, constants.COPYFILE_EXCL);
await chmod(path, 0o400);
if ((await hashAndroidFile(path)) !== sha256)
try {
const observed = await copyAndroidSnapshot(source, path, sha256, signal);
signal?.throwIfAborted();
await chmod(path, 0o400);
return { path, sha256: observed };
} catch (cause: unknown) {
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
throw new AnalysisCapabilityUnavailableError(
"jadx",
operation,
"REA_JADX_MCP_JAR bytes changed during admission; retry with a stable engine file.",
`REA_JADX_MCP_JAR bytes changed during admission at ${source}; retry with a stable engine file.`,
{ cause },
);
return { path, sha256 };
}
};
const copyAndroidSnapshot = async (
sourcePath: string,
snapshotPath: string,
expectedSha256: string,
signal?: AbortSignal,
): Promise<string> => {
signal?.throwIfAborted();
const source = await openRegularFile(sourcePath, {
symlinks: "follow",
signal,
});
let destination: FileHandle | undefined;
let inputStream: ReadStream | undefined;
let outputStream: WriteStream | undefined;
let removeSnapshot = false;
let copiedBytes = 0;
const hash = createHash("sha256");
try {
const initial = await source.stat();
signal?.throwIfAborted();
const digest = new Transform({
transform(
chunk: Buffer,
_encoding: BufferEncoding,
callback: TransformCallback,
) {
copiedBytes += chunk.length;
if (copiedBytes > initial.size) {
callback(
new AndroidSnapshotChangedError(
`Android input grew while being copied: ${sourcePath}`,
),
);
return;
}
hash.update(chunk);
callback(null, chunk);
},
});
try {
destination = await open(snapshotPath, "wx", 0o600);
signal?.throwIfAborted();
inputStream = source.createReadStream({
start: 0,
end: initial.size,
autoClose: false,
});
outputStream = destination.createWriteStream({ autoClose: false });
await pipeline(inputStream, digest, outputStream, { signal });
signal?.throwIfAborted();
await verifySourceState(source, sourcePath, initial, copiedBytes);
const observedSha256 = hash.digest("hex");
if (observedSha256 !== expectedSha256)
throw new AndroidSnapshotChangedError(
`Android snapshot digest mismatch for ${sourcePath}`,
);
return observedSha256;
} catch (cause: unknown) {
removeSnapshot = destination !== undefined;
throw cause;
}
} finally {
// Non-auto-closing streams retain a reference to their FileHandle even
// after pipeline completion. Release those references before awaiting
// descriptor closure, after the admitted source state has been verified.
inputStream?.destroy();
outputStream?.destroy();
try {
await Promise.all([source.close(), destination?.close()]);
} finally {
if (removeSnapshot)
await rm(snapshotPath, { force: true }).catch(() => undefined);
}
}
};
const verifySourceState = async (
handle: Awaited<ReturnType<typeof openRegularFile>>,
path: string,
initial: Stats,
observedBytes: number,
): Promise<void> => {
const [opened, currentPath] = await Promise.all([handle.stat(), stat(path)]);
if (
observedBytes !== initial.size ||
!sameRegularFileState(initial, opened) ||
!sameRegularFileState(initial, currentPath)
)
throw new AndroidSnapshotChangedError(
`Android input changed while being read: ${path}`,
);
};
class AndroidSnapshotChangedError extends Error {
constructor(message: string, options?: ErrorOptions) {
super(message, options);
}
}
+64 -29
View File
@@ -52,7 +52,11 @@ export const inspectJadxAvailability = async (
signal?: AbortSignal,
): Promise<ProviderAvailability> => {
try {
const configuration = await readJadxConfiguration(environment, signal);
const configuration = await readJadxConfigurationInputs(
environment,
signal,
);
await validateJadxConfigurationContents(configuration, environment, signal);
return {
status: "available",
code: null,
@@ -77,52 +81,83 @@ export const inspectJadxAvailability = async (
}
};
/** Resolve caller-selected tools and preserve the same prerequisite diagnostics. */
export const resolveJadxConfiguration = async (
/** Resolve caller-selected paths and JVM options without probing their contents. */
export const resolveJadxConfigurationInputs = async (
environment: Readonly<Record<string, string | undefined>>,
operation: AndroidOperation,
signal?: AbortSignal,
): Promise<JadxConfiguration> => {
try {
return await readJadxConfiguration(environment, signal);
return await readJadxConfigurationInputs(environment, signal);
} catch (cause) {
if (signal?.aborted === true) throw cause;
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
const output = cause.diagnostics;
throw new AnalysisCapabilityUnavailableError(
"jadx",
operation,
cause.message,
{
cause,
userMessage: cause.message,
...(typeof output.stdout === "string" &&
typeof output.stderr === "string"
? {
capturedOutput: {
stdout: output.stdout,
stderr: output.stderr,
truncated: output.output_truncated === true,
},
}
: {}),
},
);
throw projectConfigurationFailure(operation, cause);
}
};
const readJadxConfiguration = async (
/** Probe Java and archive prerequisites before starting a new engine session. */
export const validateJadxConfiguration = async (
configuration: JadxConfiguration,
environment: Readonly<Record<string, string | undefined>>,
operation: AndroidOperation,
signal?: AbortSignal,
): Promise<void> => {
try {
await validateJadxConfigurationContents(configuration, environment, signal);
} catch (cause) {
if (signal?.aborted === true) throw cause;
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
throw projectConfigurationFailure(operation, cause);
}
};
const readJadxConfigurationInputs = async (
environment: Readonly<Record<string, string | undefined>>,
signal?: AbortSignal,
): Promise<JadxConfiguration> => {
signal?.throwIfAborted();
requireSupportedHost();
const jar = await resolveJar(environment);
const java = await resolveJava(environment, signal);
await requireJavaHomeExecutable(environment.JAVA_HOME, java);
const jvmArguments = resolveJvmArguments(environment);
await inspectJavaRuntime(java, environment, signal);
await requireEngineArchive(jar, signal);
return { jar, java, jvmArguments };
signal?.throwIfAborted();
return { jar, java, jvmArguments: resolveJvmArguments(environment) };
};
const validateJadxConfigurationContents = async (
configuration: JadxConfiguration,
environment: Readonly<Record<string, string | undefined>>,
signal?: AbortSignal,
): Promise<void> => {
signal?.throwIfAborted();
await inspectJavaRuntime(configuration.java, environment, signal);
await requireEngineArchive(configuration.jar, signal);
};
const projectConfigurationFailure = (
operation: AndroidOperation,
cause: JadxConfigurationFailure,
): AnalysisCapabilityUnavailableError => {
const output = cause.diagnostics;
return new AnalysisCapabilityUnavailableError(
"jadx",
operation,
cause.message,
{
cause,
userMessage: cause.message,
...(typeof output.stdout === "string" && typeof output.stderr === "string"
? {
capturedOutput: {
stdout: output.stdout,
stderr: output.stderr,
truncated: output.output_truncated === true,
},
}
: {}),
},
);
};
const requireEngineArchive = async (
+98 -28
View File
@@ -21,12 +21,14 @@ import { err, ok, type Result } from "../domain/result.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import {
snapshotAndroidEngine,
hashAndroidFile,
snapshotAndroidTarget,
} from "./AndroidTargetSnapshot.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
import {
inspectJadxAvailability,
resolveJadxConfiguration,
resolveJadxConfigurationInputs,
validateJadxConfiguration,
} from "./JadxConfiguration.js";
import type { JadxLauncher } from "./JadxMcpTransport.js";
@@ -104,6 +106,50 @@ const executionError = (context: {
});
};
const androidTargetChanged = (
operation: string,
path: string,
cause?: unknown,
): AnalysisInputError =>
new AnalysisInputError(
operation,
cause === undefined ? undefined : { cause },
[
{
path: ["path"],
reason: "invalid_value",
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
},
],
);
const androidSourceChanged = (
operation: string,
source: string,
cause?: unknown,
): AnalysisCapabilityUnavailableError =>
new AnalysisCapabilityUnavailableError(
"jadx",
operation,
`Selected JADX input bytes changed during admission at ${source}; retry with a stable installation.`,
cause === undefined ? undefined : { cause },
);
const hashAndroidSource = async (
path: string,
operation: string,
signal: AbortSignal,
): Promise<string> => {
try {
return (await hashStableFile(path, signal)).sha256;
} catch (cause: unknown) {
if (signal.aborted) throw cause;
if (cause instanceof RegularFileChangedError)
throw androidSourceChanged(operation, path, cause);
throw cause;
}
};
interface PendingCleanup {
session: JadxSession | undefined;
root: PrivateRuntimeRoot | undefined;
@@ -147,17 +193,18 @@ const cleanup = async (
await root?.close();
resources.root = undefined;
} catch (cause) {
return err(
new ProviderCleanupError(
"jadx",
[root?.path ?? "unknown"],
{
...diagnostics,
reason: cause instanceof Error ? cause.message : String(cause),
},
{ cause: previousError ?? cause },
),
const error = new ProviderCleanupError(
"jadx",
[root?.path ?? "unknown"],
{
...diagnostics,
reason: cause instanceof Error ? cause.message : String(cause),
},
{ cause: previousError ?? cause },
);
if (previousError?.partialObservation !== undefined)
error.retainPartialObservation(previousError.partialObservation);
return err(error);
}
return ok(undefined);
};
@@ -287,23 +334,33 @@ export class JadxProvider implements AndroidAnalysisPort {
request.operation,
`Target ${target.path} is ${target.format}; provide one standalone APK.`,
);
const configuration = await resolveJadxConfiguration(
const configuration = await resolveJadxConfigurationInputs(
this.environment,
request.operation,
signal,
);
signal.throwIfAborted();
const jarHash = await hashAndroidFile(configuration.jar);
const bridgeHash = await hashAndroidFile(BRIDGE_SOURCE);
if ((await hashAndroidFile(target.path)) !== target.sha256)
throw new AnalysisInputError(request.operation, undefined, [
{
path: ["path"],
reason: "invalid_value",
message:
"APK bytes changed after admission; retry against a stable file.",
},
]);
const jarHash = await hashAndroidSource(
configuration.jar,
request.operation,
signal,
);
const bridgeHash = await hashAndroidSource(
BRIDGE_SOURCE,
request.operation,
signal,
);
let targetHash: string;
try {
targetHash = (await hashStableFile(target.path, signal)).sha256;
} catch (cause: unknown) {
if (signal.aborted) throw cause;
if (cause instanceof RegularFileChangedError)
throw androidTargetChanged(request.operation, target.path, cause);
throw cause;
}
if (targetHash !== target.sha256)
throw androidTargetChanged(request.operation, target.path);
const key = JSON.stringify({
path: target.path,
sha256: target.sha256,
@@ -326,27 +383,40 @@ export class JadxProvider implements AndroidAnalysisPort {
}
let retained = this.#retained;
if (retained === undefined) {
await validateJadxConfiguration(
configuration,
this.environment,
request.operation,
signal,
);
root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" });
const engine = await snapshotAndroidEngine(
configuration.jar,
jarHash,
root.path,
request.operation,
signal,
);
const snapshot = await snapshotAndroidTarget(
target.path,
target.sha256,
root.path,
request.operation,
signal,
);
const bridge = join(root.path, "ReaJadxBridge.java");
await copyFile(BRIDGE_SOURCE, bridge);
await chmod(bridge, 0o400);
if ((await hashAndroidFile(bridge)) !== bridgeHash)
throw new AnalysisCapabilityUnavailableError(
"jadx",
const copiedBridgeHash = await hashAndroidSource(
bridge,
request.operation,
signal,
);
if (copiedBridgeHash !== bridgeHash)
throw androidSourceChanged(
request.operation,
"REA Android bridge bytes changed during admission; retry with a stable installation.",
BRIDGE_SOURCE,
undefined,
);
signal.throwIfAborted();
session = new JadxSession(
+1 -1
View File
@@ -157,7 +157,7 @@ const runApktool = async (
timeoutMs: limits.timeoutMs,
diagnosticBytes: limits.maxOutputBytes,
},
(signal === undefined ? {} : { signal }),
signal === undefined ? {} : { signal },
);
return ok({ stdout: output.stdout.text, stderr: output.stderr.text });
} catch (cause) {
+2 -2
View File
@@ -1,5 +1,5 @@
import { err, ok, type Result } from "../domain/result.js";
import { readFile } from "node:fs/promises";
import { readRegularFileText } from "./RegularFileRead.js";
import {
effectiveClientServer,
parseClientConfiguration,
@@ -19,7 +19,7 @@ export const readConfigurationText = async (
path: string,
): Promise<Result<string | undefined, ClientConfigurationFileError>> => {
try {
return ok(await readFile(path, "utf8"));
return ok(await readRegularFileText(path));
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
return ok(undefined);
+12 -5
View File
@@ -16,8 +16,7 @@ import {
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../domain/result.js";
import { parseUtf8Json } from "./Utf8JsonInput.js";
import { readRegularFile } from "./RegularFileRead.js";
import { readJsonInputFile } from "./JsonInputFile.js";
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
/** Request control and its owning operation for an interruptible atomic write. */
@@ -37,12 +36,20 @@ export const readJsonFile = async (
> => {
const requestedPath = resolve(path);
try {
const encoded = await readRegularFile(requestedPath);
const decoded = parseUtf8Json(encoded, "read_evidence_file", requestedPath);
const decoded = await readJsonInputFile(
requestedPath,
"read_evidence_file",
);
if (!decoded.ok) {
if (decoded.error instanceof AnalysisResourceConstraintError)
return err(decoded.error);
return err(
new EvidenceFileError("read", "invalid-json", {
cause: decoded.cause ?? new SyntaxError(decoded.error),
cause:
decoded.error.cause ??
new SyntaxError(
decoded.error.issues[0]?.message ?? decoded.error.message,
),
path: requestedPath,
}),
);
@@ -6,61 +6,76 @@ import { getHeapStatistics } from "node:v8";
import parser from "stream-json/parser.js";
import streamValues from "stream-json/streamers/stream-values.js";
import { withRegularFile } from "./application/RegularFileRead.js";
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
import { withRegularFile } from "./RegularFileRead.js";
import {
JSON_BYTE_ORDER_MARK_MESSAGE,
JSON_INPUT_RESOURCE_REMEDIATION,
parseUtf8Json,
} from "./application/Utf8JsonInput.js";
} from "./Utf8JsonInput.js";
import {
AnalysisInputError,
AnalysisResourceConstraintError,
} from "./domain/analysisErrorCore.js";
import { err, ok, type Result } from "./domain/result.js";
} from "../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../domain/result.js";
type JsonFileFailure = AnalysisInputError | AnalysisResourceConstraintError;
type JsonInputFileFailure =
| AnalysisInputError
| AnalysisResourceConstraintError;
const READ_CHUNK_BYTES = 64 * 1024;
const PACKED_STRING_PART_CODE_UNITS = 64 * 1024;
const PACKED_TOKEN_PART_CODE_UNITS = 64 * 1024;
// Retain native-parser speed for modest inputs while bounding its extra copies.
// This selects an implementation; larger files remain accepted through streaming.
const NATIVE_PARSE_READ_BUDGET = 8 * 1024 * 1024;
/** Keep native parsing only when input copies and value expansion fit live headroom. */
const nativeParseFits = (inputBytes: number): boolean =>
inputBytes <= NATIVE_PARSE_READ_BUDGET &&
// The assembler estimate reserves up to 128 bytes per two-byte container.
// Allow that 64x expansion plus byte/text copies before native JSON.parse,
// whose individual container allocations cannot be intercepted.
inputBytes * 72 + READ_CHUNK_BYTES <=
getHeapStatistics().total_available_size;
/** Parse strict UTF-8 JSON without requiring a whole-document string. */
export const readCliJsonFile = (
export const readJsonInputFile = (
path: string,
operation: string,
signal?: AbortSignal,
): Promise<Result<unknown, JsonFileFailure>> =>
): Promise<Result<unknown, JsonInputFileFailure>> =>
withRegularFile(
path,
async (handle, stats) => {
let found = false;
let value: unknown;
try {
const prefix =
stats.size <= NATIVE_PARSE_READ_BUDGET
? await readPrefix(handle, stats.size + 1, signal)
: undefined;
if (prefix?.complete === true) {
const prefix = nativeParseFits(stats.size)
? await readPrefix(handle, stats.size + 1, signal)
: undefined;
if (prefix !== undefined) {
if (!prefix.complete || prefix.bytes.length !== stats.size)
throw new RegularFileChangedError(path);
const parsed = parseUtf8Json(
prefix.bytes,
operation,
path,
"cli-json-input",
"json-file-input",
);
return parsed.ok
? ok(parsed.value)
: err(invalidJson(operation, parsed.error, parsed.cause));
}
await pipeline(
decodedChunks(handle, signal, prefix?.bytes),
decodedChunks(handle, stats.size, path, signal),
parser.asStream({
jsonStreaming: false,
streamValues: false,
packKeys: false,
packStrings: false,
packNumbers: false,
}),
packJsonStrings(operation, stats.size),
packJsonScalars(operation, stats.size, path),
admitJsonAssembly(operation, stats.size, path),
streamValues.asStream(),
async (rows: AsyncIterable<unknown>) => {
for await (const row of rows) {
@@ -107,8 +122,9 @@ export const readCliJsonFile = (
new AnalysisResourceConstraintError(
operation,
"memory",
"An individual JSON string exceeds the runtime string limit",
"An individual JSON string or number token exceeds the runtime string limit",
{
input_path: path,
input_file_bytes: stats.size,
max_string_code_units: constants.MAX_STRING_LENGTH,
},
@@ -121,12 +137,16 @@ export const readCliJsonFile = (
throw cause;
}
},
signal,
{ signal },
);
const packJsonStrings = (operation: string, inputFileBytes: number) =>
const packJsonScalars = (
operation: string,
inputFileBytes: number,
path: string,
) =>
async function* (tokens: AsyncIterable<unknown>): AsyncGenerator<unknown> {
let kind: "keyValue" | "stringValue" | undefined;
let kind: "keyValue" | "stringValue" | "numberValue" | undefined;
let length = 0;
let value = "";
let parts: string[] = [];
@@ -139,23 +159,32 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
case "startString":
kind = token.name === "startKey" ? "keyValue" : "stringValue";
break;
case "startNumber":
kind = "numberValue";
break;
case "numberChunk":
case "stringChunk":
if (
kind === undefined ||
!("value" in token) ||
typeof token.value !== "string"
)
throw new Error("Unexpected JSON string fragment");
throw new Error("Unexpected JSON scalar fragment");
length += token.value.length;
if (length > constants.MAX_STRING_LENGTH)
throw new RangeError("Invalid string length");
parts.push(token.value);
partLength += token.value.length;
if (partLength >= PACKED_STRING_PART_CODE_UNITS) {
if (partLength >= PACKED_TOKEN_PART_CODE_UNITS) {
// The tokenizer emits short fragments. Coalesce them before retaining
// a string so millions of substring/rope nodes cannot exhaust the heap
// a scalar so millions of substring/rope nodes cannot exhaust the heap
// before the native string-length constraint can be reported.
requireStringAssemblyHeadroom(operation, inputFileBytes, length);
requireStringAssemblyHeadroom(
operation,
inputFileBytes,
path,
length,
);
value += parts.join("");
parts = [];
partLength = 0;
@@ -163,8 +192,14 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
break;
case "endKey":
case "endString":
if (kind === undefined) throw new Error("Unexpected JSON string end");
requireStringAssemblyHeadroom(operation, inputFileBytes, length);
case "endNumber":
if (kind === undefined) throw new Error("Unexpected JSON scalar end");
requireStringAssemblyHeadroom(
operation,
inputFileBytes,
path,
length,
);
yield { name: kind, value: value + parts.join("") };
kind = undefined;
length = 0;
@@ -178,22 +213,78 @@ const packJsonStrings = (operation: string, inputFileBytes: number) =>
}
};
/** Admit the retained value graph before the upstream assembler grows it. */
const admitJsonAssembly = (
operation: string,
inputFileBytes: number,
path: string,
) =>
async function* (tokens: AsyncIterable<unknown>): AsyncGenerator<unknown> {
let projectedBytes = 0;
let nextCheck = 0;
for await (const token of tokens) {
if (typeof token !== "object" || token === null || !("name" in token))
throw new Error("Unexpected JSON assembly token");
if (token.name === "startObject" || token.name === "startArray") {
projectedBytes += 128;
} else if (token.name === "keyValue" || token.name === "stringValue") {
if (!("value" in token) || typeof token.value !== "string")
throw new Error("Unexpected packed JSON string");
projectedBytes += 64 + token.value.length * 2;
} else if (
token.name === "numberValue" ||
token.name === "nullValue" ||
token.name === "trueValue" ||
token.name === "falseValue"
) {
projectedBytes += 64;
}
if (projectedBytes >= nextCheck) {
const heap = getHeapStatistics();
// Conservative capacity for containers, property/array slots and UTF-16
// values. Keep another whole projection available for container growth,
// plus the next chunk of estimates between samples. This is admission
// against live runtime headroom, not a fixed document-size policy.
const requiredBytes = projectedBytes + READ_CHUNK_BYTES;
if (requiredBytes > heap.total_available_size)
throw new AnalysisResourceConstraintError(
operation,
"memory",
"Insufficient heap headroom to materialize the complete JSON value",
{
input_path: path,
input_file_bytes: inputFileBytes,
projected_value_bytes: projectedBytes,
assembly_headroom_bytes: requiredBytes,
available_heap_bytes: heap.total_available_size,
heap_size_limit_bytes: heap.heap_size_limit,
},
{ remediationAction: JSON_INPUT_RESOURCE_REMEDIATION },
);
nextCheck = projectedBytes + READ_CHUNK_BYTES;
}
yield token;
}
};
const requireStringAssemblyHeadroom = (
operation: string,
inputFileBytes: number,
path: string,
stringCodeUnits: number,
): void => {
if (stringCodeUnits < PACKED_STRING_PART_CODE_UNITS) return;
if (stringCodeUnits < PACKED_TOKEN_PART_CODE_UNITS) return;
const heap = getHeapStatistics();
// Concatenation or key interning may flatten the retained rope. Reserve its
// UTF-16 representation and the next coalesced part before that allocation.
const requiredBytes = 2 * (stringCodeUnits + PACKED_STRING_PART_CODE_UNITS);
const requiredBytes = 2 * (stringCodeUnits + PACKED_TOKEN_PART_CODE_UNITS);
if (requiredBytes > heap.total_available_size)
throw new AnalysisResourceConstraintError(
operation,
"memory",
"Insufficient heap headroom to assemble a complete JSON string",
{
input_path: path,
input_file_bytes: inputFileBytes,
string_code_units: stringCodeUnits,
string_assembly_headroom_bytes: requiredBytes,
@@ -208,8 +299,9 @@ const requireStringAssemblyHeadroom = (
async function* decodedChunks(
handle: FileHandle,
admittedBytes: number,
path: string,
signal: AbortSignal | undefined,
prefix?: Uint8Array,
): AsyncGenerator<string> {
const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
let leading = true;
@@ -220,25 +312,25 @@ async function* decodedChunks(
}
return text;
};
if (prefix !== undefined)
for (let offset = 0; offset < prefix.length; offset += READ_CHUNK_BYTES) {
signal?.throwIfAborted();
yield checked(
decoder.decode(prefix.subarray(offset, offset + READ_CHUNK_BYTES), {
stream: true,
}),
);
}
const bytes = Buffer.allocUnsafe(READ_CHUNK_BYTES);
let observedBytes = 0;
while (true) {
signal?.throwIfAborted();
const read = await handle.read(bytes, 0, bytes.length, null);
const read = await handle.read(
bytes,
0,
Math.min(bytes.length, admittedBytes - observedBytes + 1),
null,
);
signal?.throwIfAborted();
if (read.bytesRead === 0) break;
observedBytes += read.bytesRead;
if (observedBytes > admittedBytes) throw new RegularFileChangedError(path);
yield checked(
decoder.decode(bytes.subarray(0, read.bytesRead), { stream: true }),
);
}
if (observedBytes !== admittedBytes) throw new RegularFileChangedError(path);
yield checked(decoder.decode());
}
@@ -264,8 +356,7 @@ const readPrefix = async (
return { bytes: bytes.subarray(0, offset), complete: true };
offset += read.bytesRead;
}
// The file grew after admission. Continue from this same handle, preserving
// the prefix already read rather than reopening or dropping its bytes.
// One byte beyond the admitted extent proves the selected file changed.
return { bytes, complete: false };
};
@@ -1,3 +1,11 @@
import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import {
createJavaScriptSemanticGraph,
createJavaScriptSemanticGraphNode,
JavaScriptSemanticEvidenceContextRegistry,
} from "../domain/javascript/javascriptSemanticGraph.js";
import { proofEvidence } from "./ReconstructionObligationLedger.fixture.js";
import { describe, expect, it } from "vitest";
import { createEvidence } from "../domain/evidence.js";
@@ -171,3 +179,115 @@ describe("reconstruction obligation candidates", () => {
);
});
});
it("keeps each semantic candidate bound to its own file and authority context", () => {
const contexts = new JavaScriptSemanticEvidenceContextRegistry();
const base = javascriptApplicationAnalysisResultSchema.parse(
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
);
const sourceRecords = [
proofEvidence("first-file"),
proofEvidence("second-file"),
];
const module = createJavaScriptSemanticGraphNode(
{
kind: "module",
identity: {
artifact_sha256: base.root_artifact_sha256,
module_path: "app.js",
source_range: null,
role_key: "module",
},
function_node_id: null,
application_node_ids: [],
label: "app.js",
properties: {},
evidence: artifactEvidence(base.root_artifact_sha256, "app.js"),
},
contexts,
);
const requests = sourceRecords.flatMap((source, fileIndex) =>
[1, 2].map((line) => {
const sha256 = String(fileIndex + 4).repeat(64);
const path = `file-${String(fileIndex)}.js`;
return createJavaScriptSemanticGraphNode(
{
kind: "request",
identity: {
artifact_sha256: sha256,
module_path: path,
source_range: {
start: { line, column: 0 },
end: { line, column: 8 },
},
role_key: `request-${String(line)}`,
},
function_node_id: null,
application_node_ids: [],
label: `${path}:${String(line)}`,
properties: {},
evidence: {
...artifactEvidence(sha256, path, "ast-static-analysis"),
evidence_ids: [source.evidence_id],
},
},
contexts,
);
}),
);
const { graph_id: _graphId, ...semantic } = base.semantic_graph;
const semanticGraph = createJavaScriptSemanticGraph({
...semantic,
root_node_ids: [module.node_id],
nodes: [module, ...requests],
evidence_contexts: contexts.contexts,
});
const example = JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE;
const application = createEvidence(
{
path: base.input_path,
sha256: base.root_artifact_sha256,
format: base.format,
},
example.provider,
{
predicateType: example.predicate_type,
operation: example.operation,
parameters: example.parameters,
result: jsonValueSchema.parse({ ...base, semantic_graph: semanticGraph }),
confidence: "derived",
authority: "shipped-artifact",
evidenceLinks: sourceRecords.map(({ evidence_id }) => evidence_id),
},
);
const generated = deriveReconstructionObligationCandidates(
createEvidenceBundle([...sourceRecords, application]),
[],
);
expect(generated.limitations).toEqual([]);
expect(generated.candidates).toHaveLength(requests.length);
for (const [index, node] of requests.entries()) {
const source = sourceRecords[Math.floor(index / 2)];
expect(
generated.candidates.find(
({ target }) => target.semantic_node_id === node.node_id,
),
).toMatchObject({
title: node.label,
family: "request",
source_state: "candidate",
target: {
artifact_sha256: node.identity.artifact_sha256,
semantic_node_id: node.node_id,
},
authority_references: [
{
evidence_id: source?.evidence_id,
authority: "controlled-replay",
state: "candidate",
location: `${semanticGraph.graph_id}/node/${node.node_id}`,
},
],
});
}
});
@@ -12,7 +12,6 @@ import type {
ReconstructionObligation,
ReviewedReconstructionObligation,
} from "../domain/reconstructionObligationLedgerSchemas.js";
import { resolveJavaScriptSemanticEvidence } from "../domain/javascript/javascriptSemanticGraph.js";
import { parseApplicationGraphEvidence } from "./javascript/JavaScriptApplicationEvidenceGraph.js";
import {
applicationObligationPolicy,
@@ -115,14 +114,21 @@ const deriveApplicationCandidates = (
}),
);
}
if (source.semanticGraph !== null)
if (source.semanticGraph !== null) {
const evidenceContexts = new Map(
source.semanticGraph.evidence_contexts.map((context) => [
context.context_id,
context,
]),
);
for (const node of source.semanticGraph.nodes) {
const candidatePolicy = semanticObligationPolicy(node.kind);
if (candidatePolicy === undefined) continue;
const evidenceContext = resolveJavaScriptSemanticEvidence(
source.semanticGraph,
node.evidence,
);
const evidenceContext = evidenceContexts.get(node.evidence.context_id);
if (evidenceContext === undefined)
throw new TypeError(
"Semantic evidence reference names an absent context",
);
addCandidate(
candidates,
generatedCandidate({
@@ -144,6 +150,7 @@ const deriveApplicationCandidates = (
}),
);
}
}
if (source.graph.coverage.status !== "complete")
limitations.add(
`Application graph ${source.graph.graph_id} coverage is ${source.graph.coverage.status}; omitted candidates remain unknown.`,
+36 -9
View File
@@ -1,7 +1,8 @@
import { readFile, realpath, stat } from "node:fs/promises";
import { lstat, realpath, stat } from "node:fs/promises";
import { join, resolve } from "node:path";
import ignore from "ignore";
import { readRegularFileText } from "./RegularFileRead.js";
import { err, ok, type Result } from "../domain/result.js";
import {
@@ -47,14 +48,27 @@ const resolveRoot = async (
const buildIgnored = async (
root: string,
excludePaths: readonly string[],
signal?: AbortSignal,
): Promise<ReturnType<typeof ignore>> => {
const ignored = ignore();
const policyPath = join(root, ".gitignore");
let present = true;
try {
ignored.add(await readFile(join(root, ".gitignore"), "utf8"));
await lstat(policyPath);
} catch (cause: unknown) {
// Missing or unreadable ignore file does not authorize broader access.
void cause;
if (
!(
typeof cause === "object" &&
cause !== null &&
"code" in cause &&
cause.code === "ENOENT"
)
)
throw cause;
present = false;
}
signal?.throwIfAborted();
if (present) ignored.add(await readRegularFileText(policyPath, { signal }));
ignored.add([...DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS]);
for (const path of excludePaths) {
ignored.add(path);
@@ -71,9 +85,22 @@ export const prepareReferenceSourceImport = async (
> => {
const root = await resolveRoot(options.root);
if (!root.ok) return root;
return ok({
root: root.value,
ignored: await buildIgnored(root.value, options.excludePaths ?? []),
secrets: ignore().add([...options.policy.secretPatterns]),
});
try {
return ok({
root: root.value,
ignored: await buildIgnored(
root.value,
options.excludePaths ?? [],
options.signal,
),
secrets: ignore().add([...options.policy.secretPatterns]),
});
} catch (cause: unknown) {
return err(
failure(
options.signal?.aborted === true ? "cancelled" : "io",
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${cause instanceof Error ? cause.message : String(cause)}`,
),
);
}
};
+62 -17
View File
@@ -1,6 +1,7 @@
import fs from "node:fs";
import { lstat, readFile } from "node:fs/promises";
import { lstat } from "node:fs/promises";
import { join, resolve } from "node:path";
import { readRegularFile, readRegularFileText } from "./RegularFileRead.js";
export type ReferenceSourceVcsInfo =
| {
@@ -29,7 +30,7 @@ export const readReferenceSourceVcs = async (
: { kind: "unknown", head: null, dirty: null };
}
try {
const head = await resolveSourceHead(root);
const head = await resolveSourceHead(root, signal);
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
return { kind: "git", head, dirty: null };
} catch (cause: unknown) {
@@ -39,13 +40,59 @@ export const readReferenceSourceVcs = async (
}
};
const resolveSourceHead = async (root: string): Promise<string> => {
const resolveSourceHead = async (
root: string,
signal?: AbortSignal,
): Promise<string> => {
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
const { resolveRef } = await import("isomorphic-git");
let failedRead: { readonly cause: unknown } | undefined;
const referenceFs = {
...fs,
promises: {
...fs.promises,
// isomorphic-git treats every read failure as absence. Preserve real
// failures so a readable packed ref cannot hide an unreadable loose ref.
readFile: async (
path: string | undefined,
options:
| BufferEncoding
| { readonly encoding?: BufferEncoding | null }
| null = null,
) => {
// The library probes readFile() without a path to detect promise APIs.
// Reject that invocation without attributing it to a source read.
if (path === undefined)
throw new TypeError("Git filesystem read requires a path");
try {
const bytes = await readRegularFile(path, { signal });
const encoding =
typeof options === "string" ? options : options?.encoding;
return encoding === undefined || encoding === null
? bytes
: bytes.toString(encoding);
} catch (cause: unknown) {
if (errorCode(cause) !== "ENOENT") failedRead ??= { cause };
throw cause;
}
},
},
};
const readRef = async (options: {
readonly gitdir?: string;
readonly ref: string;
}) => {
const head = await resolveRef({ fs: referenceFs, dir: root, ...options });
signal?.throwIfAborted();
if (failedRead !== undefined) throw failedRead.cause;
return head;
};
const dotgit = join(root, ".git");
if (!(await lstat(dotgit)).isFile())
return resolveRef({ fs, dir: root, ref: "HEAD" });
const pointer = (await readFile(dotgit, "utf8")).replace(/\r?\n$/u, "");
if (!(await lstat(dotgit)).isFile()) return readRef({ ref: "HEAD" });
const pointer = (await readRegularFileText(dotgit, { signal })).replace(
/\r?\n$/u,
"",
);
if (!pointer.startsWith("gitdir: ") || pointer.length === 8)
throw new Error("Invalid Git directory pointer");
const gitdir = resolve(root, pointer.slice(8));
@@ -53,25 +100,23 @@ const resolveSourceHead = async (root: string): Promise<string> => {
try {
commonDirectory = resolve(
gitdir,
(await readFile(join(gitdir, "commondir"), "utf8")).replace(
/\r?\n$/u,
"",
),
(
await readRegularFileText(join(gitdir, "commondir"), { signal })
).replace(/\r?\n$/u, ""),
);
} catch (cause: unknown) {
if (errorCode(cause) !== "ENOENT") throw cause;
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
return readRef({ gitdir, ref: "HEAD" });
}
const head = (await readFile(join(gitdir, "HEAD"), "utf8")).trim();
if (!head.startsWith("ref: "))
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
const head = (
await readRegularFileText(join(gitdir, "HEAD"), { signal })
).trim();
if (!head.startsWith("ref: ")) return readRef({ gitdir, ref: "HEAD" });
const ref = head.slice(5);
const privateRef = ["refs/bisect/", "refs/rewritten/", "refs/worktree/"].some(
(prefix) => ref.startsWith(prefix),
);
return resolveRef({
fs,
dir: root,
return readRef({
gitdir: privateRef ? gitdir : commonDirectory,
ref,
});
+49 -6
View File
@@ -1,20 +1,44 @@
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import { lstat, stat, type FileHandle } from "node:fs/promises";
import { openRegularFile } from "../filesystem/RegularFile.js";
import {
openRegularFile,
sameRegularFileState,
RegularFileChangedError,
} from "../filesystem/RegularFile.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
/** Filesystem admission policy for one selected regular input. */
export interface RegularFileReadOptions {
readonly signal?: AbortSignal | undefined;
readonly symlinks?: "follow" | "reject";
}
/** Admit one regular-file handle, retain it through the consumer, and always close it. */
export const withRegularFile = async <Value>(
path: string,
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
signal?: AbortSignal,
options: RegularFileReadOptions = {},
): Promise<Value> => {
const handle = await openRegularFile(path, { symlinks: "follow", signal });
const { signal } = options;
const handle = await openRegularFile(path, {
symlinks: options.symlinks ?? "follow",
signal,
});
try {
const stats = await handle.stat();
signal?.throwIfAborted();
const value = await read(handle, stats);
signal?.throwIfAborted();
const [opened, currentPath] = await Promise.all([
handle.stat(),
options.symlinks === "reject" ? lstat(path) : stat(path),
]);
if (
!sameRegularFileState(stats, opened) ||
!sameRegularFileState(stats, currentPath)
)
throw new RegularFileChangedError(path);
return value;
} finally {
await handle.close();
@@ -24,6 +48,25 @@ export const withRegularFile = async <Value>(
/** Admit a regular file without waiting for a pipe, then read its verified handle. */
export const readRegularFile = (
path: string,
signal?: AbortSignal,
options: RegularFileReadOptions = {},
): Promise<Buffer> =>
withRegularFile(path, (handle) => handle.readFile({ signal }), signal);
withRegularFile(
path,
async (handle, stats) => {
const bytes = await readBoundedFileBytes(
handle,
stats.size,
options.signal,
);
if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path);
return bytes;
},
options,
);
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
export const readRegularFileText = async (
path: string,
options: RegularFileReadOptions = {},
): Promise<string> => (await readRegularFile(path, options)).toString("utf8");
+6 -5
View File
@@ -14,7 +14,7 @@ import {
type ClientRegistrationDialect,
} from "./ClientConfigurationDocument.js";
import { constants as fsConstants } from "node:fs";
import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic";
@@ -22,6 +22,7 @@ import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { readRegularFileText } from "./RegularFileRead.js";
import type {
ClientConfigurationInspection,
ClientConfigurationResult,
@@ -64,7 +65,7 @@ const configureClientDocument = async (
return { status: "failed", reason: "path" };
let original: string | undefined;
try {
original = await readFile(transactionPath, "utf8");
original = await readRegularFileText(transactionPath);
} catch (cause: unknown) {
if (!isMissing(cause)) return { status: "failed", reason: "readback" };
}
@@ -149,7 +150,7 @@ const configureClientDocument = async (
}
try {
const readback = parseClientConfiguration(
await readFile(transactionPath, "utf8"),
await readRegularFileText(transactionPath),
format,
);
if (!registrationCurrent(readback, desired)) {
@@ -178,7 +179,7 @@ export const clientConfigurationAligned = async (
try {
const files = await readClientConfigurationFiles(client);
if (!files.ok) return false;
const original = await readFile(client.configPath, "utf8");
const original = await readRegularFileText(client.configPath);
const parsed = parseClientConfiguration(original, client.format);
const policy = await readClientPolicyBlock(
client,
@@ -246,7 +247,7 @@ export const inspectClientConfiguration = async (
};
let original: string;
try {
original = await readFile(transactionPath, "utf8");
original = await readRegularFileText(transactionPath);
} catch (cause: unknown) {
if (isMissing(cause)) return { status: "create" };
return {
+6 -14
View File
@@ -4,10 +4,8 @@ import { dirname, join } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
NonRegularFileReadError,
openRegularFile,
} from "../filesystem/RegularFile.js";
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
import { readRegularFileText } from "./RegularFileRead.js";
import { clientSkillDirectories } from "./SupportedClients.js";
export const MANAGED_SKILL_FILES = [
@@ -23,14 +21,8 @@ export const isManagedSkillManifest = (content: string): boolean =>
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
export const readSkillFile = async (path: string): Promise<string> => {
const handle = await openRegularFile(path, { symlinks: "reject" });
try {
return await handle.readFile("utf8");
} finally {
await handle.close();
}
};
export const readSkillFile = (path: string): Promise<string> =>
readRegularFileText(path, { symlinks: "reject" });
interface CanonicalSkillFile {
readonly content: string;
@@ -199,7 +191,7 @@ export const installCanonicalSkill = async (
for (const { destination, content } of changed)
await writeText(destination, content);
for (const { destination, content } of changed)
if ((await readFile(destination, "utf8")) !== content)
if ((await readSkillFile(destination)) !== content)
throw new Error(`skill readback mismatch: ${destination}`);
return "installed";
} catch (cause: unknown) {
@@ -231,7 +223,7 @@ export const readInstalledSkillIdentity = async (
let content: string | undefined;
for (const destination of destinations) {
try {
content = await readFile(join(destination.path, "SKILL.md"), "utf8");
content = await readSkillFile(join(destination.path, "SKILL.md"));
break;
} catch {
// Another selected copy may provide metadata; canonical validation below
+3 -2
View File
@@ -6,7 +6,7 @@ import {
withClientServers,
type ClientConfigurationDocument,
} from "./ClientConfigurationDocument.js";
import { copyFile, lstat, readFile, realpath, rm } from "node:fs/promises";
import { copyFile, lstat, realpath, rm } from "node:fs/promises";
import { constants as fsConstants } from "node:fs";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { join } from "node:path";
@@ -23,6 +23,7 @@ import {
} from "./SetupSkill.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { readRegularFileText } from "./RegularFileRead.js";
import {
manualRegistrationRemediation,
supportedClients,
@@ -49,7 +50,7 @@ export interface UninstallFileSystem {
}
const systemFileSystem: UninstallFileSystem = {
readText: (path) => readFile(path, "utf8"),
readText: (path) => readRegularFileText(path),
readRegularText: readSkillFile,
copy: (source, destination) =>
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
+5 -3
View File
@@ -1,4 +1,3 @@
import { readFile } from "node:fs/promises";
import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
@@ -14,6 +13,7 @@ import { supportedClients } from "./SupportedClients.js";
import { existingSkillDestinations } from "./SetupSkill.js";
import type { SetupAction } from "./SetupTypes.js";
import type { Result } from "../domain/result.js";
import { readRegularFileText } from "./RegularFileRead.js";
/** Existing REA integrations selected for maintenance, without discovering new targets. */
export interface MaintenanceScope {
@@ -73,7 +73,7 @@ export const existingMaintenanceScope = async (
const client = supported.find(({ name }) => name === registration.client);
if (client === undefined) continue;
const parsed = parseClientConfiguration(
await readFile(client.configPath, "utf8"),
await readRegularFileText(client.configPath),
client.format,
);
const enabled = z
@@ -130,7 +130,9 @@ export const planIntegrationMaintenance = async (
};
if (plan.data.plannedActions.length === 0)
return { status: "current", plannedActions: [] };
const allowedIds = new Set(scope.clients.map((client) => `configure_client:${client}`));
const allowedIds = new Set(
scope.clients.map((client) => `configure_client:${client}`),
);
const skillTargets = new Set(skillDestinations.map(({ path }) => path));
if (
plan.data.plannedActions.some((action) =>
+6 -2
View File
@@ -27,7 +27,10 @@ import { WebScriptExportError } from "../domain/webScriptExportError.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
import { WEB_SCRIPT_EXPORT_PROVIDER as PROVIDER } from "./InvestigationProviders.js";
import { readRegularFile } from "./RegularFileRead.js";
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
import {
NonRegularFileReadError,
RegularFileChangedError,
} from "../filesystem/RegularFile.js";
const OPERATION = "export_web_scripts";
@@ -153,7 +156,7 @@ const readCapture = async (
signal: AbortSignal | undefined,
): Promise<Result<Buffer, AnalysisError>> => {
try {
return ok(await readRegularFile(path, signal));
return ok(await readRegularFile(path, { signal }));
} catch (cause: unknown) {
const code =
cause instanceof Error && "code" in cause ? String(cause.code) : "";
@@ -163,6 +166,7 @@ const readCapture = async (
);
if (
cause instanceof NonRegularFileReadError ||
cause instanceof RegularFileChangedError ||
code === "ENOENT" ||
code === "ENOTDIR" ||
code === "EISDIR" ||
+1 -1
View File
@@ -14,7 +14,7 @@ import type { Result } from "../../domain/result.js";
export interface AdbDeviceAnalysisPort {
/** Probe the selected adb binary without contacting devices. */
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
/** The provider holds no long-lived resources; safe to call always. */
/** Terminally stop admission, drain work, and retry retained package rollback. */
close(): Promise<void>;
execute(
request: AdbRequest,
@@ -2,9 +2,10 @@ import type { ApplicationNode } from "../../domain/javascript/javascriptApplicat
import { sha256Text } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
import type {
JavaScriptArtifactGraphContext,
JavaScriptArtifactGraphCoverage,
import {
sourceNodeFor,
type JavaScriptArtifactGraphContext,
type JavaScriptArtifactGraphCoverage,
} from "./JavaScriptArtifactGraphContext.js";
import {
astObservationEvidence,
@@ -38,10 +39,11 @@ export const electronFindingSourceNode = (
context: JavaScriptArtifactGraphContext,
file: JavaScriptArtifactFile,
moduleKey: string | null,
runtime?: string,
): ApplicationNode | undefined =>
moduleKey === null
? (context.assetNodes.get(file.path) ?? context.fileNodes.get(file.path))
: context.moduleNodes.get(`${file.path}\0${moduleKey}`);
: sourceNodeFor(context, file.path, moduleKey, runtime);
/** Observation-scoped identity shared safely across different artifact files. */
export const electronObservationIdentity = (
@@ -73,6 +73,7 @@ const addIpcRecord = (
context,
record.file,
record.finding.module_key,
record.finding.module_runtime,
);
if (source === undefined) return;
state.sources.set(record.key, source);
@@ -255,6 +256,7 @@ const addValidationObservations = (state: IpcGraphState): void => {
state.context,
file,
validation.module_key,
validation.module_runtime,
)
: state.handlers.get(handlerRecord.key);
if (target !== undefined)
@@ -38,7 +38,12 @@ export const addElectronNativeBoundaries = (
const addNativeBinding = (input: NativeBindingInput): void => {
const { context, file, value, coverage } = input;
const source = electronFindingSourceNode(context, file, value.module_key);
const source = electronFindingSourceNode(
context,
file,
value.module_key,
value.module_runtime,
);
if (source === undefined) return;
const resolved = resolveArtifactPath(
value.specifier,
@@ -39,17 +39,32 @@ export const addElectronWindowBoundaries = (
if (javascript === null) continue;
const coverage = javascriptAnalysisCoverage(javascript);
for (const value of javascript.electron.browser_windows) {
const source = electronFindingSourceNode(context, file, value.module_key);
const source = electronFindingSourceNode(
context,
file,
value.module_key,
value.module_runtime,
);
if (source !== undefined)
addBrowserWindow({ context, file, source, value, coverage });
}
for (const value of javascript.electron.context_bridge_apis) {
const source = electronFindingSourceNode(context, file, value.module_key);
const source = electronFindingSourceNode(
context,
file,
value.module_key,
value.module_runtime,
);
if (source !== undefined)
addContextBridge({ context, file, source, value, coverage });
}
for (const value of javascript.electron.utility_processes) {
const source = electronFindingSourceNode(context, file, value.module_key);
const source = electronFindingSourceNode(
context,
file,
value.module_key,
value.module_runtime,
);
if (source !== undefined)
addUtilityProcess({ context, file, source, value, coverage });
}
@@ -31,7 +31,16 @@ const graphIdentities = async (source: string) => {
...(identity.strategy === "artifact-local-key"
? {
namespace: identity.namespace,
key: identity.key.replace(/^[0-9a-f]{64}:/u, ""),
// Module-export identity is a JSON tuple whose first entry
// binds the whole container; only that digest changes here.
key:
identity.namespace === "module-export"
? JSON.stringify(
(JSON.parse(identity.key) as unknown[]).slice(1),
)
: identity.namespace === "source-module"
? identity.key.replace(/^[0-9a-f]{64}:/u, "")
: identity.key,
}
: {}),
}))
@@ -161,7 +161,7 @@ const addBundlerModuleNodes = (
],
});
context.moduleNodes.set(
moduleLookupKey(file.path, moduleValue.module_key),
moduleLookupKey(file.path, registration.runtime, moduleValue.module_key),
module,
);
addAstContainsEdge(context, {
@@ -195,7 +195,7 @@ const addBundlerRuntimeEdges = (
const { file, registration, chunk, coverage } = record;
for (const moduleKey of registration.entry_module_keys) {
const resolved = context.moduleNodes.get(
moduleLookupKey(file.path, moduleKey),
moduleLookupKey(file.path, registration.runtime, moduleKey),
);
const target =
resolved ?? unresolvedBundlerModuleNode(context, record, moduleKey);
@@ -323,8 +323,11 @@ export const selfReferenceOmissions = (
];
/** Deterministic key for one recovered module inside a bundle asset. */
export const moduleLookupKey = (path: string, moduleKey: string): string =>
`${path}\0${moduleKey}`;
export const moduleLookupKey = (
path: string,
runtime: string,
moduleKey: string,
): string => compositeKey([path, runtime, moduleKey]);
/** Deterministic key for one recovered bundler chunk inside a bundle asset. */
export const chunkLookupKey = (
@@ -353,7 +356,10 @@ export const sourceNodeFor = (
context: JavaScriptArtifactGraphContext,
path: string,
moduleKey: string | null,
runtime?: string,
): ApplicationNode | undefined =>
moduleKey === null
? context.sourceModuleNodes.get(path)
: context.moduleNodes.get(moduleLookupKey(path, moduleKey));
: runtime === undefined
? undefined
: context.moduleNodes.get(moduleLookupKey(path, runtime, moduleKey));
@@ -78,8 +78,12 @@ const addReference = (
omissions: { selfReferences: number },
): void => {
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
sourceNodeFor(
context,
input.file.path,
input.value.module_key,
input.value.module_runtime,
) ?? input.asset;
const resolved =
input.value.specifier === null
? null
@@ -162,8 +166,12 @@ const addEndpoint = (
input: FindingInput<StaticEndpoint>,
): void => {
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
sourceNodeFor(
context,
input.file.path,
input.value.module_key,
input.value.module_runtime,
) ?? input.asset;
const key = compactStringIdentityKey(input.value.value);
const label = `${input.value.kind} endpoint`;
const node = context.accumulator.addNode({
@@ -204,8 +212,12 @@ const addStorage = (
input: FindingInput<StaticStorage>,
): void => {
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
sourceNodeFor(
context,
input.file.path,
input.value.module_key,
input.value.module_runtime,
) ?? input.asset;
const node = context.accumulator.addNode({
kind: "storage",
identity: artifactLocalIdentity(
@@ -262,8 +274,12 @@ const addDiscoveredRole = (
coverage: input.coverage,
});
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
sourceNodeFor(
context,
input.file.path,
input.value.module_key,
input.value.module_runtime,
) ?? input.asset;
addStaticInferenceEdge(context, {
source,
target: role,
@@ -359,9 +375,12 @@ const resolveReference = (
if (specifier === null) return null;
const chunk = resolveBundlerChunkReference(context, sourcePath, reference);
if (chunk !== null) return chunk;
const module = context.moduleNodes.get(
moduleLookupKey(sourcePath, specifier),
);
const module =
reference.module_runtime === undefined
? undefined
: context.moduleNodes.get(
moduleLookupKey(sourcePath, reference.module_runtime, specifier),
);
if (module !== undefined) {
const file = context.filesByPath.get(sourcePath);
return file === undefined
@@ -407,10 +426,11 @@ const resolveBundlerChunkReference = (
if (file === undefined) return null;
const registrations = context.analysis.files
.find(({ file: analyzedFile }) => analyzedFile.path === sourcePath)
?.javascript?.bundler_registrations.filter(({ modules }) =>
reference.module_key === null
? true
: modules.some(
?.javascript?.bundler_registrations.filter(({ runtime, modules }) =>
reference.module_runtime === undefined
? reference.module_key === null
: runtime === reference.module_runtime &&
modules.some(
({ module_key: moduleKey }) => moduleKey === reference.module_key,
),
);
@@ -325,8 +325,7 @@ const resolveUiAction = (
(node) =>
["control", "view", "view_controller", "scene"].includes(node.kind) &&
(node.id === query ||
node.attributes.interface_builder_object_id === query ||
node.id.endsWith(`:object:${query}`)),
node.attributes.interface_builder_object_id === query),
);
const actionIds = new Set(
graph.edges.flatMap((edge) =>
+33
View File
@@ -60,6 +60,30 @@ describe("English Unicode case collisions", () => {
]);
});
it("retains every spelling in a high-cardinality folded sibling group", () => {
const count = 8_192;
const variants = caseVariants("abcdefghijklm", count);
const { limitation, byPath } = annotate(variants);
const ordered = [...variants].sort();
const first = ordered[0];
const second = ordered[1];
const original = variants[0];
if (first === undefined || second === undefined || original === undefined) {
throw new Error("Expected at least two case variants");
}
expect(limitation).toBe(ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION);
expect(byPath.size).toBe(count);
expect(byPath.get(first)).toEqual([
`Logical path ${first} collides under English (en-US) Unicode case folding with ${second} and ${String(count - 2)} other spellings; ${NOTE}`,
]);
expect(byPath.get(original)?.[0]).toContain(
`with ${first} and ${String(count - 2)} other spellings;`,
);
expect(
[...byPath.values()].every((limitations) => limitations.length === 1),
).toBe(true);
});
it("stays linear for large case-variant subtrees", () => {
const count = 5_000;
const paths = Array.from({ length: count }, (_, index) => [
@@ -81,3 +105,12 @@ describe("English Unicode case collisions", () => {
expect(byPath.get(deep)).toEqual([]);
});
});
const caseVariants = (lowercase: string, count: number): string[] =>
Array.from({ length: count }, (_, mask) =>
[...lowercase]
.map((character, index) =>
(mask & (1 << index)) === 0 ? character : character.toUpperCase(),
)
.join(""),
);
+4 -4
View File
@@ -177,12 +177,12 @@ const englishUnicodeCaseCollisions = (
if (path.length === 0 || path === ".") continue;
let node = root;
for (const part of path.split("/")) {
const folded = englishCaseFold(part);
const spellings = node.spellingsByFold.get(folded);
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
else if (!spellings.includes(part)) spellings.push(part);
let child = node.bySpelling.get(part);
if (child === undefined) {
const folded = englishCaseFold(part);
const spellings = node.spellingsByFold.get(folded);
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
else spellings.push(part);
child = emptyCaseNode();
node.bySpelling.set(part, child);
}
+101 -24
View File
@@ -1,5 +1,5 @@
import { createReadStream } from "node:fs";
import { open as openFile, stat } from "node:fs/promises";
import { lstat } from "node:fs/promises";
import type { FileHandle } from "node:fs/promises";
import { Readable } from "node:stream";
import type { ArtifactCommand } from "../domain/artifactGraph.js";
@@ -19,11 +19,21 @@ import {
type ArtifactReader,
} from "./ArtifactReader.js";
import { streamChunkToBuffer } from "./StreamBytes.js";
import {
openRegularFile,
sameRegularFileState,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
/** Read-only universal Mach-O slice reader backed by native lipo metadata. */
export class MachOSliceArtifactReader implements ArtifactReader {
readonly format = "file" as const;
#command: ArtifactCommand | undefined;
#source: StableRegularFileDescriptor | undefined;
#pendingSource: Promise<StableRegularFileDescriptor> | undefined;
readonly #ownsSource: boolean;
#closed = false;
#closePromise: Promise<void> | undefined;
constructor(
private readonly path: string,
@@ -31,9 +41,15 @@ export class MachOSliceArtifactReader implements ArtifactReader {
private readonly runner: NativeCommandRunner = new XcrunCommandRunner(
environment,
),
) {}
source?: StableRegularFileDescriptor,
) {
this.#source = source;
this.#ownsSource = source === undefined;
}
async *entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry> {
const source = await this.#ensureSource();
await this.#verifySource(source);
const captured = await this.runner.run(
"lipo",
["-detailed_info", this.path],
@@ -53,9 +69,11 @@ export class MachOSliceArtifactReader implements ArtifactReader {
exit_code: captured.value.exitCode,
effects: ["read"],
};
const fileSize = (await stat(this.path)).size;
await this.#verifySource(source);
const fileSize = source.initial.size;
const architectures = parseLipoArchitectures(captured.value.stdout);
const structural = await readStructuralSlices(this.path, fileSize);
const structural = await readStructuralSlices(source.handle, fileSize);
await this.#verifySource(source);
if (structural.status === "malformed")
throw new ArtifactReaderFailure(
"integrity",
@@ -168,22 +186,28 @@ export class MachOSliceArtifactReader implements ArtifactReader {
"integrity",
"Invalid Mach-O slice byte range",
);
const fileSize = (await stat(this.path)).size;
if (!Number.isSafeInteger(offset + size) || offset + size > fileSize)
const source = await this.#ensureSource();
await this.#verifySource(source);
if (
!Number.isSafeInteger(offset + size) ||
offset + size > source.initial.size
)
throw new ArtifactReaderFailure(
"integrity",
`Mach-O slice range is outside the artifact: ${entry.path}`,
);
const source = createReadStream(this.path, {
const sourceStream = source.handle.createReadStream({
start: offset,
end: offset + size - 1,
autoClose: false,
...(signal === undefined ? {} : { signal }),
});
const verifySource = () => this.#verifySource(source);
return Readable.from(
(async function* () {
let observedBytes = 0;
try {
for await (const raw of source) {
for await (const raw of sourceStream) {
const chunk = streamChunkToBuffer(raw);
observedBytes += chunk.byteLength;
if (observedBytes > size)
@@ -212,6 +236,7 @@ export class MachOSliceArtifactReader implements ArtifactReader {
"integrity",
`Mach-O slice size disagrees with lipo metadata: ${entry.path}`,
);
await verifySource();
})(),
);
}
@@ -220,28 +245,80 @@ export class MachOSliceArtifactReader implements ArtifactReader {
return this.#command === undefined ? [] : [structuredClone(this.#command)];
}
close(): Promise<void> {
return Promise.resolve();
async close(): Promise<void> {
this.#closed = true;
this.#closePromise ??= this.#closeSource().catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
async #ensureSource(): Promise<StableRegularFileDescriptor> {
if (this.#closed)
throw new ArtifactReaderFailure(
"unavailable",
"Universal Mach-O reader is closed",
);
if (this.#source !== undefined) return this.#source;
this.#pendingSource ??= this.#acquireSource();
const source = await this.#pendingSource;
if (this.#closed)
throw new ArtifactReaderFailure(
"unavailable",
"Universal Mach-O reader closed during source acquisition",
);
return source;
}
async #acquireSource(): Promise<StableRegularFileDescriptor> {
const handle = await openRegularFile(this.path, { symlinks: "reject" });
try {
const initial = await handle.stat();
const source = { handle, initial };
this.#source = source;
return source;
} catch (cause: unknown) {
await handle.close().catch(() => undefined);
throw cause;
}
}
async #closeSource(): Promise<void> {
await this.#pendingSource?.catch(() => undefined);
if (!this.#ownsSource || this.#source === undefined) return;
const source = this.#source;
await source.handle.close();
this.#source = undefined;
}
async #verifySource(source: StableRegularFileDescriptor): Promise<void> {
const [opened, currentPath] = await Promise.all([
source.handle.stat(),
lstat(this.path),
]);
if (
!sameRegularFileState(source.initial, opened) ||
!sameRegularFileState(source.initial, currentPath)
)
throw new ArtifactReaderFailure(
"integrity",
`Universal Mach-O source changed during inventory: ${this.path}`,
);
}
}
/** Compare lipo's independent ranges with structurally parsed slice identity. */
const readStructuralSlices = async (
path: string,
file: FileHandle,
size: number,
): Promise<Awaited<ReturnType<typeof readMachoImage>>> => {
const file = await openFile(path, "r");
try {
const readAt: ReadAt = async (offset, length) => {
const bytes = Buffer.alloc(length);
const { bytesRead } = await file.read(bytes, 0, length, offset);
return bytes.subarray(0, bytesRead);
};
const facts = await readMachoImage(readAt, size);
return facts;
} finally {
await file.close();
}
const readAt: ReadAt = async (offset, length) => {
const bytes = Buffer.alloc(length);
const { bytesRead } = await file.read(bytes, 0, length, offset);
return bytes.subarray(0, bytesRead);
};
return readMachoImage(readAt, size);
};
const lipoMatchesSlice = (
@@ -295,23 +295,53 @@ export const createArtifactEdges = (
return edges;
};
export const nearestParent = (
path: string,
occurrences: ReadonlyMap<string, MutableOccurrence>,
interface ArtifactParentIndex {
readonly children: Map<string, ArtifactParentIndex>;
occurrence?: MutableOccurrence;
}
/** Resolve strict observed ancestors without rebuilding every path prefix. */
export const resolveOccurrenceParents = (
occurrences: readonly MutableOccurrence[],
byPath: ReadonlyMap<string, MutableOccurrence>,
expandedContainerIds: ReadonlySet<string>,
): MutableOccurrence | undefined => {
const parts = path.split("/");
while (parts.length > 1) {
parts.pop();
const candidate = occurrences.get(parts.join("/"));
): void => {
const root: ArtifactParentIndex = { children: new Map() };
for (const occurrence of byPath.values()) {
if (
candidate !== undefined &&
(candidate.entry_kind === "directory" ||
expandedContainerIds.has(candidate.occurrence_id))
occurrence.entry_kind !== "directory" &&
!expandedContainerIds.has(occurrence.occurrence_id)
)
return candidate;
continue;
let branch = root;
for (const segment of occurrence.logical_path.split("/")) {
let child = branch.children.get(segment);
if (child === undefined) {
child = { children: new Map() };
branch.children.set(segment, child);
}
branch = child;
}
branch.occurrence = occurrence;
}
for (const occurrence of occurrences) {
const path = occurrence.logical_path;
let branch = root;
let parent: MutableOccurrence | undefined;
// Walk parent segments only. Stop scanning the path once no observed
// directory or expanded container can be an ancestor.
for (let start = 0; ;) {
const slash = path.indexOf("/", start);
if (slash < 0) break;
const segment = path.slice(start, slash);
const child = branch.children.get(segment);
if (child === undefined) break;
branch = child;
parent = branch.occurrence ?? parent;
start = slash + 1;
}
occurrence.parent_occurrence_id = parent?.occurrence_id ?? null;
}
return undefined;
};
/**
+6 -6
View File
@@ -24,7 +24,7 @@ interface RootClassification {
}
type RootInventoryClassification = RootClassification & {
readonly zipSource: StableRegularFileDescriptor | undefined;
readonly rootSource: StableRegularFileDescriptor | undefined;
};
/** Classify and hash one file root through the same stable open descriptor. */
@@ -43,7 +43,7 @@ export const classifyAndHashRoot = async (
try {
return { format: result.format, digest: result.digest };
} finally {
await result.zipSource?.handle.close();
await result.rootSource?.handle.close();
}
};
@@ -55,7 +55,7 @@ export const classifyAndHashRootForInventory = async (
signal?: AbortSignal,
): Promise<RootInventoryClassification> => {
if (directory)
return { format: "directory", digest: null, zipSource: undefined };
return { format: "directory", digest: null, rootSource: undefined };
const handle = await openRootFile(path, signal);
let transferHandle = false;
try {
@@ -72,12 +72,12 @@ export const classifyAndHashRootForInventory = async (
initial,
signal,
);
const zipSource = isZipFormat(format);
if (zipSource) transferHandle = true;
const retainSource = isZipFormat(format) || format === "mach-o-universal";
if (retainSource) transferHandle = true;
return {
format,
digest,
zipSource: zipSource ? { handle, initial } : undefined,
rootSource: retainSource ? { handle, initial } : undefined,
};
} finally {
if (!transferHandle) await handle.close();
+3 -3
View File
@@ -13,7 +13,7 @@ export const createReader = async (
format: ArtifactOccurrence["artifact_format"],
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
zipSource?: StableRegularFileDescriptor,
rootSource?: StableRegularFileDescriptor,
): Promise<ArtifactReader | undefined> => {
switch (format) {
case "directory":
@@ -23,12 +23,12 @@ export const createReader = async (
case "apk":
case "msix":
case "appx":
return new ZipArtifactReader(path, format, undefined, zipSource);
return new ZipArtifactReader(path, format, undefined, rootSource);
case "asar":
return new AsarArtifactReader(path);
case "mach-o-universal":
return process.platform === "darwin"
? new MachOSliceArtifactReader(path, environment)
? new MachOSliceArtifactReader(path, environment, undefined, rootSource)
: undefined;
case "dmg":
if (process.platform !== "darwin") return undefined;
+13 -13
View File
@@ -58,7 +58,7 @@ export const scanCanonicalArtifactInventory = async (
const {
format: rootFormat,
digest: rootDigest,
zipSource,
rootSource,
} = await classifyAndHashRootForInventory(
path,
metadata.isDirectory(),
@@ -73,7 +73,7 @@ export const scanCanonicalArtifactInventory = async (
rootFormat,
options.environment ?? {},
options.signal,
zipSource,
rootSource,
);
} catch (cause: unknown) {
readerCreationFailure = { cause };
@@ -102,7 +102,7 @@ export const scanCanonicalArtifactInventory = async (
metadata,
rootFormat,
rootDigest,
zipSource,
rootSource,
reader,
signal: options.signal,
nodes,
@@ -133,13 +133,13 @@ export const scanCanonicalArtifactInventory = async (
);
}
}
if (zipSource !== undefined) {
if (rootSource !== undefined) {
try {
await zipSource.handle.close();
await rootSource.handle.close();
} catch (cleanupCause: unknown) {
const cleanup = ArtifactReaderFailure.cleanupObservation(
cleanupCause,
`root ZIP descriptor for ${path}`,
`root artifact descriptor for ${path}`,
);
cleanupFailure = ArtifactReaderFailure.withCleanup(
cleanupFailure ??
@@ -161,7 +161,7 @@ interface SnapshotBuildInput {
readonly metadata: Stats;
readonly rootFormat: ArtifactOccurrence["artifact_format"];
readonly rootDigest: HashResult | null;
readonly zipSource: StableRegularFileDescriptor | undefined;
readonly rootSource: StableRegularFileDescriptor | undefined;
readonly reader: ArtifactReader | undefined;
readonly signal: AbortSignal | undefined;
readonly nodes: Map<string, ArtifactNode>;
@@ -173,7 +173,7 @@ interface SnapshotBuildInput {
const buildInventorySnapshot = async (
input: SnapshotBuildInput,
): Promise<ArtifactInventorySnapshot> => {
const { path, metadata, rootFormat, rootDigest, zipSource, reader, signal } =
const { path, metadata, rootFormat, rootDigest, rootSource, reader, signal } =
input;
materializeDirectoryNodes(input.occurrences, input.nodes);
const rootNode = createRootNode({
@@ -218,7 +218,7 @@ const buildInventorySnapshot = async (
ordinal,
}));
await verifyRootDigest(path, rootDigest, zipSource, signal);
await verifyRootDigest(path, rootDigest, rootSource, signal);
const graphSha256 = artifactGraphDigest({
nodes: orderedNodes,
@@ -293,17 +293,17 @@ const buildIntegrityContradictions = (
const verifyRootDigest = async (
path: string,
rootDigest: HashResult | null,
zipSource: StableRegularFileDescriptor | undefined,
rootSource: StableRegularFileDescriptor | undefined,
signal: AbortSignal | undefined,
): Promise<void> => {
if (rootDigest === null) return;
const verified =
zipSource === undefined
rootSource === undefined
? await hashStableRootArtifact(path, signal)
: await hashStableRootArtifactHandle(
path,
zipSource.handle,
zipSource.initial,
rootSource.handle,
rootSource.initial,
signal,
);
if (
+6 -8
View File
@@ -18,7 +18,7 @@ import {
createArtifactNode,
createOccurrence,
noteUnreadContainerSuffix,
nearestParent,
resolveOccurrenceParents,
type MutableOccurrence,
} from "./ArtifactGraphConstruction.js";
import {
@@ -91,13 +91,11 @@ export const scanReader = async (
await visitArtifactEntries(context, reader, "");
// Archive directories may appear after their children. Resolve containment
// against the complete index before directory identities are materialized.
for (const occurrence of occurrences)
occurrence.parent_occurrence_id =
nearestParent(
occurrence.logical_path,
context.occurrenceByPath,
context.expandedContainerIds,
)?.occurrence_id ?? null;
resolveOccurrenceParents(
occurrences,
context.occurrenceByPath,
context.expandedContainerIds,
);
return {
nodes,
occurrences,
+13 -11
View File
@@ -68,18 +68,20 @@ export const closeCdpTargetSession = async (
// Best-effort domain cleanup; transport close is the definitive boundary.
}
}
if (!cleanup.signal.aborted && sessionId !== undefined)
try {
await connection.send(
"Target.detachFromTarget",
{ sessionId },
undefined,
cleanup.signal,
);
} catch {
// Best-effort detach; transport close is the definitive boundary.
}
}
// Cancellation skips optional domain shutdown but still releases the
// attached session using its independent, bounded cleanup signal.
if (!cleanup.signal.aborted && sessionId !== undefined)
try {
await connection.send(
"Target.detachFromTarget",
{ sessionId },
undefined,
cleanup.signal,
);
} catch {
// Best-effort detach; transport close is the definitive boundary.
}
} finally {
clearTimeout(timeout);
cleanup.abort();
+40 -11
View File
@@ -365,20 +365,30 @@ const ingestWebMcpEvent = (options: WebMcpIngestOptions): void => {
transientFrames,
resolvedTransientFrames,
} = options;
if (
event.method !== "WebMCP.toolsAdded" &&
event.method !== "WebMCP.toolsRemoved"
)
return;
const params = recordValue(event.params);
if (params === undefined) return;
if (params === undefined) {
completeness.exclude("webmcp_tools", "invalid_protocol_value");
return;
}
if (event.method === "WebMCP.toolsRemoved") {
for (const removed of recordsValue(params.tools)) {
for (const removed of webMcpEventTools(params.tools, completeness)) {
const name = cdpStringValue(removed.name);
const frameId = cdpStringValue(removed.frameId);
if (name === undefined || frameId === undefined) continue;
if (name === undefined || frameId === undefined) {
completeness.exclude("webmcp_tools", "invalid_protocol_value");
continue;
}
for (const [key, tool] of tools)
if (tool.name === name && tool.frame_id === frameId) tools.delete(key);
}
return;
}
if (event.method !== "WebMCP.toolsAdded") return;
for (const declared of recordsValue(params.tools)) {
for (const declared of webMcpEventTools(params.tools, completeness)) {
const frameId = cdpStringValue(declared.frameId);
if (frameId !== undefined && transientFrames.has(frameId)) {
if (!resolvedTransientFrames.has(frameId)) continue;
@@ -391,6 +401,26 @@ const ingestWebMcpEvent = (options: WebMcpIngestOptions): void => {
}
};
const webMcpEventTools = (
value: unknown,
completeness: CdpCaptureCompleteness,
): UnknownRecord[] => {
if (!Array.isArray(value)) {
completeness.exclude("webmcp_tools", "invalid_protocol_value");
return [];
}
const records: UnknownRecord[] = [];
for (const item of value) {
const record = recordValue(item);
if (record === undefined) {
completeness.exclude("webmcp_tools", "invalid_protocol_value");
continue;
}
records.push(record);
}
return records;
};
const normalizeTool = (
value: UnknownRecord,
input: DiscoverWebMcpToolsInput,
@@ -403,12 +433,11 @@ const normalizeTool = (
const frameId = cdpStringValue(value.frameId);
const name = cdpStringValue(value.name);
const frame = frameId === undefined ? undefined : frames.get(frameId);
if (
frameId === undefined ||
name === undefined ||
frame?.origin === null ||
frame === undefined
) {
if (frameId === undefined || name === undefined) {
completeness.exclude("webmcp_tools", "invalid_protocol_value");
return undefined;
}
if (frame?.origin === null || frame === undefined) {
completeness.exclude("webmcp_tools", "out_of_target_scope");
return undefined;
}
+4 -5
View File
@@ -42,7 +42,6 @@ export class PlaywrightScenarioBrowserCleanupOwner {
#profileRemoved: boolean;
#closePromise: Promise<void> | undefined;
#eventFinalizationPromise: Promise<void> | undefined;
#eventFinalizationFailureReported = false;
/** Keep the narrow provider actions needed to release one opened browser. */
constructor(private readonly resources: PlaywrightScenarioCleanupResources) {
@@ -98,10 +97,10 @@ export class PlaywrightScenarioBrowserCleanupOwner {
try {
await eventFinalization;
} catch (cause: unknown) {
if (!this.#eventFinalizationFailureReported) {
failures.push(cause);
this.#eventFinalizationFailureReported = true;
}
// A failed finalization stays failed: the memoized attempt rejects on
// every retry so cleanup can never report success for unfinalized
// events.
failures.push(cause);
}
if (!this.#browserClosed) {
try {
+2 -2
View File
@@ -1,6 +1,6 @@
import { resolve } from "node:path";
import { readCliJsonFile } from "./cliJsonFile.js";
import { readJsonInputFile } from "./application/JsonInputFile.js";
import { NonRegularFileReadError } from "./filesystem/RegularFile.js";
import {
AnalysisAccessDeniedError,
@@ -21,7 +21,7 @@ export const parseCliJsonInput = async (
const inline = safeParseJson(value);
if (inline.ok) return { ok: true, value: inline.value };
try {
const parsed = await readCliJsonFile(value, operation);
const parsed = await readJsonInputFile(value, operation);
return parsed.ok
? { ok: true, value: parsed.value }
: {
+36
View File
@@ -699,3 +699,39 @@ export const adbResultSchemas = {
package_name: packageNameSchema,
}),
} as const;
/** Acquired package facts retained when cancellation leaves caller-owned output. */
export type AdbPackagePullPartialObservation = {
readonly provider_id: "adb";
readonly operation: "pull_adb_package";
readonly result: z.infer<typeof adbResultSchemas.pull_adb_package>;
};
/** A completed acquisition whose digest read was cancelled before verification. */
export type AdbAcquiredFilePartialObservation =
| {
readonly provider_id: "adb";
readonly operation: "pull_adb_file";
readonly result: {
readonly client: z.infer<
typeof adbResultSchemas.pull_adb_file
>["client"];
readonly serial: string;
readonly local_path: string;
readonly device_path: string;
readonly digest_status: "unknown";
};
}
| {
readonly provider_id: "adb";
readonly operation: "collect_adb_bugreport";
readonly result: {
readonly client: z.infer<
typeof adbResultSchemas.collect_adb_bugreport
>["client"];
readonly serial: string;
readonly local_path: string;
readonly adb_reported_path: string;
readonly digest_status: "unknown";
};
};
+7 -1
View File
@@ -99,6 +99,10 @@ import type { BrowserScenarioPartialObservation } from "./browserScenarioCapture
import type { Evidence } from "./evidence.js";
import type { AndroidPartialObservation } from "./android/androidPartialObservation.js";
import type { FirmwarePartialObservation } from "./firmware/firmwareAnalysis.js";
import type {
AdbAcquiredFilePartialObservation,
AdbPackagePullPartialObservation,
} from "./adb/adbDeviceAnalysis.js";
import type { NativeUiObservationResult } from "./native/nativeUiObservation.js";
/** Provider-neutral evidence collected before a typed analysis failure. */
@@ -113,4 +117,6 @@ export type AnalysisPartialObservation =
| Evidence
| AndroidPartialObservation
| FirmwarePartialObservation
| NativeUiObservationResult;
| NativeUiObservationResult
| AdbAcquiredFilePartialObservation
| AdbPackagePullPartialObservation;
@@ -39,6 +39,8 @@ export type ElectronBrowserWindowFinding = ElectronBrowserWindowPreload & {
readonly web_preferences_status: "object-literal" | "dynamic" | "missing";
readonly web_preferences: readonly ElectronWebPreference[];
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
};
@@ -56,6 +58,8 @@ export type ElectronContextBridgeFinding = ElectronContextBridgeApiKey & {
readonly members: readonly string[];
readonly unknown_members: number;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
};
@@ -109,6 +113,8 @@ type ElectronIpcHandler =
interface ElectronIpcFindingContext {
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -134,12 +140,16 @@ export interface ElectronSenderValidationFinding {
readonly expected: ElectronStaticValue;
readonly enforcement: "unknown";
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
interface ElectronUtilityProcessFindingState {
readonly service_name: string | null;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -169,6 +179,8 @@ export interface ElectronNativeAddonBindingFinding {
readonly namespace_access?: boolean;
readonly dynamic_member_access?: boolean;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -62,9 +62,7 @@ const classifyLayer = (
options.reconciliationComplete &&
layer.graph.coverage.status === "complete" &&
scopedEntities.length > 0 &&
scopedEntities.every(({ capture }) =>
captureScriptsComplete(capture.evidence.evidence_id, runtimeEntities),
);
scopedEntities.every(({ capture }) => capture.scriptsCompleteWithinScope);
const states = nodes.map((node) => {
const runtimeNodeIds = uniqueSorted(direct.get(node.node_id) ?? []);
if (runtimeNodeIds.length > 0)
@@ -134,13 +132,6 @@ const append = (
else current.push(value);
};
const captureScriptsComplete = (
evidenceId: string,
entities: readonly RuntimeReconciliationEntity[],
): boolean =>
entities.find(({ capture }) => capture.evidence.evidence_id === evidenceId)
?.capture.scriptsCompleteWithinScope === true;
const isLoadStateNode = (
node: ApplicationNode,
): node is ApplicationNode & {
@@ -212,8 +212,11 @@ const graphCoverage = (
const mergeNodes = (nodes: readonly ApplicationNode[]): ApplicationNode[] => {
const grouped = new Map<string, ApplicationNode[]>();
for (const node of nodes)
grouped.set(node.node_id, [...(grouped.get(node.node_id) ?? []), node]);
for (const node of nodes) {
const group = grouped.get(node.node_id);
if (group === undefined) grouped.set(node.node_id, [node]);
else group.push(node);
}
const merged = [...grouped.values()].map((group) => {
const first = group[0];
if (first === undefined) throw new TypeError("Empty JAG node group");
@@ -23,7 +23,12 @@ it("preserves a 12,000-member native import through both analyzers", () => {
const ir = analyzeParsedJavaScriptSemantics(parsed);
expect(staticAnalysis.parse_status).toBe("complete");
expect(staticAnalysis.electron.native_addon_bindings).toEqual([
expect.objectContaining({ specifier: "./addon.node", members: ["last"] }),
expect.objectContaining({
specifier: "./addon.node",
members: ["next"],
namespace_access: false,
dynamic_member_access: false,
}),
]);
expect(ir.coverage).toEqual({ status: "complete", omittedCount: 0 });
const path = origin(onlyBinding(ir, "value"));
@@ -19,15 +19,11 @@ import {
JAVASCRIPT_SEMANTIC_RELATION_FAMILIES,
JAVASCRIPT_SEMANTIC_RELATION_FAMILY,
javaScriptSemanticFingerprintInputSchema,
javaScriptSemanticFingerprintSchema,
javaScriptSemanticGraphInputSchema,
javaScriptSemanticGraphRecordSchema,
javaScriptSemanticNodeInputSchema,
javaScriptSemanticNodeSchema,
javaScriptSemanticRelationInputSchema,
javaScriptSemanticRelationSchema,
javaScriptSemanticUnknownInputSchema,
javaScriptSemanticUnknownSchema,
type JavaScriptSemanticGraphInput,
type JavaScriptSemanticGraphNode,
type JavaScriptSemanticGraphRelation,
@@ -132,10 +128,10 @@ export const createJavaScriptSemanticGraphNode = (
application_node_ids: uniqueSorted(parsed.application_node_ids),
evidence: evidenceContexts.intern(parsed.evidence),
};
return javaScriptSemanticNodeSchema.parse({
return {
...semantic,
node_id: javaScriptSemanticNodeId(semantic),
});
};
};
/** Normalize one semantic relationship and derive its exact identifier. */
@@ -148,10 +144,10 @@ export const createJavaScriptSemanticGraphRelation = (
...parsed,
evidence: evidenceContexts.intern(parsed.evidence),
};
return javaScriptSemanticRelationSchema.parse({
return {
...semantic,
relation_id: `jsrg_relation_${digestCanonicalValue(semantic, "JavaScript semantic graph")}`,
});
};
};
/** Normalize one unresolved semantic frontier and derive its identifier. */
@@ -166,10 +162,10 @@ export const createJavaScriptSemanticGraphUnknown = (
candidate_node_ids: uniqueSorted(parsed.candidate_node_ids),
evidence: evidenceContexts.intern(parsed.evidence),
};
return javaScriptSemanticUnknownSchema.parse({
return {
...semantic,
unknown_id: `jsrg_unknown_${digestCanonicalValue(semantic, "JavaScript semantic graph")}`,
});
};
};
/** Normalize one function fingerprint and derive its component commitment. */
@@ -191,7 +187,7 @@ export const createJavaScriptSemanticFingerprint = (
semantic.components,
"JavaScript semantic graph",
);
return javaScriptSemanticFingerprintSchema.parse({
return {
...semantic,
fingerprint_sha256: fingerprintSha256,
fingerprint_id: `jsrg_fingerprint_${digestCanonicalValue(
@@ -202,7 +198,7 @@ export const createJavaScriptSemanticFingerprint = (
},
"JavaScript semantic graph",
)}`,
});
};
};
type GraphRecord = z.infer<typeof javaScriptSemanticGraphRecordSchema>;
@@ -4,6 +4,7 @@ import { z } from "zod";
import { digestCanonicalValue } from "../canonicalDigest.js";
import type { ApplicationGraphEvidence } from "./javascriptApplicationEvidenceSchemas.js";
import {
JAVASCRIPT_SEMANTIC_RELATION_FAMILY,
JAVASCRIPT_SEMANTIC_NODE_KINDS,
JAVASCRIPT_SEMANTIC_RELATION_FAMILIES,
JAVASCRIPT_SEMANTIC_RELATIONS,
@@ -27,6 +28,7 @@ import {
parseJavaScriptSemanticGraph,
serializeJavaScriptSemanticGraph,
} from "./javascriptSemanticGraphSerialization.js";
import { queryJavaScriptSemanticGraph } from "./javascriptSemanticQuery.js";
const SHA = "a".repeat(64);
const JAG_ID = `jag_${"b".repeat(64)}`;
@@ -140,6 +142,44 @@ it("accepts semantic module paths longer than the former schema ceiling", () =>
).toBe(modulePath);
});
it("captures caller-owned node identity, properties, and provenance", () => {
const contexts = new JavaScriptSemanticEvidenceContextRegistry();
const input = {
kind: "literal",
identity: {
artifact_sha256: SHA,
module_path: "bundle.js",
source_range: null,
role_key: "literal",
},
function_node_id: null,
application_node_ids: [],
label: "literal",
properties: { nested: ["original"] },
evidence: evidence(),
};
const captured = createJavaScriptSemanticGraphNode(input, contexts);
const expected = structuredClone(captured);
const expectedEvidence = structuredClone(input.evidence);
input.identity.module_path = "replaced.js";
input.properties.nested[0] = "replaced";
input.evidence.limitations.push("later mutation");
if (input.evidence.location.available)
input.evidence.location.value = {
kind: "artifact-path",
path: "replaced.js",
};
expect(captured).toEqual(expected);
expect(
resolveJavaScriptSemanticEvidence(
{ evidence_contexts: contexts.contexts },
captured.evidence,
),
).toEqual(expectedEvidence);
});
const fixtureInput = (withUnknown = false): JavaScriptSemanticGraphInput => {
const evidenceContexts = new JavaScriptSemanticEvidenceContextRegistry();
const module = node("module", "module", {}, evidenceContexts);
@@ -285,6 +325,104 @@ const registryForGraph = (
return registry;
};
const graphWithCandidateRelations = (): JavaScriptSemanticGraph => {
const base = fixtureGraph();
const evidenceContexts = registryForGraph(base);
const relations = base.relations.map(
({ relation_id: _relationId, ...relation }) =>
createJavaScriptSemanticGraphRelation(
{
...relation,
resolution: "candidate",
evidence: resolveJavaScriptSemanticEvidence(base, relation.evidence),
},
evidenceContexts,
),
);
const binding = base.nodes.find(({ label }) => label === "binding");
const callable = base.nodes.find(({ label }) => label === "function");
if (binding === undefined || callable === undefined)
throw new Error("Expected candidate-cycle endpoints");
for (const [source, target] of [
[binding, callable],
[callable, binding],
] as const)
relations.push(
createJavaScriptSemanticGraphRelation(
{
source_node_id: source.node_id,
target_node_id: target.node_id,
relation: "owns",
resolution: "candidate",
properties: {},
evidence: evidence("inferred"),
},
evidenceContexts,
),
);
const { graph_id: _graphId, ...input } = base;
return createJavaScriptSemanticGraph({
...input,
evidence_contexts: evidenceContexts.contexts,
relations,
coverage: {
...input.coverage,
families: input.coverage.families.map((family) => ({
...family,
retained_relations: relations.filter(
({ relation }) =>
JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation] === family.family,
).length,
})),
},
});
};
it("counts candidate edges by query direction and relation filter", () => {
const graph = graphWithCandidateRelations();
const binding = graph.nodes.find(({ label }) => label === "binding");
if (binding === undefined) throw new Error("Expected binding seed");
const query = (
direction: "forward-influence" | "backward-provenance",
allowedRelations: (typeof JAVASCRIPT_SEMANTIC_RELATIONS)[number][],
) =>
queryJavaScriptSemanticGraph(graph, {
seed: { kind: "semantic-node", node_id: binding.node_id },
direction,
allowed_relations: allowedRelations,
});
expect(query("forward-influence", ["captures"]).status).toBe("ambiguous");
expect(query("forward-influence", ["defines"]).status).toBe("found");
expect(query("backward-provenance", ["defines"]).status).toBe("ambiguous");
expect(query("backward-provenance", ["captures"]).status).toBe("found");
});
it("counts candidate ownership cycles once and traverses them finitely", () => {
const graph = graphWithCandidateRelations();
const binding = graph.nodes.find(({ label }) => label === "binding");
if (binding === undefined) throw new Error("Expected binding seed");
const result = queryJavaScriptSemanticGraph(graph, {
seed: { kind: "semantic-node", node_id: binding.node_id },
direction: "ownership",
allowed_relations: ["owns"],
include_ambiguous_dynamic_edges: true,
});
expect(result.status).toBe("ambiguous");
expect(result.relations).toHaveLength(2);
expect(result.summary.traversed_nodes).toBe(2);
expect(result.summary.traversed_relations).toBe(2);
expect(
queryJavaScriptSemanticGraph(graph, {
seed: { kind: "semantic-node", node_id: binding.node_id },
direction: "ownership",
allowed_relations: ["constructs-request"],
}).status,
).toBe("found");
});
it("defines every required relation family and canonicalizes records", () => {
expect(JAVASCRIPT_SEMANTIC_RELATIONS).toContain("argument-to-parameter");
expect(JAVASCRIPT_SEMANTIC_RELATIONS).toContain("detaches-task");
@@ -316,6 +316,20 @@ it("returns every relevant unknown inline", () => {
expect(result.unknowns).toHaveLength(1);
});
it("does not admit a semantic-node seed absent from the graph", () => {
const result = queryJavaScriptSemanticGraph(fixtureGraph(), {
seed: { kind: "semantic-node", node_id: `jsrg_node_${"f".repeat(64)}` },
direction: "forward-influence",
});
expect(result).toMatchObject({
status: "no-match",
seed_node_ids: [],
nodes: [],
summary: { total_seed_matches: 0, traversed_nodes: 0 },
});
});
it("returns the complete deterministic forward influence result inline", () => {
const graph = fixtureGraph();
const input = {
@@ -30,10 +30,6 @@ interface Traversal {
readonly modules: Set<string>;
}
interface SeedAdmission {
readonly nodeIds: string[];
}
interface QueryResult {
readonly retainedNodes: JavaScriptSemanticGraphNode[];
readonly retainedRelations: JavaScriptSemanticGraphRelation[];
@@ -47,10 +43,10 @@ export const queryJavaScriptSemanticGraph = (
const input = javaScriptSemanticQueryInputSchema.parse(rawInput);
const queryId = queryIdentifier(graph, input);
const seeds = resolveSeeds(graph, input);
const admission = admitSeeds(graph, seeds);
const retainedSeeds = admission.nodeIds;
const nodes = new Map(graph.nodes.map((node) => [node.node_id, node]));
const retainedSeeds = admitSeeds(nodes, seeds);
const adjacency = buildAdjacency(graph.relations, input);
const traversal = traverse(graph, retainedSeeds, adjacency);
const traversal = traverse(nodes, retainedSeeds, adjacency);
const result = createQueryResult(graph, traversal);
const allRelevantUnknowns = relevantUnknownFrontiers(
graph,
@@ -122,17 +118,16 @@ const createQueryResult = (
};
const admitSeeds = (
graph: JavaScriptSemanticGraph,
nodes: ReadonlyMap<string, JavaScriptSemanticGraphNode>,
seeds: readonly string[],
): SeedAdmission => {
const nodes = new Map(graph.nodes.map((node) => [node.node_id, node]));
): string[] => {
const retained: string[] = [];
for (const nodeId of seeds) {
const node = nodes.get(nodeId);
if (node === undefined) continue;
retained.push(nodeId);
}
return { nodeIds: retained };
return retained;
};
const buildAdjacency = (
@@ -222,11 +217,10 @@ const addEntry = (
};
const traverse = (
graph: JavaScriptSemanticGraph,
nodes: ReadonlyMap<string, JavaScriptSemanticGraphNode>,
seeds: readonly string[],
adjacency: ReadonlyMap<string, TraversalEntry[]>,
): Traversal => {
const nodes = new Map(graph.nodes.map((node) => [node.node_id, node]));
const nodeIds = new Set(seeds);
const relationIds = new Set<string>();
const functionIds = new Set<string>();
@@ -289,15 +283,32 @@ const relevantCandidateRelationCount = (
nodeIds: ReadonlySet<string>,
input: JavaScriptSemanticQueryInput,
): number => {
const adjacency = buildAdjacency(
graph.relations.filter(({ resolution }) => resolution === "candidate"),
{ ...input, include_ambiguous_dynamic_edges: true },
);
const relevant = new Set<string>();
for (const nodeId of nodeIds)
for (const { relation } of adjacency.get(nodeId) ?? [])
relevant.add(relation.relation_id);
return relevant.size;
const allowed =
input.allowed_relations === undefined
? null
: new Set(input.allowed_relations);
let relevant = 0;
for (const relation of graph.relations) {
if (
relation.resolution !== "candidate" ||
(allowed !== null && !allowed.has(relation.relation))
)
continue;
const sourceIsRelevant = nodeIds.has(relation.source_node_id);
const targetIsRelevant = nodeIds.has(relation.target_node_id);
if (
(input.direction === "callers" &&
relation.relation === "calls" &&
targetIsRelevant) ||
(input.direction === "ownership" &&
ownershipRelation(relation.relation) &&
(sourceIsRelevant || targetIsRelevant)) ||
(input.direction === "backward-provenance" && targetIsRelevant) ||
(input.direction === "forward-influence" && sourceIsRelevant)
)
relevant += 1;
}
return relevant;
};
const expectedMatchesFor = (
@@ -40,7 +40,7 @@ export const inspectBundlerRegistration = (
const table = entry.elements[1];
const runtimeValue = runtimeMetadata(entry.elements[2]);
if (!t.isArrayExpression(chunkIds) || !t.isObjectExpression(table)) return;
const recovered = recoverBundlerModules(source, table, accumulator);
const recovered = recoverBundlerModules(source, table, runtime, accumulator);
const chunkKeys = staticArrayValues(chunkIds);
accumulator.unknownFindings += chunkKeys.unknown;
accumulator.unknownFindings +=
@@ -92,12 +92,13 @@ export const inspectEsbuildWrapper = (
: null;
const table = call.arguments[0];
if (wrapperKind === null || !t.isObjectExpression(table)) return;
const recovered = recoverBundlerModules(source, table, accumulator);
const runtime = `esbuild-${wrapperKind}`;
const recovered = recoverBundlerModules(source, table, runtime, accumulator);
if (recovered.modules.length === 0) return;
accumulator.unknownFindings += recovered.unknownAsyncChunkKeys;
const registration: JavaScriptBundlerRegistration = {
bundler: "esbuild",
runtime: `esbuild-${wrapperKind}`,
runtime,
chunk_keys: [`${wrapperKind}@${String(call.start ?? 0)}`],
unknown_chunk_keys: 0,
runtime_require_name: null,
@@ -127,13 +128,19 @@ interface RecoveredBundlerModules {
const recoverBundlerModules = (
source: string,
table: t.ObjectExpression,
runtime: string,
accumulator: AnalysisAccumulator,
): RecoveredBundlerModules => {
const modules: JavaScriptBundlerModule[] = [];
const asyncChunkKeys: string[] = [];
let unknownAsyncChunkKeys = 0;
for (const property of table.properties) {
const recovered = recoverBundlerModule(source, property, accumulator);
const recovered = recoverBundlerModule(
source,
property,
runtime,
accumulator,
);
if (recovered === null) continue;
modules.push(recovered.module);
asyncChunkKeys.push(...recovered.asyncChunkKeys);
@@ -149,6 +156,7 @@ const recoverBundlerModules = (
const recoverBundlerModule = (
source: string,
property: t.ObjectMethod | t.ObjectProperty | t.SpreadElement,
runtime: string,
accumulator: AnalysisAccumulator,
): {
readonly module: JavaScriptBundlerModule;
@@ -169,6 +177,7 @@ const recoverBundlerModule = (
start: factory.start,
end: factory.end,
key,
runtime,
requireName,
};
accumulator.moduleRangeIndex.add(moduleRange);
@@ -130,7 +130,11 @@ export const finalizeLocatedFindings = <
values: readonly LocatedJavaScriptFinding<Value>[],
modules: JavaScriptModuleRangeIndex,
): Value[] =>
values.map(({ offset, value }) => ({
...value,
module_key: modules.find(offset)?.key ?? null,
}));
values.map(({ offset, value }) => {
const module = modules.find(offset);
return {
...value,
module_key: module?.key ?? null,
...(module === undefined ? {} : { module_runtime: module.runtime }),
};
});
@@ -6,9 +6,27 @@ import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
it("preserves first insertion precedence and inclusive interval bounds", () => {
const nested = new JavaScriptModuleRangeIndex(100);
expect(nested.find(0)).toBeUndefined();
nested.add({ start: 10, end: 90, key: "outer-first", requireName: null });
nested.add({ start: 20, end: 30, key: "inner-second", requireName: null });
nested.add({ start: 30, end: 40, key: "later-overlap", requireName: null });
nested.add({
start: 10,
end: 90,
key: "outer-first",
runtime: "webpackChunkApp",
requireName: null,
});
nested.add({
start: 20,
end: 30,
key: "inner-second",
runtime: "webpackChunkApp",
requireName: null,
});
nested.add({
start: 30,
end: 40,
key: "later-overlap",
runtime: "webpackChunkApp",
requireName: null,
});
expect(nested.find(9)).toBeUndefined();
expect(nested.find(10)?.key).toBe("outer-first");
expect(nested.find(20)?.key).toBe("outer-first");
@@ -20,7 +38,13 @@ it("preserves first insertion precedence and inclusive interval bounds", () => {
it("rejects missing and out-of-range lookup offsets", () => {
const index = new JavaScriptModuleRangeIndex(20);
index.add({ start: 5, end: 10, key: "bounded", requireName: null });
index.add({
start: 5,
end: 10,
key: "bounded",
runtime: "webpackChunkApp",
requireName: null,
});
expect(index.find(undefined)).toBeUndefined();
expect(index.find(null)).toBeUndefined();
expect(index.find(-1)).toBeUndefined();
@@ -58,8 +82,11 @@ it("keeps module references attributed across many real factory ranges", () => {
expect(
analysis.references.find(
({ specifier }) => specifier === `dependency-${index}`,
)?.module_key,
).toBe(`module-${index}`);
),
).toMatchObject({
module_key: `module-${index}`,
module_runtime: "webpackChunkApp",
});
}
expect(
analysis.references.find(
@@ -23,6 +23,7 @@ export interface JavaScriptModuleRange {
readonly start: number;
readonly end: number;
readonly key: string;
readonly runtime: string;
readonly requireName: string | null;
}
@@ -25,6 +25,8 @@ interface JavaScriptStaticReferenceState {
| "worker"
| "service-worker";
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -41,6 +43,8 @@ export interface JavaScriptStaticEndpoint {
readonly value: string;
readonly mechanism: string;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -55,6 +59,8 @@ export interface JavaScriptStaticStorage {
readonly name: string | null;
readonly mechanism: string;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
@@ -95,6 +101,8 @@ export interface JavaScriptRolePath {
readonly resolution_context: JavaScriptStaticPathContext;
readonly mechanism: string;
readonly module_key: string | null;
/** Recovered factory runtime; absent in older or unbundled findings. */
readonly module_runtime?: string;
readonly location: JavaScriptSourceRange;
}
+9
View File
@@ -16,6 +16,15 @@ export class NonRegularFileReadError extends Error {
}
}
/** A regular input no longer matches the extent or identity admitted for reading. */
export class RegularFileChangedError extends Error {
readonly code = "ESTALE";
constructor(readonly path: string) {
super(`Selected input changed while being read: ${path}`);
this.name = "RegularFileChangedError";
}
}
/** One admitted file descriptor and the metadata that identified its contents. */
export interface StableRegularFileDescriptor {
readonly handle: FileHandle;
+55
View File
@@ -0,0 +1,55 @@
import { createHash } from "node:crypto";
import { stat } from "node:fs/promises";
import {
openRegularFile,
RegularFileChangedError,
sameRegularFileState,
} from "./RegularFile.js";
/** Hash one regular file from its admitted descriptor and verify its path state. */
export const hashStableFile = async (
path: string,
signal?: AbortSignal,
): Promise<{ readonly sha256: string; readonly bytes: number }> => {
const handle = await openRegularFile(path, {
symlinks: "follow",
signal,
});
try {
const initial = await handle.stat();
if (!Number.isSafeInteger(initial.size) || initial.size < 0)
throw new RangeError(
`Regular file size cannot be represented safely: ${path}`,
);
const hash = createHash("sha256");
const chunk = Buffer.allocUnsafe(64 * 1024);
let position = 0;
while (position <= initial.size) {
signal?.throwIfAborted();
const length = Math.min(chunk.length, initial.size - position + 1);
if (length <= 0) break;
const read = await handle.read(chunk, 0, length, position);
signal?.throwIfAborted();
if (read.bytesRead === 0) break;
hash.update(chunk.subarray(0, read.bytesRead));
position += read.bytesRead;
if (position > initial.size) throw new RegularFileChangedError(path);
}
signal?.throwIfAborted();
const [opened, currentPath] = await Promise.all([
handle.stat(),
stat(path),
]);
signal?.throwIfAborted();
if (
position !== initial.size ||
!sameRegularFileState(initial, opened) ||
!sameRegularFileState(initial, currentPath)
)
throw new RegularFileChangedError(path);
return { sha256: hash.digest("hex"), bytes: position };
} finally {
await handle.close();
}
};
+8 -2
View File
@@ -262,11 +262,17 @@ export const createDarwinProcessRunTokenReader = (
const read = async (
processes: readonly ProcessTableEntry[],
signal?: AbortSignal,
): Promise<ReadonlyMap<number, ProcessRunTokenObservation>> => {
signal?.throwIfAborted();
if (processes.length === 0) return new Map();
const binary = await compile();
const binary = await compile(signal);
const pids = processes.map(({ pid }) => String(pid));
const { stdout } = await execFileOutput(binary, pids, { timeout: 15_000 });
const { stdout } = await execFileOutput(binary, pids, {
timeout: 15_000,
...(signal === undefined ? {} : { signal }),
});
signal?.throwIfAborted();
const decoded = safeParseJson(stdout);
if (!decoded.ok)
throw new Error(
+18 -13
View File
@@ -377,7 +377,7 @@ describe("process identity during token validation", () => {
expect(reusedDuringFinalRead).toBe(true);
});
it("fails closed when a candidate PID changes identity during token reading", async () => {
it("fails closed when candidate PIDs change identity during token reading", async () => {
let identityReads = 0;
const process: ProcessTableEntry = {
pid: 911,
@@ -386,8 +386,10 @@ describe("process identity during token validation", () => {
state: "S",
command: "reused-during-token-read",
};
const processes = [process, { ...process, pid: 912, processGroupId: 912 }];
const listProcesses = vi.fn(async () => processes);
const host: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve([process]),
listProcesses,
environment: () => Promise.resolve({}),
processIdentities: (entries) => {
identityReads += 1;
@@ -414,21 +416,24 @@ describe("process identity during token validation", () => {
};
await expect(
verifyNoTokenOwnedProcesses("run-token", host, [
{ pid: process.pid, identity: "old-start" },
]),
verifyNoTokenOwnedProcesses(
"run-token",
host,
processes.map(({ pid }) => ({ pid, identity: "old-start" })),
),
).resolves.toEqual({
cleaned: false,
reason: "process identity could not be revalidated",
failures: [
{
pid: process.pid,
reason: "process-identity-unavailable",
diagnostic:
"process identity changed or became unavailable during token validation",
},
],
failures: processes.map(({ pid }) => ({
pid,
reason: "process-identity-unavailable",
diagnostic:
"process identity changed or became unavailable during token validation",
})),
});
// Initial observation, final token classification, then one fresh recheck
// shared by both changed identities.
expect(listProcesses).toHaveBeenCalledTimes(3);
});
});
@@ -0,0 +1,328 @@
import { describe, expect, it, vi } from "vitest";
import {
cleanupOwnedProcessGroup,
type ProcessOwnershipHost,
} from "./ProcessOwnership.js";
import {
observeOwnedProcessGroups,
observeOwnedProcessLineage,
} from "./ProcessOwnershipObservation.js";
import { host, ownership } from "./ProcessOwnership.fixture.js";
const processes = [100, 101].map((pid) => ({
pid,
parentPid: pid === 100 ? 1 : 100,
processGroupId: 100,
state: "S",
command: "fixture",
}));
describe("multi-group ownership observation", () => {
it("observes multiple groups from one process snapshot and token batch", async () => {
const rows = [
...processes,
{
pid: 200,
parentPid: 1,
processGroupId: 200,
state: "S",
command: "other-owned-group",
},
{
pid: 300,
parentPid: 1,
processGroupId: 300,
state: "Z",
command: "[exited]",
},
];
const listProcesses = vi.fn(() => Promise.resolve(rows));
const environment = vi.fn(() => Promise.resolve({}));
const runTokens = vi.fn<NonNullable<ProcessOwnershipHost["runTokens"]>>(
(members) =>
Promise.resolve(
new Map(
members.map(({ pid }) => [
pid,
{
state: "readable" as const,
runId: pid === 200 ? "other-run" : ownership.runId,
},
]),
),
),
);
await expect(
observeOwnedProcessGroups(ownership.runId, [100, 200, 300], {
listProcesses,
environment,
runTokens,
signalGroup: vi.fn(),
}),
).resolves.toEqual(
new Map([
[100, { state: "alive" }],
[
200,
{ state: "unverifiable", reason: "process ownership did not match" },
],
[300, { state: "empty" }],
]),
);
expect(listProcesses).toHaveBeenCalledTimes(1);
expect(runTokens).toHaveBeenCalledTimes(1);
expect(runTokens).toHaveBeenCalledWith(
rows.filter(({ state }) => state !== "Z"),
);
expect(environment).not.toHaveBeenCalled();
});
it("rechecks unavailable members individually before classifying their groups", async () => {
const rows = [
...processes,
{
pid: 200,
parentPid: 1,
processGroupId: 200,
state: "S",
command: "unreadable-group",
},
];
const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce(rows)
.mockResolvedValueOnce(rows.filter(({ pid }) => pid !== 101))
.mockResolvedValueOnce(rows);
const runTokens = vi.fn<NonNullable<ProcessOwnershipHost["runTokens"]>>(
(members) =>
Promise.resolve(
new Map(
members.map(({ pid }) => [
pid,
pid === 101 || pid === 200
? {
state: "unavailable" as const,
reason: `unreadable-${String(pid)}`,
}
: { state: "readable" as const, runId: ownership.runId },
]),
),
),
);
await expect(
observeOwnedProcessGroups(ownership.runId, [100, 200], {
listProcesses,
environment: vi.fn(() => Promise.resolve({})),
runTokens,
signalGroup: vi.fn(),
}),
).resolves.toEqual(
new Map([
[100, { state: "alive" }],
[
200,
{
state: "unverifiable",
reason:
"process ownership could not be revalidated for PID 200: unreadable-200",
},
],
]),
);
expect(listProcesses).toHaveBeenCalledTimes(3);
expect(runTokens).toHaveBeenCalledTimes(1);
});
});
describe("multi-group observation short-circuit", () => {
it("stops missing-row fallbacks once every live group is unverifiable", async () => {
const listProcesses = vi.fn(() => Promise.resolve(processes));
const environment = vi.fn((pid: number) =>
Promise.resolve({
REA_PROCESS_RUN_ID: pid === 100 ? "other-run" : ownership.runId,
}),
);
const runTokens = vi.fn<NonNullable<ProcessOwnershipHost["runTokens"]>>(
() => Promise.resolve(new Map()),
);
await expect(
observeOwnedProcessGroups(ownership.runId, [100], {
listProcesses,
environment,
runTokens,
signalGroup: vi.fn(),
}),
).resolves.toEqual(
new Map([
[
100,
{ state: "unverifiable", reason: "process ownership did not match" },
],
]),
);
expect(runTokens).toHaveBeenCalledTimes(1);
expect(environment.mock.calls).toEqual([[100]]);
expect(listProcesses).toHaveBeenCalledTimes(1);
});
});
describe("batched ownership observation", () => {
it("observes group and lineage with one batch each and falls back only for missing rows", async () => {
const { adapter } = host({
100: { REA_PROCESS_RUN_ID: ownership.runId },
101: { REA_PROCESS_RUN_ID: ownership.runId },
});
const environment = vi.fn(adapter.environment);
const runTokens = vi.fn<NonNullable<ProcessOwnershipHost["runTokens"]>>(
() =>
Promise.resolve(
new Map([[100, { state: "readable", runId: ownership.runId }]]),
),
);
const observedHost = { ...adapter, environment, runTokens };
await expect(
observeOwnedProcessGroups(
ownership.runId,
[ownership.processGroupId],
observedHost,
),
).resolves.toEqual(
new Map([[ownership.processGroupId, { state: "alive" }]]),
);
await expect(
observeOwnedProcessLineage(ownership, observedHost),
).resolves.toMatchObject({
status: "verified",
lineage: {
descendants: [{ pid: 101, parentPid: 100, processGroupId: 100 }],
},
});
expect(runTokens.mock.calls).toEqual([[processes], [processes]]);
expect(environment.mock.calls).toEqual([[101], [101]]);
});
it.each(["batch", "environment"] as const)(
"preserves cancellation during %s reads, including successful completion races",
async (boundary) => {
const controller = new AbortController();
const reason = new Error("cancelled ownership read");
const listProcesses = vi.fn(() => Promise.resolve(processes));
const environment = vi.fn<ProcessOwnershipHost["environment"]>(
async (_pid, signal) => {
expect(signal).toBe(controller.signal);
controller.abort(reason);
return { REA_PROCESS_RUN_ID: ownership.runId };
},
);
const runTokens = vi.fn<NonNullable<ProcessOwnershipHost["runTokens"]>>(
async (_processes, signal) => {
expect(signal).toBe(controller.signal);
if (boundary === "batch") controller.abort(reason);
return new Map();
},
);
await expect(
observeOwnedProcessGroups(
ownership.runId,
[ownership.processGroupId],
{ listProcesses, environment, runTokens, signalGroup: vi.fn() },
controller.signal,
),
).rejects.toBe(reason);
expect(listProcesses).toHaveBeenCalledTimes(1);
expect(environment).toHaveBeenCalledTimes(boundary === "batch" ? 0 : 1);
},
);
it.each([100, 101])(
"rechecks unavailable batch rows for exited PID %i without overriding them through fallback",
async (exitedPid) => {
const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce(processes)
.mockResolvedValue(processes.filter(({ pid }) => pid !== exitedPid));
const environment = vi.fn(() =>
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
);
const observedHost: ProcessOwnershipHost = {
listProcesses,
environment,
signalGroup: vi.fn(),
runTokens: () =>
Promise.resolve(
new Map(
processes.map(({ pid }) => [
pid,
pid === exitedPid
? { state: "unavailable" as const, reason: "process exited" }
: { state: "readable" as const, runId: ownership.runId },
]),
),
),
};
const result = await observeOwnedProcessLineage(ownership, observedHost);
if (exitedPid === 100)
expect(result).toMatchObject({
status: "unavailable",
reason: "owned launcher exited during lineage validation",
});
else
expect(result).toMatchObject({
status: "verified",
lineage: { descendants: [] },
});
expect(environment).not.toHaveBeenCalled();
},
);
});
describe("token sweep batch failures", () => {
it("cleans token-owned groups when a failed batch has readable fallback observations", async () => {
const { adapter, signalGroup } = host({
100: { REA_PROCESS_RUN_ID: ownership.runId },
101: { REA_PROCESS_RUN_ID: ownership.runId },
});
await expect(
cleanupOwnedProcessGroup(
{ ...ownership, sweepTokenOwnedProcesses: true, captureBaseline: [] },
{
...adapter,
runTokens: () =>
Promise.reject(new Error("batch reader unavailable")),
},
),
).resolves.toEqual({ cleaned: true, signaled: true });
expect(signalGroup.mock.calls).toEqual([[100, "SIGKILL"]]);
});
it("uses fallback ownership when the initial batch fails and preserves both failures through retries", async () => {
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve(processes),
environment: (pid) =>
pid === 100
? Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId })
: Promise.reject(new Error("individual unreadable")),
runTokens: () => Promise.reject(new Error("batch reader unavailable")),
signalGroup,
};
const result = await cleanupOwnedProcessGroup(
{ ...ownership, sweepTokenOwnedProcesses: true, captureBaseline: [] },
adapter,
);
expect(result).toMatchObject({
cleaned: false,
failures: expect.arrayContaining([
expect.objectContaining({
pid: 101,
diagnostic:
"individual unreadable; run-token batch failed: batch reader unavailable",
}),
]),
});
expect(signalGroup).not.toHaveBeenCalled();
});
});
+93 -103
View File
@@ -7,6 +7,7 @@ import {
} from "./ProcessOwnershipObservation.js";
import { launcherIdentityFailure } from "./ProcessOwnershipIdentity.js";
import { descendantsOf, liveProcesses } from "./ProcessOwnershipProcessTree.js";
import { readProcessRunTokens } from "./ProcessRunTokenObservations.js";
const execFileAsync = promisify(execFile);
@@ -90,10 +91,14 @@ export interface ProcessOwnershipHost {
close?(): Promise<void>;
/** List current processes; startup inspection may be cancelled by the caller. */
listProcesses(signal?: AbortSignal): Promise<readonly ProcessTableEntry[]>;
environment(pid: number): Promise<Readonly<Record<string, string>>>;
environment(
pid: number,
signal?: AbortSignal,
): Promise<Readonly<Record<string, string>>>;
/** Read only run-token values in one host operation when the OS supports it. */
runTokens?(
processes: readonly ProcessTableEntry[],
signal?: AbortSignal,
): Promise<ReadonlyMap<number, ProcessRunTokenObservation>>;
/** Read stable identities; omit the signal during cleanup so it can finish. */
processIdentities?(
@@ -623,6 +628,7 @@ const processOwnershipFailures = async (
expectedIdentities.has(pid),
);
const observed = await host.processIdentities(expectedMembers);
const identityFailures: ProcessOwnershipValidationFailure[] = [];
for (const member of expectedMembers) {
const expected = expectedIdentities.get(member.pid);
const current = observed.get(member.pid);
@@ -631,8 +637,7 @@ const processOwnershipFailures = async (
current?.state !== "readable" ||
current.identity !== expected
) {
if (await processIsGone(host, member.pid)) continue;
failures.push({
identityFailures.push({
pid: member.pid,
reason: "process-identity-unavailable",
diagnostic:
@@ -640,30 +645,9 @@ const processOwnershipFailures = async (
});
}
}
failures.push(...(await recheckOwnershipFailures(identityFailures, host)));
}
for (const member of members) {
try {
if ((await host.environment(member.pid)).REA_PROCESS_RUN_ID !== runId)
failures.push({ pid: member.pid, reason: "run-token-mismatch" });
} catch (cause: unknown) {
try {
const live = liveProcesses(await host.listProcesses());
if (!live.some(({ pid }) => pid === member.pid)) continue;
} catch (recheckCause: unknown) {
failures.push({
pid: member.pid,
reason: "environment-unreadable",
diagnostic: `${errorMessage(cause)}; process liveness recheck failed: ${errorMessage(recheckCause)}`,
});
continue;
}
failures.push({
pid: member.pid,
reason: "environment-unreadable",
diagnostic: errorMessage(cause),
});
}
}
failures.push(...(await processRunTokenFailures(members, runId, host)));
if (
failures.length === 0 &&
expectedIdentities.size > 0 &&
@@ -673,6 +657,7 @@ const processOwnershipFailures = async (
expectedIdentities.has(pid),
);
const observed = await host.processIdentities(expectedMembers);
const identityFailures: ProcessOwnershipValidationFailure[] = [];
for (const member of expectedMembers) {
const expected = expectedIdentities.get(member.pid);
const current = observed.get(member.pid);
@@ -682,18 +667,72 @@ const processOwnershipFailures = async (
current.identity === expected
)
continue;
if (await processIsGone(host, member.pid)) continue;
failures.push({
identityFailures.push({
pid: member.pid,
reason: "process-identity-unavailable",
diagnostic:
"process identity changed or became unavailable during token validation",
});
}
failures.push(...(await recheckOwnershipFailures(identityFailures, host)));
}
return failures;
};
const processRunTokenFailures = async (
members: readonly ProcessTableEntry[],
runId: string,
host: ProcessOwnershipHost,
): Promise<readonly ProcessOwnershipValidationFailure[]> => {
const failures: ProcessOwnershipValidationFailure[] = [];
for await (const { process: member, observation } of readProcessRunTokens(
host,
members,
)) {
if (observation.state === "readable") {
if (observation.runId !== runId)
failures.push({ pid: member.pid, reason: "run-token-mismatch" });
continue;
}
failures.push({
pid: member.pid,
reason: "environment-unreadable",
diagnostic: observation.reason,
});
}
return recheckOwnershipFailures(failures, host);
};
/**
* Take a fresh liveness snapshot after each ownership-read phase. Only an
* absent PID resolves an unreadable observation; a readable token mismatch
* remains a failure. Never reuse this snapshot across identity/token reads.
*/
const recheckOwnershipFailures = async (
failures: readonly ProcessOwnershipValidationFailure[],
host: ProcessOwnershipHost,
): Promise<readonly ProcessOwnershipValidationFailure[]> => {
if (!failures.some(({ reason }) => reason !== "run-token-mismatch"))
return failures;
try {
const live = new Set(
liveProcesses(await host.listProcesses()).map(({ pid }) => pid),
);
return failures.filter(
({ pid, reason }) => reason === "run-token-mismatch" || live.has(pid),
);
} catch (cause: unknown) {
return failures.map((failure) =>
failure.reason === "run-token-mismatch"
? failure
: {
...failure,
diagnostic: `${failure.diagnostic}; process liveness recheck failed: ${errorMessage(cause)}`,
},
);
}
};
interface TokenOwnedProcessScan {
readonly owned: readonly ProcessTableEntry[];
readonly failures: readonly ProcessOwnershipValidationFailure[];
@@ -767,18 +806,19 @@ const scanTokenOwnedProcesses = async (
const failures: ProcessOwnershipValidationFailure[] = [];
if (captureBaseline !== undefined && host.processIdentities !== undefined) {
const stableCandidates: ProcessTableEntry[] = [];
const identityFailures: ProcessOwnershipValidationFailure[] = [];
for (const process of candidates) {
if (candidateIdentities.get(process.pid)?.state === "readable") {
stableCandidates.push(process);
continue;
}
if (!(await processIsGone(host, process.pid)))
failures.push({
pid: process.pid,
reason: "process-identity-unavailable",
diagnostic: "process identity was unavailable during token scan",
});
identityFailures.push({
pid: process.pid,
reason: "process-identity-unavailable",
diagnostic: "process identity was unavailable during token scan",
});
}
failures.push(...(await recheckOwnershipFailures(identityFailures, host)));
candidates = stableCandidates;
}
const owned: ProcessTableEntry[] = [];
@@ -791,26 +831,14 @@ const scanTokenOwnedProcesses = async (
identity?.state === "readable" ? identity.identity : null,
);
};
let bulkTokens: ReadonlyMap<number, ProcessRunTokenObservation> | undefined;
if (host.runTokens !== undefined)
bulkTokens = await host.runTokens(candidates);
const unreadable: UnreadableTokenCandidate[] = [];
for (const process of candidates) {
const bulkToken = bulkTokens?.get(process.pid);
if (bulkToken !== undefined) {
if (bulkToken.state === "readable") {
if (bulkToken.runId === runId) recordOwned(process);
continue;
}
unreadable.push({ process, diagnostic: bulkToken.reason });
continue;
}
try {
if ((await host.environment(process.pid)).REA_PROCESS_RUN_ID === runId)
recordOwned(process);
} catch (cause: unknown) {
unreadable.push({ process, diagnostic: errorMessage(cause) });
}
for await (const { process, observation } of readProcessRunTokens(
host,
candidates,
)) {
if (observation.state === "readable") {
if (observation.runId === runId) recordOwned(process);
} else unreadable.push({ process, diagnostic: observation.reason });
}
const remaining = await settleUnreadableTokenCandidates(
unreadable,
@@ -876,6 +904,7 @@ const scanTokenOwnedProcesses = async (
if (candidates.length > 0 && host.processIdentities !== undefined) {
const afterRead = await host.processIdentities(candidates);
const stillOwned: ProcessTableEntry[] = [];
const identityFailures: ProcessOwnershipValidationFailure[] = [];
const ownedPids = new Set(owned.map(({ pid }) => pid));
for (const process of candidates) {
const before = candidateIdentities.get(process.pid);
@@ -888,14 +917,14 @@ const scanTokenOwnedProcesses = async (
if (ownedPids.has(process.pid)) stillOwned.push(process);
continue;
}
if (await processIsGone(host, process.pid)) continue;
failures.push({
identityFailures.push({
pid: process.pid,
reason: "process-identity-unavailable",
diagnostic:
"process identity changed or became unavailable during token validation",
});
}
failures.push(...(await recheckOwnershipFailures(identityFailures, host)));
return {
owned: stillOwned,
failures,
@@ -936,31 +965,20 @@ const settleUnreadableTokenCandidates = async (
void cause;
live = undefined;
}
let observations:
| ReadonlyMap<number, ProcessRunTokenObservation>
| undefined;
if (host.runTokens !== undefined) {
try {
observations = await host.runTokens(
remaining.map(({ process }) => process),
);
} catch (cause: unknown) {
void cause;
observations = undefined;
}
}
const stillUnreadable: UnreadableTokenCandidate[] = [];
for (const candidate of remaining) {
const pid = candidate.process.pid;
if (live !== undefined && !live.has(pid)) continue;
const observation =
observations?.get(pid) ?? (await readEnvironmentRunToken(host, pid));
const candidates = remaining
.map(({ process }) => process)
.filter(({ pid }) => live === undefined || live.has(pid));
for await (const { process, observation } of readProcessRunTokens(
host,
candidates,
)) {
if (observation.state === "readable") {
if (observation.runId === runId) recordOwned(candidate.process);
if (observation.runId === runId) recordOwned(process);
continue;
}
stillUnreadable.push({
process: candidate.process,
process,
diagnostic: observation.reason,
});
}
@@ -968,31 +986,3 @@ const settleUnreadableTokenCandidates = async (
}
return remaining;
};
const readEnvironmentRunToken = async (
host: ProcessOwnershipHost,
pid: number,
): Promise<ProcessRunTokenObservation> => {
try {
return {
state: "readable",
runId: (await host.environment(pid)).REA_PROCESS_RUN_ID,
};
} catch (cause: unknown) {
return { state: "unavailable", reason: errorMessage(cause) };
}
};
const processIsGone = async (
host: ProcessOwnershipHost,
pid: number,
): Promise<boolean> => {
try {
return !liveProcesses(await host.listProcesses()).some(
(process) => process.pid === pid,
);
} catch (cause: unknown) {
void cause;
return false;
}
};
+234 -6
View File
@@ -6,7 +6,7 @@ import {
type WindowsProcessTreeHost,
} from "./ProcessOwnership.js";
import {
observeOwnedProcessGroup,
observeOwnedProcessGroups,
observeOwnedProcessLineage,
} from "./ProcessOwnershipObservation.js";
import { host, ownership } from "./ProcessOwnership.fixture.js";
@@ -115,6 +115,101 @@ describe("owned process-group cleanup validation: ownership and lineage", () =>
});
expect(signalGroup).not.toHaveBeenCalled();
});
});
describe("owned process-group cleanup liveness rechecks", () => {
it("rechecks all unreadable members after the token phase in one fresh snapshot", async () => {
const members = [100, 101, 102, 103].map((pid) => ({
pid,
parentPid: pid === 100 ? 1 : 100,
processGroupId: 100,
state: "S",
command: "fixture",
}));
const tokenReads: number[] = [];
const listProcesses = vi.fn(async () => {
if (tokenReads.length > 0)
expect(tokenReads).toEqual([100, 101, 102, 103]);
return members;
});
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses,
environment: async (pid) => {
tokenReads.push(pid);
throw new Error(`unreadable ${pid}`);
},
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{
cleaned: false,
reason:
"process ownership token could not be read for 4 live process(es): other_unavailable=4; live candidates 100=other_unavailable, 101=other_unavailable, 102=other_unavailable, 103=other_unavailable",
failures: members.map(({ pid }) => ({
pid,
reason: "environment-unreadable",
diagnostic: `unreadable ${pid}`,
})),
},
);
expect(listProcesses).toHaveBeenCalledTimes(2);
expect(signalGroup).not.toHaveBeenCalled();
});
it.each([false, true])(
"retains mismatches and fails closed when the liveness recheck fails: %s",
async (recheckFails) => {
const members = [100, 101, 102].map((pid) => ({
pid,
parentPid: pid === 100 ? 1 : 100,
processGroupId: 100,
state: "S",
command: "fixture",
}));
const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce(members);
if (recheckFails)
listProcesses.mockRejectedValue(new Error("process table denied"));
else listProcesses.mockResolvedValue([]);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses,
environment: async (pid) => {
if (pid === 101) return { REA_PROCESS_RUN_ID: "other-run" };
throw new Error(`unreadable ${pid}`);
},
signalGroup,
};
await expect(
cleanupOwnedProcessGroup(ownership, adapter),
).resolves.toEqual({
cleaned: false,
reason: "process tree contains an unowned or PID-reused process",
failures: recheckFails
? [
{
pid: 100,
reason: "environment-unreadable",
diagnostic:
"unreadable 100; process liveness recheck failed: process table denied",
},
{ pid: 101, reason: "run-token-mismatch" },
{
pid: 102,
reason: "environment-unreadable",
diagnostic:
"unreadable 102; process liveness recheck failed: process table denied",
},
]
: [{ pid: 101, reason: "run-token-mismatch" }],
});
expect(listProcesses).toHaveBeenCalledTimes(2);
expect(signalGroup).not.toHaveBeenCalled();
},
);
it("accepts a member that exits during ownership revalidation", async () => {
const liveLauncher = {
pid: 100,
@@ -160,7 +255,12 @@ describe("owned process-group cleanup validation: exited members", () => {
},
};
await expect(
observeOwnedProcessGroup(ownership, adapter, controller.signal),
observeOwnedProcessGroups(
ownership.runId,
[ownership.processGroupId],
adapter,
controller.signal,
),
).rejects.toBe(reason);
});
it("does not inspect a process group after observation was cancelled", async () => {
@@ -177,7 +277,12 @@ describe("owned process-group cleanup validation: exited members", () => {
},
};
await expect(
observeOwnedProcessGroup(ownership, adapter, controller.signal),
observeOwnedProcessGroups(
ownership.runId,
[ownership.processGroupId],
adapter,
controller.signal,
),
).rejects.toBe(reason);
});
it("ignores exited zombie members during live ownership checks", async () => {
@@ -229,9 +334,13 @@ describe("owned process-group cleanup validation: exited members", () => {
environment,
signalGroup: vi.fn(),
};
expect(await observeOwnedProcessGroup(ownership, adapter)).toEqual({
state: "empty",
});
expect(
await observeOwnedProcessGroups(
ownership.runId,
[ownership.processGroupId],
adapter,
),
).toEqual(new Map([[ownership.processGroupId, { state: "empty" }]]));
expect(environment).not.toHaveBeenCalled();
});
it("is idempotent when the owned group has already exited", async () => {
@@ -278,6 +387,125 @@ describe("owned process-group cleanup validation: exited members", () => {
});
});
describe("owned process-group cleanup token batches", () => {
const root = {
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
};
it("batches validation and falls back only for a missing member row", async () => {
const child = { ...root, pid: 101, parentPid: root.pid };
const runTokens = vi.fn(() =>
Promise.resolve(
new Map([
[root.pid, { state: "readable" as const, runId: ownership.runId }],
]),
),
);
const environment = vi.fn(() =>
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve([root, child]),
environment,
runTokens,
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{ cleaned: true, signaled: true },
);
expect(runTokens).toHaveBeenCalledTimes(2);
expect(runTokens).toHaveBeenNthCalledWith(1, [root, child]);
expect(environment.mock.calls).toEqual([[child.pid], [child.pid]]);
expect(signalGroup).toHaveBeenCalledWith(root.processGroupId, "SIGKILL");
});
it("keeps explicit unreadable rows fail-closed without environment fallback", async () => {
const environment = vi.fn(() =>
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve([root]),
environment,
runTokens: () =>
Promise.resolve(
new Map([
[
root.pid,
{ state: "unavailable" as const, reason: "fixture unreadable" },
],
]),
),
signalGroup,
};
await expect(
cleanupOwnedProcessGroup(ownership, adapter),
).resolves.toMatchObject({
cleaned: false,
failures: [
{
pid: root.pid,
reason: "environment-unreadable",
diagnostic: "fixture unreadable",
},
],
});
expect(environment).not.toHaveBeenCalled();
expect(signalGroup).not.toHaveBeenCalled();
});
it("falls back to the environment reader when the batch operation throws", async () => {
const environment = vi.fn(() =>
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve([root]),
environment,
runTokens: () => Promise.reject(new Error("batch reader unavailable")),
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{ cleaned: true, signaled: true },
);
expect(environment).toHaveBeenCalledTimes(2);
expect(signalGroup).toHaveBeenCalledWith(root.processGroupId, "SIGKILL");
});
it("retains both failures when batch and individual token reads fail", async () => {
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve([root]),
environment: () => Promise.reject(new Error("individual unreadable")),
runTokens: () => Promise.reject(new Error("batch reader unavailable")),
signalGroup,
};
await expect(
cleanupOwnedProcessGroup(ownership, adapter),
).resolves.toMatchObject({
cleaned: false,
failures: [
{
pid: root.pid,
reason: "environment-unreadable",
diagnostic:
"individual unreadable; run-token batch failed: batch reader unavailable",
},
],
});
expect(signalGroup).not.toHaveBeenCalled();
});
});
describe("Windows P0 process-tree cleanup", () => {
it("reports whether taskkill signaled or found an exited tree", async () => {
const terminated: WindowsProcessTreeHost = {
+76 -40
View File
@@ -5,6 +5,7 @@ import { launcherIdentityFailure } from "./ProcessOwnershipIdentity.js";
import { descendantsOf, liveProcesses } from "./ProcessOwnershipProcessTree.js";
import { execFileOutput } from "./ExecFileOutput.js";
import { createDarwinProcessRunTokenReader } from "./DarwinProcessRunTokenReader.js";
import { readProcessRunTokens } from "./ProcessRunTokenObservations.js";
import type {
OwnedProcessGroup,
ProcessGroupObservation,
@@ -15,49 +16,78 @@ import type {
ProcessTableEntry,
} from "./ProcessOwnership.js";
/** Observe one group without signaling it, failing closed on identity doubt. */
export const observeOwnedProcessGroup = async (
ownership: OwnedProcessGroup,
/** Observe multiple groups from one process snapshot and one token batch. */
export const observeOwnedProcessGroups = async (
runId: string,
processGroupIds: readonly number[],
host: ProcessOwnershipHost = systemProcessOwnershipHost,
signal?: AbortSignal,
): Promise<ProcessGroupObservation> => {
): Promise<ReadonlyMap<number, ProcessGroupObservation>> => {
signal?.throwIfAborted();
let members: readonly ProcessTableEntry[];
const groupIds = [...new Set(processGroupIds)];
if (groupIds.length === 0) return new Map();
let processes: readonly ProcessTableEntry[];
try {
members = (await host.listProcesses(signal)).filter(
({ processGroupId }) => processGroupId === ownership.processGroupId,
);
processes = liveProcesses(await host.listProcesses(signal));
} catch (cause: unknown) {
signal?.throwIfAborted();
return {
state: "unverifiable",
const failure = {
state: "unverifiable" as const,
reason: `process group could not be inspected: ${errorMessage(cause)}`,
};
return new Map(groupIds.map((groupId) => [groupId, failure]));
}
signal?.throwIfAborted();
const liveMembers = liveProcesses(members);
if (liveMembers.length === 0) return { state: "empty" };
for (const member of liveMembers) {
signal?.throwIfAborted();
try {
const runId = (await host.environment(member.pid)).REA_PROCESS_RUN_ID;
signal?.throwIfAborted();
if (runId !== ownership.runId)
return {
const membersByGroup = new Map(
groupIds.map((groupId) => [groupId, [] as ProcessTableEntry[]]),
);
for (const process of processes)
membersByGroup.get(process.processGroupId)?.push(process);
const observations = new Map<number, ProcessGroupObservation>(
groupIds.map((groupId) => [
groupId,
(membersByGroup.get(groupId)?.length ?? 0) > 0
? { state: "alive" }
: { state: "empty" },
]),
);
const liveMembers = groupIds.flatMap(
(groupId) => membersByGroup.get(groupId) ?? [],
);
const pendingGroups = new Set(
groupIds.filter(
(groupId) => (membersByGroup.get(groupId)?.length ?? 0) > 0,
),
);
for await (const { process: member, observation } of readProcessRunTokens(
host,
liveMembers,
signal,
)) {
if (!pendingGroups.has(member.processGroupId)) continue;
if (observation.state === "readable") {
if (observation.runId !== runId) {
observations.set(member.processGroupId, {
state: "unverifiable",
reason: "process ownership did not match",
};
} catch (cause: unknown) {
});
pendingGroups.delete(member.processGroupId);
}
} else {
const gone = await processIsGone(host, member.pid);
signal?.throwIfAborted();
if (!(await processIsGone(host, member.pid))) {
return {
if (!gone) {
observations.set(member.processGroupId, {
state: "unverifiable",
reason: `process ownership could not be revalidated for PID ${member.pid}: ${errorMessage(cause)}`,
};
reason: `process ownership could not be revalidated for PID ${member.pid}: ${observation.reason}`,
});
pendingGroups.delete(member.processGroupId);
}
}
if (pendingGroups.size === 0) break;
}
return { state: "alive" };
signal?.throwIfAborted();
return observations;
};
/**
@@ -88,18 +118,18 @@ export const observeOwnedProcessLineage = async (
return unavailableLineage(ownership, identityFailure);
const descendants = descendantsOf(launcher.pid, processes);
const verifiedDescendants: ProcessTableEntry[] = [];
for (const member of [launcher, ...descendants]) {
try {
if (
(await host.environment(member.pid)).REA_PROCESS_RUN_ID !==
ownership.runId
)
for await (const { process: member, observation } of readProcessRunTokens(
host,
[launcher, ...descendants],
)) {
if (observation.state === "readable") {
if (observation.runId !== ownership.runId)
return unavailableLineage(
ownership,
"process lineage contains an unowned or PID-reused process",
);
if (member.pid !== launcher.pid) verifiedDescendants.push(member);
} catch (cause: unknown) {
} else {
if (await processIsGone(host, member.pid)) {
if (member.pid === launcher.pid)
return unavailableLineage(
@@ -110,7 +140,7 @@ export const observeOwnedProcessLineage = async (
}
return unavailableLineage(
ownership,
`process ownership could not be revalidated for PID ${member.pid}: ${errorMessage(cause)}`,
`process ownership could not be revalidated for PID ${member.pid}: ${observation.reason}`,
);
}
}
@@ -330,18 +360,21 @@ export const createSystemProcessOwnershipHost = (
}
},
listProcesses,
async environment(pid) {
async environment(pid, signal) {
signal?.throwIfAborted();
if (platform === "linux")
return parseProcessEnvironment(
await readFile(`/proc/${pid}/environ`, "utf8"),
await readFile(`/proc/${pid}/environ`, { encoding: "utf8", signal }),
);
if (platform === "darwin") {
const process = (await listProcesses()).find(
const process = (await listProcesses(signal)).find(
(entry) => entry.pid === pid,
);
if (process === undefined)
throw new Error(`process ${String(pid)} is not live`);
const observation = (await darwinTokens?.read([process]))?.get(pid);
const observation = (await darwinTokens?.read([process], signal))?.get(
pid,
);
if (observation === undefined || observation.state === "unavailable")
throw new Error(
observation?.reason ?? "process run token could not be read",
@@ -355,6 +388,7 @@ export const createSystemProcessOwnershipHost = (
["eww", "-p", String(pid)],
{
env: hostEnvironment,
...(signal === undefined ? {} : { signal }),
},
);
const observedEnvironment: Record<string, string> = {};
@@ -366,11 +400,13 @@ export const createSystemProcessOwnershipHost = (
}
return observedEnvironment;
},
async runTokens(processes) {
async runTokens(processes, signal) {
signal?.throwIfAborted();
if (platform !== "darwin" || processes.length === 0) return new Map();
try {
return (await darwinTokens?.read(processes)) ?? new Map();
return (await darwinTokens?.read(processes, signal)) ?? new Map();
} catch (cause: unknown) {
if (isExpectedAbort(cause, signal)) throw cause;
void cause;
return new Map(
processes.map(({ pid }) => [
@@ -0,0 +1,62 @@
import type {
ProcessOwnershipHost,
ProcessRunTokenObservation,
ProcessTableEntry,
} from "./ProcessOwnership.js";
const errorMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Read only ownership tokens, using per-process fallback for missing batch rows. */
export async function* readProcessRunTokens(
host: ProcessOwnershipHost,
processes: readonly ProcessTableEntry[],
signal?: AbortSignal,
): AsyncGenerator<{
readonly process: ProcessTableEntry;
readonly observation: ProcessRunTokenObservation;
}> {
signal?.throwIfAborted();
if (processes.length === 0) return;
let batch: ReadonlyMap<number, ProcessRunTokenObservation> | undefined;
let batchFailure: string | undefined;
if (host.runTokens !== undefined) {
try {
batch =
signal === undefined
? await host.runTokens(processes)
: await host.runTokens(processes, signal);
} catch (cause: unknown) {
signal?.throwIfAborted();
batchFailure = errorMessage(cause);
}
}
for (const process of processes) {
signal?.throwIfAborted();
let observation = batch?.get(process.pid);
if (observation === undefined) {
try {
const environment =
signal === undefined
? await host.environment(process.pid)
: await host.environment(process.pid, signal);
observation = {
state: "readable",
runId: environment.REA_PROCESS_RUN_ID,
};
} catch (cause: unknown) {
signal?.throwIfAborted();
const reason = errorMessage(cause);
observation = {
state: "unavailable",
reason:
batchFailure === undefined
? reason
: `${reason}; run-token batch failed: ${batchFailure}`,
};
}
}
signal?.throwIfAborted();
yield { process, observation };
}
}
+25 -13
View File
@@ -16,29 +16,41 @@ import { classifyFilesystemEffects } from "./ProcessFilesystemEffects.js";
it.each(["after", "before"] as const)(
"keeps absence unknown when the %s snapshot omits an existing file",
async (partialSide) => {
// Enumeration order is filesystem-dependent. Choose a genuinely omitted
// entry by comparing a complete snapshot with a capacity-limited one.
const root = await createTestTempDirectory("rea-fs-partial-absence-");
await writeFile(join(root, "z.txt"), "unchanged");
await writeFile(join(root, "a1.txt"), "first");
await writeFile(join(root, "a2.txt"), "second");
await writeFile(join(root, "z.txt"), "target");
const scenario = parseProcessScenario({
executable: process.execPath,
working_directory: root,
filesystem_observation_paths: [root],
limits: { files: 2 },
});
if (partialSide === "before") await writeFile(join(root, "a.txt"), "other");
const before = await snapshotRoots(scenario);
if (partialSide === "after") await writeFile(join(root, "a.txt"), "other");
else await rm(join(root, "a.txt"));
const after = await snapshotRoots(scenario);
expect(await readFile(join(root, "z.txt"), "utf8")).toBe("unchanged");
const complete = await snapshotRoots({
...scenario,
limits: { ...scenario.limits, files: 4 },
});
const partial = await snapshotRoots(scenario);
expect(complete.completeRoots).toEqual(["root_0"]);
expect(partial.truncated).toBe(true);
const omitted = complete.files.find(
(file) => !partial.files.some(({ path }) => path === file.path),
);
if (omitted === undefined) throw new Error("Expected an omitted entry");
const before = partialSide === "before" ? partial : complete;
const after = partialSide === "after" ? partial : complete;
const effect = classifyFilesystemEffects(before, after).find(
({ path }) => path === "root_0:z.txt",
({ path }) => path === omitted.path,
);
expect(effect?.status).toBe("unknown");
expect(effect).toMatchObject(
partialSide === "after"
? { before: { path: "root_0:z.txt" }, after: null }
: { before: null, after: { path: "root_0:z.txt" } },
);
expect(effect).toMatchObject({
...(partialSide === "after"
? { before: omitted, after: null }
: { before: null, after: omitted }),
reason: expect.stringContaining("did not exhaust path enumeration"),
});
},
);
+10 -11
View File
@@ -25,7 +25,7 @@ import {
type ProcessOwnershipBaseline,
} from "../ProcessOwnership.js";
import {
observeOwnedProcessGroup,
observeOwnedProcessGroups,
prepareProcessOwnershipInspection,
systemProcessOwnershipHost,
} from "../ProcessOwnershipObservation.js";
@@ -616,21 +616,20 @@ export const observeSettlement = async (
while (!deadlineReached) {
onLiveProgress?.();
assertNotCancelled(signal);
const observations = await Promise.all(
[...new Set(processGroupIds)].map((processGroupId) =>
observeOwnedProcessGroup(
{ runId, leaderPid: processGroupId, processGroupId },
undefined,
signal,
),
),
const observations = await observeOwnedProcessGroups(
runId,
processGroupIds,
undefined,
signal,
);
assertNotCancelled(signal);
if (observations.some(({ state }) => state === "unverifiable")) {
if (
[...observations.values()].some(({ state }) => state === "unverifiable")
) {
recordEvent("lifecycle", 1);
return { state: "unverifiable", elapsed_ms: Date.now() - started };
}
if (observations.every(({ state }) => state === "empty")) {
if ([...observations.values()].every(({ state }) => state === "empty")) {
consecutiveEmpty += 1;
if (consecutiveEmpty >= 2) {
recordEvent("lifecycle", 1);
+23 -7
View File
@@ -18,6 +18,7 @@ import {
} from "./ProcessCaptureError.js";
import {
releaseProcessResources,
ProcessCaptureResourceScope,
createProcessCaptureObservationBuffer,
prepareProcessCapture,
resolveProcessResult,
@@ -291,26 +292,31 @@ it("keeps empty cleanup exception messages actionable in the report", async () =
});
it.each(["termination", "verification"] as const)(
"retains observations and releases the temporary root when process %s throws",
"retains observations and retries the temporary root when process %s throws",
async (stage) => {
const cleanupFailure = new Error(`${stage} inspection failed`);
const removeTemporaryRoot = vi.fn(async () => undefined);
let failing = true;
const scope = new ProcessCaptureResourceScope();
const host: ProcessCaptureCleanupHost = {
platform: "linux",
cleanupProcessGroup: async () => {
if (stage === "termination") throw cleanupFailure;
if (failing && stage === "termination") throw cleanupFailure;
return {
cleaned: true,
signaled: true,
unverified: [{ pid: 900, diagnostic: "environment unavailable" }],
unverified: failing
? [{ pid: 900, diagnostic: "environment unavailable" }]
: [],
};
},
verifyTokenOwnedProcesses: async () => {
throw cleanupFailure;
if (failing) throw cleanupFailure;
return { cleaned: true, signaled: false };
},
removeTemporaryRoot,
};
const cleanup = await releaseProcessResources({
const cleanup = await scope.release({
timers: new Set(),
terminal: { pid: 321 },
renderer: undefined,
@@ -319,8 +325,11 @@ it.each(["termination", "verification"] as const)(
host,
});
expect(removeTemporaryRoot).toHaveBeenCalledWith("/fixture/root");
expect(cleanup.temporary_root).toEqual({ state: "cleaned", reason: null });
expect(removeTemporaryRoot).not.toHaveBeenCalled();
expect(cleanup.temporary_root).toEqual({
state: "unverified",
reason: `temporary root retained because owned process cleanup is failed: ${cleanupFailure.message}`,
});
expect(cleanup.owned_process_group).toMatchObject({
state: "failed",
reason: cleanupFailure.message,
@@ -357,6 +366,13 @@ it.each(["termination", "verification"] as const)(
},
});
}
failing = false;
await scope.retryCleanup();
expect(removeTemporaryRoot).toHaveBeenCalledExactlyOnceWith(
"/fixture/root",
);
await scope.close();
expect(removeTemporaryRoot).toHaveBeenCalledTimes(1);
},
);
+1
View File
@@ -242,6 +242,7 @@ export const createServer = (
android.close(),
apktool.close(),
jeb.close(),
adbDevice.close(),
evmInterface.close(),
webSourceLocation.close(),
javascriptRecovery.close?.() ?? Promise.resolve(),
@@ -0,0 +1,47 @@
import { stat, truncate, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { snapshotAndroidTarget } from "../../../src/android/AndroidTargetSnapshot.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
it.skipIf(process.platform === "win32")(
"cancels an in-flight APK snapshot and removes its partial output",
async () => {
const root = await createTestTempDirectory("rea-android-snapshot-cancel-");
const source = join(root, "large.apk");
const snapshot = join(root, "target.apk");
await writeFile(source, "");
// A sparse extent keeps fixture setup cheap while ensuring the copy cannot
// finish before the test observes its first output bytes and aborts it.
await truncate(source, 64 * 1024 * 1024 * 1024);
const controller = new AbortController();
const pending = snapshotAndroidTarget(
source,
"0".repeat(64),
root,
"inspect_android_package",
controller.signal,
);
try {
await expect
.poll(
() =>
stat(snapshot).then(
({ size }) => size > 0,
() => false,
),
{ timeout: 10_000, interval: 10 },
)
.toBe(true);
controller.abort();
await expect(pending).rejects.toMatchObject({ name: "AbortError" });
await expect(stat(snapshot)).rejects.toMatchObject({ code: "ENOENT" });
} finally {
controller.abort();
await pending.catch(() => undefined);
}
},
);
@@ -5,7 +5,8 @@ import { expect, it as test } from "vitest";
import {
inspectJadxAvailability,
resolveJadxConfiguration,
resolveJadxConfigurationInputs,
validateJadxConfiguration,
} from "../../../src/android/JadxConfiguration.js";
import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
@@ -14,6 +15,18 @@ import { writeJadxJarInventory } from "../../fixtures/android/jadxJar.js";
const it = test.skipIf(process.platform === "win32");
const resolveConfiguration = async (
environment: Parameters<typeof resolveJadxConfigurationInputs>[0],
operation: Parameters<typeof resolveJadxConfigurationInputs>[1],
) => {
const configuration = await resolveJadxConfigurationInputs(
environment,
operation,
);
await validateJadxConfiguration(configuration, environment, operation);
return configuration;
};
it("rejects malformed or unrelated engine archives with JAR-specific recovery", async () => {
const root = await createTestTempDirectory("rea-jadx-invalid-jar-");
const javaHome = await createFakeJdk(root);
@@ -29,7 +42,7 @@ it("rejects malformed or unrelated engine archives with JAR-specific recovery",
diagnostics: { jar_path: await realpath(jar), phase: "jar-inspection" },
});
await expect(
resolveJadxConfiguration(environment, "inspect_android_package"),
resolveConfiguration(environment, "inspect_android_package"),
).rejects.toMatchObject({
_tag: "AnalysisCapabilityUnavailableError",
reason: expect.stringContaining("REA_JADX_MCP_JAR"),
@@ -45,7 +58,7 @@ it("rejects an explicit JAVA_HOME without an executable java binary", async () =
await writeJadxJarInventory(jar);
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{ REA_JADX_MCP_JAR: jar, JAVA_HOME: javaHome },
"inspect_android_package",
),
@@ -62,7 +75,7 @@ it("accepts a full JDK and canonicalizes the selected JAR", async () => {
await writeJadxJarInventory(jar);
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{ REA_JADX_MCP_JAR: jar, JAVA_HOME: javaHome },
"inspect_android_package",
),
@@ -78,7 +91,7 @@ it("configures heap and visible processors independently without forcing either"
const jar = join(root, "engine.jar");
const javaHome = await createFakeJdk(root);
await writeJadxJarInventory(jar);
const configuration = await resolveJadxConfiguration(
const configuration = await resolveConfiguration(
{
REA_JADX_MCP_JAR: jar,
JAVA_HOME: javaHome,
@@ -92,7 +105,7 @@ it("configures heap and visible processors independently without forcing either"
"-XX:ActiveProcessorCount=4",
]);
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{
REA_JADX_MCP_JAR: jar,
JAVA_HOME: javaHome,
@@ -107,7 +120,7 @@ it("configures heap and visible processors independently without forcing either"
for (const name of ["REA_JADX_HEAP_MIB", "REA_JADX_ACTIVE_PROCESSOR_COUNT"])
for (const value of ["", "0", "-1", "1.5", "8g", " 4", "9007199254740992"])
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{
REA_JADX_MCP_JAR: jar,
JAVA_HOME: javaHome,
@@ -127,7 +140,7 @@ it("distinguishes a runnable JRE from a missing Java executable", async () => {
await writeJadxJarInventory(jar);
const jreHome = await createFakeJdk(root, false);
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{ REA_JADX_MCP_JAR: jar, JAVA_HOME: jreHome },
"inspect_android_package",
),
@@ -175,7 +188,7 @@ it("preserves exact Java probe output and marks max-buffer capture as truncated"
);
await chmod(java, 0o700);
const failure = await resolveJadxConfiguration(
const failure = await resolveConfiguration(
{ REA_JADX_MCP_JAR: jar, JAVA_HOME: javaHome },
"inspect_android_package",
).then(
@@ -241,7 +254,7 @@ it("distinguishes an unreported Java version from an unsupported old version", a
);
await chmod(join(oldHome, "bin", "java"), 0o700);
await expect(
resolveJadxConfiguration(
resolveConfiguration(
{ REA_JADX_MCP_JAR: jar, JAVA_HOME: oldHome },
"inspect_android_package",
),
@@ -294,6 +294,15 @@ describe("CdpBrowserProvider: WebMCP discovery and completeness", () => {
],
},
});
expect(result.value.completeness.excluded).toContainEqual(
expect.objectContaining({
section: "webmcp_tools",
reason: "out_of_target_scope",
}),
);
expect(result.value.completeness.unavailable_sections).not.toContain(
"webmcp_tools",
);
const serialized = JSON.stringify(result.value);
expect(serialized).toContain("tool-secret");
expect(serialized).toContain("tool-source-secret");
@@ -354,7 +363,9 @@ describe("CdpBrowserProvider: WebMCP discovery and completeness", () => {
"WebMCP.invokeTool",
);
});
});
describe("CdpBrowserProvider: WebMCP replay inventory", () => {
it("returns every WebMCP registration in the replay", async () => {
const browser = await startFakeCdpBrowser({
webMcpTools: true,
@@ -410,7 +421,98 @@ describe("CdpBrowserProvider: WebMCP discovery and completeness", () => {
});
});
describe("CdpBrowserProvider WebMCP inventory completeness", () => {
describe("CdpBrowserProvider WebMCP malformed replay completeness", () => {
it.each(["toolsAdded", "toolsRemoved"] as const)(
"keeps malformed %s replay entries from proving WebMCP equality",
async (malformedEvent) => {
let malformed = false;
const registration = {
name: "read_item",
frameId: "frame-main",
inputSchema: { type: "object" },
};
const browser = await startFakeCdpBrowser({
commandEvents: (command) => {
if (command.method !== "WebMCP.enable") return undefined;
const events: {
readonly method: string;
readonly params: unknown;
readonly sessionId?: string;
}[] = [
{
method: "WebMCP.toolsAdded",
...(command.sessionId === undefined
? {}
: { sessionId: command.sessionId }),
params: {
tools:
malformed && malformedEvent === "toolsAdded"
? [registration, null]
: [registration],
},
},
];
if (malformed && malformedEvent === "toolsRemoved")
events.push({
method: "WebMCP.toolsRemoved",
...(command.sessionId === undefined
? {}
: { sessionId: command.sessionId }),
params: { tools: [{ name: registration.name }, null] },
});
return events;
},
});
trackBrowser(browser);
const provider = new CdpBrowserProvider();
const input = {
cdp_endpoint: browser.endpoint,
allowed_origins: [browser.allowedOrigin],
target_id: "allowed-page",
observation_ms: 0,
};
const inspection = await provider.inspectPage(
inspectWebPageInputSchema.parse(input),
);
if (!inspection.ok) throw inspection.error;
const discover = async () => {
const result = await provider.discoverWebMcpTools(
discoverWebMcpToolsInputSchema.parse(input),
);
if (!result.ok) throw result.error;
return result.value;
};
const before = await discover();
expect(before.tools.items).toHaveLength(1);
expect(before.completeness.unavailable_sections).not.toContain(
"webmcp_tools",
);
malformed = true;
const after = await discover();
expect(after.tools.items).toHaveLength(1);
expect(after.completeness.unavailable_sections).toContain("webmcp_tools");
expect(after.completeness.excluded).toContainEqual(
expect.objectContaining({
section: "webmcp_tools",
reason: "invalid_protocol_value",
}),
);
const comparison = compareWebCaptures(
compareWebCapturesInputSchema.parse({
before: { inspection: inspection.value, webmcp: before },
after: { inspection: inspection.value, webmcp: after },
}),
);
expect(comparison.dimensions.webmcp).toMatchObject({
status: "unknown",
total_changes: 0,
});
},
);
});
describe("CdpBrowserProvider WebMCP schema comparison", () => {
it("compares full untrusted declarations independently of object-key order", async () => {
const declaration: JsonValue = {
type: "object",
@@ -47,6 +47,7 @@ it.each(["Page.disable", "Target.detachFromTarget"])(
expect(methods).toContain("Target.detachFromTarget");
expect(methods).not.toContain("Target.closeTarget");
expect(methods).not.toContain("Browser.close");
await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0));
},
);
@@ -73,9 +74,11 @@ it("releases cleanup when cancellation arrives after capture completes", async (
controller.abort();
const result = await pending;
expect(result.ok).toBe(true);
expect(browser.commands.map(({ method }) => method)).toContain(
"Target.detachFromTarget",
);
await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0));
const methods = browser.commands.map(({ method }) => method);
expect(methods).not.toContain("Target.closeTarget");
expect(methods).not.toContain("Browser.close");
expect((await fetch(`${browser.endpoint}/json/version`)).ok).toBe(true);
});
it("closes an Electron inspection session when cleanup is cancelled", async () => {
@@ -122,11 +125,10 @@ it("closes an Electron inspection session when cleanup is cancelled", async () =
operation: "inspect_electron_page",
},
});
expect(browser.commands.map(({ method }) => method)).toContain(
"Page.disable",
);
const methods = browser.commands.map(({ method }) => method);
expect(methods).toContain("Target.detachFromTarget");
expect(methods).not.toContain("Target.closeTarget");
expect(methods).not.toContain("Browser.close");
await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0));
expect((await fetch(`${browser.endpoint}/json/version`)).ok).toBe(true);
});
@@ -0,0 +1,241 @@
import { expect } from "vitest";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
import { writeFixtureFiles } from "../../support/javascriptApplicationFixture.js";
const runtimeSource = (
runtime: string,
name: string,
entry = 1,
dependency = 2,
) => `
globalThis.${runtime}.push([["main"], {
${String(entry)}: (module, exports, req) => {
req(${String(dependency)}); req.e("lazy"); fetch("/${name}");
localStorage.setItem("${name}", "value");
const { BrowserWindow } = require("electron"); new BrowserWindow({});
},
${String(dependency)}: (module) => { module.exports = "${name}"; }
}, (req) => req(${String(entry)})]);
globalThis.${runtime}.push([["lazy"], {9: () => {}}]);
`;
const appA = runtimeSource("webpackChunkAppA", "a");
const appB = runtimeSource("webpackChunkAppB", "b");
cliTest(
"keeps source-derived runtime, module and chunk tuples collision-free",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-bundler-runtime-tuple-");
const registrations = [
{
runtime: "webpackChunkA",
module: "x\0y",
chunk: "x\0lazy",
endpoint: "/a",
},
{
runtime: "webpackChunkA\0x",
module: "y",
chunk: "lazy",
endpoint: "/b",
},
];
const source = registrations
.map(
({ runtime, module, chunk, endpoint }) => `
globalThis[${JSON.stringify(runtime)}].push([["main"], {
[${JSON.stringify(module)}]: (m, e, req) => { fetch(${JSON.stringify(endpoint)}); req.e(${JSON.stringify(chunk)}); }
}, (req) => req(${JSON.stringify(module)})]);
globalThis[${JSON.stringify(runtime)}].push([[${JSON.stringify(chunk)}], {}]);
`,
)
.join("\n");
await writeFixtureFiles(root, { "bundle.js": source });
const output = await cli.run({
arguments: ["analyze-javascript-application", root, "--json"],
});
expect(output.exitCode).toBe(0);
const { graph } = javascriptApplicationAnalysisResultSchema.parse(
parseEvidence(output.json).normalized_result,
);
const nodes = new Map(graph.nodes.map((node) => [node.node_id, node]));
for (const { runtime, module, chunk, endpoint } of registrations) {
const entry = graph.edges.find(
(edge) =>
edge.properties.kind === "bundler-entry-module" &&
edge.properties.runtime === runtime,
);
expect(nodes.get(entry?.target_node_id ?? "")?.identity).toMatchObject({
namespace: `${runtime}:module`,
key: module,
});
const call = graph.edges.find(
(edge) =>
edge.relation === "calls" &&
nodes.get(edge.target_node_id)?.observations[0]?.properties.value ===
endpoint,
);
expect(call?.source_node_id).toBe(entry?.target_node_id);
const lazy = graph.edges.find(
(edge) =>
edge.source_node_id === entry?.target_node_id &&
edge.properties.specifier === `chunk:${chunk}`,
);
expect(
nodes.get(lazy?.target_node_id ?? "")?.observations[0]?.properties,
).toMatchObject({ runtime, chunk_keys: [chunk] });
}
},
);
for (const scenario of [
{
name: "colliding module IDs",
source: appA + appB,
runtimes: ["webpackChunkAppA", "webpackChunkAppB"],
keys: [1, 1],
},
{
name: "reversed registrations",
source: appB + appA,
runtimes: ["webpackChunkAppA", "webpackChunkAppB"],
keys: [1, 1],
},
{
name: "single runtime",
source: appA,
runtimes: ["webpackChunkAppA"],
keys: [1],
},
{
name: "disjoint module IDs",
source: appA + runtimeSource("webpackChunkAppB", "b", 3, 4),
runtimes: ["webpackChunkAppA", "webpackChunkAppB"],
keys: [1, 3],
},
{
name: "Webpack and Rspack collision",
source: appA + runtimeSource("rspackChunkAppB", "b"),
runtimes: ["webpackChunkAppA", "rspackChunkAppB"],
keys: [1, 1],
},
]) {
cliTest(
`keeps bundled relationships within their runtime for ${scenario.name}`,
async ({ cli }) => {
const root = await createTestTempDirectory("rea-bundler-runtime-scope-");
await writeFixtureFiles(root, { "bundle.js": scenario.source });
const output = await cli.run({
arguments: ["analyze-javascript-application", root, "--json"],
});
expect(output.exitCode).toBe(0);
const { graph } = javascriptApplicationAnalysisResultSchema.parse(
parseEvidence(output.json).normalized_result,
);
const nodes = new Map(graph.nodes.map((node) => [node.node_id, node]));
expect(graph.coverage.status).toBe("complete");
for (const [index, runtime] of scenario.runtimes.entries()) {
const name = index === 0 ? "a" : "b";
const key = String(scenario.keys[index]);
const module = graph.nodes.find(
(node) =>
node.identity.strategy === "artifact-local-key" &&
node.identity.namespace === `${runtime}:module` &&
node.identity.key === key,
);
expect(module).toBeDefined();
const entry = graph.edges.find(
(edge) =>
edge.properties.kind === "bundler-entry-module" &&
edge.properties.runtime === runtime,
);
expect(entry?.target_node_id).toBe(module?.node_id);
expect(entry?.properties.resolution_status).toBe("resolved");
const ownedEdges = graph.edges.filter(
(edge) => edge.source_node_id === module?.node_id,
);
expect(
ownedEdges.find(
(edge) =>
edge.relation === "calls" &&
nodes.get(edge.target_node_id)?.observations[0]?.properties
.value === `/${name}`,
),
).toBeDefined();
expect(
ownedEdges.find(
(edge) =>
edge.relation === "persists_to" &&
nodes.get(edge.target_node_id)?.observations[0]?.properties
.name === name,
),
).toBeDefined();
expect(
ownedEdges.find(
(edge) =>
edge.relation === "contains" &&
nodes.get(edge.target_node_id)?.kind === "browser-window",
),
).toBeDefined();
const dependency = ownedEdges.find(
(edge) =>
edge.properties.kind === "require" &&
edge.properties.specifier === String(Number(key) + 1),
);
expect(
nodes.get(dependency?.target_node_id ?? "")?.identity,
).toMatchObject({
namespace: `${runtime}:module`,
key: String(Number(key) + 1),
});
const lazy = ownedEdges.find(
(edge) => edge.properties.specifier === "chunk:lazy",
);
expect(
nodes.get(lazy?.target_node_id ?? "")?.observations[0]?.properties,
).toMatchObject({ runtime, chunk_keys: ["lazy"] });
}
},
);
}
cliTest(
"shares module IDs across chunks only within the same runtime",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-bundler-shared-runtime-");
await writeFixtureFiles(root, {
"bundle.js": `
globalThis.webpackChunkApp.push([["main"], {1: (module, exports, req) => { req(2); }} , (req) => req(1)]);
globalThis.webpackChunkApp.push([["other"], {2: (module) => { module.exports = "dependency"; }}]);
`,
});
const output = await cli.run({
arguments: ["analyze-javascript-application", root, "--json"],
});
expect(output.exitCode).toBe(0);
const { graph } = javascriptApplicationAnalysisResultSchema.parse(
parseEvidence(output.json).normalized_result,
);
const module = (key: string) =>
graph.nodes.find(
(node) =>
node.identity.strategy === "artifact-local-key" &&
node.identity.namespace === "webpackChunkApp:module" &&
node.identity.key === key,
);
expect(
graph.edges.find(
(edge) =>
edge.properties.kind === "require" &&
edge.properties.specifier === "2",
),
).toMatchObject({
source_node_id: module("1")?.node_id,
target_node_id: module("2")?.node_id,
});
},
);
+7 -12
View File
@@ -8,23 +8,18 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
cliTest(
"rejects malformed oversized CLI input and preserves Evidence decoder resource failures",
"rejects malformed oversized JSON through both CLI and Evidence readers",
async ({ cli, processes }) => {
const root = await createTestTempDirectory("rea-json-string-limit-");
const input = join(root, "oversized.json");
const file = await open(input, "wx");
try {
// Sparse zero bytes are valid UTF-8. The Evidence reader reaches the
// decoded-length limit; the streamed CLI rejects their invalid JSON syntax.
// Sparse zero bytes are valid UTF-8 but invalid JSON. Both readers
// stream the input and reject the syntax before assembling a value.
await file.truncate(constants.MAX_STRING_LENGTH + 1);
} finally {
await file.close();
}
const limits = {
input_path: input,
input_bytes: constants.MAX_STRING_LENGTH + 1,
max_string_code_units: constants.MAX_STRING_LENGTH,
};
// Run readers sequentially in owned child processes, releasing each large
// read buffer when its process exits instead of retaining it in Vitest.
const result = await cli.run({
@@ -48,11 +43,11 @@ cliTest(
input,
]);
expect(JSON.parse(evidence.stdout)).toMatchObject({
code: "resource_constraint",
code: "evidence_integrity_mismatch",
details: {
operation: "read_evidence_file",
resource: "memory",
reported_limits: limits,
operation: "read",
reason: "invalid-json",
path: input,
},
});
expect(evidence.stderr).toBe("");
@@ -7,6 +7,7 @@ import { projectAnalysisError } from "../../../src/domain/analysisErrorProjectio
import { evmInterfaceSchema } from "../../../src/domain/evm/evmInterface.js";
import { EvmoleInterfaceProvider } from "../../../src/evm/EvmoleInterfaceProvider.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import { cleanupOwnedProcessGroup } from "../../../src/process/ProcessOwnership.js";
import { waitForProviderProcessReady } from "../../fixtures/providerProcess.js";
import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
import {
@@ -128,6 +129,7 @@ it.runIf(!unsupportedHost)(
async () => {
const { path } = await fixture();
let ownedPath = "";
let failCleanupReport = true;
const provider = new EvmoleInterfaceProvider({}, async (spawn) => {
ownedPath = spawn.cwd ?? "";
const launched = await spawnOwnedProviderProcess({
@@ -135,24 +137,48 @@ it.runIf(!unsupportedHost)(
command: process.execPath,
arguments: ["-e", 'process.stdout.write("x".repeat(1048577))'],
});
const ownership = launched.ownership;
if (ownership === undefined)
throw new Error("Owned launcher must expose process identity");
const cleanup =
launched.cleanup ?? (() => cleanupOwnedProcessGroup(ownership));
return {
...launched,
cleanup: async () => {
await launched.cleanup?.();
throw new Error("injected post-cleanup reporting failure");
const result = await cleanup();
if (failCleanupReport)
throw new Error("injected post-cleanup reporting failure");
return result;
},
};
});
onTestFinished(async () => {
failCleanupReport = false;
await provider.close();
});
const result = await provider.inspect({ path, encoding: "hex" });
if (result.ok) throw new Error("Expected compounded lifecycle failure");
expect(result.error).toMatchObject({
cleanupIncomplete: true,
diagnostics: {
reason: "injected post-cleanup reporting failure",
previous_error: { failure_kind: "output-limit" },
captured_output: { truncated: true, stderr: "" },
expect(projectAnalysisError(result.error)).toMatchObject({
code: "cleanup_incomplete",
details: {
diagnostics: {
reason: "Owned EVM worker cleanup could not be confirmed",
previous_error: {
code: "cleanup_incomplete",
details: {
diagnostics: {
reason: "injected post-cleanup reporting failure",
previous_error: { failure_kind: "output-limit" },
captured_output: { truncated: true, stderr: "" },
},
},
},
},
},
});
await access(ownedPath);
failCleanupReport = false;
await provider.close();
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
@@ -69,6 +69,35 @@ describe("artifact inventory entry order", () => {
},
);
it("resolves deep ZIP members without inventing missing directories", async () => {
const root = await createTestTempDirectory("rea-deep-archive-parent-");
const archive = join(root, "deep.zip");
// Fits the ZIP filename field while exceeding the formerly expensive depth.
const members = ["data.txt", "second.txt", "third.bin"].map(
(name) => `observed/${"d/".repeat(30_000)}${name}`,
);
await writeOrderedZip(archive, [...members, "observed/", "unrelated/"]);
const inventory = await inventoryArtifact(archive);
expect(artifactParentPaths(inventory)).toEqual({
".": null,
observed: ".",
unrelated: ".",
...Object.fromEntries(members.map((member) => [member, "observed"])),
});
const observedParent = artifactOccurrenceAt(inventory, "observed");
for (const member of members) {
const occurrence = artifactOccurrenceAt(inventory, member);
expect(occurrence.hash_status).toBe("verified");
expect(inventory.edges).toContainEqual(
expect.objectContaining({
occurrence_id: occurrence.occurrence_id,
parent_artifact_id: observedParent.artifact_id,
relation: "contains",
}),
);
}
});
it.each([
{
entries: ["pkg/sub/data.txt"],
@@ -129,6 +158,61 @@ describe("entry-order validation and reader controls", () => {
}
});
it("retains high-cardinality case-folded directories and their descendants", async () => {
const root = await createTestTempDirectory(
"rea-entry-order-case-capacity-",
);
const archive = join(root, "case-capacity.zip");
const variants = Array.from({ length: 256 }, (_, mask) =>
[..."abcdefgh"]
.map((character, index) =>
(mask & (1 << index)) === 0 ? character : character.toUpperCase(),
)
.join(""),
);
await writeOrderedZip(
archive,
variants.flatMap((variant) => [variant + "/", variant + "/entry.txt"]),
);
const inventory = await inventoryArtifact(archive);
const directories = variants.map((variant) =>
artifactOccurrenceAt(inventory, variant),
);
const descendants = variants.map((variant) =>
artifactOccurrenceAt(inventory, `${variant}/entry.txt`),
);
const ordered = [...variants].sort();
const first = ordered[0];
const second = ordered[1];
expect(inventory.occurrences).toHaveLength(1 + variants.length * 2);
expect(
new Set(inventory.occurrences.map(({ logical_path }) => logical_path))
.size,
).toBe(1 + variants.length * 2);
expect(
new Set(
[...directories, ...descendants].map(
({ occurrence_id }) => occurrence_id,
),
).size,
).toBe(variants.length * 2);
expect(
directories.every(({ limitations }) => limitations.length === 1),
).toBe(true);
expect(
descendants.every(({ limitations }) => limitations.length === 1),
).toBe(true);
expect(
directories.find(({ logical_path }) => logical_path === first)
?.limitations[0],
).toContain(`with ${second} and 254 other spellings;`);
expect(descendants[0]?.limitations[0]).toContain(
`is under ${variants[0]}, which collides under English (en-US) Unicode case folding with ${first} and 254 other spellings;`,
);
});
it("preserves ordinary directory and ASAR structure", async () => {
const root = await createTestTempDirectory("rea-entry-order-readers-");
const directory = join(root, "source");
@@ -7,7 +7,7 @@ import { describe, expect, it, onTestFinished } from "vitest";
import { createCli } from "../../../src/cli.js";
import { parseCliJsonInput } from "../../../src/cliJsonInput.js";
import { readCliJsonFile } from "../../../src/cliJsonFile.js";
import { readJsonInputFile } from "../../../src/application/JsonInputFile.js";
import { analysisCliErrorEnvelopeSchema } from "../../../src/contracts/errorSchemas.js";
import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../../../src/application/Utf8JsonInput.js";
import { readWithoutFifoWriter } from "../../fixtures/fifoInput.js";
@@ -203,11 +203,11 @@ describe("CLI JSON streamed input", () => {
const path = join(root, "input.json");
await writeFile(path, `${" ".repeat(9 * 1024 * 1024)}{"after":true}`);
const controller = new AbortController();
const cancelled = readCliJsonFile(path, "test-input", controller.signal);
const cancelled = readJsonInputFile(path, "test-input", controller.signal);
setImmediate(() => controller.abort(new Error("cancel JSON file parse")));
await expect(cancelled).rejects.toThrow("cancel JSON file parse");
await writeFile(path, '{"after":true}');
expect(await readCliJsonFile(path, "test-input")).toEqual({
expect(await readJsonInputFile(path, "test-input")).toEqual({
ok: true,
value: { after: true },
});
@@ -17,6 +17,7 @@ import { parse as parseJsonc, type ParseError } from "jsonc-parser";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
import { supportedClients } from "../../../src/application/SupportedClients.js";
import { PRODUCT_IDENTITY } from "../../../src/identity.js";
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
@@ -31,6 +32,7 @@ import {
} from "../../../src/application/Uninstall.js";
const roots: string[] = [];
const ownedSkillManifest = `---\nname: ${PRODUCT_IDENTITY.skillName}\n---\n# REA\n`;
beforeEach(clearClientLocationEnvironment);
afterEach(async () => {
vi.unstubAllEnvs();
@@ -1353,7 +1355,7 @@ describe("client configuration filesystem removal", () => {
},
}),
);
await writeFile(skill, "managed");
await writeFile(skill, ownedSkillManifest);
await mkdir(join(home, ".rea"), { recursive: true });
await import("node:fs/promises").then(({ symlink }) =>
symlink(home, join(home, ".rea/cache")),
@@ -1422,7 +1424,9 @@ describe("client configuration filesystem removal", () => {
readFile(`${configPath}.rea.backup`, "utf8"),
).rejects.toThrow();
});
});
describe("uninstall configuration preflight", () => {
it("fails closed on malformed client configuration", async () => {
const home = await createTestTempDirectory("rea-uninstall-bad-");
roots.push(home);
@@ -1447,8 +1451,12 @@ describe("client configuration filesystem removal", () => {
roots.push(home);
const codex = join(home, ".codex/config.toml");
const cursor = join(home, ".cursor/mcp.json");
const skillMarker = join(
home,
".agents/skills/reverse-engineer-anything/marker.txt",
);
const managed = [
join(home, ".agents/skills/reverse-engineer-anything/marker.txt"),
skillMarker,
join(home, ".rea/cache/marker.txt"),
join(home, ".rea/state/marker.txt"),
];
@@ -1463,6 +1471,8 @@ describe("client configuration filesystem removal", () => {
await writeFile(codex, "[mcp_servers.rea]\ncommand = [broken\n");
await writeFile(cursor, cursorConfig);
for (const path of managed) await writeFile(path, "marker");
const manifest = join(dirname(skillMarker), "SKILL.md");
await writeFile(manifest, ownedSkillManifest);
const stopped = await runUninstall(true, systemUninstallHost(home));
@@ -1478,6 +1488,7 @@ describe("client configuration filesystem removal", () => {
expect(await readFile(cursor, "utf8")).toBe(cursorConfig);
for (const path of managed)
expect(await readFile(path, "utf8")).toBe("marker");
expect(await readFile(manifest, "utf8")).toBe(ownedSkillManifest);
await writeFile(codex, "");
const repaired = await runUninstall(true, systemUninstallHost(home));
@@ -1485,7 +1496,12 @@ describe("client configuration filesystem removal", () => {
expect(JSON.parse(await readFile(cursor, "utf8"))).toEqual({
mcpServers: { other: { command: "echo", args: ["hello"] } },
});
for (const path of managed)
// Uninstall removes only the managed bundle, never arbitrary skill files.
expect(await readFile(skillMarker, "utf8")).toBe("marker");
await expect(readFile(manifest, "utf8")).rejects.toMatchObject({
code: "ENOENT",
});
for (const path of managed.slice(1))
await expect(readFile(path, "utf8")).rejects.toThrow();
});
});
@@ -1625,6 +1641,7 @@ describe("client configuration filesystem failure reporting", () => {
roots.push(home);
const skillRoot = join(home, ".agents/skills/reverse-engineer-anything");
await mkdir(skillRoot, { recursive: true });
await writeFile(join(skillRoot, "SKILL.md"), ownedSkillManifest);
const result = await runUninstall(
false,
systemUninstallHost(home, {
@@ -1637,11 +1654,14 @@ describe("client configuration filesystem failure reporting", () => {
name: "skill",
status: "failed",
detail: expect.stringContaining(
"This item could not be removed. Check file permissions, then rerun uninstall.",
`Managed file could not be removed: ${join(skillRoot, "SKILL.md")}`,
),
}),
);
expect(JSON.stringify(result)).not.toContain("SECRET");
expect(await readFile(join(skillRoot, "SKILL.md"), "utf8")).toBe(
ownedSkillManifest,
);
});
it("uninstalls all managed skill roots, including a custom Claude Code directory", async () => {
@@ -1653,7 +1673,11 @@ describe("client configuration filesystem failure reporting", () => {
join(customClaudeDirectory, "skills/reverse-engineer-anything"),
];
await Promise.all(
rootsToRemove.map((path) => mkdir(path, { recursive: true })),
rootsToRemove.map(async (path) => {
await mkdir(path, { recursive: true });
await writeFile(join(path, "SKILL.md"), ownedSkillManifest);
await writeFile(join(path, "user-notes.txt"), "user-authored");
}),
);
const result = await systemUninstallHost(home, testFileSystem, {
@@ -1663,7 +1687,12 @@ describe("client configuration filesystem failure reporting", () => {
expect(result).toMatchObject({ status: "removed" });
await Promise.all(
rootsToRemove.map(async (path) => {
await expect(lstat(path)).rejects.toMatchObject({ code: "ENOENT" });
await expect(
readFile(join(path, "SKILL.md"), "utf8"),
).rejects.toMatchObject({ code: "ENOENT" });
expect(await readFile(join(path, "user-notes.txt"), "utf8")).toBe(
"user-authored",
);
}),
);
});
@@ -157,6 +157,8 @@ describe("Qwen Code profile setup", () => {
expect(await setup.clientNeedsConfigure(client, {}, command)).toBe(false);
expect(await setup.skillNeedsInstall([client.name])).toBe(false);
expect(await setup.installSkill([client.name])).toBe("unchanged");
const userFile = join(skill, "user-note.txt");
await writeFile(userFile, "caller-owned skill notes\n");
expect(
(
await readClientRegistrationStatuses(
@@ -173,7 +175,12 @@ describe("Qwen Code profile setup", () => {
expect(registered).toEqual(client);
expect((await uninstall.removeClient(client)).status).toBe("removed");
expect((await uninstall.removeSkill()).status).toBe("removed");
await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
await expect(access(join(skill, "SKILL.md"))).rejects.toMatchObject({
code: "ENOENT",
});
expect(await readFile(userFile, "utf8")).toBe(
"caller-owned skill notes\n",
);
if (customHome)
await expect(access(join(home, ".qwen"))).rejects.toMatchObject({
code: "ENOENT",
@@ -84,6 +84,14 @@ it("reads a linked worktree's own branch and detached HEAD through shared Git re
head: mainHead,
dirty: null,
});
// A non-regular loose ref must remain unknown even when packed-refs has
// a readable value. The library's promise-API probe is not a source failure.
await mkdir(join(repository, ".git", "refs", "heads", "main"));
expect(await readReferenceSourceVcs(repository)).toEqual({
kind: "unknown",
head: null,
dirty: null,
});
const controller = new AbortController();
controller.abort();
expect(await readReferenceSourceVcs(linked, controller.signal)).toEqual({
@@ -324,7 +324,7 @@ it("cancels a waiting request promptly while keeping later launches behind the a
await assertFirmwareCleanup(fixture.launches);
});
it("retains an uncertain workspace and lets the next operation create a new root", async () => {
it("retains an uncertain worker's workspace and blocks later launches", async () => {
const fixture = await firmwareFixture("cleanup-failure");
const first = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
@@ -335,18 +335,15 @@ it("retains an uncertain workspace and lets the next operation create a new root
});
const workspace = fixture.launches[0]?.cwd;
expect(workspace).toBeDefined();
if (workspace === undefined) return;
if (workspace === undefined) throw new Error("Expected retained workspace");
await access(workspace);
const acquiredLaunches = [...fixture.launches];
const next = await fixture.service.execute("inspect_firmware_regions", {
path: fixture.path,
});
expect(next).toMatchObject({ ok: false, error: { cleanupIncomplete: true } });
expect(fixture.launches).toHaveLength(2);
const nextWorkspace = fixture.launches[1]?.cwd;
expect(nextWorkspace).toBeDefined();
expect(nextWorkspace).not.toBe(workspace);
expect(fixture.launches).toEqual(acquiredLaunches);
await access(workspace);
if (nextWorkspace !== undefined) await access(nextWorkspace);
});
it("accepts another build on the verified release line and reports it", async () => {
@@ -567,9 +567,9 @@ it.skipIf(process.platform === "win32")(
},
);
it.skipIf(process.platform === "win32")(
"preserves cancellation through system-host baseline identity inspection",
async () => {
it.skipIf(process.platform === "win32").each(["baseline", "tokens"] as const)(
"preserves cancellation through system-host %s inspection",
async (kind) => {
const directory = await mkdtemp(
join(tmpdir(), "rea-process-baseline-abort-test-"),
);
@@ -579,8 +579,23 @@ it.skipIf(process.platform === "win32")(
});
const controller = new AbortController();
let helperRoot: string | undefined;
const inspection =
kind === "baseline"
? host.captureBaseline?.(controller.signal)
: host.runTokens?.(
[
{
pid: process.pid,
parentPid: process.ppid,
processGroupId: process.pid,
state: "S",
command: process.execPath,
},
],
controller.signal,
);
const baseline =
host.captureBaseline?.(controller.signal).then(
inspection?.then(
() => ({ state: "fulfilled" as const }),
(cause: unknown) => ({ state: "rejected" as const, cause }),
) ?? Promise.resolve({ state: "missing" as const });
+10 -8
View File
@@ -4,10 +4,8 @@ import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver
import { AndroidAnalysisService } from "../../../src/application/android/AndroidAnalysisService.js";
import { JadxProvider } from "../../../src/android/JadxProvider.js";
import { androidResultSchemas } from "../../../src/domain/android/androidAnalysis.js";
import {
hashAndroidFile,
snapshotAndroidEngine,
} from "../../../src/android/AndroidTargetSnapshot.js";
import { snapshotAndroidEngine } from "../../../src/android/AndroidTargetSnapshot.js";
import { hashStableFile } from "../../../src/filesystem/StableFileHash.js";
import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
import {
createJadxProtocolFixture as setup,
@@ -43,7 +41,7 @@ it("cancels Android admission while hashing the selected APK", async () => {
it("rejects a copied JAR that differs from its admitted session identity", async () => {
const { jar } = await setup();
const admitted = await hashAndroidFile(jar);
const admitted = (await hashStableFile(jar)).sha256;
await writeFile(jar, "engine changed between hashing and copying");
const root = await PrivateRuntimeRoot.create({
prefix: "rea-jadx-admission-",
@@ -93,8 +91,9 @@ it("reuses immutable engine state while keeping each Evidence observation separa
await verifyCleanup(launches);
});
it("bounds output per operation across a retained session rather than accumulating earlier captures", async () => {
const { service, apk, launches, provider } = await setup("large-manifest");
it("bounds output per operation without repeating prerequisite probes for a retained session", async () => {
const { service, apk, launches, provider, javaProbeCount } =
await setup("large-manifest");
const expectedManifest = "x".repeat(1024 * 1024);
for (let index = 0; index < 34; index += 1) {
const result = await service.execute("inspect_android_package", {
@@ -108,12 +107,14 @@ it("bounds output per operation across a retained session rather than accumulati
).toBe(expectedManifest);
}
expect(launches).toHaveLength(1);
expect(await javaProbeCount()).toBe(1);
await provider.close();
await verifyCleanup(launches);
});
it("retire sessions when APK bytes, engine bytes or JVM options change", async () => {
const { service, apk, jar, launches, provider, environment } = await setup();
const { service, apk, jar, launches, provider, environment, javaProbeCount } =
await setup();
const inspect = async () => {
const result = await service.execute("inspect_android_package", {
path: apk,
@@ -135,6 +136,7 @@ it("retire sessions when APK bytes, engine bytes or JVM options change", async (
environment._JAVA_OPTIONS = "-XX:+UseG1GC";
await inspect();
expect(launches).toHaveLength(4);
expect(await javaProbeCount()).toBe(4);
expect(launches[3]).toMatchObject({
javaToolOptions: "-Xmx8g",
legacyJavaOptions: "-XX:+UseG1GC",
@@ -59,7 +59,10 @@ const expectUnverifiedHostCleanup = (error: ProcessCaptureError): void => {
/^(?:[1-9][0-9]*=(?:environment_unavailable|environment_errno_(?:EACCES|EPERM)))(?:, [1-9][0-9]*=(?:environment_unavailable|environment_errno_(?:EACCES|EPERM)))*$/u,
);
expect(report?.terminal_renderer.state).toBe("cleaned");
expect(report?.temporary_root.state).toBe("cleaned");
expect(report?.temporary_root).toEqual({
state: "unverified",
reason: `temporary root retained because owned process cleanup is unverified: ${report?.owned_process_group.reason}`,
});
};
const captureObservations = (
@@ -275,7 +278,10 @@ itWithCaptureCapability(
const projected = projectAnalysisError(result.error);
expect(projected).toMatchObject({
code: "cleanup_incomplete",
details: { cleanup: "incomplete", resources: ["owned_process_group"] },
details: {
cleanup: "incomplete",
resources: ["owned_process_group", "temporary_root"],
},
});
expect(projected).not.toHaveProperty("stack");
expect(projected).not.toHaveProperty("cause");
@@ -169,7 +169,13 @@ itWithCaptureCapability.each(["cli", "mcp"] as const)(
await writeFile(join(root, "z.txt"), "unchanged");
const scenario = {
executable: process.execPath,
arguments: ["-e", "require('node:fs').writeFileSync('a.txt','added')"],
// Delete the known file and overflow the remaining directory capacity.
// Incomplete enumeration still cannot prove its absence, regardless of
// which new entry the filesystem returns first.
arguments: [
"-e",
"const fs=require('node:fs');fs.unlinkSync('z.txt');fs.writeFileSync('a.txt','added');fs.writeFileSync('b.txt','added');",
],
working_directory: root,
filesystem_observation_paths: [root],
limits: { files: 2 },
@@ -179,7 +185,9 @@ itWithCaptureCapability.each(["cli", "mcp"] as const)(
);
const capture = parseProcessCapture(evidence.normalized_result);
expect(capture.truncated).toBe(true);
expect(await readFile(join(root, "z.txt"), "utf8")).toBe("unchanged");
await expect(readFile(join(root, "z.txt"), "utf8")).rejects.toMatchObject({
code: "ENOENT",
});
expect(
capture.filesystem_effects.find(({ path }) => path === "root_0:z.txt")
?.status,
@@ -194,9 +194,14 @@ itWithCaptureCapability.each(["cli", "mcp"] as const)(
expect(details.filesystem_after.enumeration_reasons).toContain(
"files_limit",
);
expect(
result.files_after.find(({ path }) => path === "root_0:b.txt")?.sha256,
).toBeNull();
// The truncated subset follows filesystem enumeration order. The large
// file may be retained without a digest, or not enumerated at all.
const largeFile = result.files_after.find(
({ path }) => path === "root_0:b.txt",
);
if (largeFile !== undefined) expect(largeFile.sha256).toBeNull();
expect(details.filesystem_after.enumeration_truncated).toBe(true);
expect(result.files_after.length).toBeLessThanOrEqual(3);
expect(processSourceTruncated(result, "terminal_rendered")).toBe(true);
expect(compareProcessCaptures(result, result)).toMatchObject({
terminal: "unknown",
@@ -1,7 +1,7 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { describe, expect, it, onTestFinished } from "vitest";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
@@ -40,6 +40,7 @@ describe("artifact Mach-O slices", () => {
),
};
const reader = new MachOSliceArtifactReader(binary, {}, runner);
onTestFinished(() => reader.close());
const enumerate = async (): Promise<void> => {
for await (const _entry of reader.entries()) {
// Enumeration must reject out-of-bounds lipo metadata before yielding.
@@ -78,6 +79,7 @@ describe("artifact Mach-O slices", () => {
),
};
const reader = new MachOSliceArtifactReader(binary, {}, runner);
onTestFinished(() => reader.close());
const entries = [];
for await (const entry of reader.entries()) entries.push(entry);
expect(entries).toHaveLength(2);
@@ -105,7 +107,9 @@ describe("artifact Mach-O slices", () => {
Reflect.set(provenance[0], "tool", "forged");
expect(reader.provenance()[0]?.tool).toBe("lipo");
});
});
describe("artifact Mach-O slice integrity", () => {
it.each([
["CPU type", "cputype 16777223"],
["CPU subtype", "cpusubtype 2"],
@@ -139,6 +143,7 @@ describe("artifact Mach-O slices", () => {
{},
lipoRunner(`${details.join("\n")}\n`),
);
onTestFinished(() => reader.close());
await expect(async () => {
for await (const _entry of reader.entries()) {
// The structural comparison runs before the reader yields a slice.
@@ -161,6 +166,7 @@ describe("artifact Mach-O slices", () => {
`architecture arm64\n cputype 16777228\n cpusubtype 0\n offset 4096\n size ${thin.length}\n align 2^12 (4096)\n`,
),
);
onTestFinished(() => reader.close());
await expect(async () => {
for await (const _entry of reader.entries()) {
// Malformed structural facts must fail before any slice is yielded.
@@ -215,6 +221,7 @@ describe("artifact Mach-O arm64e variant slices", () => {
`architecture x86_64\n cputype CPU_TYPE_X86_64\n cpusubtype CPU_SUBTYPE_X86_64_ALL\n offset 4096\n size ${String(x86.length)}\n align 2^12 (4096)\n${lipo(arm.length)}`,
),
);
onTestFinished(() => reader.close());
const entries = [];
for await (const entry of reader.entries()) entries.push(entry);
return entries.map(({ path, byteOffset }) => [path, byteOffset]);
@@ -72,12 +72,16 @@ describe("static Electron application analysis", () => {
const result = await reconstructJavaScriptArtifact({ input_path: root });
const graph = parseJavaScriptApplicationGraph(result.graph);
const requestedMembers = graph.nodes.flatMap((node) =>
// Computed access keeps no known members; the unknown access is an
// explicit dynamic flag rather than a "*" sentinel inside members.
const accesses = graph.nodes.flatMap((node) =>
node.kind === "native-export"
? node.observations.flatMap(({ properties }) => {
const members = properties.requested_members;
return Array.isArray(members) ? members : [];
})
? node.observations.map(({ properties }) => ({
members: Array.isArray(properties.requested_members)
? properties.requested_members
: [],
dynamicMemberAccess: properties.dynamic_member_access ?? false,
}))
: [],
);
@@ -86,7 +90,7 @@ describe("static Electron application analysis", () => {
native_addon_bindings: 1,
resolved_native_addon_bindings: 1,
});
expect(requestedMembers).toEqual(["*"]);
expect(accesses).toEqual([{ members: [], dynamicMemberAccess: true }]);
});
it("maps Electron boundaries through bundler-renamed bindings", async () => {
@@ -186,6 +186,40 @@ it("matches renderer, frame, script bytes, and worker without claiming execution
);
});
it("keeps static load states unknown when one scoped capture omits scripts", async () => {
const fixture = await applicationFixture();
const staticEvidence = await analyzeFixture(fixture);
const result = reconcileInput({
static_layers: [{ role: "application", analysis: staticEvidence }],
runtime_observations: [
electronRuntimeEvidence(fixture, SOURCE, {
includeWorker: false,
targetId: "target-complete",
}),
electronRuntimeEvidence(fixture, SOURCE, {
includeWorker: false,
scriptsUnavailable: true,
targetId: "target-incomplete",
}),
],
});
expect(
result.runtime_captures.map(
({ scripts_complete_within_scope }) => scripts_complete_within_scope,
),
).toEqual(expect.arrayContaining([false, true]));
expect(result.summary.static_not_observed).toBe(0);
expect(result.static_load_states).toEqual(
expect.arrayContaining([
expect.objectContaining({
status: "unknown",
reason: "static-or-runtime-coverage-incomplete",
}),
]),
);
});
it("reports a captured digest disagreement instead of accepting a path match", async () => {
const fixture = await applicationFixture();
const staticEvidence = await analyzeFixture(fixture);
@@ -316,7 +350,7 @@ it("imports an operator-provided cache layer through an explicit file mapping",
const application = await applicationFixture();
const cache = await createTestTempDirectory("rea-runtime-cache-static-");
const runtimeCache = await createTestTempDirectory("rea-runtime-cache-live-");
const cacheSource = "export const cachedFeature = 'fixture';\n";
const cacheSource = SOURCE;
await mkdir(join(cache, "mapped"));
await Promise.all([
writeFile(join(cache, "mapped", "chunk.js"), cacheSource),
@@ -373,6 +407,16 @@ it("imports an operator-provided cache layer through an explicit file mapping",
status: "unknown",
reason: "layer-outside-runtime-scope",
});
const sharedFile = result.graph.nodes.find(
({ observations }) =>
observations.some(({ properties }) => properties.path === "app.js") &&
observations.some(
({ properties }) => properties.path === "mapped/chunk.js",
),
);
expect(
sharedFile?.observations.map(({ properties }) => properties.path),
).toEqual(expect.arrayContaining(["app.js", "mapped/chunk.js"]));
});
const applicationFixture = async (): Promise<string> => {
@@ -410,6 +454,7 @@ const electronRuntimeEvidence = (
readonly targetId?: string;
readonly sourceIncluded?: boolean;
readonly workersUnavailable?: boolean;
readonly scriptsUnavailable?: boolean;
} = {},
) => {
const scriptFile = options.scriptFile ?? "app.js";
@@ -445,15 +490,7 @@ const electronRuntimeEvidence = (
ended_at: "2026-07-15T00:00:00.100Z",
observation_ms: 100,
},
completeness: options.workersUnavailable
? {
...completeCapture(),
status: "attach_limited" as const,
conditions: ["attach_limited" as const],
attach_limited_sections: ["workers" as const],
unavailable_sections: ["workers" as const],
}
: completeCapture(),
completeness: captureCompleteness(options),
frames: [
{
frame_id: "frame-main",
@@ -522,3 +559,21 @@ const completeCapture = () => ({
total: 0,
},
});
const captureCompleteness = (options: {
readonly workersUnavailable?: boolean;
readonly scriptsUnavailable?: boolean;
}) => {
const unavailableSections = [
...(options.scriptsUnavailable ? (["scripts"] as const) : []),
...(options.workersUnavailable ? (["workers"] as const) : []),
];
if (unavailableSections.length === 0) return completeCapture();
return {
...completeCapture(),
status: "attach_limited" as const,
conditions: ["attach_limited" as const],
attach_limited_sections: unavailableSections,
unavailable_sections: unavailableSections,
};
};
+14 -2
View File
@@ -1,4 +1,11 @@
import { access, chmod, mkdir, rm, writeFile } from "node:fs/promises";
import {
access,
chmod,
mkdir,
readFile,
rm,
writeFile,
} from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { expect, onTestFinished } from "vitest";
@@ -27,10 +34,11 @@ export const createJadxProtocolFixture = async (
const javaHome = join(root, "jdk");
const javaBin = join(javaHome, "bin");
const java = join(javaBin, "java");
const javaProbeLog = join(root, "java-probes.log");
await mkdir(javaBin, { recursive: true });
await writeFile(
java,
`#!/usr/bin/env node\nif (process.argv[2] === "--list-modules") console.log("java.base@21.0.0\\njdk.compiler@21.0.0");\n`,
`#!/usr/bin/env node\nif (process.argv[2] === "--list-modules") { require("node:fs").appendFileSync(${JSON.stringify(javaProbeLog)}, "probe\\n"); console.log("java.base@21.0.0\\njdk.compiler@21.0.0"); }\n`,
);
await chmod(java, 0o755);
await writeOrderedZip(apk, ["AndroidManifest.xml", "classes.dex"]);
@@ -115,6 +123,10 @@ export const createJadxProtocolFixture = async (
jar,
environment,
launches,
javaProbeCount: async () => {
const contents = await readFile(javaProbeLog, "utf8");
return contents.split("\n").filter(Boolean).length;
},
};
};
+1
View File
@@ -143,6 +143,7 @@ export const startFakeCdpBrowser = async (
browserWebSocketUrl: `ws://127.0.0.1:${String(port)}/devtools/browser/fake`,
allowedOrigin: endpoint,
commands,
activeConnectionCount: () => sockets.size,
emitRawMessage(message) {
for (const socket of sockets) socket.send(message);
},
+2
View File
@@ -11,6 +11,8 @@ export interface FakeCdpBrowser {
readonly browserWebSocketUrl: string;
readonly allowedOrigin: string;
readonly commands: readonly FakeCdpCommand[];
/** Actual WebSocket connections still held by the wire fixture. */
activeConnectionCount(): number;
/** Deliver actual malformed wire data without bypassing the production transport parser. */
emitRawMessage(message: string): void;
emitEvent(event: {
+10 -1
View File
@@ -148,6 +148,7 @@ describe("existing REA integration maintenance", () => {
).resolves.toEqual({
clients: ["codex", "vscode"],
skill: false,
skillDestinations: [],
});
});
@@ -157,6 +158,7 @@ describe("existing REA integration maintenance", () => {
).resolves.toEqual({
clients: [],
skill: false,
skillDestinations: [],
});
await writeSkill("---\nname: another-skill\n---\nUser-authored content\n");
await expect(
@@ -164,6 +166,7 @@ describe("existing REA integration maintenance", () => {
).resolves.toEqual({
clients: [],
skill: false,
skillDestinations: [],
});
});
@@ -177,6 +180,12 @@ describe("existing REA integration maintenance", () => {
).resolves.toEqual({
clients: [],
skill: true,
skillDestinations: [
{
client: "shared",
path: join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName),
},
],
});
expect(await readFile(path, "utf8")).toBe(original);
});
@@ -224,7 +233,7 @@ describe("existing REA integration maintenance", () => {
]);
expect(result).toEqual({
status: "planned",
scope: { clients: ["codex"], skill: false },
scope: { clients: ["codex"], skill: false, skillDestinations: [] },
command: recorded[0]?.slice(0, -2),
plannedActions: [action],
});