Compare commits

..
Author SHA1 Message Date
DragoloneandGitHub 94072364c7 fix(sqlite): keep triggers whose declaration spells the table in another case (#2807)
* fix(sqlite): keep triggers whose declaration spells the table in another case

SQLite stores a trigger's target in sqlite_schema.tbl_name exactly as
written in CREATE TRIGGER and resolves it by ASCII case folding, while an
index stores the table's own name. The selected_table scope filtered
tbl_name with COLLATE BINARY, so a trigger declared ON users for table
Users was omitted while the schema still reported complete. Match
dependents with COLLATE NOCASE, and report each trigger's table as the
stored table or view name so the selected-table contract still holds.

* fix(sqlite): refuse an inexact table before the case-folded schema scan

With the NOCASE expansion, a selection such as users for table Users
expanded the stored table's declarations before readRows reported the
missing exact name, so large trigger definitions could surface as an
output-limit failure instead of the selection error. Check the exact
table name against PRAGMA table_list first and share the selection
failure with readRows.

* fix(sqlite): fold dependent names only for a row-selectable table

An exact view, virtual or possible shadow table passed the exact-name
guard, so the NOCASE scan could still expand a differently cased trigger
past the reply budget before readRows refused the selection. Fold case
only when the selection is an ordinary table that is not a possible
shadow table, and keep the exact filter otherwise so every refused
selection fails as it did before the case-folding change.

* fix(sqlite): resolve trigger tables without folding unrelated names

The stored-name lookup case-folded every PRAGMA table_list name up front,
so a selected-table inspection of a tiny table paid for large unrelated
identifiers. A selected-table scan only returns the selected table's
dependents, so report the selected name directly; other scopes use an
exact stored name when one matches and fold the stored names once, only
when a trigger spells its target differently.

* perf(sqlite): fold each table name once when matching shadow parents

The extracted shadow-parent helper folded the table name inside the find
callback, once per unresolved virtual table, and the selected table was
checked both before and during the schema scan. Fold the name once per
call as before, and reuse the selected table's result in the scan.
2026-10-13 22:37:47 +08:00
e103a89889 fix(artifacts): preserve NIB key and class name identity (#2109)
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 22:36:06 +08:00
edd9939879 fix(browser): give the DevToolsActivePort wait one deadline owner (#2265)
* Single owner for the Chrome DevToolsActivePort startup deadline

Cold Chrome startup tolerance was spelled three ways for one wait: the helper
fell back to 15,000 ms, the runtime fixture lifecycle passed a literal 60,000,
and the real-browser verifier kept a local 60,000 constant. A per-caller literal
quietly changes how long a lane tolerates a cold browser before it fails, and
the failure reported only elapsed_ms, so a reader could not see which ceiling
was hit.

Export BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS as the owner, use it for the helper
fallback and both call sites, and report startup_timeout_ms in
BrowserStartupError. A source guard keeps callers from restating the number.

* Pin the DevToolsActivePort deadline guard to the startup call argument

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 22:35:45 +08:00
1e38a038c6 fix(process): release handles and close worker on incomplete cleanup (#2321)
* fix(process): unreference handles on incomplete provider process stop

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

* fix(javascript): close worker and release handles on incomplete cleanup

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

* fix(javascript): defer worker stdin close until incomplete cleanup

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

* test(javascript): assert cleanup error instance on inspection uncertainty

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

---------

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 22:34:54 +08:00
morluto 6e25cc3c76 fix(tests): clear lint and typecheck errors on main
Split two over-long test functions into module-level helpers
(max-lines-per-function): IdaSessionClient transport cases, IDA provider
composition, and function comparison identity cases keep their names and
assertions.

Also fix three typecheck errors: default omitted
direct_invokes_exhaustive to null so the result schema stays JsonValue
compatible, parse the ghidra dossier before spreading it, and delete the
optional beforeCall hook instead of assigning undefined under
exactOptionalPropertyTypes.
2026-10-13 22:32:01 +08:00
17 changed files with 603 additions and 135 deletions
@@ -8,7 +8,10 @@ import { spawnOwnedProviderProcess } from "../../dist/process/ProviderProcess.js
import { createSystemProcessOwnershipHost } from "../../dist/process/ProcessOwnershipObservation.js";
import { liveProcesses } from "../../dist/process/ProcessOwnershipProcessTree.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { waitForBrowserDevtoolsPort } from "../../dist/browser/BrowserProcessStartup.js";
import {
BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
waitForBrowserDevtoolsPort,
} from "../../dist/browser/BrowserProcessStartup.js";
/** Launch the synthetic fixture browser and await CDP in its own verified process group. */
export async function startRuntimeFixtureBrowser(executable, origin) {
@@ -50,7 +53,7 @@ export async function startRuntimeFixtureBrowser(executable, origin) {
stderr: () => stderr,
// Match the broader real-browser verifier's allowance for cold Chrome
// startup on shared runners; early process failures still reject at once.
timeoutMs: 60_000,
timeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
});
return {
port,
+5 -3
View File
@@ -9,7 +9,10 @@ import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { CdpBrowserProvider } from "../dist/browser/CdpBrowserProvider.js";
import { waitForBrowserDevtoolsPort } from "../dist/browser/BrowserProcessStartup.js";
import {
BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
waitForBrowserDevtoolsPort,
} from "../dist/browser/BrowserProcessStartup.js";
import { createBrowserScenarioProvider } from "../dist/composition/browserScenario.js";
import {
inspectWebPageInputSchema,
@@ -47,7 +50,6 @@ import { artifactCliEvidence, artifactMcpResult } from "./lib/artifact-e2e.mjs";
import { verifyScenarioEnvironment } from "./lib/browser-scenario-environment-e2e.mjs";
import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs";
const REAL_BROWSER_STARTUP_TIMEOUT_MS = 60_000;
const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret";
const SCENARIO_URL_SECRET_VALUE = "rea-browser-url-verifier-secret";
process.env.REA_BROWSER_VERIFIER_SECRET = SCENARIO_SECRET_VALUE;
@@ -92,7 +94,7 @@ try {
executable,
activePortPath: join(profile, "DevToolsActivePort"),
stderr: () => stderr,
timeoutMs: REAL_BROWSER_STARTUP_TIMEOUT_MS,
timeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
});
const endpoint = `http://127.0.0.1:${String(port)}`;
const provider = new CdpBrowserProvider();
+16
View File
@@ -36,6 +36,22 @@ const archiveWithOverlappingObjectRanges = (objectCount: number): Buffer => {
};
describe("NIBArchive decoder", () => {
it("keeps leading U+FEFF in length-prefixed keys and class names", () => {
const archive = encodeArchive({
classes: ["\uFEFFWidget"],
objects: [{ classIndex: 0, values: { count: 1, "\uFEFFcount": 2 } }],
});
expect(decodeNibArchive(archive).objects).toEqual([
{
id: 0,
class_name: "\uFEFFWidget",
values: { count: 1, "\uFEFFcount": 2 },
},
]);
const invalid = Buffer.from(archive);
invalid[invalid.indexOf(Buffer.from("\uFEFFcount"))] = 0xff;
expect(() => decodeNibArchive(invalid)).toThrow(/not valid UTF-8/u);
});
it("decodes bounded object, key, class, and reference tables", () => {
const archive = encodeArchive({
classes: ["NSView\0", "NSString\0"],
+23 -6
View File
@@ -14,6 +14,7 @@ export class BrowserStartupError extends Error {
readonly failure: BrowserStartupFailure;
readonly executable: string;
readonly elapsedMs: number;
readonly startupTimeoutMs: number;
readonly exitCode: number | null;
readonly signalCode: NodeJS.Signals | null;
readonly stderr: string;
@@ -23,6 +24,7 @@ export class BrowserStartupError extends Error {
readonly failure: BrowserStartupFailure;
readonly executable: string;
readonly elapsedMs: number;
readonly startupTimeoutMs: number;
readonly exitCode: number | null;
readonly signalCode: NodeJS.Signals | null;
readonly stderr: string;
@@ -33,12 +35,22 @@ export class BrowserStartupError extends Error {
this.failure = diagnostics.failure;
this.executable = diagnostics.executable;
this.elapsedMs = diagnostics.elapsedMs;
this.startupTimeoutMs = diagnostics.startupTimeoutMs;
this.exitCode = diagnostics.exitCode;
this.signalCode = diagnostics.signalCode;
this.stderr = diagnostics.stderr;
}
}
/**
* Single owner of the wait for Chrome to publish its DevToolsActivePort.
*
* Callers that need a different budget pass `timeoutMs`; nobody else should
* restate this number, because a per-caller literal silently changes how long
* a cold Chrome launch is tolerated before the failure names its ceiling.
*/
export const BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS = 60_000;
/** Owned child process and startup policy for Chrome CDP discovery. */
export interface BrowserStartupOptions {
readonly child: ChildProcess;
@@ -53,23 +65,23 @@ export interface BrowserStartupOptions {
export const waitForBrowserDevtoolsPort = async (
options: BrowserStartupOptions,
): Promise<number> => {
const timeoutMs = options.timeoutMs ?? 15_000;
const timeoutMs = options.timeoutMs ?? BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS;
const pollIntervalMs = options.pollIntervalMs ?? 25;
const startedAt = performance.now();
const terminal = childTermination(options, startedAt);
const terminal = childTermination(options, startedAt, timeoutMs);
const deadline = startedAt + timeoutMs;
while (true) {
const terminalState = currentTerminalState(options.child);
if (terminalState !== null)
throw startupError(options, startedAt, terminalState);
throw startupError(options, startedAt, timeoutMs, terminalState);
const port = await readDevtoolsPort(options.activePortPath);
if (port !== null) return port;
const remainingMs = deadline - performance.now();
if (remainingMs <= 0)
throw startupError(options, startedAt, { failure: "timeout" });
throw startupError(options, startedAt, timeoutMs, { failure: "timeout" });
const outcome = await Promise.race([
terminal,
@@ -88,11 +100,12 @@ type TerminalState =
const childTermination = (
options: BrowserStartupOptions,
startedAt: number,
startupTimeoutMs: number,
): Promise<BrowserStartupError> =>
new Promise((resolve) => {
options.child.once("error", (cause: unknown) => {
resolve(
startupError(options, startedAt, {
startupError(options, startedAt, startupTimeoutMs, {
failure: "spawn-error",
cause,
}),
@@ -103,6 +116,7 @@ const childTermination = (
startupError(
options,
startedAt,
startupTimeoutMs,
options.child.signalCode === null
? { failure: "exited" }
: { failure: "signalled" },
@@ -120,6 +134,7 @@ const currentTerminalState = (child: ChildProcess): TerminalState | null => {
const startupError = (
options: BrowserStartupOptions,
startedAt: number,
startupTimeoutMs: number,
state: TerminalState,
): BrowserStartupError =>
new BrowserStartupError(
@@ -127,6 +142,7 @@ const startupError = (
failure: state.failure,
executable: options.executable,
elapsedMs: Math.max(0, Math.round(performance.now() - startedAt)),
startupTimeoutMs,
exitCode: options.child.exitCode,
signalCode: options.child.signalCode,
stderr: options.stderr(),
@@ -156,9 +172,10 @@ const browserStartupMessage = (
diagnostics: {
readonly executable: string;
readonly elapsedMs: number;
readonly startupTimeoutMs: number;
readonly exitCode: number | null;
readonly signalCode: NodeJS.Signals | null;
readonly stderr: string;
},
): string =>
`Chrome CDP startup ${failure}; executable=${diagnostics.executable}; elapsed_ms=${String(diagnostics.elapsedMs)}; exit_code=${diagnostics.exitCode === null ? "null" : String(diagnostics.exitCode)}; signal=${diagnostics.signalCode ?? "null"}; stderr=${diagnostics.stderr.length === 0 ? "<empty>" : diagnostics.stderr}`;
`Chrome CDP startup ${failure}; executable=${diagnostics.executable}; elapsed_ms=${String(diagnostics.elapsedMs)}; startup_timeout_ms=${String(diagnostics.startupTimeoutMs)}; exit_code=${diagnostics.exitCode === null ? "null" : String(diagnostics.exitCode)}; signal=${diagnostics.signalCode ?? "null"}; stderr=${diagnostics.stderr.length === 0 ? "<empty>" : diagnostics.stderr}`;
+1 -1
View File
@@ -795,7 +795,7 @@ it("rejects unsupported parent operations and incompatible collections", () => {
it("preserves selected native facet unavailability instead of an exhausted empty page", () => {
const parent = nativeParent({
...ghidraFunctionDossier(),
...functionDossierSchema.parse(ghidraFunctionDossier()),
facet_availability: {
comments: { available: false, reason: "Typed comments unavailable" },
references: { available: false, reason: "Typed references unavailable" },
+51 -41
View File
@@ -128,6 +128,49 @@ const observe = (
},
);
const rejectsMalformedFacetAvailability = (): void => {
expect(
functionDossierSchema.safeParse({
...dossier("return 0;", "0x1000"),
facet_availability: { cfg: { available: false, reason: "" } },
}).success,
).toBe(false);
expect(
functionDossierSchema.safeParse({
...dossier("return 0;", "0x1000"),
facet_availability: { cfg: undefined },
}).success,
).toBe(false);
};
const keepsGeneratedNamesAmbiguous = (): void => {
const named = (base: "0x1000" | "0x2000", name: string) =>
functionDossierSchema.parse({
...dossier("return 0;", base),
procedure: {
...procedure(base, name),
signature: "int helper(void)",
locals: [],
},
});
const generated = compareFunctions(
observe("1", named("0x1000", "sub_1000")),
observe("2", named("0x2000", "sub_2000")),
);
expect(generated.function_match.status).toBe("ambiguous");
expect(generated.status).toBe("unknown");
expect(
generated.dimensions.find(({ dimension }) => dimension === "identity"),
).toMatchObject({ status: "unknown" });
const stable = compareFunctions(
observe("3", named("0x1000", "firstFunction")),
observe("4", named("0x2000", "secondFunction")),
);
expect(stable.function_match.status).toBe("mismatched");
expect(stable.status).toBe("changed");
};
describe("function comparison normalized identity", () => {
it("retains unresolved call details alongside missing facet coverage", () => {
const value: Record<string, unknown> = {
@@ -187,47 +230,14 @@ describe("function comparison normalized identity", () => {
),
).toMatchObject({ status: "unknown", left_count: null, right_count: null });
});
it("rejects malformed facet availability instead of treating it as absence", () => {
expect(
functionDossierSchema.safeParse({
...dossier("return 0;", "0x1000"),
facet_availability: { cfg: { available: false, reason: "" } },
}).success,
).toBe(false);
expect(
functionDossierSchema.safeParse({
...dossier("return 0;", "0x1000"),
facet_availability: { cfg: undefined },
}).success,
).toBe(false);
});
it("keeps differing generated names ambiguous and stable names mismatched", () => {
const named = (base: "0x1000" | "0x2000", name: string) =>
functionDossierSchema.parse({
...dossier("return 0;", base),
procedure: {
...procedure(base, name),
signature: "int helper(void)",
locals: [],
},
});
const generated = compareFunctions(
observe("1", named("0x1000", "sub_1000")),
observe("2", named("0x2000", "sub_2000")),
);
expect(generated.function_match.status).toBe("ambiguous");
expect(generated.status).toBe("unknown");
expect(
generated.dimensions.find(({ dimension }) => dimension === "identity"),
).toMatchObject({ status: "unknown" });
const stable = compareFunctions(
observe("3", named("0x1000", "firstFunction")),
observe("4", named("0x2000", "secondFunction")),
);
expect(stable.function_match.status).toBe("mismatched");
expect(stable.status).toBe("changed");
});
it(
"rejects malformed facet availability instead of treating it as absence",
rejectsMalformedFacetAvailability,
);
it(
"keeps differing generated names ambiguous and stable names mismatched",
keepsGeneratedNamesAmbiguous,
);
it.each(["sub_deallocate", "sub_deadbeef_handler", "fcn.dispatch"])(
"recognizes the explicit symbol %s without a generated-name prefix false positive",
+1 -1
View File
@@ -1459,7 +1459,7 @@ it.each(["idb_close", "idb_list", "connection_close"])(
});
expect(result.error.cleanupIncomplete).toBe(true);
} finally {
producer.beforeCall = undefined;
delete producer.beforeCall;
producer.close = originalClose;
await client.close();
}
@@ -83,6 +83,7 @@ export class JavaScriptAnalysisWorker {
readonly #resultBudgetBytes: number;
/** Null when the runtime enforces the V8 heap flags instead. */
readonly #processMemoryLimitBytes: number | null;
readonly #onStdinError = (): void => {};
readonly #onStdout = (chunk: Buffer | string): void => {
try {
this.#pending +=
@@ -784,20 +785,40 @@ export class JavaScriptAnalysisWorker {
if (this.#supervisor !== undefined) {
const supervisor = this.#supervisor;
const stopped = await supervisor.stop();
supervisor.launch.process.stdout?.off("data", this.#onStdout);
if (stopped.status === "incomplete") {
try {
const stdin = supervisor.launch.process.stdin;
if (
stdin !== undefined &&
stdin !== null &&
!stdin.destroyed &&
!stdin.writableEnded
) {
stdin.off("error", this.#onStdinError);
stdin.once("error", this.#onStdinError);
stdin.end();
}
} catch {
// Cooperative protocol stdin may already be closed or broken.
}
this.#resetState();
pendingCleanup.add(this);
return this.#cleanupError(stopped.reason, original);
}
supervisor.launch.process.stdout?.off("data", this.#onStdout);
supervisor.dispose();
this.#supervisor = undefined;
}
this.#resetState();
return undefined;
}
#resetState(): void {
this.#ready = false;
this.#frames = [];
this.#pending = "";
this.#decoder = new StringDecoder("utf8");
this.#protocolFailure = undefined;
return undefined;
}
}
+22
View File
@@ -61,6 +61,8 @@ export interface ProviderProcessHandle extends Pick<
/** True once an enforced process memory limit reported an allocation failure. */
readonly memoryLimitReached?: boolean;
kill(signal?: NodeJS.Signals | number): boolean;
unref?(): void;
ref?(): void;
}
/** Process handle returned by a provider-specific launcher. */
@@ -371,13 +373,33 @@ export class ProviderProcessSupervisor {
this.launch.process.off("error", this.#onError);
}
/** Release event-loop references so incomplete cleanup does not prevent host exit. */
unref(): void {
const process = this.launch.process;
process.unref?.();
(process.stdin as { unref?: () => void } | null | undefined)?.unref?.();
(process.stdout as { unref?: () => void } | null | undefined)?.unref?.();
(process.stderr as { unref?: () => void } | null | undefined)?.unref?.();
}
/** Restore event-loop references when retrying supervised process cleanup. */
ref(): void {
const process = this.launch.process;
process.ref?.();
(process.stdin as { ref?: () => void } | null | undefined)?.ref?.();
(process.stdout as { ref?: () => void } | null | undefined)?.ref?.();
(process.stderr as { ref?: () => void } | null | undefined)?.ref?.();
}
async #stop(
options: ProviderProcessStopOptions,
): Promise<ProviderProcessStopResult> {
this.ref();
let completed = false;
try {
const result = await this.#stopWithin(options);
completed = result.status !== "incomplete";
if (!completed) this.unref();
return result;
} finally {
if (completed) this.dispose();
+37
View File
@@ -121,3 +121,40 @@ it("reports a reached process memory limit only when the handle observed one", (
supervisor.dispose();
}
});
it("unreferences process and stream handles when stop reports incomplete cleanup", async () => {
const unrefProcess = vi.fn();
const unrefStdin = vi.fn();
const unrefStdout = vi.fn();
const unrefStderr = vi.fn();
const producer = Object.assign(new EventEmitter(), {
stdout: Object.assign(new PassThrough(), { unref: unrefStdout }),
stdin: Object.assign(new PassThrough(), { unref: unrefStdin }),
stderr: Object.assign(new PassThrough(), { unref: unrefStderr }),
exitCode: null,
signalCode: null,
kill: () => false,
unref: unrefProcess,
});
const supervisor = new ProviderProcessSupervisor({
process: producer,
ownsProcessLifetime: true,
cleanup: async () => ({
cleaned: false,
reason: "process table could not be inspected: spawn EPERM",
}),
});
try {
const result = await supervisor.stop();
expect(result).toMatchObject({
status: "incomplete",
reason: "process table could not be inspected: spawn EPERM",
});
expect(unrefProcess).toHaveBeenCalledTimes(1);
expect(unrefStdin).toHaveBeenCalledTimes(1);
expect(unrefStdout).toHaveBeenCalledTimes(1);
expect(unrefStderr).toHaveBeenCalledTimes(1);
} finally {
supervisor.dispose();
}
});
+50 -13
View File
@@ -109,6 +109,11 @@ class OutputBudget {
}
const quoteIdentifier = (name: string): string =>
`"${name.replaceAll('"', '""')}"`;
const missingSelectedTable = (name: string): SqliteInspectionFailure =>
new SqliteInspectionFailure(
"selection",
`Selected ordinary table does not exist with that exact name: ${name}`,
);
// SQLite folds only ASCII letters when matching virtual/shadow table names.
const sqliteIdentifierKey = (name: string): string =>
name.replace(/[A-Z]/g, (letter) => letter.toLowerCase());
@@ -236,12 +241,49 @@ const readSchema = (
metadata.set(value.name, value);
}
}
// A trigger stores its target as written in CREATE TRIGGER, which SQLite
// resolves by ASCII case folding; report the stored table or view name.
// A selected-table scan returns only the selected table's dependents, and
// other scopes fold stored names once, only when a spelling differs.
let foldedNames: Map<string, string> | undefined;
const triggerTable = (written: string): string => {
if (selectedName !== undefined) return selectedName;
if (metadata.has(written)) return written;
foldedNames ??= new Map(
[...metadata.keys()].map((name) => [sqliteIdentifierKey(name), name]),
);
return foldedNames.get(sqliteIdentifierKey(written)) ?? written;
};
const unresolvedVirtualTables = [...metadata.values()]
.filter((value) => value.type === "virtual" && value.ncol === 0)
.map((value) => ({
name: value.name,
prefix: sqliteIdentifierKey(`${value.name}_`),
}));
const shadowParentOf = (
name: string,
listed: z.output<typeof tableRow>,
): (typeof unresolvedVirtualTables)[number] | undefined => {
if (listed.type !== "table") return undefined;
const identifier = sqliteIdentifierKey(name);
return unresolvedVirtualTables.find((parent) =>
identifier.startsWith(parent.prefix),
);
};
const selected =
selectedName === undefined ? undefined : metadata.get(selectedName);
if (selectedName !== undefined && selected === undefined)
throw missingSelectedTable(selectedName);
const selectedShadowParent =
selectedName === undefined || selected === undefined
? undefined
: shadowParentOf(selectedName, selected);
// NOCASE folds only ASCII letters, as SQLite does when it resolves the
// table named by an index or trigger. Fold only for an ordinary table whose
// rows can be selected; a selection refused later keeps the exact filter,
// so its failure does not depend on declarations under another spelling.
const foldDependents =
selected?.type === "table" && selectedShadowParent === undefined;
const schema: SqliteDatabase["schema"] = {
scope,
completeness: "complete",
@@ -254,7 +296,9 @@ const readSchema = (
// duplicate large names or definitions beyond SQLite's native record limit.
const declaration = scope === "inventory" ? "" : ", CAST(sql AS BLOB) AS sql";
const selection =
scope === "selected_table" ? " WHERE tbl_name = ? COLLATE BINARY" : "";
scope === "selected_table"
? ` WHERE tbl_name = ? COLLATE ${foldDependents ? "NOCASE" : "BINARY"}`
: "";
const statement = db.prepare(
`SELECT type, CAST(name AS BLOB) AS name, CAST(tbl_name AS BLOB) AS tbl_name, rootpage${declaration} FROM main.sqlite_schema${selection}`,
);
@@ -280,13 +324,10 @@ const readSchema = (
"format",
`SQLite schema/table inventory mismatch: ${value.name}`,
);
const identifier = sqliteIdentifierKey(value.name);
const possibleShadowParent =
listed.type === "table"
? unresolvedVirtualTables.find((parent) =>
identifier.startsWith(parent.prefix),
)
: undefined;
value.name === selectedName
? selectedShadowParent
: shadowParentOf(value.name, listed);
const table: SqliteDatabase["schema"]["tables"][number] = {
name: value.name,
...(scope === "inventory" ? {} : { sql: value.sql }),
@@ -349,7 +390,7 @@ const readSchema = (
} else {
const trigger = {
name: value.name,
table: value.tbl_name,
table: triggerTable(value.tbl_name),
...(scope === "inventory" ? {} : { sql: value.sql }),
};
budget.retain(trigger);
@@ -370,11 +411,7 @@ const readRows = (
): SqliteDatabase["rows"] => {
const { encoding, budget } = context;
if (input.table === undefined) return null;
if (table === undefined)
throw new SqliteInspectionFailure(
"selection",
`Selected ordinary table does not exist with that exact name: ${input.table}`,
);
if (table === undefined) throw missingSelectedTable(input.table);
if (table.kind === "unknown")
throw new SqliteInspectionFailure(
"selection",
+14
View File
@@ -67,6 +67,7 @@ export class WindowsOwnedProcess extends EventEmitter {
#closed = false;
#failure: string | undefined;
#timer: NodeJS.Timeout | undefined;
#unreferenced = false;
#cleanup: Promise<ProcessCleanupResult> | undefined;
readonly #memoryLimitBytes: number | undefined;
#memoryLimitReached = false;
@@ -222,6 +223,18 @@ export class WindowsOwnedProcess extends EventEmitter {
return this.#cleanup;
}
/** Release event-loop references so unsettled jobs do not prevent host exit. */
unref(): void {
this.#unreferenced = true;
this.#timer?.unref();
}
/** Restore event-loop references when retrying process lifecycle observation. */
ref(): void {
this.#unreferenced = false;
this.#timer?.ref();
}
async #stop(): Promise<ProcessCleanupResult> {
if (this.#closed) return { cleaned: true, signaled: false };
let timer: NodeJS.Timeout | undefined;
@@ -255,6 +268,7 @@ export class WindowsOwnedProcess extends EventEmitter {
#schedule(): void {
if (this.#closed || this.#timer !== undefined) return;
this.#timer = setTimeout(() => this.#poll(), 25);
if (this.#unreferenced) this.#timer.unref();
}
#poll(): void {
@@ -1,3 +1,4 @@
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect } from "vitest";
@@ -9,6 +10,7 @@ import {
} from "../../support/process/processFixture.js";
import {
BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
BrowserStartupError,
waitForBrowserDevtoolsPort,
} from "../../../src/browser/BrowserProcessStartup.js";
@@ -86,15 +88,68 @@ describe("browser process startup", () => {
failure: "timeout",
exitCode: null,
signalCode: null,
startupTimeoutMs: 20,
});
expect(String(failure)).toContain("stderr=<empty>");
expect(String(failure)).toContain("startup_timeout_ms=20");
} finally {
child.kill("SIGKILL");
expect(await waitForExit(child, 5_000)).toBe(true);
}
});
it("names the owner's deadline when a caller omits one", async ({
processes,
}) => {
const root = await temporaryRoot();
const child = processes.spawn(process.execPath, [
"-e",
"setTimeout(() => process.exit(3), 10)",
]);
const failure = await waitForBrowserDevtoolsPort({
child,
executable: process.execPath,
activePortPath: join(root, "DevToolsActivePort"),
stderr: () => "owner default fixture",
pollIntervalMs: 5,
}).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(BrowserStartupError);
expect(failure).toMatchObject({
failure: "exited",
startupTimeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
});
expect(String(failure)).toContain(
`startup_timeout_ms=${String(BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS)}`,
);
});
it("keeps one owner for the DevToolsActivePort deadline", async () => {
expect(BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS).toBe(60_000);
const callers = [
"../../../scripts/verify-real-browser.mjs",
"../../../scripts/lib/browser-runtime-fixture-lifecycle.mjs",
];
for (const caller of callers) {
const source = await readFile(new URL(caller, import.meta.url), "utf8");
expect(devtoolsPortDeadlines(source)).toEqual([
["BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS"],
]);
}
});
});
const temporaryRoot = async (): Promise<string> => {
return createTestTempDirectory("rea-browser-startup-");
};
const devtoolsPortDeadlines = (source: string): string[][] => {
const calls = /waitForBrowserDevtoolsPort\(\{(?<body>[\s\S]*?)\}\)/gu;
const deadlines = /\btimeoutMs:\s*(?<value>[^\n,]+)/gu;
return [...source.matchAll(calls)].map((call) =>
[...String(call.groups?.body).matchAll(deadlines)].map((deadline) =>
String(deadline.groups?.value).trim(),
),
);
};
@@ -696,6 +696,49 @@ describe("NIB projection byte amplification", () => {
});
});
cliTest(
"compiled CLI preserves distinct NIB keys and class identity",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-nib-name-identity-");
const bundle = join(root, "Example.app");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
await mkdir(join(bundle, "Contents", "MacOS"), { recursive: true });
await writeFile(
join(bundle, "Contents", "MacOS", "App"),
thinMach(0xfeedfacf, 0x0100000c),
);
await writeFile(
join(bundle, "Contents", "Info.plist"),
"<plist><dict><key>CFBundleExecutable</key><string>App</string></dict></plist>",
);
await writeFile(
join(resources, "Identity.nib"),
encodeNibArchiveFixture({
classes: ["\uFEFFWidget"],
objects: [{ classIndex: 0, values: { count: 1, "\uFEFFcount": 2 } }],
}),
);
const output = await cli.run({
arguments: ["decode-interface-builder", bundle, "--json"],
environment: { REA_LOG_LEVEL: "silent", REA_ANALYSIS_PROVIDER: "auto" },
});
expect(output.exitCode, output.stderr).toBe(0);
const result = interfaceBuilderAnalysisSchema.parse(
parseEvidence(output.json).normalized_result,
);
expect(result.graph.nodes).toContainEqual(
expect.objectContaining({
attributes: expect.objectContaining({
class_name: "\uFEFFWidget",
nibValues: { count: 1, "\uFEFFcount": 2 },
}),
}),
);
expect(JSON.stringify(result)).toContain("\uFEFFWidget");
},
);
cliTest(
"compiled CLI preserves a good archive and reports repeated key and data omissions",
async ({ cli }) => {
@@ -1,7 +1,13 @@
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { JavaScriptAnalysisWorker } from "../../../src/javascript/analysis/JavaScriptAnalysisWorker.js";
import { cleanupOwnedProcessGroup } from "../../../src/process/ProcessOwnership.js";
import type { OwnedProcessGroup } from "../../../src/process/ProcessOwnership.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
@@ -151,6 +157,121 @@ it("preserves a failed owned cleanup retry when the next caller cancels before a
}
});
it("ends worker stdin and unreferences process handles when cleanup reports inspection uncertainty", async () => {
let stdinEnded = false;
let unreferenced = false;
let permitCleanup = false;
let childStdin: typeof process.stdin | undefined;
const worker = new JavaScriptAnalysisWorker(
{ heapMb: 128, timeoutMs: 30_000 },
async (request) => {
const launched = await spawnOwnedProviderProcess(request);
const originalUnref = launched.process.unref;
launched.process.unref = () => {
unreferenced = true;
originalUnref?.call(launched.process);
};
childStdin = launched.process.stdin as unknown as typeof process.stdin;
launched.process.stdin?.once("finish", () => {
stdinEnded = true;
});
return {
...launched,
cleanup: async () =>
permitCleanup
? launched.cleanup === undefined
? cleanupOwnedProcessGroup(launched.ownership)
: launched.cleanup()
: {
cleaned: false,
reason: "process table could not be inspected: spawn EPERM",
},
};
},
);
try {
const file = source("app.mjs", "export const observed = 7;");
const result = await worker.analyze(file, 100);
if (!result.ok) throw result.error.error;
const closing = worker.close();
await expect(closing).rejects.toBeInstanceOf(ProviderCleanupError);
await expect(closing).rejects.toMatchObject({
cause: {
message: "process table could not be inspected: spawn EPERM",
},
});
expect(childStdin?.writableEnded || stdinEnded).toBe(true);
expect(unreferenced).toBe(true);
const errorListeners = childStdin?.listenerCount("error");
for (let retry = 0; retry < 3; retry++)
await expect(worker.close()).rejects.toBeInstanceOf(ProviderCleanupError);
expect(childStdin?.listenerCount("error")).toBeLessThanOrEqual(
errorListeners ?? 0,
);
permitCleanup = true;
await worker.close();
} finally {
permitCleanup = true;
await worker.close().catch(() => {});
}
});
it("lets a separate host exit while its live worker cleanup remains incomplete", async () => {
const root = await createTestTempDirectory("rea-worker-incomplete-host-");
const ownershipPath = join(root, "ownership.json");
const workerUrl = new URL(
"../../../dist/javascript/analysis/JavaScriptAnalysisWorker.js",
import.meta.url,
).href;
const launcherUrl = new URL(
"../../../dist/process/ProviderProcess.js",
import.meta.url,
).href;
const script = `
import { writeFile } from "node:fs/promises";
import { pathToFileURL } from "node:url";
import { JavaScriptAnalysisWorker } from ${JSON.stringify(workerUrl)};
import { spawnOwnedProviderProcess } from ${JSON.stringify(launcherUrl)};
const worker = new JavaScriptAnalysisWorker({ heapMb: 128, timeoutMs: 10000 }, async request => {
const entry = pathToFileURL(request.arguments.at(-1)).href;
const launched = await spawnOwnedProviderProcess({ ...request, arguments: [
...request.arguments.slice(0, -1), "--input-type=module", "-e",
"const interval = setInterval(() => {}, 1000); setTimeout(() => clearInterval(interval), 60000); await import(" + JSON.stringify(entry) + ");"
] });
// The launcher is deliberately orphaned when this host exits; cleanup still
// requires its run token and executable identity, rather than its old parent.
const { expectedParentPid, ...ownership } = launched.ownership;
await writeFile(${JSON.stringify(ownershipPath)}, JSON.stringify(ownership));
return { ...launched, cleanup: async () => ({ cleaned: false, reason: "fixture inspection unavailable" }) };
});
const analyzed = await worker.analyze(${JSON.stringify(source("app.mjs", "export const value = 1;"))}, 100);
if (!analyzed.ok) throw analyzed.error.error;
try { await worker.close(); throw new Error("cleanup unexpectedly completed"); }
catch (error) {
if (error.cleanupIncomplete !== true) throw error;
process.stdout.write(JSON.stringify({ code: "cleanup_incomplete", reason: error.cause.message }) + "\\n");
}
`;
try {
const result = await promisify(execFile)(
process.execPath,
["--input-type=module", "-e", script],
{ timeout: 10_000 },
);
expect(JSON.parse(result.stdout)).toMatchObject({
code: "cleanup_incomplete",
reason: "fixture inspection unavailable",
});
} finally {
const ownership = JSON.parse(
await readFile(ownershipPath, "utf8"),
) as OwnedProcessGroup;
expect(await cleanupOwnedProcessGroup(ownership)).toMatchObject({
cleaned: true,
});
}
});
// The handshake mirrors the real server: only Node.js reports an enforced V8 heap limit.
const resourceFailureWorker = (heapLimit: string) =>
new JavaScriptAnalysisWorker({ heapMb: 128, timeoutMs: 30_000 }, (request) =>
@@ -130,6 +130,71 @@ it("accounts for JSON escape expansion of a stored declaration in the reply limi
]);
});
it("attributes triggers declared with another ASCII case to the stored table name", async () => {
const path = await createDatabase(`
CREATE TABLE Users(id INTEGER PRIMARY KEY, email TEXT);
CREATE INDEX users_email ON users(email);
CREATE TRIGGER audit_insert AFTER INSERT ON users BEGIN SELECT 1; END;
CREATE TRIGGER audit_delete AFTER DELETE ON USERS BEGIN SELECT 1; END;
CREATE VIEW recent AS SELECT 1;
CREATE TRIGGER recent_insert INSTEAD OF INSERT ON Recent BEGIN SELECT 1; END;
`);
const selected = inspectSqliteDatabaseSnapshot(path, {
path,
table: "Users",
});
expect(selected.schema).toMatchObject({
scope: "selected_table",
completeness: "complete",
tables: [{ name: "Users" }],
indexes: [{ name: "users_email", table: "Users" }],
views: [],
});
expect(
selected.schema.triggers.map(({ name, table }) => ({ name, table })),
).toEqual([
{ name: "audit_insert", table: "Users" },
{ name: "audit_delete", table: "Users" },
]);
const inventory = inspectSqliteDatabaseSnapshot(path, { path });
expect(inventory.schema.triggers).toEqual([
{ name: "audit_insert", table: "Users" },
{ name: "audit_delete", table: "Users" },
{ name: "recent_insert", table: "recent" },
]);
});
it.each([
{
selection: "a differently cased table",
schema: "CREATE TABLE Users(id INTEGER PRIMARY KEY)",
trigger: "AFTER INSERT ON Users",
table: "users",
},
{
selection: "an exact view",
schema: "CREATE VIEW Recent AS SELECT 1 AS id",
trigger: "INSTEAD OF INSERT ON recent",
table: "Recent",
},
])(
"refuses $selection before expanding declarations under another spelling",
async ({ schema, trigger, table }) => {
const path = await createDatabase(`
${schema};
CREATE TRIGGER expanded ${trigger} BEGIN /*${"\u0001".repeat(3 * 1024 * 1024)}*/ SELECT 1; END;
`);
expect(() =>
inspectSqliteDatabaseSnapshot(path, { path, table }),
).toThrowError(
expect.objectContaining({
reason: "selection",
message: `Selected ordinary table does not exist with that exact name: ${table}`,
}),
);
},
);
it.each(["UTF-8", "UTF-16le", "UTF-16be"])(
"preserves exact %s schema text when decoding native bytes",
async (encoding) => {
+71 -66
View File
@@ -22,6 +22,73 @@ afterEach(async () => {
await rm(root, { recursive: true, force: true });
});
const reobservesExternallyMutableState = async (): Promise<void> => {
const { root, target } = await createIdaTarget();
roots.push(root);
const path = join(root, "registration.json");
await writeFile(path, JSON.stringify({ command: "fixture" }));
const config = parseConfig({ REA_IDA_MCP_CONFIG: path });
if (!config.ok) throw config.error;
const producer = new RecordingIdaMcp(target);
const session = createTestBinarySession(
new IdaProvider(config.value, {}, () => producer),
);
expect((await session.open(target.path)).ok).toBe(true);
const query = { procedure: "main", document: target.path };
const first = await session.execute("procedure_pseudo_code", query);
expect(first.ok && first.value.result).toContain("42");
const snapshot = session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
expect(snapshot.value.entries).toEqual([]);
const profile = session.analysisProfile();
if (profile === undefined) throw new Error("Missing profile");
const cached = createAnalysisExecution(
"stale output",
IDA_PROVIDER_IDENTITY,
{
analysisProfile: profile,
subject: {
path: target.path,
sha256: target.sha256,
format: target.format,
...(target.architecture === undefined
? {}
: { architecture: target.architecture }),
},
},
);
snapshot.value.entries.push(
createAnalysisSnapshotEntry({
target: snapshotTarget(target),
binding: snapshotBinding(profile),
operation: "procedure_pseudo_code",
parameters: query,
execution: cached,
}),
);
snapshot.value.evidence_bundle = createEvidenceBundle([
...snapshot.value.evidence_bundle.records,
createEvidence(target, IDA_PROVIDER_IDENTITY, {
operation: "procedure_pseudo_code",
parameters: query,
result: cached.result,
rawResult: cached.rawResult,
analysisProfile: profile,
}),
]);
expect(session.importAnalysisSnapshot(snapshot.value).ok).toBe(true);
expect(session.allowsSnapshotReplay("procedure_pseudo_code")).toBe(false);
expect(session.allowsSnapshotReplay("binary_overview")).toBe(false);
producer.pseudocode = "int main(void) { return 99; }";
const second = await session.execute("procedure_pseudo_code", query);
expect(second.ok && second.value.result).toContain("99");
expect(
producer.calls.filter(({ name }) => name === "decompile_function"),
).toHaveLength(2);
await session.close();
expect(producer.closes).toBe(1);
};
describe("IDA provider composition", () => {
it("discovers without launching and commits adapter semantics without persisting credentials", async () => {
const { root, target } = await createIdaTarget();
@@ -59,72 +126,10 @@ describe("IDA provider composition", () => {
.find(({ operation }) => operation === "procedure_callers")?.available,
).toBe(false);
});
it("re-observes externally mutable state even when a document is explicit and a snapshot is imported", async () => {
const { root, target } = await createIdaTarget();
roots.push(root);
const path = join(root, "registration.json");
await writeFile(path, JSON.stringify({ command: "fixture" }));
const config = parseConfig({ REA_IDA_MCP_CONFIG: path });
if (!config.ok) throw config.error;
const producer = new RecordingIdaMcp(target);
const session = createTestBinarySession(
new IdaProvider(config.value, {}, () => producer),
);
expect((await session.open(target.path)).ok).toBe(true);
const query = { procedure: "main", document: target.path };
const first = await session.execute("procedure_pseudo_code", query);
expect(first.ok && first.value.result).toContain("42");
const snapshot = session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
expect(snapshot.value.entries).toEqual([]);
const profile = session.analysisProfile();
if (profile === undefined) throw new Error("Missing profile");
const cached = createAnalysisExecution(
"stale output",
IDA_PROVIDER_IDENTITY,
{
analysisProfile: profile,
subject: {
path: target.path,
sha256: target.sha256,
format: target.format,
...(target.architecture === undefined
? {}
: { architecture: target.architecture }),
},
},
);
snapshot.value.entries.push(
createAnalysisSnapshotEntry({
target: snapshotTarget(target),
binding: snapshotBinding(profile),
operation: "procedure_pseudo_code",
parameters: query,
execution: cached,
}),
);
snapshot.value.evidence_bundle = createEvidenceBundle([
...snapshot.value.evidence_bundle.records,
createEvidence(target, IDA_PROVIDER_IDENTITY, {
operation: "procedure_pseudo_code",
parameters: query,
result: cached.result,
rawResult: cached.rawResult,
analysisProfile: profile,
}),
]);
expect(session.importAnalysisSnapshot(snapshot.value).ok).toBe(true);
expect(session.allowsSnapshotReplay("procedure_pseudo_code")).toBe(false);
expect(session.allowsSnapshotReplay("binary_overview")).toBe(false);
producer.pseudocode = "int main(void) { return 99; }";
const second = await session.execute("procedure_pseudo_code", query);
expect(second.ok && second.value.result).toContain("99");
expect(
producer.calls.filter(({ name }) => name === "decompile_function"),
).toHaveLength(2);
await session.close();
expect(producer.closes).toBe(1);
});
it(
"re-observes externally mutable state even when a document is explicit and a snapshot is imported",
reobservesExternallyMutableState,
);
it("rejects unsupported target kinds and reports incomplete upstream profiles on first analysis", async () => {
const { root, target } = await createIdaTarget();
roots.push(root);