Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
426aa62c10 | ||
|
|
1268cb328f | ||
|
|
c73cf0ab76 | ||
|
|
aa7008f8a5 | ||
|
|
1887e1120e | ||
|
|
ca7ef69c0c | ||
|
|
b9c439831d | ||
|
|
bace2c5a4f | ||
|
|
3cd80cbb88 | ||
|
|
6cd5ea0fb2 | ||
|
|
706bba6e13 | ||
|
|
9964af6b79 | ||
|
|
f2f56b5804 | ||
|
|
04875e7817 | ||
|
|
3059d8a40f | ||
|
|
86b223b33e | ||
|
|
92741187ec | ||
|
|
5fdb0636b4 | ||
|
|
72b3209f3a |
@@ -31,7 +31,7 @@ When tools are absent or registration is stale:
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
`commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`, `hermes`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
|
||||
@@ -208,10 +208,53 @@ Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Herme
|
||||
connects an entry unless `enabled` is false, so setup states `enabled: true`.
|
||||
|
||||
Hermes personal skills install under the active Hermes home's `skills` directory.
|
||||
Setup, doctor and uninstall follow the root's `active_profile` selection, as
|
||||
plain Hermes CLI commands do. An explicit `HERMES_HOME` pointing to
|
||||
`profiles/<name>` selects that named profile directly. An invalid, missing or
|
||||
deleted sticky profile is reported instead of writing into the default home;
|
||||
repair the profile or run `hermes profile use default` before retrying.
|
||||
REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
|
||||
and home expansion before resolving that profile. Setup previews the resolved
|
||||
configuration and skill destinations; doctor and uninstall use those same paths.
|
||||
|
||||
### DeepSeek Harness
|
||||
|
||||
DeepSeek Harness uses a Cordis YAML patch sequence rather than a `mcpServers`
|
||||
object. REA's `setup --client` does not edit this format. For Harness
|
||||
`0.2.0-rc.2`, append the following entry to `$DSH_HOME/cordis.patch.yml`
|
||||
(`~/.dsh/cordis.patch.yml` when `DSH_HOME` is unset). Preserve existing patches
|
||||
and replace the executable paths with absolute paths from your installation:
|
||||
|
||||
```yaml
|
||||
- insert:
|
||||
- id: rea-mcp
|
||||
name: "@deepseek-ai/dsh-mcp-client"
|
||||
config:
|
||||
serverName: rea
|
||||
transport: stdio
|
||||
command: /absolute/path/to/node
|
||||
args: ["/absolute/path/to/rea/scripts/rea.mjs", mcp]
|
||||
cwd: !!js process.cwd()
|
||||
```
|
||||
|
||||
For a global npm installation, the launcher is under
|
||||
`$(npm root -g)/rea-agents/scripts/rea.mjs`. Check the Node executable with
|
||||
`command -v node`. Register `serverName: rea` only once in each active profile,
|
||||
then restart Harness. It exposes tools as `mcp__rea__<tool_name>`.
|
||||
Run `rea setup --skill --dry-run`, review the destination, then
|
||||
`rea setup --skill --yes` to install the workflow in `~/.agents/skills`.
|
||||
Harness discovers that directory, or `$DSH_AGENTS_HOME/skills` when configured;
|
||||
its native `skill` tool can load `reverse-engineer-anything`.
|
||||
|
||||
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
|
||||
checks headless chat, skill loading, all tool input schemas and actual analysis
|
||||
using the installed Harness and a local model fixture. This establishes the
|
||||
Harness integration, with live DeepSeek API compatibility remaining unverified.
|
||||
See the [Harness MCP guide](https://github.com/deepseek-ai/deepseek-harness/blob/639ed015397290b3745d163aafe02ffee4aa3f84/packages/mcp/mcp-client/README.md)
|
||||
for its native configuration contract.
|
||||
|
||||
### Other client settings
|
||||
|
||||
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
|
||||
both load. If the configuration already uses OpenCode V2's native
|
||||
`mcp.servers` table, setup registers REA there instead and replaces any earlier
|
||||
|
||||
@@ -144,6 +144,12 @@ if retention is unavailable or fails, the diagnostic reports that reason.
|
||||
REA accepts ordinary `tools/call` progress tokens. Updates are monotonic,
|
||||
rate-bounded to at most one intermediate update per 100 ms, and always allow a
|
||||
terminal update. Unknown totals are omitted; REA does not fabricate percentages.
|
||||
`capture_process_scenario` uses that boundary only when the client supplies a
|
||||
progress token. Notifications set `progress` to the count of collected terminal
|
||||
frames, process samples, and interaction events, omit `total`, and put the
|
||||
lifecycle phase, elapsed time, and final cleanup status in `message`. A client
|
||||
that does not send a progress token receives no progress notifications. The
|
||||
tool result is unchanged either way.
|
||||
Provider calls receive the request cancellation signal. Artifact traversal,
|
||||
hashing, version comparisons, Hopper requests, and process capture
|
||||
check the same signal. Cancellation is distinct from timeout. A cleanup failure
|
||||
|
||||
@@ -51,6 +51,34 @@ format is TOON. The file contains the complete capture Evidence record.
|
||||
Choose an output file distinct from the scenario input: shell redirection opens
|
||||
and truncates the output before REA reads the scenario.
|
||||
|
||||
While the capture runs, REA writes live status to stderr as one JSON object per
|
||||
line. Stdout stays the final Evidence document:
|
||||
|
||||
```json
|
||||
{
|
||||
"rea_progress": {
|
||||
"phase": "running",
|
||||
"completed": 3,
|
||||
"total": null,
|
||||
"message": "elapsed_ms=1200 frames=1 samples=2 interactions=0",
|
||||
"sequence": 4
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`phase` is the observed lifecycle stage: `prepare`, `running`, `settling`, or
|
||||
`cleanup`. `completed` counts collected terminal frames, process samples, and
|
||||
interaction events. `total` stays null; REA does not invent a unit total or a
|
||||
percentage. `message` reports elapsed time and those counts. The final line uses
|
||||
`phase` `cleanup`, sets `terminal` to true, and adds `disposition` (`exited`,
|
||||
`timeout`, `idle_timeout`, `cancelled`, or `failed`) plus the owned-process,
|
||||
renderer, and temporary-root cleanup states. Updates use the shared progress
|
||||
boundary and are limited to one intermediate line per 100 ms; the terminal line
|
||||
is always emitted. If a receiver is slow, intermediate observations are merged
|
||||
into the latest status, and stale live observations are discarded before final
|
||||
cleanup status. Progress never copies child output. Child output remains in the
|
||||
capture Evidence on stdout.
|
||||
|
||||
For example:
|
||||
|
||||
```json
|
||||
|
||||
@@ -102,6 +102,36 @@ cleanup. These POSIX lanes use a deterministic loopback model, so they do not
|
||||
prove live model-provider or native Windows compatibility. Qwen and Pi use
|
||||
caller-configured skill directories to isolate the fixture account from the OS
|
||||
home; this does not prove their default home-directory discovery.
|
||||
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
||||
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
||||
|
||||
`verify:gemini-client` requires an installed Gemini CLI (verified with
|
||||
`@google/gemini-cli@0.63.0`); select it with `REA_VERIFY_GEMINI_COMMAND`.
|
||||
The optional POSIX lane uses native `GEMINI_CLI_HOME` discovery in an isolated
|
||||
Git project, checks setup plans, backups and idempotence, activates the installed
|
||||
personal skill, validates all forwarded input JSON Schemas against their declared
|
||||
dialect, forwards the complete REA catalog and checks full JavaScript
|
||||
Evidence for a Unicode path. Use `-- chat` for ordinary chat with REA enabled,
|
||||
or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed REA package.
|
||||
The local fixture exercises the real Gemini API adapter, including its
|
||||
`untrusted_context` tool-result envelope. Token counts are synthetic; live
|
||||
Google API and native Windows compatibility remain unverified. The lane disables
|
||||
the client's memory-based relaunch to preserve the caller's Node heap budget.
|
||||
|
||||
`verify:deepseek-client` requires an installed DeepSeek Harness (`dsh`;
|
||||
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
|
||||
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
|
||||
installs the skill through `rea setup --skill`, writes the native Cordis MCP
|
||||
patch into an isolated `DSH_HOME`, and uses `DSH_AGENTS_HOME` for native personal
|
||||
skill discovery. A separate Git root prevents inherited project skills from
|
||||
masking a missing installation. The default `call` mode checks skill loading,
|
||||
complete catalog discovery, the actual forwarded input schemas and regexes,
|
||||
and full JavaScript Evidence for a Unicode path. Use `-- chat` for ordinary
|
||||
chat with REA enabled, or `REA_VERIFY_RUNTIME_ROOT` for an installed package.
|
||||
The fixture uses Harness's custom OpenAI adapter and the one-million-token
|
||||
context capacity declared by its default DeepSeek Flash model; it does not
|
||||
establish live DeepSeek API or native Windows coverage. Receipts include
|
||||
request artifacts, result digests and owned-process lineage.
|
||||
|
||||
Full E2E tests invoke the production command dispatcher and real providers,
|
||||
without fake launchers, runners or responses. `verify:keyed-archive` writes an
|
||||
|
||||
@@ -180,8 +180,10 @@
|
||||
"verify:package": "npm run build:cached && node scripts/verify-package.mjs",
|
||||
"verify:package:windows": "npm run build:cached && node scripts/verify-windows-package.mjs",
|
||||
"verify:qwen-client": "npm run build:cached && node scripts/verify-qwen-client.mjs",
|
||||
"verify:gemini-client": "npm run build:cached && node scripts/verify-gemini-client.mjs",
|
||||
"verify:pi-client": "npm run build:cached && node scripts/verify-native-agent-client.mjs pi",
|
||||
"verify:hermes-client": "npm run build:cached && node scripts/verify-native-agent-client.mjs hermes",
|
||||
"verify:deepseek-client": "npm run build:cached && node scripts/verify-deepseek-client.mjs",
|
||||
"probe:windows-capabilities": "npm run build:cached && node scripts/windows-capabilities.mjs",
|
||||
"verify:published": "node scripts/verify-published-package.mjs",
|
||||
"config:print": "node scripts/print-mcp-config.mjs",
|
||||
|
||||
@@ -40,7 +40,11 @@ export async function verifyGhidraBoundaries(
|
||||
ajv.validateSchema(tool[key]),
|
||||
`${tool.name} ${key}: ${JSON.stringify(ajv.errors)}`,
|
||||
);
|
||||
validators.set(tool.name, ajv.compile(tool.outputSchema));
|
||||
validators.set(
|
||||
tool.name,
|
||||
(tool.outputSchema?.$id && ajv.getSchema(tool.outputSchema.$id)) ||
|
||||
ajv.compile(tool.outputSchema),
|
||||
);
|
||||
}
|
||||
let successfulCalls = 0;
|
||||
let rejectedCalls = 0;
|
||||
|
||||
@@ -42,7 +42,11 @@ export async function verifyHopperBoundaryContracts(
|
||||
ajv.validateSchema(tool.outputSchema),
|
||||
`${tool.name} output schema`,
|
||||
);
|
||||
validators.set(tool.name, ajv.compile(tool.outputSchema));
|
||||
validators.set(
|
||||
tool.name,
|
||||
(tool.outputSchema?.$id && ajv.getSchema(tool.outputSchema.$id)) ||
|
||||
ajv.compile(tool.outputSchema),
|
||||
);
|
||||
}
|
||||
let successfulCalls = 0;
|
||||
let rejectedCalls = 0;
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
|
||||
|
||||
const mode = process.argv[2] ?? "call";
|
||||
assert(["chat", "call"].includes(mode), "Select chat or call.");
|
||||
assert(process.platform !== "win32", "Native Windows coverage is unverified.");
|
||||
const repo = fileURLToPath(new URL("..", import.meta.url));
|
||||
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_DEEPSEEK_COMMAND ?? "dsh";
|
||||
const lab = await mkdtemp(join(tmpdir(), "rea-deepseek-client-"));
|
||||
const account = join(lab, "account");
|
||||
const profile = join(lab, "selected profile");
|
||||
const workspace = join(lab, "workspace");
|
||||
const target = join(workspace, "fixture α测试");
|
||||
const skillPath = join(
|
||||
account,
|
||||
".agents",
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
"SKILL.md",
|
||||
);
|
||||
const cli = join(runtimeRoot, "scripts", "rea.mjs");
|
||||
const verifier = createVerifierRun();
|
||||
await Promise.all([
|
||||
mkdir(account),
|
||||
mkdir(profile),
|
||||
mkdir(target, { recursive: true }),
|
||||
]);
|
||||
const environment = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
USERPROFILE: account,
|
||||
DSH_HOME: profile,
|
||||
DSH_AGENTS_HOME: join(account, ".agents"),
|
||||
OPENAI_API_KEY: "local-fixture-key",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
// A temporary directory under the checkout must not inherit its project skills or AGENTS.md.
|
||||
await exec(
|
||||
"git",
|
||||
["-c", "init.defaultBranch=main", "init", "--quiet", workspace],
|
||||
{
|
||||
env: environment,
|
||||
timeout: 30_000,
|
||||
},
|
||||
);
|
||||
await writeFile(
|
||||
join(target, "app.js"),
|
||||
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
|
||||
);
|
||||
const clientVersion = (
|
||||
await exec(command, ["--version"], {
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout.trim();
|
||||
for (const args of [
|
||||
["setup", "--skill", "--dry-run"],
|
||||
["setup", "--skill", "--yes"],
|
||||
]) {
|
||||
const result = await exec(process.execPath, [cli, ...args, "--json"], {
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 60_000,
|
||||
});
|
||||
const parsed = JSON.parse(result.stdout);
|
||||
assert.equal(parsed.status, args.includes("--dry-run") ? "planned" : "ready");
|
||||
if (args.includes("--dry-run")) {
|
||||
assert.equal(parsed.plannedActions.length, 1);
|
||||
assert.equal(parsed.plannedActions[0].kind, "install_skill");
|
||||
assert.equal(parsed.plannedActions[0].target, dirname(skillPath));
|
||||
}
|
||||
await writeFile(
|
||||
join(lab, args.includes("--dry-run") ? "setup-plan.json" : "setup.json"),
|
||||
result.stdout,
|
||||
);
|
||||
}
|
||||
assert(
|
||||
(await readFile(skillPath, "utf8")).includes(
|
||||
"name: reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
|
||||
let catalog;
|
||||
let patternCount = 0;
|
||||
let skillRequested = false;
|
||||
let skillLoaded = false;
|
||||
let evidenceSeen = false;
|
||||
let finalSeen = false;
|
||||
let artifactDigest;
|
||||
let mainRequests = 0;
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
const patterns = new Set();
|
||||
const validatePatterns = (value) => {
|
||||
if (!value || typeof value !== "object") return;
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (key === "pattern" && typeof child === "string") {
|
||||
// Check the representation the native client actually forwards to its model adapter.
|
||||
for (const flags of ["", "u", "v"]) new RegExp(child, flags);
|
||||
patterns.add(child);
|
||||
patternCount++;
|
||||
} else validatePatterns(child);
|
||||
}
|
||||
};
|
||||
const extractEvidence = (value) => {
|
||||
if (typeof value === "string") {
|
||||
try {
|
||||
return extractEvidence(JSON.parse(value));
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
if (
|
||||
typeof value.evidence_id === "string" &&
|
||||
value.normalized_result?.graph?.nodes
|
||||
)
|
||||
return value;
|
||||
for (const child of Object.values(value)) {
|
||||
const found = extractEvidence(child);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
};
|
||||
const fixture = await createOpenAiModelFixture({
|
||||
directory: lab,
|
||||
prefix: `deepseek-${mode}`,
|
||||
onRequest: async ({ body, tools, requestIndex }) => {
|
||||
// Harness generates a session title in a separate, tool-free model request.
|
||||
if (tools.length === 0)
|
||||
return { role: "assistant", content: "REA validation" };
|
||||
assert(
|
||||
++mainRequests <= 4,
|
||||
"Harness did not complete the bounded fixture workflow.",
|
||||
);
|
||||
const rea = tools.filter((tool) => tool.name.startsWith("mcp__rea__"));
|
||||
const observedCatalog = rea
|
||||
.map((tool) => tool.name.slice("mcp__rea__".length))
|
||||
.sort();
|
||||
assert.deepEqual(
|
||||
observedCatalog,
|
||||
TOOL_CONTRACTS.map((tool) => tool.name).sort(),
|
||||
);
|
||||
if (catalog === undefined) {
|
||||
catalog = observedCatalog;
|
||||
for (const tool of rea) {
|
||||
assert(
|
||||
ajv.validateSchema(tool.parameters),
|
||||
`${tool.name}: ${ajv.errorsText(ajv.errors)}`,
|
||||
);
|
||||
validatePatterns(tool.parameters);
|
||||
}
|
||||
assert(
|
||||
patternCount > 0,
|
||||
"The advertised regex constraints must reach the model.",
|
||||
);
|
||||
}
|
||||
const toolMessages = (body.messages ?? []).filter(
|
||||
(message) => message.role === "tool",
|
||||
);
|
||||
const serialized = JSON.stringify(toolMessages);
|
||||
skillLoaded ||=
|
||||
serialized.includes(
|
||||
'<skill_content name=\\"reverse-engineer-anything\\">',
|
||||
) && serialized.includes(dirname(skillPath));
|
||||
for (const message of toolMessages) {
|
||||
const found = extractEvidence(message.content);
|
||||
if (found === undefined) continue;
|
||||
const evidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(found);
|
||||
assert.equal(evidence.subject.local_path, target);
|
||||
const shape = evidence.normalized_result.graph.nodes
|
||||
.flatMap((node) => node.observations)
|
||||
.map((observation) => observation.properties)
|
||||
.find(
|
||||
(properties) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "result",
|
||||
);
|
||||
assert(shape?.return_shape_coverage?.projection_complete);
|
||||
const fields = shape.static_return_shapes.flatMap(
|
||||
(entry) => entry.fields,
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/clientCompatibility" &&
|
||||
field.state === "literal" &&
|
||||
field.value === "REA_CLIENT_FIXTURE",
|
||||
),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/count" &&
|
||||
field.state === "literal" &&
|
||||
field.value === 7,
|
||||
),
|
||||
);
|
||||
const text = JSON.stringify(evidence);
|
||||
await writeFile(join(lab, "evidence.json"), text);
|
||||
artifactDigest = createHash("sha256").update(text).digest("hex");
|
||||
evidenceSeen = true;
|
||||
}
|
||||
if (mode === "chat" || evidenceSeen) {
|
||||
finalSeen = true;
|
||||
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
|
||||
}
|
||||
const call = (name, args) => ({
|
||||
role: "assistant",
|
||||
content: null,
|
||||
tool_calls: [
|
||||
{
|
||||
id: `call_rea_${requestIndex}`,
|
||||
type: "function",
|
||||
function: { name, arguments: JSON.stringify(args) },
|
||||
},
|
||||
],
|
||||
});
|
||||
if (!skillRequested) {
|
||||
assert(tools.some((tool) => tool.name === "skill"));
|
||||
skillRequested = true;
|
||||
return call("skill", { name: "reverse-engineer-anything" });
|
||||
}
|
||||
assert(
|
||||
skillLoaded,
|
||||
"Harness must load the installed skill before analysis.",
|
||||
);
|
||||
return call("mcp__rea__analyze_javascript_application", {
|
||||
input_path: target,
|
||||
format: "directory",
|
||||
});
|
||||
},
|
||||
});
|
||||
let failure;
|
||||
try {
|
||||
// Use the client's native Cordis patch DSL; generic `rea setup --client` does not edit it.
|
||||
await writeFile(
|
||||
join(profile, "cordis.patch.yml"),
|
||||
`
|
||||
- id: llm-pi-ai
|
||||
config:
|
||||
providers:
|
||||
rea-fixture:
|
||||
apiKeyEnv: OPENAI_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: ${fixture.baseUrl}
|
||||
defaultMaxTokens: 4096
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 0
|
||||
models:
|
||||
- id: rea-client-fixture
|
||||
contextWindow: 1000000
|
||||
- id: agent-default-model
|
||||
config:
|
||||
provider: rea-fixture
|
||||
model: rea-client-fixture
|
||||
- insert:
|
||||
- id: rea-mcp
|
||||
name: '@deepseek-ai/dsh-mcp-client'
|
||||
config:
|
||||
serverName: rea
|
||||
transport: stdio
|
||||
command: ${JSON.stringify(process.execPath)}
|
||||
args: ${JSON.stringify([cli, "mcp"])}
|
||||
cwd: !!js process.cwd()
|
||||
env:
|
||||
REA_PROCESS_RUN_ID: ${JSON.stringify(verifier.run_id)}
|
||||
`,
|
||||
);
|
||||
const execution = exec(
|
||||
command,
|
||||
[
|
||||
"--profile",
|
||||
"headless",
|
||||
"--json",
|
||||
mode === "chat" ? "Say hello briefly." : `Use REA to analyze ${target}.`,
|
||||
],
|
||||
{
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 120_000,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
execution.child.stdin.end();
|
||||
const result = await execution;
|
||||
await writeFile(join(lab, "client.stdout"), result.stdout);
|
||||
await writeFile(join(lab, "client.stderr"), result.stderr);
|
||||
assert.equal(fixture.failure, undefined);
|
||||
assert(finalSeen && result.stdout.includes("REA_CLIENT_COMPATIBILITY_OK"));
|
||||
if (mode === "call") assert(evidenceSeen && skillLoaded);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
await writeFile(join(lab, "client.stdout"), cause.stdout);
|
||||
if (typeof cause?.stderr === "string")
|
||||
await writeFile(join(lab, "client.stderr"), cause.stderr);
|
||||
} finally {
|
||||
await fixture.close();
|
||||
const receipt = {
|
||||
client: "deepseek-harness",
|
||||
clientVersion,
|
||||
mode,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
host: {
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
node: process.versions.node,
|
||||
},
|
||||
modelFixture:
|
||||
"loopback custom OpenAI-compatible adapter; no live DeepSeek provider claim",
|
||||
modelContextWindow: 1000000,
|
||||
skillDiscovery:
|
||||
"native DSH_AGENTS_HOME personal directory; isolated project root",
|
||||
catalogSize: catalog?.length,
|
||||
patternCount,
|
||||
uniquePatterns: patterns.size,
|
||||
skillLoaded,
|
||||
evidenceSeen,
|
||||
artifactDigest,
|
||||
finalSeen,
|
||||
requests: fixture.requests,
|
||||
fixtureFailure: fixture.failure,
|
||||
verifier: await completeVerifierRun(verifier),
|
||||
};
|
||||
await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
|
||||
console.log(JSON.stringify(receipt));
|
||||
}
|
||||
if (failure !== undefined) throw failure;
|
||||
@@ -0,0 +1,378 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createGeminiModelFixture } from "./verify/clients/gemini-model-fixture.mjs";
|
||||
|
||||
assert(
|
||||
process.platform !== "win32",
|
||||
"Native Gemini verifier requires POSIX; Windows is unverified",
|
||||
);
|
||||
const repo = fileURLToPath(new URL("..", import.meta.url));
|
||||
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_GEMINI_COMMAND ?? "gemini";
|
||||
const mode = process.argv[2] ?? "call";
|
||||
assert(
|
||||
["chat", "call"].includes(mode),
|
||||
"Usage: verify-gemini-client.mjs [chat|call]",
|
||||
);
|
||||
const verifier = createVerifierRun();
|
||||
const lab = await mkdtemp(join(tmpdir(), "rea-gemini-client-"));
|
||||
const account = join(lab, "account");
|
||||
const profile = join(lab, "selected profile");
|
||||
const workspace = join(lab, "workspace");
|
||||
const configPath = join(profile, ".gemini", "settings.json");
|
||||
const skillPath = join(
|
||||
profile,
|
||||
".agents",
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
"SKILL.md",
|
||||
);
|
||||
await Promise.all([
|
||||
mkdir(account),
|
||||
mkdir(workspace),
|
||||
mkdir(join(profile, ".gemini"), { recursive: true }),
|
||||
]);
|
||||
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
||||
await writeFile(join(lab, "system-settings.json"), "{}\n");
|
||||
await writeFile(join(lab, "system-defaults.json"), "{}\n");
|
||||
const environment = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
USERPROFILE: account,
|
||||
GEMINI_CLI_HOME: profile,
|
||||
GEMINI_CLI_SYSTEM_SETTINGS_PATH: join(lab, "system-settings.json"),
|
||||
GEMINI_CLI_SYSTEM_DEFAULTS_PATH: join(lab, "system-defaults.json"),
|
||||
// Preserve the caller's bounded Node heap instead of the launcher's host-memory auto-sizing.
|
||||
GEMINI_CLI_NO_RELAUNCH: "1",
|
||||
GEMINI_CLI_TRUST_WORKSPACE: "true",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
let clientVersion;
|
||||
try {
|
||||
clientVersion = (
|
||||
await exec(command, ["--version"], { env: environment, timeout: 30_000 })
|
||||
).stdout.trim();
|
||||
} catch (cause) {
|
||||
throw new Error(
|
||||
`Gemini preflight failed for ${command}; install Gemini CLI or set REA_VERIFY_GEMINI_COMMAND. ${String(cause)}`,
|
||||
);
|
||||
}
|
||||
const original = JSON.stringify(
|
||||
{
|
||||
security: {
|
||||
auth: { selectedType: "gemini-api-key" },
|
||||
folderTrust: { enabled: false },
|
||||
},
|
||||
telemetry: { enabled: false },
|
||||
advanced: { autoConfigureMemory: false },
|
||||
model: { name: "gemini-2.5-flash", maxSessionTurns: 8 },
|
||||
mcpServers: { other: { command: "unrelated-server" } },
|
||||
},
|
||||
null,
|
||||
2,
|
||||
);
|
||||
await writeFile(configPath, original);
|
||||
for (const args of [
|
||||
["setup", "--client", "gemini_cli", "--dry-run"],
|
||||
["setup", "--client", "gemini_cli", "--yes"],
|
||||
["doctor", "--client", "gemini_cli", "--skill"],
|
||||
]) {
|
||||
const result = await exec(
|
||||
process.execPath,
|
||||
[join(runtimeRoot, "scripts/rea.mjs"), ...args, "--json"],
|
||||
{ env: environment, cwd: workspace, timeout: 60_000 },
|
||||
);
|
||||
const parsed = JSON.parse(result.stdout);
|
||||
if (args[0] === "doctor") assert.equal(parsed.healthy, true);
|
||||
else {
|
||||
assert.equal(
|
||||
parsed.status,
|
||||
args.includes("--dry-run") ? "planned" : "ready",
|
||||
);
|
||||
if (args.includes("--dry-run")) {
|
||||
assert(parsed.plannedActions.some((a) => a.target === configPath));
|
||||
assert(
|
||||
parsed.plannedActions.some((a) =>
|
||||
a.target.startsWith(join(profile, ".agents", "skills") + "/"),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
parsed.plannedActions.every((a) => a.target.startsWith(profile + "/")),
|
||||
);
|
||||
}
|
||||
}
|
||||
await writeFile(
|
||||
join(lab, `${args[0]}${args.includes("--dry-run") ? "-plan" : ""}.json`),
|
||||
result.stdout,
|
||||
);
|
||||
}
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
const configured = await readFile(configPath, "utf8");
|
||||
assert.deepEqual(JSON.parse(configured).mcpServers.other, {
|
||||
command: "unrelated-server",
|
||||
});
|
||||
const repeated = await exec(
|
||||
process.execPath,
|
||||
[
|
||||
join(runtimeRoot, "scripts/rea.mjs"),
|
||||
"setup",
|
||||
"--client",
|
||||
"gemini_cli",
|
||||
"--yes",
|
||||
"--json",
|
||||
],
|
||||
{ env: environment, cwd: workspace, timeout: 60_000 },
|
||||
);
|
||||
assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []);
|
||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
assert(
|
||||
(await readFile(skillPath, "utf8")).includes(
|
||||
"name: reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
const target = join(lab, "fixture α测试");
|
||||
await mkdir(target);
|
||||
await writeFile(
|
||||
join(target, "app.js"),
|
||||
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
|
||||
);
|
||||
let evidenceSeen = false;
|
||||
let skillLoaded = false;
|
||||
let skillRequested = false;
|
||||
let finalSeen = false;
|
||||
let catalogSize = 0;
|
||||
let schemasChecked = 0;
|
||||
let artifactDigest;
|
||||
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
|
||||
const extractEvidence = (value) => {
|
||||
if (typeof value === "string") {
|
||||
if (
|
||||
value.startsWith("<untrusted_context>\n") &&
|
||||
value.endsWith("\n</untrusted_context>")
|
||||
) {
|
||||
return extractEvidence(
|
||||
value.slice(
|
||||
"<untrusted_context>\n".length,
|
||||
-"\n</untrusted_context>".length,
|
||||
),
|
||||
);
|
||||
}
|
||||
try {
|
||||
return extractEvidence(JSON.parse(value));
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
if (
|
||||
typeof value.evidence_id === "string" &&
|
||||
value.normalized_result?.graph?.nodes?.length > 0
|
||||
)
|
||||
return value;
|
||||
for (const nested of Object.values(value)) {
|
||||
const evidence = extractEvidence(nested);
|
||||
if (evidence) return evidence;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
const fixture = await createGeminiModelFixture({
|
||||
directory: lab,
|
||||
onRequest: async ({ body, tools }) => {
|
||||
const analysis = tools.find((t) =>
|
||||
t.name.endsWith("analyze_javascript_application"),
|
||||
);
|
||||
assert(analysis, "Gemini must expose the REA analysis tool");
|
||||
const prefix = analysis.name.slice(
|
||||
0,
|
||||
-"analyze_javascript_application".length,
|
||||
);
|
||||
const expected = TOOL_CONTRACTS.map((t) => prefix + t.name).sort();
|
||||
assert.deepEqual(
|
||||
tools
|
||||
.filter((t) => expected.includes(t.name))
|
||||
.map((t) => t.name)
|
||||
.sort(),
|
||||
expected,
|
||||
"Gemini must forward the complete REA tool catalog",
|
||||
);
|
||||
catalogSize = expected.length;
|
||||
if (schemasChecked === 0) {
|
||||
for (const tool of tools.filter((t) => expected.includes(t.name))) {
|
||||
assert(
|
||||
tool.parametersJsonSchema,
|
||||
`${tool.name}: missing advertised JSON Schema`,
|
||||
);
|
||||
assert(
|
||||
schemaValidator.validateSchema(tool.parametersJsonSchema),
|
||||
`${tool.name}: invalid advertised JSON Schema: ${JSON.stringify(schemaValidator.errors)}`,
|
||||
);
|
||||
schemasChecked++;
|
||||
}
|
||||
}
|
||||
const responses = body.contents
|
||||
.flatMap((c) => c.parts ?? [])
|
||||
.filter((p) => p.functionResponse)
|
||||
.map((p) => p.functionResponse);
|
||||
skillLoaded ||= responses.some(
|
||||
(r) =>
|
||||
r.name === "activate_skill" &&
|
||||
r.response?.output?.includes(
|
||||
`<activated_skill name="reverse-engineer-anything">`,
|
||||
) &&
|
||||
r.response.output.includes(
|
||||
join(profile, ".agents", "skills", "reverse-engineer-anything"),
|
||||
),
|
||||
);
|
||||
const observed = extractEvidence(responses);
|
||||
if (observed) {
|
||||
const evidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(observed);
|
||||
assert.equal(evidence.subject.local_path, target);
|
||||
const shape = evidence.normalized_result.graph.nodes
|
||||
.flatMap((n) => n.observations)
|
||||
.map((o) => o.properties)
|
||||
.find(
|
||||
(p) =>
|
||||
p.semantic_role === "export-return-shapes" &&
|
||||
p.exported_name === "result",
|
||||
);
|
||||
assert(shape?.return_shape_coverage?.projection_complete);
|
||||
const fields = shape.static_return_shapes.flatMap((s) => s.fields);
|
||||
assert(
|
||||
fields.some(
|
||||
(f) =>
|
||||
f.path === "/clientCompatibility" &&
|
||||
f.state === "literal" &&
|
||||
f.value === "REA_CLIENT_FIXTURE",
|
||||
),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(f) => f.path === "/count" && f.state === "literal" && f.value === 7,
|
||||
),
|
||||
);
|
||||
const bytes = JSON.stringify(evidence);
|
||||
artifactDigest = createHash("sha256").update(bytes).digest("hex");
|
||||
await writeFile(join(lab, "gemini-evidence.json"), bytes);
|
||||
evidenceSeen = true;
|
||||
}
|
||||
if (mode === "chat" || evidenceSeen) {
|
||||
finalSeen = true;
|
||||
return [{ text: "REA_CLIENT_COMPATIBILITY_OK" }];
|
||||
}
|
||||
if (!skillRequested) {
|
||||
const skill = tools.find((t) => t.name === "activate_skill");
|
||||
assert(skill, "Gemini must expose its native skill activation tool");
|
||||
skillRequested = true;
|
||||
return [
|
||||
{
|
||||
functionCall: {
|
||||
name: skill.name,
|
||||
args: { name: "reverse-engineer-anything" },
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
assert(skillLoaded, "Gemini must load the installed personal REA skill");
|
||||
return [
|
||||
{
|
||||
functionCall: {
|
||||
name: analysis.name,
|
||||
args: { input_path: target, format: "directory" },
|
||||
},
|
||||
},
|
||||
];
|
||||
},
|
||||
});
|
||||
let failure;
|
||||
try {
|
||||
const response = await exec(
|
||||
command,
|
||||
[
|
||||
"--model",
|
||||
"gemini-2.5-flash",
|
||||
"--approval-mode",
|
||||
mode === "chat" ? "default" : "yolo",
|
||||
"--allowed-mcp-server-names",
|
||||
"rea",
|
||||
"--output-format",
|
||||
"stream-json",
|
||||
"-p",
|
||||
mode === "chat"
|
||||
? "Say hello briefly."
|
||||
: `Use REA to analyze the JavaScript directory ${target}. Report the result.`,
|
||||
],
|
||||
{
|
||||
env: {
|
||||
...environment,
|
||||
GEMINI_API_KEY: "local-fixture-key",
|
||||
GOOGLE_GEMINI_BASE_URL: fixture.baseUrl,
|
||||
},
|
||||
cwd: workspace,
|
||||
timeout: 120_000,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
await writeFile(join(lab, `${mode}.stdout`), response.stdout);
|
||||
await writeFile(join(lab, `${mode}.stderr`), response.stderr);
|
||||
const events = response.stdout
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line));
|
||||
assert.equal(events.findLast((e) => e.type === "result")?.status, "success");
|
||||
assert.equal(fixture.failure, undefined);
|
||||
assert(finalSeen && response.stdout.includes("REA_CLIENT_COMPATIBILITY_OK"));
|
||||
if (mode === "call") assert(evidenceSeen && skillLoaded);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
await writeFile(join(lab, `${mode}.stdout`), cause.stdout);
|
||||
if (typeof cause?.stderr === "string")
|
||||
await writeFile(join(lab, `${mode}.stderr`), cause.stderr);
|
||||
} finally {
|
||||
await fixture.close();
|
||||
const lineage = await completeVerifierRun(verifier);
|
||||
const receipt = {
|
||||
client: "Gemini CLI",
|
||||
clientVersion,
|
||||
mode,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
host: {
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
node: process.versions.node,
|
||||
},
|
||||
modelFixture:
|
||||
"loopback native Gemini API adapter; token counts synthetic; no live Google API claim",
|
||||
skillDiscovery:
|
||||
"native GEMINI_CLI_HOME personal shared directory; isolated Git project",
|
||||
catalogSize,
|
||||
schemasChecked,
|
||||
evidenceSeen,
|
||||
skillLoaded,
|
||||
artifactDigest,
|
||||
finalSeen,
|
||||
requests: fixture.requests,
|
||||
probes: fixture.probes,
|
||||
fixtureFailure: fixture.failure,
|
||||
verifier: lineage,
|
||||
};
|
||||
await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
|
||||
console.log(JSON.stringify(receipt));
|
||||
}
|
||||
if (failure !== undefined) throw failure;
|
||||
@@ -27,11 +27,16 @@ const command =
|
||||
const lab = await mkdtemp(join(tmpdir(), `rea-${client}-client-`));
|
||||
const account = join(lab, "account");
|
||||
const profile = join(lab, "selected profile");
|
||||
const stickyHermes =
|
||||
client === "hermes" && process.env.REA_VERIFY_HERMES_STICKY_PROFILE === "1";
|
||||
const selectedProfile = stickyHermes
|
||||
? join(profile, "profiles", "coder")
|
||||
: profile;
|
||||
const workspace = join(lab, "workspace");
|
||||
const target = join(workspace, "fixture α测试");
|
||||
const artifactPath = join(workspace, "evidence.json");
|
||||
const skillPath = join(
|
||||
client === "pi" ? join(account, ".agents") : profile,
|
||||
client === "pi" ? join(account, ".agents") : selectedProfile,
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
"SKILL.md",
|
||||
@@ -39,7 +44,8 @@ const skillPath = join(
|
||||
const verifier = createVerifierRun();
|
||||
await Promise.all([
|
||||
mkdir(account),
|
||||
mkdir(profile),
|
||||
mkdir(profile, { recursive: true }),
|
||||
...(stickyHermes ? [mkdir(selectedProfile, { recursive: true })] : []),
|
||||
mkdir(target, { recursive: true }),
|
||||
]);
|
||||
await writeFile(
|
||||
@@ -53,7 +59,11 @@ const environment = {
|
||||
PI_OFFLINE: "1",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
const configPath = join(profile, client === "pi" ? "mcp.json" : "config.yaml");
|
||||
const configPath = join(
|
||||
selectedProfile,
|
||||
client === "pi" ? "mcp.json" : "config.yaml",
|
||||
);
|
||||
if (stickyHermes) await writeFile(join(profile, "active_profile"), "coder\n");
|
||||
const original =
|
||||
client === "pi"
|
||||
? JSON.stringify({
|
||||
@@ -440,6 +450,7 @@ try {
|
||||
client === "pi"
|
||||
? "caller-configured isolated shared directory; default OS home discovery unverified"
|
||||
: "native HERMES_HOME personal directory",
|
||||
stickyHermes,
|
||||
catalogSize: catalog?.length,
|
||||
evidenceSeen,
|
||||
skillLoaded,
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
/** Exercise the native Gemini API adapter against a local model, without cloud claims. */
|
||||
export const createGeminiModelFixture = async ({ directory, onRequest }) => {
|
||||
const requests = [];
|
||||
const probes = [];
|
||||
let incoming = 0;
|
||||
let failure;
|
||||
const server = createServer(async (request, response) => {
|
||||
try {
|
||||
const chunks = [];
|
||||
let bytes = 0;
|
||||
for await (const chunk of request) {
|
||||
bytes += chunk.length;
|
||||
assert(
|
||||
bytes <= 10 * 1024 * 1024,
|
||||
"Model fixture request exceeds 10 MiB budget",
|
||||
);
|
||||
chunks.push(chunk);
|
||||
}
|
||||
const raw = Buffer.concat(chunks).toString("utf8");
|
||||
const body = raw.length === 0 ? null : JSON.parse(raw);
|
||||
await writeFile(
|
||||
join(directory, `gemini-incoming-${++incoming}.json`),
|
||||
JSON.stringify({ method: request.method, path: request.url, body }),
|
||||
);
|
||||
const path = new URL(request.url, "http://localhost").pathname;
|
||||
if (path.endsWith(":countTokens")) {
|
||||
probes.push({ path, kind: "synthetic-token-count" });
|
||||
response.setHeader("Content-Type", "application/json");
|
||||
response.end(JSON.stringify({ totalTokens: 100 }));
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!/\/models\/[^/]+:(?:streamGenerateContent|generateContent)$/u.test(
|
||||
path,
|
||||
)
|
||||
) {
|
||||
probes.push({ path, status: 404 });
|
||||
response.statusCode = 404;
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
error: { message: "Unsupported fixture endpoint" },
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
assert.equal(request.method, "POST");
|
||||
assert(body && typeof body === "object" && !Array.isArray(body));
|
||||
assert(Array.isArray(body.contents));
|
||||
const tools = (body.tools ?? []).flatMap(
|
||||
(t) => t.functionDeclarations ?? [],
|
||||
);
|
||||
requests.push({
|
||||
path,
|
||||
tools: tools.map((t) => t.name),
|
||||
contentRoles: body.contents.map((c) => c.role),
|
||||
});
|
||||
const parts = await onRequest({
|
||||
body,
|
||||
tools,
|
||||
requestIndex: requests.length,
|
||||
});
|
||||
const result = {
|
||||
candidates: [
|
||||
{ index: 0, content: { role: "model", parts }, finishReason: "STOP" },
|
||||
],
|
||||
usageMetadata: {
|
||||
promptTokenCount: 100,
|
||||
candidatesTokenCount: 20,
|
||||
totalTokenCount: 120,
|
||||
},
|
||||
modelVersion: "local-rea-fixture",
|
||||
};
|
||||
if (path.endsWith(":streamGenerateContent")) {
|
||||
response.setHeader("Content-Type", "text/event-stream");
|
||||
response.end(`data: ${JSON.stringify(result)}\n\n`);
|
||||
} else {
|
||||
response.setHeader("Content-Type", "application/json");
|
||||
response.end(JSON.stringify(result));
|
||||
}
|
||||
} catch (cause) {
|
||||
failure = String(cause);
|
||||
response.statusCode = 400;
|
||||
response.setHeader("Content-Type", "application/json");
|
||||
response.end(JSON.stringify({ error: { message: failure, code: 400 } }));
|
||||
}
|
||||
});
|
||||
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||
const address = server.address();
|
||||
assert(address !== null && typeof address !== "string");
|
||||
return {
|
||||
baseUrl: `http://127.0.0.1:${address.port}`,
|
||||
requests,
|
||||
probes,
|
||||
get failure() {
|
||||
return failure;
|
||||
},
|
||||
close: () => new Promise((resolve) => server.close(resolve)),
|
||||
};
|
||||
};
|
||||
@@ -88,7 +88,10 @@ window.loadFile("renderer.html");require("node:fs").writeFileSync(${JSON.stringi
|
||||
ajv.compile(tool.inputSchema);
|
||||
return [
|
||||
tool.name,
|
||||
tool.outputSchema ? ajv.compile(tool.outputSchema) : undefined,
|
||||
tool.outputSchema
|
||||
? (tool.outputSchema.$id && ajv.getSchema(tool.outputSchema.$id)) ||
|
||||
ajv.compile(tool.outputSchema)
|
||||
: undefined,
|
||||
];
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
import { lstatSync, readFileSync, realpathSync, statSync } from "node:fs";
|
||||
import {
|
||||
basename,
|
||||
dirname,
|
||||
isAbsolute,
|
||||
join,
|
||||
relative,
|
||||
resolve,
|
||||
sep,
|
||||
} from "node:path";
|
||||
|
||||
/** An unresolved Hermes profile must not be configured as the default home. */
|
||||
export class HermesClientPathError extends Error {
|
||||
constructor(
|
||||
readonly path: string,
|
||||
message: string,
|
||||
) {
|
||||
super(message);
|
||||
this.name = "HermesClientPathError";
|
||||
}
|
||||
}
|
||||
|
||||
const defaultDirectory = (
|
||||
home: string,
|
||||
platform: NodeJS.Platform,
|
||||
env: Readonly<NodeJS.ProcessEnv>,
|
||||
): string => {
|
||||
const suffix = env.HERMES_DATA_DIR_SUFFIX ?? "";
|
||||
return platform === "win32"
|
||||
? join(
|
||||
env.LOCALAPPDATA?.trim() || join(home, "AppData", "Local"),
|
||||
`hermes${suffix}`,
|
||||
)
|
||||
: join(home, `.hermes${suffix}`);
|
||||
};
|
||||
const expandedDirectory = (
|
||||
home: string,
|
||||
platform: NodeJS.Platform,
|
||||
env: Readonly<NodeJS.ProcessEnv>,
|
||||
): string | undefined => {
|
||||
const override = env.HERMES_HOME?.trim();
|
||||
if (!override) return undefined;
|
||||
const variables = override.replace(
|
||||
/\$(\w+|\{[^}]*\})/gu,
|
||||
(match, variable: string) => {
|
||||
const key = variable.startsWith("{") ? variable.slice(1, -1) : variable;
|
||||
return env[key] ?? match;
|
||||
},
|
||||
);
|
||||
const expanded =
|
||||
platform === "win32"
|
||||
? variables.replace(
|
||||
/%([^%]+)%/gu,
|
||||
(match, variable: string) => env[variable] ?? match,
|
||||
)
|
||||
: variables;
|
||||
if (expanded === "~") return home;
|
||||
if (
|
||||
expanded.startsWith("~/") ||
|
||||
(platform === "win32" && expanded.startsWith("~\\"))
|
||||
)
|
||||
return join(home, expanded.slice(2));
|
||||
return resolve(expanded);
|
||||
};
|
||||
const physicalPath = (path: string): string => {
|
||||
try {
|
||||
return realpathSync(path);
|
||||
} catch {
|
||||
return resolve(path);
|
||||
}
|
||||
};
|
||||
const within = (root: string, path: string): boolean => {
|
||||
const difference = relative(physicalPath(root), physicalPath(path));
|
||||
return (
|
||||
difference === "" ||
|
||||
(!isAbsolute(difference) &&
|
||||
difference !== ".." &&
|
||||
!difference.startsWith(`..${sep}`))
|
||||
);
|
||||
};
|
||||
const PROFILE_MARKERS = [
|
||||
"config.yaml",
|
||||
".env",
|
||||
"SOUL.md",
|
||||
"profile.yaml",
|
||||
"auth.json",
|
||||
"state.db",
|
||||
];
|
||||
const RESERVED_PROFILES = new Set(["hermes", "test", "tmp", "root", "sudo"]);
|
||||
const liveProfile = (path: string): boolean => {
|
||||
try {
|
||||
if (!statSync(path).isDirectory()) return false;
|
||||
try {
|
||||
statSync(join(dirname(path), ".deleted", basename(path)));
|
||||
return false;
|
||||
} catch {
|
||||
/* No tombstone. */
|
||||
}
|
||||
return PROFILE_MARKERS.some((marker) => {
|
||||
try {
|
||||
const status = lstatSync(join(path, marker));
|
||||
return status.isFile() || status.isSymbolicLink();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/** Match Hermes CLI sticky-profile selection while retaining the configured path spelling. */
|
||||
export const resolveHermesClientHome = (
|
||||
home: string,
|
||||
platform: NodeJS.Platform,
|
||||
env: Readonly<NodeJS.ProcessEnv>,
|
||||
): string => {
|
||||
const native = defaultDirectory(home, platform, env);
|
||||
const root = expandedDirectory(home, platform, env) ?? native;
|
||||
// An explicit named profile bypasses the root's sticky selection in Hermes CLI.
|
||||
if (basename(dirname(root)) === "profiles") return root;
|
||||
const stickyRoot = within(native, root) ? native : root;
|
||||
let name: string;
|
||||
try {
|
||||
// Python uses utf-8-sig and ignores unreadable / invalid-UTF8 active_profile files.
|
||||
name = new TextDecoder("utf-8", { fatal: true })
|
||||
.decode(readFileSync(join(stickyRoot, "active_profile")))
|
||||
.trim();
|
||||
} catch {
|
||||
return root;
|
||||
}
|
||||
if (name === "" || name === "default") return root;
|
||||
const canonical = name.toLowerCase();
|
||||
if (canonical === "default") return root;
|
||||
if (
|
||||
!/^[a-z0-9][a-z0-9_-]{0,63}$/u.test(canonical) ||
|
||||
RESERVED_PROFILES.has(canonical)
|
||||
)
|
||||
throw new HermesClientPathError(
|
||||
join(stickyRoot, "active_profile"),
|
||||
`Invalid saved Hermes profile ${JSON.stringify(name)}. Run 'hermes profile use default' or repair active_profile.`,
|
||||
);
|
||||
const selected = join(root, "profiles", canonical);
|
||||
if (!liveProfile(selected))
|
||||
throw new HermesClientPathError(
|
||||
selected,
|
||||
`Saved Hermes profile ${JSON.stringify(name)} is missing, deleted, or has no profile identity. Run 'hermes profile use default' or restore that profile before configuring it.`,
|
||||
);
|
||||
return selected;
|
||||
};
|
||||
@@ -0,0 +1,33 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { createProgressReporter } from "./ProgressReporter.js";
|
||||
|
||||
it("limits intermediate sends to 100 ms and always delivers terminal status", async () => {
|
||||
let now = 0;
|
||||
const sent: number[] = [];
|
||||
const reporter = createProgressReporter(
|
||||
async () => {
|
||||
sent.push(now);
|
||||
},
|
||||
{ now: () => now },
|
||||
);
|
||||
for (const [completed, elapsed] of [0, 25, 50, 99, 100, 199, 200].entries()) {
|
||||
now = elapsed;
|
||||
await reporter.report({
|
||||
phase: "running",
|
||||
completed,
|
||||
total: null,
|
||||
message: "Observed capture status",
|
||||
});
|
||||
}
|
||||
expect(sent).toEqual([0, 100, 200]);
|
||||
now = 201;
|
||||
await reporter.report({
|
||||
phase: "cleanup",
|
||||
completed: 7,
|
||||
total: null,
|
||||
message: "Owned cleanup complete",
|
||||
terminal: true,
|
||||
});
|
||||
expect(sent).toEqual([0, 100, 200, 201]);
|
||||
});
|
||||
@@ -1,6 +1,10 @@
|
||||
import { dirname, isAbsolute, join, resolve, win32 } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { lstatSync } from "node:fs";
|
||||
import {
|
||||
HermesClientPathError,
|
||||
resolveHermesClientHome,
|
||||
} from "./HermesClientHome.js";
|
||||
|
||||
/** One supported client configuration location. */
|
||||
export interface SetupClient {
|
||||
@@ -115,49 +119,8 @@ const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => {
|
||||
const grokDirectory = ({ home, env }: ClientPathContext): string =>
|
||||
env.GROK_HOME || join(home, ".grok");
|
||||
|
||||
/**
|
||||
* Hermes Agent home. `HERMES_HOME` wins; otherwise the platform default mirrors
|
||||
* Hermes's own resolver (`%LOCALAPPDATA%\\hermes` on Windows, `~/.hermes`
|
||||
* elsewhere), including the optional `HERMES_DATA_DIR_SUFFIX` that is appended
|
||||
* to the literal `hermes`/`.hermes` directory name.
|
||||
*/
|
||||
const hermesDirectory = ({
|
||||
home,
|
||||
platform,
|
||||
env,
|
||||
}: ClientPathContext): string => {
|
||||
const override = env.HERMES_HOME?.trim();
|
||||
if (override) {
|
||||
const expandedVariables = override.replace(
|
||||
/\$(\w+|\{[^}]*\})/gu,
|
||||
(match, variable: string) => {
|
||||
const key = variable.startsWith("{") ? variable.slice(1, -1) : variable;
|
||||
return env[key] ?? match;
|
||||
},
|
||||
);
|
||||
const expanded =
|
||||
platform === "win32"
|
||||
? expandedVariables.replace(
|
||||
/%([^%]+)%/gu,
|
||||
(match, variable: string) => env[variable] ?? match,
|
||||
)
|
||||
: expandedVariables;
|
||||
if (expanded === "~") return home;
|
||||
if (
|
||||
expanded.startsWith("~/") ||
|
||||
(platform === "win32" && expanded.startsWith("~\\"))
|
||||
)
|
||||
return join(home, expanded.slice(2));
|
||||
return resolve(expanded);
|
||||
}
|
||||
const suffix = env.HERMES_DATA_DIR_SUFFIX ?? "";
|
||||
return platform === "win32"
|
||||
? join(
|
||||
env.LOCALAPPDATA?.trim() || join(home, "AppData", "Local"),
|
||||
`hermes${suffix}`,
|
||||
)
|
||||
: join(home, `.hermes${suffix}`);
|
||||
};
|
||||
const hermesDirectory = ({ home, platform, env }: ClientPathContext): string =>
|
||||
resolveHermesClientHome(home, platform, env);
|
||||
|
||||
/** Grok Bot uses an absolute SAND_DATA_ROOT; anything else stays ~/.grokbot. */
|
||||
const grokBotDirectory = ({ home, env }: ClientPathContext): string => {
|
||||
@@ -577,6 +540,14 @@ export const supportedClients = (
|
||||
format: definition.format,
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof HermesClientPathError)
|
||||
return {
|
||||
name: definition.name,
|
||||
displayName: definition.displayName,
|
||||
format: definition.format,
|
||||
configPath: cause.path,
|
||||
configPathError: cause.message,
|
||||
};
|
||||
if (definition.name !== "pi") throw cause;
|
||||
return {
|
||||
name: definition.name,
|
||||
@@ -611,14 +582,21 @@ export const clientSkillDirectories = (
|
||||
const destinations = new Map<string, { client: string; directory: string }>();
|
||||
for (const client of selected) {
|
||||
const definition = definitions.find(({ name }) => name === client);
|
||||
const directory =
|
||||
definition?.skillPath === undefined
|
||||
? join(home, ".agents", "skills")
|
||||
: resolvePath(definition.skillPath, {
|
||||
home,
|
||||
platform,
|
||||
env: environment,
|
||||
});
|
||||
let directory: string;
|
||||
try {
|
||||
directory =
|
||||
definition?.skillPath === undefined
|
||||
? join(home, ".agents", "skills")
|
||||
: resolvePath(definition.skillPath, {
|
||||
home,
|
||||
platform,
|
||||
env: environment,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
// Registration diagnostics own an unresolved profile; never fall back to another skill root.
|
||||
if (cause instanceof HermesClientPathError) continue;
|
||||
throw cause;
|
||||
}
|
||||
if (!destinations.has(directory))
|
||||
destinations.set(directory, {
|
||||
client:
|
||||
|
||||
@@ -42,6 +42,46 @@ describe("JavaScript analysis cancellation before publication", () => {
|
||||
expect(phases).not.toContain("validate_javascript_application_result");
|
||||
});
|
||||
|
||||
it("handles control turns and cancellation inside one large source file (#1462)", async () => {
|
||||
const root = await createTestTempDirectory("rea-js-cancel-in-file-");
|
||||
await writeFile(
|
||||
join(root, "index.js"),
|
||||
Array.from(
|
||||
{ length: 1200 },
|
||||
(_, index) =>
|
||||
`export function f${index}(){return {name:'f${index}',value:${index}};}`,
|
||||
).join("\n"),
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const phases: string[] = [];
|
||||
let turns = 0;
|
||||
// Count event-loop turns once the file starts; parsing, analysis and
|
||||
// projection of this one file must leave room for them.
|
||||
const tick = (): void => {
|
||||
turns += 1;
|
||||
if (turns === 3) controller.abort();
|
||||
else if (!controller.signal.aborted) setImmediate(tick);
|
||||
};
|
||||
const result = await analyzeJavaScriptApplication(
|
||||
{ input_path: root, format: "directory" },
|
||||
{
|
||||
signal: controller.signal,
|
||||
progress: {
|
||||
async report(event) {
|
||||
phases.push(event.phase);
|
||||
if (event.phase === "parse_javascript_source") setImmediate(tick);
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
if (result.ok) throw new Error("Cancelled analysis must not publish");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "cancelled",
|
||||
});
|
||||
expect(phases).toContain("parse_javascript_source");
|
||||
expect(phases).not.toContain("build_javascript_application_graph");
|
||||
});
|
||||
|
||||
it.each([
|
||||
"parse_javascript_source",
|
||||
"build_javascript_application_graph",
|
||||
|
||||
@@ -4,7 +4,7 @@ import { setImmediate } from "node:timers/promises";
|
||||
import { resolveJavaScriptSourceMapReference } from "../../domain/javascript/javascriptSourceMapPaths.js";
|
||||
import { analyzeParsedJavaScriptStaticSource } from "../../domain/javascript/javascriptStaticAnalysis.js";
|
||||
import {
|
||||
analyzeParsedJavaScriptSemantics,
|
||||
analyzeParsedJavaScriptSemanticsSteps,
|
||||
classifyParsedJavaScriptOpenReceivers,
|
||||
} from "../../domain/javascript/javascriptSemanticAnalysis.js";
|
||||
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
|
||||
@@ -33,6 +33,7 @@ import type {
|
||||
JavaScriptSourceMapOriginal,
|
||||
} from "./JavaScriptArtifactAnalysisTypes.js";
|
||||
import { analyzeJavaScriptJsonModule } from "./JavaScriptJsonModules.js";
|
||||
import { completeJavaScriptAnalysisSteps } from "./JavaScriptAnalysisControl.js";
|
||||
import { htmlArtifactReferences } from "../../domain/javascript/htmlArtifactReferences.js";
|
||||
|
||||
interface MutableArtifactAnalysis<
|
||||
@@ -56,34 +57,44 @@ interface ArtifactAnalysisContext<
|
||||
readonly projectSemantics: (
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
) => SemanticIr;
|
||||
) => Generator<void, SemanticIr>;
|
||||
}
|
||||
|
||||
/** Analyze every text source from the artifact without a result quota. */
|
||||
export const analyzeJavaScriptArtifactFiles = (
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
): JavaScriptArtifactAnalysis =>
|
||||
analyzeArtifactFiles(fileSet, (_file, ir) => ir);
|
||||
analyzeArtifactFiles(fileSet, function* (_file, ir) {
|
||||
// The full IR needs no projection; one step keeps the shared protocol.
|
||||
yield;
|
||||
return ir;
|
||||
});
|
||||
|
||||
/** Project each file's full IR immediately, retaining only later module facts. */
|
||||
/**
|
||||
* Project each file's full IR immediately, retaining only later module facts.
|
||||
* Parsing, analysis and each projection pass run as separate steps, so control
|
||||
* messages and cancellation are handled within one large file as well.
|
||||
*/
|
||||
export const analyzeAndProjectJavaScriptArtifactFiles = async (
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
projectSemantics: (
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
) => void,
|
||||
) => Iterator<void, void> | void,
|
||||
beforeFile?: (
|
||||
file: JavaScriptArtifactFile,
|
||||
completed: number,
|
||||
total: number,
|
||||
) => Promise<void>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JavaScriptModuleArtifactAnalysis> => {
|
||||
const state = emptyArtifactAnalysis<JavaScriptModuleSemanticIr>();
|
||||
const projectFileSemantics = (
|
||||
const projectFileSemantics = function* (
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
): JavaScriptModuleSemanticIr => {
|
||||
projectSemantics(file, ir);
|
||||
): Generator<void, JavaScriptModuleSemanticIr> {
|
||||
const projection = projectSemantics(file, ir);
|
||||
if (projection !== undefined) while (projection.next().done !== true) yield;
|
||||
const programScopes = ir.scopes.filter(({ kind }) => kind === "program");
|
||||
const programScopeId = programScopes[0]?.scopeId;
|
||||
const localNames = new Set(
|
||||
@@ -111,7 +122,10 @@ export const analyzeAndProjectJavaScriptArtifactFiles = async (
|
||||
// Let progress, cancellation and garbage collection run between files.
|
||||
await setImmediate();
|
||||
await beforeFile?.(file, index, fileSet.files.length);
|
||||
analyzeArtifactFile(file, context);
|
||||
await completeJavaScriptAnalysisSteps(
|
||||
analyzeArtifactFileSteps(file, context),
|
||||
signal,
|
||||
);
|
||||
}
|
||||
return finalizeArtifactAnalysis(state);
|
||||
};
|
||||
@@ -122,7 +136,10 @@ const analyzeArtifactFiles = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||
): JavaScriptArtifactAnalysis<SemanticIr> => {
|
||||
const state = emptyArtifactAnalysis<SemanticIr>();
|
||||
const context = { state, projectSemantics };
|
||||
for (const file of fileSet.files) analyzeArtifactFile(file, context);
|
||||
for (const file of fileSet.files) {
|
||||
const steps = analyzeArtifactFileSteps(file, context);
|
||||
while (steps.next().done !== true);
|
||||
}
|
||||
return finalizeArtifactAnalysis(state);
|
||||
};
|
||||
|
||||
@@ -148,10 +165,12 @@ const finalizeArtifactAnalysis = <
|
||||
};
|
||||
};
|
||||
|
||||
const analyzeArtifactFile = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||
function* analyzeArtifactFileSteps<
|
||||
SemanticIr extends JavaScriptModuleSemanticIr,
|
||||
>(
|
||||
file: JavaScriptArtifactFile,
|
||||
context: ArtifactAnalysisContext<SemanticIr>,
|
||||
): void => {
|
||||
): Generator<void, void> {
|
||||
const { state } = context;
|
||||
addStructuredObservations(file, state);
|
||||
if (file.kind === "html" && file.text.included)
|
||||
@@ -162,6 +181,7 @@ const analyzeArtifactFile = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||
return;
|
||||
}
|
||||
const parsed = parseJavaScriptSource(file.text.value, file.path);
|
||||
yield;
|
||||
if (parsed === null) {
|
||||
const analysis = failedJavaScriptStaticAnalysis();
|
||||
state.files.push({ file, javascript: analysis, semantic: null });
|
||||
@@ -169,23 +189,26 @@ const analyzeArtifactFile = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||
return;
|
||||
}
|
||||
const openReceiverFacts = classifyParsedJavaScriptOpenReceivers(parsed);
|
||||
yield;
|
||||
const analysis = analyzeParsedJavaScriptStaticSource(
|
||||
file.text.value,
|
||||
parsed,
|
||||
openReceiverFacts,
|
||||
);
|
||||
const staticFindings = findingCount(analysis);
|
||||
yield;
|
||||
const semantics =
|
||||
analysis.parse_status === "complete" || analysis.parse_status === "partial"
|
||||
? analyzeParsedJavaScriptSemantics(parsed)
|
||||
? yield* analyzeParsedJavaScriptSemanticsSteps(parsed)
|
||||
: null;
|
||||
const projected =
|
||||
semantics === null
|
||||
? null
|
||||
: yield* context.projectSemantics(file, semantics);
|
||||
state.files.push({
|
||||
file,
|
||||
javascript: analysis,
|
||||
semantic:
|
||||
semantics === null
|
||||
? null
|
||||
: { ir: context.projectSemantics(file, semantics) },
|
||||
semantic: projected === null ? null : { ir: projected },
|
||||
});
|
||||
state.visitedNodes += analysis.visited_ast_nodes;
|
||||
state.findings += staticFindings + (semantics?.moduleLinks.length ?? 0);
|
||||
@@ -194,7 +217,7 @@ const analyzeArtifactFile = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||
0,
|
||||
);
|
||||
if (analysis.parse_status === "failed") state.parseFailures += 1;
|
||||
};
|
||||
}
|
||||
|
||||
const addSourceMap = (
|
||||
file: JavaScriptArtifactFile,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import {
|
||||
createJavaScriptApplicationGraph,
|
||||
createImmutableJavaScriptApplicationGraphSteps,
|
||||
sealTransferredJavaScriptApplicationGraphSteps,
|
||||
type JavaScriptApplicationGraph,
|
||||
} from "../../domain/javascript/javascriptApplicationGraph.js";
|
||||
import type { JavaScriptModuleArtifactAnalysis } from "./JavaScriptArtifactAnalysisTypes.js";
|
||||
@@ -15,9 +15,9 @@ import {
|
||||
addJavaScriptHtmlRoles,
|
||||
addJavaScriptSourceMapOriginals,
|
||||
} from "./JavaScriptArtifactGraphDocuments.js";
|
||||
import { addJavaScriptStaticFindings } from "./JavaScriptArtifactGraphFindings.js";
|
||||
import { addJavaScriptStaticFindingsSteps } from "./JavaScriptArtifactGraphFindings.js";
|
||||
import {
|
||||
addJavaScriptModuleRelationships,
|
||||
addJavaScriptModuleRelationshipsSteps,
|
||||
type JavaScriptModuleRelationshipOmissions,
|
||||
addJavaScriptSourceModules,
|
||||
} from "./JavaScriptModuleRelationships.js";
|
||||
@@ -51,24 +51,42 @@ export const buildJavaScriptArtifactGraph = (
|
||||
analysis: JavaScriptModuleArtifactAnalysis,
|
||||
): JavaScriptApplicationGraph =>
|
||||
createJavaScriptApplicationGraph(
|
||||
buildJavaScriptArtifactGraphInput(snapshot, fileSet, analysis),
|
||||
completeSynchronously(
|
||||
buildJavaScriptArtifactGraphInputSteps(snapshot, fileSet, analysis),
|
||||
),
|
||||
);
|
||||
|
||||
/** Build a validated application graph with cooperative immutable ownership. */
|
||||
export const buildImmutableJavaScriptArtifactGraphSteps = (
|
||||
/**
|
||||
* Build a validated application graph with cooperative immutable ownership.
|
||||
* Constructing the graph input also runs in steps: projecting every module
|
||||
* relationship and finding at once held the event loop for seconds.
|
||||
*/
|
||||
export function* buildImmutableJavaScriptArtifactGraphSteps(
|
||||
snapshot: ArtifactInventorySnapshot,
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
analysis: JavaScriptModuleArtifactAnalysis,
|
||||
): Generator<void, JavaScriptApplicationGraph> =>
|
||||
createImmutableJavaScriptApplicationGraphSteps(
|
||||
buildJavaScriptArtifactGraphInput(snapshot, fileSet, analysis),
|
||||
): Generator<void, JavaScriptApplicationGraph> {
|
||||
const input = yield* buildJavaScriptArtifactGraphInputSteps(
|
||||
snapshot,
|
||||
fileSet,
|
||||
analysis,
|
||||
);
|
||||
// The input was built here and is not retained, so it can be transferred.
|
||||
return yield* sealTransferredJavaScriptApplicationGraphSteps(input);
|
||||
}
|
||||
|
||||
const buildJavaScriptArtifactGraphInput = (
|
||||
const completeSynchronously = <Value>(steps: Iterator<void, Value>): Value => {
|
||||
for (;;) {
|
||||
const step = steps.next();
|
||||
if (step.done === true) return step.value;
|
||||
}
|
||||
};
|
||||
|
||||
function* buildJavaScriptArtifactGraphInputSteps(
|
||||
snapshot: ArtifactInventorySnapshot,
|
||||
fileSet: JavaScriptArtifactFileSet,
|
||||
analysis: JavaScriptModuleArtifactAnalysis,
|
||||
): unknown => {
|
||||
): Generator<void, unknown> {
|
||||
const accumulator = new JavaScriptArtifactGraphAccumulator();
|
||||
const root = createJavaScriptArtifactRootNode(accumulator, snapshot);
|
||||
const context: JavaScriptArtifactGraphContext = {
|
||||
@@ -86,15 +104,23 @@ const buildJavaScriptArtifactGraphInput = (
|
||||
containerNodes: new Map([[snapshot.manifest.root_sha256, root]]),
|
||||
};
|
||||
addJavaScriptArtifactContainers(context);
|
||||
yield;
|
||||
addJavaScriptArtifactFiles(context);
|
||||
yield;
|
||||
const packageRoots = addJavaScriptPackageNodes(context);
|
||||
yield;
|
||||
addJavaScriptSourceModules(context);
|
||||
yield;
|
||||
const bundlerLimitations = addJavaScriptBundlerNodes(context);
|
||||
const relationshipOmissions = addJavaScriptModuleRelationships(context);
|
||||
const findingLimitations = addJavaScriptStaticFindings(context);
|
||||
const relationshipOmissions =
|
||||
yield* addJavaScriptModuleRelationshipsSteps(context);
|
||||
const findingLimitations = yield* addJavaScriptStaticFindingsSteps(context);
|
||||
addElectronBoundaries(context);
|
||||
yield;
|
||||
addJavaScriptHtmlRoles(context);
|
||||
yield;
|
||||
addJavaScriptSourceMapOriginals(context);
|
||||
yield;
|
||||
const coverage = graphCoverage(context);
|
||||
return {
|
||||
schema: "JavaScriptApplicationGraph",
|
||||
@@ -111,7 +137,7 @@ const buildJavaScriptArtifactGraphInput = (
|
||||
...graphLimitations(context, coverage.status, relationshipOmissions),
|
||||
],
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
const graphCoverage = (context: JavaScriptArtifactGraphContext) => {
|
||||
const resourceLimits = semanticResourceLimits(context);
|
||||
|
||||
@@ -44,11 +44,12 @@ interface ResolvedReference {
|
||||
}
|
||||
|
||||
/** Project imports, workers, roles, endpoints, storage, and source-map links. */
|
||||
export const addJavaScriptStaticFindings = (
|
||||
export function* addJavaScriptStaticFindingsSteps(
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): readonly string[] => {
|
||||
): Generator<void, readonly string[]> {
|
||||
const omissions = { selfReferences: 0 };
|
||||
for (const analyzed of context.analysis.files) {
|
||||
yield;
|
||||
const { file, javascript } = analyzed;
|
||||
const asset = context.assetNodes.get(file.path);
|
||||
if (javascript === null || asset === undefined) continue;
|
||||
@@ -69,7 +70,7 @@ export const addJavaScriptStaticFindings = (
|
||||
["static reference", "static references"],
|
||||
"the referencing module",
|
||||
);
|
||||
};
|
||||
}
|
||||
|
||||
const addReference = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
|
||||
@@ -90,7 +90,7 @@ export const reconstructJavaScriptArtifact = async (
|
||||
const semanticProjection = createJavaScriptSemanticGraphProjection();
|
||||
const analysis = await analyzeAndProjectJavaScriptArtifactFiles(
|
||||
files,
|
||||
semanticProjection.projectFile,
|
||||
semanticProjection.projectFileSteps,
|
||||
async (file, completed, total) => {
|
||||
abortIfNeeded(signal);
|
||||
await progress?.report({
|
||||
@@ -101,6 +101,7 @@ export const reconstructJavaScriptArtifact = async (
|
||||
});
|
||||
abortIfNeeded(signal);
|
||||
},
|
||||
signal,
|
||||
);
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
abortIfNeeded(signal);
|
||||
|
||||
@@ -136,15 +136,17 @@ export interface JavaScriptModuleRelationshipOmissions {
|
||||
type RelationshipOmissionCounter = { selfImports: number };
|
||||
|
||||
/** Compose bounded CommonJS and ESM binding relationships across artifact files. */
|
||||
export const addJavaScriptModuleRelationships = (
|
||||
export function* addJavaScriptModuleRelationshipsSteps(
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): JavaScriptModuleRelationshipOmissions => {
|
||||
): Generator<void, JavaScriptModuleRelationshipOmissions> {
|
||||
const omissions = { selfImports: 0 };
|
||||
for (const analyzed of context.analysis.files) {
|
||||
const { file, semantic } = analyzed;
|
||||
const source = context.sourceModuleNodes.get(file.path);
|
||||
if (semantic === null || source === undefined) continue;
|
||||
for (const link of semantic.ir.moduleLinks) {
|
||||
// One module can export thousands of bindings, each with return shapes.
|
||||
yield;
|
||||
const input = { context, file, semantic, source, link };
|
||||
if (isExportLink(link)) addExportRelationship(input, omissions);
|
||||
else if (link.specifier !== null)
|
||||
@@ -155,7 +157,7 @@ export const addJavaScriptModuleRelationships = (
|
||||
}
|
||||
}
|
||||
return omissions;
|
||||
};
|
||||
}
|
||||
|
||||
const addExportRelationship = (
|
||||
input: RelationshipInput,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import {
|
||||
createJavaScriptSemanticGraph,
|
||||
createImmutableJavaScriptSemanticGraphSteps,
|
||||
sealTransferredJavaScriptSemanticGraphSteps,
|
||||
type JavaScriptSemanticGraph,
|
||||
} from "../../domain/javascript/javascriptSemanticGraph.js";
|
||||
import type {
|
||||
@@ -130,6 +130,11 @@ export interface JavaScriptSemanticGraphProjection {
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
) => void;
|
||||
/** Project one file in steps between its projection passes. */
|
||||
readonly projectFileSteps: (
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
) => Generator<void, void>;
|
||||
readonly finish: (
|
||||
rootArtifactSha256: string,
|
||||
applicationGraph: BuilderInput["applicationGraph"],
|
||||
@@ -146,10 +151,10 @@ export const createJavaScriptSemanticGraphProjection =
|
||||
const state = emptyState({ nodes: [] });
|
||||
const fingerprints: JavaScriptSemanticFingerprint[] = [];
|
||||
let truncatedFiles = 0;
|
||||
const projectSource = (
|
||||
const projectSourceSteps = function* (
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
): void => {
|
||||
): Generator<void, void> {
|
||||
if (state.nodes.size >= SEMANTIC_GRAPH_NODE_CEILING) {
|
||||
truncatedFiles += 1;
|
||||
return;
|
||||
@@ -161,7 +166,7 @@ export const createJavaScriptSemanticGraphProjection =
|
||||
remainingTreeBudget,
|
||||
);
|
||||
state.fileNodesDropped = false;
|
||||
fingerprints.push(...projectFile(file, ir, state));
|
||||
fingerprints.push(...(yield* projectFileSteps(file, ir, state)));
|
||||
// Nodes were dropped only if the budget actually blocked creation. A file
|
||||
// that exactly fills its share ends with a zero budget without dropping
|
||||
// anything, so the remaining budget alone cannot decide truncation.
|
||||
@@ -198,7 +203,8 @@ export const createJavaScriptSemanticGraphProjection =
|
||||
evidence_contexts: state.evidenceContexts.contexts,
|
||||
nodes: [...state.nodes.values()],
|
||||
relations,
|
||||
fingerprints,
|
||||
// The input is transferred to the factory; `fingerprints` is cleared below.
|
||||
fingerprints: [...fingerprints],
|
||||
unknowns,
|
||||
coverage: {
|
||||
status: truncatedFiles > 0 ? "partial" : "unknown",
|
||||
@@ -246,7 +252,11 @@ export const createJavaScriptSemanticGraphProjection =
|
||||
return graph;
|
||||
};
|
||||
return {
|
||||
projectFile: projectSource,
|
||||
projectFile: (file, ir) => {
|
||||
const steps = projectSourceSteps(file, ir);
|
||||
while (steps.next().done !== true);
|
||||
},
|
||||
projectFileSteps: projectSourceSteps,
|
||||
finish: (rootArtifactSha256, applicationGraph) =>
|
||||
finishWith(
|
||||
createJavaScriptSemanticGraph,
|
||||
@@ -255,18 +265,19 @@ export const createJavaScriptSemanticGraphProjection =
|
||||
),
|
||||
finishImmutableSteps: (rootArtifactSha256, applicationGraph) =>
|
||||
finishWith(
|
||||
createImmutableJavaScriptSemanticGraphSteps,
|
||||
sealTransferredJavaScriptSemanticGraphSteps,
|
||||
rootArtifactSha256,
|
||||
applicationGraph,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
const projectFile = (
|
||||
function* projectFileSteps(
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
state: BuilderState,
|
||||
): JavaScriptSemanticFingerprint[] => {
|
||||
): Generator<void, JavaScriptSemanticFingerprint[]> {
|
||||
yield;
|
||||
const moduleNode = retainNode(state, file, {
|
||||
kind: "module",
|
||||
roleKey: "module",
|
||||
@@ -276,38 +287,50 @@ const projectFile = (
|
||||
});
|
||||
if (moduleNode === null) return [];
|
||||
state.roots.add(moduleNode.node_id);
|
||||
const callableNodes = new Map(
|
||||
ir.callables.flatMap((callable) => {
|
||||
const node = retainNode(state, file, {
|
||||
kind: "function",
|
||||
roleKey: `callable:${callable.callableId}`,
|
||||
location: callable.location,
|
||||
label: callable.name,
|
||||
functionNodeId: null,
|
||||
// The label is display text; keep the exact name, which may be "".
|
||||
properties: { name: callable.name },
|
||||
});
|
||||
return node === null ? [] : [[callable.callableId, node] as const];
|
||||
}),
|
||||
);
|
||||
// Node creation keeps source order, which the per-file node budget relies
|
||||
// on; yields between nodes do not change it.
|
||||
const callableNodes = new Map<string, JavaScriptSemanticGraphNode>();
|
||||
for (const [index, callable] of ir.callables.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const node = retainNode(state, file, {
|
||||
kind: "function",
|
||||
roleKey: `callable:${callable.callableId}`,
|
||||
location: callable.location,
|
||||
label: callable.name,
|
||||
functionNodeId: null,
|
||||
// The label is display text; keep the exact name, which may be "".
|
||||
properties: { name: callable.name },
|
||||
});
|
||||
if (node !== null) callableNodes.set(callable.callableId, node);
|
||||
}
|
||||
const callableOwnerAt = createSemanticCallableOwnerLookup(ir, callableNodes);
|
||||
const bindingNodes = new Map(
|
||||
ir.bindings.flatMap((binding) => {
|
||||
const location = binding.definitions[0]?.location ?? null;
|
||||
const kind = binding.kind === "parameter" ? "parameter" : "binding";
|
||||
const owner = callableOwnerAt(location);
|
||||
const node = retainNode(state, file, {
|
||||
kind,
|
||||
roleKey: `binding:${binding.bindingId}`,
|
||||
location,
|
||||
label: binding.name,
|
||||
functionNodeId: owner?.node_id ?? null,
|
||||
});
|
||||
return node === null ? [] : [[binding.bindingId, node] as const];
|
||||
}),
|
||||
const bindingNodes = new Map<string, JavaScriptSemanticGraphNode>();
|
||||
for (const [index, binding] of ir.bindings.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const location = binding.definitions[0]?.location ?? null;
|
||||
const kind = binding.kind === "parameter" ? "parameter" : "binding";
|
||||
const owner = callableOwnerAt(location);
|
||||
const node = retainNode(state, file, {
|
||||
kind,
|
||||
roleKey: `binding:${binding.bindingId}`,
|
||||
location,
|
||||
label: binding.name,
|
||||
functionNodeId: owner?.node_id ?? null,
|
||||
});
|
||||
if (node !== null) bindingNodes.set(binding.bindingId, node);
|
||||
}
|
||||
const returnSiteNodes = yield* createReturnSiteNodes(
|
||||
file,
|
||||
ir,
|
||||
callableNodes,
|
||||
state,
|
||||
);
|
||||
const callSiteNodes = yield* createCallSiteNodes(
|
||||
file,
|
||||
ir,
|
||||
callableNodes,
|
||||
state,
|
||||
);
|
||||
const returnSiteNodes = createReturnSiteNodes(file, ir, callableNodes, state);
|
||||
const callSiteNodes = createCallSiteNodes(file, ir, callableNodes, state);
|
||||
const definitions: Omit<FileContext, "referenceNodesWithin"> = {
|
||||
file,
|
||||
ir,
|
||||
@@ -328,42 +351,38 @@ const projectFile = (
|
||||
callableOwnerAt,
|
||||
callSiteAt: createSemanticCallSiteLookup(ir, callSiteNodes),
|
||||
};
|
||||
projectDefinitionsAndReferences(definitions);
|
||||
yield* projectDefinitionsAndReferences(definitions);
|
||||
const context: FileContext = {
|
||||
...definitions,
|
||||
referenceNodesWithin: createSemanticNodeRangeLookup(
|
||||
definitions.referenceNodes,
|
||||
),
|
||||
};
|
||||
projectSemanticValues(context);
|
||||
projectSemanticObjects(context);
|
||||
projectCalls(context);
|
||||
projectSemanticPromises(context);
|
||||
projectSemanticEvents(context);
|
||||
projectSemanticTimers(context);
|
||||
projectSemanticChildProcesses(context);
|
||||
projectSemanticConfiguration(context);
|
||||
projectSemanticRequests(context);
|
||||
projectSemanticBoundaries(context);
|
||||
projectSemanticResources(context);
|
||||
projectSemanticClosureCaptures(context);
|
||||
projectSemanticFrontiers(context);
|
||||
// Each pass completes before the next starts, so yielding between passes
|
||||
// keeps projection order and results unchanged.
|
||||
for (const project of FILE_PROJECTION_PASSES) {
|
||||
yield;
|
||||
const steps = project(context);
|
||||
if (steps !== undefined) while (steps.next().done !== true) yield;
|
||||
}
|
||||
yield;
|
||||
return projectSemanticFunctionFingerprints(
|
||||
file,
|
||||
ir,
|
||||
callableNodes,
|
||||
state.evidenceContexts,
|
||||
);
|
||||
};
|
||||
}
|
||||
|
||||
const createReturnSiteNodes = (
|
||||
function* createReturnSiteNodes(
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
callables: ReadonlyMap<string, JavaScriptSemanticGraphNode>,
|
||||
state: BuilderState,
|
||||
): Map<string, JavaScriptSemanticGraphNode> => {
|
||||
): Generator<void, Map<string, JavaScriptSemanticGraphNode>> {
|
||||
const result = new Map<string, JavaScriptSemanticGraphNode>();
|
||||
for (const callable of ir.callables) {
|
||||
for (const [index, callable] of ir.callables.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const owner = callables.get(callable.callableId);
|
||||
if (owner === undefined) continue;
|
||||
for (const site of callable.returnSites) {
|
||||
@@ -379,16 +398,17 @@ const createReturnSiteNodes = (
|
||||
}
|
||||
}
|
||||
return result;
|
||||
};
|
||||
}
|
||||
|
||||
const createCallSiteNodes = (
|
||||
function* createCallSiteNodes(
|
||||
file: JavaScriptArtifactFile,
|
||||
ir: JavaScriptSemanticIr,
|
||||
callables: ReadonlyMap<string, JavaScriptSemanticGraphNode>,
|
||||
state: BuilderState,
|
||||
): Map<string, JavaScriptSemanticGraphNode> => {
|
||||
): Generator<void, Map<string, JavaScriptSemanticGraphNode>> {
|
||||
const result = new Map<string, JavaScriptSemanticGraphNode>();
|
||||
for (const call of ir.callSites) {
|
||||
for (const [index, call] of ir.callSites.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const owner =
|
||||
call.callerCallableId === null
|
||||
? null
|
||||
@@ -403,12 +423,13 @@ const createCallSiteNodes = (
|
||||
if (node !== null) result.set(call.callSiteId, node);
|
||||
}
|
||||
return result;
|
||||
};
|
||||
}
|
||||
|
||||
const projectDefinitionsAndReferences = (
|
||||
function* projectDefinitionsAndReferences(
|
||||
context: Omit<FileContext, "referenceNodesWithin">,
|
||||
): void => {
|
||||
for (const binding of context.ir.bindings) {
|
||||
): Generator<void, void> {
|
||||
for (const [bindingIndex, binding] of context.ir.bindings.entries()) {
|
||||
if (bindingIndex % 64 === 0) yield;
|
||||
const bindingNode = context.bindingNodes.get(binding.bindingId);
|
||||
if (bindingNode === undefined) continue;
|
||||
for (const [index, definition] of binding.definitions.entries()) {
|
||||
@@ -428,6 +449,7 @@ const projectDefinitionsAndReferences = (
|
||||
}
|
||||
}
|
||||
for (const [index, reference] of context.ir.references.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const expression = retainNode(context.state, context.file, {
|
||||
kind: "expression",
|
||||
roleKey: `reference:${String(index)}:${reference.role}:${reference.name}`,
|
||||
@@ -458,7 +480,7 @@ const projectDefinitionsAndReferences = (
|
||||
resolution: "resolved",
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const projectCalls = (context: FileContext): void => {
|
||||
for (const call of context.ir.callSites) {
|
||||
@@ -543,3 +565,22 @@ const createArgumentNode = (
|
||||
: `argument ${String(argument.index)}`,
|
||||
functionNodeId: callNode.function_node_id,
|
||||
});
|
||||
|
||||
/** Per-file projection passes, run in this order with a yield between each. */
|
||||
const FILE_PROJECTION_PASSES: readonly ((
|
||||
context: FileContext,
|
||||
) => Iterator<void, void> | void)[] = [
|
||||
projectSemanticValues,
|
||||
projectSemanticObjects,
|
||||
projectCalls,
|
||||
projectSemanticPromises,
|
||||
projectSemanticEvents,
|
||||
projectSemanticTimers,
|
||||
projectSemanticChildProcesses,
|
||||
projectSemanticConfiguration,
|
||||
projectSemanticRequests,
|
||||
projectSemanticBoundaries,
|
||||
projectSemanticResources,
|
||||
projectSemanticClosureCaptures,
|
||||
projectSemanticFrontiers,
|
||||
];
|
||||
|
||||
@@ -240,10 +240,11 @@ export const projectSemanticClosureCaptures = (
|
||||
};
|
||||
|
||||
/** Retain bounded unresolved dynamic-call, dynamic-property and dynamic-scope frontiers. */
|
||||
export const projectSemanticFrontiers = (
|
||||
export function* projectSemanticFrontiers(
|
||||
context: SemanticFlowProjectionContext,
|
||||
): void => {
|
||||
for (const frontier of context.ir.frontiers) {
|
||||
): Generator<void, void> {
|
||||
for (const [index, frontier] of context.ir.frontiers.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
// Dynamic environments leave identifier reads and writes unresolved.
|
||||
const family =
|
||||
frontier.kind === "dynamic-call"
|
||||
@@ -275,4 +276,4 @@ export const projectSemanticFrontiers = (
|
||||
);
|
||||
addSemanticGraphUnknown(context.state, unknown);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@@ -20,13 +20,14 @@ import type { SemanticFlowProjectionContext } from "./JavaScriptSemanticGraphFlo
|
||||
import { semanticPropertySlot } from "./JavaScriptSemanticGraphValueProjection.js";
|
||||
|
||||
/** Project static object reads, writes, spreads, and destructuring. */
|
||||
export const projectSemanticObjects = (
|
||||
export function* projectSemanticObjects(
|
||||
context: SemanticFlowProjectionContext,
|
||||
): void => {
|
||||
): Generator<void, void> {
|
||||
const values = new Map(
|
||||
context.ir.bindings.map((binding) => [binding.bindingId, binding.value]),
|
||||
);
|
||||
for (const operation of context.ir.objectOperations) {
|
||||
for (const [index, operation] of context.ir.objectOperations.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const occurrence = addObjectOccurrence(context, operation);
|
||||
const target = objectTarget(context, values, operation);
|
||||
const ends = objectRelationEnds(context, operation, occurrence, target);
|
||||
@@ -43,7 +44,7 @@ export const projectSemanticObjects = (
|
||||
if (!resolved)
|
||||
addObjectUnknown(context, operation, target.slot ?? occurrence, target);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const OBJECT_RELATIONS = {
|
||||
read: "reads-property",
|
||||
|
||||
@@ -15,6 +15,7 @@ import { processTraceSpecificationSchema } from "../../domain/process/processTra
|
||||
import { processScenarioSchema } from "../../domain/process/processScenario.js";
|
||||
import { compareProcessCaptures } from "../../domain/process/processComparison.js";
|
||||
import { parseProcessCapture } from "../../domain/process/processCaptureParsing.js";
|
||||
import { createProgressReporter } from "../ProgressReporter.js";
|
||||
import { captureProcessScenario } from "../../process/capture/ProcessHarness.js";
|
||||
import { PROCESS_PROVIDER } from "../../domain/process/processEvidenceProvider.js";
|
||||
import { createProcessCaptureEvidence } from "./ProcessEvidence.js";
|
||||
@@ -56,11 +57,20 @@ export const captureProcessScenarioFile = async (
|
||||
input,
|
||||
{ cause: parsed.error },
|
||||
);
|
||||
const progress = createProgressReporter(
|
||||
async (update) => {
|
||||
process.stderr.write(`${JSON.stringify({ rea_progress: update })}\n`);
|
||||
},
|
||||
{ minimumIntervalMs: 100 },
|
||||
);
|
||||
const captured = await captureProcessScenario(
|
||||
parsed.data,
|
||||
signal,
|
||||
process.platform,
|
||||
environment,
|
||||
undefined,
|
||||
undefined,
|
||||
progress,
|
||||
);
|
||||
if (!captured.ok) return cliAnalysisError(captured.error);
|
||||
return createProcessCaptureEvidence(parsed.data, captured.value);
|
||||
|
||||
@@ -11,11 +11,13 @@ import {
|
||||
createJavaScriptApplicationGraph,
|
||||
createImmutableJavaScriptApplicationGraphSteps,
|
||||
isValidatedImmutableJavaScriptApplicationGraph,
|
||||
sealTransferredJavaScriptApplicationGraphSteps,
|
||||
} from "./javascriptApplicationGraph.js";
|
||||
import {
|
||||
createJavaScriptSemanticGraph,
|
||||
createImmutableJavaScriptSemanticGraphSteps,
|
||||
isValidatedImmutableJavaScriptSemanticGraph,
|
||||
sealTransferredJavaScriptSemanticGraphSteps,
|
||||
} from "./javascriptSemanticGraph.js";
|
||||
|
||||
const complete = <Value>(steps: Iterator<void, Value>): Value => {
|
||||
@@ -265,3 +267,60 @@ describe("owned JavaScript result bindings", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("graphs transferred to their sealing steps", () => {
|
||||
it("seal the same validated graphs as the copying factories", () => {
|
||||
const input = example();
|
||||
const { graph_id: _applicationId, ...application } = input.graph;
|
||||
const { graph_id: _semanticId, ...semantic } = input.semantic_graph;
|
||||
const expectedApplication = createJavaScriptApplicationGraph(application);
|
||||
const expectedSemantic = createJavaScriptSemanticGraph(semantic);
|
||||
const sealedApplication = complete(
|
||||
sealTransferredJavaScriptApplicationGraphSteps(
|
||||
structuredClone(application),
|
||||
),
|
||||
);
|
||||
const sealedSemantic = complete(
|
||||
sealTransferredJavaScriptSemanticGraphSteps(structuredClone(semantic)),
|
||||
);
|
||||
expect(sealedApplication).toEqual(expectedApplication);
|
||||
expect(sealedSemantic).toEqual(expectedSemantic);
|
||||
expect(
|
||||
isValidatedImmutableJavaScriptApplicationGraph(sealedApplication),
|
||||
).toBe(true);
|
||||
expect(isValidatedImmutableJavaScriptSemanticGraph(sealedSemantic)).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("report the issues a whole-graph parse reports", () => {
|
||||
const input = example();
|
||||
const { graph_id: _applicationId, ...application } = input.graph;
|
||||
const { graph_id: _semanticId, ...semantic } = input.semantic_graph;
|
||||
const node = semantic.nodes[0];
|
||||
const applicationNode = application.nodes[0];
|
||||
if (node === undefined || applicationNode === undefined)
|
||||
throw new Error("Expected semantic and application nodes");
|
||||
const malformed = {
|
||||
...semantic,
|
||||
nodes: [{ ...node, label: 7 }, ...semantic.nodes.slice(1)],
|
||||
};
|
||||
expect(
|
||||
issuesFrom(() =>
|
||||
complete(sealTransferredJavaScriptSemanticGraphSteps(malformed)),
|
||||
),
|
||||
).toEqual(issuesFrom(() => createJavaScriptSemanticGraph(malformed)));
|
||||
const stale = {
|
||||
...application,
|
||||
nodes: [
|
||||
{ ...applicationNode, node_id: `jag_node_${"f".repeat(64)}` },
|
||||
...application.nodes.slice(1),
|
||||
],
|
||||
};
|
||||
expect(
|
||||
issuesFrom(() =>
|
||||
complete(sealTransferredJavaScriptApplicationGraphSteps(stale)),
|
||||
),
|
||||
).toEqual(issuesFrom(() => createJavaScriptApplicationGraph(stale)));
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { z } from "zod";
|
||||
import {
|
||||
arrayContainerSchema,
|
||||
parseElementSteps,
|
||||
} from "../zodCollectionSteps.js";
|
||||
import { canonicalJsonDigestSteps } from "../canonicalJsonDigestSteps.js";
|
||||
import { canonicalJson } from "../comparisonSemantics.js";
|
||||
import { digestCanonicalValue } from "../canonicalDigest.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
@@ -93,11 +98,14 @@ type EdgeSemantic = Omit<ApplicationEdge, "edge_id">;
|
||||
const edgeId = (edge: EdgeSemantic): string =>
|
||||
`jag_edge_${digestCanonicalValue(edge, "JavaScript Application Graph")}`;
|
||||
|
||||
/** The issue sink shared by schema refinement and stepwise checking. */
|
||||
type GraphIssueContext = Pick<z.RefinementCtx, "addIssue">;
|
||||
|
||||
const sortedUniqueIssue = (
|
||||
values: readonly string[],
|
||||
path: PropertyKey[],
|
||||
label: string,
|
||||
context: z.RefinementCtx,
|
||||
context: GraphIssueContext,
|
||||
): void => {
|
||||
for (let index = 1; index < values.length; index += 1) {
|
||||
if (
|
||||
@@ -116,7 +124,7 @@ const sortedUniqueIssue = (
|
||||
const checkEvidenceNormalization = (
|
||||
evidence: ApplicationGraphEvidence,
|
||||
path: PropertyKey[],
|
||||
context: z.RefinementCtx,
|
||||
context: GraphIssueContext,
|
||||
): void => {
|
||||
sortedUniqueIssue(
|
||||
evidence.limitations,
|
||||
@@ -226,7 +234,7 @@ const observationMatchesIdentity = (
|
||||
const checkNode = (
|
||||
node: ApplicationNode,
|
||||
index: number,
|
||||
context: z.RefinementCtx,
|
||||
context: GraphIssueContext,
|
||||
): void => {
|
||||
if (node.node_id !== nodeId(node))
|
||||
context.addIssue({
|
||||
@@ -306,8 +314,16 @@ type GraphRecord = z.infer<typeof javascriptApplicationGraphRecordSchema>;
|
||||
|
||||
const checkGraphContent = (
|
||||
graph: JavaScriptApplicationGraphInput,
|
||||
context: z.RefinementCtx,
|
||||
context: GraphIssueContext,
|
||||
): void => {
|
||||
const steps = checkGraphContentSteps(graph, context);
|
||||
while (steps.next().done !== true);
|
||||
};
|
||||
|
||||
function* checkGraphContentSteps(
|
||||
graph: JavaScriptApplicationGraphInput,
|
||||
context: GraphIssueContext,
|
||||
): Generator<void, void> {
|
||||
sortedUniqueIssue(
|
||||
graph.root_node_ids,
|
||||
["root_node_ids"],
|
||||
@@ -353,9 +369,12 @@ const checkGraphContent = (
|
||||
message: "Root identifier must name a graph node",
|
||||
path: ["root_node_ids"],
|
||||
});
|
||||
for (const [index, node] of graph.nodes.entries())
|
||||
for (const [index, node] of graph.nodes.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
checkNode(node, index, context);
|
||||
}
|
||||
for (const [index, edge] of graph.edges.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const { edge_id: identifier, ...semantic } = edge;
|
||||
if (identifier !== edgeId(semantic))
|
||||
context.addIssue({
|
||||
@@ -394,11 +413,11 @@ const checkGraphContent = (
|
||||
context,
|
||||
);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const checkGraphInvariants = (
|
||||
graph: GraphRecord,
|
||||
context: z.RefinementCtx,
|
||||
context: GraphIssueContext,
|
||||
): void => {
|
||||
checkGraphContent(graph, context);
|
||||
const { graph_id: identifier, ...semantic } = graph;
|
||||
@@ -475,22 +494,9 @@ export const createJavaScriptApplicationEdge = (
|
||||
export const createJavaScriptApplicationGraph = (
|
||||
input: unknown,
|
||||
): JavaScriptApplicationGraph => {
|
||||
const parsed = javascriptApplicationGraphInputSchema.parse(input);
|
||||
const semantic: JavaScriptApplicationGraphInput = {
|
||||
...parsed,
|
||||
root_node_ids: uniqueSorted(parsed.root_node_ids),
|
||||
nodes: [...parsed.nodes].sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.node_id, right.node_id),
|
||||
),
|
||||
edges: [...parsed.edges].sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.edge_id, right.edge_id),
|
||||
),
|
||||
coverage: {
|
||||
...parsed.coverage,
|
||||
limits: normalizeLimits(parsed.coverage.limits),
|
||||
},
|
||||
limitations: uniqueSorted(parsed.limitations),
|
||||
};
|
||||
const semantic = normalizeGraphInput(
|
||||
javascriptApplicationGraphInputSchema.parse(input),
|
||||
);
|
||||
const content = javascriptApplicationGraphContentSchema.parse(semantic);
|
||||
return {
|
||||
...content,
|
||||
@@ -498,6 +504,24 @@ export const createJavaScriptApplicationGraph = (
|
||||
};
|
||||
};
|
||||
|
||||
const normalizeGraphInput = (
|
||||
parsed: JavaScriptApplicationGraphInput,
|
||||
): JavaScriptApplicationGraphInput => ({
|
||||
...parsed,
|
||||
root_node_ids: uniqueSorted(parsed.root_node_ids),
|
||||
nodes: [...parsed.nodes].sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.node_id, right.node_id),
|
||||
),
|
||||
edges: [...parsed.edges].sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.edge_id, right.edge_id),
|
||||
),
|
||||
coverage: {
|
||||
...parsed.coverage,
|
||||
limits: normalizeLimits(parsed.coverage.limits),
|
||||
},
|
||||
limitations: uniqueSorted(parsed.limitations),
|
||||
});
|
||||
|
||||
const validatedImmutableApplicationGraphs = new WeakSet<object>();
|
||||
|
||||
/** Clone and validate synchronously, then return steps to seal the owned graph. */
|
||||
@@ -506,6 +530,65 @@ export const createImmutableJavaScriptApplicationGraphSteps = (
|
||||
): Generator<void, JavaScriptApplicationGraph> =>
|
||||
sealValidatedApplicationGraphSteps(createJavaScriptApplicationGraph(input));
|
||||
|
||||
const applicationGraphShape = javascriptApplicationGraphInputSchema.shape;
|
||||
const applicationGraphContainerSchema =
|
||||
javascriptApplicationGraphInputSchema.extend({
|
||||
nodes: arrayContainerSchema(applicationGraphShape.nodes),
|
||||
edges: arrayContainerSchema(applicationGraphShape.edges),
|
||||
});
|
||||
|
||||
/**
|
||||
* Seal a graph whose input the caller transfers: the caller neither retains nor
|
||||
* mutates any part of it afterwards. Only the graph's own fields are validated
|
||||
* synchronously; nodes and edges are validated, checked, hashed and frozen in
|
||||
* steps, because doing so at once held the event loop for large applications.
|
||||
*/
|
||||
export const sealTransferredJavaScriptApplicationGraphSteps = (
|
||||
input: unknown,
|
||||
): Generator<void, JavaScriptApplicationGraph> =>
|
||||
parseAndSealApplicationGraphSteps(
|
||||
applicationGraphContainerSchema.parse(input),
|
||||
);
|
||||
|
||||
function* parseAndSealApplicationGraphSteps(
|
||||
container: z.output<typeof applicationGraphContainerSchema>,
|
||||
): Generator<void, JavaScriptApplicationGraph> {
|
||||
const issues: z.core.$ZodIssue[] = [];
|
||||
const parsed: JavaScriptApplicationGraphInput = {
|
||||
...container,
|
||||
nodes: yield* parseElementSteps(
|
||||
applicationGraphShape.nodes.element,
|
||||
container.nodes,
|
||||
["nodes"],
|
||||
issues,
|
||||
),
|
||||
edges: yield* parseElementSteps(
|
||||
applicationGraphShape.edges.element,
|
||||
container.edges,
|
||||
["edges"],
|
||||
issues,
|
||||
),
|
||||
};
|
||||
if (issues.length > 0) throw new z.ZodError(issues);
|
||||
const content = normalizeGraphInput(parsed);
|
||||
let invalid = false;
|
||||
yield* checkGraphContentSteps(content, {
|
||||
addIssue: () => {
|
||||
invalid = true;
|
||||
},
|
||||
});
|
||||
// An invalid owned graph is a construction defect; report it with the exact
|
||||
// issues the content schema produces.
|
||||
if (invalid) {
|
||||
javascriptApplicationGraphContentSchema.parse(content);
|
||||
throw new TypeError("Application graph checks disagree with its schema");
|
||||
}
|
||||
return yield* sealValidatedApplicationGraphSteps({
|
||||
...content,
|
||||
graph_id: `jag_${yield* canonicalJsonDigestSteps(content)}`,
|
||||
});
|
||||
}
|
||||
|
||||
function* sealValidatedApplicationGraphSteps(
|
||||
graph: JavaScriptApplicationGraph,
|
||||
): Generator<void, JavaScriptApplicationGraph> {
|
||||
|
||||
@@ -597,11 +597,9 @@ describe("property mutation collection on minified bundles", () => {
|
||||
{ length: 2000 },
|
||||
(_, index) => `source.p${index} = ${index};`,
|
||||
).join(" ");
|
||||
const start = performance.now();
|
||||
const value = resultValue(
|
||||
`const source = { token: "TOKEN" }; ${writes} return [source.token, source.p1999];`,
|
||||
);
|
||||
expect(performance.now() - start).toBeLessThan(2000);
|
||||
expect(value).toMatchObject({
|
||||
status: "array",
|
||||
items: [
|
||||
@@ -610,6 +608,21 @@ describe("property mutation collection on minified bundles", () => {
|
||||
],
|
||||
});
|
||||
}, 30000);
|
||||
|
||||
it("indexes many returned object methods instead of rescanning the object", () => {
|
||||
const methods = Array.from(
|
||||
{ length: 2000 },
|
||||
(_, index) => `m${index}() { return shared; }`,
|
||||
).join(",");
|
||||
const writes = Array.from(
|
||||
{ length: 2000 },
|
||||
(_, index) => `box.m${index}().p${index} = ${index};`,
|
||||
).join(" ");
|
||||
const value = resultValue(
|
||||
`const shared = { token: "TOKEN" }; const box = { ${methods} }; ${writes} return shared.token;`,
|
||||
);
|
||||
expect(value?.status).toBe("unknown");
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
describe("results of methods on an escaped receiver", () => {
|
||||
@@ -628,35 +641,22 @@ describe("results of methods on an escaped receiver", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("expands a receiver's methods once rather than once per call (#1495)", () => {
|
||||
const receiverCalls = (count: number) => {
|
||||
const methods = Array.from(
|
||||
{ length: count },
|
||||
(_, index) => `m${index}() { return shared; },`,
|
||||
).join(" ");
|
||||
const calls = Array.from(
|
||||
{ length: count },
|
||||
(_, index) => `box.m${index}().p${index} = ${index};`,
|
||||
).join(" ");
|
||||
return `const shared = { token: "TOKEN" }; const box = { ${methods} }; ${calls} return [shared.token];`;
|
||||
};
|
||||
const fastest = (body: string) =>
|
||||
Math.min(
|
||||
...[0, 1].map(() => {
|
||||
const start = performance.now();
|
||||
resultValue(body);
|
||||
return performance.now() - start;
|
||||
}),
|
||||
);
|
||||
resultValue(receiverCalls(100));
|
||||
// Compare growth rather than wall time, which depends on the runner and on
|
||||
// coverage instrumentation. Four times the calls costs about 5-6x here;
|
||||
// re-expanding every method per call cost about 13-14x.
|
||||
const ratio = fastest(receiverCalls(1600)) / fastest(receiverCalls(400));
|
||||
expect(ratio).toBeLessThan(10);
|
||||
expect(resultValue(receiverCalls(400))).toMatchObject({
|
||||
it("keeps a shared result uncertain across a wide escaped receiver (#1495)", () => {
|
||||
const count = 2000;
|
||||
const methods = Array.from(
|
||||
{ length: count },
|
||||
(_, index) => `m${index}() { return shared; },`,
|
||||
).join(" ");
|
||||
const calls = Array.from(
|
||||
{ length: count },
|
||||
(_, index) => `box.m${index}().p${index} = ${index};`,
|
||||
).join(" ");
|
||||
const value = resultValue(
|
||||
`const shared = { token: "TOKEN" }; const box = { ${methods} }; ${calls} return [shared.token];`,
|
||||
);
|
||||
expect(value).toMatchObject({
|
||||
status: "array",
|
||||
items: [{ value: { status: "unknown" } }],
|
||||
});
|
||||
}, 60000);
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
@@ -38,7 +38,7 @@ import {
|
||||
collectSemanticReturns,
|
||||
resolveSemanticModuleCallables,
|
||||
} from "./javascriptSemanticReturns.js";
|
||||
import { collectJavaScriptDerivedSemantics } from "./javascriptSemanticDerivedAnalysis.js";
|
||||
import { collectJavaScriptDerivedSemanticsSteps } from "./javascriptSemanticDerivedAnalysis.js";
|
||||
import { range } from "./javascriptStaticAnalysisHelpers.js";
|
||||
import { propertyName } from "./javascriptAstValues.js";
|
||||
import { semanticCoverage } from "./javascriptSemanticCoverage.js";
|
||||
@@ -107,18 +107,40 @@ export const analyzeParsedJavaScriptReferences = (
|
||||
export const analyzeParsedJavaScriptSemantics = (
|
||||
file: ParsedJavaScriptSource,
|
||||
): JavaScriptSemanticIr => {
|
||||
const steps = analyzeParsedJavaScriptSemanticsSteps(file);
|
||||
for (;;) {
|
||||
const step = steps.next();
|
||||
if (step.done === true) return step.value;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Recover semantics from a parsed artifact with a yield between analysis
|
||||
* phases, so a caller can serve control messages within one large file.
|
||||
* Each phase completes before the next starts; results equal the synchronous
|
||||
* form.
|
||||
*/
|
||||
export function* analyzeParsedJavaScriptSemanticsSteps(
|
||||
file: ParsedJavaScriptSource,
|
||||
): Generator<void, JavaScriptSemanticIr> {
|
||||
const state = createState(file.program);
|
||||
collectDefinitions(file.program, state);
|
||||
yield;
|
||||
collectSemanticMemberMutations(file.program, state);
|
||||
yield;
|
||||
traverseJavaScriptAst(file.program, {
|
||||
enter: (node) => collectSemanticModuleLink(node, state),
|
||||
});
|
||||
yield;
|
||||
const references = collectSemanticReferences(file.program, state);
|
||||
yield;
|
||||
const parserPartial = file.errors.length > 0;
|
||||
const bindings = immutableSemanticBindings(state);
|
||||
yield;
|
||||
const callables = collectSemanticReturns(file.program, state, parserPartial);
|
||||
yield;
|
||||
const moduleLinks = resolveSemanticModuleCallables(state, callables);
|
||||
const derived = collectJavaScriptDerivedSemantics(
|
||||
const derived = yield* collectJavaScriptDerivedSemanticsSteps(
|
||||
file.program,
|
||||
state,
|
||||
callables,
|
||||
@@ -154,7 +176,7 @@ export const analyzeParsedJavaScriptSemantics = (
|
||||
"Cross-function mutation and dynamic property resolution remain unknown.",
|
||||
],
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
/** Receiver identity facts for the overloaded `.open` syntax only. */
|
||||
export type JavaScriptOpenReceiverFact =
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
analyzeParsedJavaScriptSemantics,
|
||||
analyzeParsedJavaScriptSemanticsSteps,
|
||||
} from "./javascriptSemanticAnalysis.js";
|
||||
import { parseJavaScriptSource } from "./javascriptSourceParser.js";
|
||||
|
||||
const source = `
|
||||
import { readFile } from "node:fs";
|
||||
const shared = { token: "TOKEN" };
|
||||
export function load(path) {
|
||||
return new Promise((resolve) => setTimeout(() => resolve(readFile(path)), 1));
|
||||
}
|
||||
export const box = { get() { return shared; } };
|
||||
box.get().value = process.env.MODE ?? "default";
|
||||
`;
|
||||
|
||||
it("yields between semantic phases and returns the synchronous result (#1462)", () => {
|
||||
const parsed = parseJavaScriptSource(source, "main.js");
|
||||
if (parsed === null) throw new Error("Expected parsed source");
|
||||
const steps = analyzeParsedJavaScriptSemanticsSteps(parsed);
|
||||
let yields = 0;
|
||||
let step = steps.next();
|
||||
while (step.done !== true) {
|
||||
yields += 1;
|
||||
step = steps.next();
|
||||
}
|
||||
// Every top-level phase and derived collector is a separate step.
|
||||
expect(yields).toBeGreaterThanOrEqual(13);
|
||||
const reparsed = parseJavaScriptSource(source, "main.js");
|
||||
if (reparsed === null) throw new Error("Expected parsed source");
|
||||
expect(step.value).toEqual(analyzeParsedJavaScriptSemantics(reparsed));
|
||||
});
|
||||
@@ -36,43 +36,54 @@ type DerivedSemanticAnalysis = Pick<
|
||||
>;
|
||||
|
||||
/** Compose bounded semantic passes after lexical definitions are available. */
|
||||
export const collectJavaScriptDerivedSemantics = (
|
||||
/**
|
||||
* Collect derived semantics with a yield between collectors. Each collector
|
||||
* completes before the next starts, so results equal the synchronous form.
|
||||
*/
|
||||
export function* collectJavaScriptDerivedSemanticsSteps(
|
||||
program: t.Program,
|
||||
state: JavaScriptSemanticAnalysisState,
|
||||
callables: readonly JavaScriptSemanticCallable[],
|
||||
parserPartial: boolean,
|
||||
): DerivedSemanticAnalysis => {
|
||||
): Generator<void, DerivedSemanticAnalysis> {
|
||||
const calls = collectJavaScriptSemanticCalls(program, state, callables);
|
||||
yield;
|
||||
const promiseOperations = collectJavaScriptSemanticPromises(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const asyncEffects = collectJavaScriptSemanticAsyncEffects(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const childProcesses = collectJavaScriptSemanticChildProcesses(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const dataEffects = collectJavaScriptSemanticDataEffects(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const resourceOperations = collectJavaScriptSemanticResources(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const objectOperations = collectJavaScriptSemanticObjects(
|
||||
program,
|
||||
state,
|
||||
callables,
|
||||
);
|
||||
yield;
|
||||
const functionFingerprints = collectJavaScriptSemanticFingerprints({
|
||||
state,
|
||||
callables,
|
||||
@@ -96,4 +107,4 @@ export const collectJavaScriptDerivedSemantics = (
|
||||
objectOperations,
|
||||
functionFingerprints,
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
@@ -6,6 +6,10 @@ import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
|
||||
import { canonicalJsonDigestSteps } from "../canonicalJsonDigestSteps.js";
|
||||
import { freezeOwnedJsonSnapshotSteps } from "../immutableJson.js";
|
||||
import {
|
||||
arrayContainerSchema,
|
||||
parseElementSteps,
|
||||
} from "../zodCollectionSteps.js";
|
||||
import type {
|
||||
ApplicationGraphEvidence,
|
||||
ApplicationGraphEvidenceContext,
|
||||
@@ -765,6 +769,15 @@ export const createJavaScriptSemanticGraph = (
|
||||
|
||||
const validatedImmutableSemanticGraphs = new WeakSet<object>();
|
||||
|
||||
const semanticGraphShape = javaScriptSemanticGraphInputSchema.shape;
|
||||
const semanticGraphContainerSchema = javaScriptSemanticGraphInputSchema.extend({
|
||||
evidence_contexts: arrayContainerSchema(semanticGraphShape.evidence_contexts),
|
||||
nodes: arrayContainerSchema(semanticGraphShape.nodes),
|
||||
relations: arrayContainerSchema(semanticGraphShape.relations),
|
||||
fingerprints: arrayContainerSchema(semanticGraphShape.fingerprints),
|
||||
unknowns: arrayContainerSchema(semanticGraphShape.unknowns),
|
||||
});
|
||||
|
||||
/** Clone input synchronously; commit, check and seal the owned graph in steps. */
|
||||
export const createImmutableJavaScriptSemanticGraphSteps = (
|
||||
input: unknown,
|
||||
@@ -773,6 +786,58 @@ export const createImmutableJavaScriptSemanticGraphSteps = (
|
||||
normalizeGraphInput(javaScriptSemanticGraphInputSchema.parse(input)),
|
||||
);
|
||||
|
||||
/**
|
||||
* Seal a graph whose input the caller transfers: the caller neither retains nor
|
||||
* mutates any part of it afterwards. Only the graph's own fields are validated
|
||||
* synchronously; collection elements are validated and cloned in steps, because
|
||||
* a whole-graph parse of a large application held the event loop for seconds.
|
||||
*/
|
||||
export const sealTransferredJavaScriptSemanticGraphSteps = (
|
||||
input: unknown,
|
||||
): Generator<void, JavaScriptSemanticGraph> =>
|
||||
parseAndSealSemanticGraphSteps(semanticGraphContainerSchema.parse(input));
|
||||
|
||||
function* parseAndSealSemanticGraphSteps(
|
||||
container: z.output<typeof semanticGraphContainerSchema>,
|
||||
): Generator<void, JavaScriptSemanticGraph> {
|
||||
const issues: z.core.$ZodIssue[] = [];
|
||||
const parsed: JavaScriptSemanticGraphInput = {
|
||||
...container,
|
||||
evidence_contexts: yield* parseElementSteps(
|
||||
semanticGraphShape.evidence_contexts.element,
|
||||
container.evidence_contexts,
|
||||
["evidence_contexts"],
|
||||
issues,
|
||||
),
|
||||
nodes: yield* parseElementSteps(
|
||||
semanticGraphShape.nodes.element,
|
||||
container.nodes,
|
||||
["nodes"],
|
||||
issues,
|
||||
),
|
||||
relations: yield* parseElementSteps(
|
||||
semanticGraphShape.relations.element,
|
||||
container.relations,
|
||||
["relations"],
|
||||
issues,
|
||||
),
|
||||
fingerprints: yield* parseElementSteps(
|
||||
semanticGraphShape.fingerprints.element,
|
||||
container.fingerprints,
|
||||
["fingerprints"],
|
||||
issues,
|
||||
),
|
||||
unknowns: yield* parseElementSteps(
|
||||
semanticGraphShape.unknowns.element,
|
||||
container.unknowns,
|
||||
["unknowns"],
|
||||
issues,
|
||||
),
|
||||
};
|
||||
if (issues.length > 0) throw new z.ZodError(issues);
|
||||
return yield* validateAndSealSemanticGraphSteps(normalizeGraphInput(parsed));
|
||||
}
|
||||
|
||||
function* validateAndSealSemanticGraphSteps(
|
||||
semantic: JavaScriptSemanticGraphInput,
|
||||
): Generator<void, JavaScriptSemanticGraph> {
|
||||
|
||||
@@ -343,7 +343,8 @@ export const collectSemanticMemberMutations = (
|
||||
if (paths.has(pathKey)) continue;
|
||||
paths.add(pathKey);
|
||||
expanded.set(node, paths);
|
||||
pending.push(...expandedValues(node, value.path, "escape"));
|
||||
for (const reference of expandedValues(node, value.path, "escape"))
|
||||
pending.push(reference);
|
||||
}
|
||||
}
|
||||
const byPath = escapedLeaves.get(root) ?? new Map();
|
||||
|
||||
@@ -30,6 +30,22 @@ interface Candidate {
|
||||
readonly bindings: ReadonlySet<string>;
|
||||
}
|
||||
|
||||
interface ObjectTargetIndex {
|
||||
readonly propertyByName: ReadonlyMap<
|
||||
string,
|
||||
t.ObjectMethod | t.ObjectProperty
|
||||
>;
|
||||
readonly trailingUnknownsByProperty: ReadonlyMap<
|
||||
t.ObjectMethod | t.ObjectProperty,
|
||||
ObjectTargetUnknown
|
||||
>;
|
||||
}
|
||||
|
||||
interface ObjectTargetUnknown {
|
||||
readonly node: t.ObjectMethod | t.ObjectProperty | t.SpreadElement;
|
||||
readonly previous: ObjectTargetUnknown | undefined;
|
||||
}
|
||||
|
||||
/** Resolve local result references conservatively, without claiming call execution. */
|
||||
export const createSemanticReturnedReferences = (
|
||||
state: JavaScriptSemanticAnalysisState,
|
||||
@@ -39,6 +55,7 @@ export const createSemanticReturnedReferences = (
|
||||
);
|
||||
const bindingCache = new Map<string, LocalCallableResolution>();
|
||||
const results = new WeakMap<t.Node, readonly SemanticReturnedReference[]>();
|
||||
const objectTargets = new WeakMap<t.ObjectExpression, ObjectTargetIndex>();
|
||||
return (callee) => {
|
||||
const existing = results.get(callee);
|
||||
if (existing !== undefined) return existing;
|
||||
@@ -115,7 +132,7 @@ export const createSemanticReturnedReferences = (
|
||||
} else if (t.isFunction(node) && item.path.length === 0)
|
||||
functions.add(node);
|
||||
else if (t.isObjectExpression(node))
|
||||
selectObjectTargets(node, item, pending);
|
||||
selectObjectTargets(node, item, pending, objectTargets);
|
||||
else if (t.isArrayExpression(node)) {
|
||||
const [key, ...path] = item.path;
|
||||
const index =
|
||||
@@ -187,8 +204,38 @@ const selectObjectTargets = (
|
||||
node: t.ObjectExpression,
|
||||
item: Candidate,
|
||||
pending: Candidate[],
|
||||
indexes: WeakMap<t.ObjectExpression, ObjectTargetIndex>,
|
||||
): void => {
|
||||
const [key, ...path] = item.path;
|
||||
if (key !== null && key !== undefined) {
|
||||
const index = objectTargetIndex(node, indexes);
|
||||
const property = index.propertyByName.get(String(key));
|
||||
if (property !== undefined) {
|
||||
const unknowns: ObjectTargetUnknown[] = [];
|
||||
for (
|
||||
let unknown = index.trailingUnknownsByProperty.get(property);
|
||||
unknown !== undefined;
|
||||
unknown = unknown.previous
|
||||
)
|
||||
unknowns.push(unknown);
|
||||
for (const { node: unknown } of unknowns.reverse())
|
||||
pending.push({
|
||||
...item,
|
||||
node: t.isSpreadElement(unknown)
|
||||
? unknown.argument
|
||||
: t.isObjectProperty(unknown)
|
||||
? unknown.value
|
||||
: unknown,
|
||||
...(t.isSpreadElement(unknown) ? {} : { path }),
|
||||
});
|
||||
pending.push({
|
||||
...item,
|
||||
node: t.isObjectProperty(property) ? property.value : property,
|
||||
path,
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
let replaced = false;
|
||||
for (const property of [...node.properties].reverse()) {
|
||||
if (replaced) break;
|
||||
@@ -211,3 +258,35 @@ const selectObjectTargets = (
|
||||
replaced = true;
|
||||
}
|
||||
};
|
||||
|
||||
const objectTargetIndex = (
|
||||
node: t.ObjectExpression,
|
||||
indexes: WeakMap<t.ObjectExpression, ObjectTargetIndex>,
|
||||
): ObjectTargetIndex => {
|
||||
const existing = indexes.get(node);
|
||||
if (existing !== undefined) return existing;
|
||||
const propertyByName = new Map<string, t.ObjectMethod | t.ObjectProperty>();
|
||||
const trailingUnknownsByProperty = new Map<
|
||||
t.ObjectMethod | t.ObjectProperty,
|
||||
ObjectTargetUnknown
|
||||
>();
|
||||
let trailingUnknown: ObjectTargetUnknown | undefined;
|
||||
for (const property of [...node.properties].reverse()) {
|
||||
if (t.isSpreadElement(property)) {
|
||||
trailingUnknown = { node: property, previous: trailingUnknown };
|
||||
continue;
|
||||
}
|
||||
const name = semanticStaticPropertyKey(property.key, property.computed);
|
||||
if (name === null) {
|
||||
trailingUnknown = { node: property, previous: trailingUnknown };
|
||||
continue;
|
||||
}
|
||||
if (propertyByName.has(name)) continue;
|
||||
propertyByName.set(name, property);
|
||||
if (trailingUnknown !== undefined)
|
||||
trailingUnknownsByProperty.set(property, trailingUnknown);
|
||||
}
|
||||
const created = { propertyByName, trailingUnknownsByProperty };
|
||||
indexes.set(node, created);
|
||||
return created;
|
||||
};
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
import { z } from "zod";
|
||||
|
||||
/**
|
||||
* An array schema's own constraints, such as a minimum length, without its
|
||||
* element schema. Pair it with {@link parseElementSteps} to validate a large
|
||||
* collection in cooperative steps.
|
||||
*/
|
||||
export const arrayContainerSchema = (
|
||||
array: z.ZodArray<z.ZodType>,
|
||||
): z.ZodType<unknown[]> => array.clone({ ...array.def, element: z.unknown() });
|
||||
|
||||
/** Elements validated between yields, in the issues and paths a whole-array parse reports. */
|
||||
export function* parseElementSteps<Element extends z.ZodType>(
|
||||
element: Element,
|
||||
values: readonly unknown[],
|
||||
path: readonly PropertyKey[],
|
||||
issues: z.core.$ZodIssue[],
|
||||
): Generator<void, z.output<Element>[]> {
|
||||
const parsed: z.output<Element>[] = [];
|
||||
for (const [index, value] of values.entries()) {
|
||||
if (index % 64 === 0) yield;
|
||||
const result = element.safeParse(value);
|
||||
if (result.success) parsed.push(result.data);
|
||||
else
|
||||
issues.push(
|
||||
...result.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: [...path, index, ...issue.path],
|
||||
})),
|
||||
);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
@@ -75,6 +75,7 @@ interface TerminalExitOptions {
|
||||
readonly interactions: InteractionEvent[];
|
||||
readonly dispatchedEventIndexes: ReadonlySet<number>;
|
||||
readonly recordEvent: RecordProcessCaptureEvent;
|
||||
readonly onLiveProgress?: () => void;
|
||||
}
|
||||
|
||||
interface CaptureResultOptions {
|
||||
@@ -601,8 +602,10 @@ export const observeSettlement = async (
|
||||
recordEvent: RecordProcessCaptureEvent = () => undefined,
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
signal?: AbortSignal,
|
||||
onLiveProgress?: () => void,
|
||||
): Promise<ObservedProcessSettlement> => {
|
||||
assertNotCancelled(signal);
|
||||
onLiveProgress?.();
|
||||
if (platform === "win32") {
|
||||
recordEvent("lifecycle", 1);
|
||||
return { state: "unverifiable", elapsed_ms: 0 };
|
||||
@@ -611,6 +614,7 @@ export const observeSettlement = async (
|
||||
let consecutiveEmpty = 0;
|
||||
let deadlineReached = false;
|
||||
while (!deadlineReached) {
|
||||
onLiveProgress?.();
|
||||
assertNotCancelled(signal);
|
||||
const observations = await Promise.all(
|
||||
[...new Set(processGroupIds)].map((processGroupId) =>
|
||||
@@ -651,6 +655,7 @@ export const awaitTerminalExit = async ({
|
||||
interactions,
|
||||
dispatchedEventIndexes,
|
||||
recordEvent,
|
||||
onLiveProgress,
|
||||
}: TerminalExitOptions): Promise<{
|
||||
exitCode: number;
|
||||
signal?: number;
|
||||
@@ -661,6 +666,7 @@ export const awaitTerminalExit = async ({
|
||||
// Keep the initiating lifecycle reason so comparisons distinguish a target
|
||||
// exit from harness-owned timeout, idle-timeout, and cancellation cleanup.
|
||||
let reason: "exited" | "timeout" | "idle_timeout" | "cancelled" = "exited";
|
||||
onLiveProgress?.();
|
||||
terminal.onExit((exit) => {
|
||||
recordEvent("lifecycle", 0);
|
||||
for (const [eventIndex, event] of scenario.events.entries()) {
|
||||
@@ -688,6 +694,7 @@ export const awaitTerminalExit = async ({
|
||||
resolveExit({ ...exit, reason });
|
||||
});
|
||||
const timeout = scheduleProcessInterval(() => {
|
||||
onLiveProgress?.();
|
||||
if (signal?.aborted === true) {
|
||||
reason = "cancelled";
|
||||
terminal.kill("SIGKILL");
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
createProcessCaptureProgressTracker,
|
||||
type ProcessCaptureProgress,
|
||||
} from "./ProcessCaptureProgress.js";
|
||||
|
||||
const blockedProgress = () => {
|
||||
let release: () => void = () => undefined;
|
||||
const blocked = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const delivered: Parameters<ProcessCaptureProgress["report"]>[0][] = [];
|
||||
const progress: ProcessCaptureProgress = {
|
||||
async report(update) {
|
||||
delivered.push(update);
|
||||
if (delivered.length === 1) await blocked;
|
||||
},
|
||||
};
|
||||
return { progress, delivered, release: () => release() };
|
||||
};
|
||||
|
||||
it("coalesces intermediate observations while the progress receiver is blocked", async () => {
|
||||
const receiver = blockedProgress();
|
||||
const tracker = createProcessCaptureProgressTracker(receiver.progress);
|
||||
void tracker.report({
|
||||
phase: "running",
|
||||
counts: { frames: 1, samples: 0, interactions: 0 },
|
||||
});
|
||||
await Promise.resolve();
|
||||
let latest: Promise<void> | undefined;
|
||||
for (let frames = 2; frames <= 1_000; frames += 1) {
|
||||
latest = tracker.report({
|
||||
phase: "running",
|
||||
counts: { frames, samples: 0, interactions: 0 },
|
||||
});
|
||||
}
|
||||
expect(receiver.delivered).toHaveLength(1);
|
||||
receiver.release();
|
||||
await latest;
|
||||
expect(receiver.delivered.map((update) => update.completed)).toEqual([
|
||||
1, 1_000,
|
||||
]);
|
||||
expect(receiver.delivered[1]?.message).toContain("frames=1000");
|
||||
});
|
||||
|
||||
it("discards stale live observations before delivering final cleanup", async () => {
|
||||
const receiver = blockedProgress();
|
||||
const tracker = createProcessCaptureProgressTracker(receiver.progress);
|
||||
void tracker.report({
|
||||
phase: "running",
|
||||
counts: { frames: 1, samples: 0, interactions: 0 },
|
||||
});
|
||||
await Promise.resolve();
|
||||
void tracker.report({
|
||||
phase: "running",
|
||||
counts: { frames: 2, samples: 0, interactions: 0 },
|
||||
});
|
||||
void tracker.report({
|
||||
phase: "settling",
|
||||
counts: { frames: 2, samples: 1, interactions: 0 },
|
||||
});
|
||||
tracker.closeLive();
|
||||
await tracker.report({
|
||||
phase: "running",
|
||||
counts: { frames: 10, samples: 10, interactions: 10 },
|
||||
});
|
||||
const terminal = tracker.report({
|
||||
phase: "cleanup",
|
||||
counts: { frames: 2, samples: 2, interactions: 0 },
|
||||
disposition: "exited",
|
||||
terminal: true,
|
||||
});
|
||||
receiver.release();
|
||||
await terminal;
|
||||
expect(receiver.delivered.map((update) => update.phase)).toEqual([
|
||||
"running",
|
||||
"cleanup",
|
||||
]);
|
||||
expect(receiver.delivered.at(-1)).toMatchObject({
|
||||
completed: 4,
|
||||
total: null,
|
||||
terminal: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("still delivers terminal status after an observer rejects an update", async () => {
|
||||
const delivered: Parameters<ProcessCaptureProgress["report"]>[0][] = [];
|
||||
const tracker = createProcessCaptureProgressTracker({
|
||||
async report(update) {
|
||||
delivered.push(update);
|
||||
if (delivered.length === 1) throw new Error("Observer disconnected");
|
||||
},
|
||||
});
|
||||
await tracker.report({
|
||||
phase: "prepare",
|
||||
counts: { frames: 0, samples: 0, interactions: 0 },
|
||||
});
|
||||
tracker.closeLive();
|
||||
await expect(
|
||||
tracker.report({
|
||||
phase: "cleanup",
|
||||
counts: { frames: 1, samples: 1, interactions: 0 },
|
||||
disposition: "failed",
|
||||
terminal: true,
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
expect(delivered.at(-1)).toMatchObject({
|
||||
phase: "cleanup",
|
||||
completed: 2,
|
||||
terminal: true,
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
import type { ProcessCaptureCleanupReport } from "../../domain/process/processCapture.js";
|
||||
|
||||
/**
|
||||
* Progress port accepted by process capture.
|
||||
* Structurally matches the shared application `ProgressReporter`.
|
||||
*/
|
||||
export interface ProcessCaptureProgress {
|
||||
report(update: {
|
||||
readonly phase: string;
|
||||
readonly completed: number;
|
||||
readonly total: null;
|
||||
readonly message: string;
|
||||
readonly terminal?: boolean;
|
||||
}): Promise<void>;
|
||||
}
|
||||
|
||||
/** Observed collection counts. These are not a percentage of an unknown total. */
|
||||
export interface ProcessCaptureProgressCounts {
|
||||
readonly frames: number;
|
||||
readonly samples: number;
|
||||
readonly interactions: number;
|
||||
}
|
||||
|
||||
export type ProcessCaptureProgressPhase =
|
||||
| "prepare"
|
||||
| "running"
|
||||
| "settling"
|
||||
| "cleanup";
|
||||
|
||||
export type ProcessCaptureProgressDisposition =
|
||||
| "exited"
|
||||
| "timeout"
|
||||
| "idle_timeout"
|
||||
| "cancelled"
|
||||
| "failed";
|
||||
|
||||
export interface ProcessCaptureProgressReport {
|
||||
readonly phase: ProcessCaptureProgressPhase;
|
||||
readonly counts: ProcessCaptureProgressCounts;
|
||||
readonly terminal?: boolean;
|
||||
readonly disposition?: ProcessCaptureProgressDisposition;
|
||||
readonly cleanup?: ProcessCaptureCleanupReport;
|
||||
}
|
||||
|
||||
const formatProgressMessage = (
|
||||
elapsedMs: number,
|
||||
counts: ProcessCaptureProgressCounts,
|
||||
disposition: ProcessCaptureProgressDisposition | undefined,
|
||||
cleanup: ProcessCaptureCleanupReport | undefined,
|
||||
): string => {
|
||||
const status = `elapsed_ms=${String(elapsedMs)} frames=${String(counts.frames)} samples=${String(counts.samples)} interactions=${String(counts.interactions)}`;
|
||||
const dispositionText =
|
||||
disposition === undefined ? "" : ` disposition=${disposition}`;
|
||||
const cleanupText =
|
||||
cleanup === undefined
|
||||
? ""
|
||||
: ` owned_process_group=${cleanup.owned_process_group.state} terminal_renderer=${cleanup.terminal_renderer.state} temporary_root=${cleanup.temporary_root.state}`;
|
||||
return `${status}${dispositionText}${cleanupText}`;
|
||||
};
|
||||
|
||||
/** Serialize observed capture status through one rate-limited reporter. */
|
||||
export const createProcessCaptureProgressTracker = (
|
||||
progress: ProcessCaptureProgress | undefined,
|
||||
now: () => number = Date.now,
|
||||
) => {
|
||||
const started = now();
|
||||
let completed = 0;
|
||||
let live = true;
|
||||
let next: Parameters<ProcessCaptureProgress["report"]>[0] | undefined;
|
||||
let pending: Promise<void> | undefined;
|
||||
const flush = async (): Promise<void> => {
|
||||
try {
|
||||
while (next !== undefined) {
|
||||
const update = next;
|
||||
next = undefined;
|
||||
try {
|
||||
await progress?.report(update);
|
||||
} catch {
|
||||
// Progress delivery is observational; capture and cleanup still finish.
|
||||
continue;
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
pending = undefined;
|
||||
}
|
||||
};
|
||||
return {
|
||||
closeLive(): void {
|
||||
live = false;
|
||||
if (next?.terminal !== true) next = undefined;
|
||||
},
|
||||
report(update: ProcessCaptureProgressReport): Promise<void> {
|
||||
if (progress === undefined) return Promise.resolve();
|
||||
if (update.terminal !== true && !live) return Promise.resolve();
|
||||
const units =
|
||||
update.counts.frames +
|
||||
update.counts.samples +
|
||||
update.counts.interactions;
|
||||
if (units > completed) completed = units;
|
||||
// A slow observer needs the latest observation, not a queue of old states.
|
||||
next = {
|
||||
phase: update.phase,
|
||||
completed,
|
||||
total: null as null,
|
||||
message: formatProgressMessage(
|
||||
Math.max(0, now() - started),
|
||||
update.counts,
|
||||
update.disposition,
|
||||
update.cleanup,
|
||||
),
|
||||
...(update.terminal === true ? { terminal: true as const } : {}),
|
||||
};
|
||||
pending ??= Promise.resolve().then(flush);
|
||||
return pending;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export type ProcessCaptureProgressTracker = ReturnType<
|
||||
typeof createProcessCaptureProgressTracker
|
||||
>;
|
||||
@@ -55,6 +55,13 @@ import {
|
||||
import { makeProcessCaptureEnvironment } from "./ProcessCaptureEnvironment.js";
|
||||
import { classifyFilesystemEffects } from "./ProcessFilesystemEffects.js";
|
||||
import { processCaptureOwnershipUnavailableReason } from "./ProcessCaptureCapability.js";
|
||||
import {
|
||||
createProcessCaptureProgressTracker,
|
||||
type ProcessCaptureProgress,
|
||||
type ProcessCaptureProgressCounts,
|
||||
type ProcessCaptureProgressDisposition,
|
||||
type ProcessCaptureProgressTracker,
|
||||
} from "./ProcessCaptureProgress.js";
|
||||
import type { ProcessOwnershipBaseline } from "../ProcessOwnership.js";
|
||||
|
||||
interface StartedCaptureRuntime {
|
||||
@@ -262,7 +269,10 @@ const finishProcessRun = async (options: {
|
||||
readonly executionFailure: unknown;
|
||||
readonly observations?: ProcessCaptureObservationBuffer;
|
||||
readonly rootPid?: number;
|
||||
readonly progress: ProcessCaptureProgressTracker;
|
||||
readonly progressCounts: () => ProcessCaptureProgressCounts;
|
||||
}): Promise<ProcessCapture> => {
|
||||
options.progress.closeLive();
|
||||
await options.stopSampler();
|
||||
const sampledProcessGroupIds =
|
||||
options.runtime === undefined
|
||||
@@ -281,6 +291,20 @@ const finishProcessRun = async (options: {
|
||||
temporaryRoot: options.temporaryRoot,
|
||||
...(options.cleanupHost === undefined ? {} : { host: options.cleanupHost }),
|
||||
});
|
||||
const observedExit =
|
||||
options.observations?.exit.state === "available"
|
||||
? options.observations.exit.value.reason
|
||||
: options.capture?.exit.reason;
|
||||
await options.progress.report({
|
||||
phase: "cleanup",
|
||||
counts: options.progressCounts(),
|
||||
terminal: true,
|
||||
disposition: captureProgressDisposition(
|
||||
observedExit,
|
||||
options.executionFailure,
|
||||
),
|
||||
cleanup,
|
||||
});
|
||||
const cleanupFailure = cleanupReportFailure(cleanup);
|
||||
let { capture, executionFailure } = options;
|
||||
let verifiedCapture: ProcessCapture | undefined;
|
||||
@@ -336,6 +360,7 @@ const completeCapture = async (options: {
|
||||
readonly eventJournal: readonly ProcessCaptureEventJournalEntry[];
|
||||
readonly observationBuffer: ProcessCaptureObservationBuffer;
|
||||
readonly recordEvent: RecordProcessCaptureEvent;
|
||||
readonly onLiveProgress?: () => void;
|
||||
}): Promise<PendingProcessCapture> => {
|
||||
const unavailableReason = (stage: string, cause: unknown): string =>
|
||||
`${stage} failed: ${cause instanceof Error ? cause.message : "unknown failure"}`;
|
||||
@@ -351,6 +376,7 @@ const completeCapture = async (options: {
|
||||
options.recordEvent,
|
||||
options.hostPlatform,
|
||||
options.signal,
|
||||
options.onLiveProgress,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
options.observationBuffer.settlement = {
|
||||
@@ -460,6 +486,25 @@ const completeCapture = async (options: {
|
||||
});
|
||||
};
|
||||
|
||||
const captureProgressDisposition = (
|
||||
exitReason: string | undefined,
|
||||
executionFailure: unknown,
|
||||
): ProcessCaptureProgressDisposition => {
|
||||
if (
|
||||
exitReason === "exited" ||
|
||||
exitReason === "timeout" ||
|
||||
exitReason === "idle_timeout" ||
|
||||
exitReason === "cancelled"
|
||||
)
|
||||
return exitReason;
|
||||
if (
|
||||
executionFailure instanceof ProcessCaptureError &&
|
||||
executionFailure.reason === "cancelled"
|
||||
)
|
||||
return "cancelled";
|
||||
return "failed";
|
||||
};
|
||||
|
||||
/** Execute one caller-selected scenario and return bounded observations. */
|
||||
const runProcessScenario = async (
|
||||
scenario: ProcessScenario,
|
||||
@@ -468,12 +513,44 @@ const runProcessScenario = async (
|
||||
hostPlatform: NodeJS.Platform = process.platform,
|
||||
captureSnapshot: typeof snapshotRoots = snapshotRoots,
|
||||
cleanupHost?: ProcessCaptureCleanupHost,
|
||||
progress?: ProcessCaptureProgress,
|
||||
): Promise<ProcessCapture> => {
|
||||
const { temporaryRoot, runId, ownershipBaseline, before } =
|
||||
await prepareProcessCapture(scenario, signal, captureSnapshot);
|
||||
const progressTracker = createProcessCaptureProgressTracker(progress);
|
||||
const frames: TerminalFrame[] = [];
|
||||
const samples: ProcessSample[] = [];
|
||||
const interactions: InteractionEvent[] = [];
|
||||
const progressCounts = (): ProcessCaptureProgressCounts => ({
|
||||
frames: frames.length,
|
||||
samples: samples.length,
|
||||
interactions: interactions.length,
|
||||
});
|
||||
const reportRunning = () => {
|
||||
void progressTracker.report({ phase: "running", counts: progressCounts() });
|
||||
};
|
||||
const reportSettling = () => {
|
||||
void progressTracker.report({
|
||||
phase: "settling",
|
||||
counts: progressCounts(),
|
||||
});
|
||||
};
|
||||
let prepared: Awaited<ReturnType<typeof prepareProcessCapture>>;
|
||||
try {
|
||||
await progressTracker.report({
|
||||
phase: "prepare",
|
||||
counts: progressCounts(),
|
||||
});
|
||||
prepared = await prepareProcessCapture(scenario, signal, captureSnapshot);
|
||||
} catch (cause: unknown) {
|
||||
progressTracker.closeLive();
|
||||
await progressTracker.report({
|
||||
phase: "cleanup",
|
||||
counts: progressCounts(),
|
||||
terminal: true,
|
||||
disposition: signal?.aborted === true ? "cancelled" : "failed",
|
||||
});
|
||||
throw cause;
|
||||
}
|
||||
const { temporaryRoot, runId, ownershipBaseline, before } = prepared;
|
||||
const journal = createProcessCaptureJournal();
|
||||
const { entries: eventJournal, recordEvent } = journal;
|
||||
recordEvent("filesystem_checkpoints", 0);
|
||||
@@ -523,6 +600,7 @@ const runProcessScenario = async (
|
||||
interactions,
|
||||
dispatchedEventIndexes,
|
||||
recordEvent,
|
||||
...(progress === undefined ? {} : { onLiveProgress: reportRunning }),
|
||||
});
|
||||
observations.exit = {
|
||||
state: "available",
|
||||
@@ -549,6 +627,7 @@ const runProcessScenario = async (
|
||||
recordEvent,
|
||||
captureSnapshot,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
...(progress === undefined ? {} : { onLiveProgress: reportSettling }),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
executionFailure = normalizeCaptureFailure(cause, signal);
|
||||
@@ -590,6 +669,8 @@ const runProcessScenario = async (
|
||||
executionFailure,
|
||||
observations,
|
||||
...(actualRootPid === undefined ? {} : { rootPid: actualRootPid }),
|
||||
progress: progressTracker,
|
||||
progressCounts,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -599,8 +680,9 @@ export const captureProcessScenario = async (
|
||||
signal?: AbortSignal,
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
environment: Readonly<Record<string, string | undefined>> = process.env,
|
||||
captureSnapshot: typeof snapshotRoots = snapshotRoots,
|
||||
captureSnapshot?: typeof snapshotRoots,
|
||||
cleanupHost?: ProcessCaptureCleanupHost,
|
||||
progress?: ProcessCaptureProgress,
|
||||
): Promise<Result<ProcessCapture, ProcessCaptureError | AnalysisError>> => {
|
||||
const ownershipReason = processCaptureOwnershipUnavailableReason(platform);
|
||||
if (ownershipReason !== undefined)
|
||||
@@ -624,8 +706,9 @@ export const captureProcessScenario = async (
|
||||
signal,
|
||||
environment,
|
||||
platform,
|
||||
captureSnapshot,
|
||||
captureSnapshot ?? snapshotRoots,
|
||||
cleanupHost,
|
||||
progress,
|
||||
),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
|
||||
@@ -90,24 +90,20 @@ const registerProcessTools = ({
|
||||
toolRegistrationOptions(captureContract),
|
||||
async (input, context) => {
|
||||
const progress = mcpProgressReporter(context);
|
||||
await progress.report({
|
||||
phase: captureContract.name,
|
||||
completed: 0,
|
||||
total: 1,
|
||||
message: "started",
|
||||
});
|
||||
const captured = await logToolExecution(
|
||||
logger,
|
||||
captureContract.name,
|
||||
() => captureProcessScenario(input, context.mcpReq.signal),
|
||||
() =>
|
||||
captureProcessScenario(
|
||||
input,
|
||||
context.mcpReq.signal,
|
||||
process.platform,
|
||||
process.env,
|
||||
undefined,
|
||||
undefined,
|
||||
progress,
|
||||
),
|
||||
);
|
||||
await progress.report({
|
||||
phase: captureContract.name,
|
||||
completed: 1,
|
||||
total: 1,
|
||||
message: captured.ok ? "completed" : "failed",
|
||||
terminal: true,
|
||||
});
|
||||
if (!captured.ok)
|
||||
return server.delivery.toCallToolResult(captured, captureContract);
|
||||
const evidence = createProcessCaptureEvidence(input, captured.value);
|
||||
|
||||
@@ -47,6 +47,10 @@ const writes = {
|
||||
constructor:
|
||||
"function Box() { return shared; } const t = new Box(); t.x = 2;",
|
||||
callback: "const get = () => shared; [0].forEach(() => { get().x = 2; });",
|
||||
wideMethods: `const box = { ${Array.from(
|
||||
{ length: 64 },
|
||||
(_, index) => `m${index}() { return shared; }`,
|
||||
).join(", ")} }; box.m0().x = 2; box.m63().other = 3;`,
|
||||
};
|
||||
const source = [
|
||||
...Object.entries(writes).map(
|
||||
@@ -61,6 +65,8 @@ const source = [
|
||||
"export function unused(){ const shared = {x: 1}; const get = () => shared; get(); return shared.x; }",
|
||||
"export function nestedCallable(){ const shared = {x: 1}; const get = () => { const inner = () => shared; return {x: 0}; }; get().x = 2; return shared.x; }",
|
||||
"export function unrelated(){ const shared = {x: 1}; const other = {x: 9}; const get = () => shared; get().x = 2; return other.x; }",
|
||||
"export function receiverDistinctScopes(){ const shared = {x: 1}; const box = {outer(){ return shared; }, inner(){ const shared = {x: 9}; return shared; }, noop(){}}; box.noop(); return {outer: shared.x, inner: box.inner().x}; }",
|
||||
"export function receiverDistinctPaths(){ const parent = {left: {x: 1}, right: {x: 9}, keep: 7}; const box = {left(){ return parent.left; }, right(){ return parent.right; }, noop(){}}; box.noop(); return {left: parent.left.x, right: parent.right.x, keep: parent.keep}; }",
|
||||
].join("\n");
|
||||
|
||||
const assertReturns = (fields: JavaScriptReturnFields): void => {
|
||||
@@ -91,6 +97,17 @@ const assertReturns = (fields: JavaScriptReturnFields): void => {
|
||||
expect(fields("unrelated")).toContainEqual(
|
||||
expect.objectContaining({ path: "", state: "literal", value: 9 }),
|
||||
);
|
||||
for (const path of ["/outer", "/inner"])
|
||||
expect(fields("receiverDistinctScopes")).toContainEqual(
|
||||
expect.objectContaining({ path, state: "unknown" }),
|
||||
);
|
||||
for (const path of ["/left", "/right"])
|
||||
expect(fields("receiverDistinctPaths")).toContainEqual(
|
||||
expect.objectContaining({ path, state: "unknown" }),
|
||||
);
|
||||
expect(fields("receiverDistinctPaths")).toContainEqual(
|
||||
expect.objectContaining({ path: "/keep", state: "literal", value: 7 }),
|
||||
);
|
||||
};
|
||||
|
||||
cliTest(
|
||||
@@ -98,3 +115,39 @@ cliTest(
|
||||
({ cli }) => verifyJavaScriptReturnShapes(cli, source, assertReturns),
|
||||
120_000,
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"analyzes wide receiver returns without losing escaped reference facts",
|
||||
({ cli }) => {
|
||||
const width = 160_000;
|
||||
const object = Array.from(
|
||||
{ length: width },
|
||||
(_, index) => `p${index}:0`,
|
||||
).join(",");
|
||||
const array = "0,".repeat(width);
|
||||
const wideSource = [
|
||||
["wideObject", `{${object},last:shared}`],
|
||||
["wideArray", `[${array}shared]`],
|
||||
]
|
||||
.map(
|
||||
([name, value]) =>
|
||||
`export function ${name}(){const shared={x:1};const box={wide(){return ${value}},noop(){}};box.noop();return {x:shared.x,keep:7};}`,
|
||||
)
|
||||
.join("\n");
|
||||
return verifyJavaScriptReturnShapes(cli, wideSource, (fields) => {
|
||||
for (const name of ["wideObject", "wideArray"]) {
|
||||
expect(fields(name)).toContainEqual(
|
||||
expect.objectContaining({ path: "/x", state: "unknown" }),
|
||||
);
|
||||
expect(fields(name)).toContainEqual(
|
||||
expect.objectContaining({
|
||||
path: "/keep",
|
||||
state: "literal",
|
||||
value: 7,
|
||||
}),
|
||||
);
|
||||
}
|
||||
});
|
||||
},
|
||||
120_000,
|
||||
);
|
||||
|
||||
@@ -22,9 +22,9 @@ cliTest(
|
||||
arguments: arguments_,
|
||||
cwd: home,
|
||||
environment: {
|
||||
HOME: home,
|
||||
USERPROFILE: home,
|
||||
HERMES_HOME: " ${HOME}/profile ",
|
||||
REA_CLIENT_PROFILE_HOME: home,
|
||||
HERMES_HOME: " ${REA_CLIENT_PROFILE_HOME}/profile ",
|
||||
},
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
@@ -96,3 +96,195 @@ cliTest(
|
||||
await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
},
|
||||
);
|
||||
|
||||
for (const selection of ["default", "custom", "suffix", "explicit"] as const) {
|
||||
cliTest(
|
||||
`Hermes ${selection} sticky-profile selection preserves the root configuration`,
|
||||
async ({ cli }) => {
|
||||
const home = await createTestTempDirectory("rea-hermes-sticky-");
|
||||
const suffix = selection === "suffix" ? "-dev" : "";
|
||||
const native =
|
||||
process.platform === "win32"
|
||||
? join(home, "AppData", "Local", `hermes${suffix}`)
|
||||
: join(home, `.hermes${suffix}`);
|
||||
const root = selection === "custom" ? join(home, "custom root") : native;
|
||||
const profile = join(
|
||||
root,
|
||||
"profiles",
|
||||
selection === "explicit" ? "other" : "coder",
|
||||
);
|
||||
await mkdir(profile, { recursive: true });
|
||||
const original =
|
||||
"# profile settings\nmodel: custom\nmcp_servers:\n other:\n command: other-server\n";
|
||||
const config = join(profile, "config.yaml");
|
||||
const rootConfig = join(root, "config.yaml");
|
||||
await writeFile(config, original);
|
||||
await writeFile(rootConfig, "model: root-model\n");
|
||||
await writeFile(join(root, "active_profile"), "\uFEFFCoDeR\r\n");
|
||||
const run = (args: readonly string[]) =>
|
||||
cli.run({
|
||||
arguments: args,
|
||||
cwd: home,
|
||||
environment: {
|
||||
USERPROFILE: home,
|
||||
LOCALAPPDATA: join(home, "AppData", "Local"),
|
||||
HERMES_DATA_DIR_SUFFIX: suffix,
|
||||
HERMES_HOME:
|
||||
selection === "explicit"
|
||||
? profile
|
||||
: selection === "custom"
|
||||
? root
|
||||
: undefined,
|
||||
},
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
const plan = await run([
|
||||
"setup",
|
||||
"--client",
|
||||
"hermes",
|
||||
"--dry-run",
|
||||
"--json",
|
||||
]);
|
||||
expect(plan.exitCode).toBe(0);
|
||||
expect(plan.json).toMatchObject({
|
||||
status: "planned",
|
||||
plannedActions: expect.arrayContaining([
|
||||
expect.objectContaining({ target: config }),
|
||||
expect.objectContaining({
|
||||
target: join(profile, "skills", "reverse-engineer-anything"),
|
||||
}),
|
||||
]),
|
||||
});
|
||||
expect(
|
||||
(await run(["setup", "--client", "hermes", "--yes", "--json"]))
|
||||
.exitCode,
|
||||
).toBe(0);
|
||||
expect(parse(await readFile(config, "utf8"))).toMatchObject({
|
||||
mcp_servers: {
|
||||
rea: { enabled: true },
|
||||
other: { command: "other-server" },
|
||||
},
|
||||
});
|
||||
expect(await readFile(`${config}.rea.backup`, "utf8")).toBe(original);
|
||||
const skill = join(
|
||||
profile,
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
"SKILL.md",
|
||||
);
|
||||
expect(await readFile(skill, "utf8")).toContain(
|
||||
"name: reverse-engineer-anything",
|
||||
);
|
||||
if (selection === "custom") {
|
||||
const doctor = await run([
|
||||
"doctor",
|
||||
"--client",
|
||||
"hermes",
|
||||
"--skill",
|
||||
"--json",
|
||||
]);
|
||||
expect(doctor.exitCode).toBe(0);
|
||||
expect(doctor.json).toMatchObject({
|
||||
healthy: true,
|
||||
identity: {
|
||||
skill: { state: "aligned" },
|
||||
registrations: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
client: "hermes",
|
||||
config_path: config,
|
||||
state: "aligned",
|
||||
}),
|
||||
]),
|
||||
},
|
||||
});
|
||||
expect(
|
||||
(await run(["setup", "--client", "hermes", "--yes", "--json"])).json,
|
||||
).toMatchObject({ appliedActions: [] });
|
||||
expect((await run(["uninstall", "--json"])).exitCode).toBe(0);
|
||||
expect(parse(await readFile(config, "utf8"))).toEqual(parse(original));
|
||||
await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
}
|
||||
expect(await readFile(rootConfig, "utf8")).toBe("model: root-model\n");
|
||||
await expect(access(`${rootConfig}.rea.backup`)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await expect(
|
||||
access(join(root, "skills", "reverse-engineer-anything")),
|
||||
).rejects.toMatchObject({ code: "ENOENT" });
|
||||
},
|
||||
);
|
||||
}
|
||||
for (const name of ["missing", "deleted", "ghost", "../escape", "root"]) {
|
||||
cliTest(
|
||||
`Hermes rejects unusable sticky profile ${name} without writing a fallback home`,
|
||||
async ({ cli }) => {
|
||||
const home = await createTestTempDirectory("rea-hermes-invalid-profile-");
|
||||
const root = join(home, "hermes root");
|
||||
await mkdir(root);
|
||||
await writeFile(join(root, "active_profile"), name);
|
||||
if (name === "deleted" || name === "ghost") {
|
||||
await mkdir(join(root, "profiles", name), { recursive: true });
|
||||
if (name === "deleted") {
|
||||
await writeFile(
|
||||
join(root, "profiles", name, "config.yaml"),
|
||||
"model: custom\n",
|
||||
);
|
||||
await mkdir(join(root, "profiles", ".deleted"));
|
||||
await writeFile(
|
||||
join(root, "profiles", ".deleted", name),
|
||||
"deleted\n",
|
||||
);
|
||||
}
|
||||
}
|
||||
const run = (args: readonly string[]) =>
|
||||
cli.run({
|
||||
arguments: args,
|
||||
cwd: home,
|
||||
environment: { USERPROFILE: home, HERMES_HOME: root },
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
const rejected = await run([
|
||||
"setup",
|
||||
"--client",
|
||||
"hermes",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(rejected.exitCode).not.toBe(0);
|
||||
expect(rejected.json).toMatchObject({
|
||||
status: "needs_human",
|
||||
plannedActions: [],
|
||||
appliedActions: [],
|
||||
remediation: expect.stringContaining("Hermes profile"),
|
||||
});
|
||||
await expect(access(join(root, "config.yaml"))).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await expect(access(join(root, "skills"))).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
if (name !== "missing") return;
|
||||
const doctor = await run(["doctor", "--client", "hermes", "--json"]);
|
||||
expect(doctor.exitCode).not.toBe(0);
|
||||
expect(doctor.json).toMatchObject({
|
||||
identity: {
|
||||
registrations: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
client: "hermes",
|
||||
state: "invalid",
|
||||
remediation: expect.stringContaining("Hermes profile"),
|
||||
}),
|
||||
]),
|
||||
},
|
||||
});
|
||||
const unrelated = await run([
|
||||
"setup",
|
||||
"--client",
|
||||
"codex",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(unrelated.exitCode).toBe(0);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
@@ -32,157 +32,156 @@ afterEach(async () => {
|
||||
await Promise.all(browsers.splice(0).map(async (browser) => browser.close()));
|
||||
});
|
||||
|
||||
describe("browser CLI parity", () => {
|
||||
it(
|
||||
"returns the same Evidence discovery and inspection contracts",
|
||||
async () => {
|
||||
const browser = await startFakeCdpBrowser({
|
||||
sessionTimeline: "same_origin",
|
||||
webMcpTools: true,
|
||||
sensitiveShapes: true,
|
||||
commandEvents: partialScriptEvents,
|
||||
});
|
||||
browsers.push(browser);
|
||||
const listed = await runCli(
|
||||
["list-browser-targets", browser.endpoint, "--json"],
|
||||
process.env,
|
||||
);
|
||||
expect(listed).toMatchObject({
|
||||
operation: "list_browser_targets",
|
||||
provider: { id: "rea-cdp-browser" },
|
||||
normalized_result: {
|
||||
targets: expect.arrayContaining([
|
||||
expect.objectContaining({ target_id: "allowed-page" }),
|
||||
expect.objectContaining({ target_id: "disallowed-page" }),
|
||||
]),
|
||||
it(
|
||||
"browser CLI parity returns the same Evidence discovery and inspection contracts",
|
||||
async () => {
|
||||
const browser = await startFakeCdpBrowser({
|
||||
sessionTimeline: "same_origin",
|
||||
webMcpTools: true,
|
||||
sensitiveShapes: true,
|
||||
commandEvents: partialScriptEvents,
|
||||
eventsBeforeReply: ["Network.enable"],
|
||||
});
|
||||
browsers.push(browser);
|
||||
const listed = await runCli(
|
||||
["list-browser-targets", browser.endpoint, "--json"],
|
||||
process.env,
|
||||
);
|
||||
expect(listed).toMatchObject({
|
||||
operation: "list_browser_targets",
|
||||
provider: { id: "rea-cdp-browser" },
|
||||
normalized_result: {
|
||||
targets: expect.arrayContaining([
|
||||
expect.objectContaining({ target_id: "allowed-page" }),
|
||||
expect.objectContaining({ target_id: "disallowed-page" }),
|
||||
]),
|
||||
},
|
||||
});
|
||||
const inspected = await runCli(
|
||||
[
|
||||
"inspect-web-page",
|
||||
browser.endpoint,
|
||||
"allowed-page",
|
||||
"--observation-ms",
|
||||
"0",
|
||||
"--include-console-text",
|
||||
"--include-json-body-shapes",
|
||||
"--include-websocket-shapes",
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(inspected).toMatchObject({
|
||||
operation: "inspect_web_page",
|
||||
provider: { id: "rea-cdp-browser" },
|
||||
normalized_result: {
|
||||
target: { target_id: "allowed-page" },
|
||||
scripts: {
|
||||
items: [{ cdp_hash: null, length: null, is_module: null }],
|
||||
},
|
||||
});
|
||||
const inspected = await runCli(
|
||||
[
|
||||
"inspect-web-page",
|
||||
browser.endpoint,
|
||||
"allowed-page",
|
||||
"--observation-ms",
|
||||
"0",
|
||||
"--include-console-text",
|
||||
"--include-json-body-shapes",
|
||||
"--include-websocket-shapes",
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(inspected).toMatchObject({
|
||||
operation: "inspect_web_page",
|
||||
provider: { id: "rea-cdp-browser" },
|
||||
normalized_result: {
|
||||
target: { target_id: "allowed-page" },
|
||||
scripts: {
|
||||
items: [{ cdp_hash: null, length: null, is_module: null }],
|
||||
},
|
||||
network: {
|
||||
prior_activity_available: false,
|
||||
requests: [
|
||||
expect.objectContaining({
|
||||
body_shapes: expect.objectContaining({ status: "included" }),
|
||||
}),
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
const inspection = normalizedResult(inspected);
|
||||
const compared = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify({ inspection }),
|
||||
JSON.stringify({ inspection }),
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(compared).toMatchObject({
|
||||
operation: "compare_web_captures",
|
||||
normalized_result: {
|
||||
overall_status: "unknown",
|
||||
dimensions: {
|
||||
network: { status: "unknown", total_changes: 0 },
|
||||
},
|
||||
},
|
||||
});
|
||||
const scenarioCapture = completeScenarioCapture();
|
||||
const scenarioCompared = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
'{"rules":[]}',
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(scenarioCompared).toMatchObject({
|
||||
operation: "compare_web_captures",
|
||||
normalized_result: {
|
||||
comparison_kind: "browser_scenario",
|
||||
overall_status: "unchanged",
|
||||
alignment: { status: "aligned", aligned_steps: 2 },
|
||||
},
|
||||
});
|
||||
const intentionalPolicy = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
JSON.stringify({
|
||||
rules: [
|
||||
{
|
||||
rule_id: "domain",
|
||||
artifacts: ["url"],
|
||||
match: "app.example.test",
|
||||
replacement: "normalized.test",
|
||||
},
|
||||
],
|
||||
}),
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(intentionalPolicy).toMatchObject({
|
||||
normalized_result: {
|
||||
normalization: { rules: [{ rule_id: "domain" }] },
|
||||
},
|
||||
});
|
||||
|
||||
const malformedPolicy = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
"{invalid-policy",
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(malformedPolicy).toMatchObject({
|
||||
error: "Browser observation failed",
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_captures",
|
||||
issues: [
|
||||
{
|
||||
path: ["normalization"],
|
||||
reason: "invalid_format",
|
||||
expected: "JSON",
|
||||
},
|
||||
network: {
|
||||
prior_activity_available: false,
|
||||
requests: [
|
||||
expect.objectContaining({
|
||||
body_shapes: expect.objectContaining({ status: "included" }),
|
||||
}),
|
||||
],
|
||||
},
|
||||
});
|
||||
},
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
});
|
||||
},
|
||||
});
|
||||
const inspection = normalizedResult(inspected);
|
||||
const compared = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify({ inspection }),
|
||||
JSON.stringify({ inspection }),
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(compared).toMatchObject({
|
||||
operation: "compare_web_captures",
|
||||
normalized_result: {
|
||||
overall_status: "unknown",
|
||||
dimensions: {
|
||||
network: { status: "unknown", total_changes: 0 },
|
||||
},
|
||||
},
|
||||
});
|
||||
const scenarioCapture = completeScenarioCapture();
|
||||
const scenarioCompared = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
'{"rules":[]}',
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(scenarioCompared).toMatchObject({
|
||||
operation: "compare_web_captures",
|
||||
normalized_result: {
|
||||
comparison_kind: "browser_scenario",
|
||||
overall_status: "unchanged",
|
||||
alignment: { status: "aligned", aligned_steps: 2 },
|
||||
},
|
||||
});
|
||||
const intentionalPolicy = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
JSON.stringify({
|
||||
rules: [
|
||||
{
|
||||
rule_id: "domain",
|
||||
artifacts: ["url"],
|
||||
match: "app.example.test",
|
||||
replacement: "normalized.test",
|
||||
},
|
||||
],
|
||||
}),
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(intentionalPolicy).toMatchObject({
|
||||
normalized_result: {
|
||||
normalization: { rules: [{ rule_id: "domain" }] },
|
||||
},
|
||||
});
|
||||
|
||||
const malformedPolicy = await runCli(
|
||||
[
|
||||
"compare-web-captures",
|
||||
JSON.stringify(scenarioCapture),
|
||||
JSON.stringify(scenarioCapture),
|
||||
"--normalization-json",
|
||||
"{invalid-policy",
|
||||
"--json",
|
||||
],
|
||||
process.env,
|
||||
);
|
||||
expect(malformedPolicy).toMatchObject({
|
||||
error: "Browser observation failed",
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_captures",
|
||||
issues: [
|
||||
{
|
||||
path: ["normalization"],
|
||||
reason: "invalid_format",
|
||||
expected: "JSON",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
},
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
describe("browser CLI capture parity", () => {
|
||||
it(
|
||||
|
||||
@@ -0,0 +1,558 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { expect, onTestFinished } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { analysisCliErrorEnvelopeSchema } from "../../../src/contracts/errorSchemas.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { parseProcessCapture } from "../../../src/domain/process/processCaptureParsing.js";
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { itWithCaptureCapability } from "./processCaptureCapability.js";
|
||||
|
||||
const PROGRESS_INTERVAL_MS = 100;
|
||||
|
||||
const progressLineSchema = z.object({
|
||||
rea_progress: z.object({
|
||||
phase: z.string(),
|
||||
completed: z.number().nonnegative(),
|
||||
total: z.null(),
|
||||
message: z.string(),
|
||||
sequence: z.number().int().positive(),
|
||||
terminal: z.boolean().optional(),
|
||||
}),
|
||||
});
|
||||
|
||||
const progressMessageSchema = z.object({
|
||||
elapsed_ms: z.number().int().nonnegative(),
|
||||
frames: z.number().int().nonnegative(),
|
||||
samples: z.number().int().nonnegative(),
|
||||
interactions: z.number().int().nonnegative(),
|
||||
disposition: z
|
||||
.enum(["exited", "timeout", "idle_timeout", "cancelled", "failed"])
|
||||
.optional(),
|
||||
owned_process_group: z
|
||||
.enum(["cleaned", "failed", "unverified", "not_required"])
|
||||
.optional(),
|
||||
terminal_renderer: z
|
||||
.enum(["cleaned", "failed", "unverified", "not_required"])
|
||||
.optional(),
|
||||
temporary_root: z
|
||||
.enum(["cleaned", "failed", "unverified", "not_required"])
|
||||
.optional(),
|
||||
});
|
||||
|
||||
type ProgressLine = z.infer<typeof progressLineSchema>["rea_progress"];
|
||||
type ParsedProgress = ProgressLine & {
|
||||
readonly details: z.infer<typeof progressMessageSchema>;
|
||||
};
|
||||
|
||||
const parseProgressMessage = (message: string) => {
|
||||
const matched =
|
||||
/^elapsed_ms=(?<elapsed_ms>\d+) frames=(?<frames>\d+) samples=(?<samples>\d+) interactions=(?<interactions>\d+)(?: disposition=(?<disposition>exited|timeout|idle_timeout|cancelled|failed))?(?: owned_process_group=(?<owned_process_group>cleaned|failed|unverified|not_required) terminal_renderer=(?<terminal_renderer>cleaned|failed|unverified|not_required) temporary_root=(?<temporary_root>cleaned|failed|unverified|not_required))?$/u.exec(
|
||||
message,
|
||||
);
|
||||
if (matched?.groups === undefined)
|
||||
throw new Error(`Progress message is not capture status: ${message}`);
|
||||
return progressMessageSchema.parse({
|
||||
elapsed_ms: Number(matched.groups.elapsed_ms),
|
||||
frames: Number(matched.groups.frames),
|
||||
samples: Number(matched.groups.samples),
|
||||
interactions: Number(matched.groups.interactions),
|
||||
...(matched.groups.disposition === undefined
|
||||
? {}
|
||||
: { disposition: matched.groups.disposition }),
|
||||
...(matched.groups.owned_process_group === undefined
|
||||
? {}
|
||||
: {
|
||||
owned_process_group: matched.groups.owned_process_group,
|
||||
terminal_renderer: matched.groups.terminal_renderer,
|
||||
temporary_root: matched.groups.temporary_root,
|
||||
}),
|
||||
});
|
||||
};
|
||||
|
||||
interface CliCapture {
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
readonly exitCode: number | null;
|
||||
readonly lines: readonly {
|
||||
readonly update: ParsedProgress;
|
||||
}[];
|
||||
}
|
||||
|
||||
const writeScenario = async (
|
||||
prefix: string,
|
||||
scenario: Readonly<Record<string, unknown>>,
|
||||
): Promise<string> => {
|
||||
const root = await createTestTempDirectory(prefix);
|
||||
const path = join(root, "scenario.json");
|
||||
await writeFile(path, JSON.stringify(scenario));
|
||||
return path;
|
||||
};
|
||||
|
||||
const runCaptureCli = async (
|
||||
scenarioPath: string,
|
||||
options: { readonly stopOnRunning?: boolean } = {},
|
||||
): Promise<CliCapture> => {
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
["scripts/rea.mjs", "capture-process", scenarioPath, "--json"],
|
||||
{ stdio: ["ignore", "pipe", "pipe"] },
|
||||
);
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let pending = "";
|
||||
const lines: string[] = [];
|
||||
let sawRunning: (() => void) | undefined;
|
||||
const running = options.stopOnRunning
|
||||
? new Promise<void>((resolve) => {
|
||||
sawRunning = resolve;
|
||||
})
|
||||
: undefined;
|
||||
child.stdout.on("data", (chunk: Buffer) => {
|
||||
stdout += chunk.toString();
|
||||
});
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
pending += chunk.toString();
|
||||
const parts = pending.split("\n");
|
||||
pending = parts.pop() ?? "";
|
||||
for (const line of parts) {
|
||||
if (line.length === 0) continue;
|
||||
lines.push(line);
|
||||
if (line.includes('"phase":"running"')) sawRunning?.();
|
||||
}
|
||||
});
|
||||
const closed = new Promise<{ code: number | null }>((resolve, reject) => {
|
||||
child.once("error", reject);
|
||||
child.once("close", (code) => resolve({ code }));
|
||||
});
|
||||
if (running !== undefined) {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const appeared = await Promise.race([
|
||||
running.then(() => true),
|
||||
new Promise<boolean>((resolve) => {
|
||||
timer = setTimeout(() => resolve(false), 8_000);
|
||||
}),
|
||||
]).finally(() => {
|
||||
clearTimeout(timer);
|
||||
});
|
||||
if (!appeared) {
|
||||
child.kill("SIGKILL");
|
||||
await closed;
|
||||
throw new Error(
|
||||
`capture produced no running progress on stderr:\n${lines.join("\n")}`,
|
||||
);
|
||||
}
|
||||
child.kill("SIGINT");
|
||||
}
|
||||
const result = await closed;
|
||||
if (pending.length > 0) lines.push(pending);
|
||||
stderr = lines.join("\n");
|
||||
return {
|
||||
stdout,
|
||||
stderr,
|
||||
exitCode: result.code,
|
||||
lines: lines.map((line) => {
|
||||
const update = progressLineSchema.parse(JSON.parse(line)).rea_progress;
|
||||
return {
|
||||
update: { ...update, details: parseProgressMessage(update.message) },
|
||||
};
|
||||
}),
|
||||
};
|
||||
};
|
||||
|
||||
const assertMonotonic = (updates: readonly ParsedProgress[]): void => {
|
||||
for (const [index, update] of updates.entries()) {
|
||||
expect(update.total).toBeNull();
|
||||
expect(update.message).not.toMatch(/%|\bpercent/iu);
|
||||
const previous = updates[index - 1];
|
||||
if (previous === undefined) continue;
|
||||
expect(update.completed).toBeGreaterThanOrEqual(previous.completed);
|
||||
expect(update.sequence).toBe(previous.sequence + 1);
|
||||
}
|
||||
};
|
||||
|
||||
const assertRateBound = (
|
||||
intermediateCount: number,
|
||||
elapsedMs: number,
|
||||
): void => {
|
||||
// Pipe/transport buffering can deliver correctly spaced sends together.
|
||||
// Exact send spacing belongs to the reporter's controlled-clock test.
|
||||
expect(intermediateCount).toBeGreaterThan(1);
|
||||
expect(intermediateCount).toBeLessThanOrEqual(
|
||||
Math.floor(elapsedMs / PROGRESS_INTERVAL_MS) + 1,
|
||||
);
|
||||
};
|
||||
|
||||
const childScript = (source: string) => ["-e", source];
|
||||
|
||||
const connectProgressClient = async (
|
||||
recordNotifications = false,
|
||||
): Promise<{
|
||||
readonly client: Client;
|
||||
readonly notifications: string[];
|
||||
}> => {
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("Process capture must not launch a binary provider");
|
||||
});
|
||||
const server = createServer(
|
||||
{ kind: "session", session },
|
||||
{ logger: silentLogger },
|
||||
);
|
||||
const client = new Client({
|
||||
name: "process-capture-progress",
|
||||
version: "1",
|
||||
});
|
||||
const notifications: string[] = [];
|
||||
if (recordNotifications) {
|
||||
client.setNotificationHandler("notifications/progress", (notification) => {
|
||||
notifications.push(JSON.stringify(notification));
|
||||
});
|
||||
}
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
onTestFinished(async () => {
|
||||
await Promise.allSettled([client.close(), server.close(), session.close()]);
|
||||
});
|
||||
return { client, notifications };
|
||||
};
|
||||
|
||||
itWithCaptureCapability(
|
||||
"keeps capture Evidence on stdout and live status on stderr for a sparse child",
|
||||
async () => {
|
||||
const scenario = await writeScenario("rea-process-progress-sparse-", {
|
||||
executable: process.execPath,
|
||||
arguments: childScript(
|
||||
"process.stdout.write('SPARSE_CHILD_STARTED\\n'); setTimeout(() => process.stdout.write('SPARSE_CHILD_DONE\\n'), 1600);",
|
||||
),
|
||||
timeout_ms: 10_000,
|
||||
idle_timeout_ms: 10_000,
|
||||
});
|
||||
const capture = await runCaptureCli(scenario);
|
||||
expect(capture.exitCode, capture.stderr).toBe(0);
|
||||
expect(capture.stdout).not.toContain("rea_progress");
|
||||
const evidence = parseEvidence(JSON.parse(capture.stdout));
|
||||
const recorded = parseProcessCapture(evidence.normalized_result);
|
||||
expect(evidence).toMatchObject({
|
||||
operation: "capture_process_scenario",
|
||||
predicate_type: "rea.process-capture",
|
||||
});
|
||||
expect(recorded.frames.map(({ data }) => data).join("")).toContain(
|
||||
"SPARSE_CHILD_STARTED",
|
||||
);
|
||||
expect(capture.stderr).not.toContain("SPARSE_CHILD_STARTED");
|
||||
expect(capture.stderr).not.toContain("SPARSE_CHILD_DONE");
|
||||
const updates = capture.lines.map(({ update }) => update);
|
||||
assertMonotonic(updates);
|
||||
const running = updates.filter(({ phase }) => phase === "running");
|
||||
expect(running.length).toBeGreaterThanOrEqual(3);
|
||||
const elapsed = running.map(({ details }) => details.elapsed_ms);
|
||||
expect(Math.max(...elapsed) - Math.min(...elapsed)).toBeGreaterThanOrEqual(
|
||||
1_000,
|
||||
);
|
||||
expect(running.every(({ details }) => details.frames <= 2)).toBe(true);
|
||||
expect(updates.at(-1)).toMatchObject({
|
||||
phase: "cleanup",
|
||||
terminal: true,
|
||||
details: {
|
||||
disposition: "exited",
|
||||
owned_process_group: "cleaned",
|
||||
terminal_renderer: "cleaned",
|
||||
temporary_root: "cleaned",
|
||||
},
|
||||
});
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"bounds stderr progress while a busy child emits many output chunks",
|
||||
async () => {
|
||||
const scenario = await writeScenario("rea-process-progress-busy-", {
|
||||
executable: process.execPath,
|
||||
arguments: childScript(
|
||||
"let count = 0; const timer = setInterval(() => { process.stdout.write('BUSY_CHILD_LINE\\n'); count += 1; if (count >= 50) clearInterval(timer); }, 12);",
|
||||
),
|
||||
timeout_ms: 10_000,
|
||||
idle_timeout_ms: 10_000,
|
||||
});
|
||||
const capture = await runCaptureCli(scenario);
|
||||
expect(capture.exitCode, capture.stderr).toBe(0);
|
||||
const evidence = parseEvidence(JSON.parse(capture.stdout));
|
||||
const recorded = parseProcessCapture(evidence.normalized_result);
|
||||
const occurrences =
|
||||
recorded.frames
|
||||
.map(({ data }) => data)
|
||||
.join("")
|
||||
.split("BUSY_CHILD_LINE").length - 1;
|
||||
expect(occurrences).toBeGreaterThanOrEqual(40);
|
||||
expect(capture.stderr).not.toContain("BUSY_CHILD_LINE");
|
||||
const updates = capture.lines.map(({ update }) => update);
|
||||
assertMonotonic(updates);
|
||||
expect(updates.length).toBeLessThan(occurrences / 2);
|
||||
const nonTerminal = capture.lines.filter(
|
||||
({ update }) => update.terminal !== true,
|
||||
);
|
||||
assertRateBound(
|
||||
nonTerminal.length,
|
||||
updates.at(-1)?.details.elapsed_ms ?? 0,
|
||||
);
|
||||
expect(
|
||||
Math.max(...updates.map(({ details }) => details.frames)),
|
||||
).toBeGreaterThanOrEqual(10);
|
||||
expect(updates.at(-1)?.details.owned_process_group).toBe("cleaned");
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"reports timeout cleanup on stderr without changing Evidence shape",
|
||||
async () => {
|
||||
const scenario = await writeScenario("rea-process-progress-timeout-", {
|
||||
executable: process.execPath,
|
||||
arguments: childScript("setInterval(() => undefined, 1000);"),
|
||||
timeout_ms: 700,
|
||||
idle_timeout_ms: 10_000,
|
||||
});
|
||||
const capture = await runCaptureCli(scenario);
|
||||
expect(capture.exitCode, capture.stderr).toBe(0);
|
||||
const evidence = parseEvidence(JSON.parse(capture.stdout));
|
||||
const recorded = parseProcessCapture(evidence.normalized_result);
|
||||
expect(recorded.exit.reason).toBe("timeout");
|
||||
expect(capture.stdout).not.toContain("rea_progress");
|
||||
const updates = capture.lines.map(({ update }) => update);
|
||||
assertMonotonic(updates);
|
||||
expect(updates.at(-1)).toMatchObject({
|
||||
phase: "cleanup",
|
||||
terminal: true,
|
||||
details: {
|
||||
disposition: "timeout",
|
||||
owned_process_group: "cleaned",
|
||||
},
|
||||
});
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"reports cleanup status on stderr when capture is cancelled",
|
||||
async () => {
|
||||
const scenario = await writeScenario("rea-process-progress-cancel-", {
|
||||
executable: process.execPath,
|
||||
arguments: childScript("setInterval(() => undefined, 1000);"),
|
||||
timeout_ms: 15_000,
|
||||
idle_timeout_ms: 15_000,
|
||||
});
|
||||
const capture = await runCaptureCli(scenario, { stopOnRunning: true });
|
||||
expect(capture.exitCode).toBe(130);
|
||||
expect(
|
||||
analysisCliErrorEnvelopeSchema.parse(JSON.parse(capture.stdout)),
|
||||
).toMatchObject({
|
||||
code: "cancelled",
|
||||
details: { operation: "process_capture", cleanup: "complete" },
|
||||
});
|
||||
expect(capture.stdout).not.toContain("rea_progress");
|
||||
const updates = capture.lines.map(({ update }) => update);
|
||||
assertMonotonic(updates);
|
||||
expect(updates.at(-1)).toMatchObject({
|
||||
phase: "cleanup",
|
||||
terminal: true,
|
||||
details: {
|
||||
disposition: "cancelled",
|
||||
owned_process_group: "cleaned",
|
||||
},
|
||||
});
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"forwards live capture progress only when the MCP client supplies a token",
|
||||
async () => {
|
||||
const silentClient = await connectProgressClient(true);
|
||||
const { client: tokenClient } = await connectProgressClient();
|
||||
const { notifications } = silentClient;
|
||||
const silent = await silentClient.client.callTool({
|
||||
name: "capture_process_scenario",
|
||||
arguments: {
|
||||
executable: process.execPath,
|
||||
arguments: childScript("process.stdout.write('SILENT_CHILD_OK\\n');"),
|
||||
timeout_ms: 10_000,
|
||||
idle_timeout_ms: 10_000,
|
||||
},
|
||||
});
|
||||
expect(silent.isError, JSON.stringify(silent)).not.toBe(true);
|
||||
expect(notifications).toEqual([]);
|
||||
const evidence = toolContract(
|
||||
"capture_process_scenario",
|
||||
).outputSchema.parse(silent.structuredContent);
|
||||
expect(
|
||||
evidence.normalized_result.frames.map((frame) => frame.data).join(""),
|
||||
).toContain("SILENT_CHILD_OK");
|
||||
|
||||
const progress: Array<{
|
||||
readonly progress: number;
|
||||
readonly total: number | undefined;
|
||||
readonly message: string | undefined;
|
||||
}> = [];
|
||||
const captured = await tokenClient.callTool(
|
||||
{
|
||||
name: "capture_process_scenario",
|
||||
arguments: {
|
||||
executable: process.execPath,
|
||||
arguments: childScript(
|
||||
"process.stdout.write('TOKEN_CHILD_STARTED\\n'); setTimeout(() => process.stdout.write('TOKEN_CHILD_DONE\\n'), 1200);",
|
||||
),
|
||||
timeout_ms: 10_000,
|
||||
idle_timeout_ms: 10_000,
|
||||
},
|
||||
},
|
||||
{
|
||||
onprogress: (update) => {
|
||||
progress.push({
|
||||
progress: update.progress,
|
||||
total: update.total,
|
||||
message: update.message,
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(captured.isError, JSON.stringify(captured)).not.toBe(true);
|
||||
expect(progress.length).toBeGreaterThanOrEqual(3);
|
||||
expect(progress.every(({ total }) => total === undefined)).toBe(true);
|
||||
for (const [index, update] of progress.entries()) {
|
||||
const previous = progress[index - 1];
|
||||
if (previous !== undefined)
|
||||
expect(update.progress).toBeGreaterThanOrEqual(previous.progress);
|
||||
expect(update.message ?? "").not.toContain("TOKEN_CHILD_STARTED");
|
||||
expect(update.message ?? "").not.toContain("TOKEN_CHILD_DONE");
|
||||
}
|
||||
const messages = progress.map(({ message }) => message ?? "");
|
||||
expect(messages.some((message) => message.startsWith("running:"))).toBe(
|
||||
true,
|
||||
);
|
||||
expect(messages.at(-1)).toContain("cleanup:");
|
||||
expect(messages.at(-1)).toContain("owned_process_group=cleaned");
|
||||
expect(messages.at(-1)).toContain("disposition=exited");
|
||||
const nonTerminal = progress.slice(0, -1);
|
||||
assertRateBound(
|
||||
nonTerminal.length,
|
||||
parseProgressMessage((messages.at(-1) ?? "").slice("cleanup: ".length))
|
||||
.elapsed_ms,
|
||||
);
|
||||
const tokenEvidence = toolContract(
|
||||
"capture_process_scenario",
|
||||
).outputSchema.parse(captured.structuredContent);
|
||||
expect(
|
||||
tokenEvidence.normalized_result.frames
|
||||
.map((frame) => frame.data)
|
||||
.join(""),
|
||||
).toContain("TOKEN_CHILD_STARTED");
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"reports timeout and cancellation cleanup through MCP progress",
|
||||
async () => {
|
||||
const { client } = await connectProgressClient();
|
||||
const timeoutProgress: Array<{
|
||||
readonly message: string | undefined;
|
||||
readonly total: number | undefined;
|
||||
}> = [];
|
||||
const timedOut = await client.callTool(
|
||||
{
|
||||
name: "capture_process_scenario",
|
||||
arguments: {
|
||||
executable: process.execPath,
|
||||
arguments: childScript("setInterval(() => undefined, 1000);"),
|
||||
timeout_ms: 600,
|
||||
idle_timeout_ms: 10_000,
|
||||
},
|
||||
},
|
||||
{
|
||||
onprogress: (update) => {
|
||||
timeoutProgress.push({
|
||||
message: update.message,
|
||||
total: update.total,
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(timedOut.isError, JSON.stringify(timedOut)).not.toBe(true);
|
||||
const timedOutEvidence = toolContract(
|
||||
"capture_process_scenario",
|
||||
).outputSchema.parse(timedOut.structuredContent);
|
||||
expect(timedOutEvidence.normalized_result.exit.reason).toBe("timeout");
|
||||
expect(timeoutProgress.every(({ total }) => total === undefined)).toBe(
|
||||
true,
|
||||
);
|
||||
expect(timeoutProgress.at(-1)?.message).toContain("disposition=timeout");
|
||||
expect(timeoutProgress.at(-1)?.message).toContain(
|
||||
"owned_process_group=cleaned",
|
||||
);
|
||||
|
||||
const cancelling = await connectProgressClient(true);
|
||||
// Prepare the SDK's full output-schema cache before timing capture control.
|
||||
await cancelling.client.callTool({ name: "binary_session", arguments: {} });
|
||||
const controller = new AbortController();
|
||||
const stall = setTimeout(() => controller.abort(), 5_000);
|
||||
onTestFinished(() => clearTimeout(stall));
|
||||
const call = cancelling.client.callTool(
|
||||
{
|
||||
name: "capture_process_scenario",
|
||||
arguments: {
|
||||
executable: process.execPath,
|
||||
arguments: childScript("setInterval(() => undefined, 1000);"),
|
||||
timeout_ms: 15_000,
|
||||
idle_timeout_ms: 15_000,
|
||||
},
|
||||
_meta: { progressToken: "capture-cancel" },
|
||||
},
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
const completion = call.then(
|
||||
(reply) => ({ kind: "reply" as const, reply }),
|
||||
(cause: unknown) => ({ kind: "error" as const, cause }),
|
||||
);
|
||||
try {
|
||||
await expect
|
||||
.poll(
|
||||
() =>
|
||||
cancelling.notifications.some((note) => note.includes("running:")),
|
||||
{ timeout: 4_000 },
|
||||
)
|
||||
.toBe(true);
|
||||
controller.abort();
|
||||
const outcome = await completion;
|
||||
if (outcome.kind !== "error")
|
||||
throw new Error("Cancelled capture must reject its client wait");
|
||||
expect(outcome.cause).toMatchObject({
|
||||
message: expect.stringMatching(/abort/iu),
|
||||
});
|
||||
} finally {
|
||||
clearTimeout(stall);
|
||||
controller.abort();
|
||||
await completion;
|
||||
}
|
||||
await expect
|
||||
.poll(() => cancelling.notifications.join("\n"))
|
||||
.toContain("disposition=cancelled");
|
||||
expect(cancelling.notifications.join("\n")).toContain(
|
||||
"owned_process_group=cleaned",
|
||||
);
|
||||
expect(cancelling.notifications.join("\n")).not.toMatch(
|
||||
/setInterval|CHILD/u,
|
||||
);
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
Vendored
+16
-10
@@ -196,22 +196,28 @@ const replyToCommand = (
|
||||
);
|
||||
return;
|
||||
}
|
||||
socket.send(
|
||||
JSON.stringify({
|
||||
id: command.id,
|
||||
result,
|
||||
}),
|
||||
);
|
||||
if (options.malformedEventOnMethod === command.method)
|
||||
socket.send("{not-json");
|
||||
if (options.malformedEventShapeOnMethod === command.method)
|
||||
socket.send(JSON.stringify({ method: 42 }));
|
||||
const reply = () => {
|
||||
socket.send(
|
||||
JSON.stringify({
|
||||
id: command.id,
|
||||
result,
|
||||
}),
|
||||
);
|
||||
if (options.malformedEventOnMethod === command.method)
|
||||
socket.send("{not-json");
|
||||
if (options.malformedEventShapeOnMethod === command.method)
|
||||
socket.send(JSON.stringify({ method: 42 }));
|
||||
};
|
||||
const eventsFirst =
|
||||
options.eventsBeforeReply?.includes(command.method) === true;
|
||||
if (!eventsFirst) reply();
|
||||
const customEvents = options.commandEvents?.(
|
||||
command,
|
||||
`http://127.0.0.1:${String(port)}`,
|
||||
);
|
||||
if (customEvents === undefined) emitEvents(socket, command, port, options);
|
||||
else for (const event of customEvents) socket.send(JSON.stringify(event));
|
||||
if (eventsFirst) reply();
|
||||
if (options.closeAfterMethod === command.method)
|
||||
setImmediate(() => socket.close());
|
||||
};
|
||||
|
||||
+2
@@ -28,6 +28,8 @@ export interface FakeCdpBrowser {
|
||||
}
|
||||
|
||||
export interface FakeOptions {
|
||||
/** Send these commands' producer events before acknowledging completion, as a wire barrier. */
|
||||
readonly eventsBeforeReply?: readonly string[];
|
||||
/** Return a native-shaped command rejection through the actual wire parser. */
|
||||
readonly commandError?: (
|
||||
command: FakeCdpCommand,
|
||||
|
||||
+5
-1
@@ -43,7 +43,7 @@ HTML files. No URL list needs to be maintained.
|
||||
- `public/first-investigation/index.html`: a guided Notes export investigation, from setup to a checked CSV prediction.
|
||||
- `public/showcase/index.html`: the case-study index.
|
||||
- `public/blog/index.html`: articles about reconstruction methods, ports, mods and reverse engineering.
|
||||
- `public/blog/touhou-reconstruction/index.html`: a personal essay about agent-driven reconstruction, knowledge that compounds across TH08 and TH095, and the workflow's applicability to PC-98 TH04.
|
||||
- `public/blog/touhou-reconstruction/index.html`: a personal essay about agent autonomy, oracle quality, knowledge carried between TH08 and TH095, PC-98 TH04, and the path from exact reconstruction to readable source and modern ports.
|
||||
- `public/showcase/aegis/index.html`: Aegis's Android login-code calculation, with an adjustable clock and reference checks.
|
||||
- `public/showcase/dx-ball/index.html`: sound-pan investigation and project status.
|
||||
- `public/showcase/notion/index.html`: Notion's Electron clipboard bridge and rich clipboard format.
|
||||
@@ -136,6 +136,10 @@ quoted README passage and milestone counting rules are in
|
||||
The completed text has normal search metadata and is included in the generated
|
||||
sitemap.
|
||||
|
||||
Its opening flywheel connects agent autonomy, reference-based oracles and
|
||||
repository memory. The essay develops those ideas through the project examples.
|
||||
Figure labels are selectable HTML; inline SVG supplies the feedback arrows.
|
||||
|
||||
Keep the copy direct and specific. Explain the task and the result before listing
|
||||
tool names. Setup commands and runtime requirements should match the released
|
||||
package. Keep the core learning path inside the site. Source, issue and evidence
|
||||
|
||||
@@ -173,3 +173,115 @@ observations, also confirmed by the author in this revision request. It is
|
||||
not attributed to a new automated website test or a fresh runtime replay.
|
||||
The 64-bit port is described as in development. The earlier four-product build
|
||||
and function-count milestones retain their historical scopes.
|
||||
|
||||
## Autonomy, oracles and the compounding flywheel
|
||||
|
||||
The author's requested revision emphasizes two principles: agents have broad
|
||||
autonomy to investigate, and their hypotheses are tested by oracles with concrete
|
||||
references and explicit acceptance conditions. These ideas are developed in the
|
||||
Factory's agent-autonomy and semantic-reconstruction documents listed above.
|
||||
An oracle result supports the property and inputs actually checked; the essay
|
||||
retains human responsibility for project milestones and release decisions.
|
||||
|
||||
The flywheel is an explanatory model of this relationship. Its passing branch
|
||||
retains a checked result and its evidence; a mismatch returns to investigation.
|
||||
Code, checks and recorded lessons improve the next starting point, with TH08,
|
||||
TH095 and TH04 illustrating knowledge carried between projects. It does not
|
||||
depict a measured performance ratio or give REA sole credit for reconstruction.
|
||||
|
||||
The first inline reference to GensokyoClub's public README links to the author's
|
||||
selected [Internet Archive capture](https://web.archive.org/web/20260909051514/https://github.com/GensokyoClub/th08#important-notice).
|
||||
The pinned source caption and 23-word excerpt retain their original provenance.
|
||||
The author approved this revision for publication on 10 October 2026.
|
||||
|
||||
## TH08's floating-literal oracle failure
|
||||
|
||||
The author recalled a floating-point comparison bug. The source history and
|
||||
public GitHub record identify it as a reference-data verification gap, rather
|
||||
than a rounding-tolerance problem. The original Japanese TH08 1.00d executable
|
||||
uses double `128.0` in the power arm of `ItemManager::OnUpdate`'s item-attraction
|
||||
condition. The reconstructed source used `0.0`.
|
||||
|
||||
- [Issue #10](https://github.com/N0zoM1z0/th08/issues/10), opened on 2 September
|
||||
2026, records the downstream Switch-port report, the target instructions and
|
||||
the constant bytes. Normal power is non-negative, so the erroneous power arm
|
||||
was effectively unconditional after crossing the collection line. Focus and
|
||||
shot-type exceptions were separate, correct arms of the condition.
|
||||
- [PR #11](https://github.com/N0zoM1z0/th08/pull/11), merged on 2 September,
|
||||
contains commit
|
||||
[`7148a76b`](https://github.com/N0zoM1z0/th08/commit/7148a76ba224742677895c7160807a68897957d5).
|
||||
It changes the threshold from `0.0` to `128.0`, fixes the manifest literal
|
||||
identity, and adds source-symbol/declared-data/target-data checks for
|
||||
explicitly recorded floating literals.
|
||||
- [PR #13](https://github.com/N0zoM1z0/th08/pull/13), merged the same day,
|
||||
contains commit
|
||||
[`a393f400`](https://github.com/N0zoM1z0/th08/commit/a393f400786e4cc6cbb1000af0dad3276da12c78).
|
||||
It audits 1,548 configured floating-literal references, corrects twelve more
|
||||
stale references across five accepted functions, and makes verification
|
||||
automatic for every 32-bit or 64-bit `__real@...` relocation. Its tests include
|
||||
deliberately invalid literal declarations and representations.
|
||||
|
||||
The older comparison replayed the target address into the instruction's
|
||||
relocation field, then compared instruction bytes without checking the
|
||||
floating-literal data at that address. Source `0.0` could therefore appear
|
||||
exact against target `128.0`. The essay explains this mechanism in ordinary
|
||||
language and keeps implementation details in the linked record. The count is
|
||||
of references, not distinct values or new functions.
|
||||
|
||||
The website work reexamined the historical diffs and retrieved the issue and
|
||||
PR metadata with `gh`; it did not rerun the game's historical compiler or
|
||||
runtime. The author supplies the first-person lesson about working as one
|
||||
human with agents and about responsibility for oracle quality.
|
||||
|
||||
## Readability and the learned phase order
|
||||
|
||||
The semantic discussion is based on the author's requested explanation,
|
||||
[TH08's semantic playbook](https://github.com/N0zoM1z0/th08/blob/d2a00f4acb4a13916e07a00bd13f714961714628/docs/SEMANTIC_PLAYBOOK.md),
|
||||
and the Factory's
|
||||
[semantic-reconstruction workflow](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/717e1b8560783ed17a39fdf159d30aa16bb307a7/docs/semantic-reconstruction.md).
|
||||
The workflow's latest public change was checked using `gh` on 10 October.
|
||||
|
||||
The recommended order is an exact baseline, a working historical-platform
|
||||
product, semantic reconstruction with both validation paths, then portable
|
||||
products. The original target/compiler comparisons and the historical
|
||||
build/runtime checks answer different questions. Both are preserved during
|
||||
semantic work. Names, ownership and protocol interpretations also need
|
||||
producer/consumer evidence; a passing check does not establish their English
|
||||
meaning. Modern ports compare behavior with the established historical build
|
||||
without requiring the new architecture's machine code to match the old one.
|
||||
|
||||
This is a method learned from the projects. TH08's earlier portable releases
|
||||
preceded some later historical-platform audits. The essay states that history
|
||||
explicitly rather than presenting the current order as the original chronology.
|
||||
The game-RE background and the assembly/raw-offset aside are author-provided
|
||||
perspective. Transfer to other fields is a proposal to carry the principles
|
||||
while choosing references appropriate to the new domain.
|
||||
|
||||
## Industrial analogy: instruments develop with machines
|
||||
|
||||
The [Science Museum Group's steam-engine indicator record](https://collection.sciencemuseumgroup.org.uk/objects/co51439/watts-steam-engine-indicator),
|
||||
read on 10 October 2026, describes Boulton and Watt's introduction of the
|
||||
instrument in 1796 to assist with valve adjustment. It also explains how adding
|
||||
a pencil and moving paper made cylinder pressure through the stroke recordable.
|
||||
The essay paraphrases this description and links it directly. It does not use
|
||||
the record's images, quote its text, or date the displayed object or pressure
|
||||
diagram to 1796.
|
||||
|
||||
The connection to reconstruction tools is the author's analogy: rapid
|
||||
execution needs measurement, and the instruments themselves require improvement.
|
||||
The claim that this is an early industrial stage is the essay's interpretation
|
||||
of the project experience. The historical example supports the measurement
|
||||
comparison rather than a measured prediction about AI productivity.
|
||||
|
||||
## Prose revision
|
||||
|
||||
The later prose pass reduces repeated enumerations and develops each lesson
|
||||
through its project example. It preserves the milestone figures, quotation,
|
||||
source links and four-stage method. The two author-selected emphasized sentences
|
||||
retain their wording.
|
||||
|
||||
The opening takeaway now uses five bullets: agent autonomy, repository/Git
|
||||
memory, evidence and explicit unknowns, testing the oracle, and the early stage
|
||||
of the industrial shift. Each has a direct lead and a short explanation. The
|
||||
historical-build/semantic/port sequence remains in the body rather than becoming
|
||||
another takeaway item.
|
||||
|
||||
+19
-3
@@ -96,9 +96,25 @@ formula, default settings, slider outputs and reference cases consistent.
|
||||
|
||||
## Touhou reconstruction essay
|
||||
|
||||
The essay opens with a takeaway and develops its argument in prose. It has no
|
||||
opening flow figure. Project dates and the attributed README excerpt support
|
||||
the narrative; detailed milestone references are available in a native
|
||||
The essay opens with a takeaway and a flywheel explaining how experience
|
||||
compounds. The figure uses selectable HTML labels and inline SVG connectors:
|
||||
an autonomous agent proposes an experiment, an oracle tests it against a
|
||||
reference, and code, evidence, checks and lessons remain in the repository.
|
||||
A dashed mismatch branch returns to the agent for revision. The repository
|
||||
supplies a better starting point for the next investigation; TH08 → TH095 →
|
||||
TH04 illustrates knowledge carried between projects. Human direction names
|
||||
the goal and acceptance criteria above the cycle.
|
||||
|
||||
The caption connects reuse with improving the checks themselves. The TH08
|
||||
floating-literal case in the essay explains why a passing result can need
|
||||
reexamination when an oracle's blind spot is discovered. Preserve that link
|
||||
between the figure's ideal loop and the infrastructure lessons in the prose.
|
||||
|
||||
The shared stylesheet positions the labels and gives narrow screens separate
|
||||
connector geometry. Keep the arrows clear of the text and retain the same
|
||||
meaning at every width. This is an explanatory model, with no measured speed
|
||||
or productivity multiplier. Project dates and the attributed README excerpt
|
||||
support the narrative; detailed milestone references are available in a native
|
||||
disclosure. Their provenance is documented in
|
||||
[evidence/touhou-reconstruction-method.md](evidence/touhou-reconstruction-method.md).
|
||||
|
||||
|
||||
@@ -692,6 +692,178 @@ pre code {
|
||||
.blog-takeaway p:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
.blog-takeaway ul {
|
||||
margin: 0;
|
||||
padding-left: 1.15rem;
|
||||
}
|
||||
.blog-takeaway li + li {
|
||||
margin-top: 0.75rem;
|
||||
}
|
||||
.reconstruction-flywheel {
|
||||
max-width: 880px;
|
||||
margin: 0 auto 4rem;
|
||||
padding-block: 1.5rem 1.75rem;
|
||||
border-block: 1px solid var(--line);
|
||||
}
|
||||
.flywheel-heading {
|
||||
text-align: center;
|
||||
}
|
||||
.flywheel-heading .eyebrow {
|
||||
margin-bottom: 0.6rem;
|
||||
color: var(--accent);
|
||||
}
|
||||
.flywheel-heading > p:last-child {
|
||||
color: var(--muted);
|
||||
font-size: 0.875rem;
|
||||
}
|
||||
.flywheel-diagram {
|
||||
position: relative;
|
||||
max-width: 760px;
|
||||
height: 440px;
|
||||
margin-inline: auto;
|
||||
}
|
||||
.flywheel-arrows {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
.flywheel-path-mobile {
|
||||
display: none;
|
||||
}
|
||||
.flywheel-node {
|
||||
position: absolute;
|
||||
width: 30%;
|
||||
text-align: center;
|
||||
transform: translate(-50%, -50%);
|
||||
}
|
||||
.flywheel-node p {
|
||||
margin: 0.6rem 0 0;
|
||||
color: var(--muted);
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
.flywheel-node .flywheel-kicker {
|
||||
margin: 0 0 0.4rem;
|
||||
color: var(--accent);
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 600;
|
||||
letter-spacing: 0.09em;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.flywheel-node h3 {
|
||||
margin: 0;
|
||||
font-size: 1.5rem;
|
||||
line-height: 1.2;
|
||||
}
|
||||
.flywheel-agent {
|
||||
top: 15%;
|
||||
left: 50%;
|
||||
width: 38%;
|
||||
}
|
||||
.flywheel-oracle {
|
||||
top: 58%;
|
||||
left: 79%;
|
||||
}
|
||||
.flywheel-memory {
|
||||
top: 58%;
|
||||
left: 21%;
|
||||
}
|
||||
.flywheel-retry,
|
||||
.flywheel-center,
|
||||
.flywheel-reuse,
|
||||
.flywheel-pass {
|
||||
position: absolute;
|
||||
margin: 0;
|
||||
padding: 0.25rem;
|
||||
background: var(--page);
|
||||
color: var(--muted);
|
||||
font-size: 0.75rem;
|
||||
line-height: 1.5;
|
||||
text-align: center;
|
||||
transform: translate(-50%, -50%);
|
||||
}
|
||||
.flywheel-retry {
|
||||
top: 36%;
|
||||
left: 68%;
|
||||
width: 23%;
|
||||
}
|
||||
.flywheel-center {
|
||||
top: 57%;
|
||||
left: 50%;
|
||||
width: 20%;
|
||||
color: var(--accent);
|
||||
font-size: 0.9375rem;
|
||||
}
|
||||
.flywheel-reuse {
|
||||
top: 33%;
|
||||
left: 23%;
|
||||
color: var(--accent);
|
||||
}
|
||||
.flywheel-pass {
|
||||
top: 94%;
|
||||
left: 50%;
|
||||
width: 75%;
|
||||
color: var(--accent);
|
||||
font-size: 0.8125rem;
|
||||
}
|
||||
.flywheel-projects {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
gap: 1.4rem;
|
||||
margin-top: 0.3rem;
|
||||
}
|
||||
.flywheel-projects a {
|
||||
font-size: 0.9375rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
.flywheel-projects span {
|
||||
color: var(--muted);
|
||||
}
|
||||
.reconstruction-flywheel figcaption {
|
||||
margin: 1rem auto 0;
|
||||
max-width: 600px;
|
||||
color: var(--muted);
|
||||
font-size: 0.875rem;
|
||||
text-align: center;
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
.reconstruction-flywheel {
|
||||
margin-bottom: 2.75rem;
|
||||
}
|
||||
.flywheel-diagram {
|
||||
height: 405px;
|
||||
}
|
||||
.flywheel-agent {
|
||||
width: 52%;
|
||||
}
|
||||
.flywheel-path-desktop {
|
||||
display: none;
|
||||
}
|
||||
.flywheel-path-mobile {
|
||||
display: initial;
|
||||
}
|
||||
.flywheel-node h3 {
|
||||
font-size: 1.125rem;
|
||||
}
|
||||
.flywheel-node p {
|
||||
font-size: 0.75rem;
|
||||
}
|
||||
.flywheel-node .flywheel-kicker {
|
||||
font-size: 0.6rem;
|
||||
letter-spacing: 0.05em;
|
||||
}
|
||||
.flywheel-center {
|
||||
font-size: 0.75rem;
|
||||
}
|
||||
.flywheel-retry {
|
||||
font-size: 0.625rem;
|
||||
}
|
||||
.flywheel-projects {
|
||||
gap: 1rem;
|
||||
}
|
||||
}
|
||||
.blog-article blockquote {
|
||||
margin: 1.6rem 0;
|
||||
padding-left: 1.25rem;
|
||||
|
||||
@@ -147,21 +147,166 @@
|
||||
<p class="page-meta">By N0zoM1z0 · 10 October 2026</p>
|
||||
<div class="blog-takeaway">
|
||||
<p class="eyebrow">Takeaway</p>
|
||||
<p>
|
||||
<strong
|
||||
>Agent-driven reverse engineering changes how the work can
|
||||
accumulate.</strong
|
||||
>
|
||||
An investigation can become a checked result, a failed experiment
|
||||
can become a reusable test, and one project's experience can become
|
||||
the next project's starting point.
|
||||
</p>
|
||||
<p>
|
||||
Reconstructing TH08, TH095 and TH04 convinced me that RE is entering
|
||||
its industrial age.
|
||||
</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Give agents autonomy over the investigation.</strong> Set
|
||||
a clear goal and acceptance criteria, then let the agent decide
|
||||
what to try next.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Use the repository and Git as working memory.</strong>
|
||||
Keep the reasoning with the code. Git checkpoints let the next
|
||||
session continue after the conversation ends.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Let evidence decide what you claim.</strong> A confident
|
||||
explanation still needs testing. “Unknown” is a valid answer when
|
||||
the evidence is incomplete.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Test the oracle too.</strong> Try cases it should reject.
|
||||
If it misses a bug, revisit the earlier results that depended on
|
||||
it.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
>We are in the early days of RE's industrial revolution.</strong
|
||||
>
|
||||
The methods are still taking shape. There is plenty left to
|
||||
explore.
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<figure class="reconstruction-flywheel" aria-labelledby="flywheel-title">
|
||||
<div class="flywheel-heading">
|
||||
<p class="eyebrow" id="flywheel-title">How experience compounds</p>
|
||||
<p><strong>Human direction</strong> · Goal and acceptance criteria</p>
|
||||
</div>
|
||||
<div class="flywheel-diagram">
|
||||
<svg
|
||||
class="flywheel-arrows"
|
||||
viewBox="0 0 720 470"
|
||||
preserveAspectRatio="none"
|
||||
aria-hidden="true"
|
||||
focusable="false"
|
||||
>
|
||||
<defs>
|
||||
<marker
|
||||
id="flywheel-forward"
|
||||
markerWidth="10"
|
||||
markerHeight="10"
|
||||
refX="7"
|
||||
refY="4"
|
||||
orient="auto"
|
||||
markerUnits="userSpaceOnUse"
|
||||
>
|
||||
<path
|
||||
d="M 0 0 L 8 4 L 0 8"
|
||||
fill="none"
|
||||
stroke="#294f82"
|
||||
stroke-width="2"
|
||||
/>
|
||||
</marker>
|
||||
<marker
|
||||
id="flywheel-revise"
|
||||
markerWidth="10"
|
||||
markerHeight="10"
|
||||
refX="7"
|
||||
refY="4"
|
||||
orient="auto"
|
||||
markerUnits="userSpaceOnUse"
|
||||
>
|
||||
<path
|
||||
d="M 0 0 L 8 4 L 0 8"
|
||||
fill="none"
|
||||
stroke="#5b6570"
|
||||
stroke-width="1.5"
|
||||
/>
|
||||
</marker>
|
||||
</defs>
|
||||
<g
|
||||
fill="none"
|
||||
stroke="#294f82"
|
||||
stroke-width="2.5"
|
||||
marker-end="url(#flywheel-forward)"
|
||||
>
|
||||
<path
|
||||
class="flywheel-path-desktop"
|
||||
d="M 444 115 C 537 111 624 157 627 208"
|
||||
/>
|
||||
<path d="M 528 365 C 456 437 263 437 191 365" />
|
||||
<path
|
||||
class="flywheel-path-desktop"
|
||||
d="M 99 209 C 94 145 170 97 271 104"
|
||||
/>
|
||||
<path
|
||||
class="flywheel-path-mobile"
|
||||
d="M 530 115 C 614 108 681 164 674 215"
|
||||
/>
|
||||
<path
|
||||
class="flywheel-path-mobile"
|
||||
d="M 87 198 C 74 143 135 119 190 120"
|
||||
/>
|
||||
</g>
|
||||
<path
|
||||
class="flywheel-path-desktop"
|
||||
d="M 515 215 C 500 165 450 166 430 127"
|
||||
fill="none"
|
||||
stroke="#5b6570"
|
||||
stroke-width="1.5"
|
||||
stroke-dasharray="5 5"
|
||||
marker-end="url(#flywheel-revise)"
|
||||
/>
|
||||
<path
|
||||
class="flywheel-path-mobile"
|
||||
d="M 510 204 C 404 204 391 187 412 136"
|
||||
fill="none"
|
||||
stroke="#5b6570"
|
||||
stroke-width="1.5"
|
||||
stroke-dasharray="5 5"
|
||||
marker-end="url(#flywheel-revise)"
|
||||
/>
|
||||
</svg>
|
||||
<div class="flywheel-node flywheel-agent">
|
||||
<p class="flywheel-kicker">Autonomy</p>
|
||||
<h3>Agent</h3>
|
||||
<p>Choose an experiment.<br />Propose a hypothesis.</p>
|
||||
</div>
|
||||
<div class="flywheel-node flywheel-oracle">
|
||||
<p class="flywheel-kicker">Verification</p>
|
||||
<h3>Oracle</h3>
|
||||
<p>Test against a<br />concrete reference.</p>
|
||||
</div>
|
||||
<div class="flywheel-node flywheel-memory">
|
||||
<p class="flywheel-kicker">Repository</p>
|
||||
<h3>Working memory</h3>
|
||||
<p>Code, evidence,<br />checks and lessons.</p>
|
||||
</div>
|
||||
<p class="flywheel-retry">
|
||||
Mismatch<br /><strong>Revise the hypothesis</strong>
|
||||
</p>
|
||||
<p class="flywheel-center">
|
||||
A better<br /><strong>starting point</strong>
|
||||
</p>
|
||||
<p class="flywheel-reuse">Reuse</p>
|
||||
<p class="flywheel-pass">
|
||||
<strong>Pass</strong> · Keep the result and its evidence
|
||||
</p>
|
||||
</div>
|
||||
<div
|
||||
class="flywheel-projects"
|
||||
aria-label="Experience carried between projects"
|
||||
>
|
||||
<a href="#th08">TH08</a><span aria-hidden="true">→</span
|
||||
><a href="#th095">TH095</a><span aria-hidden="true">→</span
|
||||
><a href="#th04">TH04</a>
|
||||
</div>
|
||||
<figcaption>
|
||||
Each project leaves the next with a better starting point. That
|
||||
includes fixing the checks when we discover a gap.
|
||||
</figcaption>
|
||||
</figure>
|
||||
<div class="article-layout">
|
||||
<nav class="contents" aria-label="On this page">
|
||||
<p>In this article</p>
|
||||
@@ -171,6 +316,9 @@
|
||||
<li><a href="#different-approaches">Two approaches</a></li>
|
||||
<li><a href="#th095">TH095: compounding</a></li>
|
||||
<li><a href="#th04">TH04: across architectures</a></li>
|
||||
<li>
|
||||
<a href="#semantic-reconstruction">From exact to readable</a>
|
||||
</li>
|
||||
<li><a href="#industrial-shift">The industrial shift</a></li>
|
||||
<li><a href="#possibilities">What becomes possible</a></li>
|
||||
</ul>
|
||||
@@ -179,82 +327,145 @@
|
||||
<section class="article-section" id="august">
|
||||
<h2>What changed in August 2026</h2>
|
||||
<p>
|
||||
Before this reconstruction work, I had done a lot of manual
|
||||
reverse engineering. The familiar loop was to inspect
|
||||
instructions, follow references, form an explanation, test it, and
|
||||
revise it. Progress depended on how much of that loop I could
|
||||
carry out myself, and how well I could keep the growing picture in
|
||||
my head.
|
||||
Before this work, I had done a lot of manual reverse engineering.
|
||||
I would inspect a function until I had an explanation, then test
|
||||
that explanation against the program. Getting further meant
|
||||
spending more of my own time on the next function. It also meant
|
||||
keeping an increasingly large picture in my head.
|
||||
</p>
|
||||
<p>
|
||||
In August 2026, I began exploring agent-driven RE seriously. I
|
||||
wanted to see how far an agent could go when it had access to the
|
||||
program, ordinary development tools, and feedback from real
|
||||
checks. Touhou reconstruction became the place where I could
|
||||
pursue that question through a substantial project.
|
||||
Most of my RE work is on games. I reconstruct them so their
|
||||
behavior can be understood and eventually carried into a port or a
|
||||
mod. That is the experience behind this article. The method can be
|
||||
useful elsewhere, but each field needs to establish what it can
|
||||
check against a reliable reference.
|
||||
</p>
|
||||
<p>
|
||||
The early progress was startling. Tasks that I would have budgeted
|
||||
around my own time and attention could keep moving while the agent
|
||||
investigated, tried source formulations, compiled them, and
|
||||
examined the result. My work increasingly involved choosing the
|
||||
direction, improving the tools and checks, and deciding what the
|
||||
accumulated evidence meant.
|
||||
In August 2026, I started exploring agent-driven RE seriously. I
|
||||
wanted to see how far an agent could get with access to the
|
||||
program and ordinary development tools. Touhou reconstruction gave
|
||||
me a substantial project in which to find out.
|
||||
</p>
|
||||
<p>
|
||||
That changed which projects seemed reasonable to attempt. A large
|
||||
executable began to look like a collection of investigations we
|
||||
could organize and keep advancing. Recovering source, making it
|
||||
readable, and eventually using it for a port became parts of an
|
||||
achievable project.
|
||||
The early progress was startling. An investigation could keep
|
||||
moving without me choosing every individual step. A failed
|
||||
comparison might send the agent to another caller. It could write
|
||||
a small diagnostic and use the result to decide what to try next.
|
||||
Giving it that freedom made a difference: I could spend more
|
||||
attention on the project's direction and on whether its checks
|
||||
were trustworthy.
|
||||
</p>
|
||||
<p>
|
||||
The numbers first got my attention. The more interesting question
|
||||
was what made that progress possible—and whether the process could
|
||||
improve as it worked.
|
||||
The pace got my attention first. Then I began wondering what would
|
||||
survive beyond the current project. Would the next game benefit
|
||||
from everything we had just learned?
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="th08">
|
||||
<h2>TH08: building on human work</h2>
|
||||
<p>
|
||||
<a href="https://github.com/N0zoM1z0/th08">TH08</a>, Imperishable
|
||||
Night, began for me as a continuation of
|
||||
Night, began as a continuation of
|
||||
<a href="https://github.com/GensokyoClub/th08"
|
||||
>GensokyoClub's reconstruction</a
|
||||
>. Their public work already contained source, names, build
|
||||
knowledge and a contribution history. It gave the continuation a
|
||||
foundation. That foundation matters when discussing everything
|
||||
that followed.
|
||||
>. Their public source gave me a substantial foundation. It came
|
||||
with knowledge of the build and a history of contributions that I
|
||||
preserved in the continuation.
|
||||
</p>
|
||||
<p>
|
||||
I preserved the imported history through the public 10 August
|
||||
checkpoint. The independent continuation began on 13 August. By 19
|
||||
August, the project's ledger recorded source for all 1,107
|
||||
identified game functions. A playable Linux reconstruction port
|
||||
was committed on 24 August, roughly eleven days after the
|
||||
continuation began. A Web edition followed, then a native Linux
|
||||
64-bit release on 30 August.
|
||||
The imported history ends at the public 10 August checkpoint. My
|
||||
independent continuation began on 13 August. By 19 August, the
|
||||
ledger recorded source for all 1,107 identified game functions.
|
||||
</p>
|
||||
<p>
|
||||
Those dates made the change in pace tangible. What interested me
|
||||
most was that the work extended beyond producing plausible C++. It
|
||||
included comparisons, build repairs, recovery of meaningful names
|
||||
and types, and the work needed to turn reconstructed pieces into a
|
||||
program people could run.
|
||||
A playable Linux reconstruction port was committed on 24 August,
|
||||
roughly eleven days after the continuation began. A Web edition
|
||||
followed. The native Linux 64-bit release arrived on 30 August.
|
||||
</p>
|
||||
<p>
|
||||
This also exposed the importance of feedback. A function can look
|
||||
correct and compile correctly while its place in the whole program
|
||||
is wrong. Two functions can accidentally use separate copies of
|
||||
state that should be shared. A port can hide a problem that
|
||||
appears when the historical build runs. Each failure adds
|
||||
something useful if we can reproduce it and keep the check.
|
||||
What struck me was that the work had reached a program people
|
||||
could run. Getting there required following problems beyond an
|
||||
individual function. Even when two functions looked correct in
|
||||
isolation, they could accidentally use separate copies of state
|
||||
that should have been shared.
|
||||
</p>
|
||||
<p>
|
||||
Over time, more of the project lived outside any one conversation:
|
||||
source, evidence, comparison scripts, regression tests, and
|
||||
focused notes about what to do next. A fresh agent could inspect
|
||||
that state and continue.
|
||||
This made reference checks central to the work. I call them
|
||||
<strong>oracles</strong>. A code comparison can tell us whether a
|
||||
reconstructed function reproduces the original instructions. A
|
||||
runtime check can tell us whether an exercised route reaches the
|
||||
expected state. The agent proposes an explanation and tests it; a
|
||||
mismatch gives it something specific to investigate.
|
||||
</p>
|
||||
<h3>An exact match with the wrong constant</h3>
|
||||
<p>
|
||||
An oracle is software someone wrote. TH08 gave me a clear reminder
|
||||
of how much can depend on that software.
|
||||
</p>
|
||||
<p>
|
||||
In September, a bug reported by a downstream Switch port led back
|
||||
to item auto-collection. The reconstructed source checked player
|
||||
power against <code>0.0</code>. The original used
|
||||
<code>128.0</code>, the maximum-power threshold.
|
||||
</p>
|
||||
<p>
|
||||
Normal power is non-negative, so the reconstructed power check was
|
||||
effectively always satisfied. Above the collection line, the game
|
||||
could attract items without requiring maximum power. The other
|
||||
exceptions in the condition were correct. This one constant
|
||||
changed the behavior.
|
||||
</p>
|
||||
<p>
|
||||
<strong
|
||||
>Yet the function had already passed the exact
|
||||
comparison.</strong
|
||||
>
|
||||
</p>
|
||||
<p>
|
||||
A rebuild can put a constant at a different address. The
|
||||
comparison tool accounted for that by adjusting addresses in the
|
||||
compiled instructions before comparing them with the original. But
|
||||
it never checked the floating-point value stored at the referenced
|
||||
address.
|
||||
</p>
|
||||
<p>
|
||||
That left a hole in the check. It could point the reconstructed
|
||||
instruction at the original's <code>128.0</code> while the source
|
||||
still said <code>0.0</code>. The instruction bytes matched. The
|
||||
source meant something different.
|
||||
</p>
|
||||
<p>
|
||||
The
|
||||
<a href="https://github.com/N0zoM1z0/th08/pull/11"
|
||||
>2 September fix</a
|
||||
>
|
||||
corrected the source and made the comparison check the referenced
|
||||
constant's actual bytes. A
|
||||
<a href="https://github.com/N0zoM1z0/th08/pull/13">wider audit</a>
|
||||
then examined 1,548 floating-literal references. It found another
|
||||
twelve incorrect references across five accepted functions.
|
||||
</p>
|
||||
<p>
|
||||
The comparison now checks every such floating literal. Its tests
|
||||
include deliberately wrong values to make sure they are rejected.
|
||||
We also had to revisit results that had passed the weaker check.
|
||||
</p>
|
||||
<p>
|
||||
<strong>The oracle needed reconstructing too.</strong> Repairing
|
||||
it was part of repairing the game.
|
||||
</p>
|
||||
<p>
|
||||
That matters when the team is mostly one person working with
|
||||
agents. I cannot personally inspect every line they produce, so
|
||||
much of my confidence rests on their checks. A shared oracle's
|
||||
blind spot can affect many investigations before I notice it. I
|
||||
have to understand what the checker actually verifies and test it
|
||||
against cases that ought to fail.
|
||||
</p>
|
||||
<p>
|
||||
As the work continued, those checks became part of the repository.
|
||||
So did the reasons for source changes and notes that let a later
|
||||
session pick up where an earlier one stopped.
|
||||
<strong
|
||||
>The repository was becoming the project's working
|
||||
memory.</strong
|
||||
@@ -270,10 +481,14 @@
|
||||
<section class="article-section" id="different-approaches">
|
||||
<h2>Two different ideas of reconstruction</h2>
|
||||
<p>
|
||||
There is a real disagreement around this work, and GensokyoClub's
|
||||
public README expresses it directly. Its notice announces that
|
||||
further development will happen privately until completion. One
|
||||
passage reads:
|
||||
GensokyoClub's
|
||||
<a
|
||||
href="https://web.archive.org/web/20260909051514/https://github.com/GensokyoClub/th08#important-notice"
|
||||
>public README</a
|
||||
>
|
||||
makes its disagreement with this kind of work explicit. The notice
|
||||
announces that further development will happen privately until
|
||||
completion. One passage reads:
|
||||
</p>
|
||||
<blockquote>
|
||||
<p>
|
||||
@@ -285,56 +500,50 @@
|
||||
<p>
|
||||
The notice also describes the psychological toll on the
|
||||
maintainers. Their contribution policy excludes pull requests
|
||||
produced primarily with AI. These are people putting their free
|
||||
time into difficult work, and their published work helped make my
|
||||
continuation possible. That effort deserves respect. The
|
||||
disagreement I want to discuss concerns how reconstruction is done
|
||||
and how contributions are evaluated.
|
||||
produced primarily with AI. They have put their free time into
|
||||
difficult work, and that work helped make my continuation
|
||||
possible. I respect the effort behind it. The disagreement I want
|
||||
to discuss is about how reconstruction should proceed and how a
|
||||
contribution should be judged.
|
||||
</p>
|
||||
<p>
|
||||
In the manual RE workflow I knew, expertise and execution were
|
||||
closely connected. The person developing an understanding of a
|
||||
function also carried out much of the investigation and
|
||||
reconstruction. Personal knowledge, careful authorship and trust
|
||||
in contributors were central to the project.
|
||||
In the manual workflow I knew, developing an understanding of a
|
||||
function and reconstructing it usually fell to the same person. A
|
||||
project relied heavily on that person's expertise. Trust in the
|
||||
contributor mattered because so much of the reasoning happened
|
||||
while they worked.
|
||||
</p>
|
||||
<p>
|
||||
Manual projects already preserve knowledge in code, build tools
|
||||
and tests. What changed for me was having an agent use that
|
||||
accumulated knowledge to investigate, implement and test the next
|
||||
change.
|
||||
Existing projects already preserve knowledge in their source and
|
||||
build tools. What changed for me was that an agent could use that
|
||||
knowledge to pursue the next investigation on its own.
|
||||
</p>
|
||||
<p>
|
||||
My continuation takes an agent-led approach. Agents carry out the
|
||||
new engineering; the human sets direction and release decisions.
|
||||
Understanding has to be retained in source and evidence, while
|
||||
comparisons, builds and runtime experiments test the results. A
|
||||
contribution can be useful even when an agent performed most of
|
||||
the work, provided its reasoning and result can be examined.
|
||||
In my continuation, I decide the objective and the standard for
|
||||
accepting a result. The agent has broad freedom to investigate.
|
||||
Its proposed reconstruction then has to survive the relevant
|
||||
checks. I want another person to be able to examine why we chose
|
||||
an implementation, even if an agent did most of the work.
|
||||
</p>
|
||||
<p>
|
||||
This creates a difficult transition for communities built around
|
||||
the earlier model. Open source makes existing work available for
|
||||
continuation, and agents can change the pace of that continuation
|
||||
dramatically. Years of careful work may become the starting point
|
||||
for something that moves much faster. Recognition, provenance and
|
||||
the standards for accepting new work become more important in that
|
||||
situation.
|
||||
This can be a difficult transition. Years of careful work may
|
||||
become the foundation for a continuation that moves much faster.
|
||||
That raises real questions about credit. It also changes what
|
||||
maintainers need to know before accepting a contribution.
|
||||
</p>
|
||||
<p>
|
||||
The industrial analogy helps me make sense of the disagreement. A
|
||||
craft contains skill, judgment and accumulated knowledge. When
|
||||
machinery changes how execution happens, those skills find new
|
||||
roles in designing the process, detecting defects and deciding
|
||||
what should be produced. Different communities can reasonably
|
||||
choose different ways to organize that work.
|
||||
The industrial analogy helps me think about this. In a craft, much
|
||||
of the process depends on the skill of the person carrying it out.
|
||||
Machinery changes where that skill is needed. Someone still has to
|
||||
design the process and recognize when its output is wrong.
|
||||
Different communities can choose how much of that change they want
|
||||
to take on.
|
||||
</p>
|
||||
<p>
|
||||
My choice is to develop this continuation openly, preserve its
|
||||
sources and history, and make the new work reviewable. The
|
||||
technical question I want to pursue is how far agent-driven
|
||||
reconstruction can go with strong evidence and a process that
|
||||
keeps improving.
|
||||
My choice is to continue openly, with the inherited work credited
|
||||
and its history preserved. I want the new work to be reviewable.
|
||||
That gives us a way to ask how far this approach can go and to
|
||||
learn from what goes wrong along the way.
|
||||
</p>
|
||||
<p class="caption">
|
||||
Source: GensokyoClub's
|
||||
@@ -357,103 +566,100 @@
|
||||
<h2>TH095: experience starts compounding</h2>
|
||||
<p>
|
||||
<a href="https://github.com/N0zoM1z0/th095">TH095</a>, Shoot the
|
||||
Bullet, made the accumulated experience much easier to see. Its
|
||||
new repository began on 29 August with zero confirmed game
|
||||
functions in the reconstruction ledger. It already had a way to
|
||||
work: identify the target, bring up analysis and comparison tools,
|
||||
keep tasks manageable, preserve evidence, and leave checkpoints
|
||||
another session could use.
|
||||
Bullet, made the value of that experience much easier to see. Its
|
||||
repository began on 29 August with zero confirmed game functions
|
||||
in the ledger. We still had to learn the game. But we already knew
|
||||
much more about how to begin a reconstruction and how to keep it
|
||||
moving.
|
||||
</p>
|
||||
<p>
|
||||
By 7 September, all 697 identified game functions had source. By 8
|
||||
September, 696 of them were accepted as exact comparisons. The
|
||||
whole program linked on 9 September, and the Windows i386
|
||||
reconstruction was marked playable on 10 September, about twelve
|
||||
days after initialization.
|
||||
September, 696 had been accepted as exact comparisons. The whole
|
||||
program linked on 9 September. The Windows i386 reconstruction was
|
||||
marked playable on 10 September, about twelve days after
|
||||
initialization.
|
||||
</p>
|
||||
<p>
|
||||
I found this more exciting than the first project's speed. TH095
|
||||
began with the process and experience developed during TH08 and
|
||||
the other reconstruction work. We could approach a new target with
|
||||
better questions, better checks and fewer assumptions left
|
||||
implicit.
|
||||
I found this more exciting than the first project's speed. A fresh
|
||||
target could benefit from work done on another game. The
|
||||
experience was already present in the tools and in the way the
|
||||
project was organized.
|
||||
</p>
|
||||
<p>
|
||||
For example, matching individual functions had taught us to pay
|
||||
attention to the assembled program early. Does the source build
|
||||
together? Do different parts of the game use the same state? Does
|
||||
the reconstructed executable survive the transitions we need to
|
||||
exercise? Those questions became part of how the next project was
|
||||
organized.
|
||||
For example, TH08 had taught us to pay attention to the assembled
|
||||
program early. If several recovered functions depend on the same
|
||||
state, their isolated comparisons leave an important question
|
||||
open. We need to see them working together. That lesson helped
|
||||
shape how we approached TH095's whole-program build.
|
||||
</p>
|
||||
<p>
|
||||
This is what I mean by knowledge compounding. A lesson that stays
|
||||
in my head helps while I am present. A lesson retained as a test,
|
||||
a script or a precise evidence note can help every later session.
|
||||
A failure becomes valuable when the next agent can recognize it
|
||||
without repeating the whole investigation.
|
||||
A lesson that stays in my head is useful while I am there. Once it
|
||||
becomes a check that another session can run, it can keep helping
|
||||
after I have moved on. The next agent can use the result without
|
||||
repeating the investigation that led to it.
|
||||
</p>
|
||||
<p>
|
||||
The floating-literal bug belongs in that memory too. It explains
|
||||
why checking a reference also requires checking the data behind
|
||||
it. Keeping that correction with the code helps later projects
|
||||
avoid inheriting the old check's blind spot.
|
||||
</p>
|
||||
<p>
|
||||
<strong
|
||||
>The method becomes part of the starting material for the next
|
||||
game.</strong
|
||||
>
|
||||
As that material improves, we can spend more attention on what is
|
||||
new about the target.
|
||||
We can spend more of the next project's effort on what is actually
|
||||
new about its target.
|
||||
</p>
|
||||
<p>
|
||||
This compounds across people as well as projects. Someone joining
|
||||
later can rerun a check, inspect the reason for a source choice,
|
||||
or continue from a recorded unknown. The project can make use of
|
||||
experience without requiring everyone to reconstruct its entire
|
||||
history from scratch.
|
||||
The same benefit is available to people joining later. They can
|
||||
inspect a decision and rerun its check before continuing the work.
|
||||
They do not have to reconstruct the entire project's history to
|
||||
find out why the source looks the way it does.
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="th04">
|
||||
<h2>TH04: the workflow survives a different architecture</h2>
|
||||
<p>
|
||||
<a href="https://github.com/N0zoM1z0/th04">TH04</a>, Lotus Land
|
||||
Story, took the work into the PC-98 DOS era. We moved from 32-bit
|
||||
Windows games to a 16-bit environment with four cooperating
|
||||
programs and hardware behavior that had to be understood on its
|
||||
own terms. Existing
|
||||
<a href="https://github.com/nmlgc/ReC98">ReC98 work</a> provided
|
||||
Story, took this work into the PC-98 DOS era. The target was now a
|
||||
16-bit environment with four cooperating programs. Understanding
|
||||
its hardware behavior required different evidence from the Windows
|
||||
games. Existing
|
||||
<a href="https://github.com/nmlgc/ReC98">ReC98 work</a> gave us
|
||||
valuable knowledge and source material here too.
|
||||
</p>
|
||||
<p>
|
||||
The DOS reconstruction is now working, including complete Normal
|
||||
routes, endings and saves from my manual testing. The current work
|
||||
is a native 64-bit port. These are distinct phases: first
|
||||
establish a working reconstruction on the original platform, then
|
||||
carry its behavior into a new environment.
|
||||
The DOS reconstruction is now working. In my manual testing, I
|
||||
played complete Normal routes through their endings and checked
|
||||
saves. The current work is a native 64-bit port. Establishing a
|
||||
working DOS version first gives that port a reference.
|
||||
</p>
|
||||
<p>
|
||||
TH04's architecture changed the evidence we needed and the tools
|
||||
used to collect it. It did not require inventing a different way
|
||||
to organize the work. We could still isolate a question, inspect
|
||||
the original behavior, reconstruct it, compare the result and
|
||||
retain the lesson.
|
||||
The architecture changed what we needed to investigate. It also
|
||||
changed the compiler and runtime against which we checked our
|
||||
work. But the agent could still follow a question through to a
|
||||
result and let that result guide the next experiment.
|
||||
</p>
|
||||
<p>
|
||||
The difficult parts became concrete experiments. Which part of
|
||||
startup establishes a piece of state? What happens when gameplay
|
||||
hands control to an ending? Which hardware effect must the
|
||||
reconstructed code preserve? With observations and a useful check,
|
||||
an agent could work through these questions in the same iterative
|
||||
way as it did on the Windows titles.
|
||||
Consider the transition from gameplay to an ending. We need to
|
||||
know what state carries across that boundary and which program is
|
||||
responsible for it. That is something we can investigate against
|
||||
the DOS product. Once the evidence and checks are available, an
|
||||
agent can work through the question much as it did on a Windows
|
||||
title.
|
||||
</p>
|
||||
<p>
|
||||
That is why TH04 matters to the larger argument. The process
|
||||
generalized across a substantial change in platform. Architecture
|
||||
determined the subject of the investigation; it did not become a
|
||||
fundamental barrier to the workflow.
|
||||
This is why TH04 matters to the argument. A substantial platform
|
||||
change did not force us to start over with a new way of working.
|
||||
The architecture defined the problem; the workflow still gave us a
|
||||
way to solve it.
|
||||
</p>
|
||||
<p>
|
||||
The 64-bit port now gives that work another use. We can revisit
|
||||
the recovered behavior, separate it from old platform services and
|
||||
test new implementations against what we learned. The
|
||||
reconstruction supplies both source and a growing body of
|
||||
knowledge about how the game should behave.
|
||||
For the 64-bit port, we can now examine the new implementation
|
||||
against behavior already recovered on DOS. The knowledge from
|
||||
reconstruction gives the port something to build on.
|
||||
</p>
|
||||
<p class="caption">
|
||||
Project state as of 10 October 2026:
|
||||
@@ -467,39 +673,171 @@
|
||||
>. The port remains in development.
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="semantic-reconstruction">
|
||||
<h2>From exact code to readable code</h2>
|
||||
<p>
|
||||
Once a reconstruction works, I want someone else to be able to
|
||||
understand it.
|
||||
</p>
|
||||
<p>
|
||||
To me, assembly and raw offsets can feel like old friends. I
|
||||
realise this is a slightly unusual definition of “readable.” Most
|
||||
people would prefer to follow the game's logic without keeping the
|
||||
executable's memory layout in their head.
|
||||
</p>
|
||||
<p>
|
||||
That is where <strong>semantic reconstruction</strong> comes in. A
|
||||
recovered field may still be known mainly by its offset. We follow
|
||||
how the game uses it until we can explain its role. Then we can
|
||||
give it a meaningful name and a type that fits the evidence. We
|
||||
keep the reasoning with the source so the next person can see
|
||||
where that interpretation came from.
|
||||
</p>
|
||||
<p>
|
||||
This becomes especially important for a port. An absolute address
|
||||
tells me where something lived in the old executable. It gives a
|
||||
64-bit implementation little help in deciding which object should
|
||||
own that state. To move the behavior safely, we need to recover
|
||||
the relationship behind the old memory access.
|
||||
</p>
|
||||
<p>The order I now use is:</p>
|
||||
<ol>
|
||||
<li>
|
||||
<strong>Recover an exact baseline.</strong> Compile the
|
||||
reconstructed pieces with the historical compiler. Compare the
|
||||
relevant code and data with the original executable. Record
|
||||
unresolved differences so the next phase has a clear starting
|
||||
point.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Build and play it on the original platform.</strong>
|
||||
Link those pieces into the real program using the original
|
||||
architecture and compiler. Exercise important gameplay paths.
|
||||
This is where we can find problems with shared state or
|
||||
initialization that an isolated function comparison missed.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
>Reconstruct the semantics against both references.</strong
|
||||
>
|
||||
Take one coherent part of the game at a time and establish what
|
||||
its recovered source means. Improve its representation while
|
||||
preserving the exact comparisons and the playable historical
|
||||
build.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Make the modern port.</strong> Move the established
|
||||
behavior into the new environment, such as a native 64-bit
|
||||
build. The reconstructed original-platform game remains a
|
||||
reference for comparing how the port behaves.
|
||||
</li>
|
||||
</ol>
|
||||
<p>
|
||||
The playable build from the second stage becomes a
|
||||
<strong>second oracle</strong> during the third. The first oracle
|
||||
checks whether our changed source still reproduces the relevant
|
||||
original code and data. The second checks that the reconstructed
|
||||
program still builds and behaves correctly along the paths we
|
||||
exercise.
|
||||
</p>
|
||||
<p>
|
||||
They catch different mistakes. A type change can alter the
|
||||
generated instructions. An ownership change can leave two parts of
|
||||
the game using different copies of state. Keeping both checks
|
||||
available gives the agent a concrete failure to investigate before
|
||||
carrying a refactor further.
|
||||
</p>
|
||||
<p>
|
||||
A name needs evidence of its own. An exact comparison cannot tell
|
||||
us whether a field really means “invulnerability time.” We have to
|
||||
establish that from how the game writes and uses it. If the
|
||||
meaning remains uncertain, a neutral name is more helpful to the
|
||||
next reader than a confident guess.
|
||||
</p>
|
||||
<p>
|
||||
We learned this order through the projects. TH08 already had
|
||||
playable ports before some of its later historical-platform
|
||||
audits. That made certain defects harder to see. The
|
||||
<a
|
||||
href="https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/717e1b8560783ed17a39fdf159d30aa16bb307a7/docs/semantic-reconstruction.md"
|
||||
>Factory's current workflow</a
|
||||
>
|
||||
puts the original-platform build first, so semantic work can use
|
||||
it as a reference before porting begins.
|
||||
</p>
|
||||
<p>
|
||||
<strong
|
||||
>Exact reconstruction gives us a reference. Semantic
|
||||
reconstruction makes the recovered knowledge usable.</strong
|
||||
>
|
||||
A port can then build on both.
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="industrial-shift">
|
||||
<h2>What makes this an industrial shift</h2>
|
||||
<p>
|
||||
These projects changed my view of the bottleneck. As more
|
||||
investigation and implementation can be carried out by agents, the
|
||||
working environment becomes increasingly important. The agent
|
||||
needs access to the target, tools it can compose, feedback it can
|
||||
act on, and memory that survives its current session.
|
||||
These projects changed where I spent my attention. Once agents
|
||||
could carry much of an investigation forward, improving their
|
||||
working environment became one of the most useful things I could
|
||||
do. A better tool could help with every later function that needed
|
||||
it.
|
||||
</p>
|
||||
<p>
|
||||
REA fits into that picture by making reverse-engineering tools
|
||||
available through interfaces an agent can use. It can inspect
|
||||
code, follow relationships and return evidence for the next
|
||||
question. Reconstruction projects add their own compilers,
|
||||
comparisons and runtime checks. Together, these give an agent a
|
||||
way to move from an explanation to an experiment with an
|
||||
observable result.
|
||||
Autonomy matters here. The useful next step often becomes clear
|
||||
only after a failed experiment. An agent needs enough freedom to
|
||||
follow that result somewhere unexpected. If it has to wait for me
|
||||
to prescribe each step, much of the work remains tied to my
|
||||
attention.
|
||||
</p>
|
||||
<p>
|
||||
Fast execution makes good feedback more valuable. A bad assumption
|
||||
can produce a large amount of plausible source. A useful check can
|
||||
reject that assumption early and point to the next investigation.
|
||||
Improving a check improves many later decisions, which is why
|
||||
tooling and verification deserve as much attention as the
|
||||
recovered code.
|
||||
I expect the agent to make wrong hypotheses. What matters is
|
||||
whether we can test them and learn from the result. A failed check
|
||||
should help it understand the mistake well enough to try again. I
|
||||
still have to decide whether the accumulated evidence supports a
|
||||
project milestone.
|
||||
</p>
|
||||
<p>
|
||||
The work also needs continuity. A conversation can end while the
|
||||
campaign continues. Its result should leave the repository in a
|
||||
state another agent can understand: what changed, what was tested,
|
||||
what remains uncertain, and where to continue. This lets a large
|
||||
reconstruction proceed through many small experiments without
|
||||
relying on one uninterrupted session.
|
||||
REA gives the agent access to analysis tools. A question about a
|
||||
function's caller can lead straight into inspecting that caller.
|
||||
The reconstruction project supplies the compiler and its own
|
||||
reference checks. The agent can use those to test the source it
|
||||
proposes and see where its explanation holds up.
|
||||
</p>
|
||||
<p>
|
||||
The TH08 bug shows why those checks deserve engineering attention
|
||||
of their own. When the same comparison is used across hundreds of
|
||||
functions, a gap in it can spread much further than a mistake in
|
||||
one implementation. Testing the checker improves the feedback
|
||||
available to all that later work.
|
||||
</p>
|
||||
<p>
|
||||
The industrial analogy has a useful historical example here.
|
||||
Boulton and Watt introduced a
|
||||
<a
|
||||
href="https://collection.sciencemuseumgroup.org.uk/objects/co51439/watts-steam-engine-indicator"
|
||||
>steam-engine indicator in 1796</a
|
||||
>
|
||||
to help adjust engine valves. A recording version traced pressure
|
||||
through the piston stroke. It made the engine's internal behavior
|
||||
available for inspection. Our comparison tools serve a similar
|
||||
purpose: they let us examine what the machinery is doing while we
|
||||
improve it.
|
||||
</p>
|
||||
<p>
|
||||
<strong
|
||||
>We are in the early stages of this industrial shift.</strong
|
||||
>
|
||||
Much of the infrastructure is still immature. Agents can move
|
||||
faster than our checks were designed to support, so the process
|
||||
has to develop alongside them. When we find a defect in a shared
|
||||
tool, we have to repair it and revisit the results it affected.
|
||||
The next project can then inherit a stronger tool.
|
||||
</p>
|
||||
<p>
|
||||
There is also a practical limit to any one conversation. It will
|
||||
end before a large reconstruction is finished. The repository has
|
||||
to make it possible for another session to continue without losing
|
||||
the reason for the last decision.
|
||||
</p>
|
||||
<p>
|
||||
The
|
||||
@@ -507,42 +845,31 @@
|
||||
href="https://github.com/N0zoM1z0/touhou-reconstruction-factory"
|
||||
>Touhou Reconstruction Factory</a
|
||||
>
|
||||
grew out of these experiences. It gives the projects shared ways
|
||||
to retain evidence, run checks and carry useful lessons between
|
||||
games. The point is to make the next investigation easier to
|
||||
execute, easier to judge and easier to resume.
|
||||
grew out of that need. It gives projects a shared way to carry
|
||||
their checks and lessons forward. Work on one game can improve the
|
||||
starting conditions for another.
|
||||
</p>
|
||||
<p>
|
||||
This is the industrial shift I see: expertise increasingly lives
|
||||
in a working process that can be reused and improved. Human
|
||||
judgment shapes the process. Agents supply much of the execution.
|
||||
Verified results and accumulated lessons make the process more
|
||||
capable over time.
|
||||
That is what makes the industrial analogy meaningful to me.
|
||||
Experience starts becoming part of tools that others can use.
|
||||
Improving those tools changes how much the next person—or the next
|
||||
agent—can get done.
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="possibilities">
|
||||
<h2>The projects we can now consider</h2>
|
||||
<p>
|
||||
The pace matters because it changes the cost of beginning. When I
|
||||
imagined doing every investigation manually, I also imagined
|
||||
committing my own attention to each one. A sufficiently large
|
||||
project could be interesting and still remain something I would
|
||||
never realistically attempt.
|
||||
The pace matters because it changes the decision to begin. A game
|
||||
could be fascinating to reverse engineer and still demand more of
|
||||
my own attention than I could realistically give it. Many projects
|
||||
would stay ideas.
|
||||
</p>
|
||||
<p>
|
||||
An agent with access, feedback and project memory changes that
|
||||
calculation. Recovering a game, making its source readable,
|
||||
building a port, or exploring a mod can become a sustained
|
||||
engineering effort with a practical way to keep going. The work
|
||||
still involves difficult questions, but there is now a process for
|
||||
giving those questions sustained attention.
|
||||
</p>
|
||||
<p>
|
||||
TH08 showed me what an agent-led continuation could achieve on top
|
||||
of human work. TH095 showed how the experience could carry
|
||||
forward. TH04 showed that the method could cross into a very
|
||||
different platform. Together, they made the industrial analogy
|
||||
feel concrete.
|
||||
Now I can see a way to keep such a project moving through repeated
|
||||
investigations. Getting a working reconstruction makes a port more
|
||||
practical. Recovering readable semantics makes it easier for
|
||||
someone else to explore a mod. The effort put into understanding
|
||||
the game can keep paying off after the first version runs.
|
||||
</p>
|
||||
<p>
|
||||
I now look at an unfamiliar program and ask:
|
||||
@@ -552,10 +879,9 @@
|
||||
>
|
||||
</p>
|
||||
<p>
|
||||
That question opens up projects I would previously have left
|
||||
alone. Each one can add code, checks and experience that make the
|
||||
next more approachable. That is the future of reverse engineering
|
||||
I want to keep exploring.
|
||||
That question makes me consider projects I would previously have
|
||||
left alone. Each one can improve the way we approach the next. I
|
||||
want to keep exploring how far that can take us.
|
||||
</p>
|
||||
</section>
|
||||
<section class="article-section" id="sources">
|
||||
@@ -624,6 +950,41 @@
|
||||
records the working DOS reconstruction, the maintainer's
|
||||
complete Normal-route testing and the current 64-bit phase.
|
||||
</li>
|
||||
<li>
|
||||
<strong>TH08 oracle correction:</strong>
|
||||
<a href="https://github.com/N0zoM1z0/th08/issues/10"
|
||||
>the reported auto-collection bug</a
|
||||
>,
|
||||
<a
|
||||
href="https://github.com/N0zoM1z0/th08/commit/7148a76ba224742677895c7160807a68897957d5"
|
||||
>the source and comparison fix</a
|
||||
>, and
|
||||
<a
|
||||
href="https://github.com/N0zoM1z0/th08/commit/a393f400786e4cc6cbb1000af0dad3276da12c78"
|
||||
>the full floating-literal audit</a
|
||||
>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Semantic reconstruction:</strong>
|
||||
<a
|
||||
href="https://github.com/N0zoM1z0/th08/blob/d2a00f4acb4a13916e07a00bd13f714961714628/docs/SEMANTIC_PLAYBOOK.md"
|
||||
>TH08's readability playbook</a
|
||||
>
|
||||
and
|
||||
<a
|
||||
href="https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/717e1b8560783ed17a39fdf159d30aa16bb307a7/docs/semantic-reconstruction.md"
|
||||
>the Factory's phase order and two validation paths</a
|
||||
>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Industrial history:</strong>
|
||||
<a
|
||||
href="https://collection.sciencemuseumgroup.org.uk/objects/co51439/watts-steam-engine-indicator"
|
||||
>the Science Museum Group's steam-engine indicator record</a
|
||||
>
|
||||
describes the 1796 introduction and the pressure-recording
|
||||
mechanism.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Method:</strong> the Factory's
|
||||
<a
|
||||
|
||||
Reference in New Issue
Block a user