Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 N0zoM1z0
|
28b1e5cadc | ||
|
N0zoM1z0
|
456aa4f6de | ||
|
N0zoM1z0
|
73c8b36615 | ||
|
N0zoM1z0
|
0751974d49 | ||
|
Rudy Celekli
|
3ae5ac3359 | ||
|
Rudy Celekli
|
a47517c67c |
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "37"
|
||||
version: "36"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -25,18 +25,57 @@ a skill installed from repository main may describe capabilities absent from
|
||||
an older npm release. Keep complete inline Evidence when that server does not
|
||||
advertise retained references.
|
||||
|
||||
If tools are absent, a call is blocked by the client, or a required provider is
|
||||
unavailable, read [connection and recovery](references/connection-and-recovery.md).
|
||||
Diagnose the observed blocker, continue useful supported work, and resume the
|
||||
original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
When tools are absent or registration is stale:
|
||||
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Binwalk or Unblob.
|
||||
1. Diagnose without changing files. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`, `hermes`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair
|
||||
MCP registration. For an unsupported client, use manual stdio registration
|
||||
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
npx -y rea-agents@latest inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
No MCP registration or native engine is required. The CLI returns the complete
|
||||
Evidence record directly, and a real application can produce hundreds of
|
||||
megabytes, so save it to a file rather than reading it into context. Start from
|
||||
the summary view, then request a module page
|
||||
(`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}`) or one
|
||||
module the same way, or pass the saved Evidence inline to
|
||||
`trace-application-feature`. Read graph, limitations, and unknowns with the
|
||||
same care as an MCP result. This fallback does not establish that MCP is
|
||||
configured. Native CLI tasks still require their selected engine.
|
||||
|
||||
## Route the target first
|
||||
|
||||
@@ -101,11 +140,9 @@ after opening; IDA does not supply `binary_overview`. Consult session availabili
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
facets remain unknown; live IDA results are not replayed from snapshots.
|
||||
|
||||
If the user says only "my app" without a name, path, or target already selected
|
||||
for this task, ask which app before searching for artifacts. Paths in unrelated
|
||||
saved reports and nearby test fixtures do not select the user's target. Resolve
|
||||
an actual human-readable app name to one clear installed artifact when possible;
|
||||
ask when matches are ambiguous. Never choose an example app on the user's behalf.
|
||||
If the app is missing, ask which app to inspect. Resolve a human-readable app
|
||||
name to one clear installed artifact when possible; ask only when matches are
|
||||
ambiguous. Never choose an example app on the user's behalf.
|
||||
|
||||
In a target-free session, use `open_binary` to bind any archive/package or
|
||||
native target whose analysis tool operates on the active target. Do not call an
|
||||
@@ -120,11 +157,7 @@ For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
Recheck session state when a lifecycle change makes that status useful.
|
||||
Make a focused follow-up only when the
|
||||
Do not repeat an identical tool call. Make a focused follow-up only when the
|
||||
returned result leaves a specific question unanswered.
|
||||
|
||||
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
||||
@@ -135,14 +168,6 @@ session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
canonical JSON or repeat the producer. For JavaScript, read the
|
||||
[module-view and trace recipes](references/javascript-applications.md) before
|
||||
drilling into a large graph; a broad bidirectional trace can expand unrelated
|
||||
containment relationships even after summary analysis.
|
||||
|
||||
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
||||
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
||||
that static analysis observed execution. Runtime requests execute the declared
|
||||
@@ -150,9 +175,7 @@ target and lifecycle; do not broaden the target or action beyond those fields.
|
||||
|
||||
Within the user's requested investigation, call available analysis tools
|
||||
directly. REA does not require permission grants or per-call approval flags.
|
||||
The agent client can independently require approval. Follow the declared request
|
||||
scope and the host's actual access requirements; use the recovery guide for an
|
||||
observed client-policy rejection.
|
||||
Follow the declared request scope and the host's actual access requirements.
|
||||
|
||||
## Plan broader investigations
|
||||
|
||||
|
||||
@@ -1,127 +0,0 @@
|
||||
# Connection and task recovery
|
||||
|
||||
Read this guide only for an observed connection, command, approval or provider
|
||||
blocker. Keep the user's target and question in view. Explain what prevented the
|
||||
operation and the next useful step; after repair, continue the investigation.
|
||||
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Resolve only the prerequisite needed for the task; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
CODEX_HOME is under `/tmp`; use an authorized profile outside the OS temporary
|
||||
directory. REA setup cannot repair the client's executor. Preserve the sandbox
|
||||
instead of silently disabling it to make an analysis command work.
|
||||
|
||||
Static JavaScript and ordinary ZIP inventory need neither Hopper nor Ghidra.
|
||||
HAR inspection is offline and does not require a live browser. Native
|
||||
decompilation, Android analysis and firmware extraction have their own
|
||||
bring-your-own prerequisites. Use supported inventory/static evidence when it
|
||||
answers part of the question, and name the unavailable evidence separately.
|
||||
Do not substitute another target. Label observations from ordinary shell tools
|
||||
as such; only actual REA output supplies REA Evidence.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
|
||||
```bash
|
||||
rea doctor --client codex --json
|
||||
rea setup --client codex --dry-run --json
|
||||
```
|
||||
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
then apply the same scope with `rea setup --client codex --yes`. Add
|
||||
`--install-hopper` only when separately needed and explicitly approved. Do not
|
||||
install or upgrade unrelated prerequisites. Restart/reconnect, verify tools
|
||||
actually appear, and resume the original task. If they remain absent, inspect
|
||||
the launch failure rather than looping through setup.
|
||||
|
||||
## Codex tool approval is separate from REA readiness
|
||||
|
||||
REA tools have no permission-grant handshake. Codex can still require approval
|
||||
because an operation creates owned resources, launches a helper or has cleanup
|
||||
effects. An advertised tool or successful doctor result does not override that
|
||||
client policy. `workspace-write` alone does not resolve an MCP approval
|
||||
rejection, and `approval_policy = "never"` does not mean every MCP tool is
|
||||
preapproved.
|
||||
|
||||
When interactive approval is available, use the client's approval flow for the
|
||||
requested operation. In an unattended run, continue through an authorized CLI
|
||||
workflow if the execution environment permits it, or report the blocked
|
||||
operation and the approval needed to proceed. Do not retry the same rejected
|
||||
call, silently broaden approvals, or rewrite MCP effects as read-only.
|
||||
|
||||
For an operator who explicitly wants to preapprove named operations in a
|
||||
disposable Codex profile, current Codex supports per-tool configuration. For
|
||||
example, opening and closing an analysis session can be scoped independently:
|
||||
|
||||
```toml
|
||||
[mcp_servers.rea.tools.open_binary]
|
||||
approval_mode = "approve"
|
||||
|
||||
[mcp_servers.rea.tools.close_binary]
|
||||
approval_mode = "approve"
|
||||
```
|
||||
|
||||
Use the actual server ID and only the operations whose effects were approved;
|
||||
this example does not approve inspection, extraction or runtime execution.
|
||||
Show the proposed config edit and preserve unrelated entries and backups before
|
||||
writing. Managed policy can constrain client settings. Consult the
|
||||
[Codex configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
```bash
|
||||
command -v rea
|
||||
command -v node
|
||||
node --version
|
||||
rea --version
|
||||
```
|
||||
|
||||
Use the public launcher at its known installation path with a supported existing
|
||||
Node on PATH when the child shell loses a prefix installation. Avoid internal
|
||||
`dist/cli.js` entrypoints or global toolchain changes. MCP may have its own valid
|
||||
launcher even when a shell cannot resolve `rea`.
|
||||
|
||||
For a user-supplied JavaScript tree or ASAR, select output files in the authorized
|
||||
workspace and save complete Evidence before reading a view:
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
These commands do not register MCP or install an analysis engine. CLI JavaScript
|
||||
JSON is complete and can be very large; redirect it rather than loading it into
|
||||
context. Use an existing JSON processor if `jq` is absent, without installing
|
||||
one just for this recipe. Inspect a module page or item using the same inline
|
||||
source and the [JavaScript recipes](javascript-applications.md). Preserve graph
|
||||
coverage, limitations and unknowns. Native CLI analysis still requires its
|
||||
selected engine, and all CLI operations retain the host's execution permissions.
|
||||
@@ -1,90 +1,20 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
Evidence record with a distinct ID and a `normalized_result.parent_evidence_id`
|
||||
for the complete retained application analysis. Source-map contents are part of
|
||||
that static analysis. Read findings, coverage, limitations and unknowns, then
|
||||
follow a specific unanswered question. Older servers without `detail` return
|
||||
the complete application Evidence inline when it fits.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
The following IDs are placeholders to replace with returned values. When the
|
||||
module's node ID is not already known, page identities before selecting an item.
|
||||
Pages return only node IDs, paths and roles; choose a useful page size and follow
|
||||
`coverage.next_offset` when another page matters:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": { "kind": "page", "collection": "modules", "offset": 0, "limit": 16 }
|
||||
}
|
||||
```
|
||||
|
||||
Select the relevant page entry by its exact `node_id` and recorded role:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "item",
|
||||
"collection": "modules",
|
||||
"selector": { "node_id": "jag_node_ACTUAL_NODE_ID" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Module pages include asset, bundled and source module roles, so one path can
|
||||
identify several entries. Use `selector.path` only when uniqueness is established;
|
||||
the returned node ID avoids needing a complete inventory to prove uniqueness. On ambiguity,
|
||||
select the matching node ID from the page or the error's candidates. Follow
|
||||
`next_offset` when another page matters. An item exposes that module node's
|
||||
recorded observations, identity, hashes, exports and source ranges; it does not
|
||||
include every child syntax observation, related edge or original source file.
|
||||
An absent item fact is not evidence of absence from the whole application.
|
||||
|
||||
## Trace only the remaining question
|
||||
|
||||
Use `trace_application_feature` on the parent application Evidence for one
|
||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||
Select `incoming` for relationships leading to the seed, `outgoing` for those
|
||||
leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
matches exactly.
|
||||
|
||||
## Recover an oversized display
|
||||
extracted tree. The complete result, graph, and Evidence context are returned
|
||||
inline when they fit the MCP frame. Source-map contents are part of the static analysis. Start from its
|
||||
findings, coverage, unknowns, and graph context, then make a focused follow-up
|
||||
only when a specific question remains unanswered.
|
||||
|
||||
An oversized response reports `resource_constraint` and
|
||||
`details.resource: "transport"`. Reuse
|
||||
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
||||
summary, one module, or a stable page of module identities.
|
||||
summary, one module, or a stable page of module identities. Use
|
||||
`trace_application_feature` once a module, route, or string seed is known.
|
||||
`export_evidence_bundle` writes the complete canonical session to a
|
||||
caller-selected file. These workflows keep the original Evidence and coverage;
|
||||
a broad follow-up can also exceed framing.
|
||||
|
||||
A client can truncate a successful result inside its text envelope without a
|
||||
REA `resource_constraint`. Treat a truncation notice or incomplete JSON as a
|
||||
display limitation; do not claim the full record was examined. Reuse the known
|
||||
parent ID for smaller views, or export complete Evidence to a selected file
|
||||
while the session remains open. If the parent ID was lost from the preview,
|
||||
use the prior analysis summary or explicit ledger entries to recover it; do
|
||||
not invent an ID or rerun analysis just to reproduce the same display.
|
||||
|
||||
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
||||
registrations, utility processes, and native binding requests are static syntax
|
||||
observations. Only a unique exact literal IPC channel match supports an inferred
|
||||
@@ -92,6 +22,13 @@ pairing. Dynamic or ambiguous channels remain unresolved. A requested `.node`
|
||||
member is not a verified native export. Never claim runtime reachability,
|
||||
registration, defaults, or policy enforcement from static analysis.
|
||||
|
||||
Use `trace_application_feature` on existing application Evidence for one
|
||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||
Choose a direction and reuse the returned application Evidence ID when the
|
||||
connected server advertises retained references, or supply complete Evidence
|
||||
inline. Include
|
||||
Hopper or Ghidra Evidence only when its artifact digest matches exactly.
|
||||
|
||||
For version comparison, analyze each version once, then call
|
||||
`compare_application_versions`. Accept only its digest, source-map, structural
|
||||
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
||||
|
||||
@@ -152,7 +152,7 @@ context. PSP/stack/device state remains unmodeled. See [DOS guide](https://githu
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"_comment": "Test files are intentionally NOT excluded: the gate used to report zero clones because it skipped **/*.test.ts, hiding all test-tree duplication. Current actual is ~2.47% lines across src+ and tests/, down from 3.4% when tests were included. The 2.6% limit leaves modest headroom for ordinary changes; lower it as duplication is removed.",
|
||||
"threshold": 2.6,
|
||||
"_comment": "Test files are intentionally NOT excluded: the gate used to report zero clones because it skipped **/*.test.ts, hiding all test-tree duplication. Current actual is ~2.4% lines across src+ and tests/, down from 3.4% when tests were included. Lower this as duplication is removed.",
|
||||
"threshold": 2.5,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
|
||||
@@ -120,8 +120,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
);
|
||||
private static final String[] CAPABILITIES = {
|
||||
"annotate_native_function",
|
||||
"set_address_name",
|
||||
"set_addresses_names",
|
||||
"inspect_native_load_image",
|
||||
"read_bytes",
|
||||
"address_to_file_offset",
|
||||
@@ -376,12 +374,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
case "resolve_native_call_targets" -> resolveNativeCallTargets(request.params);
|
||||
case "xrefs" -> xrefs(request.params);
|
||||
case "analyze_function" -> analyzeFunction(request.params);
|
||||
case "set_address_name", "set_addresses_names" -> {
|
||||
if (!descriptor.transport.equals("unix-socket"))
|
||||
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
||||
yield request.method.equals("set_address_name")
|
||||
? setAddressName(request.params) : setAddressesNames(request.params);
|
||||
}
|
||||
case "annotate_native_function" -> {
|
||||
if (!descriptor.transport.equals("unix-socket"))
|
||||
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
||||
@@ -425,8 +417,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.addProperty("analysis_timed_out", timedOut);
|
||||
result.add("capabilities", GSON.toJsonTree(
|
||||
java.util.Arrays.stream(CAPABILITIES)
|
||||
.filter(value -> !readOnly || !(value.equals("annotate_native_function")
|
||||
|| value.equals("set_address_name") || value.equals("set_addresses_names")))
|
||||
.filter(value -> !readOnly || !value.equals("annotate_native_function"))
|
||||
.toArray(String[]::new)
|
||||
));
|
||||
result.add("target", target);
|
||||
@@ -1036,11 +1027,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
for (String field : List.of("name", "comment", "inline_comment")) {
|
||||
if (params.has(field)) validateAnnotationText(requireText(params, field), field);
|
||||
}
|
||||
String leafName = params.has("name")
|
||||
? annotationLeafName(function, requireString(params, "name")) : null;
|
||||
int transaction = currentProgram.startTransaction("REA function annotations");
|
||||
boolean commit = false;
|
||||
try {
|
||||
String leafName = params.has("name")
|
||||
? symbolLeafName(function.getSymbol(), requireString(params, "name")) : null;
|
||||
// Validate edits through Ghidra's own setters, including name rules.
|
||||
// A later invalid name must roll back earlier comment writes.
|
||||
for (String field : List.of("comment", "inline_comment")) {
|
||||
@@ -1089,33 +1080,22 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
}
|
||||
|
||||
private String symbolLeafName(Symbol symbol, String requested) throws ghidra.util.exception.InvalidInputException {
|
||||
private String annotationLeafName(Function function, String requested) {
|
||||
// Fully qualified readback must be usable as an idempotent rename input.
|
||||
if (requested.equals(symbol.getName(true))) return symbol.getName();
|
||||
String namespace = symbol.getParentNamespace().isGlobal()
|
||||
? "" : symbol.getParentNamespace().getName(true);
|
||||
if (requested.equals(procedureName(function))) return function.getName();
|
||||
String namespace = function.getParentNamespace().isGlobal()
|
||||
? "" : function.getParentNamespace().getName(true);
|
||||
String prefix = namespace + "::";
|
||||
if (!namespace.isEmpty() && requested.startsWith(prefix)) {
|
||||
String leaf = requested.substring(prefix.length());
|
||||
if (!leaf.isEmpty()) return leaf;
|
||||
throw new ghidra.util.exception.InvalidInputException(
|
||||
"Symbol name has an empty leaf within namespace " + namespace + " at " +
|
||||
canonicalAddress(symbol.getAddress()) + ": " + requested);
|
||||
throw new RequestFailure("invalid_function_name",
|
||||
"Function name has an empty leaf within namespace " + namespace + " at " +
|
||||
canonicalAddress(function.getEntryPoint()) + ": " + requested);
|
||||
}
|
||||
return requested;
|
||||
}
|
||||
|
||||
private void requireAvailableAddressName(Address address,
|
||||
ghidra.program.model.symbol.Namespace namespace, String name)
|
||||
throws ghidra.util.exception.DuplicateNameException {
|
||||
for (Symbol owner : currentProgram.getSymbolTable().getSymbols(name, namespace)) {
|
||||
if (!owner.getAddress().equals(Address.NO_ADDRESS) && !owner.getAddress().equals(address))
|
||||
throw new ghidra.util.exception.DuplicateNameException(
|
||||
"Symbol name " + name + " is already owned at " +
|
||||
canonicalAddress(owner.getAddress()) + " in namespace " + namespace.getName(true));
|
||||
}
|
||||
}
|
||||
|
||||
private static String requireText(JsonObject params, String field) {
|
||||
JsonElement value = params.get(field);
|
||||
if (value == null || !value.isJsonPrimitive() || !value.getAsJsonPrimitive().isString())
|
||||
@@ -1436,67 +1416,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
||||
}
|
||||
|
||||
// Analyst naming: a function entry renames the function, other addresses get a primary label.
|
||||
private boolean applyAddressName(Address address, String name) throws Exception {
|
||||
if (!currentProgram.getMemory().contains(address))
|
||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
||||
validateAnnotationText(name, "name");
|
||||
int transaction = currentProgram.startTransaction("REA address name");
|
||||
boolean commit = false;
|
||||
try {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
var table = currentProgram.getSymbolTable();
|
||||
Symbol primary = function == null ? table.getPrimarySymbol(address) : function.getSymbol();
|
||||
boolean existing = primary != null && !primary.isDynamic();
|
||||
String expected = existing ? symbolLeafName(primary, name) : name;
|
||||
requireAvailableAddressName(address,
|
||||
existing ? primary.getParentNamespace() : currentProgram.getGlobalNamespace(), expected);
|
||||
if (function != null) {
|
||||
function.setName(expected, SourceType.USER_DEFINED);
|
||||
} else if (existing) {
|
||||
primary.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = table.createLabel(address, expected, SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
}
|
||||
invalidateAnalysisCaches();
|
||||
Symbol readback = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
||||
commit = readback != null && expected.equals(readback.getName());
|
||||
return commit;
|
||||
}
|
||||
catch (ghidra.util.exception.InvalidInputException | ghidra.util.exception.DuplicateNameException exception) {
|
||||
return false;
|
||||
}
|
||||
finally {
|
||||
currentProgram.endTransaction(transaction, commit);
|
||||
}
|
||||
}
|
||||
|
||||
private JsonElement setAddressName(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "address", "name"));
|
||||
requireDocument(params);
|
||||
return GSON.toJsonTree(applyAddressName(requireAddress(params, "address"), requireString(params, "name")));
|
||||
}
|
||||
|
||||
private JsonElement setAddressesNames(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "names"));
|
||||
requireDocument(params);
|
||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
||||
JsonObject result = new JsonObject();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
||||
boolean ok;
|
||||
try {
|
||||
ok = applyAddressName(parseReaAddress(entry.getKey()), entry.getValue().getAsString());
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
ok = false;
|
||||
}
|
||||
result.addProperty(entry.getKey(), ok);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private JsonArray listDocuments(JsonObject params) {
|
||||
requireKeys(params, Set.of());
|
||||
JsonArray result = new JsonArray();
|
||||
|
||||
@@ -136,12 +136,6 @@ existing analysis families directly, for example
|
||||
- Pull and capture outputs are additive: the destination must not already
|
||||
exist. Re-running a completed pull refuses the existing directory instead
|
||||
of overwriting evidence.
|
||||
- Cancellation during a package pull preserves caller-selected output. The
|
||||
error includes partial observations identifying completed artifacts and the
|
||||
interrupted local path; remaining APKs were not attempted.
|
||||
- Failed package acquisition rollback retains its directory identity and retries
|
||||
cleanup before later work and during shutdown. Cleanup refuses a replacement
|
||||
directory. Terminal shutdown stops admission and drains active operations.
|
||||
- The provider holds no long-lived processes; every invocation is one
|
||||
bounded adb run. The adb server belongs to the caller.
|
||||
|
||||
|
||||
+3
-10
@@ -166,16 +166,9 @@ rea import-reference-source /absolute/path/to/source
|
||||
```
|
||||
|
||||
The import records hashes and metadata for the supplied files, separately
|
||||
from observations of the current app. It honors the source tree's `.gitignore`
|
||||
and applies default exclusions for common generated and dependency paths such
|
||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
||||
matched pattern and whether it came from project, default, caller, or sensitive
|
||||
path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||
patterns when you want to mark selected paths as sensitive; those patterns are
|
||||
reported separately as configured-secret exclusions.
|
||||
|
||||
Re-import saved graphs whose pattern-based exclusions lack `pattern` before
|
||||
using them in source-to-bundle comparisons.
|
||||
from observations of the current app. File names do not automatically exclude
|
||||
files. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||
patterns when you want to exclude selected paths.
|
||||
|
||||
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
||||
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
||||
|
||||
@@ -25,12 +25,6 @@ extraction have independent prerequisites; one does not require the other.
|
||||
Both require util-linux `prlimit`, normally `/usr/bin/prlimit`; override its
|
||||
absolute path with `REA_FIRMWARE_PRLIMIT_COMMAND` if needed.
|
||||
|
||||
REA retains at most 8 MiB of combined stdout and stderr per command. Exceeding
|
||||
that diagnostic budget stops analysis with an explicit truncated-output error;
|
||||
child process limits do not bound parent-side pipe retention. Completed commands
|
||||
drain their output streams before recording diagnostics. Failed worker cleanup
|
||||
retains the worker and workspace for retry before another request or at shutdown.
|
||||
|
||||
Unblob needs the external extractor for the selected format on its inherited
|
||||
`PATH`: for example, `debugfs` for ext2/ext3/ext4. Its default Landlock sandbox
|
||||
and skip rules stay enabled. Sandbox startup failures remain actionable errors;
|
||||
|
||||
@@ -29,13 +29,6 @@ section names or the record being inside the initial 4 KiB header probe.
|
||||
Program and section declarations must agree; duplicate, conflicting, truncated
|
||||
or out-of-file records are rejected. Stripped section tables and section-only
|
||||
records are supported. Extended table numbering is explicitly unsupported.
|
||||
For carved ELFs whose section table extends beyond the captured file, REA
|
||||
examines every complete captured section header and records the unavailable
|
||||
headers as a limitation. A valid program-header ABI record can still establish
|
||||
the supported declaration. Available section declarations must agree with it;
|
||||
their duplicate, conflicting or unreadable records remain errors. A missing
|
||||
program-header record does not make ABI declarations in uncaptured sections
|
||||
absent or supply a default ABI.
|
||||
The field layout follows
|
||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||
|
||||
@@ -99,11 +92,6 @@ observations and does not relabel the upstream result as verified.
|
||||
|
||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||
identity, snapshot round-trip and final process cleanup remain covered.
|
||||
For both byte orders, the lane also carves away the section table while retaining
|
||||
all program-header payloads. It checks the actual Ghidra language, entry-address
|
||||
decompilation, distinct artifact digest and missing-section limitations through
|
||||
CLI Evidence and an MCP-produced snapshot. Caller-selected Ghidra heap limits
|
||||
are passed to these child workflows.
|
||||
Pseudocode presence is a liveness check, not proof of semantic equivalence.
|
||||
Instruction decoding and reader agreement likewise do not establish runtime,
|
||||
delay-slot execution or complete floating-point semantics. Assertion regressions
|
||||
|
||||
+4
-45
@@ -68,15 +68,7 @@ selects the npm release, and persistent MCP registrations are pinned to the
|
||||
version that performed setup. Installing newer instructions does not update a
|
||||
running server or its registration.
|
||||
|
||||
The npm release checked on **2026-10-10** was **6.3.0**, published from
|
||||
[`74cf9e14`](https://github.com/morluto/rea/commit/74cf9e1401fcd89482fe39c27cc898f5858e9b72),
|
||||
with bundled skill version 34. It does not include automatic Pi or Hermes
|
||||
registration: `setup --client pi` and `setup --client hermes` reject those IDs
|
||||
before writing configuration. Their setup workflows below require a source build
|
||||
or a later release containing those integrations; `@latest` alone cannot select
|
||||
unpublished changes.
|
||||
|
||||
The earlier release checked on **2026-10-07** was **5.0.0** (133 MCP tools), published
|
||||
The release checked on **2026-10-07** was **5.0.0** (133 MCP tools), published
|
||||
from the fixed checkpoint
|
||||
[`b33236ec`](https://github.com/morluto/rea/releases/tag/rea-agents-5.0.0).
|
||||
The public CLI, MCP catalog and target-free session, and isolated update from
|
||||
@@ -150,29 +142,7 @@ listed after the table because its connector is not one of these files:
|
||||
| Pi | `pi` |
|
||||
| Hermes | `hermes` |
|
||||
|
||||
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
|
||||
chat with REA's full input-schema profile before sending a model request
|
||||
(`compaction_static_context_blocked`; [#1554](https://github.com/morluto/rea/issues/1554)).
|
||||
For the native offline OpenAI-compatible BYOK adapter, full-profile chat and
|
||||
analysis also passed with a distinct `COPILOT_PROVIDER_MODEL_ID` such as
|
||||
`rea-gpt-4.1`, `COPILOT_PROVIDER_WIRE_MODEL=gpt-4.1`, and that distinct ID selected
|
||||
through `copilot --model rea-gpt-4.1`. The local fixture requested
|
||||
`COPILOT_PROVIDER_MAX_PROMPT_TOKENS=1000000`; configure the limit supported by
|
||||
your backend. This keeps every full schema and the actual wire model intact.
|
||||
It does not establish the effective client budget or fix the built-in-ID path.
|
||||
The existing compact profile passed native chat and analysis with the complete
|
||||
tool inventory on Linux using a loopback model fixture. After setup, add
|
||||
`"REA_MCP_INPUT_SCHEMA_PROFILE": "compact"` to the `env` object of the
|
||||
`mcpServers.rea` entry in Copilot's `mcp-config.json`, preserving its other
|
||||
settings, then restart the client. Compact advertisements can reduce nested
|
||||
schema constraints; REA still validates complete canonical inputs. Full-profile
|
||||
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
|
||||
Windows remain unverified; see [native client verification](testing.md).
|
||||
|
||||
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
|
||||
automatic setup requires a build containing the integration; npm 6.3.0 does not
|
||||
include it. See [released package and main](#released-package-and-main).
|
||||
In a supported build,
|
||||
setup writes a `type: "stdio"` entry under `mcpServers.rea` in the user-level
|
||||
`~/.pi/agent/mcp.json` and installs the bundled skill under `~/.agents/skills`.
|
||||
Use `rea setup --client pi --dry-run --json` to inspect the plan, then
|
||||
@@ -231,10 +201,7 @@ registration. Doctor treats an `enabled: false` entry as active when
|
||||
`enabledServers` lists `rea`, as OMP does, unless `disabledServers` also lists
|
||||
it. Run setup under each profile that should load REA.
|
||||
|
||||
For Hermes, automatic setup also requires a build containing the integration;
|
||||
npm 6.3.0 does not include it. See
|
||||
[released package and main](#released-package-and-main).
|
||||
In a supported build, setup writes a `mcp_servers.rea` entry to `config.yaml`, preserving
|
||||
For Hermes, setup writes a `mcp_servers.rea` entry to `config.yaml`, preserving
|
||||
comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
||||
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
||||
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
||||
@@ -367,12 +334,6 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no
|
||||
changes. Agent configuration writes preserve unrelated entries and comments,
|
||||
create backups, use atomic replacement, and verify their result. Setup and
|
||||
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
|
||||
Setup also retains valid environment overrides in REA's existing server entry,
|
||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
|
||||
Detected provider settings replace only matching environment keys.
|
||||
Setup and doctor report a malformed REA environment map or invalid runtime
|
||||
setting before setup writes files. Repair the named setting in the client
|
||||
configuration; setup does not silently discard the rest of the map.
|
||||
|
||||
After setup, REA reports which agents, analysis tools, and workflow files passed
|
||||
its final checks. Restart any agent named in the completion message, then begin
|
||||
@@ -637,10 +598,8 @@ dossiers, instructions, recovered data types, measured load mappings, loaded
|
||||
memory bytes, and observed file offsets. Independent load-image attestation
|
||||
supports DOS MZ and explicitly selected COM; PE returns its measurements with that limitation.
|
||||
On Linux and macOS, `annotate_native_function` also edits a function name and/or
|
||||
entry comments atomically and returns refreshed analysis, and `set_address_name`
|
||||
and `set_addresses_names` name a function entry or any other mapped address
|
||||
(creating or renaming its primary label) with per-address success results. These
|
||||
session metadata edits leave executable bytes unchanged and are discarded on close. GUI controls
|
||||
entry comments atomically and returns refreshed analysis. These session metadata
|
||||
edits leave executable bytes unchanged and are discarded on close. GUI controls
|
||||
require Hopper; Windows P0 remains read-only.
|
||||
|
||||
Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
|
||||
|
||||
@@ -38,20 +38,6 @@ Inspect separately obtained components explicitly and preserve their identities.
|
||||
A valid implementation marker establishes a candidate native boundary, not
|
||||
recovered native semantics or runtime behavior.
|
||||
|
||||
### Resource admission
|
||||
|
||||
The static provider admits regular input files up to 128 MiB before retaining
|
||||
their bytes. Reads remain bounded if a file grows after admission. Metadata,
|
||||
CIL decoding, and projected inspection records have a separate conservative
|
||||
64 MiB representation budget; shared heap strings are cached, but repeated
|
||||
output occurrences still consume that budget. These are inspection resource
|
||||
policies, not PE format limits or a hard bound on total V8 heap use.
|
||||
|
||||
Input overflow returns a resource constraint. Decode or projection overflow
|
||||
reports `resource-limit` with unavailable or partial coverage; omitted facts do
|
||||
not become empty observations. Ambiguous PE section mappings are rejected
|
||||
rather than interpreted according to section order.
|
||||
|
||||
## Analysis objective
|
||||
|
||||
REA's managed-code track is intended to answer five different questions without
|
||||
|
||||
@@ -397,15 +397,6 @@ unpaired Unicode surrogate; a rejection identifies the field and UTF-16 index
|
||||
and leaves every annotation unchanged. CRLF, supplementary Unicode characters,
|
||||
and combining characters are preserved. Windows P0 remains read-only. Ghidra has no
|
||||
GUI authority, and REA never falls back automatically to Hopper.
|
||||
`set_address_name` and `set_addresses_names` give Ghidra sessions the same
|
||||
compact naming results as Hopper: a local function entry renames that function,
|
||||
and any other mapped address, such as a global, receives a new or renamed primary
|
||||
label. Names owned by another address in the same namespace are rejected;
|
||||
the same leaf name remains valid in a different namespace. A batch applies each
|
||||
address separately and reports one success boolean per address, preserving the
|
||||
other entries when one fails. Ghidra batches do not move or swap names. Use them
|
||||
to name many globals and helpers cheaply; use
|
||||
`annotate_native_function` when you also want comments and a refreshed dossier.
|
||||
Ghidra name edits preserve the existing namespace. Supply either a leaf name
|
||||
such as `renamed` or a fully qualified name in that namespace, such as
|
||||
`alpha::renamed`. The returned qualified name can be reused as an idempotent
|
||||
|
||||
@@ -114,12 +114,6 @@ from launch, must be ordered, and must fall within `timeout_ms`. Inputs marked
|
||||
Environment overrides are recorded in the scenario commitment; inherited
|
||||
values are not copied into Evidence.
|
||||
|
||||
Terminal dimensions must satisfy `columns × (rows + scrollback) ≤ 1,000,000`.
|
||||
Every scheduled resize uses the same selected scrollback and must fit this
|
||||
working-buffer budget. REA rejects oversized scenarios before launching the
|
||||
command; it never clamps the requested dimensions. This limit is independent
|
||||
of `limits.output_bytes`, which bounds retained terminal observations.
|
||||
|
||||
The capture contains raw PTY output chunks and rendered terminal states,
|
||||
interaction dispatch outcomes, exit reason, sampled process-tree observations,
|
||||
and settlement status. When filesystem paths are selected, REA records
|
||||
@@ -193,10 +187,6 @@ Every capture requires `truncation_details`, with separate accounting for:
|
||||
input, so raw omissions also limit rendered coverage.
|
||||
- `filesystem_before` and `filesystem_after`: file-count/depth limits,
|
||||
enumeration failures, whole-file hash budget and bytes successfully hashed.
|
||||
Directory names are read incrementally within the remaining entry capacity.
|
||||
When a directory exceeds that capacity, the retained subset follows filesystem
|
||||
enumeration order and is then sorted; it is explicitly incomplete and cannot
|
||||
establish that an omitted path is absent.
|
||||
Each retained regular file without a digest has an aliased path, size,
|
||||
remaining budget and reason: `file_bytes_budget`,
|
||||
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is
|
||||
|
||||
+10
-113
@@ -99,10 +99,9 @@ result that its client offloads, and Pi exercises default codemode execution.
|
||||
Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package.
|
||||
Receipts retain client versions, result digests, host coverage and owned-process
|
||||
cleanup. These POSIX lanes use a deterministic loopback model, so they do not
|
||||
prove live model-provider or native Windows compatibility. Qwen, Pi and Hermes
|
||||
bind both `HOME` and `USERPROFILE` to the disposable account. Qwen and Pi discover
|
||||
the shared personal skill through their default search paths without an explicit
|
||||
skill setting; Hermes discovers it through the selected `HERMES_HOME`.
|
||||
prove live model-provider or native Windows compatibility. Qwen and Pi use
|
||||
caller-configured skill directories to isolate the fixture account from the OS
|
||||
home; this does not prove their default home-directory discovery.
|
||||
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
||||
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
||||
|
||||
@@ -160,39 +159,11 @@ Use `-- chat` for ordinary
|
||||
chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
|
||||
The native offline BYOK adapter uses a loopback OpenAI completions/SSE fixture
|
||||
and `gpt-5.4` model metadata; model inference and token usage are synthetic.
|
||||
Set `REA_VERIFY_COPILOT_MODEL` to examine another client model ID and
|
||||
`REA_VERIFY_COPILOT_WIRE_MODEL` for its backend model name (defaults to the client
|
||||
ID). Every actual HTTP request's model is checked and recorded separately.
|
||||
Both `HOME` and `USERPROFILE` point to the disposable account; native `skill add`
|
||||
still selects the installed skill directory explicitly.
|
||||
|
||||
The same native client passed full-profile chat and analysis with the actual
|
||||
wire model `gpt-4.1` under a distinct BYOK model ID:
|
||||
|
||||
```bash
|
||||
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- chat
|
||||
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- call
|
||||
```
|
||||
|
||||
These runs retain full canonical input/output schemas, all tools, installed-skill
|
||||
loading and complete Evidence readback. They use the fixture's requested
|
||||
one-million-token BYOK prompt setting; effective capacity and live API acceptance
|
||||
remain unknown. With the built-in `gpt-4.1` model ID, the verified client still
|
||||
blocks the full input-schema profile before HTTP with
|
||||
Set `REA_VERIFY_COPILOT_MODEL` to examine another model configuration. In the
|
||||
verified client, `gpt-4.1` blocks the complete catalog before HTTP with
|
||||
`compaction_static_context_blocked`, even when requesting a larger BYOK prompt
|
||||
capacity. The effective capacity is unknown. The default verifier preserves
|
||||
the complete catalog and full schemas. To verify the existing compact profile:
|
||||
|
||||
```bash
|
||||
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- chat
|
||||
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- call
|
||||
```
|
||||
|
||||
Both native workflows passed with the complete tool inventory. The verifier selects the
|
||||
profile only in its disposable Copilot registration and records it in the
|
||||
receipt. Compact advertisements can reduce nested validation structure; the
|
||||
server still validates complete canonical inputs. This is a tested alternate
|
||||
workflow, not a fix for the full-profile client/model limit tracked in #1554.
|
||||
capacity. The effective capacity is unknown; this lane does not establish a
|
||||
fix for that client/model limit. It preserves the complete catalog and schemas.
|
||||
Live model APIs, native Windows and default OS-home skill discovery remain
|
||||
unverified.
|
||||
|
||||
@@ -216,30 +187,6 @@ The loopback OpenAI completions/SSE custom model declares a one-million-token
|
||||
context and synthetic usage. Live xAI APIs, native Windows and default OS-home
|
||||
skill discovery remain unverified.
|
||||
|
||||
`verify:commandcode-client` requires installed Command Code (verified with npm
|
||||
1.79.2); select it with `REA_VERIFY_COMMANDCODE_COMMAND`. Auth, MCP tokens and
|
||||
sessions use the actual OS home, so provision a disposable POSIX account rather
|
||||
than overriding `HOME`. Set `REA_VERIFY_COMMANDCODE_ACCOUNT_HOME` to that account's
|
||||
actual home and create `.rea-client-verification` there with exactly
|
||||
`Disposable REA client verification account` followed by a newline. The lane
|
||||
owns that account's `.commandcode` configuration and shared REA skill. Build the
|
||||
runtime as the checkout owner, then run `node scripts/verify-commandcode-client.mjs`
|
||||
as the disposable account; the npm shortcut also needs a writable checkout.
|
||||
The verifier does not create accounts or install clients. It guards setup
|
||||
targets, preserves an unrelated registration and backups, checks idempotence,
|
||||
and verifies native default shared-skill discovery and complete activation.
|
||||
Default deferred-schema delivery stays enabled: the native prompt advertises
|
||||
every REA tool, exact-name `search_tools` returns every input schema, and a native
|
||||
shell query recovers the saved catalog's count and digest when the client spills
|
||||
it. Actual analysis of a Unicode-path fixture delivers full named-schema Evidence
|
||||
to the next model request. Use `-- chat` for ordinary chat or
|
||||
`REA_VERIFY_RUNTIME_ROOT` for a production-only installed package. The loopback
|
||||
OpenAI completions/SSE model is a keyless BYOK endpoint declaring a million-token
|
||||
context with synthetic usage. A synthetic account-key value satisfies the
|
||||
client's print-mode gate; native local-only mode refuses hosted API calls.
|
||||
Updates, telemetry and cron are disabled. Hosted authentication, live models
|
||||
and Windows remain unverified.
|
||||
|
||||
`verify:omp-client` requires installed OMP (verified with the official Linux x64
|
||||
18.8.7 binary); select it with `REA_VERIFY_OMP_COMMAND`. This optional POSIX lane
|
||||
isolates the default agent, global config and XDG roots, guards setup targets,
|
||||
@@ -261,9 +208,8 @@ unverified; skill discovery uses an explicit isolated custom directory.
|
||||
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
|
||||
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
|
||||
installs the skill through `rea setup --skill`, writes the native Cordis MCP
|
||||
patch into an isolated `DSH_HOME`, and binds `HOME` and `USERPROFILE` to the
|
||||
disposable account. The native default `~/.agents/skills` search path discovers
|
||||
the installed skill without a `DSH_AGENTS_HOME` override. A separate Git root prevents inherited project skills from
|
||||
patch into an isolated `DSH_HOME`, and uses `DSH_AGENTS_HOME` for native personal
|
||||
skill discovery. A separate Git root prevents inherited project skills from
|
||||
masking a missing installation. The default `call` mode checks skill loading,
|
||||
complete catalog discovery, the actual forwarded input schemas and regexes,
|
||||
and full JavaScript Evidence for a Unicode path. Use `-- chat` for ordinary
|
||||
@@ -1083,39 +1029,6 @@ real local Codex CLI with:
|
||||
npm run verify:agent
|
||||
```
|
||||
|
||||
The lane runs scenarios sequentially in standalone Codex (`--no-daemon`) with a
|
||||
disposable HOME and CODEX_HOME under `~/.cache/rea-agent-evaluations/`. Set
|
||||
`REA_AGENT_EVAL_ROOT` to choose another parent directory outside the OS temporary
|
||||
directory: current Codex refuses helper aliases under `/tmp`, preventing shell
|
||||
commands and skill reads even if MCP works. Each owned run directory is removed
|
||||
after verification unless fixture retention is requested. The lane
|
||||
installs the packaged skill and MCP registration through public `rea setup`,
|
||||
rather than copying a skill into a project that also inherits the caller's
|
||||
skills and configuration. Only authentication is copied, when present, and it
|
||||
is removed even with `REA_AGENT_EVAL_KEEP_FIXTURES=true`. No user configuration,
|
||||
rules, plugins or history is copied; shell snapshots and additional agents are
|
||||
disabled. Existing API-key environment authentication remains available.
|
||||
The native missing-provider scenario preapproves only `open_binary` and
|
||||
`close_binary` in the disposable client so it reaches REA's actual provider
|
||||
boundary instead of stopping at Codex approval. This does not validate a real
|
||||
native engine or authorize arbitrary runtime execution.
|
||||
|
||||
Select a smaller run with `REA_AGENT_EVAL_SCENARIOS`, a comma-separated list of
|
||||
scenario IDs. `asar-zh` exercises the packaged desktop rubric from a Chinese
|
||||
request. `javascript-module-view` exercises summary analysis, module paging and
|
||||
item inspection in a tree with unrelated modules; its routing and text checks
|
||||
remain heuristic, so review the retained transcript for selector choice and
|
||||
coverage claims. Set `REA_CODEX_CLI` to an installed Codex executable and
|
||||
`REA_AGENT_EVAL_MODEL` to an account-supported model. Use an external resource
|
||||
limit for the whole process tree when testing on a constrained host; a Node
|
||||
heap limit alone does not bound client and child-process memory.
|
||||
|
||||
`missing-target` leaves desktop fixtures visible without selecting one in the
|
||||
request. It requires zero REA calls and a structured request for an app name or
|
||||
artifact path. Its `expectedFirstTool` is `null`, and
|
||||
`targetClarificationPassed` grades that limited routing outcome independently
|
||||
of analysis prose heuristics. Analyzing a nearby example cannot pass this case.
|
||||
|
||||
The packaged desktop and parser-comparison scenarios now assess a closed set of
|
||||
known fixture claims. Each final answer must be a strict JSON manifest containing
|
||||
exactly the requested claim IDs, values, Evidence IDs, and Evidence authority and
|
||||
@@ -1140,21 +1053,9 @@ navigation-context, and address-context scenarios have no factual rubric and
|
||||
remain `not_assessed`; the managed workflow requires both artifact and member
|
||||
inspection to answer its type and entry-point question.
|
||||
|
||||
The closed factual scenarios request complete producer Evidence on the initial
|
||||
analysis so their source selectors can authenticate all configured facts.
|
||||
Summary/page/item workflows are exercised separately by `javascript-module-view`;
|
||||
its heuristic gate does not establish full-producer factual correctness.
|
||||
|
||||
All scenarios retain routing, workflow, validation, repetition, process-exit,
|
||||
and token-use gates. Configured factual scenarios additionally require a factual
|
||||
pass. Their answer-text heuristics are diagnostic and do not affect the gate.
|
||||
Validation counts include complete text-only REA `invalid_request` envelopes
|
||||
and failed SDK calls whose arguments violate the named current REA contract.
|
||||
Client approval rejections and provider unavailability remain distinct failures;
|
||||
truncated JSON previews are not parsed into invented error codes.
|
||||
Repeated `binary_session` observations separated by a successful `open_binary`
|
||||
or `close_binary` are treated as lifecycle verification. An unchanged retry,
|
||||
including one after a failed lifecycle call, still counts as repetition.
|
||||
Scenarios without a rubric retain the legacy text-heuristic gate.
|
||||
`answerTermCoverageMet` checks case-insensitive substrings,
|
||||
`epistemicCuePresent` checks keywords, and `finalCitesEvidence` checks only an ID's
|
||||
@@ -1162,11 +1063,7 @@ presence. These metrics can still accept fabricated prose and must not be read
|
||||
as factual assessments. Set `REA_AGENT_EVAL_TRANSCRIPT_DIR` to retain complete
|
||||
tool results and final answers for review.
|
||||
|
||||
Report schema version 4 adds negative routing scenarios with nullable
|
||||
`expectedFirstTool` and a `targetClarificationPassed` outcome. Consumers must
|
||||
allow an expected absence of REA calls; this does not make an analysis scenario
|
||||
pass without its required tools. Version 3 changed the top-level
|
||||
`factualCorrectness` from a constant
|
||||
Report schema version 3 changes the top-level `factualCorrectness` from a constant
|
||||
string to an assessment summary with status, scope, and assessed/passed/failed/
|
||||
not-assessed scenario counts. Scenario records include the factual assessment and
|
||||
configured claim IDs. The evaluation scope is
|
||||
|
||||
+4
-7
@@ -85,22 +85,19 @@ if [[ "$version" =~ \+ ]]; then
|
||||
fail "npm cannot install an exact version with build metadata. Pass a version without build metadata, then retry."
|
||||
fi
|
||||
|
||||
install_args=(install --global)
|
||||
prefix_args=()
|
||||
if [[ "$platform" == "Linux" ]]; then
|
||||
install_args+=(--prefix "$HOME/.local")
|
||||
prefix_args=(--prefix "$HOME/.local")
|
||||
install_bin="$HOME/.local/bin/rea"
|
||||
else
|
||||
npm_prefix="$(npm prefix --global)" || fail "the npm global prefix could not be read. Repair the npm configuration, then retry."
|
||||
install_bin="$npm_prefix/bin/rea"
|
||||
fi
|
||||
install_args+=("$PACKAGE@$version")
|
||||
|
||||
printf 'REA install plan\n'
|
||||
printf ' Version: %s\n' "$version"
|
||||
printf ' Runtime: Node.js %s\n' "$node_version"
|
||||
printf ' Command:'
|
||||
printf ' %q' npm "${install_args[@]}"
|
||||
printf '\n'
|
||||
printf ' Command: npm install --global %s@%s\n' "$PACKAGE" "$version"
|
||||
printf ' Binary: %s\n' "$install_bin"
|
||||
printf ' Setup: %s\n' "$([[ "$start_setup" == true ]] && printf 'start when a terminal is available' || printf 'skipped')"
|
||||
|
||||
@@ -110,7 +107,7 @@ if [[ "$dry_run" == true ]]; then
|
||||
fi
|
||||
|
||||
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
|
||||
npm "${install_args[@]}" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
|
||||
npm install --global ${prefix_args[@]+"${prefix_args[@]}"} "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
|
||||
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
|
||||
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
|
||||
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
|
||||
|
||||
@@ -188,7 +188,6 @@
|
||||
"verify:gemini-client": "npm run build:cached && node scripts/verify-gemini-client.mjs",
|
||||
"verify:opencode-client": "npm run build:cached && node scripts/verify-opencode-client.mjs",
|
||||
"verify:grok-client": "npm run build:cached && node scripts/verify-grok-client.mjs",
|
||||
"verify:commandcode-client": "npm run build:cached && node scripts/verify-commandcode-client.mjs",
|
||||
"verify:omp-client": "npm run build:cached && node scripts/verify-omp-client.mjs",
|
||||
"verify:copilot-client": "npm run build:cached && node scripts/verify-copilot-client.mjs",
|
||||
"verify:claude-client": "npm run build:cached && node scripts/verify-claude-client.mjs",
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
import { analyzeJavaScriptApplication } from "../../dist/application/javascript/JavaScriptApplicationService.js";
|
||||
import { ArtifactResourceScope } from "../../dist/artifacts/ArtifactResourceScope.js";
|
||||
|
||||
/** Analyze one JavaScript application and close its caller-owned resources. */
|
||||
export const analyzeJavaScriptApplicationWithOwnedScope = async (input) => {
|
||||
const scope = new ArtifactResourceScope();
|
||||
let outcome;
|
||||
try {
|
||||
const result = await analyzeJavaScriptApplication(input, scope);
|
||||
outcome = result.ok
|
||||
? { kind: "completed", value: result.value }
|
||||
: { kind: "failed", cause: result.error };
|
||||
} catch (cause) {
|
||||
outcome = { kind: "failed", cause };
|
||||
}
|
||||
try {
|
||||
await scope.close();
|
||||
} catch (cleanupCause) {
|
||||
if (outcome.kind === "failed")
|
||||
throw new AggregateError(
|
||||
[outcome.cause, cleanupCause],
|
||||
"JavaScript analysis and artifact cleanup both failed",
|
||||
);
|
||||
throw cleanupCause;
|
||||
}
|
||||
if (outcome.kind === "failed") throw outcome.cause;
|
||||
return outcome.value;
|
||||
};
|
||||
@@ -21,6 +21,7 @@ export async function verifyGhidraEntryAliases({
|
||||
new URL("../../tests/conformance/ghidra/entry-aliases.c", import.meta.url),
|
||||
);
|
||||
const path = join(workspace, "entry-aliases");
|
||||
const prefix = process.platform === "darwin" ? "_" : "";
|
||||
try {
|
||||
await exec(compiler, ["-O0", "-g", "-fno-inline", source, "-o", path], {
|
||||
env,
|
||||
@@ -28,8 +29,7 @@ export async function verifyGhidraEntryAliases({
|
||||
});
|
||||
const bytes = await readFile(path);
|
||||
await call("close_binary");
|
||||
const imported = await call("open_binary", { path, provider_id: "ghidra" });
|
||||
const prefix = imported.format === "mach-o" ? "_" : "";
|
||||
await call("open_binary", { path, provider_id: "ghidra" });
|
||||
const names = await call("list_names");
|
||||
const alias = names.find(
|
||||
({ value }) => value === `${prefix}rea_entry_alias`,
|
||||
@@ -46,10 +46,8 @@ export async function verifyGhidraEntryAliases({
|
||||
);
|
||||
assert.ok(canonical, "Fixture alias is not at a function entry");
|
||||
const xrefsSelectors = [];
|
||||
// CLI xrefs keeps bare hexadecimal selectors as addresses. A Mach-O
|
||||
// primary label such as "dead" is tested below as a procedure name instead.
|
||||
for (const value of [
|
||||
`${prefix}rea_alias_target`,
|
||||
canonical.value,
|
||||
alias.value,
|
||||
`${prefix}rea_xrefs_data`,
|
||||
`${prefix}rea_xrefs_unreferenced`,
|
||||
@@ -91,18 +89,6 @@ export async function verifyGhidraEntryAliases({
|
||||
);
|
||||
assert.ok(interior);
|
||||
assert.notEqual(interior.address, alias.address);
|
||||
assert.equal(
|
||||
interior.symbol.type,
|
||||
"label",
|
||||
"Fixture interior control must remain an imported label",
|
||||
);
|
||||
assert.equal(
|
||||
(await call("list_procedures")).some(
|
||||
({ address }) => address === interior.address,
|
||||
),
|
||||
false,
|
||||
"Fixture interior control must not be a Ghidra procedure entry",
|
||||
);
|
||||
const failure = await reject("procedure_address", {
|
||||
procedure: interior.value,
|
||||
});
|
||||
@@ -167,11 +153,7 @@ export async function verifyGhidraEntryAliases({
|
||||
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const evidence = JSON.parse(stdout);
|
||||
assert.deepEqual(
|
||||
evidence.normalized_result,
|
||||
expected,
|
||||
`CLI xrefs mismatch for ${JSON.stringify(selector)}`,
|
||||
);
|
||||
assert.deepEqual(evidence.normalized_result, expected);
|
||||
assert.equal(evidence.parameters.address, selector.at(-1));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -54,15 +54,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
);
|
||||
return { namespace, address: procedure.address };
|
||||
});
|
||||
const names = await call("list_names");
|
||||
const globals = ["alpha", "outer::inner"].map((namespace) => {
|
||||
const symbol = names.find(
|
||||
(item) => item.value === `${namespace}::value` && item.symbol.primary,
|
||||
);
|
||||
assert.ok(symbol, `Missing demangled fixture global ${namespace}::value`);
|
||||
return { namespace, address: symbol.address };
|
||||
});
|
||||
await verifyAddressNameCollisions(call, procedures, names);
|
||||
const ambiguous = await reject("procedure_address", { procedure: "same" });
|
||||
assert.equal(ambiguous.code, "invalid_request");
|
||||
for (const { address } of entries)
|
||||
@@ -134,20 +125,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
});
|
||||
assert.deepEqual(unchanged.annotations, qualified.annotations);
|
||||
}
|
||||
await verifyAddressNames(call, [
|
||||
{ ...entries[0], kind: "function" },
|
||||
{ ...entries[2], kind: "function" },
|
||||
...globals.map((symbol) => ({ ...symbol, kind: "label" })),
|
||||
]);
|
||||
for (const { namespace, address } of entries) {
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address,
|
||||
name: `${namespace}::qualified`,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
}
|
||||
const { stdout } = await exec(
|
||||
process.execPath,
|
||||
[
|
||||
@@ -182,224 +159,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyAddressNameCollisions(call, procedures, names) {
|
||||
const find = (items, name) => {
|
||||
const symbol = items.find(
|
||||
(item) => item.value === name || item.value === `_${name}`,
|
||||
);
|
||||
assert.ok(symbol, `Missing naming collision fixture ${name}`);
|
||||
return symbol;
|
||||
};
|
||||
const first = find(procedures, "rea_name_first");
|
||||
const second = find(procedures, "rea_name_second");
|
||||
const firstLabel = find(names, "rea_name_data_first");
|
||||
const secondLabel = find(names, "rea_name_data_second");
|
||||
for (const [owner, other, name] of [
|
||||
[first, second, "owned_function"],
|
||||
[firstLabel, secondLabel, "owned_label"],
|
||||
]) {
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: owner.address, name }),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: other.address, name }),
|
||||
false,
|
||||
"A different address must not acquire the existing name",
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: other.address }),
|
||||
other.value,
|
||||
);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: owner.address, name }),
|
||||
true,
|
||||
"The owning address may repeat the same name",
|
||||
);
|
||||
}
|
||||
for (const [address, name, original] of [
|
||||
[first.address, "owned_label", "owned_function"],
|
||||
[firstLabel.address, "owned_function", "owned_label"],
|
||||
]) {
|
||||
assert.equal(await call("set_address_name", { address, name }), false);
|
||||
assert.equal(await call("address_name", { address }), original);
|
||||
}
|
||||
const alphaFunction = find(procedures, "alpha::same");
|
||||
const alphaLabel = find(names, "alpha::value");
|
||||
for (const name of ["value", "alpha::value"]) {
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: alphaFunction.address, name }),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: alphaFunction.address }),
|
||||
alphaFunction.value,
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address: alphaLabel.address,
|
||||
name: "same",
|
||||
}),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: alphaLabel.address }),
|
||||
alphaLabel.value,
|
||||
);
|
||||
assert.deepEqual(
|
||||
await call("set_addresses_names", {
|
||||
names: {
|
||||
[secondLabel.address]: "batch_available",
|
||||
[second.address]: "owned_function",
|
||||
[firstLabel.address]: "owned_label",
|
||||
},
|
||||
}),
|
||||
{
|
||||
[secondLabel.address]: true,
|
||||
[second.address]: false,
|
||||
[firstLabel.address]: true,
|
||||
},
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: second.address }),
|
||||
second.value,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: first.address }),
|
||||
"owned_function",
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: secondLabel.address }),
|
||||
"batch_available",
|
||||
);
|
||||
const before = await call("annotate_native_function", {
|
||||
procedure: second.address,
|
||||
comment: "Keep this comment",
|
||||
inline_comment: "Keep this inline comment",
|
||||
});
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address: second.address,
|
||||
name: "owned_function",
|
||||
}),
|
||||
false,
|
||||
);
|
||||
const unchanged = await call("annotate_native_function", {
|
||||
procedure: second.address,
|
||||
name: second.value,
|
||||
});
|
||||
assert.deepEqual(unchanged.annotations, before.annotations);
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address: first.address,
|
||||
name: "released_function",
|
||||
}),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address: second.address,
|
||||
name: "owned_function",
|
||||
}),
|
||||
true,
|
||||
"A name becomes available after its old owner releases it",
|
||||
);
|
||||
}
|
||||
|
||||
async function verifyAddressNames(call, symbols) {
|
||||
for (const { namespace, address, kind } of symbols) {
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address,
|
||||
name: `address_renamed_${kind}`,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
const qualified = `${namespace}::address_qualified_${kind}`;
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: qualified }),
|
||||
true,
|
||||
);
|
||||
assert.equal(await call("address_name", { address }), qualified);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: "other::literal" }),
|
||||
true,
|
||||
);
|
||||
const literal = await call("address_name", { address });
|
||||
assert.equal(literal, `${namespace}::other::literal`);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: literal }),
|
||||
true,
|
||||
);
|
||||
assert.equal(await call("address_name", { address }), literal);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: qualified }),
|
||||
true,
|
||||
);
|
||||
for (let repeat = 0; repeat < 2; repeat++) {
|
||||
const readback = await call("address_name", { address });
|
||||
assert.deepEqual(
|
||||
await call("set_addresses_names", { names: { [address]: readback } }),
|
||||
{ [address]: true },
|
||||
);
|
||||
assert.equal(await call("address_name", { address }), qualified);
|
||||
}
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: `${namespace}::` }),
|
||||
false,
|
||||
);
|
||||
const session = await call("binary_session");
|
||||
for (const operation of [
|
||||
"set_address_name",
|
||||
"set_addresses_names",
|
||||
"address_name",
|
||||
"list_names",
|
||||
])
|
||||
assert.equal(
|
||||
session.capabilities.find((item) => item.operation === operation)
|
||||
?.available,
|
||||
true,
|
||||
`An invalid name must not mark ${operation} unavailable`,
|
||||
);
|
||||
assert.equal(await call("address_name", { address }), qualified);
|
||||
}
|
||||
const [valid, invalid] = symbols;
|
||||
assert.deepEqual(
|
||||
await call("set_addresses_names", {
|
||||
names: {
|
||||
[valid.address]: `${valid.namespace}::batch_renamed`,
|
||||
[invalid.address]: `${invalid.namespace}::`,
|
||||
},
|
||||
}),
|
||||
{ [valid.address]: true, [invalid.address]: false },
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: valid.address }),
|
||||
`${valid.namespace}::batch_renamed`,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: invalid.address }),
|
||||
`${invalid.namespace}::address_qualified_${invalid.kind}`,
|
||||
);
|
||||
const names = await call("list_names");
|
||||
for (const { namespace, address, kind } of symbols) {
|
||||
const expected =
|
||||
address === valid.address
|
||||
? `${namespace}::batch_renamed`
|
||||
: `${namespace}::address_qualified_${kind}`;
|
||||
assert.ok(
|
||||
names.some(
|
||||
(item) =>
|
||||
item.address === address &&
|
||||
item.value === expected &&
|
||||
item.symbol.primary,
|
||||
),
|
||||
`Missing refreshed primary symbol ${expected}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function requireNamespaceDemangler(env) {
|
||||
const inspection = inspectGhidraInstallation({
|
||||
environment: process.env,
|
||||
|
||||
@@ -1,18 +1,9 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import {
|
||||
chmod,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
rm,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { homedir } from "node:os";
|
||||
import { delimiter, dirname, join, resolve } from "node:path";
|
||||
import { cp, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { createInterface } from "node:readline";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { z } from "zod";
|
||||
|
||||
import { evaluateCodexEvents } from "../dist/evaluation/CodexAgentEval.js";
|
||||
import {
|
||||
@@ -24,95 +15,32 @@ import {
|
||||
agentEvaluationPassed,
|
||||
summarizeFactualCorrectness,
|
||||
} from "../dist/evaluation/AgentEvaluationReport.js";
|
||||
import { PRODUCT_IDENTITY } from "../dist/identity.js";
|
||||
import { MCP_STARTUP_POLICY } from "../dist/mcpStartupPolicy.js";
|
||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const verifierRun = createVerifierRun();
|
||||
const evaluationRoot = await mkdtemp(join(tmpdir(), "rea-agent-eval-"));
|
||||
const fixtureRoot = join(evaluationRoot, "targets");
|
||||
const skillDestination = join(
|
||||
evaluationRoot,
|
||||
".agents/skills/reverse-engineer-anything",
|
||||
);
|
||||
const timeoutMs = Number(process.env.REA_AGENT_EVAL_TIMEOUT_MS ?? 480_000);
|
||||
const codex = process.env.REA_CODEX_CLI ?? "codex";
|
||||
const optionalModel = process.env.REA_AGENT_EVAL_MODEL;
|
||||
|
||||
if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 10_000)
|
||||
throw new Error("REA_AGENT_EVAL_TIMEOUT_MS must be an integer >= 10000");
|
||||
// Codex refuses helper aliases when CODEX_HOME is under the OS temp directory.
|
||||
// Keep the disposable account in a private cache, then remove the owned run.
|
||||
const evaluationParent = resolve(
|
||||
process.env.REA_AGENT_EVAL_ROOT ??
|
||||
join(homedir(), ".cache", "rea-agent-evaluations"),
|
||||
);
|
||||
await mkdir(evaluationParent, { recursive: true, mode: 0o700 });
|
||||
const evaluationRoot = await mkdtemp(join(evaluationParent, "run-"));
|
||||
const fixtureRoot = join(evaluationRoot, "targets");
|
||||
const account = join(evaluationRoot, "account");
|
||||
const codexHome = join(evaluationRoot, "codex");
|
||||
const authPath = join(codexHome, "auth.json");
|
||||
const skillDestination = join(
|
||||
account,
|
||||
".agents/skills/reverse-engineer-anything",
|
||||
);
|
||||
const evaluationEnvironment = {
|
||||
...Object.fromEntries(
|
||||
Object.entries(process.env).filter(
|
||||
([name]) => !/^(?:REA_|CODEX_|GHIDRA_|HOPPER_|IDA_)/u.test(name),
|
||||
),
|
||||
),
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
CODEX_HOME: codexHome,
|
||||
XDG_CONFIG_HOME: join(account, ".config"),
|
||||
XDG_DATA_HOME: join(account, ".local/share"),
|
||||
XDG_CACHE_HOME: join(account, ".cache"),
|
||||
REA_PROCESS_RUN_ID: verifierRun.run_id,
|
||||
PATH: `${dirname(process.execPath)}${delimiter}${process.env.PATH ?? ""}`,
|
||||
};
|
||||
const codexVersion = await readCommandVersion(codex);
|
||||
|
||||
try {
|
||||
const codexVersion = await readCommandVersion(codex);
|
||||
await Promise.all([
|
||||
mkdir(fixtureRoot, { recursive: true }),
|
||||
mkdir(account),
|
||||
mkdir(codexHome, { mode: 0o700 }),
|
||||
]);
|
||||
// Reuse authentication only. User config, plugins, rules, history and skills
|
||||
// must not make a fresh installed-skill trial pass accidentally.
|
||||
try {
|
||||
await copyFile(
|
||||
join(process.env.CODEX_HOME ?? join(homedir(), ".codex"), "auth.json"),
|
||||
authPath,
|
||||
);
|
||||
await chmod(authPath, 0o600);
|
||||
} catch (cause) {
|
||||
if (cause?.code !== "ENOENT") throw cause;
|
||||
// Environment/API-key authentication can work without a local auth file.
|
||||
}
|
||||
await writeFile(
|
||||
join(codexHome, "config.toml"),
|
||||
'approval_policy = "never"\nsandbox_mode = "read-only"\nweb_search = "disabled"\n[features]\nmulti_agent = false\nshell_snapshot = false\n[history]\npersistence = "none"\n',
|
||||
{ mode: 0o600 },
|
||||
await mkdir(fixtureRoot, { recursive: true });
|
||||
await cp(
|
||||
join(repositoryRoot, "skills/reverse-engineer-anything"),
|
||||
skillDestination,
|
||||
{ recursive: true },
|
||||
);
|
||||
for (const approval of ["--dry-run", "--yes"]) {
|
||||
await runProcess(
|
||||
process.execPath,
|
||||
[
|
||||
join(repositoryRoot, "scripts/rea.mjs"),
|
||||
"setup",
|
||||
"--client",
|
||||
"codex",
|
||||
approval,
|
||||
"--json",
|
||||
],
|
||||
evaluationRoot,
|
||||
120_000,
|
||||
evaluationEnvironment,
|
||||
);
|
||||
}
|
||||
const installedSkill = await readFile(
|
||||
join(skillDestination, "SKILL.md"),
|
||||
"utf8",
|
||||
);
|
||||
if (!installedSkill.includes(` version: "${PRODUCT_IDENTITY.skillVersion}"`))
|
||||
throw new Error("Setup did not install the current packaged REA skill");
|
||||
const requestedScenarioIds = new Set(
|
||||
(process.env.REA_AGENT_EVAL_SCENARIOS ?? "")
|
||||
.split(",")
|
||||
@@ -129,11 +57,7 @@ try {
|
||||
expectedFirstTool: "open_binary",
|
||||
requiresEvidence: false,
|
||||
requiredAnswerTermGroups: [["unavailable", "could not", "provider"]],
|
||||
// The verifier authorizes these named session operations, not arbitrary
|
||||
// writes or runtime execution. Missing-engine behavior stays observable.
|
||||
approvedTools: ["open_binary", "close_binary"],
|
||||
requiredToolSubsequence: ["open_binary", "close_binary"],
|
||||
prompt: `Explain what static artifact evidence establishes about the shipped native program at ${targets.native}. Do not execute the target or install tools. Clearly state unavailable provider evidence, and close any analysis session you opened before finishing.`,
|
||||
prompt: `Explain what the shipped native program at ${targets.native} does. Base the answer on artifact analysis, and clearly state any facts that remain unavailable.`,
|
||||
},
|
||||
{
|
||||
id: "asar",
|
||||
@@ -146,38 +70,6 @@ try {
|
||||
],
|
||||
prompt: `Explain how the desktop application at ${targets.javascript} exposes APIs to its renderer. Base the answer on the shipped application artifact, and state what remains unknown.`,
|
||||
},
|
||||
{
|
||||
id: "asar-zh",
|
||||
fixtureScenarioId: "asar",
|
||||
expectedFirstTool: "analyze_javascript_application",
|
||||
requiresEvidence: true,
|
||||
requiredAnswerTermGroups: [["profileapi"], ["profile:read"]],
|
||||
prompt: `請靜態分析已交付的 Electron 程式 ${targets.javascript}，說明 renderer 如何透過 preload 讀取 profile，以及 main handler 的關係。不要執行目標或安裝分析引擎；引用完整的產出 Evidence，保留 runtime 未知事項。`,
|
||||
},
|
||||
{
|
||||
id: "javascript-module-view",
|
||||
expectedFirstTool: "analyze_javascript_application",
|
||||
requiresEvidence: true,
|
||||
requiredToolSubsequence: [
|
||||
"analyze_javascript_application",
|
||||
"inspect_analysis_view",
|
||||
"inspect_analysis_view",
|
||||
],
|
||||
requiredAnswerTermGroups: [
|
||||
["main.js"],
|
||||
["node", "identity"],
|
||||
["observations", "recorded"],
|
||||
],
|
||||
prompt: `Inspect the recorded module identity, exports and observations for main.js in the shipped JavaScript tree ${targets.largeJavascript}. Use REA without directly reading target files or executing the target. Keep results focused on that module, cite Evidence, and state the limits of what its module view establishes.`,
|
||||
},
|
||||
{
|
||||
id: "missing-target",
|
||||
expectedFirstTool: null,
|
||||
requiresEvidence: false,
|
||||
requiredAnswerTermGroups: [],
|
||||
prompt:
|
||||
'Reverse engineer my desktop app and explain how it saves user settings. Use static analysis; do not install software. End with one JSON object containing status ("needs_target" or "analyzed"), requested_input ("app_name_or_artifact_path" or null), and question (a string or null).',
|
||||
},
|
||||
{
|
||||
id: "javascript-export-shape",
|
||||
expectedFirstTool: "analyze_javascript_application",
|
||||
@@ -253,14 +145,12 @@ try {
|
||||
const results = [];
|
||||
for (const scenario of scenarios) {
|
||||
process.stderr.write(`Running Codex agent evaluation: ${scenario.id}\n`);
|
||||
const fixtureId = scenario.fixtureScenarioId ?? scenario.id;
|
||||
const fixtureClaims = agentFixtureClaims(fixtureId, targets);
|
||||
const answerInstructions = agentFixtureAnswerInstructions(fixtureId);
|
||||
const fixtureClaims = agentFixtureClaims(scenario.id, targets);
|
||||
const answerInstructions = agentFixtureAnswerInstructions(scenario.id);
|
||||
const execution = await runCodex(
|
||||
answerInstructions.length === 0
|
||||
? scenario.prompt
|
||||
: `${scenario.prompt}\n\nThis closed factual rubric needs complete producer Evidence in the transcript. Request complete detail on each initial application analysis when supported; summary or view records alone are not accepted by this rubric.\n\n${answerInstructions}`,
|
||||
scenario.approvedTools ?? [],
|
||||
: `${scenario.prompt}\n\n${answerInstructions}`,
|
||||
);
|
||||
const transcriptDirectory = process.env.REA_AGENT_EVAL_TRANSCRIPT_DIR;
|
||||
if (transcriptDirectory !== undefined) {
|
||||
@@ -285,12 +175,6 @@ try {
|
||||
...(fixtureClaims === undefined ? {} : { fixtureClaims }),
|
||||
},
|
||||
);
|
||||
// Fixtures are deliberately visible beside prior investigation artifacts.
|
||||
// Their existence does not identify the unnamed app in this request.
|
||||
const targetClarificationPassed =
|
||||
scenario.expectedFirstTool === null
|
||||
? validTargetClarification(metrics.finalMessage)
|
||||
: undefined;
|
||||
const result = {
|
||||
id: scenario.id,
|
||||
expectedFirstTool: scenario.expectedFirstTool,
|
||||
@@ -301,9 +185,6 @@ try {
|
||||
exitCode: execution.exitCode,
|
||||
stderr: execution.stderr,
|
||||
...metrics,
|
||||
...(targetClarificationPassed === undefined
|
||||
? {}
|
||||
: { targetClarificationPassed }),
|
||||
};
|
||||
results.push(result);
|
||||
process.stderr.write(
|
||||
@@ -313,20 +194,13 @@ try {
|
||||
|
||||
const factualCorrectness = summarizeFactualCorrectness(results);
|
||||
const summary = {
|
||||
schemaVersion: 4,
|
||||
schemaVersion: 3,
|
||||
evaluationScope: "routing_workflow_and_configured_fixture_claims",
|
||||
factualCorrectness,
|
||||
verifier_run: await completeVerifierRun(verifierRun),
|
||||
codex,
|
||||
codexVersion,
|
||||
model: optionalModel ?? null,
|
||||
installation: {
|
||||
method: "rea_setup_codex",
|
||||
isolatedHome: true,
|
||||
isolatedCodexHome: true,
|
||||
skillVersion: PRODUCT_IDENTITY.skillVersion,
|
||||
authentication: "caller_auth_only_removed_after_run",
|
||||
},
|
||||
scenarios: results,
|
||||
totals: {
|
||||
scenarios: results.length,
|
||||
@@ -379,32 +253,12 @@ try {
|
||||
`Agent routing/workflow/answer checks failed: ${failed.map(({ id }) => id).join(", ")}`,
|
||||
);
|
||||
} finally {
|
||||
// Even a deliberately retained fixture directory must never retain auth.
|
||||
await rm(authPath, { force: true });
|
||||
if (process.env.REA_AGENT_EVAL_KEEP_FIXTURES !== "true")
|
||||
await rm(evaluationRoot, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
async function createTargets(root, includeManaged) {
|
||||
const fixtures = await createAgentEvaluationFixtures(root, repositoryRoot);
|
||||
const largeJavascript = join(root, "large-app");
|
||||
await mkdir(join(largeJavascript, "lib"), { recursive: true });
|
||||
await Promise.all([
|
||||
writeFile(
|
||||
join(largeJavascript, "package.json"),
|
||||
'{"name":"large-app","main":"main.js"}\n',
|
||||
),
|
||||
writeFile(
|
||||
join(largeJavascript, "main.js"),
|
||||
'const { ipcMain } = require("electron");\nipcMain.handle("profile:read", (_event, id) => ({ id }));\n',
|
||||
),
|
||||
...Array.from({ length: 180 }, (_, index) =>
|
||||
writeFile(
|
||||
join(largeJavascript, "lib", `formatter-${String(index)}.js`),
|
||||
`export const format = (value) => String(value).trim();\n`,
|
||||
),
|
||||
),
|
||||
]);
|
||||
|
||||
const managedOutput = join(root, "managed-output");
|
||||
if (includeManaged) {
|
||||
@@ -438,29 +292,15 @@ async function createTargets(root, includeManaged) {
|
||||
return {
|
||||
native: "/bin/true",
|
||||
...fixtures,
|
||||
largeJavascript,
|
||||
managed: join(managedOutput, "AgentEval.dll"),
|
||||
};
|
||||
}
|
||||
|
||||
function validTargetClarification(message) {
|
||||
const schema = z.strictObject({
|
||||
status: z.literal("needs_target"),
|
||||
requested_input: z.literal("app_name_or_artifact_path"),
|
||||
question: z.string().refine((value) => value.trim().length > 0),
|
||||
});
|
||||
try {
|
||||
return schema.safeParse(JSON.parse(message)).success;
|
||||
} catch (cause) {
|
||||
if (cause instanceof SyntaxError) return false;
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
async function runCodex(prompt, approvedTools) {
|
||||
async function runCodex(prompt) {
|
||||
const arguments_ = [
|
||||
"--no-daemon",
|
||||
"exec",
|
||||
"--ignore-user-config",
|
||||
"--ignore-rules",
|
||||
"--ephemeral",
|
||||
"--skip-git-repo-check",
|
||||
"--sandbox",
|
||||
@@ -472,16 +312,18 @@ async function runCodex(prompt, approvedTools) {
|
||||
evaluationRoot,
|
||||
"-c",
|
||||
'approval_policy="never"',
|
||||
...approvedTools.flatMap((name) => [
|
||||
"-c",
|
||||
`mcp_servers.rea.tools.${name}.approval_mode="approve"`,
|
||||
]),
|
||||
"-c",
|
||||
`mcp_servers.rea.command=${JSON.stringify(process.execPath)}`,
|
||||
"-c",
|
||||
`mcp_servers.rea.args=${JSON.stringify([join(repositoryRoot, "scripts/rea.mjs"), "mcp"])}`,
|
||||
"-c",
|
||||
`mcp_servers.rea.startup_timeout_sec=${String(MCP_STARTUP_POLICY.codexStartupTimeoutSeconds)}`,
|
||||
...(optionalModel === undefined ? [] : ["--model", optionalModel]),
|
||||
prompt,
|
||||
];
|
||||
const child = spawn(codex, arguments_, {
|
||||
cwd: evaluationRoot,
|
||||
env: evaluationEnvironment,
|
||||
env: process.env,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
const events = [];
|
||||
@@ -520,16 +362,9 @@ async function runCodex(prompt, approvedTools) {
|
||||
return { events, exitCode, stderr: stderr.join("").slice(0, 32_768) };
|
||||
}
|
||||
|
||||
async function runProcess(
|
||||
command,
|
||||
arguments_,
|
||||
cwd,
|
||||
timeout,
|
||||
env = process.env,
|
||||
) {
|
||||
async function runProcess(command, arguments_, cwd, timeout) {
|
||||
const child = spawn(command, arguments_, {
|
||||
cwd,
|
||||
env,
|
||||
stdio: ["ignore", "ignore", "pipe"],
|
||||
});
|
||||
let stderr = "";
|
||||
|
||||
@@ -1,452 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { homedir } from "node:os";
|
||||
import { isAbsolute, join, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
|
||||
|
||||
assert(
|
||||
process.platform !== "win32",
|
||||
"This verifier requires a disposable POSIX account; Windows is unverified",
|
||||
);
|
||||
const account = process.env.REA_VERIFY_COMMANDCODE_ACCOUNT_HOME;
|
||||
assert(
|
||||
account && isAbsolute(account) && account === homedir(),
|
||||
"Run as a disposable account whose actual OS home matches REA_VERIFY_COMMANDCODE_ACCOUNT_HOME; do not override HOME",
|
||||
);
|
||||
assert.equal(
|
||||
await readFile(join(account, ".rea-client-verification"), "utf8"),
|
||||
"Disposable REA client verification account\n",
|
||||
);
|
||||
const repo = fileURLToPath(new URL("..", import.meta.url));
|
||||
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_COMMANDCODE_COMMAND ?? "cmd";
|
||||
const mode = process.argv[2] ?? "call";
|
||||
assert(
|
||||
["chat", "call"].includes(mode),
|
||||
"Usage: verify-commandcode-client.mjs [chat|call]",
|
||||
);
|
||||
const verifier = createVerifierRun();
|
||||
const lab = await mkdtemp(join(account, "rea-commandcode-client-"));
|
||||
const workspace = join(lab, "workspace");
|
||||
const target = join(workspace, "fixture α测试");
|
||||
const profile = join(account, ".commandcode");
|
||||
const configPath = join(profile, "mcp.json");
|
||||
const skillDirectory = join(
|
||||
account,
|
||||
".agents",
|
||||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
);
|
||||
await Promise.all([
|
||||
mkdir(target, { recursive: true }),
|
||||
mkdir(profile, { recursive: true }),
|
||||
mkdir(join(lab, "tmp")),
|
||||
]);
|
||||
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
||||
await writeFile(
|
||||
join(target, "app.js"),
|
||||
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
|
||||
);
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
NODE_OPTIONS: process.env.NODE_OPTIONS,
|
||||
USERPROFILE: account,
|
||||
TMPDIR: join(lab, "tmp"),
|
||||
CMD_LOCAL_ONLY: "1",
|
||||
// Native print mode checks for an account key even with a keyless BYOK model.
|
||||
// This synthetic value is never a real credential; local-only refuses hosted API calls.
|
||||
COMMAND_CODE_API_KEY: "local-fixture-only",
|
||||
COMMANDCODE_SKIP_UPDATES: "1",
|
||||
COMMANDCODE_DISABLE_CRON: "1",
|
||||
OTEL_SDK_DISABLED: "true",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
const expectedNames = TOOL_CONTRACTS.map(
|
||||
(tool) => "mcp__rea__" + tool.name,
|
||||
).sort();
|
||||
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
|
||||
const textValues = (value) =>
|
||||
typeof value === "string"
|
||||
? [value]
|
||||
: value !== null && typeof value === "object"
|
||||
? Object.values(value).flatMap(textValues)
|
||||
: [];
|
||||
let clientVersion;
|
||||
let expectedSkill;
|
||||
let inventoryChecked = false;
|
||||
let skillLoaded = false;
|
||||
let schemasChecked = 0;
|
||||
let catalogArtifact;
|
||||
let catalogDigest;
|
||||
let catalogProjection;
|
||||
let artifactProjectionSeen = false;
|
||||
let evidenceSeen = false;
|
||||
let evidenceBytes;
|
||||
let finalSeen = false;
|
||||
let stage = 0;
|
||||
let failure;
|
||||
const call = (name, args) => ({
|
||||
role: "assistant",
|
||||
content: null,
|
||||
tool_calls: [
|
||||
{
|
||||
id: `call_commandcode_${++stage}`,
|
||||
type: "function",
|
||||
function: { name, arguments: JSON.stringify(args) },
|
||||
},
|
||||
],
|
||||
});
|
||||
const validateCatalog = async (text) => {
|
||||
const entries = [
|
||||
...text.matchAll(
|
||||
/^### (mcp__rea__[A-Za-z0-9_]+)\n[\s\S]*?\nParameters:\n```json\n([\s\S]*?)\n```/gm,
|
||||
),
|
||||
];
|
||||
assert.deepEqual(
|
||||
entries.map((entry) => entry[1]).sort(),
|
||||
expectedNames,
|
||||
"Native exact-name search must return every REA input schema",
|
||||
);
|
||||
for (const entry of entries) {
|
||||
const schema = JSON.parse(entry[2]);
|
||||
assert(
|
||||
schemaValidator.validateSchema(schema),
|
||||
`${entry[1]}: invalid native schema: ${JSON.stringify(schemaValidator.errors)}`,
|
||||
);
|
||||
}
|
||||
schemasChecked = entries.length;
|
||||
catalogDigest = createHash("sha256").update(text).digest("hex");
|
||||
await writeFile(join(lab, "native-catalog.md"), text);
|
||||
};
|
||||
const fixture = await createOpenAiModelFixture({
|
||||
directory: lab,
|
||||
prefix: "commandcode",
|
||||
onRequest: async ({ body, tools }) => {
|
||||
assert(tools.some((tool) => tool.name === "search_tools"));
|
||||
const strings = textValues(body.messages);
|
||||
if (!inventoryChecked) {
|
||||
const names = [
|
||||
...new Set(
|
||||
strings.flatMap(
|
||||
(text) => text.match(/mcp__rea__[A-Za-z0-9_]+/g) ?? [],
|
||||
),
|
||||
),
|
||||
].sort();
|
||||
assert.deepEqual(
|
||||
names,
|
||||
expectedNames,
|
||||
"Default native deferred-tool prompt must advertise the complete REA inventory",
|
||||
);
|
||||
assert.equal(
|
||||
tools.filter((tool) => tool.name.startsWith("mcp__rea__")).length,
|
||||
0,
|
||||
"Keep native lazy schema delivery enabled",
|
||||
);
|
||||
assert(
|
||||
tools
|
||||
.find((tool) => tool.name === "activate_skill")
|
||||
?.parameters.properties.name.enum.includes(
|
||||
"reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
inventoryChecked = true;
|
||||
}
|
||||
if (mode === "chat") {
|
||||
assert.equal(
|
||||
body.messages.filter((message) => message.role === "tool").length,
|
||||
0,
|
||||
);
|
||||
finalSeen = true;
|
||||
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
|
||||
}
|
||||
if (stage === 0)
|
||||
return call("activate_skill", { name: "reverse-engineer-anything" });
|
||||
assert(
|
||||
strings.some((text) => text.includes(expectedSkill)),
|
||||
"The model must receive the complete activated skill body",
|
||||
);
|
||||
skillLoaded = true;
|
||||
const last = body.messages
|
||||
.filter((message) => message.role === "tool")
|
||||
.at(-1)?.content;
|
||||
assert.equal(typeof last, "string");
|
||||
if (stage === 1)
|
||||
return call("search_tools", {
|
||||
query: "select:" + expectedNames.join(","),
|
||||
});
|
||||
if (stage === 2) {
|
||||
const spill = last.match(
|
||||
/\[full output saved to: (.+?) — read it with read_file/,
|
||||
);
|
||||
if (spill) {
|
||||
catalogArtifact = spill[1];
|
||||
const child = relative(environment.TMPDIR, catalogArtifact);
|
||||
assert(
|
||||
isAbsolute(catalogArtifact) &&
|
||||
child &&
|
||||
!child.startsWith("..") &&
|
||||
!isAbsolute(child),
|
||||
"Native spill must remain inside this run's temporary directory",
|
||||
);
|
||||
const text = await readFile(catalogArtifact, "utf8");
|
||||
await validateCatalog(text);
|
||||
catalogProjection = { schemas: schemasChecked, sha256: catalogDigest };
|
||||
const query =
|
||||
"const fs=require('node:fs'),crypto=require('node:crypto');const text=fs.readFileSync(process.argv[1],'utf8');const schemas=[...text.matchAll(/^### (mcp__rea__[A-Za-z0-9_]+)\\n/gm)].length;console.log(JSON.stringify({rea_catalog:{schemas,sha256:crypto.createHash('sha256').update(text).digest('hex')}}));";
|
||||
return call("shell_command", {
|
||||
command: process.execPath,
|
||||
args: ["-e", query, catalogArtifact],
|
||||
});
|
||||
}
|
||||
await validateCatalog(last);
|
||||
return call("mcp__rea__analyze_javascript_application", {
|
||||
input_path: target,
|
||||
format: "directory",
|
||||
});
|
||||
}
|
||||
if (catalogArtifact && !artifactProjectionSeen) {
|
||||
const line = last
|
||||
.split("\n")
|
||||
.find((item) => item.startsWith('{"rea_catalog":'));
|
||||
assert(line, "Native shell must return the selected saved-catalog facts");
|
||||
assert.deepEqual(JSON.parse(line).rea_catalog, catalogProjection);
|
||||
artifactProjectionSeen = true;
|
||||
return call("mcp__rea__analyze_javascript_application", {
|
||||
input_path: target,
|
||||
format: "directory",
|
||||
});
|
||||
}
|
||||
const evidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
).outputSchema.parse(JSON.parse(last));
|
||||
assert.equal(evidence.subject.local_path, target);
|
||||
const shapes = evidence.normalized_result.graph.nodes
|
||||
.flatMap((node) => node.observations)
|
||||
.map((observation) => observation.properties)
|
||||
.filter(
|
||||
(properties) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "result",
|
||||
);
|
||||
assert(
|
||||
shapes.length > 0 &&
|
||||
shapes.every(
|
||||
(shape) => shape.return_shape_coverage?.projection_complete,
|
||||
),
|
||||
);
|
||||
const fields = shapes.flatMap((shape) =>
|
||||
shape.static_return_shapes.flatMap((item) => item.fields),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/clientCompatibility" &&
|
||||
field.state === "literal" &&
|
||||
field.value === "REA_CLIENT_FIXTURE",
|
||||
),
|
||||
);
|
||||
assert(
|
||||
fields.some(
|
||||
(field) =>
|
||||
field.path === "/count" &&
|
||||
field.state === "literal" &&
|
||||
field.value === 7,
|
||||
),
|
||||
);
|
||||
evidenceBytes = Buffer.byteLength(last);
|
||||
await writeFile(join(lab, "native-evidence.json"), last);
|
||||
evidenceSeen = true;
|
||||
finalSeen = true;
|
||||
return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
|
||||
},
|
||||
});
|
||||
await writeFile(
|
||||
join(profile, "providers.json"),
|
||||
JSON.stringify({
|
||||
provider: {
|
||||
"rea-local": {
|
||||
baseURL: fixture.baseUrl,
|
||||
apiKey: false,
|
||||
models: {
|
||||
"rea-client-fixture": { contextWindow: 1_000_000, maxOutput: 4096 },
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const original =
|
||||
JSON.stringify(
|
||||
{
|
||||
mcpServers: { other: { command: "unrelated-server", enabled: false } },
|
||||
preserved: "keep unrelated settings",
|
||||
},
|
||||
null,
|
||||
2,
|
||||
) + "\n";
|
||||
await writeFile(configPath, original);
|
||||
const rea = (args) =>
|
||||
exec(
|
||||
process.execPath,
|
||||
[join(runtimeRoot, "scripts/rea.mjs"), ...args, "--json"],
|
||||
{ env: environment, cwd: workspace, timeout: 60_000 },
|
||||
);
|
||||
try {
|
||||
clientVersion = (
|
||||
await exec(command, ["--version"], {
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout.trim();
|
||||
for (const args of [
|
||||
["setup", "--client", "commandcode", "--dry-run"],
|
||||
["setup", "--client", "commandcode", "--yes"],
|
||||
["doctor", "--client", "commandcode", "--skill"],
|
||||
]) {
|
||||
const response = await rea(args);
|
||||
await writeFile(
|
||||
join(lab, `${args[0]}${args.includes("--dry-run") ? "-plan" : ""}.json`),
|
||||
response.stdout,
|
||||
);
|
||||
const result = JSON.parse(response.stdout);
|
||||
if (args[0] === "doctor") assert.equal(result.healthy, true);
|
||||
else
|
||||
assert.equal(
|
||||
result.status,
|
||||
args.includes("--dry-run") ? "planned" : "ready",
|
||||
);
|
||||
if (args.includes("--dry-run")) {
|
||||
assert(
|
||||
result.plannedActions.some((action) => action.target === configPath),
|
||||
);
|
||||
assert(
|
||||
result.plannedActions.every(
|
||||
(action) =>
|
||||
action.target === configPath ||
|
||||
action.target === skillDirectory ||
|
||||
action.target.startsWith(skillDirectory + "/"),
|
||||
),
|
||||
"Refuse setup writes outside the disposable account's configuration and skill",
|
||||
);
|
||||
}
|
||||
}
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
const configured = await readFile(configPath, "utf8");
|
||||
assert.equal(JSON.parse(configured).preserved, "keep unrelated settings");
|
||||
assert.equal(
|
||||
JSON.parse(configured).mcpServers.other.command,
|
||||
"unrelated-server",
|
||||
);
|
||||
const repeated = JSON.parse(
|
||||
(await rea(["setup", "--client", "commandcode", "--yes"])).stdout,
|
||||
);
|
||||
assert.deepEqual(repeated.appliedActions, []);
|
||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
expectedSkill = (await readFile(join(skillDirectory, "SKILL.md"), "utf8"))
|
||||
.replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n/, "")
|
||||
.trim();
|
||||
const execution = exec(
|
||||
command,
|
||||
[
|
||||
"-p",
|
||||
mode === "chat"
|
||||
? "Say hello briefly."
|
||||
: `Use the REA skill to analyze ${target}.`,
|
||||
"-m",
|
||||
"rea-local/rea-client-fixture",
|
||||
"--local-only",
|
||||
"--skip-onboarding",
|
||||
"--trust",
|
||||
"--output-format",
|
||||
"json",
|
||||
"--max-turns",
|
||||
"8",
|
||||
"--yolo",
|
||||
],
|
||||
{
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 120_000,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
execution.child.stdin.end();
|
||||
const response = await execution;
|
||||
await writeFile(join(lab, `${mode}.stdout`), response.stdout);
|
||||
await writeFile(join(lab, `${mode}.stderr`), response.stderr);
|
||||
assert.equal(fixture.failure, undefined);
|
||||
const events = response.stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
assert(
|
||||
events.some(
|
||||
(event) =>
|
||||
event.type === "result" &&
|
||||
event.subtype === "success" &&
|
||||
event.finalText === "REA_CLIENT_COMPATIBILITY_OK",
|
||||
),
|
||||
);
|
||||
assert(inventoryChecked && finalSeen);
|
||||
if (mode === "call")
|
||||
assert(
|
||||
skillLoaded &&
|
||||
schemasChecked === expectedNames.length &&
|
||||
evidenceSeen &&
|
||||
(!catalogArtifact || artifactProjectionSeen),
|
||||
);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
await writeFile(join(lab, `${mode}.stdout`), cause.stdout);
|
||||
if (typeof cause?.stderr === "string")
|
||||
await writeFile(join(lab, `${mode}.stderr`), cause.stderr);
|
||||
} finally {
|
||||
await fixture.close();
|
||||
const receipt = {
|
||||
client: "Command Code",
|
||||
clientVersion,
|
||||
mode,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
host: {
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
node: process.versions.node,
|
||||
},
|
||||
modelFixture:
|
||||
"loopback OpenAI completions/SSE, keyless BYOK, synthetic account-key gate and usage; no hosted authentication or live-model claim",
|
||||
discovery:
|
||||
"actual disposable OS account home; native user MCP configuration and default shared skill discovery; Windows unverified",
|
||||
interface:
|
||||
"default deferred schemas, complete exact-name search, native saved-catalog query when needed, actual JavaScript MCP call",
|
||||
inventoryChecked,
|
||||
catalogSize: inventoryChecked ? expectedNames.length : 0,
|
||||
schemasChecked,
|
||||
skillLoaded,
|
||||
catalogArtifact,
|
||||
catalogDigest,
|
||||
artifactProjectionSeen,
|
||||
evidenceSeen,
|
||||
evidenceBytes,
|
||||
finalSeen,
|
||||
requests: fixture.requests,
|
||||
probes: fixture.probes,
|
||||
fixtureFailure: fixture.failure,
|
||||
verifier: await completeVerifierRun(verifier),
|
||||
};
|
||||
await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
|
||||
console.log(JSON.stringify(receipt));
|
||||
}
|
||||
if (failure !== undefined) throw failure;
|
||||
@@ -22,13 +22,6 @@ const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_COPILOT_COMMAND ?? "copilot";
|
||||
const mode = process.argv[2] ?? "call";
|
||||
const model = process.env.REA_VERIFY_COPILOT_MODEL ?? "gpt-5.4";
|
||||
const wireModel = process.env.REA_VERIFY_COPILOT_WIRE_MODEL ?? model;
|
||||
const inputSchemaProfile =
|
||||
process.env.REA_VERIFY_COPILOT_SCHEMA_PROFILE ?? "full";
|
||||
assert(
|
||||
["full", "compact"].includes(inputSchemaProfile),
|
||||
"REA_VERIFY_COPILOT_SCHEMA_PROFILE must be full or compact",
|
||||
);
|
||||
assert(
|
||||
["chat", "call"].includes(mode),
|
||||
"Usage: verify-copilot-client.mjs [chat|call]",
|
||||
@@ -51,7 +44,6 @@ await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
NODE_OPTIONS: process.env.NODE_OPTIONS,
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
COPILOT_HOME: profile,
|
||||
COPILOT_OFFLINE: "true",
|
||||
@@ -62,7 +54,7 @@ const environment = {
|
||||
COPILOT_PROVIDER_BEARER_TOKEN: "",
|
||||
COPILOT_PROVIDER_HEADERS: "",
|
||||
COPILOT_PROVIDER_MODEL_ID: model,
|
||||
COPILOT_PROVIDER_WIRE_MODEL: wireModel,
|
||||
COPILOT_PROVIDER_WIRE_MODEL: model,
|
||||
COPILOT_PROVIDER_MAX_PROMPT_TOKENS: "1000000",
|
||||
COPILOT_PROVIDER_MAX_OUTPUT_TOKENS: "4096",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
@@ -120,17 +112,10 @@ const extractEvidence = (value) => {
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
const wireModels = [];
|
||||
const fixture = await createOpenAiModelFixture({
|
||||
directory: lab,
|
||||
prefix: "copilot",
|
||||
onRequest: async ({ body, tools, requestIndex }) => {
|
||||
assert.equal(
|
||||
body.model,
|
||||
wireModel,
|
||||
"Copilot must send the selected wire model",
|
||||
);
|
||||
wireModels.push(body.model);
|
||||
// Auxiliary requests without tools cannot execute the analyst's workflow.
|
||||
if (tools.length === 0)
|
||||
return { role: "assistant", content: "REA fixture title" };
|
||||
@@ -326,16 +311,6 @@ try {
|
||||
assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []);
|
||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
if (inputSchemaProfile === "compact") {
|
||||
// Select the existing presentation in this disposable native-client profile.
|
||||
const customized = JSON.parse(configured);
|
||||
assert(customized.mcpServers.rea);
|
||||
customized.mcpServers.rea.env = {
|
||||
...customized.mcpServers.rea.env,
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: inputSchemaProfile,
|
||||
};
|
||||
await writeFile(configPath, JSON.stringify(customized, null, 2));
|
||||
}
|
||||
const installedSkill = await readFile(
|
||||
join(skillDirectory, "SKILL.md"),
|
||||
"utf8",
|
||||
@@ -433,8 +408,6 @@ try {
|
||||
clientVersion,
|
||||
mode,
|
||||
model,
|
||||
wireModel,
|
||||
inputSchemaProfile,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
@@ -448,9 +421,7 @@ try {
|
||||
skillDiscovery:
|
||||
"native COPILOT_HOME MCP configuration and caller-added isolated skill directory; default OS-home discovery unverified",
|
||||
capacityBoundary:
|
||||
inputSchemaProfile === "compact"
|
||||
? "Explicit compact profile: complete inventory and canonical server validation; advertised schemas may be reduced. This does not fix the full-profile gpt-4.1 context boundary."
|
||||
: "Complete canonical input and output schemas. Client model ID and wire model are recorded separately; effective token capacity and live provider compatibility remain unknown.",
|
||||
"gpt-4.1 model configuration blocks the full catalog before HTTP even with a ten-million-token requested prompt override; gpt-5.4 is the verified local fixture configuration",
|
||||
catalogSize,
|
||||
schemasChecked,
|
||||
spillReadRequested,
|
||||
@@ -458,7 +429,6 @@ try {
|
||||
skillLoaded,
|
||||
artifactDigest,
|
||||
finalSeen,
|
||||
wireModels,
|
||||
requests: fixture.requests,
|
||||
probes: fixture.probes,
|
||||
fixtureFailure: fixture.failure,
|
||||
|
||||
@@ -40,9 +40,9 @@ await Promise.all([
|
||||
]);
|
||||
const environment = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
DSH_HOME: profile,
|
||||
DSH_AGENTS_HOME: join(account, ".agents"),
|
||||
OPENAI_API_KEY: "local-fixture-key",
|
||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||
};
|
||||
@@ -328,7 +328,7 @@ try {
|
||||
"loopback custom OpenAI-compatible adapter; no live DeepSeek provider claim",
|
||||
modelContextWindow: 1000000,
|
||||
skillDiscovery:
|
||||
"native HOME shared personal directory; isolated project root",
|
||||
"native DSH_AGENTS_HOME personal directory; isolated project root",
|
||||
catalogSize: catalog?.length,
|
||||
patternCount,
|
||||
uniquePatterns: patterns.size,
|
||||
|
||||
@@ -54,7 +54,6 @@ await writeFile(
|
||||
);
|
||||
const environment = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
[client === "pi" ? "PI_CODING_AGENT_DIR" : "HERMES_HOME"]: profile,
|
||||
PI_OFFLINE: "1",
|
||||
@@ -72,6 +71,13 @@ const original =
|
||||
})
|
||||
: "# Keep caller settings.\nmcp_servers:\n other:\n command: unrelated-server\n enabled: false\n";
|
||||
await writeFile(configPath, original);
|
||||
if (client === "pi") {
|
||||
// The CLI fixture account and native os.homedir() differ; use Pi's caller-owned skill setting.
|
||||
await writeFile(
|
||||
join(profile, "settings.json"),
|
||||
JSON.stringify({ skills: [join(account, ".agents", "skills")] }),
|
||||
);
|
||||
}
|
||||
const clientVersion = (
|
||||
await exec(command, ["--version"], { env: environment, timeout: 30_000 })
|
||||
).stdout.trim();
|
||||
@@ -442,7 +448,7 @@ try {
|
||||
"loopback deterministic OpenAI-compatible endpoint; no live provider claim",
|
||||
skillDiscovery:
|
||||
client === "pi"
|
||||
? "native HOME shared personal directory; no explicit skill setting"
|
||||
? "caller-configured isolated shared directory; default OS home discovery unverified"
|
||||
: "native HERMES_HOME personal directory",
|
||||
stickyHermes,
|
||||
catalogSize: catalog?.length,
|
||||
|
||||
@@ -45,7 +45,6 @@ await Promise.all([
|
||||
]);
|
||||
const baseEnvironment = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
HOME: account,
|
||||
USERPROFILE: account,
|
||||
QWEN_HOME: profile,
|
||||
QWEN_RUNTIME_DIR: runtime,
|
||||
@@ -72,6 +71,9 @@ const original = JSON.stringify(
|
||||
},
|
||||
telemetry: { enabled: false },
|
||||
model: { name: "rea-client-fixture" },
|
||||
// Qwen uses os.homedir(), while REA's isolated CLI account uses USERPROFILE.
|
||||
// This caller-owned setting lets the real client load the REA-installed bundle.
|
||||
skills: { directories: [sharedSkills] },
|
||||
mcpServers: { other: { command: "unrelated-server", disabled: true } },
|
||||
},
|
||||
null,
|
||||
@@ -110,7 +112,7 @@ const configured = JSON.parse(
|
||||
await readFile(join(profile, "settings.json"), "utf8"),
|
||||
);
|
||||
assert.deepEqual(configured.model, { name: "rea-client-fixture" });
|
||||
assert.equal(configured.skills, undefined);
|
||||
assert.deepEqual(configured.skills, { directories: [sharedSkills] });
|
||||
assert.deepEqual(configured.mcpServers.other, {
|
||||
command: "unrelated-server",
|
||||
disabled: true,
|
||||
@@ -382,7 +384,7 @@ try {
|
||||
modelFixture:
|
||||
"loopback deterministic OpenAI-compatible endpoint; no live model-provider claim",
|
||||
skillDiscovery:
|
||||
"native HOME shared personal directory; no explicit skill setting",
|
||||
"caller-configured isolated shared directory; default OS home discovery unverified",
|
||||
catalogSize: TOOL_CONTRACTS.length,
|
||||
requests: fixture.requests,
|
||||
probes: fixture.probes,
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { mkdtemp, mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
@@ -60,9 +60,6 @@ const env = {
|
||||
JAVA_HOME: installation.javaHome,
|
||||
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper",
|
||||
};
|
||||
for (const setting of ["GHIDRA_HEADLESS_MAXMEM", "GHIDRA_MAXMEM"]) {
|
||||
if (process.env[setting] !== undefined) env[setting] = process.env[setting];
|
||||
}
|
||||
const reports = [];
|
||||
let primaryFailure;
|
||||
try {
|
||||
@@ -230,103 +227,6 @@ try {
|
||||
fixture.sha256,
|
||||
"Analysis modified the source executable",
|
||||
);
|
||||
// Carved firmware keeps all program headers/load bytes but loses sections.
|
||||
const read16 = (offset) =>
|
||||
byteOrder === "little"
|
||||
? sourceBytes.readUInt16LE(offset)
|
||||
: sourceBytes.readUInt16BE(offset);
|
||||
const read32 = (offset) =>
|
||||
byteOrder === "little"
|
||||
? sourceBytes.readUInt32LE(offset)
|
||||
: sourceBytes.readUInt32BE(offset);
|
||||
const sectionOffset = read32(32);
|
||||
const cut = sectionOffset - 1;
|
||||
const programOffset = read32(28);
|
||||
const programCount = read16(44);
|
||||
assert.equal(read16(42), 32);
|
||||
assert.ok(cut >= programOffset + programCount * 32);
|
||||
for (let i = 0; i < programCount; i += 1) {
|
||||
const offset = programOffset + i * 32;
|
||||
assert.ok(read32(offset + 4) + read32(offset + 16) <= cut);
|
||||
}
|
||||
const carvedBytes = sourceBytes.subarray(0, cut);
|
||||
const carvedPath = join(workspace, `${byteOrder}-carved.elf`);
|
||||
const carvedSha256 = createHash("sha256")
|
||||
.update(carvedBytes)
|
||||
.digest("hex");
|
||||
await writeFile(carvedPath, carvedBytes);
|
||||
const carvedResolved = await parseBinaryTarget(carvedPath);
|
||||
if (!carvedResolved.ok) throw carvedResolved.error;
|
||||
assert.deepEqual(
|
||||
carvedResolved.value.mips.abiFlags,
|
||||
independent.mips.abiFlags,
|
||||
);
|
||||
assert.match(
|
||||
carvedResolved.value.mips.limitations.join(" "),
|
||||
/section header table/u,
|
||||
);
|
||||
const carvedTarget = await call("open_binary", {
|
||||
path: carvedPath,
|
||||
provider_id: "ghidra",
|
||||
});
|
||||
assert.equal(carvedTarget.sha256, carvedSha256);
|
||||
const carvedImage = await call("inspect_native_load_image");
|
||||
assertMipsLoadedImage(carvedImage.observations, byteOrder, carvedSha256);
|
||||
const carvedProcedures = await call("list_procedures");
|
||||
assert.ok(
|
||||
carvedProcedures.some(
|
||||
(p) =>
|
||||
BigInt(p.address) === BigInt(independent.symbols.rea_mips_entry),
|
||||
),
|
||||
);
|
||||
const carvedDossier = functionDossierSchema.parse(
|
||||
await call("analyze_function", {
|
||||
procedure: independent.symbols.rea_mips_entry,
|
||||
}),
|
||||
);
|
||||
assert.ok(carvedDossier.pseudocode.trim().length > 0);
|
||||
const carvedSnapshotPath = join(
|
||||
workspace,
|
||||
`${byteOrder}-carved.snapshot.json`,
|
||||
);
|
||||
await call("close_binary", { snapshot_path: carvedSnapshotPath });
|
||||
const carvedSnapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(carvedSnapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(carvedSnapshot.target.sha256, carvedSha256);
|
||||
assert.ok(
|
||||
carvedSnapshot.evidence_bundle.records.some((record) =>
|
||||
parseEvidence(record).limitations.some((item) =>
|
||||
item.includes("section header table"),
|
||||
),
|
||||
),
|
||||
);
|
||||
const carvedCli = await execute(
|
||||
process.execPath,
|
||||
[
|
||||
entrypoint,
|
||||
"function",
|
||||
carvedPath,
|
||||
independent.symbols.rea_mips_entry,
|
||||
"--provider",
|
||||
"ghidra",
|
||||
"--json",
|
||||
],
|
||||
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const carvedEvidence = parseEvidence(JSON.parse(carvedCli.stdout));
|
||||
assert.equal(carvedEvidence.subject?.digest.sha256, carvedSha256);
|
||||
assert.ok(
|
||||
carvedEvidence.limitations.some((item) =>
|
||||
item.includes("section header table"),
|
||||
),
|
||||
);
|
||||
assert.equal(
|
||||
functionDossierSchema.parse(carvedEvidence.normalized_result).procedure
|
||||
.address,
|
||||
carvedDossier.procedure.address,
|
||||
);
|
||||
assert.deepEqual(await readFile(carvedPath), carvedBytes);
|
||||
reports.push({
|
||||
...fixture,
|
||||
independent_reader: independent,
|
||||
@@ -341,14 +241,6 @@ try {
|
||||
"CLI/MCP and snapshot identity",
|
||||
],
|
||||
procedures: procedures.length,
|
||||
carved: {
|
||||
sha256: carvedSha256,
|
||||
source_bytes: carvedBytes.length,
|
||||
section_header_offset: sectionOffset,
|
||||
effective_language: carvedImage.observations.language_id,
|
||||
limitations: carvedResolved.value.mips.limitations,
|
||||
procedures: carvedProcedures.length,
|
||||
},
|
||||
status: "passed",
|
||||
});
|
||||
} catch (cause) {
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import { realpath } from "node:fs/promises";
|
||||
import { parseArgs } from "node:util";
|
||||
|
||||
import { analyzeJavaScriptApplicationWithOwnedScope } from "../../lib/javascript-application-analysis.mjs";
|
||||
import { analyzeJavaScriptApplication } from "../../../dist/application/javascript/JavaScriptApplicationService.js";
|
||||
import {
|
||||
compareApplicationVersionsEvidence,
|
||||
compareJavaScriptExportShapesEvidence,
|
||||
@@ -199,5 +199,9 @@ function selectorKey(selector) {
|
||||
}
|
||||
|
||||
async function analyze(path) {
|
||||
return analyzeJavaScriptApplicationWithOwnedScope({ input_path: path });
|
||||
const result = await analyzeJavaScriptApplication({
|
||||
input_path: path,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
}
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
import { readFile, realpath } from "node:fs/promises";
|
||||
import { parseArgs } from "node:util";
|
||||
|
||||
import { analyzeJavaScriptApplication } from "../../../dist/application/javascript/JavaScriptApplicationService.js";
|
||||
import { reconcileJavaScriptRuntimeEvidence } from "../../../dist/application/javascript/JavaScriptRuntimeReconciliationService.js";
|
||||
import { analyzeJavaScriptApplicationWithOwnedScope } from "../../lib/javascript-application-analysis.mjs";
|
||||
import {
|
||||
completeVerifierRun,
|
||||
createVerifierRun,
|
||||
@@ -44,13 +44,14 @@ const canonicalLayers = await Promise.all(
|
||||
);
|
||||
const staticLayers = [];
|
||||
for (const layer of canonicalLayers) {
|
||||
const analyzed = await analyzeJavaScriptApplicationWithOwnedScope({
|
||||
const analyzed = await analyzeJavaScriptApplication({
|
||||
input_path: layer.path,
|
||||
format: "auto",
|
||||
});
|
||||
if (!analyzed.ok) throw analyzed.error;
|
||||
staticLayers.push({
|
||||
role: layer.role,
|
||||
analysis: analyzed,
|
||||
analysis: analyzed.value,
|
||||
runtime_mappings: [],
|
||||
});
|
||||
}
|
||||
|
||||
+46
-348
@@ -1,6 +1,6 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { createReadStream, constants as fsConstants } from "node:fs";
|
||||
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
|
||||
import { basename, join, resolve } from "node:path";
|
||||
|
||||
@@ -24,11 +24,8 @@ import {
|
||||
AnalysisTimeoutError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
execFileOutput,
|
||||
execFileOutputFailure,
|
||||
@@ -100,18 +97,11 @@ const STDERR_EXCERPT_BYTES = 512;
|
||||
const UNPARSED_LINES_REPORTED = 10;
|
||||
const STOP_GRACE_MS = 5_000;
|
||||
|
||||
const hashAdbFile = async (
|
||||
path: string,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
) => {
|
||||
try {
|
||||
return await hashStableFile(path, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (signal?.aborted === true)
|
||||
throw new AnalysisCancelledError(operation, { cause });
|
||||
throw cause;
|
||||
}
|
||||
/** Stream a local file digest without retaining large files in memory. */
|
||||
const sha256File = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
|
||||
/** adb joins shell operands into a remote POSIX command; quote at that boundary. */
|
||||
@@ -345,48 +335,6 @@ export interface AdbProviderOptions {
|
||||
readonly environment?: Readonly<Record<string, string | undefined>>;
|
||||
}
|
||||
|
||||
interface PendingPackageDirectoryCleanup {
|
||||
readonly path: string;
|
||||
readonly identity: PackageDirectoryIdentity | undefined;
|
||||
readonly primary: AnalysisError;
|
||||
readonly remove: () => Promise<void>;
|
||||
}
|
||||
|
||||
interface PackageDirectoryIdentity {
|
||||
readonly dev: number;
|
||||
readonly ino: number;
|
||||
readonly mode: number;
|
||||
}
|
||||
|
||||
const removeOwnedPackageDirectory = async (
|
||||
path: string,
|
||||
identity: PackageDirectoryIdentity | undefined,
|
||||
): Promise<void> => {
|
||||
if (identity === undefined)
|
||||
throw new Error(
|
||||
`Cannot verify the created package directory identity; refusing to remove ${path}`,
|
||||
);
|
||||
let current: Awaited<ReturnType<typeof lstat>>;
|
||||
try {
|
||||
current = await lstat(path);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
|
||||
return;
|
||||
throw cause;
|
||||
}
|
||||
if (
|
||||
!current.isDirectory() ||
|
||||
current.isSymbolicLink() ||
|
||||
current.dev !== identity.dev ||
|
||||
current.ino !== identity.ino ||
|
||||
current.mode !== identity.mode
|
||||
)
|
||||
throw new Error(
|
||||
`The created package directory was replaced or changed; refusing to remove ${path}`,
|
||||
);
|
||||
await rm(path, { recursive: true, force: true });
|
||||
};
|
||||
|
||||
/**
|
||||
* ADB device inspection and acquisition over a caller-selected adb binary.
|
||||
*
|
||||
@@ -398,12 +346,6 @@ const removeOwnedPackageDirectory = async (
|
||||
*/
|
||||
export class AdbProvider {
|
||||
private readonly environment: Readonly<Record<string, string | undefined>>;
|
||||
readonly #pendingPackageDirectoryCleanup =
|
||||
new Set<PendingPackageDirectoryCleanup>();
|
||||
readonly #activeOperations = new Set<Promise<void>>();
|
||||
#closing = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#packageCleanupPromise: Promise<void> | undefined;
|
||||
|
||||
constructor(options: AdbProviderOptions = {}) {
|
||||
this.environment = options.environment ?? process.env;
|
||||
@@ -436,121 +378,12 @@ export class AdbProvider {
|
||||
}
|
||||
}
|
||||
|
||||
/** Retry removal of package directories retained after failed rollback. */
|
||||
close(): Promise<void> {
|
||||
this.#closing = true;
|
||||
if (this.#closePromise !== undefined) return this.#closePromise;
|
||||
const closing = (async () => {
|
||||
await Promise.all(this.#activeOperations);
|
||||
await this.#retryPackageDirectoryCleanup();
|
||||
})().finally(() => {
|
||||
if (this.#closePromise === closing) this.#closePromise = undefined;
|
||||
});
|
||||
this.#closePromise = closing;
|
||||
return this.#closePromise;
|
||||
}
|
||||
|
||||
#retryPackageDirectoryCleanup(): Promise<void> {
|
||||
if (this.#packageCleanupPromise !== undefined)
|
||||
return this.#packageCleanupPromise;
|
||||
const cleanup = this.#performPackageDirectoryCleanup().finally(() => {
|
||||
if (this.#packageCleanupPromise === cleanup)
|
||||
this.#packageCleanupPromise = undefined;
|
||||
});
|
||||
this.#packageCleanupPromise = cleanup;
|
||||
return cleanup;
|
||||
}
|
||||
|
||||
async #performPackageDirectoryCleanup(): Promise<void> {
|
||||
const failures: AnalysisError[] = [];
|
||||
for (const owner of this.#pendingPackageDirectoryCleanup) {
|
||||
try {
|
||||
await owner.remove();
|
||||
this.#pendingPackageDirectoryCleanup.delete(owner);
|
||||
} catch (cause: unknown) {
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[owner.path],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: "pull_adb_package", cause },
|
||||
);
|
||||
failures.push(
|
||||
analysisErrorWithCleanupFailure(
|
||||
owner.primary,
|
||||
cleanup,
|
||||
"pull_adb_package",
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (failures.length === 1) throw failures[0];
|
||||
if (failures.length > 1)
|
||||
throw new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[...this.#pendingPackageDirectoryCleanup].map(({ path }) => path),
|
||||
{
|
||||
reason:
|
||||
"Failed package output directories remain after cleanup retry",
|
||||
failures: failures.map(({ message, cleanupResources }) => ({
|
||||
message,
|
||||
resources: [...cleanupResources],
|
||||
})),
|
||||
},
|
||||
{
|
||||
operation: "pull_adb_package",
|
||||
cause: new AggregateError(
|
||||
failures,
|
||||
"Multiple ADB package rollback cleanups failed",
|
||||
),
|
||||
},
|
||||
);
|
||||
}
|
||||
/** The provider holds no long-lived resources; the adb server is caller-owned. */
|
||||
async close(): Promise<void> {}
|
||||
|
||||
async execute(
|
||||
request: AdbRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
if (this.#closing)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
let release!: () => void;
|
||||
const active = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
this.#activeOperations.add(active);
|
||||
try {
|
||||
if (this.#pendingPackageDirectoryCleanup.size > 0) {
|
||||
try {
|
||||
await this.#retryPackageDirectoryCleanup();
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
cause instanceof AnalysisError
|
||||
? cause
|
||||
: new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[...this.#pendingPackageDirectoryCleanup].map(
|
||||
({ path }) => path,
|
||||
),
|
||||
{
|
||||
reason:
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: request.operation, cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
return await this.#executeOperation(request, options);
|
||||
} finally {
|
||||
this.#activeOperations.delete(active);
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
async #executeOperation(
|
||||
request: AdbRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const signal = options?.signal;
|
||||
if (signal?.aborted === true)
|
||||
@@ -1177,11 +1010,9 @@ export class AdbProvider {
|
||||
);
|
||||
if (!directory.ok) return directory;
|
||||
const packageDirectory = join(directory.value, packageName);
|
||||
let packageDirectoryCreated = false;
|
||||
let packageDirectoryIdentity: PackageDirectoryIdentity | undefined;
|
||||
try {
|
||||
if (
|
||||
await lstat(packageDirectory).then(
|
||||
await stat(packageDirectory).then(
|
||||
() => true,
|
||||
() => false,
|
||||
)
|
||||
@@ -1190,44 +1021,14 @@ export class AdbProvider {
|
||||
`The package output directory already exists: ${packageDirectory}`,
|
||||
);
|
||||
await mkdir(packageDirectory);
|
||||
packageDirectoryCreated = true;
|
||||
const created = await lstat(packageDirectory);
|
||||
if (!created.isDirectory() || created.isSymbolicLink())
|
||||
throw new Error(
|
||||
`The created package output is not a regular directory: ${packageDirectory}`,
|
||||
);
|
||||
packageDirectoryIdentity = {
|
||||
dev: created.dev,
|
||||
ino: created.ino,
|
||||
mode: created.mode,
|
||||
};
|
||||
} catch (cause) {
|
||||
const primary = new AnalysisOutputError(
|
||||
operation,
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
{ cause },
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
cause instanceof Error ? cause.message : String(cause),
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
if (!packageDirectoryCreated) return err(primary);
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{
|
||||
reason: "Could not establish an owned package directory identity",
|
||||
},
|
||||
{ operation, cause },
|
||||
);
|
||||
return err(analysisErrorWithCleanupFailure(primary, cleanup, operation));
|
||||
}
|
||||
const artifacts: {
|
||||
device_path: string;
|
||||
@@ -1243,37 +1044,6 @@ export class AdbProvider {
|
||||
stage: "resolve_paths" | "pull" | "digest";
|
||||
message: string;
|
||||
}[] = [];
|
||||
const cancelledPackagePull = (
|
||||
error: AnalysisCancelledError,
|
||||
localPath: string,
|
||||
devicePath: string,
|
||||
stage: "pull" | "digest",
|
||||
) => {
|
||||
error.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial,
|
||||
package_name: packageName,
|
||||
output_directory: directory.value,
|
||||
artifacts,
|
||||
failures: [
|
||||
...failures,
|
||||
{
|
||||
device_path: devicePath,
|
||||
stage,
|
||||
message:
|
||||
stage === "pull"
|
||||
? `Pull cancelled; any incomplete file is retained at ${localPath}. Remaining APKs were not attempted.`
|
||||
: `Digesting the pulled file was cancelled; the file is retained at ${localPath}. Remaining APKs were not attempted.`,
|
||||
},
|
||||
],
|
||||
coverage: "partial",
|
||||
},
|
||||
});
|
||||
return err(error);
|
||||
};
|
||||
const seenFileNames = new Set<string>();
|
||||
for (const entry of parsed.paths) {
|
||||
if (seenFileNames.has(entry.file_name)) {
|
||||
@@ -1295,13 +1065,7 @@ export class AdbProvider {
|
||||
signal,
|
||||
);
|
||||
if (!pulled.ok) {
|
||||
if (pulled.error instanceof AnalysisCancelledError)
|
||||
return cancelledPackagePull(
|
||||
pulled.error,
|
||||
localPath,
|
||||
entry.device_path,
|
||||
"pull",
|
||||
);
|
||||
if (pulled.error instanceof AnalysisCancelledError) return pulled;
|
||||
failures.push({
|
||||
device_path: entry.device_path,
|
||||
stage: "pull",
|
||||
@@ -1310,24 +1074,18 @@ export class AdbProvider {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const digest = await hashAdbFile(localPath, operation, signal);
|
||||
const info = await stat(localPath);
|
||||
const sha256 = await sha256File(localPath);
|
||||
artifacts.push({
|
||||
device_path: entry.device_path,
|
||||
file_name: entry.file_name,
|
||||
local_path: localPath,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
role: entry.role,
|
||||
role_basis: "file_name",
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError)
|
||||
return cancelledPackagePull(
|
||||
cause,
|
||||
localPath,
|
||||
entry.device_path,
|
||||
"digest",
|
||||
);
|
||||
} catch {
|
||||
failures.push({
|
||||
device_path: entry.device_path,
|
||||
stage: "digest",
|
||||
@@ -1336,36 +1094,14 @@ export class AdbProvider {
|
||||
}
|
||||
}
|
||||
if (artifacts.length === 0 && failures.length > 0) {
|
||||
const primary = new AnalysisProtocolError(
|
||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||
await rm(packageDirectory, { recursive: true, force: true }).catch(
|
||||
() => undefined,
|
||||
);
|
||||
return err(
|
||||
new AnalysisProtocolError(
|
||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||
),
|
||||
);
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
try {
|
||||
await owner.remove();
|
||||
} catch (cause: unknown) {
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: operation, cause },
|
||||
);
|
||||
return err(
|
||||
analysisErrorWithCleanupFailure(primary, cleanup, operation),
|
||||
);
|
||||
}
|
||||
return err(primary);
|
||||
}
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
@@ -1420,7 +1156,8 @@ export class AdbProvider {
|
||||
);
|
||||
if (!pulled.ok) return pulled;
|
||||
try {
|
||||
const digest = await hashAdbFile(localPath, operation, signal);
|
||||
const info = await stat(localPath);
|
||||
const sha256 = await sha256File(localPath);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
@@ -1428,8 +1165,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
device_path: input.device_path,
|
||||
local_path: localPath,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
},
|
||||
provider,
|
||||
{
|
||||
@@ -1440,20 +1177,6 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) {
|
||||
cause.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
digest_status: "unknown",
|
||||
},
|
||||
});
|
||||
return err(cause);
|
||||
}
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
@@ -1488,13 +1211,7 @@ export class AdbProvider {
|
||||
cause: new Error(`The local file does not exist: ${localPath}`),
|
||||
}),
|
||||
);
|
||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
||||
try {
|
||||
digest = await hashAdbFile(localPath, operation, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
throw cause;
|
||||
}
|
||||
const sha256 = await sha256File(localPath);
|
||||
const existsProbe = await runAdb(
|
||||
operation,
|
||||
input.device_path,
|
||||
@@ -1540,7 +1257,7 @@ export class AdbProvider {
|
||||
);
|
||||
if (deviceDigest.ok) {
|
||||
const digestMatch = /^([a-f0-9]{64})\s/u.exec(deviceDigest.value.stdout);
|
||||
if (digestMatch !== null && digestMatch[1] !== digest.sha256)
|
||||
if (digestMatch !== null && digestMatch[1] !== sha256)
|
||||
return err(
|
||||
new AnalysisArtifactChangedError(
|
||||
operation,
|
||||
@@ -1555,8 +1272,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
device_sha256: digestMatch?.[1] ?? null,
|
||||
overwritten: deviceExists,
|
||||
device_digest_source: "device_sha256sum",
|
||||
@@ -1577,8 +1294,8 @@ export class AdbProvider {
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
device_path: input.device_path,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
device_sha256: null,
|
||||
overwritten: deviceExists,
|
||||
device_digest_source: "unavailable",
|
||||
@@ -1617,13 +1334,7 @@ export class AdbProvider {
|
||||
cause: new Error(`The local APK does not exist: ${apkPath}`),
|
||||
}),
|
||||
);
|
||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
||||
try {
|
||||
digest = await hashAdbFile(apkPath, operation, signal);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
throw cause;
|
||||
}
|
||||
const sha256 = await sha256File(apkPath);
|
||||
const installed = await runAdb(
|
||||
operation,
|
||||
input.serial,
|
||||
@@ -1652,8 +1363,8 @@ export class AdbProvider {
|
||||
client,
|
||||
serial: input.serial,
|
||||
apk_path: apkPath,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
replaced: input.replace,
|
||||
},
|
||||
provider,
|
||||
@@ -2023,15 +1734,16 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
try {
|
||||
const digest = await hashAdbFile(reportedPath, operation, signal);
|
||||
const info = await stat(reportedPath);
|
||||
const sha256 = await sha256File(reportedPath);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: reportedPath,
|
||||
bytes: digest.bytes,
|
||||
sha256: digest.sha256,
|
||||
bytes: info.size,
|
||||
sha256,
|
||||
adb_reported_path: reportedPath,
|
||||
},
|
||||
provider,
|
||||
@@ -2043,20 +1755,6 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) {
|
||||
cause.retainPartialObservation({
|
||||
provider_id: "adb",
|
||||
operation,
|
||||
result: {
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: reportedPath,
|
||||
adb_reported_path: reportedPath,
|
||||
digest_status: "unknown",
|
||||
},
|
||||
});
|
||||
return err(cause);
|
||||
}
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
|
||||
@@ -1,43 +1,40 @@
|
||||
import { constants, createReadStream } from "node:fs";
|
||||
import { chmod, copyFile } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { ReadStream, Stats, WriteStream } from "node:fs";
|
||||
import { chmod, open, rm, stat, type FileHandle } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { Transform, type TransformCallback } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
||||
|
||||
/** Stream a local file digest without retaining the APK or JAR in memory. */
|
||||
export const hashAndroidFile = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
|
||||
/** Bind decompiler input to the admitted artifact, even if the original changes. */
|
||||
export const snapshotAndroidTarget = async (
|
||||
path: string,
|
||||
sha256: string,
|
||||
root: string,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const snapshot = join(root, "target.apk");
|
||||
try {
|
||||
await copyAndroidSnapshot(path, snapshot, sha256, signal);
|
||||
signal?.throwIfAborted();
|
||||
await chmod(snapshot, 0o400);
|
||||
return snapshot;
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
||||
throw new AnalysisInputError(operation, { cause }, [
|
||||
await copyFile(path, snapshot, constants.COPYFILE_EXCL);
|
||||
await chmod(snapshot, 0o400);
|
||||
if ((await hashAndroidFile(snapshot)) !== sha256)
|
||||
throw new AnalysisInputError(operation, undefined, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
||||
message:
|
||||
"APK bytes changed after admission; retry against a stable file.",
|
||||
},
|
||||
]);
|
||||
}
|
||||
return snapshot;
|
||||
};
|
||||
|
||||
/** Fingerprint the actual immutable engine bytes executed for this observation. */
|
||||
@@ -46,120 +43,15 @@ export const snapshotAndroidEngine = async (
|
||||
sha256: string,
|
||||
root: string,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ path: string; sha256: string }> => {
|
||||
const path = join(root, "engine.jar");
|
||||
try {
|
||||
const observed = await copyAndroidSnapshot(source, path, sha256, signal);
|
||||
signal?.throwIfAborted();
|
||||
await chmod(path, 0o400);
|
||||
return { path, sha256: observed };
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
||||
await copyFile(source, path, constants.COPYFILE_EXCL);
|
||||
await chmod(path, 0o400);
|
||||
if ((await hashAndroidFile(path)) !== sha256)
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
`REA_JADX_MCP_JAR bytes changed during admission at ${source}; retry with a stable engine file.`,
|
||||
{ cause },
|
||||
"REA_JADX_MCP_JAR bytes changed during admission; retry with a stable engine file.",
|
||||
);
|
||||
}
|
||||
return { path, sha256 };
|
||||
};
|
||||
|
||||
const copyAndroidSnapshot = async (
|
||||
sourcePath: string,
|
||||
snapshotPath: string,
|
||||
expectedSha256: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
signal?.throwIfAborted();
|
||||
const source = await openRegularFile(sourcePath, {
|
||||
symlinks: "follow",
|
||||
signal,
|
||||
});
|
||||
let destination: FileHandle | undefined;
|
||||
let inputStream: ReadStream | undefined;
|
||||
let outputStream: WriteStream | undefined;
|
||||
let removeSnapshot = false;
|
||||
let copiedBytes = 0;
|
||||
const hash = createHash("sha256");
|
||||
try {
|
||||
const initial = await source.stat();
|
||||
signal?.throwIfAborted();
|
||||
const digest = new Transform({
|
||||
transform(
|
||||
chunk: Buffer,
|
||||
_encoding: BufferEncoding,
|
||||
callback: TransformCallback,
|
||||
) {
|
||||
copiedBytes += chunk.length;
|
||||
if (copiedBytes > initial.size) {
|
||||
callback(
|
||||
new AndroidSnapshotChangedError(
|
||||
`Android input grew while being copied: ${sourcePath}`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
hash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
try {
|
||||
destination = await open(snapshotPath, "wx", 0o600);
|
||||
signal?.throwIfAborted();
|
||||
inputStream = source.createReadStream({
|
||||
start: 0,
|
||||
end: initial.size,
|
||||
autoClose: false,
|
||||
});
|
||||
outputStream = destination.createWriteStream({ autoClose: false });
|
||||
await pipeline(inputStream, digest, outputStream, { signal });
|
||||
signal?.throwIfAborted();
|
||||
await verifySourceState(source, sourcePath, initial, copiedBytes);
|
||||
const observedSha256 = hash.digest("hex");
|
||||
if (observedSha256 !== expectedSha256)
|
||||
throw new AndroidSnapshotChangedError(
|
||||
`Android snapshot digest mismatch for ${sourcePath}`,
|
||||
);
|
||||
return observedSha256;
|
||||
} catch (cause: unknown) {
|
||||
removeSnapshot = destination !== undefined;
|
||||
throw cause;
|
||||
}
|
||||
} finally {
|
||||
// Non-auto-closing streams retain a reference to their FileHandle even
|
||||
// after pipeline completion. Release those references before awaiting
|
||||
// descriptor closure, after the admitted source state has been verified.
|
||||
inputStream?.destroy();
|
||||
outputStream?.destroy();
|
||||
try {
|
||||
await Promise.all([source.close(), destination?.close()]);
|
||||
} finally {
|
||||
if (removeSnapshot)
|
||||
await rm(snapshotPath, { force: true }).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const verifySourceState = async (
|
||||
handle: Awaited<ReturnType<typeof openRegularFile>>,
|
||||
path: string,
|
||||
initial: Stats,
|
||||
observedBytes: number,
|
||||
): Promise<void> => {
|
||||
const [opened, currentPath] = await Promise.all([handle.stat(), stat(path)]);
|
||||
if (
|
||||
observedBytes !== initial.size ||
|
||||
!sameRegularFileState(initial, opened) ||
|
||||
!sameRegularFileState(initial, currentPath)
|
||||
)
|
||||
throw new AndroidSnapshotChangedError(
|
||||
`Android input changed while being read: ${path}`,
|
||||
);
|
||||
};
|
||||
|
||||
class AndroidSnapshotChangedError extends Error {
|
||||
constructor(message: string, options?: ErrorOptions) {
|
||||
super(message, options);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,11 +52,7 @@ export const inspectJadxAvailability = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<ProviderAvailability> => {
|
||||
try {
|
||||
const configuration = await readJadxConfigurationInputs(
|
||||
environment,
|
||||
signal,
|
||||
);
|
||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
||||
const configuration = await readJadxConfiguration(environment, signal);
|
||||
return {
|
||||
status: "available",
|
||||
code: null,
|
||||
@@ -81,83 +77,52 @@ export const inspectJadxAvailability = async (
|
||||
}
|
||||
};
|
||||
|
||||
/** Resolve caller-selected paths and JVM options without probing their contents. */
|
||||
export const resolveJadxConfigurationInputs = async (
|
||||
/** Resolve caller-selected tools and preserve the same prerequisite diagnostics. */
|
||||
export const resolveJadxConfiguration = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JadxConfiguration> => {
|
||||
try {
|
||||
return await readJadxConfigurationInputs(environment, signal);
|
||||
return await readJadxConfiguration(environment, signal);
|
||||
} catch (cause) {
|
||||
if (signal?.aborted === true) throw cause;
|
||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
||||
throw projectConfigurationFailure(operation, cause);
|
||||
const output = cause.diagnostics;
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
cause.message,
|
||||
{
|
||||
cause,
|
||||
userMessage: cause.message,
|
||||
...(typeof output.stdout === "string" &&
|
||||
typeof output.stderr === "string"
|
||||
? {
|
||||
capturedOutput: {
|
||||
stdout: output.stdout,
|
||||
stderr: output.stderr,
|
||||
truncated: output.output_truncated === true,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** Probe Java and archive prerequisites before starting a new engine session. */
|
||||
export const validateJadxConfiguration = async (
|
||||
configuration: JadxConfiguration,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
operation: AndroidOperation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
||||
} catch (cause) {
|
||||
if (signal?.aborted === true) throw cause;
|
||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
||||
throw projectConfigurationFailure(operation, cause);
|
||||
}
|
||||
};
|
||||
|
||||
const readJadxConfigurationInputs = async (
|
||||
const readJadxConfiguration = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JadxConfiguration> => {
|
||||
signal?.throwIfAborted();
|
||||
requireSupportedHost();
|
||||
const jar = await resolveJar(environment);
|
||||
const java = await resolveJava(environment, signal);
|
||||
await requireJavaHomeExecutable(environment.JAVA_HOME, java);
|
||||
signal?.throwIfAborted();
|
||||
return { jar, java, jvmArguments: resolveJvmArguments(environment) };
|
||||
};
|
||||
|
||||
const validateJadxConfigurationContents = async (
|
||||
configuration: JadxConfiguration,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
signal?.throwIfAborted();
|
||||
await inspectJavaRuntime(configuration.java, environment, signal);
|
||||
await requireEngineArchive(configuration.jar, signal);
|
||||
};
|
||||
|
||||
const projectConfigurationFailure = (
|
||||
operation: AndroidOperation,
|
||||
cause: JadxConfigurationFailure,
|
||||
): AnalysisCapabilityUnavailableError => {
|
||||
const output = cause.diagnostics;
|
||||
return new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
cause.message,
|
||||
{
|
||||
cause,
|
||||
userMessage: cause.message,
|
||||
...(typeof output.stdout === "string" && typeof output.stderr === "string"
|
||||
? {
|
||||
capturedOutput: {
|
||||
stdout: output.stdout,
|
||||
stderr: output.stderr,
|
||||
truncated: output.output_truncated === true,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
const jvmArguments = resolveJvmArguments(environment);
|
||||
await inspectJavaRuntime(java, environment, signal);
|
||||
await requireEngineArchive(jar, signal);
|
||||
return { jar, java, jvmArguments };
|
||||
};
|
||||
|
||||
const requireEngineArchive = async (
|
||||
|
||||
+28
-98
@@ -21,14 +21,12 @@ import { err, ok, type Result } from "../domain/result.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
snapshotAndroidEngine,
|
||||
hashAndroidFile,
|
||||
snapshotAndroidTarget,
|
||||
} from "./AndroidTargetSnapshot.js";
|
||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
||||
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
inspectJadxAvailability,
|
||||
resolveJadxConfigurationInputs,
|
||||
validateJadxConfiguration,
|
||||
resolveJadxConfiguration,
|
||||
} from "./JadxConfiguration.js";
|
||||
import type { JadxLauncher } from "./JadxMcpTransport.js";
|
||||
|
||||
@@ -106,50 +104,6 @@ const executionError = (context: {
|
||||
});
|
||||
};
|
||||
|
||||
const androidTargetChanged = (
|
||||
operation: string,
|
||||
path: string,
|
||||
cause?: unknown,
|
||||
): AnalysisInputError =>
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
cause === undefined ? undefined : { cause },
|
||||
[
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
const androidSourceChanged = (
|
||||
operation: string,
|
||||
source: string,
|
||||
cause?: unknown,
|
||||
): AnalysisCapabilityUnavailableError =>
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
operation,
|
||||
`Selected JADX input bytes changed during admission at ${source}; retry with a stable installation.`,
|
||||
cause === undefined ? undefined : { cause },
|
||||
);
|
||||
|
||||
const hashAndroidSource = async (
|
||||
path: string,
|
||||
operation: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<string> => {
|
||||
try {
|
||||
return (await hashStableFile(path, signal)).sha256;
|
||||
} catch (cause: unknown) {
|
||||
if (signal.aborted) throw cause;
|
||||
if (cause instanceof RegularFileChangedError)
|
||||
throw androidSourceChanged(operation, path, cause);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
interface PendingCleanup {
|
||||
session: JadxSession | undefined;
|
||||
root: PrivateRuntimeRoot | undefined;
|
||||
@@ -193,18 +147,17 @@ const cleanup = async (
|
||||
await root?.close();
|
||||
resources.root = undefined;
|
||||
} catch (cause) {
|
||||
const error = new ProviderCleanupError(
|
||||
"jadx",
|
||||
[root?.path ?? "unknown"],
|
||||
{
|
||||
...diagnostics,
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause: previousError ?? cause },
|
||||
return err(
|
||||
new ProviderCleanupError(
|
||||
"jadx",
|
||||
[root?.path ?? "unknown"],
|
||||
{
|
||||
...diagnostics,
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause: previousError ?? cause },
|
||||
),
|
||||
);
|
||||
if (previousError?.partialObservation !== undefined)
|
||||
error.retainPartialObservation(previousError.partialObservation);
|
||||
return err(error);
|
||||
}
|
||||
return ok(undefined);
|
||||
};
|
||||
@@ -334,33 +287,23 @@ export class JadxProvider implements AndroidAnalysisPort {
|
||||
request.operation,
|
||||
`Target ${target.path} is ${target.format}; provide one standalone APK.`,
|
||||
);
|
||||
const configuration = await resolveJadxConfigurationInputs(
|
||||
const configuration = await resolveJadxConfiguration(
|
||||
this.environment,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
signal.throwIfAborted();
|
||||
const jarHash = await hashAndroidSource(
|
||||
configuration.jar,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const bridgeHash = await hashAndroidSource(
|
||||
BRIDGE_SOURCE,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
let targetHash: string;
|
||||
try {
|
||||
targetHash = (await hashStableFile(target.path, signal)).sha256;
|
||||
} catch (cause: unknown) {
|
||||
if (signal.aborted) throw cause;
|
||||
if (cause instanceof RegularFileChangedError)
|
||||
throw androidTargetChanged(request.operation, target.path, cause);
|
||||
throw cause;
|
||||
}
|
||||
if (targetHash !== target.sha256)
|
||||
throw androidTargetChanged(request.operation, target.path);
|
||||
const jarHash = await hashAndroidFile(configuration.jar);
|
||||
const bridgeHash = await hashAndroidFile(BRIDGE_SOURCE);
|
||||
if ((await hashAndroidFile(target.path)) !== target.sha256)
|
||||
throw new AnalysisInputError(request.operation, undefined, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"APK bytes changed after admission; retry against a stable file.",
|
||||
},
|
||||
]);
|
||||
const key = JSON.stringify({
|
||||
path: target.path,
|
||||
sha256: target.sha256,
|
||||
@@ -383,40 +326,27 @@ export class JadxProvider implements AndroidAnalysisPort {
|
||||
}
|
||||
let retained = this.#retained;
|
||||
if (retained === undefined) {
|
||||
await validateJadxConfiguration(
|
||||
configuration,
|
||||
this.environment,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" });
|
||||
const engine = await snapshotAndroidEngine(
|
||||
configuration.jar,
|
||||
jarHash,
|
||||
root.path,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const snapshot = await snapshotAndroidTarget(
|
||||
target.path,
|
||||
target.sha256,
|
||||
root.path,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
const bridge = join(root.path, "ReaJadxBridge.java");
|
||||
await copyFile(BRIDGE_SOURCE, bridge);
|
||||
await chmod(bridge, 0o400);
|
||||
const copiedBridgeHash = await hashAndroidSource(
|
||||
bridge,
|
||||
request.operation,
|
||||
signal,
|
||||
);
|
||||
if (copiedBridgeHash !== bridgeHash)
|
||||
throw androidSourceChanged(
|
||||
if ((await hashAndroidFile(bridge)) !== bridgeHash)
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
"jadx",
|
||||
request.operation,
|
||||
BRIDGE_SOURCE,
|
||||
undefined,
|
||||
"REA Android bridge bytes changed during admission; retry with a stable installation.",
|
||||
);
|
||||
signal.throwIfAborted();
|
||||
session = new JadxSession(
|
||||
|
||||
@@ -78,14 +78,12 @@ export const parseApktoolVersionOutput = (
|
||||
export const inspectApktoolClient = async (
|
||||
selection: ApktoolCommandSelection,
|
||||
signal?: AbortSignal,
|
||||
environment: Readonly<Record<string, string | undefined>> = process.env,
|
||||
): Promise<ApktoolClientIdentity> => {
|
||||
try {
|
||||
const { stdout } = await execFileOutput(selection.command, ["--version"], {
|
||||
timeout: VERSION_TIMEOUT_MS,
|
||||
stopSignal: "SIGTERM",
|
||||
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
|
||||
env: { ...process.env, ...environment },
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
return parseApktoolVersionOutput(stdout);
|
||||
|
||||
+130
-324
@@ -1,20 +1,18 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { createWriteStream, constants as fsConstants } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import { createReadStream, constants as fsConstants } from "node:fs";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
opendir,
|
||||
open,
|
||||
readdir,
|
||||
rm,
|
||||
stat,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { Transform, type TransformCallback } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
import type {
|
||||
ExecutionOptions,
|
||||
@@ -35,19 +33,11 @@ import {
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../process/OwnedCommand.js";
|
||||
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
|
||||
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
|
||||
execFileOutput,
|
||||
execFileOutputFailure,
|
||||
} from "../process/ExecFileOutput.js";
|
||||
import {
|
||||
ApktoolConfigurationFailure,
|
||||
apktoolConfigurationAvailability,
|
||||
@@ -74,22 +64,20 @@ const DECODE_MAX_OUTPUT_BYTES = 4 * 1024 * 1024;
|
||||
const MANIFEST_MAX_BYTES = 512 * 1024;
|
||||
const STRINGS_XML_MAX_BYTES = 8 * 1024 * 1024;
|
||||
const MAX_STRINGS = 2_000;
|
||||
const MAX_DECODED_ENTRIES_VISITED = 100_000;
|
||||
const MAX_DECODED_FILES_COUNTED = 100_000;
|
||||
const STDERR_EXCERPT_BYTES = 512;
|
||||
|
||||
const throwIfCancelled = (
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): void => {
|
||||
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
|
||||
/** Stream a local file digest without retaining large APKs in memory. */
|
||||
const sha256File = async (path: string): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
|
||||
/** Read at most the byte budget plus one byte, before decoding UTF-8. */
|
||||
const readBoundedText = async (
|
||||
path: string,
|
||||
budget: number,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<{ text: string | null; truncated: boolean }> => {
|
||||
const file = await open(path, "r").catch((cause: unknown) => {
|
||||
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return null;
|
||||
@@ -97,11 +85,9 @@ const readBoundedText = async (
|
||||
});
|
||||
if (file === null) return { text: null, truncated: false };
|
||||
try {
|
||||
throwIfCancelled(operation, signal);
|
||||
const bytes = Buffer.alloc(budget + 1);
|
||||
let length = 0;
|
||||
while (length < bytes.length) {
|
||||
throwIfCancelled(operation, signal);
|
||||
const result = await file.read(
|
||||
bytes,
|
||||
length,
|
||||
@@ -111,7 +97,6 @@ const readBoundedText = async (
|
||||
if (result.bytesRead === 0) break;
|
||||
length += result.bytesRead;
|
||||
}
|
||||
throwIfCancelled(operation, signal);
|
||||
const truncated = length > budget;
|
||||
// A prefix may end within a UTF-8 character; omit those trailing bytes.
|
||||
const decoder = new TextDecoder("utf-8");
|
||||
@@ -140,141 +125,97 @@ const runApktool = async (
|
||||
arguments_: readonly string[],
|
||||
limits: DecodeLimits,
|
||||
signal: AbortSignal | undefined,
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
onCleanupOwner: (owner: ProviderProcessSupervisor) => void,
|
||||
): Promise<Result<{ stdout: string; stderr: string }, AnalysisError>> => {
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
try {
|
||||
const output = await runOwnedCommand(
|
||||
{
|
||||
command,
|
||||
arguments: arguments_,
|
||||
runId: randomUUID(),
|
||||
hostEnvironment: { ...process.env, ...environment },
|
||||
},
|
||||
{
|
||||
timeoutMs: limits.timeoutMs,
|
||||
diagnosticBytes: limits.maxOutputBytes,
|
||||
},
|
||||
signal === undefined ? {} : { signal },
|
||||
return ok(
|
||||
await execFileOutput(command, arguments_, {
|
||||
timeout: limits.timeoutMs,
|
||||
maxBuffer: limits.maxOutputBytes,
|
||||
stopSignal: "SIGTERM",
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
}),
|
||||
);
|
||||
return ok({ stdout: output.stdout.text, stderr: output.stderr.text });
|
||||
} catch (cause) {
|
||||
if (!(cause instanceof OwnedCommandFailure))
|
||||
if (signal?.aborted || execFileOutputFailure(cause)?.code === "ABORT_ERR")
|
||||
return err(new AnalysisCancelledError(operation, { cause }));
|
||||
const failure = execFileOutputFailure(cause);
|
||||
if (failure === undefined)
|
||||
return err(
|
||||
new AnalysisProtocolError(`apktool failed for ${operation}`, { cause }),
|
||||
);
|
||||
if (cause.cleanupOwner !== undefined) onCleanupOwner(cause.cleanupOwner);
|
||||
const output = cause.snapshot;
|
||||
const capturedOutput =
|
||||
output === null
|
||||
? undefined
|
||||
: {
|
||||
stdout: output.stdout.text,
|
||||
stderr: output.stderr.text,
|
||||
truncated: output.diagnosticTruncated === true,
|
||||
stdout_bytes: output.stdout.bytes,
|
||||
stderr_bytes: output.stderr.bytes,
|
||||
exit_code: output.exitCode ?? null,
|
||||
signal: output.signal ?? null,
|
||||
};
|
||||
const cleanup =
|
||||
cause.cleanupFailure === null
|
||||
? undefined
|
||||
: {
|
||||
reason: cause.cleanupFailure,
|
||||
resources: cause.resources,
|
||||
};
|
||||
if (cause.reason === "cancelled")
|
||||
return err(
|
||||
new AnalysisCancelledError(operation, {
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
}),
|
||||
);
|
||||
if (cause.reason === "timeout")
|
||||
return err(
|
||||
new AnalysisTimeoutError(operation, limits.timeoutMs, {
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
}),
|
||||
);
|
||||
if (cause.reason === "output-limit")
|
||||
if (failure.outputTruncated)
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
`apktool output exceeded the ${String(limits.maxOutputBytes)}-byte capture budget`,
|
||||
{ max_output_bytes: limits.maxOutputBytes },
|
||||
{
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
},
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
const excerpt = failure.stderr.trim().slice(0, STDERR_EXCERPT_BYTES);
|
||||
if (failure.killed)
|
||||
return err(
|
||||
new AnalysisTimeoutError(operation, limits.timeoutMs, { cause }),
|
||||
);
|
||||
return err(
|
||||
new AnalysisUnsupportedTargetError(
|
||||
operation,
|
||||
command,
|
||||
output?.stderr.text.trim().slice(0, STDERR_EXCERPT_BYTES) ||
|
||||
`apktool exited with code ${String(output?.exitCode ?? "unknown")} for ${operation}`,
|
||||
{
|
||||
cause,
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(capturedOutput === undefined ? {} : { capturedOutput }),
|
||||
},
|
||||
excerpt === ""
|
||||
? `apktool exited with code ${String(failure.code)} for ${operation}`
|
||||
: excerpt,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** Count files while bounding streamed directory-entry traversal. */
|
||||
/** Count files in the decoded workspace with a hard ceiling. */
|
||||
const countDecodedFiles = async (
|
||||
root: string,
|
||||
signal: AbortSignal | undefined,
|
||||
operation: string,
|
||||
): Promise<{
|
||||
count: number;
|
||||
capped: boolean;
|
||||
locales: readonly string[];
|
||||
}> => {
|
||||
): Promise<{ count: number; capped: boolean }> => {
|
||||
let count = 0;
|
||||
let capped = false;
|
||||
let visited = 0;
|
||||
const directories = [root];
|
||||
const locales: string[] = [];
|
||||
while (directories.length > 0 && !capped) {
|
||||
throwIfCancelled(operation, signal);
|
||||
const directory = directories.pop()!;
|
||||
for await (const entry of await opendir(directory)) {
|
||||
throwIfCancelled(operation, signal);
|
||||
if (visited >= MAX_DECODED_ENTRIES_VISITED) {
|
||||
const walk = async (directory: string): Promise<void> => {
|
||||
const entries = await readdir(directory, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
if (count >= MAX_DECODED_FILES_COUNTED) {
|
||||
capped = true;
|
||||
break;
|
||||
return;
|
||||
}
|
||||
visited += 1;
|
||||
const path = join(directory, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
directories.push(path);
|
||||
if (directory === join(root, "res")) {
|
||||
const match = /^values-(.+)$/u.exec(entry.name);
|
||||
if (
|
||||
match !== null &&
|
||||
(await access(join(path, "strings.xml")).then(
|
||||
() => true,
|
||||
() => false,
|
||||
))
|
||||
)
|
||||
locales.push(match[1]!);
|
||||
}
|
||||
} else if (entry.isFile()) count += 1;
|
||||
if (entry.isDirectory()) await walk(path);
|
||||
else if (entry.isFile()) count += 1;
|
||||
}
|
||||
};
|
||||
await walk(root);
|
||||
return { count, capped };
|
||||
};
|
||||
|
||||
/** Locale codes present as res/values-<locale>/strings.xml. */
|
||||
const listStringLocales = async (
|
||||
workspace: string,
|
||||
): Promise<readonly string[]> => {
|
||||
const res = join(workspace, "res");
|
||||
const entries = await readdir(res, { withFileTypes: true }).catch(() => []);
|
||||
const locales: string[] = [];
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const match = /^values-(.+)$/u.exec(entry.name);
|
||||
if (match === null) continue;
|
||||
const strings = join(res, entry.name, "strings.xml");
|
||||
if (
|
||||
await access(strings).then(
|
||||
() => true,
|
||||
() => false,
|
||||
)
|
||||
)
|
||||
locales.push(match[1]!);
|
||||
}
|
||||
return { count, capped, locales: locales.sort() };
|
||||
return locales.sort();
|
||||
};
|
||||
|
||||
export interface ApktoolProviderOptions {
|
||||
@@ -294,14 +235,9 @@ export class ApktoolProvider {
|
||||
|
||||
private readonly workspaces = new Set<string>();
|
||||
private readonly removeWorkspace: (path: string) => Promise<void>;
|
||||
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
|
||||
readonly #shutdown = new AbortController();
|
||||
readonly #active = new Set<Promise<void>>();
|
||||
#closing = false;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
|
||||
constructor(options: ApktoolProviderOptions = {}) {
|
||||
this.environment = snapshotEnvironment(options.environment ?? process.env);
|
||||
this.environment = options.environment ?? process.env;
|
||||
this.removeWorkspace =
|
||||
options.removeWorkspace ??
|
||||
((path) => rm(path, { recursive: true, force: true }));
|
||||
@@ -312,25 +248,12 @@ export class ApktoolProvider {
|
||||
}
|
||||
|
||||
/** Probe the selected apktool launcher; decodes nothing. */
|
||||
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability> {
|
||||
if (this.#closing)
|
||||
return Promise.reject(
|
||||
new AnalysisCancelledError("inspect_apktool_client"),
|
||||
);
|
||||
const operationSignal =
|
||||
signal === undefined
|
||||
? this.#shutdown.signal
|
||||
: AbortSignal.any([signal, this.#shutdown.signal]);
|
||||
return this.#track(this.#inspectAvailability(operationSignal));
|
||||
}
|
||||
|
||||
async #inspectAvailability(
|
||||
signal: AbortSignal,
|
||||
async inspectAvailability(
|
||||
signal?: AbortSignal,
|
||||
): Promise<ProviderAvailability> {
|
||||
const selection = this.selection();
|
||||
try {
|
||||
await inspectApktoolClient(selection, signal, this.environment);
|
||||
throwIfCancelled("inspect_apktool_client", signal);
|
||||
await inspectApktoolClient(selection, signal);
|
||||
return {
|
||||
status: "available",
|
||||
code: null,
|
||||
@@ -349,44 +272,30 @@ export class ApktoolProvider {
|
||||
|
||||
/** Retain failed cleanup ownership and retry it on close. */
|
||||
async close(): Promise<void> {
|
||||
this.#closing = true;
|
||||
this.#shutdown.abort();
|
||||
this.#closePromise ??= this.#drainAndClean().catch((cause: unknown) => {
|
||||
this.#closePromise = undefined;
|
||||
throw cause;
|
||||
});
|
||||
return this.#closePromise;
|
||||
const failures: unknown[] = [];
|
||||
for (const path of this.workspaces) {
|
||||
try {
|
||||
await this.removeWorkspace(path);
|
||||
this.workspaces.delete(path);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool workspace cleanup failed");
|
||||
}
|
||||
|
||||
execute(
|
||||
async execute(
|
||||
request: ApktoolRequest,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
if (this.#closing)
|
||||
return Promise.resolve(
|
||||
err(new AnalysisCancelledError(request.operation)),
|
||||
);
|
||||
const signal =
|
||||
options?.signal === undefined
|
||||
? this.#shutdown.signal
|
||||
: AbortSignal.any([options.signal, this.#shutdown.signal]);
|
||||
return this.#track(this.#execute(request, signal));
|
||||
}
|
||||
|
||||
async #execute(
|
||||
request: ApktoolRequest,
|
||||
signal: AbortSignal,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const signal = options?.signal;
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
const selection = this.selection();
|
||||
let identity: ApktoolClientIdentity;
|
||||
try {
|
||||
identity = await inspectApktoolClient(
|
||||
selection,
|
||||
signal,
|
||||
this.environment,
|
||||
);
|
||||
identity = await inspectApktoolClient(selection, signal);
|
||||
} catch (cause) {
|
||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
||||
if (cause instanceof ApktoolConfigurationFailure)
|
||||
@@ -407,8 +316,6 @@ export class ApktoolProvider {
|
||||
),
|
||||
);
|
||||
}
|
||||
if (signal.aborted)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
const client = {
|
||||
command: selection.command,
|
||||
command_source: selection.commandSource,
|
||||
@@ -430,47 +337,6 @@ export class ApktoolProvider {
|
||||
);
|
||||
}
|
||||
|
||||
#track<T>(operation: Promise<T>): Promise<T> {
|
||||
const settled = operation.then(
|
||||
() => undefined,
|
||||
() => undefined,
|
||||
);
|
||||
this.#active.add(settled);
|
||||
void settled.then(() => this.#active.delete(settled));
|
||||
return operation;
|
||||
}
|
||||
|
||||
async #drainAndClean(): Promise<void> {
|
||||
await Promise.all(this.#active);
|
||||
const failures: unknown[] = [];
|
||||
for (const [owner, root] of this.#pendingCommandOwners) {
|
||||
try {
|
||||
const stopped = await owner.stop();
|
||||
if (stopped.status === "incomplete") throw new Error(stopped.reason);
|
||||
this.#pendingCommandOwners.delete(owner);
|
||||
} catch (cause) {
|
||||
failures.push(
|
||||
new Error(
|
||||
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool process cleanup failed");
|
||||
for (const path of this.workspaces) {
|
||||
try {
|
||||
await this.removeWorkspace(path);
|
||||
this.workspaces.delete(path);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Apktool workspace cleanup failed");
|
||||
}
|
||||
|
||||
private async decodeResources(
|
||||
operation: "decode_android_resources",
|
||||
selection: ApktoolCommandSelection,
|
||||
@@ -487,6 +353,28 @@ export class ApktoolProvider {
|
||||
},
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||
const info = await stat(input.path).then(
|
||||
(value) => value,
|
||||
() => null,
|
||||
);
|
||||
if (info === null || !info.isFile())
|
||||
return err(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK is not a readable file: ${input.path}`,
|
||||
),
|
||||
}),
|
||||
);
|
||||
const readable = await access(input.path, fsConstants.R_OK).then(
|
||||
() => true,
|
||||
() => false,
|
||||
);
|
||||
if (!readable)
|
||||
return err(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(`The selected APK is not readable: ${input.path}`),
|
||||
}),
|
||||
);
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-apktool-"));
|
||||
this.workspaces.add(root);
|
||||
const workspace = join(root, "decoded");
|
||||
@@ -496,10 +384,6 @@ export class ApktoolProvider {
|
||||
// The workspace is removed before the result is returned, so an
|
||||
// unremovable workspace is reported instead of silently abandoned.
|
||||
const removeWorkspace = async (): Promise<void> => {
|
||||
if ([...this.#pendingCommandOwners.values()].includes(root)) {
|
||||
cleanupFailure = `The decode workspace is retained until its apktool process stops: ${root}`;
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await this.removeWorkspace(root);
|
||||
this.workspaces.delete(root);
|
||||
@@ -510,78 +394,20 @@ export class ApktoolProvider {
|
||||
const failureWithCleanup = (failure: AnalysisError): AnalysisError =>
|
||||
cleanupFailure === null
|
||||
? failure
|
||||
: analysisErrorWithCleanupFailure(
|
||||
failure,
|
||||
new ProviderCleanupError(
|
||||
APKTOOL_PROVIDER_IDENTITY.id,
|
||||
[root],
|
||||
{ reason: cleanupFailure },
|
||||
{ operation },
|
||||
),
|
||||
operation,
|
||||
);
|
||||
let source: FileHandle | undefined;
|
||||
let sourceAdmissionFailed = false;
|
||||
: failure instanceof AnalysisCancelledError
|
||||
? new AnalysisCancelledError(operation, {
|
||||
cause: failure,
|
||||
cleanup: { reason: cleanupFailure, resources: [root] },
|
||||
})
|
||||
: new AnalysisProtocolError(failure.message, {
|
||||
cause: failure,
|
||||
cleanup: { reason: cleanupFailure, resources: [root] },
|
||||
});
|
||||
try {
|
||||
throwIfCancelled(operation, signal);
|
||||
sourceAdmissionFailed = true;
|
||||
source = await openRegularFile(input.path, {
|
||||
symlinks: "follow",
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
const sourceInfo = await source.stat();
|
||||
sourceAdmissionFailed = false;
|
||||
const sourceSize = sourceInfo.size;
|
||||
let copiedBytes = 0;
|
||||
const sourceHash = createHash("sha256");
|
||||
const boundedSource = new Transform({
|
||||
transform(
|
||||
chunk: Buffer,
|
||||
_encoding: BufferEncoding,
|
||||
callback: TransformCallback,
|
||||
) {
|
||||
copiedBytes += chunk.length;
|
||||
if (copiedBytes > sourceSize) {
|
||||
callback(
|
||||
new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK changed while being copied: ${input.path}`,
|
||||
),
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
sourceHash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
await pipeline(
|
||||
source.createReadStream({
|
||||
start: 0,
|
||||
end: sourceSize,
|
||||
autoClose: false,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
}),
|
||||
boundedSource,
|
||||
createWriteStream(snapshot, { flags: "wx", mode: 0o600 }),
|
||||
signal === undefined ? {} : { signal },
|
||||
);
|
||||
const finalSourceInfo = await source.stat();
|
||||
if (
|
||||
copiedBytes !== sourceSize ||
|
||||
!sameRegularFileState(sourceInfo, finalSourceInfo)
|
||||
)
|
||||
throw new AnalysisInputError(operation, {
|
||||
cause: new Error(
|
||||
`The selected APK changed while being copied: ${input.path}`,
|
||||
),
|
||||
});
|
||||
const sha256 = sourceHash.digest("hex");
|
||||
await source.close();
|
||||
source = undefined;
|
||||
await copyFile(input.path, snapshot, fsConstants.COPYFILE_EXCL);
|
||||
await chmod(snapshot, 0o400);
|
||||
throwIfCancelled(operation, signal);
|
||||
const snapshotInfo = await stat(snapshot);
|
||||
const sha256 = await sha256File(snapshot);
|
||||
await mkdir(workspace);
|
||||
await mkdir(framework);
|
||||
const decoded = await runApktool(
|
||||
@@ -602,19 +428,14 @@ export class ApktoolProvider {
|
||||
maxOutputBytes: DECODE_MAX_OUTPUT_BYTES,
|
||||
},
|
||||
signal,
|
||||
this.environment,
|
||||
(owner) => this.#pendingCommandOwners.set(owner, root),
|
||||
);
|
||||
if (!decoded.ok) {
|
||||
await removeWorkspace();
|
||||
return err(failureWithCleanup(decoded.error));
|
||||
}
|
||||
throwIfCancelled(operation, signal);
|
||||
const yml = await readBoundedText(
|
||||
join(workspace, "apktool.yml"),
|
||||
MANIFEST_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
if (yml.text === null || yml.truncated) {
|
||||
throw new AnalysisProtocolError(
|
||||
@@ -625,12 +446,11 @@ export class ApktoolProvider {
|
||||
const manifestRead = await readBoundedText(
|
||||
join(workspace, "AndroidManifest.xml"),
|
||||
MANIFEST_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
const manifest = manifestRead.text;
|
||||
const packageName =
|
||||
manifest === null ? null : parseManifestPackage(manifest);
|
||||
const locales = await listStringLocales(workspace);
|
||||
const stringsDirectory =
|
||||
input.locale === undefined
|
||||
? "values"
|
||||
@@ -639,8 +459,6 @@ export class ApktoolProvider {
|
||||
? await readBoundedText(
|
||||
join(workspace, "res", stringsDirectory, "strings.xml"),
|
||||
STRINGS_XML_MAX_BYTES,
|
||||
operation,
|
||||
signal,
|
||||
)
|
||||
: { text: null, truncated: false };
|
||||
// Do not parse an incomplete XML document as though its projection were complete.
|
||||
@@ -650,8 +468,7 @@ export class ApktoolProvider {
|
||||
? { entries: [], unparsedCount: 0 }
|
||||
: parseDecodedStrings(stringsXml);
|
||||
const bounded = parsedStrings.entries.slice(0, MAX_STRINGS);
|
||||
const files = await countDecodedFiles(workspace, signal, operation);
|
||||
throwIfCancelled(operation, signal);
|
||||
const files = await countDecodedFiles(workspace);
|
||||
const coverage =
|
||||
(stringsXml !== null && parsedStrings.entries.length > MAX_STRINGS) ||
|
||||
manifestRead.truncated ||
|
||||
@@ -676,7 +493,7 @@ export class ApktoolProvider {
|
||||
manifest,
|
||||
strings: bounded,
|
||||
locale: input.locale ?? null,
|
||||
locales: files.locales,
|
||||
locales,
|
||||
decoded_file_count: files.count,
|
||||
coverage,
|
||||
},
|
||||
@@ -718,7 +535,7 @@ export class ApktoolProvider {
|
||||
: []),
|
||||
...(files.capped
|
||||
? [
|
||||
`Decoded workspace enumeration stopped after ${String(MAX_DECODED_ENTRIES_VISITED)} entries; decoded_file_count and locales may be partial`,
|
||||
`decoded_file_count is capped at ${String(MAX_DECODED_FILES_COUNTED)}`,
|
||||
]
|
||||
: []),
|
||||
...(cleanupFailure === null ? [] : [cleanupFailure]),
|
||||
@@ -730,25 +547,14 @@ export class ApktoolProvider {
|
||||
await removeWorkspace();
|
||||
return err(
|
||||
failureWithCleanup(
|
||||
signal?.aborted === true
|
||||
? new AnalysisCancelledError(operation, { cause })
|
||||
: sourceAdmissionFailed
|
||||
? new AnalysisInputError(operation, { cause })
|
||||
: cause instanceof AnalysisInputError
|
||||
? cause
|
||||
: cause instanceof AnalysisCancelledError ||
|
||||
cause instanceof AnalysisProtocolError
|
||||
? cause
|
||||
: new AnalysisProtocolError(
|
||||
"Apktool resource projection failed",
|
||||
{
|
||||
cause,
|
||||
},
|
||||
),
|
||||
cause instanceof AnalysisCancelledError ||
|
||||
cause instanceof AnalysisProtocolError
|
||||
? cause
|
||||
: new AnalysisProtocolError("Apktool resource projection failed", {
|
||||
cause,
|
||||
}),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
await source?.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { lstat, readdir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -156,14 +156,13 @@ const wrappedBundle = async (bundle: string): Promise<string | undefined> => {
|
||||
if (isAbsence(cause)) return undefined;
|
||||
throw cause;
|
||||
}
|
||||
let app: string | undefined;
|
||||
for await (const entry of await opendir(wrapper)) {
|
||||
if (!entry.isDirectory() || !entry.name.toLowerCase().endsWith(".app"))
|
||||
continue;
|
||||
if (app !== undefined) return undefined;
|
||||
app = join(wrapper, entry.name);
|
||||
}
|
||||
return app;
|
||||
const apps = (await readdir(wrapper, { withFileTypes: true })).filter(
|
||||
(entry) => entry.isDirectory() && entry.name.toLowerCase().endsWith(".app"),
|
||||
);
|
||||
const [app] = apps;
|
||||
return apps.length === 1 && app !== undefined
|
||||
? join(wrapper, app.name)
|
||||
: undefined;
|
||||
};
|
||||
|
||||
const resolveLayoutExecutable = async (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { constants, type Stats } from "node:fs";
|
||||
import { constants } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
access,
|
||||
@@ -248,7 +248,7 @@ const isArchiveFormat = (
|
||||
const sha256Handle = async (
|
||||
handle: FileHandle,
|
||||
path: string,
|
||||
initial: Stats,
|
||||
initial: Awaited<ReturnType<FileHandle["stat"]>>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
@@ -256,18 +256,11 @@ const sha256Handle = async (
|
||||
let position = 0;
|
||||
while (true) {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const observed = await handle.read(
|
||||
chunk,
|
||||
0,
|
||||
Math.min(chunk.length, initial.size - position + 1),
|
||||
position,
|
||||
);
|
||||
const observed = await handle.read(chunk, 0, chunk.length, position);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (observed.bytesRead === 0) break;
|
||||
hash.update(chunk.subarray(0, observed.bytesRead));
|
||||
position += observed.bytesRead;
|
||||
if (position > initial.size)
|
||||
throw new BinaryTargetError(path, "target grew while being read");
|
||||
}
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const [opened, currentPath] = await Promise.all([handle.stat(), lstat(path)]);
|
||||
@@ -373,13 +366,7 @@ const readExecutableMetadata = async (
|
||||
if (!abiFlags.ok) return abiFlags;
|
||||
return ok({
|
||||
...parsed.value,
|
||||
mips: {
|
||||
...parsed.value.mips,
|
||||
abiFlags: abiFlags.value.abiFlags,
|
||||
...(abiFlags.value.limitations.length === 0
|
||||
? {}
|
||||
: { limitations: abiFlags.value.limitations }),
|
||||
},
|
||||
mips: { ...parsed.value.mips, abiFlags: abiFlags.value },
|
||||
});
|
||||
};
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import type { Result } from "../domain/result.js";
|
||||
|
||||
/** Provider-neutral application boundary for controlled browser scenarios. */
|
||||
export interface BrowserScenarioCapturePort {
|
||||
close?(): Promise<void>;
|
||||
identity(): ProviderIdentity;
|
||||
captureScenario(
|
||||
scenario: BrowserScenario,
|
||||
|
||||
@@ -1,22 +1,17 @@
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
RegularFileCleanupFailure,
|
||||
readRegularFileText,
|
||||
} from "./RegularFileRead.js";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import {
|
||||
effectiveClientServer,
|
||||
parseClientConfiguration,
|
||||
type ClientConfigurationDocument,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import type { SetupClient } from "./SupportedClients.js";
|
||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
||||
|
||||
/** A named source failure that setup can explain without losing the bad path. */
|
||||
export interface ClientConfigurationFileError {
|
||||
readonly kind: "unreadable" | "malformed";
|
||||
readonly path: string;
|
||||
readonly detail: string;
|
||||
readonly cleanupFailure?: RegularFileCleanupFailure;
|
||||
}
|
||||
|
||||
/** Preserve named read failures and distinguish a missing source from an empty file. */
|
||||
@@ -24,7 +19,7 @@ export const readConfigurationText = async (
|
||||
path: string,
|
||||
): Promise<Result<string | undefined, ClientConfigurationFileError>> => {
|
||||
try {
|
||||
return ok(await readRegularFileText(path));
|
||||
return ok(await readFile(path, "utf8"));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return ok(undefined);
|
||||
@@ -32,9 +27,6 @@ export const readConfigurationText = async (
|
||||
kind: "unreadable",
|
||||
path,
|
||||
detail: `Configuration ${path} could not be read: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
...(cause instanceof RegularFileCleanupFailure
|
||||
? { cleanupFailure: cause }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
};
|
||||
@@ -71,16 +63,7 @@ export const readClientConfigurationFiles = async (
|
||||
for (const path of client.configPaths ?? [client.configPath]) {
|
||||
const document = await readConfigurationFile(path, client.format);
|
||||
if (!document.ok) return document;
|
||||
if (document.value !== undefined) {
|
||||
const environment = clientRegistrationEnvironment(document.value);
|
||||
if (!environment.ok)
|
||||
return err({
|
||||
kind: "malformed",
|
||||
path,
|
||||
detail: `Configuration ${path} has an invalid REA server environment: ${environment.error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
|
||||
});
|
||||
documents.push(document.value);
|
||||
}
|
||||
if (document.value !== undefined) documents.push(document.value);
|
||||
}
|
||||
return ok(documents);
|
||||
};
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { parseEnvironment } from "../config/environment.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
effectiveClientServer,
|
||||
type ClientConfigurationDocument,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
|
||||
/** Read explicit REA overrides without dropping peers when one value is invalid. */
|
||||
export const clientRegistrationEnvironment = (
|
||||
parsed: ClientConfigurationDocument | undefined,
|
||||
overrides: Readonly<Record<string, string>> = {},
|
||||
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
|
||||
const key =
|
||||
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
|
||||
? "environment"
|
||||
: "env";
|
||||
const registration =
|
||||
parsed === undefined
|
||||
? undefined
|
||||
: effectiveClientServer(parsed, PRODUCT_IDENTITY.mcpServerKey);
|
||||
const raw =
|
||||
typeof registration === "object" && registration !== null
|
||||
? Reflect.get(registration, key)
|
||||
: undefined;
|
||||
const existing = z
|
||||
.record(z.string(), z.string())
|
||||
.safeParse(raw === undefined ? {} : raw);
|
||||
if (!existing.success)
|
||||
return err(
|
||||
new ConfigurationError("Invalid REA server environment", {
|
||||
settings: existing.error.issues.map((issue) => ({
|
||||
setting: [key, ...issue.path].join("."),
|
||||
constraint:
|
||||
"Server environment must be an object containing only string values",
|
||||
})),
|
||||
}),
|
||||
);
|
||||
// Validate known settings with the runtime's parser, but retain the original
|
||||
// string overrides, including unknown client-specific keys and no defaults.
|
||||
const environment = { ...existing.data, ...overrides };
|
||||
const validated = parseEnvironment(environment);
|
||||
if (!validated.ok) return validated;
|
||||
return ok(
|
||||
Object.fromEntries(
|
||||
Object.entries(environment).sort(([left], [right]) =>
|
||||
left.localeCompare(right),
|
||||
),
|
||||
),
|
||||
);
|
||||
};
|
||||
@@ -6,7 +6,7 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
import type { AnalysisExecution } from "./AnalysisProvider.js";
|
||||
import type { BinarySession } from "./binary/BinarySession.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "./binary/AnalysisClientCleanup.js";
|
||||
|
||||
/** Await one-shot cleanup and retain either the outcome or both failures. */
|
||||
export const withSessionCleanup = async <Value>(
|
||||
|
||||
@@ -5,22 +5,27 @@ import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { compareBundles } from "../domain/bundleComparison.js";
|
||||
import { EvidenceLedger } from "./investigation/EvidenceLedger.js";
|
||||
import {
|
||||
readEvidenceBundle,
|
||||
writeEvidenceBundle,
|
||||
} from "./EvidenceBundleFiles.js";
|
||||
|
||||
/** Validate a standalone bundle and report its complete record/revision counts. */
|
||||
/** Validate and merge one bundle using the session Evidence ledger. */
|
||||
export const importEvidenceBundleCommand = async (
|
||||
path: string,
|
||||
): Promise<Result<JsonValue, AnalysisError>> => {
|
||||
const loaded = await readEvidenceBundle(path);
|
||||
if (!loaded.ok) return loaded;
|
||||
return ok({
|
||||
imported: loaded.value.records.length,
|
||||
unknowns_added: loaded.value.unknowns.length,
|
||||
total: loaded.value.records.length,
|
||||
});
|
||||
const ledger = createLedger();
|
||||
const imported = ledger.import(loaded.value);
|
||||
return imported.ok
|
||||
? ok({
|
||||
imported: imported.value.recordsAdded,
|
||||
unknowns_added: imported.value.unknownsAdded,
|
||||
total: ledger.export().records.length,
|
||||
})
|
||||
: imported;
|
||||
};
|
||||
|
||||
/** Validate a source bundle and atomically export canonical bytes. */
|
||||
@@ -59,6 +64,8 @@ export const compareEvidenceBundlesCommand = async (input: {
|
||||
}
|
||||
};
|
||||
|
||||
const createLedger = (): EvidenceLedger => new EvidenceLedger();
|
||||
|
||||
const projectWrite = (
|
||||
bundle: EvidenceBundle,
|
||||
written: Awaited<ReturnType<typeof writeEvidenceBundle>>,
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
/** Maximum admitted vendor response bytes, shared by Hopper setup adapters. */
|
||||
export const HOPPER_DOWNLOAD_BYTES = 100_000_000;
|
||||
|
||||
/** Bound observed response bytes before retaining them, including chunked bodies. */
|
||||
export const downloadHopperPackage = async (
|
||||
url: string,
|
||||
options: { readonly signal?: AbortSignal } = {},
|
||||
): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }> => {
|
||||
const deadline = AbortSignal.timeout(120_000);
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
signal:
|
||||
options.signal === undefined
|
||||
? deadline
|
||||
: AbortSignal.any([options.signal, deadline]),
|
||||
});
|
||||
const refused = { ok: false, bytes: new Uint8Array() };
|
||||
if (
|
||||
!response.ok ||
|
||||
Number(response.headers.get("content-length")) > HOPPER_DOWNLOAD_BYTES
|
||||
) {
|
||||
await response.body?.cancel();
|
||||
return refused;
|
||||
}
|
||||
if (response.body === null) return { ok: true, bytes: new Uint8Array() };
|
||||
const reader = response.body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let bytes = 0;
|
||||
try {
|
||||
while (true) {
|
||||
const chunk = await reader.read();
|
||||
if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) };
|
||||
if (chunk.value.byteLength > HOPPER_DOWNLOAD_BYTES - bytes) {
|
||||
await reader.cancel();
|
||||
return refused;
|
||||
}
|
||||
chunks.push(chunk.value);
|
||||
bytes += chunk.value.byteLength;
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
};
|
||||
@@ -16,7 +16,8 @@ import {
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { readJsonInputFile } from "./JsonInputFile.js";
|
||||
import { parseUtf8Json } from "./Utf8JsonInput.js";
|
||||
import { readRegularFile } from "./RegularFileRead.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
|
||||
/** Request control and its owning operation for an interruptible atomic write. */
|
||||
@@ -36,20 +37,12 @@ export const readJsonFile = async (
|
||||
> => {
|
||||
const requestedPath = resolve(path);
|
||||
try {
|
||||
const decoded = await readJsonInputFile(
|
||||
requestedPath,
|
||||
"read_evidence_file",
|
||||
);
|
||||
const encoded = await readRegularFile(requestedPath);
|
||||
const decoded = parseUtf8Json(encoded, "read_evidence_file", requestedPath);
|
||||
if (!decoded.ok) {
|
||||
if (decoded.error instanceof AnalysisResourceConstraintError)
|
||||
return err(decoded.error);
|
||||
return err(
|
||||
new EvidenceFileError("read", "invalid-json", {
|
||||
cause:
|
||||
decoded.error.cause ??
|
||||
new SyntaxError(
|
||||
decoded.error.issues[0]?.message ?? decoded.error.message,
|
||||
),
|
||||
cause: decoded.cause,
|
||||
path: requestedPath,
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -7,13 +7,10 @@ import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { execFileOutput } from "../process/ExecFileOutput.js";
|
||||
import {
|
||||
downloadHopperPackage,
|
||||
HOPPER_DOWNLOAD_BYTES,
|
||||
} from "./HopperDownload.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
|
||||
const MAX_PACKAGE_BYTES = 100_000_000;
|
||||
const SUPPORTED_HOPPER_SHA256 = new Set([
|
||||
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
|
||||
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
|
||||
@@ -234,7 +231,16 @@ export const systemLinuxHopperInstallHost = (
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): LinuxHopperInstallHost => ({
|
||||
distribution: readLinuxDistribution,
|
||||
download: downloadHopperPackage,
|
||||
async download(url, options) {
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
return {
|
||||
ok: response.ok,
|
||||
bytes: new Uint8Array(await response.arrayBuffer()),
|
||||
};
|
||||
},
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
|
||||
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
installPackage: (family, archive) =>
|
||||
@@ -344,7 +350,7 @@ const packageIntegrityMatches = (
|
||||
release: LinuxHopperRelease,
|
||||
): boolean =>
|
||||
bytes.byteLength === Number(release.file_length) &&
|
||||
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
|
||||
bytes.byteLength <= MAX_PACKAGE_BYTES &&
|
||||
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
|
||||
|
||||
const packageFamilyFor = (
|
||||
|
||||
+167
-213
@@ -15,15 +15,12 @@ import { promisify } from "node:util";
|
||||
|
||||
import { z } from "zod";
|
||||
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
||||
import {
|
||||
downloadHopperPackage,
|
||||
HOPPER_DOWNLOAD_BYTES,
|
||||
} from "./HopperDownload.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const RELEASES_URL =
|
||||
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
|
||||
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
|
||||
const MAX_PACKAGE_BYTES = 100_000_000;
|
||||
const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4";
|
||||
|
||||
/** Stable network and command effects disclosed before macOS Hopper setup. */
|
||||
@@ -39,11 +36,7 @@ export const macHopperInstallDisclosure = {
|
||||
|
||||
/** Result of installing the official Hopper macOS application. */
|
||||
export type MacHopperInstallResult =
|
||||
| {
|
||||
readonly status: "installed";
|
||||
readonly launcherPath: string;
|
||||
readonly cleanupFailure?: string;
|
||||
}
|
||||
| { readonly status: "installed"; readonly launcherPath: string }
|
||||
| {
|
||||
readonly status: "failed";
|
||||
readonly reason:
|
||||
@@ -55,7 +48,6 @@ export type MacHopperInstallResult =
|
||||
| "destination_exists"
|
||||
| "copy"
|
||||
| "launcher_missing";
|
||||
readonly cleanupFailure?: string;
|
||||
};
|
||||
|
||||
/** External effects used by the bounded macOS Hopper installer. */
|
||||
@@ -80,8 +72,6 @@ export interface MacHopperInstallHost {
|
||||
launcherReady(path: string): Promise<boolean>;
|
||||
openApplication(path: string): Promise<void>;
|
||||
cleanup(path: string): Promise<void>;
|
||||
/** Retain the cleanup capability until the owning setup host retries it. */
|
||||
retainCleanup?(cleanup: () => Promise<void>): void;
|
||||
destination(): string;
|
||||
}
|
||||
|
||||
@@ -95,221 +85,187 @@ export const installMacHopper = async (
|
||||
): Promise<MacHopperInstallResult> => {
|
||||
let temporary: string | undefined;
|
||||
let mounted = false;
|
||||
let result: MacHopperInstallResult;
|
||||
try {
|
||||
result = await (async (): Promise<MacHopperInstallResult> => {
|
||||
const metadata = await host.download(RELEASES_URL);
|
||||
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
|
||||
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
|
||||
if (!parsed.success)
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const release = parsed.data["OS X"];
|
||||
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const archive = await host.download(release.filename);
|
||||
if (!archive.ok) return { status: "failed", reason: "download" };
|
||||
if (!integrityMatches(archive.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
const metadata = await host.download(RELEASES_URL);
|
||||
if (!metadata.ok) return { status: "failed", reason: "release_metadata" };
|
||||
const parsed = releasesSchema.safeParse(parseJson(metadata.bytes));
|
||||
if (!parsed.success)
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const release = parsed.data["OS X"];
|
||||
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
|
||||
return { status: "failed", reason: "release_metadata" };
|
||||
const archive = await host.download(release.filename);
|
||||
if (!archive.ok) return { status: "failed", reason: "download" };
|
||||
if (!integrityMatches(archive.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
|
||||
const destination = host.destination();
|
||||
const destinationExists = await host.destinationExists(destination);
|
||||
if (destinationExists && options.replaceExisting !== true)
|
||||
return { status: "failed", reason: "destination_exists" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const packagePath = join(temporary, "hopper.dmg");
|
||||
const mountPath = join(temporary, "mounted");
|
||||
await host.createMountDirectory(mountPath);
|
||||
await host.writePackage(packagePath, archive.bytes);
|
||||
if (!(await host.mount(packagePath, mountPath)))
|
||||
return { status: "failed", reason: "mount" };
|
||||
mounted = true;
|
||||
const source = await host.appBundle(mountPath);
|
||||
if (source === undefined || !(await host.validBundle(source)))
|
||||
return { status: "failed", reason: "bundle" };
|
||||
const stage = `${destination}.rea-stage`;
|
||||
if (
|
||||
!(await host.installBundle(
|
||||
source,
|
||||
destination,
|
||||
stage,
|
||||
destinationExists,
|
||||
))
|
||||
)
|
||||
return { status: "failed", reason: "copy" };
|
||||
const launcherPath = join(destination, "Contents/MacOS/hopper");
|
||||
if (!(await host.launcherReady(launcherPath)))
|
||||
return { status: "failed", reason: "launcher_missing" };
|
||||
await host.openApplication(destination);
|
||||
return { status: "installed", launcherPath };
|
||||
})();
|
||||
const destination = host.destination();
|
||||
const destinationExists = await host.destinationExists(destination);
|
||||
if (destinationExists && options.replaceExisting !== true)
|
||||
return { status: "failed", reason: "destination_exists" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const packagePath = join(temporary, "hopper.dmg");
|
||||
const mountPath = join(temporary, "mounted");
|
||||
await host.createMountDirectory(mountPath);
|
||||
await host.writePackage(packagePath, archive.bytes);
|
||||
if (!(await host.mount(packagePath, mountPath)))
|
||||
return { status: "failed", reason: "mount" };
|
||||
mounted = true;
|
||||
const source = await host.appBundle(mountPath);
|
||||
if (source === undefined || !(await host.validBundle(source)))
|
||||
return { status: "failed", reason: "bundle" };
|
||||
const stage = `${destination}.rea-stage`;
|
||||
if (
|
||||
!(await host.installBundle(source, destination, stage, destinationExists))
|
||||
)
|
||||
return { status: "failed", reason: "copy" };
|
||||
const launcherPath = join(destination, "Contents/MacOS/hopper");
|
||||
if (!(await host.launcherReady(launcherPath)))
|
||||
return { status: "failed", reason: "launcher_missing" };
|
||||
await host.openApplication(destination);
|
||||
return { status: "installed", launcherPath };
|
||||
} catch (cause: unknown) {
|
||||
// Install-stage failures share the download reason; the stage is cleaned below.
|
||||
void cause;
|
||||
result = { status: "failed", reason: "download" };
|
||||
}
|
||||
|
||||
let cleanupFailure: string | undefined;
|
||||
if (temporary !== undefined) {
|
||||
const temporaryPath = temporary;
|
||||
const mountPath = join(temporary, "mounted");
|
||||
const cleanup = async (): Promise<void> => {
|
||||
if (mounted) {
|
||||
await host.unmount(mountPath);
|
||||
mounted = false;
|
||||
}
|
||||
await host.cleanup(temporaryPath);
|
||||
};
|
||||
try {
|
||||
await cleanup();
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = `Hopper DMG cleanup failed: ${failureMessage(cause)}`;
|
||||
host.retainCleanup?.(cleanup);
|
||||
return { status: "failed", reason: "download" };
|
||||
} finally {
|
||||
if (temporary !== undefined) {
|
||||
const mountPath = join(temporary, "mounted");
|
||||
if (mounted) await host.unmount(mountPath);
|
||||
await host.cleanup(temporary);
|
||||
}
|
||||
}
|
||||
|
||||
return cleanupFailure === undefined ? result : { ...result, cleanupFailure };
|
||||
};
|
||||
|
||||
/** Compose installer paths and subprocesses from one selected caller context. */
|
||||
export const systemMacHopperInstallHost = (
|
||||
homeDirectory: string,
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): MacHopperInstallHost & { close(): Promise<string | undefined> } => {
|
||||
const pendingCleanup = new Set<() => Promise<void>>();
|
||||
return {
|
||||
download: downloadHopperPackage,
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
|
||||
createMountDirectory: (path) => mkdir(path),
|
||||
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
async mount(packagePath, mountPath) {
|
||||
return commandSucceeds(
|
||||
"hdiutil",
|
||||
[
|
||||
"attach",
|
||||
"-readonly",
|
||||
"-nobrowse",
|
||||
"-mountpoint",
|
||||
mountPath,
|
||||
packagePath,
|
||||
],
|
||||
): MacHopperInstallHost => ({
|
||||
download: downloadPackage,
|
||||
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-mac-")),
|
||||
createMountDirectory: (path) => mkdir(path),
|
||||
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
|
||||
async mount(packagePath, mountPath) {
|
||||
return commandSucceeds(
|
||||
"hdiutil",
|
||||
[
|
||||
"attach",
|
||||
"-readonly",
|
||||
"-nobrowse",
|
||||
"-mountpoint",
|
||||
mountPath,
|
||||
packagePath,
|
||||
],
|
||||
environment,
|
||||
);
|
||||
},
|
||||
async unmount(mountPath) {
|
||||
await commandSucceeds(
|
||||
"hdiutil",
|
||||
["detach", mountPath, "-force"],
|
||||
environment,
|
||||
);
|
||||
},
|
||||
async appBundle(mountPath) {
|
||||
const entries = await readdir(mountPath, { withFileTypes: true });
|
||||
const app = entries.find(
|
||||
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
|
||||
);
|
||||
return app === undefined ? undefined : join(mountPath, app.name);
|
||||
},
|
||||
async validBundle(appPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"codesign",
|
||||
["--verify", "--deep", "--strict", appPath],
|
||||
environment,
|
||||
);
|
||||
},
|
||||
async unmount(mountPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"hdiutil",
|
||||
["detach", mountPath, "-force"],
|
||||
environment,
|
||||
))
|
||||
)
|
||||
throw new Error(
|
||||
`hdiutil detach failed for ${JSON.stringify(mountPath)}`,
|
||||
);
|
||||
},
|
||||
async appBundle(mountPath) {
|
||||
const entries = await readdir(mountPath, { withFileTypes: true });
|
||||
const app = entries.find(
|
||||
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
|
||||
);
|
||||
return app === undefined ? undefined : join(mountPath, app.name);
|
||||
},
|
||||
async validBundle(appPath) {
|
||||
if (
|
||||
!(await commandSucceeds(
|
||||
"codesign",
|
||||
["--verify", "--deep", "--strict", appPath],
|
||||
environment,
|
||||
))
|
||||
)
|
||||
return false;
|
||||
try {
|
||||
const identifier = (
|
||||
await execFileAsync(
|
||||
"/usr/libexec/PlistBuddy",
|
||||
[
|
||||
"-c",
|
||||
"Print :CFBundleIdentifier",
|
||||
join(appPath, "Contents/Info.plist"),
|
||||
],
|
||||
{ env: environment },
|
||||
)
|
||||
).stdout.trim();
|
||||
return identifier === BUNDLE_IDENTIFIER;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional bundle probing; unreadable means invalid.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async destinationExists(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional existence probing; absence means missing.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async installBundle(source, destination, stage, replaceExisting) {
|
||||
const previous = `${stage}-previous`;
|
||||
try {
|
||||
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
await execFileAsync("ditto", [source, stage], { env: environment });
|
||||
await access(join(stage, "Contents/MacOS/hopper"));
|
||||
if (replaceExisting) await rename(destination, previous);
|
||||
await rename(stage, destination);
|
||||
if (replaceExisting)
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// Install failure rolls back below; the false return reports it.
|
||||
void cause;
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
if (replaceExisting) {
|
||||
try {
|
||||
await rename(previous, destination);
|
||||
} catch (rollbackCause: unknown) {
|
||||
// The original destination was not moved, or rollback is impossible.
|
||||
void rollbackCause;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async launcherReady(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional launcher probing; absence means not ready.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async openApplication(path) {
|
||||
await execFileAsync("open", [path], { env: environment });
|
||||
},
|
||||
cleanup: (path) => rm(path, { recursive: true, force: true }),
|
||||
retainCleanup: (cleanup) => pendingCleanup.add(cleanup),
|
||||
async close() {
|
||||
let failure: string | undefined;
|
||||
for (const cleanup of pendingCleanup) {
|
||||
))
|
||||
)
|
||||
return false;
|
||||
try {
|
||||
const identifier = (
|
||||
await execFileAsync(
|
||||
"/usr/libexec/PlistBuddy",
|
||||
[
|
||||
"-c",
|
||||
"Print :CFBundleIdentifier",
|
||||
join(appPath, "Contents/Info.plist"),
|
||||
],
|
||||
{ env: environment },
|
||||
)
|
||||
).stdout.trim();
|
||||
return identifier === BUNDLE_IDENTIFIER;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional bundle probing; unreadable means invalid.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async destinationExists(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional existence probing; absence means missing.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async installBundle(source, destination, stage, replaceExisting) {
|
||||
const previous = `${stage}-previous`;
|
||||
try {
|
||||
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
await rm(previous, { recursive: true, force: true });
|
||||
await execFileAsync("ditto", [source, stage], { env: environment });
|
||||
await access(join(stage, "Contents/MacOS/hopper"));
|
||||
if (replaceExisting) await rename(destination, previous);
|
||||
await rename(stage, destination);
|
||||
if (replaceExisting) await rm(previous, { recursive: true, force: true });
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// Install failure rolls back below; the false return reports it.
|
||||
void cause;
|
||||
await rm(stage, { recursive: true, force: true });
|
||||
if (replaceExisting) {
|
||||
try {
|
||||
await cleanup();
|
||||
pendingCleanup.delete(cleanup);
|
||||
} catch (cause: unknown) {
|
||||
failure ??= `Hopper DMG cleanup remains incomplete: ${failureMessage(cause)}`;
|
||||
await rename(previous, destination);
|
||||
} catch (rollbackCause: unknown) {
|
||||
// The original destination was not moved, or rollback is impossible.
|
||||
void rollbackCause;
|
||||
}
|
||||
}
|
||||
return failure;
|
||||
},
|
||||
destination: () =>
|
||||
join(homeDirectory, "Applications/Hopper Disassembler.app"),
|
||||
};
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async launcherReady(path) {
|
||||
try {
|
||||
await access(path);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// best-effort cleanup: optional launcher probing; absence means not ready.
|
||||
void cause;
|
||||
return false;
|
||||
}
|
||||
},
|
||||
async openApplication(path) {
|
||||
await execFileAsync("open", [path], { env: environment });
|
||||
},
|
||||
cleanup: (path) => rm(path, { recursive: true, force: true }),
|
||||
destination: () =>
|
||||
join(homeDirectory, "Applications/Hopper Disassembler.app"),
|
||||
});
|
||||
|
||||
const downloadPackage = async (url: string) => {
|
||||
const response = await fetch(url, {
|
||||
headers: { accept: "application/json", "user-agent": "rea-installer" },
|
||||
signal: AbortSignal.timeout(120_000),
|
||||
});
|
||||
const length = Number(response.headers.get("content-length") ?? "0");
|
||||
if (length > MAX_PACKAGE_BYTES) return { ok: false, bytes: new Uint8Array() };
|
||||
const bytes = new Uint8Array(await response.arrayBuffer());
|
||||
return { ok: response.ok && bytes.byteLength <= MAX_PACKAGE_BYTES, bytes };
|
||||
};
|
||||
|
||||
const parseJson = (bytes: Uint8Array): unknown =>
|
||||
@@ -319,7 +275,7 @@ const integrityMatches = (
|
||||
release: z.infer<typeof releaseSchema>,
|
||||
): boolean =>
|
||||
bytes.byteLength === Number(release.file_length) &&
|
||||
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
|
||||
bytes.byteLength <= MAX_PACKAGE_BYTES &&
|
||||
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
|
||||
const commandSucceeds = async (
|
||||
command: string,
|
||||
@@ -335,8 +291,6 @@ const commandSucceeds = async (
|
||||
return false;
|
||||
}
|
||||
};
|
||||
const failureMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
const releaseSchema = z.object({
|
||||
filename: z.string().url(),
|
||||
file_length: z.string().regex(/^\d+$/u),
|
||||
|
||||
@@ -5,12 +5,6 @@ import type {
|
||||
} from "../domain/binaryTargetTypes.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/** Observed ABI declarations and coverage of their source tables. */
|
||||
export interface MipsAbiFlagsInspection {
|
||||
readonly abiFlags: MipsAbiFlags | null;
|
||||
readonly limitations: readonly string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Read ELF32 MIPS ABI declarations from the caller's already-open file.
|
||||
* Inspect both typed tables (not section names), including records beyond the
|
||||
@@ -21,7 +15,7 @@ export const readMipsElfAbiFlags = async (
|
||||
handle: FileHandle,
|
||||
metadata: MipsElfMetadata,
|
||||
checkCancelled: () => void,
|
||||
): Promise<Result<MipsAbiFlagsInspection, string>> => {
|
||||
): Promise<Result<MipsAbiFlags | null, string>> => {
|
||||
checkCancelled();
|
||||
if (metadata.elfClass !== 32)
|
||||
return err("MIPS ABI record inspection currently supports ELF32 only");
|
||||
@@ -87,7 +81,6 @@ export const readMipsElfAbiFlags = async (
|
||||
dataOffset: 4,
|
||||
sizeOffset: 16,
|
||||
recordType: 0x70000003, // PT_MIPS_ABIFLAGS
|
||||
optional: false,
|
||||
},
|
||||
{
|
||||
label: "section header table",
|
||||
@@ -99,7 +92,6 @@ export const readMipsElfAbiFlags = async (
|
||||
dataOffset: 16,
|
||||
sizeOffset: 20,
|
||||
recordType: 0x7000002a, // SHT_MIPS_ABIFLAGS
|
||||
optional: true,
|
||||
},
|
||||
];
|
||||
// Extended numbering needs a separate section-zero reader. Refuse it rather
|
||||
@@ -107,7 +99,6 @@ export const readMipsElfAbiFlags = async (
|
||||
if (u16(bytes, 44) === 0xffff || u16(bytes, 48) >= 0xff00)
|
||||
return err("extended or reserved MIPS ELF table numbering is unsupported");
|
||||
let record: Buffer | undefined;
|
||||
const limitations: string[] = [];
|
||||
for (const table of tables) {
|
||||
if (table.count === 0) {
|
||||
if (table.offset !== 0)
|
||||
@@ -116,27 +107,16 @@ export const readMipsElfAbiFlags = async (
|
||||
}
|
||||
if (table.offset < 52 || table.stride !== table.entrySize)
|
||||
return err(`invalid MIPS ${table.label} offset or entry size`);
|
||||
const count = table.optional
|
||||
? Math.min(
|
||||
table.count,
|
||||
Math.max(0, Math.floor((fileSize - table.offset) / table.entrySize)),
|
||||
)
|
||||
: table.count;
|
||||
if (count < table.count)
|
||||
limitations.push(
|
||||
`MIPS ${table.label} at file offset ${table.offset} extends beyond the captured file: examined ${count} of ${table.count} complete headers. ABI declarations in unavailable headers remain unknown.`,
|
||||
);
|
||||
if (count === 0) continue;
|
||||
// Counts are bounded by the ELF16 fields and strides by their exact ELF32
|
||||
// structure sizes. Never allocate according to an untrusted section size.
|
||||
const contents = await read(
|
||||
table.offset,
|
||||
count * table.entrySize,
|
||||
table.count * table.entrySize,
|
||||
table.label,
|
||||
);
|
||||
if (!contents.ok) return contents;
|
||||
let found = false;
|
||||
for (let i = 0; i < count; i += 1) {
|
||||
for (let i = 0; i < table.count; i += 1) {
|
||||
checkCancelled();
|
||||
const position = i * table.entrySize;
|
||||
if (u32(contents.value, position + table.typeOffset) !== table.recordType)
|
||||
@@ -154,10 +134,10 @@ export const readMipsElfAbiFlags = async (
|
||||
record = candidate.value;
|
||||
}
|
||||
}
|
||||
if (record === undefined) return ok({ abiFlags: null, limitations });
|
||||
if (record === undefined) return ok(null);
|
||||
// Preserve declarations verbatim, including unknown versions/values. Provider
|
||||
// admission, not parsing, decides which interpretations have been verified.
|
||||
const abiFlags: MipsAbiFlags = {
|
||||
return ok({
|
||||
version: u16(record, 0),
|
||||
isaLevel: record.readUInt8(2),
|
||||
isaRevision: record.readUInt8(3),
|
||||
@@ -169,6 +149,5 @@ export const readMipsElfAbiFlags = async (
|
||||
ases: u32(record, 12),
|
||||
flags1: u32(record, 16),
|
||||
flags2: u32(record, 20),
|
||||
};
|
||||
return ok({ abiFlags, limitations });
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1,11 +1,3 @@
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
import {
|
||||
createJavaScriptSemanticGraph,
|
||||
createJavaScriptSemanticGraphNode,
|
||||
JavaScriptSemanticEvidenceContextRegistry,
|
||||
} from "../domain/javascript/javascriptSemanticGraph.js";
|
||||
import { proofEvidence } from "./ReconstructionObligationLedger.fixture.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { createEvidence } from "../domain/evidence.js";
|
||||
@@ -179,115 +171,3 @@ describe("reconstruction obligation candidates", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps each semantic candidate bound to its own file and authority context", () => {
|
||||
const contexts = new JavaScriptSemanticEvidenceContextRegistry();
|
||||
const base = javascriptApplicationAnalysisResultSchema.parse(
|
||||
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
|
||||
);
|
||||
const sourceRecords = [
|
||||
proofEvidence("first-file"),
|
||||
proofEvidence("second-file"),
|
||||
];
|
||||
const module = createJavaScriptSemanticGraphNode(
|
||||
{
|
||||
kind: "module",
|
||||
identity: {
|
||||
artifact_sha256: base.root_artifact_sha256,
|
||||
module_path: "app.js",
|
||||
source_range: null,
|
||||
role_key: "module",
|
||||
},
|
||||
function_node_id: null,
|
||||
application_node_ids: [],
|
||||
label: "app.js",
|
||||
properties: {},
|
||||
evidence: artifactEvidence(base.root_artifact_sha256, "app.js"),
|
||||
},
|
||||
contexts,
|
||||
);
|
||||
const requests = sourceRecords.flatMap((source, fileIndex) =>
|
||||
[1, 2].map((line) => {
|
||||
const sha256 = String(fileIndex + 4).repeat(64);
|
||||
const path = `file-${String(fileIndex)}.js`;
|
||||
return createJavaScriptSemanticGraphNode(
|
||||
{
|
||||
kind: "request",
|
||||
identity: {
|
||||
artifact_sha256: sha256,
|
||||
module_path: path,
|
||||
source_range: {
|
||||
start: { line, column: 0 },
|
||||
end: { line, column: 8 },
|
||||
},
|
||||
role_key: `request-${String(line)}`,
|
||||
},
|
||||
function_node_id: null,
|
||||
application_node_ids: [],
|
||||
label: `${path}:${String(line)}`,
|
||||
properties: {},
|
||||
evidence: {
|
||||
...artifactEvidence(sha256, path, "ast-static-analysis"),
|
||||
evidence_ids: [source.evidence_id],
|
||||
},
|
||||
},
|
||||
contexts,
|
||||
);
|
||||
}),
|
||||
);
|
||||
const { graph_id: _graphId, ...semantic } = base.semantic_graph;
|
||||
const semanticGraph = createJavaScriptSemanticGraph({
|
||||
...semantic,
|
||||
root_node_ids: [module.node_id],
|
||||
nodes: [module, ...requests],
|
||||
evidence_contexts: contexts.contexts,
|
||||
});
|
||||
const example = JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE;
|
||||
const application = createEvidence(
|
||||
{
|
||||
path: base.input_path,
|
||||
sha256: base.root_artifact_sha256,
|
||||
format: base.format,
|
||||
},
|
||||
example.provider,
|
||||
{
|
||||
predicateType: example.predicate_type,
|
||||
operation: example.operation,
|
||||
parameters: example.parameters,
|
||||
result: jsonValueSchema.parse({ ...base, semantic_graph: semanticGraph }),
|
||||
confidence: "derived",
|
||||
authority: "shipped-artifact",
|
||||
evidenceLinks: sourceRecords.map(({ evidence_id }) => evidence_id),
|
||||
},
|
||||
);
|
||||
const generated = deriveReconstructionObligationCandidates(
|
||||
createEvidenceBundle([...sourceRecords, application]),
|
||||
[],
|
||||
);
|
||||
expect(generated.limitations).toEqual([]);
|
||||
expect(generated.candidates).toHaveLength(requests.length);
|
||||
for (const [index, node] of requests.entries()) {
|
||||
const source = sourceRecords[Math.floor(index / 2)];
|
||||
expect(
|
||||
generated.candidates.find(
|
||||
({ target }) => target.semantic_node_id === node.node_id,
|
||||
),
|
||||
).toMatchObject({
|
||||
title: node.label,
|
||||
family: "request",
|
||||
source_state: "candidate",
|
||||
target: {
|
||||
artifact_sha256: node.identity.artifact_sha256,
|
||||
semantic_node_id: node.node_id,
|
||||
},
|
||||
authority_references: [
|
||||
{
|
||||
evidence_id: source?.evidence_id,
|
||||
authority: "controlled-replay",
|
||||
state: "candidate",
|
||||
location: `${semanticGraph.graph_id}/node/${node.node_id}`,
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
@@ -12,6 +12,7 @@ import type {
|
||||
ReconstructionObligation,
|
||||
ReviewedReconstructionObligation,
|
||||
} from "../domain/reconstructionObligationLedgerSchemas.js";
|
||||
import { resolveJavaScriptSemanticEvidence } from "../domain/javascript/javascriptSemanticGraph.js";
|
||||
import { parseApplicationGraphEvidence } from "./javascript/JavaScriptApplicationEvidenceGraph.js";
|
||||
import {
|
||||
applicationObligationPolicy,
|
||||
@@ -114,21 +115,14 @@ const deriveApplicationCandidates = (
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (source.semanticGraph !== null) {
|
||||
const evidenceContexts = new Map(
|
||||
source.semanticGraph.evidence_contexts.map((context) => [
|
||||
context.context_id,
|
||||
context,
|
||||
]),
|
||||
);
|
||||
if (source.semanticGraph !== null)
|
||||
for (const node of source.semanticGraph.nodes) {
|
||||
const candidatePolicy = semanticObligationPolicy(node.kind);
|
||||
if (candidatePolicy === undefined) continue;
|
||||
const evidenceContext = evidenceContexts.get(node.evidence.context_id);
|
||||
if (evidenceContext === undefined)
|
||||
throw new TypeError(
|
||||
"Semantic evidence reference names an absent context",
|
||||
);
|
||||
const evidenceContext = resolveJavaScriptSemanticEvidence(
|
||||
source.semanticGraph,
|
||||
node.evidence,
|
||||
);
|
||||
addCandidate(
|
||||
candidates,
|
||||
generatedCandidate({
|
||||
@@ -150,7 +144,6 @@ const deriveApplicationCandidates = (
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (source.graph.coverage.status !== "complete")
|
||||
limitations.add(
|
||||
`Application graph ${source.graph.graph_id} coverage is ${source.graph.coverage.status}; omitted candidates remain unknown.`,
|
||||
|
||||
@@ -8,24 +8,14 @@ import {
|
||||
type HistoricalSourceGraphInput,
|
||||
} from "../domain/referenceSourceGraph.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
||||
import { readReferenceSource } from "../reference/ReferenceSourceReader.js";
|
||||
import type {
|
||||
ReferenceSourceEntryKind,
|
||||
ReferenceSourceRead,
|
||||
ReferenceSourceReaderError,
|
||||
} from "../reference/ReferenceSourceReaderTypes.js";
|
||||
import { parseReferenceSourceEntries } from "./ReferenceSourceImportEntries.js";
|
||||
import { readReferenceSourceVcs } from "./ReferenceSourceVcsAdapter.js";
|
||||
import {
|
||||
type ReferenceSourceImportError,
|
||||
type ReferenceSourceImportOptions,
|
||||
} from "./ReferenceSourceImportTypes.js";
|
||||
import {
|
||||
prepareReferenceSourceImport,
|
||||
type PreparedReferenceSourceImport,
|
||||
} from "./ReferenceSourceImportPolicy.js";
|
||||
import { prepareReferenceSourceImport } from "./ReferenceSourceImportPolicy.js";
|
||||
|
||||
const failure = (
|
||||
code: ReferenceSourceImportError["code"],
|
||||
@@ -36,12 +26,8 @@ const failure = (
|
||||
message,
|
||||
});
|
||||
|
||||
const cancelled = (
|
||||
partial?: ReferenceSourceRead,
|
||||
): ReferenceSourceImportError => ({
|
||||
...failure("cancelled", "Reference source import cancelled"),
|
||||
...(partial === undefined ? {} : { partial }),
|
||||
});
|
||||
const cancelled = (): ReferenceSourceImportError =>
|
||||
failure("cancelled", "Reference source import cancelled");
|
||||
|
||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
||||
|
||||
@@ -129,48 +115,9 @@ const sortExclusions = (
|
||||
[...exclusions].sort((left, right) => {
|
||||
const byPath = compareUnicodeCodePoints(left.path, right.path);
|
||||
if (byPath !== 0) return byPath;
|
||||
const byReason = compareUnicodeCodePoints(left.reason, right.reason);
|
||||
if (byReason !== 0) return byReason;
|
||||
return compareUnicodeCodePoints(
|
||||
"pattern" in left ? left.pattern : "",
|
||||
"pattern" in right ? right.pattern : "",
|
||||
);
|
||||
return compareUnicodeCodePoints(left.reason, right.reason);
|
||||
});
|
||||
|
||||
const createShouldExclude =
|
||||
(
|
||||
exclusions: HistoricalSourceGraphInput["exclusions"],
|
||||
secrets: PreparedReferenceSourceImport["secrets"],
|
||||
ignored: PreparedReferenceSourceImport["ignored"],
|
||||
): ((path: string, kind: ReferenceSourceEntryKind) => boolean) =>
|
||||
(path, kind) => {
|
||||
const patternPath = kind === "directory" ? `${path}/` : path;
|
||||
const secretMatch = secrets.test(patternPath);
|
||||
if (secretMatch.ignored) {
|
||||
if (!secretMatch.rule)
|
||||
throw new Error(`Ignored secret path has no matching rule: ${path}`);
|
||||
exclusions.push({
|
||||
path,
|
||||
reason: "configured-secret",
|
||||
pattern: secretMatch.rule.pattern,
|
||||
});
|
||||
return true;
|
||||
}
|
||||
const match = ignored.test(patternPath);
|
||||
if (!match.ignored) return false;
|
||||
if (!match.rule)
|
||||
throw new Error(`Ignored path has no matching rule: ${path}`);
|
||||
const reason = match.rule.mark;
|
||||
if (
|
||||
reason !== "project-ignored" &&
|
||||
reason !== "default-ignored" &&
|
||||
reason !== "caller-excluded"
|
||||
)
|
||||
throw new Error(`Ignored path has unknown rule origin: ${path}`);
|
||||
exclusions.push({ path, reason, pattern: match.rule.pattern });
|
||||
return true;
|
||||
};
|
||||
|
||||
/**
|
||||
* Import a reference source directory into a committed historical source graph.
|
||||
*
|
||||
@@ -180,41 +127,43 @@ const createShouldExclude =
|
||||
*/
|
||||
export const importReferenceSource = async (
|
||||
options: ReferenceSourceImportOptions,
|
||||
resources: ArtifactResourceScope,
|
||||
): Promise<Result<HistoricalSourceGraph, ReferenceSourceImportError>> => {
|
||||
if (isAborted(options.signal)) return err(cancelled());
|
||||
const prepared = await prepareReferenceSourceImport(options, resources);
|
||||
const prepared = await prepareReferenceSourceImport(options);
|
||||
if (!prepared.ok) return prepared;
|
||||
const { ignored, root, secrets } = prepared.value;
|
||||
if (isAborted(options.signal)) return err(cancelled());
|
||||
|
||||
const exclusions: HistoricalSourceGraphInput["exclusions"] = [];
|
||||
const shouldExclude = createShouldExclude(exclusions, secrets, ignored);
|
||||
const shouldExclude = (path: string): boolean => {
|
||||
if (secrets.ignores(path)) {
|
||||
exclusions.push({ path, reason: "configured-secret" });
|
||||
return true;
|
||||
}
|
||||
if (ignored.ignores(path)) {
|
||||
exclusions.push({ path, reason: "caller-excluded" });
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
const [readResult, vcsResult] = await Promise.all([
|
||||
readReferenceSource(root, resources, {
|
||||
const [readResult, vcs] = await Promise.all([
|
||||
readReferenceSource(root, {
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
shouldExclude,
|
||||
}),
|
||||
readReferenceSourceVcs(root, resources, options.signal),
|
||||
readReferenceSourceVcs(root, options.signal),
|
||||
]);
|
||||
|
||||
if (!readResult.ok)
|
||||
return err(
|
||||
combineReadFailures(
|
||||
readResult.error,
|
||||
vcsResult.ok ? undefined : vcsResult.error,
|
||||
readResult.error.partial,
|
||||
),
|
||||
);
|
||||
if (!vcsResult.ok)
|
||||
return err(
|
||||
combineReadFailures(vcsResult.error, undefined, readResult.value),
|
||||
);
|
||||
if (!readResult.ok) {
|
||||
const error = readResult.error;
|
||||
if (error.code === "cancelled") return err(cancelled());
|
||||
if (error.code === "unsupported")
|
||||
return err(failure("unsupported", error.message));
|
||||
return err(failure("io", error.message));
|
||||
}
|
||||
|
||||
const vcs = vcsResult.value;
|
||||
|
||||
if (isAborted(options.signal)) return err(cancelled(readResult.value));
|
||||
if (isAborted(options.signal)) return err(cancelled());
|
||||
|
||||
const read = readResult.value;
|
||||
const filePaths = new Set(
|
||||
@@ -226,7 +175,7 @@ export const importReferenceSource = async (
|
||||
const { entries, relationships, parseFailures, limitations } =
|
||||
parseReferenceSourceEntries(read, filePaths, options.signal);
|
||||
|
||||
if (isAborted(options.signal)) return err(cancelled(read));
|
||||
if (isAborted(options.signal)) return err(cancelled());
|
||||
|
||||
const uniqueRelationships = deduplicateRelationships(relationships);
|
||||
const uniqueFailures = normalizeHistoricalSourceParseFailures(parseFailures);
|
||||
@@ -272,45 +221,3 @@ export const importReferenceSource = async (
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const combineReadFailures = (
|
||||
primary: ReferenceSourceReaderError,
|
||||
other: ReferenceSourceReaderError | undefined,
|
||||
partial: ReferenceSourceRead | undefined,
|
||||
): ReferenceSourceImportError => {
|
||||
const code =
|
||||
primary.code === "cancelled" || other?.code === "cancelled"
|
||||
? "cancelled"
|
||||
: primary.code;
|
||||
const cleanup = mergeCleanup(primary.cleanup, other?.cleanup);
|
||||
const message =
|
||||
other === undefined
|
||||
? primary.message
|
||||
: `${primary.message}; additional reference metadata failure: ${other.message}`;
|
||||
const cause =
|
||||
other === undefined
|
||||
? primary.cause
|
||||
: new AggregateError(
|
||||
[primary, other],
|
||||
"Reference source inventory and metadata both failed",
|
||||
{ cause: primary },
|
||||
);
|
||||
return {
|
||||
...failure(code, message),
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
...(partial === undefined ? {} : { partial }),
|
||||
...(cause === undefined ? {} : { cause }),
|
||||
};
|
||||
};
|
||||
|
||||
const mergeCleanup = (
|
||||
first: AnalysisCleanupObservation | undefined,
|
||||
second: AnalysisCleanupObservation | undefined,
|
||||
): AnalysisCleanupObservation | undefined => {
|
||||
if (first === undefined) return second;
|
||||
if (second === undefined) return first;
|
||||
return {
|
||||
reason: `${first.reason}; ${second.reason}`,
|
||||
resources: [...new Set([...first.resources, ...second.resources])],
|
||||
};
|
||||
};
|
||||
|
||||
@@ -1,12 +1,10 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { posix } from "node:path";
|
||||
|
||||
import { logicalPathEscapesRoot } from "../domain/artifactIdentity.js";
|
||||
import {
|
||||
createReferenceSourcePathLookup,
|
||||
isPortableAbsoluteReferenceTarget,
|
||||
type ReferenceSourcePathLookup,
|
||||
} from "../domain/referenceSourcePathIdentity.js";
|
||||
logicalPathEscapesRoot,
|
||||
normalizeJoinedLogicalPath,
|
||||
} from "../domain/artifactIdentity.js";
|
||||
import {
|
||||
classifyReferenceSourcePath,
|
||||
detectReferenceSourceLanguage,
|
||||
@@ -85,11 +83,11 @@ const failedEntry = (
|
||||
const resolveInternalSpecifier = (
|
||||
fromPath: string,
|
||||
specifier: string,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
filePaths: ReadonlySet<string>,
|
||||
): { to: string; resolution: "internal" | "unresolved" } => {
|
||||
if (posix.isAbsolute(specifier))
|
||||
return { to: specifier, resolution: "unresolved" };
|
||||
const normalized = posix.normalize(
|
||||
const normalized = normalizeJoinedLogicalPath(
|
||||
posix.join(posix.dirname(fromPath), specifier),
|
||||
);
|
||||
if (logicalPathEscapesRoot(normalized))
|
||||
@@ -115,7 +113,7 @@ const resolveInternalSpecifier = (
|
||||
const stem = normalized.slice(0, -posix.extname(normalized).length);
|
||||
candidates.push(...sourceSuffixes.map((suffix) => `${stem}${suffix}`));
|
||||
}
|
||||
const match = lookupPath.resolveCandidates(candidates);
|
||||
const match = candidates.find((candidate) => filePaths.has(candidate));
|
||||
return match === undefined
|
||||
? { to: normalized, resolution: "unresolved" }
|
||||
: { to: match, resolution: "internal" };
|
||||
@@ -123,7 +121,7 @@ const resolveInternalSpecifier = (
|
||||
|
||||
const appendFile = (
|
||||
entry: Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
filePaths: ReadonlySet<string>,
|
||||
output: ParsedReferenceSourceEntries,
|
||||
): void => {
|
||||
const language = detectReferenceSourceLanguage(entry.path);
|
||||
@@ -148,7 +146,7 @@ const appendFile = (
|
||||
const resolved = resolveInternalSpecifier(
|
||||
relationship.from_path,
|
||||
relationship.to,
|
||||
lookupPath,
|
||||
filePaths,
|
||||
);
|
||||
output.relationships.push({
|
||||
...relationship,
|
||||
@@ -161,11 +159,11 @@ const appendFile = (
|
||||
|
||||
const appendReadEntry = (
|
||||
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
|
||||
lookupPath: ReferenceSourcePathLookup,
|
||||
filePaths: ReadonlySet<string>,
|
||||
output: ParsedReferenceSourceEntries,
|
||||
): void => {
|
||||
if (entry.kind === "file") {
|
||||
appendFile(entry, lookupPath, output);
|
||||
appendFile(entry, filePaths, output);
|
||||
return;
|
||||
}
|
||||
const classifications = classifyReferenceSourcePath(entry.path);
|
||||
@@ -182,14 +180,7 @@ const appendReadEntry = (
|
||||
output.entries.push({
|
||||
path: entry.path,
|
||||
kind: "symlink",
|
||||
// The reader resolves inventory identities from its root; the persisted
|
||||
// graph represents relative symlink targets from the link's directory.
|
||||
target:
|
||||
entry.targetState === "internal" ||
|
||||
(entry.targetState === "missing" &&
|
||||
!isPortableAbsoluteReferenceTarget(entry.target))
|
||||
? posix.relative(posix.dirname(entry.path), entry.target) || "."
|
||||
: entry.target,
|
||||
target: entry.target,
|
||||
target_state: entry.targetState,
|
||||
classifications,
|
||||
limitations: [],
|
||||
@@ -208,11 +199,10 @@ export const parseReferenceSourceEntries = (
|
||||
parseFailures: [],
|
||||
limitations: [...read.limitations],
|
||||
};
|
||||
const lookupPath = createReferenceSourcePathLookup(filePaths);
|
||||
for (const entry of read.entries) {
|
||||
if (signal?.aborted === true) break;
|
||||
if (entry.status === "failed") output.entries.push(failedEntry(entry));
|
||||
else appendReadEntry(entry, lookupPath, output);
|
||||
else appendReadEntry(entry, filePaths, output);
|
||||
}
|
||||
return output;
|
||||
};
|
||||
|
||||
@@ -1,22 +1,14 @@
|
||||
import { lstat, realpath, stat } from "node:fs/promises";
|
||||
import { readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
import ignore from "ignore";
|
||||
import {
|
||||
readRegularFileText,
|
||||
RegularFileCleanupFailure,
|
||||
retryRegularFileCleanup,
|
||||
} from "./RegularFileRead.js";
|
||||
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
||||
import {
|
||||
DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS,
|
||||
type ReferenceSourceImportError,
|
||||
type ReferenceSourceImportOptions,
|
||||
} from "./ReferenceSourceImportTypes.js";
|
||||
import { rootFilesystemFailure } from "../reference/ReferenceSourceReaderErrors.js";
|
||||
|
||||
/** Validated inputs ready for filesystem traversal. */
|
||||
export interface PreparedReferenceSourceImport {
|
||||
@@ -34,158 +26,54 @@ const failure = (
|
||||
message,
|
||||
});
|
||||
|
||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
||||
|
||||
const addMarkedPatterns = (
|
||||
matcher: ReturnType<typeof ignore>,
|
||||
patterns: readonly string[],
|
||||
mark: "project-ignored" | "default-ignored" | "caller-excluded",
|
||||
): void => {
|
||||
for (const pattern of patterns)
|
||||
for (const line of pattern.split(/\r?\n/u))
|
||||
matcher.add({ pattern: line, mark });
|
||||
};
|
||||
|
||||
const resolveRoot = async (
|
||||
requestedRoot: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<string, ReferenceSourceImportError>> => {
|
||||
if (isAborted(signal))
|
||||
return err(failure("cancelled", "Reference source import cancelled"));
|
||||
try {
|
||||
const metadata = await stat(requestedRoot);
|
||||
if (isAborted(signal))
|
||||
return err(failure("cancelled", "Reference source import cancelled"));
|
||||
if (!metadata.isDirectory())
|
||||
if (!(await stat(requestedRoot)).isDirectory())
|
||||
return err(
|
||||
failure(
|
||||
"invalid-root",
|
||||
`Reference source root is not a directory: ${requestedRoot}`,
|
||||
),
|
||||
failure("invalid-root", "Reference source root is not a directory"),
|
||||
);
|
||||
const canonicalRoot = await realpath(resolve(requestedRoot));
|
||||
if (isAborted(signal))
|
||||
return err(failure("cancelled", "Reference source import cancelled"));
|
||||
return ok(canonicalRoot);
|
||||
} catch (cause: unknown) {
|
||||
if (isAborted(signal))
|
||||
return err(failure("cancelled", "Reference source import cancelled"));
|
||||
const rootFailure = rootFilesystemFailure(
|
||||
cause,
|
||||
"Reference source root could not be resolved",
|
||||
void cause;
|
||||
return err(
|
||||
failure("invalid-root", "Reference source root could not be resolved"),
|
||||
);
|
||||
if (rootFailure === undefined) throw cause;
|
||||
return err({ ...failure(rootFailure.code, rootFailure.message), cause });
|
||||
}
|
||||
};
|
||||
|
||||
const buildIgnored = async (
|
||||
root: string,
|
||||
excludePaths: readonly string[],
|
||||
resources: ArtifactResourceScope,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<ReturnType<typeof ignore>, ReferenceSourceImportError>> => {
|
||||
): Promise<ReturnType<typeof ignore>> => {
|
||||
const ignored = ignore();
|
||||
const policyPath = join(root, ".gitignore");
|
||||
let present = true;
|
||||
try {
|
||||
await lstat(policyPath);
|
||||
ignored.add(await readFile(join(root, ".gitignore"), "utf8"));
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
!(
|
||||
typeof cause === "object" &&
|
||||
cause !== null &&
|
||||
"code" in cause &&
|
||||
cause.code === "ENOENT"
|
||||
)
|
||||
)
|
||||
throw cause;
|
||||
present = false;
|
||||
// Missing or unreadable ignore file does not authorize broader access.
|
||||
void cause;
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
if (present) {
|
||||
let text: string;
|
||||
try {
|
||||
text = await readRegularFileText(policyPath, { signal });
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
||||
const retry = await retryRegularFileCleanup(cause, resources);
|
||||
if (retry.cleanup !== undefined) {
|
||||
const primary =
|
||||
cause.outcome.kind === "failed"
|
||||
? cause.outcome.cause
|
||||
: cause.cleanupCause;
|
||||
return err({
|
||||
...failure(
|
||||
signal?.aborted === true ? "cancelled" : "io",
|
||||
`Reference source ignore policy could not be read at ${policyPath}: ${errorMessage(primary)}`,
|
||||
),
|
||||
cleanup: retry.cleanup,
|
||||
cause: primary,
|
||||
});
|
||||
}
|
||||
if (retry.outcome.kind === "failed") throw retry.outcome.cause;
|
||||
text = retry.outcome.value;
|
||||
}
|
||||
addMarkedPatterns(ignored, [text], "project-ignored");
|
||||
}
|
||||
addMarkedPatterns(
|
||||
ignored,
|
||||
DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS,
|
||||
"default-ignored",
|
||||
);
|
||||
ignored.add([...DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS]);
|
||||
for (const path of excludePaths) {
|
||||
addMarkedPatterns(ignored, [path, `${path}/`], "caller-excluded");
|
||||
ignored.add(path);
|
||||
ignored.add(`${path}/`);
|
||||
}
|
||||
return ok(ignored);
|
||||
return ignored;
|
||||
};
|
||||
|
||||
/** Resolve the caller-selected directory and build path filters. */
|
||||
export const prepareReferenceSourceImport = async (
|
||||
options: ReferenceSourceImportOptions,
|
||||
resources: ArtifactResourceScope,
|
||||
): Promise<
|
||||
Result<PreparedReferenceSourceImport, ReferenceSourceImportError>
|
||||
> => {
|
||||
try {
|
||||
return await resources.run(async () => {
|
||||
const root = await resolveRoot(options.root, options.signal);
|
||||
if (!root.ok) return root;
|
||||
try {
|
||||
const ignored = await buildIgnored(
|
||||
root.value,
|
||||
options.excludePaths ?? [],
|
||||
resources,
|
||||
options.signal,
|
||||
);
|
||||
if (!ignored.ok) return ignored;
|
||||
return ok({
|
||||
root: root.value,
|
||||
ignored: ignored.value,
|
||||
secrets: ignore().add([...options.policy.secretPatterns]),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err({
|
||||
...failure(
|
||||
options.signal?.aborted === true ? "cancelled" : "io",
|
||||
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${errorMessage(cause)}`,
|
||||
),
|
||||
cause,
|
||||
});
|
||||
}
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof ArtifactReaderFailure)) throw cause;
|
||||
return err({
|
||||
...failure(
|
||||
cause.reason === "cancelled" ? "cancelled" : "io",
|
||||
cause.message,
|
||||
),
|
||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
||||
cause,
|
||||
});
|
||||
}
|
||||
const root = await resolveRoot(options.root);
|
||||
if (!root.ok) return root;
|
||||
return ok({
|
||||
root: root.value,
|
||||
ignored: await buildIgnored(root.value, options.excludePaths ?? []),
|
||||
secrets: ignore().add([...options.policy.secretPatterns]),
|
||||
});
|
||||
};
|
||||
|
||||
const errorMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
projectReferenceSourceImportError,
|
||||
type ReferenceSourceImportError,
|
||||
} from "./ReferenceSourceImportTypes.js";
|
||||
import type { ReferenceSourceRead } from "../reference/ReferenceSourceReaderTypes.js";
|
||||
|
||||
describe("reference-source import error projection", () => {
|
||||
it("preserves primary evidence with category-specific recovery guidance", () => {
|
||||
const expectedCategories = {
|
||||
cancelled: "cancelled",
|
||||
"invalid-root": "invalid_input",
|
||||
unsupported: "unsupported_host",
|
||||
io: "execution_failure",
|
||||
parse: "execution_failure",
|
||||
} as const;
|
||||
for (const [code, category] of Object.entries(expectedCategories)) {
|
||||
const projected = projectReferenceSourceImportError({
|
||||
tag: "reference-source-import",
|
||||
code: code as ReferenceSourceImportError["code"],
|
||||
message: "EIO while reading /selected/source/src/main.ts",
|
||||
});
|
||||
expect(projected.category).toBe(category);
|
||||
expect(projected.message).toContain(
|
||||
"EIO while reading /selected/source/src/main.ts",
|
||||
);
|
||||
expect(projected.message).toMatch(
|
||||
/try again|when ready|Check that|REA on Linux/u,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves cancellation, partial facts, and cleanup diagnostics", () => {
|
||||
const bytes = Buffer.from("export const observed = true;\n");
|
||||
const partial: ReferenceSourceRead = {
|
||||
root: "/selected/source",
|
||||
entries: [
|
||||
{
|
||||
status: "read",
|
||||
kind: "file",
|
||||
path: "src/main.ts",
|
||||
bytes,
|
||||
size: bytes.length,
|
||||
},
|
||||
],
|
||||
bytesRead: bytes.length,
|
||||
limitations: [],
|
||||
};
|
||||
const projected = projectReferenceSourceImportError({
|
||||
tag: "reference-source-import",
|
||||
code: "cancelled",
|
||||
message: "Traversal cancelled after reading src/main.ts",
|
||||
cleanup: {
|
||||
reason: "directory close rejected",
|
||||
resources: ["/selected/source"],
|
||||
},
|
||||
partial,
|
||||
});
|
||||
|
||||
expect(projected).toMatchObject({
|
||||
category: "cancelled",
|
||||
message: expect.stringContaining(
|
||||
"Traversal cancelled after reading src/main.ts",
|
||||
),
|
||||
cleanup: {
|
||||
reason: "directory close rejected",
|
||||
resources: ["/selected/source"],
|
||||
},
|
||||
partial: {
|
||||
root: "/selected/source",
|
||||
entries: [{ path: "src/main.ts", bytes }],
|
||||
},
|
||||
});
|
||||
expect(projected.message).toContain("Cleanup failed during import");
|
||||
expect(projected.message).toContain("directory close rejected");
|
||||
expect(projected.message).toContain("/selected/source");
|
||||
expect(projected.message).not.toContain("remains unconfirmed");
|
||||
});
|
||||
});
|
||||
@@ -1,67 +1,44 @@
|
||||
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
|
||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
||||
import type { ReferenceSourceRead } from "../reference/ReferenceSourceReaderTypes.js";
|
||||
|
||||
/** Typed expected failure returned by historical-source imports. */
|
||||
export interface ReferenceSourceImportError {
|
||||
readonly tag: "reference-source-import";
|
||||
readonly code: "cancelled" | "invalid-root" | "unsupported" | "io" | "parse";
|
||||
readonly message: string;
|
||||
readonly cleanup?: AnalysisCleanupObservation;
|
||||
readonly partial?: ReferenceSourceRead;
|
||||
readonly cause?: unknown;
|
||||
}
|
||||
|
||||
const IMPORT_FAILURE_GUIDANCE = {
|
||||
cancelled: {
|
||||
category: "cancelled",
|
||||
message:
|
||||
"Reference source import was cancelled. Start it again when ready.",
|
||||
},
|
||||
"invalid-root": {
|
||||
category: "invalid_input",
|
||||
message:
|
||||
"Reference source directory could not be opened. Check that the path exists, is readable, and points to a directory.",
|
||||
},
|
||||
unsupported: {
|
||||
category: "unsupported_host",
|
||||
message:
|
||||
"Safe no-follow file opens are unavailable on this host. Import the source tree with REA on Linux (including WSL) or macOS.",
|
||||
},
|
||||
io: {
|
||||
category: "execution_failure",
|
||||
message:
|
||||
"Reference source files could not be read. Check directory permissions and try again.",
|
||||
},
|
||||
parse: {
|
||||
category: "execution_failure",
|
||||
message:
|
||||
"Reference source could not be indexed. Check that the source tree is readable, then try again.",
|
||||
},
|
||||
} satisfies Record<
|
||||
ReferenceSourceImportError["code"],
|
||||
{ readonly category: string; readonly message: string }
|
||||
>;
|
||||
|
||||
/** Safe CLI projection for a historical-source import failure. */
|
||||
export const projectReferenceSourceImportError = (
|
||||
error: ReferenceSourceImportError,
|
||||
): Readonly<{
|
||||
category: string;
|
||||
message: string;
|
||||
cleanup?: AnalysisCleanupObservation;
|
||||
partial?: ReferenceSourceRead;
|
||||
}> => {
|
||||
const primary = IMPORT_FAILURE_GUIDANCE[error.code];
|
||||
const cleanupMessage =
|
||||
error.cleanup === undefined
|
||||
? ""
|
||||
: ` Cleanup failed during import for ${error.cleanup.resources.join(", ")}: ${error.cleanup.reason}.`;
|
||||
): Readonly<{ category: string; message: string }> => {
|
||||
if (error.code === "cancelled")
|
||||
return {
|
||||
category: "cancelled",
|
||||
message:
|
||||
"Reference source import was cancelled. Start it again when ready.",
|
||||
};
|
||||
if (error.code === "invalid-root")
|
||||
return {
|
||||
category: "invalid_input",
|
||||
message:
|
||||
"Reference source directory could not be opened. Check that the path exists, is readable, and points to a directory.",
|
||||
};
|
||||
if (error.code === "unsupported")
|
||||
return {
|
||||
category: "unsupported_host",
|
||||
message:
|
||||
"Safe no-follow file opens are unavailable on this host. Import the source tree with REA on Linux (including WSL) or macOS.",
|
||||
};
|
||||
if (error.code === "io")
|
||||
return {
|
||||
category: "execution_failure",
|
||||
message:
|
||||
"Reference source files could not be read. Check directory permissions and try again.",
|
||||
};
|
||||
return {
|
||||
category: primary.category,
|
||||
message: `${error.message} ${primary.message}${cleanupMessage}`,
|
||||
...(error.cleanup === undefined ? {} : { cleanup: error.cleanup }),
|
||||
...(error.partial === undefined ? {} : { partial: error.partial }),
|
||||
category: "execution_failure",
|
||||
message:
|
||||
"Reference source could not be indexed. Check that the source tree is readable, then try again.",
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -1,15 +1,6 @@
|
||||
import fs from "node:fs";
|
||||
import { lstat } from "node:fs/promises";
|
||||
import { lstat, readFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
||||
import {
|
||||
readRegularFile,
|
||||
RegularFileCleanupFailure,
|
||||
retryRegularFileCleanup,
|
||||
} from "./RegularFileRead.js";
|
||||
import type { ReferenceSourceReaderError } from "../reference/ReferenceSourceReaderTypes.js";
|
||||
|
||||
export type ReferenceSourceVcsInfo =
|
||||
| {
|
||||
@@ -27,174 +18,34 @@ export type ReferenceSourceVcsInfo =
|
||||
*/
|
||||
export const readReferenceSourceVcs = async (
|
||||
root: string,
|
||||
resources: ArtifactResourceScope,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<ReferenceSourceVcsInfo, ReferenceSourceReaderError>> => {
|
||||
): Promise<ReferenceSourceVcsInfo> => {
|
||||
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
|
||||
try {
|
||||
return await resources.run(async () => {
|
||||
if (isAborted(signal)) return ok(unknownVcs());
|
||||
try {
|
||||
await lstat(join(root, ".git"));
|
||||
} catch (cause: unknown) {
|
||||
return ok(
|
||||
errorCode(cause) === "ENOENT"
|
||||
? { kind: "none", head: null, dirty: null }
|
||||
: unknownVcs(),
|
||||
);
|
||||
}
|
||||
try {
|
||||
const head = await resolveSourceHead(root, resources, signal);
|
||||
if (isAborted(signal)) return ok(unknownVcs());
|
||||
return ok({ kind: "git", head, dirty: null });
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof ArtifactReaderFailure) throw cause;
|
||||
return ok(unknownVcs());
|
||||
}
|
||||
});
|
||||
await lstat(join(root, ".git"));
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof ArtifactReaderFailure)) throw cause;
|
||||
return err({
|
||||
tag: "reference-source-reader",
|
||||
code: cause.reason === "cancelled" ? "cancelled" : "io",
|
||||
message: cause.message,
|
||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
||||
cause,
|
||||
});
|
||||
return errorCode(cause) === "ENOENT"
|
||||
? { kind: "none", head: null, dirty: null }
|
||||
: { kind: "unknown", head: null, dirty: null };
|
||||
}
|
||||
try {
|
||||
const head = await resolveSourceHead(root);
|
||||
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
|
||||
return { kind: "git", head, dirty: null };
|
||||
} catch (cause: unknown) {
|
||||
// Unresolvable refs mean VCS state is unknown, not absent.
|
||||
void cause;
|
||||
return { kind: "unknown", head: null, dirty: null };
|
||||
}
|
||||
};
|
||||
|
||||
const unknownVcs = (): ReferenceSourceVcsInfo => ({
|
||||
kind: "unknown",
|
||||
head: null,
|
||||
dirty: null,
|
||||
});
|
||||
|
||||
const resolveSourceHead = async (
|
||||
root: string,
|
||||
resources: ArtifactResourceScope,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const resolveSourceHead = async (root: string): Promise<string> => {
|
||||
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
|
||||
const { resolveRef } = await import("isomorphic-git");
|
||||
let failedRead: { readonly cause: unknown } | undefined;
|
||||
const cleanupFailures: ArtifactReaderFailure[] = [];
|
||||
const readMetadataBytes = async (path: string): Promise<Buffer> => {
|
||||
try {
|
||||
return await readRegularFile(path, { signal });
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
||||
const retry = await retryRegularFileCleanup(cause, resources);
|
||||
if (
|
||||
retry.outcome.kind === "failed" &&
|
||||
errorCode(retry.outcome.cause) !== "ENOENT"
|
||||
)
|
||||
failedRead ??= { cause: retry.outcome.cause };
|
||||
if (
|
||||
retry.cleanup === undefined &&
|
||||
retry.outcome.kind === "failed" &&
|
||||
!isAborted(signal)
|
||||
)
|
||||
throw retry.outcome.cause;
|
||||
const primary =
|
||||
retry.outcome.kind === "failed"
|
||||
? retry.outcome.cause
|
||||
: cause.cleanupCause;
|
||||
throw new ArtifactReaderFailure(
|
||||
isAborted(signal) ? "cancelled" : "io",
|
||||
`Git metadata could not be read safely: ${errorMessage(primary)}`,
|
||||
{
|
||||
cause: primary,
|
||||
...(retry.cleanup === undefined ? {} : { cleanup: retry.cleanup }),
|
||||
},
|
||||
);
|
||||
}
|
||||
};
|
||||
const readMetadataText = async (path: string): Promise<string> =>
|
||||
(await readMetadataBytes(path)).toString("utf8");
|
||||
const throwCleanupFailures = (primary: unknown): never => {
|
||||
const observations = cleanupFailures.flatMap((failure) =>
|
||||
failure.cleanup === undefined ? [] : [failure.cleanup],
|
||||
);
|
||||
throw new ArtifactReaderFailure(
|
||||
isAborted(signal) ? "cancelled" : "io",
|
||||
`Git metadata could not be read safely: ${errorMessage(primary)}`,
|
||||
{
|
||||
cause: new AggregateError(
|
||||
[primary, ...cleanupFailures],
|
||||
"Git metadata reads and cleanup failed",
|
||||
{ cause: primary },
|
||||
),
|
||||
...(observations.length === 0
|
||||
? {}
|
||||
: {
|
||||
cleanup: {
|
||||
reason: observations.map(({ reason }) => reason).join("; "),
|
||||
resources: [
|
||||
...new Set(
|
||||
observations.flatMap(({ resources }) => resources),
|
||||
),
|
||||
],
|
||||
},
|
||||
}),
|
||||
},
|
||||
);
|
||||
};
|
||||
const referenceFs = {
|
||||
...fs,
|
||||
promises: {
|
||||
...fs.promises,
|
||||
// Let isomorphic-git continue from a failed loose ref to packed-refs,
|
||||
// then report the original read failure after resolution. This preserves
|
||||
// both observations when packed metadata also fails cleanup.
|
||||
readFile: async (
|
||||
path: string | undefined,
|
||||
options:
|
||||
| BufferEncoding
|
||||
| { readonly encoding?: BufferEncoding | null }
|
||||
| null = null,
|
||||
) => {
|
||||
// The library probes readFile() without a path to detect promise APIs.
|
||||
// Reject that invocation without attributing it to a source read.
|
||||
if (path === undefined)
|
||||
throw new TypeError("Git filesystem read requires a path");
|
||||
try {
|
||||
const bytes = await readMetadataBytes(path);
|
||||
const encoding =
|
||||
typeof options === "string" ? options : options?.encoding;
|
||||
return encoding === undefined || encoding === null
|
||||
? bytes
|
||||
: bytes.toString(encoding);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof ArtifactReaderFailure) {
|
||||
cleanupFailures.push(cause);
|
||||
} else if (errorCode(cause) !== "ENOENT") failedRead ??= { cause };
|
||||
throw cause;
|
||||
}
|
||||
},
|
||||
},
|
||||
};
|
||||
const readRef = async (options: {
|
||||
readonly gitdir?: string;
|
||||
readonly ref: string;
|
||||
}) => {
|
||||
let head: string;
|
||||
try {
|
||||
head = await resolveRef({ fs: referenceFs, dir: root, ...options });
|
||||
} catch (cause: unknown) {
|
||||
if (cleanupFailures.length > 0)
|
||||
throwCleanupFailures(failedRead?.cause ?? cause);
|
||||
if (failedRead !== undefined) throw failedRead.cause;
|
||||
throw cause;
|
||||
}
|
||||
if (cleanupFailures.length > 0)
|
||||
throwCleanupFailures(failedRead?.cause ?? cleanupFailures[0]?.cause);
|
||||
signal?.throwIfAborted();
|
||||
if (failedRead !== undefined) throw failedRead.cause;
|
||||
return head;
|
||||
};
|
||||
const dotgit = join(root, ".git");
|
||||
if (!(await lstat(dotgit)).isFile()) return readRef({ ref: "HEAD" });
|
||||
const pointer = (await readMetadataText(dotgit)).replace(/\r?\n$/u, "");
|
||||
if (!(await lstat(dotgit)).isFile())
|
||||
return resolveRef({ fs, dir: root, ref: "HEAD" });
|
||||
const pointer = (await readFile(dotgit, "utf8")).replace(/\r?\n$/u, "");
|
||||
if (!pointer.startsWith("gitdir: ") || pointer.length === 8)
|
||||
throw new Error("Invalid Git directory pointer");
|
||||
const gitdir = resolve(root, pointer.slice(8));
|
||||
@@ -202,22 +53,25 @@ const resolveSourceHead = async (
|
||||
try {
|
||||
commonDirectory = resolve(
|
||||
gitdir,
|
||||
(await readMetadataText(join(gitdir, "commondir"))).replace(
|
||||
(await readFile(join(gitdir, "commondir"), "utf8")).replace(
|
||||
/\r?\n$/u,
|
||||
"",
|
||||
),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
if (errorCode(cause) !== "ENOENT") throw cause;
|
||||
return readRef({ gitdir, ref: "HEAD" });
|
||||
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||
}
|
||||
const head = (await readMetadataText(join(gitdir, "HEAD"))).trim();
|
||||
if (!head.startsWith("ref: ")) return readRef({ gitdir, ref: "HEAD" });
|
||||
const head = (await readFile(join(gitdir, "HEAD"), "utf8")).trim();
|
||||
if (!head.startsWith("ref: "))
|
||||
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||
const ref = head.slice(5);
|
||||
const privateRef = ["refs/bisect/", "refs/rewritten/", "refs/worktree/"].some(
|
||||
(prefix) => ref.startsWith(prefix),
|
||||
);
|
||||
return readRef({
|
||||
return resolveRef({
|
||||
fs,
|
||||
dir: root,
|
||||
gitdir: privateRef ? gitdir : commonDirectory,
|
||||
ref,
|
||||
});
|
||||
@@ -228,7 +82,4 @@ const errorCode = (cause: unknown): string | undefined =>
|
||||
? String(cause.code)
|
||||
: undefined;
|
||||
|
||||
const errorMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
||||
|
||||
@@ -1,158 +1,29 @@
|
||||
import type { Stats } from "node:fs";
|
||||
import { lstat, stat, type FileHandle } from "node:fs/promises";
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
import {
|
||||
RegularFileAdmissionFailure,
|
||||
openRegularFile,
|
||||
sameRegularFileState,
|
||||
RegularFileChangedError,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
||||
|
||||
/** Filesystem admission policy for one selected regular input. */
|
||||
export interface RegularFileReadOptions {
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
readonly symlinks?: "follow" | "reject";
|
||||
}
|
||||
|
||||
export type RegularFileReadOutcome<Value> =
|
||||
| { readonly kind: "completed"; readonly value: Value }
|
||||
| { readonly kind: "failed"; readonly cause: unknown };
|
||||
|
||||
/** A close failed after reading; this error transfers the exact handle owner. */
|
||||
export class RegularFileCleanupFailure<Value = unknown> extends Error {
|
||||
constructor(
|
||||
readonly path: string,
|
||||
readonly owner: OwnedFileHandle,
|
||||
readonly outcome: RegularFileReadOutcome<Value>,
|
||||
readonly cleanupCause: unknown,
|
||||
) {
|
||||
super(`Regular file cleanup failed: ${path}`, {
|
||||
cause: outcome.kind === "failed" ? outcome.cause : cleanupCause,
|
||||
});
|
||||
this.name = "RegularFileCleanupFailure";
|
||||
}
|
||||
}
|
||||
|
||||
/** Retry a failed close through its retained owner and report any remaining cleanup. */
|
||||
export const retryRegularFileCleanup = async <Value>(
|
||||
failure: RegularFileCleanupFailure<Value>,
|
||||
resources: ArtifactResourceScope,
|
||||
): Promise<{
|
||||
readonly outcome: RegularFileReadOutcome<Value>;
|
||||
readonly cleanup?: AnalysisCleanupObservation;
|
||||
}> => {
|
||||
const released = await resources.release({
|
||||
kind: "file-handle",
|
||||
handle: failure.owner,
|
||||
resource: failure.path,
|
||||
});
|
||||
if (released.kind === "released") return { outcome: failure.outcome };
|
||||
return {
|
||||
outcome: failure.outcome,
|
||||
cleanup: {
|
||||
reason: `${failureMessage(failure.cleanupCause)}; retry: ${failureMessage(released.cause)}`,
|
||||
resources: [failure.path],
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const failureMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
import { openRegularFile } from "../filesystem/RegularFile.js";
|
||||
|
||||
/** Admit one regular-file handle, retain it through the consumer, and always close it. */
|
||||
export const withRegularFile = async <Value>(
|
||||
path: string,
|
||||
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
|
||||
options: RegularFileReadOptions = {},
|
||||
signal?: AbortSignal,
|
||||
): Promise<Value> => {
|
||||
const { signal } = options;
|
||||
let handle: FileHandle;
|
||||
try {
|
||||
handle = await openRegularFile(path, {
|
||||
symlinks: options.symlinks ?? "follow",
|
||||
signal,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof RegularFileAdmissionFailure)) throw cause;
|
||||
throw new RegularFileCleanupFailure(
|
||||
path,
|
||||
cause.owner,
|
||||
{ kind: "failed", cause: cause.cause },
|
||||
cause.cleanupCause,
|
||||
);
|
||||
}
|
||||
const owner = new OwnedFileHandle(handle);
|
||||
let outcome: RegularFileReadOutcome<Value>;
|
||||
const handle = await openRegularFile(path, { symlinks: "follow", signal });
|
||||
try {
|
||||
const stats = await handle.stat();
|
||||
signal?.throwIfAborted();
|
||||
const value = await read(handle, stats);
|
||||
signal?.throwIfAborted();
|
||||
const [opened, currentPath] = await Promise.all([
|
||||
handle.stat(),
|
||||
options.symlinks === "reject" ? lstat(path) : stat(path),
|
||||
]);
|
||||
if (
|
||||
!sameRegularFileState(stats, opened) ||
|
||||
!sameRegularFileState(stats, currentPath)
|
||||
)
|
||||
throw new RegularFileChangedError(path);
|
||||
outcome = { kind: "completed", value };
|
||||
} catch (cause: unknown) {
|
||||
outcome = { kind: "failed", cause };
|
||||
return value;
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
try {
|
||||
await owner.close();
|
||||
} catch (cleanupCause: unknown) {
|
||||
throw new RegularFileCleanupFailure(path, owner, outcome, cleanupCause);
|
||||
}
|
||||
if (outcome.kind === "failed") throw outcome.cause;
|
||||
return outcome.value;
|
||||
};
|
||||
|
||||
/** Admit a regular file without waiting for a pipe, then read its verified handle. */
|
||||
export const readRegularFile = (
|
||||
path: string,
|
||||
options: RegularFileReadOptions = {},
|
||||
signal?: AbortSignal,
|
||||
): Promise<Buffer> =>
|
||||
withRegularFile(
|
||||
path,
|
||||
async (handle, stats) => {
|
||||
const bytes = await readBoundedFileBytes(
|
||||
handle,
|
||||
stats.size,
|
||||
options.signal,
|
||||
);
|
||||
if (bytes === undefined || bytes.length !== stats.size)
|
||||
throw new RegularFileChangedError(path);
|
||||
return bytes;
|
||||
},
|
||||
options,
|
||||
);
|
||||
|
||||
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
|
||||
export const readRegularFileText = async (
|
||||
path: string,
|
||||
options: RegularFileReadOptions = {},
|
||||
): Promise<string> => {
|
||||
try {
|
||||
return (await readRegularFile(path, options)).toString("utf8");
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
cause instanceof RegularFileCleanupFailure &&
|
||||
cause.outcome.kind === "completed" &&
|
||||
Buffer.isBuffer(cause.outcome.value)
|
||||
)
|
||||
throw new RegularFileCleanupFailure(
|
||||
cause.path,
|
||||
cause.owner,
|
||||
{ kind: "completed", value: cause.outcome.value.toString("utf8") },
|
||||
cause.cleanupCause,
|
||||
);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
withRegularFile(path, (handle) => handle.readFile({ signal }), signal);
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
|
||||
import { FakeSetupHost, options } from "./Setup.fixture.js";
|
||||
import { runSetup } from "./Setup.js";
|
||||
@@ -110,76 +107,6 @@ describe("setup workflow", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("setup failure cleanup projection", () => {
|
||||
it("projects a configuration failure together with failed host cleanup", async () => {
|
||||
const configPath = "/fixture/home/vscode/settings.json";
|
||||
const primary = new ConfigurationError("Client config could not be read", {
|
||||
settings: [
|
||||
{
|
||||
setting: configPath,
|
||||
constraint: "The selected config could not be read.",
|
||||
},
|
||||
],
|
||||
cleanup: {
|
||||
reason: "The admitted config handle remains open.",
|
||||
resources: [configPath],
|
||||
},
|
||||
});
|
||||
class SetupHostWithCleanupFailure extends FakeSetupHost {
|
||||
close = async (): Promise<string> =>
|
||||
"Setup could not release its client configuration resources.";
|
||||
}
|
||||
const host = new SetupHostWithCleanupFailure();
|
||||
host.clients = [{ name: "codex", configPath }];
|
||||
host.inspectClientConfiguration = async () => {
|
||||
throw primary;
|
||||
};
|
||||
|
||||
const failure = await runSetup(
|
||||
{ ...options(true), clientIds: ["codex"] },
|
||||
host,
|
||||
).then(
|
||||
() => undefined,
|
||||
(cause: unknown) => cause,
|
||||
);
|
||||
expect(failure).toBeInstanceOf(AnalysisError);
|
||||
if (!(failure instanceof AnalysisError))
|
||||
throw new Error("Expected setup to preserve a typed analysis failure");
|
||||
|
||||
const projected = projectAnalysisError(failure);
|
||||
expect(projected).toMatchObject({
|
||||
code: "cleanup_incomplete",
|
||||
details: {
|
||||
cleanup: "incomplete",
|
||||
resources: [configPath],
|
||||
diagnostics: {
|
||||
primary_error: {
|
||||
code: "cleanup_incomplete",
|
||||
details: { resources: [configPath] },
|
||||
},
|
||||
cleanup_error: {
|
||||
code: "cleanup_incomplete",
|
||||
details: { resources: [configPath] },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(projected.details).toMatchObject({
|
||||
diagnostics: {
|
||||
primary_error: { message: expect.stringContaining("selected config") },
|
||||
cleanup_error: {
|
||||
details: {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"Setup could not release its client configuration resources.",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("setup scoped readiness", () => {
|
||||
it("limits Hopper host checks to workflows that request Hopper", async () => {
|
||||
const platformHost = new FakeSetupHost("win32");
|
||||
|
||||
@@ -23,69 +23,12 @@ import {
|
||||
hostRemediation,
|
||||
systemSetupHost,
|
||||
} from "./SetupHost.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
|
||||
/** Discover, approve, and apply setup actions idempotently. */
|
||||
export const runSetup = async (
|
||||
options: SetupOptions,
|
||||
host: SetupHost = systemSetupHost(),
|
||||
confirm?: SetupConfirmation,
|
||||
): Promise<SetupResult> => {
|
||||
let result: SetupResult | undefined;
|
||||
let primaryFailure: unknown;
|
||||
let threw = false;
|
||||
try {
|
||||
result = await runSetupWorkflow(options, host, confirm);
|
||||
} catch (cause: unknown) {
|
||||
threw = true;
|
||||
primaryFailure = cause;
|
||||
}
|
||||
|
||||
let cleanupFailure: string | undefined;
|
||||
try {
|
||||
cleanupFailure = await host.close?.();
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = `Setup resource cleanup failed: ${failureMessage(cause)}`;
|
||||
}
|
||||
|
||||
if (threw) {
|
||||
if (cleanupFailure !== undefined && primaryFailure instanceof AnalysisError)
|
||||
throw analysisErrorWithCleanupFailure(
|
||||
primaryFailure,
|
||||
new ProviderCleanupError(
|
||||
"setup",
|
||||
primaryFailure.cleanupResources,
|
||||
{ reason: cleanupFailure },
|
||||
{ operation: "setup" },
|
||||
),
|
||||
"setup",
|
||||
);
|
||||
if (cleanupFailure !== undefined)
|
||||
throw new AggregateError(
|
||||
[primaryFailure, new Error(cleanupFailure)],
|
||||
"Setup and resource cleanup failed.",
|
||||
{ cause: primaryFailure },
|
||||
);
|
||||
throw primaryFailure;
|
||||
}
|
||||
if (result === undefined) throw new Error("Setup returned no result.");
|
||||
if (cleanupFailure === undefined) return result;
|
||||
return {
|
||||
...result,
|
||||
status: "needs_human",
|
||||
code: result.code ?? "cleanup_failed",
|
||||
remediation: result.remediation
|
||||
? `${result.remediation} ${cleanupFailure}`
|
||||
: cleanupFailure,
|
||||
};
|
||||
};
|
||||
|
||||
const runSetupWorkflow = async (
|
||||
options: SetupOptions,
|
||||
host: SetupHost,
|
||||
confirm?: SetupConfirmation,
|
||||
): Promise<SetupResult> => {
|
||||
const appliedActions: string[] = [];
|
||||
let plannedActions: readonly SetupAction[] = [];
|
||||
@@ -116,7 +59,6 @@ const runSetupWorkflow = async (
|
||||
options.readinessScope?.clients === undefined &&
|
||||
options.allDetectedClients !== true &&
|
||||
options.installHopper !== true &&
|
||||
options.skillClientIds === undefined &&
|
||||
options.installSkill === undefined;
|
||||
const interactiveSelection =
|
||||
confirm !== undefined && !options.structured && clientSelectionAllowed;
|
||||
@@ -125,12 +67,10 @@ const runSetupWorkflow = async (
|
||||
: (options.clientIds ??
|
||||
options.readinessScope?.clients ??
|
||||
discovery.defaultClientIds);
|
||||
let selectedSkillClientIds = options.skillClientIds ?? selectedClientIds;
|
||||
let skillNeedsInstall = await host.skillNeedsInstall(selectedSkillClientIds);
|
||||
let skillNeedsInstall = await host.skillNeedsInstall(selectedClientIds);
|
||||
let skillSelected =
|
||||
options.installSkill === true ||
|
||||
(options.installSkill !== false &&
|
||||
(selectedClientIds.length > 0 || selectedSkillClientIds.length > 0));
|
||||
(options.installSkill !== false && selectedClientIds.length > 0);
|
||||
let installSkill = skillSelected && skillNeedsInstall;
|
||||
let planDiscovery = discovery;
|
||||
if (interactiveSelection) {
|
||||
@@ -198,7 +138,6 @@ const runSetupWorkflow = async (
|
||||
selectedClientIds.length === 0) ||
|
||||
(options.installSkill !== false && selectedClientIds.length > 0);
|
||||
installSkill = skillSelected && skillNeedsInstall;
|
||||
selectedSkillClientIds = selectedClientIds;
|
||||
}
|
||||
const planned = await planSetupActions({
|
||||
discovery: planDiscovery,
|
||||
@@ -206,7 +145,7 @@ const runSetupWorkflow = async (
|
||||
providerEnvironment,
|
||||
command: host.registrationCommand,
|
||||
clientIds: selectedClientIds,
|
||||
skillClientIds: selectedSkillClientIds,
|
||||
skillClientIds: selectedClientIds,
|
||||
installSkill,
|
||||
});
|
||||
plannedActions = planned.plannedActions;
|
||||
@@ -317,7 +256,7 @@ const runSetupWorkflow = async (
|
||||
host,
|
||||
options,
|
||||
appliedActions,
|
||||
selectedSkillClientIds,
|
||||
selectedClientIds,
|
||||
))
|
||||
)
|
||||
return fail(
|
||||
@@ -386,11 +325,8 @@ const installHopperAction = async (input: {
|
||||
emitProgress(input.options, {
|
||||
actionId: "install_hopper",
|
||||
label,
|
||||
state: installed.cleanupFailure === undefined ? "completed" : "warning",
|
||||
detail:
|
||||
installed.cleanupFailure === undefined
|
||||
? installed.launcherPath
|
||||
: `${installed.launcherPath}; ${installed.cleanupFailure}`,
|
||||
state: "completed",
|
||||
detail: installed.launcherPath,
|
||||
});
|
||||
const providerEnvironment = {
|
||||
...input.providerEnvironment,
|
||||
@@ -556,6 +492,3 @@ const summarizeDoctor = (report: DoctorReport): SetupDoctorSummary => ({
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
const failureMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
@@ -11,22 +11,17 @@ import {
|
||||
serializeClientConfiguration,
|
||||
withClientServers,
|
||||
type ClientConfigurationDocument,
|
||||
type ClientRegistrationDialect,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import writeFileAtomic from "write-file-atomic";
|
||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||
import {
|
||||
readRegularFileText,
|
||||
RegularFileCleanupFailure,
|
||||
} from "./RegularFileRead.js";
|
||||
import type { ClientConfigurationFileError } from "./ClientConfigurationFiles.js";
|
||||
import type {
|
||||
ClientConfigurationInspection,
|
||||
ClientConfigurationResult,
|
||||
@@ -42,37 +37,25 @@ export const configureClientConfiguration = (
|
||||
): Promise<ClientConfigurationResult> => {
|
||||
if (client.configPathError !== undefined)
|
||||
return Promise.resolve({ status: "failed", reason: "path" });
|
||||
return configureClientDocument(client, environment, command);
|
||||
if (client.format === undefined || client.format === "unsupported")
|
||||
return Promise.resolve({ status: "failed", reason: "readback" });
|
||||
return configureClientDocument(client, environment, command, client.format);
|
||||
};
|
||||
|
||||
const configureClientDocument = async (
|
||||
client: SetupClient,
|
||||
environment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
format: NonNullable<SetupClient["format"]>,
|
||||
): Promise<ClientConfigurationResult> => {
|
||||
const format = client.format;
|
||||
if (format === undefined || format === "unsupported")
|
||||
return { status: "failed", reason: "readback" };
|
||||
const files = await readClientConfigurationFiles(client);
|
||||
if (!files.ok) {
|
||||
throwCleanupFailure(files.error);
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
if (!files.ok) return { status: "failed", reason: "readback" };
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
undefined,
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
environment,
|
||||
command,
|
||||
files.value.at(-1),
|
||||
),
|
||||
clientConfigurationDesired(client, environment, command, client.format),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
if (policy.value !== undefined)
|
||||
if (!policy.ok || policy.value !== undefined)
|
||||
return { status: "failed", reason: "readback" };
|
||||
const transactionPath = await resolveClientConfigTransactionPath(
|
||||
client.configPath,
|
||||
@@ -81,9 +64,8 @@ const configureClientDocument = async (
|
||||
return { status: "failed", reason: "path" };
|
||||
let original: string | undefined;
|
||||
try {
|
||||
original = await readRegularFileText(transactionPath);
|
||||
original = await readFile(transactionPath, "utf8");
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
if (!isMissing(cause)) return { status: "failed", reason: "readback" };
|
||||
}
|
||||
let parsed: ClientConfigurationDocument;
|
||||
@@ -95,16 +77,11 @@ const configureClientDocument = async (
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, environment, command, parsed),
|
||||
clientConfigurationDesired(client, environment, command, parsed.dialect),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
if (policy.value !== undefined)
|
||||
if (!policy.ok || policy.value !== undefined)
|
||||
return { status: "failed", reason: "readback" };
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
// Malformed existing configuration fails the readback gate.
|
||||
void cause;
|
||||
return { status: "failed", reason: "readback" };
|
||||
@@ -113,36 +90,9 @@ const configureClientDocument = async (
|
||||
client,
|
||||
environment,
|
||||
command,
|
||||
parsed,
|
||||
parsed.dialect,
|
||||
);
|
||||
if (registrationCurrent(parsed, desired)) return { status: "unchanged" };
|
||||
return persistClientConfiguration({
|
||||
client,
|
||||
transactionPath,
|
||||
format,
|
||||
original,
|
||||
parsed,
|
||||
desired,
|
||||
});
|
||||
};
|
||||
|
||||
interface ClientConfigurationUpdate {
|
||||
readonly client: SetupClient;
|
||||
readonly transactionPath: string;
|
||||
readonly format: NonNullable<SetupClient["format"]>;
|
||||
readonly original: string | undefined;
|
||||
readonly parsed: ClientConfigurationDocument;
|
||||
readonly desired: unknown;
|
||||
}
|
||||
|
||||
const persistClientConfiguration = async ({
|
||||
client,
|
||||
transactionPath,
|
||||
format,
|
||||
original,
|
||||
parsed,
|
||||
desired,
|
||||
}: ClientConfigurationUpdate): Promise<ClientConfigurationResult> => {
|
||||
const backupPath =
|
||||
original === undefined ? undefined : `${client.configPath}.rea.backup`;
|
||||
if (
|
||||
@@ -199,7 +149,7 @@ const persistClientConfiguration = async ({
|
||||
}
|
||||
try {
|
||||
const readback = parseClientConfiguration(
|
||||
await readRegularFileText(transactionPath),
|
||||
await readFile(transactionPath, "utf8"),
|
||||
format,
|
||||
);
|
||||
if (!registrationCurrent(readback, desired)) {
|
||||
@@ -207,7 +157,6 @@ const persistClientConfiguration = async ({
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
// Readback failure restores the transaction before reporting.
|
||||
void cause;
|
||||
await restoreConfig(transactionPath, original);
|
||||
@@ -228,22 +177,21 @@ export const clientConfigurationAligned = async (
|
||||
if (client.configPathError !== undefined) return false;
|
||||
try {
|
||||
const files = await readClientConfigurationFiles(client);
|
||||
if (!files.ok) {
|
||||
throwCleanupFailure(files.error);
|
||||
return false;
|
||||
}
|
||||
const original = await readRegularFileText(client.configPath);
|
||||
if (!files.ok) return false;
|
||||
const original = await readFile(client.configPath, "utf8");
|
||||
const parsed = parseClientConfiguration(original, client.format);
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed.dialect,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
policy.ok &&
|
||||
policy.value === undefined &&
|
||||
registrationCurrent(
|
||||
parsed,
|
||||
@@ -251,12 +199,11 @@ export const clientConfigurationAligned = async (
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
parsed.dialect,
|
||||
),
|
||||
)
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
// Unreadable configuration is treated as not aligned so setup repairs it.
|
||||
void cause;
|
||||
return false;
|
||||
@@ -273,13 +220,7 @@ export const inspectClientConfiguration = async (
|
||||
return { status: "invalid", remediation: client.configPathError };
|
||||
if (client.format === "unsupported") return { status: "already_current" };
|
||||
const files = await readClientConfigurationFiles(client);
|
||||
if (!files.ok) {
|
||||
throwCleanupFailure(files.error);
|
||||
return {
|
||||
status: "invalid",
|
||||
remediation: files.error.detail,
|
||||
};
|
||||
}
|
||||
if (!files.ok) return { status: "invalid", remediation: files.error.detail };
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
undefined,
|
||||
@@ -287,16 +228,11 @@ export const inspectClientConfiguration = async (
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
files.value.at(-1),
|
||||
client.format,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
return {
|
||||
status: "invalid",
|
||||
remediation: policy.error.detail,
|
||||
};
|
||||
}
|
||||
if (!policy.ok)
|
||||
return { status: "invalid", remediation: policy.error.detail };
|
||||
if (policy.value !== undefined)
|
||||
return { status: "invalid", remediation: policy.value };
|
||||
const transactionPath = await resolveClientConfigTransactionPath(
|
||||
@@ -310,13 +246,13 @@ export const inspectClientConfiguration = async (
|
||||
};
|
||||
let original: string;
|
||||
try {
|
||||
original = await readRegularFileText(transactionPath);
|
||||
original = await readFile(transactionPath, "utf8");
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
if (isMissing(cause)) return { status: "create" };
|
||||
return {
|
||||
status: "invalid",
|
||||
remediation: `The configuration file could not be read. Check its permissions before rerunning setup. ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
remediation:
|
||||
"The configuration file could not be read. Check its permissions before rerunning setup.",
|
||||
};
|
||||
}
|
||||
try {
|
||||
@@ -324,27 +260,26 @@ export const inspectClientConfiguration = async (
|
||||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed.dialect,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
return {
|
||||
status: "invalid",
|
||||
remediation: policy.error.detail,
|
||||
};
|
||||
}
|
||||
if (!policy.ok)
|
||||
return { status: "invalid", remediation: policy.error.detail };
|
||||
if (policy.value !== undefined)
|
||||
return { status: "invalid", remediation: policy.value };
|
||||
const desired = clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
parsed.dialect,
|
||||
);
|
||||
if (registrationCurrent(parsed, desired))
|
||||
return { status: "already_current" };
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
// Malformed configuration is reported with the invalid remediation.
|
||||
void cause;
|
||||
return {
|
||||
@@ -361,11 +296,6 @@ export const inspectClientConfiguration = async (
|
||||
|
||||
const isMissing = (cause: unknown): boolean =>
|
||||
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
|
||||
|
||||
const throwCleanupFailure = (error: ClientConfigurationFileError): void => {
|
||||
if (error.cleanupFailure !== undefined) throw error.cleanupFailure;
|
||||
};
|
||||
|
||||
const preserveConfigBackup = async (
|
||||
source: string,
|
||||
destination: string,
|
||||
@@ -407,20 +337,17 @@ const clientConfigurationDesired = (
|
||||
client: SetupClient,
|
||||
providerEnvironment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
parsed?: ClientConfigurationDocument,
|
||||
format: ClientRegistrationDialect | undefined,
|
||||
) => {
|
||||
const format = parsed?.dialect ?? client.format ?? "json";
|
||||
// Preserve explicitly configured server settings, never the ambient process
|
||||
// environment. Newly discovered provider paths win only for their own keys.
|
||||
const environment = clientRegistrationEnvironment(
|
||||
parsed,
|
||||
providerEnvironment,
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(providerEnvironment).sort(([left], [right]) =>
|
||||
left.localeCompare(right),
|
||||
),
|
||||
);
|
||||
if (!environment.ok) throw environment.error;
|
||||
const registration = clientRegistrationEntry(
|
||||
format,
|
||||
format ?? "json",
|
||||
command.length === 0 ? [PRODUCT_IDENTITY.cliBinary, "mcp"] : command,
|
||||
environment.value,
|
||||
environment,
|
||||
);
|
||||
return {
|
||||
...registration,
|
||||
|
||||
+20
-141
@@ -1,9 +1,6 @@
|
||||
import { access } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||
import {
|
||||
@@ -33,10 +30,6 @@ import {
|
||||
inspectClientConfiguration,
|
||||
} from "./SetupClientConfiguration.js";
|
||||
import { setupInstallFailure } from "./SetupInstallFailure.js";
|
||||
import {
|
||||
RegularFileCleanupFailure,
|
||||
retryRegularFileCleanup,
|
||||
} from "./RegularFileRead.js";
|
||||
import { providerRegistrationEnvironment } from "./SetupRegistrationEnvironment.js";
|
||||
import type {
|
||||
SetupHost,
|
||||
@@ -101,11 +94,6 @@ export const systemSetupHost = (
|
||||
const doctorHost = selectedDoctorHost ?? systemDoctorHost({ environment });
|
||||
const platform = doctorHost.platform;
|
||||
const { homeDirectory } = doctorHost;
|
||||
const macHopperHost =
|
||||
platform === "darwin"
|
||||
? systemMacHopperInstallHost(homeDirectory, environment)
|
||||
: undefined;
|
||||
const resources = new ArtifactResourceScope();
|
||||
return {
|
||||
platform,
|
||||
homeDirectory,
|
||||
@@ -118,7 +106,6 @@ export const systemSetupHost = (
|
||||
nodeVersion: process.versions.node,
|
||||
macosVersion: () => doctorHost.macosVersion(),
|
||||
linuxDistribution: readLinuxDistribution,
|
||||
close: () => closeSetupResources(resources, macHopperHost),
|
||||
initialSetupState: async (
|
||||
scope?: DoctorScope,
|
||||
): Promise<SetupInitialState> => {
|
||||
@@ -139,38 +126,30 @@ export const systemSetupHost = (
|
||||
};
|
||||
},
|
||||
installHopper: async (replaceExisting) => {
|
||||
if (platform === "linux") {
|
||||
const result = await installLinuxHopper(
|
||||
systemLinuxHopperInstallHost(environment),
|
||||
);
|
||||
return result.status === "installed"
|
||||
? result
|
||||
: setupInstallFailure(result.reason);
|
||||
}
|
||||
if (macHopperHost === undefined)
|
||||
return setupInstallFailure("unsupported_host");
|
||||
const result = await installMacHopper({ replaceExisting }, macHopperHost);
|
||||
if (result.status === "installed")
|
||||
return {
|
||||
status: "installed",
|
||||
launcherPath: result.launcherPath,
|
||||
...(result.cleanupFailure === undefined
|
||||
? {}
|
||||
: { cleanupFailure: result.cleanupFailure }),
|
||||
};
|
||||
const failure = setupInstallFailure(result.reason);
|
||||
if (result.cleanupFailure === undefined) return failure;
|
||||
return failure.status === "failed"
|
||||
? {
|
||||
...failure,
|
||||
remediation: `${failure.remediation} ${result.cleanupFailure}`,
|
||||
}
|
||||
: failure;
|
||||
const result =
|
||||
platform === "linux"
|
||||
? await installLinuxHopper(systemLinuxHopperInstallHost(environment))
|
||||
: await installMacHopper(
|
||||
{ replaceExisting },
|
||||
systemMacHopperInstallHost(homeDirectory, environment),
|
||||
);
|
||||
if (result.status === "installed") return result;
|
||||
return setupInstallFailure(result.reason);
|
||||
},
|
||||
detectedClients: () => detectClients(homeDirectory, platform, environment),
|
||||
supportedClients: () =>
|
||||
Promise.resolve(supportedClients(homeDirectory, platform, environment)),
|
||||
...clientConfigurationOperations(resources),
|
||||
configureClient: (client, providerEnvironment, command) =>
|
||||
client.format === "unsupported"
|
||||
? Promise.resolve({ status: "skipped" })
|
||||
: configureClientConfiguration(client, providerEnvironment, command),
|
||||
clientNeedsConfigure: (client, providerEnvironment, command) =>
|
||||
client.format === "unsupported"
|
||||
? Promise.resolve(false)
|
||||
: clientConfigurationAligned(client, providerEnvironment, command).then(
|
||||
(aligned) => !aligned,
|
||||
),
|
||||
inspectClientConfiguration: inspectClientConfiguration,
|
||||
skillNeedsInstall: (clientIds) =>
|
||||
canonicalSkillNeedsInstall(
|
||||
homeDirectory,
|
||||
@@ -184,106 +163,6 @@ export const systemSetupHost = (
|
||||
};
|
||||
};
|
||||
|
||||
const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
|
||||
configureClient: (
|
||||
client: SetupClient,
|
||||
providerEnvironment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
) =>
|
||||
runClientConfigurationOperation(resources, async () =>
|
||||
client.format === "unsupported"
|
||||
? { status: "skipped" as const }
|
||||
: configureClientConfiguration(client, providerEnvironment, command),
|
||||
),
|
||||
clientNeedsConfigure: (
|
||||
client: SetupClient,
|
||||
providerEnvironment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
) =>
|
||||
runClientConfigurationOperation(resources, async () =>
|
||||
client.format === "unsupported"
|
||||
? false
|
||||
: !(await clientConfigurationAligned(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
)),
|
||||
),
|
||||
inspectClientConfiguration: (
|
||||
client: SetupClient,
|
||||
providerEnvironment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
) =>
|
||||
runClientConfigurationOperation(resources, () =>
|
||||
inspectClientConfiguration(client, providerEnvironment, command),
|
||||
),
|
||||
});
|
||||
|
||||
const closeSetupResources = async (
|
||||
resources: ArtifactResourceScope,
|
||||
macHopperHost: ReturnType<typeof systemMacHopperInstallHost> | undefined,
|
||||
): Promise<string | undefined> => {
|
||||
const [resourceCleanup, hopperCleanup] = await Promise.allSettled([
|
||||
resources.close(),
|
||||
Promise.resolve().then(() => macHopperHost?.close()),
|
||||
]);
|
||||
const failures: string[] = [];
|
||||
if (hopperCleanup.status === "rejected")
|
||||
failures.push(
|
||||
`Hopper host cleanup failed: ${errorMessage(hopperCleanup.reason)}`,
|
||||
);
|
||||
else if (hopperCleanup.value !== undefined)
|
||||
failures.push(hopperCleanup.value);
|
||||
if (resourceCleanup.status === "rejected")
|
||||
failures.push(errorMessage(resourceCleanup.reason));
|
||||
return failures.length === 0 ? undefined : failures.join("; ");
|
||||
};
|
||||
|
||||
const runClientConfigurationOperation = async <Value>(
|
||||
resources: ArtifactResourceScope,
|
||||
operation: () => Promise<Value>,
|
||||
): Promise<Value> => {
|
||||
try {
|
||||
return await resources.run(async () => {
|
||||
try {
|
||||
return await operation();
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
||||
const cleanup = await retryRegularFileCleanup(cause, resources);
|
||||
const primary =
|
||||
cleanup.outcome.kind === "failed"
|
||||
? cleanup.outcome.cause
|
||||
: cause.cleanupCause;
|
||||
throw configurationCleanupError(
|
||||
`Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}`,
|
||||
cause,
|
||||
cleanup.cleanup,
|
||||
);
|
||||
}
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof ArtifactReaderFailure)
|
||||
throw configurationCleanupError(cause.message, cause, cause.cleanup);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const configurationCleanupError = (
|
||||
message: string,
|
||||
cause: unknown,
|
||||
cleanup:
|
||||
| { readonly reason: string; readonly resources: readonly string[] }
|
||||
| undefined,
|
||||
): ConfigurationError =>
|
||||
new ConfigurationError(message, {
|
||||
cause,
|
||||
settings: [{ setting: "client configuration", constraint: message }],
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
});
|
||||
|
||||
const errorMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
/** Detect supported agents from their config files or stable installation markers. */
|
||||
export const detectClients = async (
|
||||
home: string,
|
||||
|
||||
@@ -10,15 +10,10 @@ export type SetupFailureCode =
|
||||
| "destination_exists"
|
||||
| "mount_failed"
|
||||
| "bundle_invalid"
|
||||
| "copy_failed"
|
||||
| "cleanup_failed";
|
||||
| "copy_failed";
|
||||
|
||||
export type SetupHopperInstallResult =
|
||||
| {
|
||||
readonly status: "installed";
|
||||
readonly launcherPath: string;
|
||||
readonly cleanupFailure?: string;
|
||||
}
|
||||
| { readonly status: "installed"; readonly launcherPath: string }
|
||||
| {
|
||||
readonly status: "failed";
|
||||
readonly code: SetupFailureCode;
|
||||
|
||||
@@ -312,8 +312,8 @@ const clientConfigurationDetail = (
|
||||
.map(([name, value]) => `${name}=${value}`);
|
||||
const environmentDetail =
|
||||
entries.length === 0
|
||||
? " No additional provider environment variables. Existing server environment overrides are retained."
|
||||
: ` Detected provider environment: ${entries.join(", ")}. Existing server environment overrides are retained; detected values replace matching keys.`;
|
||||
? " No additional environment variables."
|
||||
: ` Environment: ${entries.join(", ")}.`;
|
||||
const hopperDetail = installHopper
|
||||
? environment.HOPPER_LAUNCHER_PATH === undefined
|
||||
? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation."
|
||||
|
||||
@@ -1,16 +1,12 @@
|
||||
import { copyFile, lstat, mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { mkdir, readFile, rm } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import writeFileAtomic from "write-file-atomic";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import { clientSkillDirectories } from "./SupportedClients.js";
|
||||
|
||||
export const MANAGED_SKILL_FILES = [
|
||||
const SKILL_FILES = [
|
||||
"SKILL.md",
|
||||
"references/connection-and-recovery.md",
|
||||
"references/native-and-artifacts.md",
|
||||
"references/javascript-applications.md",
|
||||
"references/android-applications.md",
|
||||
@@ -18,13 +14,6 @@ export const MANAGED_SKILL_FILES = [
|
||||
"references/evidence-workflows.md",
|
||||
] as const;
|
||||
|
||||
export const isManagedSkillManifest = (content: string): boolean =>
|
||||
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
||||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
||||
|
||||
export const readSkillFile = (path: string): Promise<string> =>
|
||||
readRegularFileText(path, { symlinks: "reject" });
|
||||
|
||||
interface CanonicalSkillFile {
|
||||
readonly content: string;
|
||||
readonly destination: string;
|
||||
@@ -58,38 +47,9 @@ export const skillDestinations = (
|
||||
}),
|
||||
);
|
||||
|
||||
/** Resolve existing REA-owned skill roots using setup's canonical destinations. */
|
||||
export const existingSkillDestinations = async (
|
||||
home: string,
|
||||
environment: Readonly<NodeJS.ProcessEnv> = {},
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
): Promise<readonly SkillDestination[]> => {
|
||||
const existing: SkillDestination[] = [];
|
||||
for (const destination of skillDestinations(
|
||||
home,
|
||||
undefined,
|
||||
environment,
|
||||
platform,
|
||||
)) {
|
||||
try {
|
||||
const root = await lstat(destination.path);
|
||||
if (root.isSymbolicLink() || !root.isDirectory()) continue;
|
||||
const manifest = join(destination.path, "SKILL.md");
|
||||
const content = await readSkillFile(manifest);
|
||||
if (isManagedSkillManifest(content)) existing.push(destination);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
continue;
|
||||
if (cause instanceof NonRegularFileReadError) continue;
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
return existing;
|
||||
};
|
||||
|
||||
const readOptionalText = async (path: string): Promise<string | undefined> => {
|
||||
try {
|
||||
return await readSkillFile(path);
|
||||
return await readFile(path, "utf8");
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||
return undefined;
|
||||
@@ -106,7 +66,7 @@ const canonicalSkillFiles = async (
|
||||
Promise.all(
|
||||
skillDestinations(home, clientIds, environment, platform).flatMap(
|
||||
({ path: root }) =>
|
||||
MANAGED_SKILL_FILES.map(async (relativePath) => {
|
||||
SKILL_FILES.map(async (relativePath) => {
|
||||
const destination = join(root, relativePath);
|
||||
return {
|
||||
destination,
|
||||
@@ -144,19 +104,6 @@ export const canonicalSkillNeedsInstall = async (
|
||||
const writeText = (path: string, content: string): Promise<void> =>
|
||||
writeFileAtomic(path, content, { encoding: "utf8", mode: 0o600 });
|
||||
|
||||
const preserveSkillBackup = async (
|
||||
source: string,
|
||||
destination: string,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
await copyFile(source, destination, fsConstants.COPYFILE_EXCL);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "EEXIST")
|
||||
return;
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const restoreSkillFiles = async (
|
||||
changed: readonly CanonicalSkillFile[],
|
||||
): Promise<void> => {
|
||||
@@ -187,12 +134,12 @@ export const installCanonicalSkill = async (
|
||||
for (const { destination, original } of changed) {
|
||||
await mkdir(dirname(destination), { recursive: true });
|
||||
if (original !== undefined)
|
||||
await preserveSkillBackup(destination, `${destination}.rea.backup`);
|
||||
await writeText(`${destination}.rea.backup`, original);
|
||||
}
|
||||
for (const { destination, content } of changed)
|
||||
await writeText(destination, content);
|
||||
for (const { destination, content } of changed)
|
||||
if ((await readSkillFile(destination)) !== content)
|
||||
if ((await readFile(destination, "utf8")) !== content)
|
||||
throw new Error(`skill readback mismatch: ${destination}`);
|
||||
return "installed";
|
||||
} catch (cause: unknown) {
|
||||
@@ -224,7 +171,7 @@ export const readInstalledSkillIdentity = async (
|
||||
let content: string | undefined;
|
||||
for (const destination of destinations) {
|
||||
try {
|
||||
content = await readSkillFile(join(destination.path, "SKILL.md"));
|
||||
content = await readFile(join(destination.path, "SKILL.md"), "utf8");
|
||||
break;
|
||||
} catch {
|
||||
// Another selected copy may provide metadata; canonical validation below
|
||||
|
||||
@@ -50,8 +50,6 @@ export interface SetupHost {
|
||||
linuxDistribution(): Promise<LinuxDistribution | undefined>;
|
||||
initialSetupState(scope?: DoctorScope): Promise<SetupInitialState>;
|
||||
installHopper(replaceExisting: boolean): Promise<SetupHopperInstallResult>;
|
||||
/** Retry and report any cleanup resources still owned by this setup host. */
|
||||
close?(): Promise<string | undefined>;
|
||||
detectedClients(): Promise<readonly SetupClient[]>;
|
||||
supportedClients(): Promise<readonly SetupClient[]>;
|
||||
configureClient(
|
||||
@@ -157,8 +155,6 @@ export interface SetupOptions {
|
||||
readonly structured: boolean;
|
||||
readonly proposeHopper?: boolean;
|
||||
readonly clientIds?: readonly string[];
|
||||
/** Skill roots to update independently of MCP registrations (maintenance). */
|
||||
readonly skillClientIds?: readonly string[];
|
||||
readonly installSkill?: boolean;
|
||||
readonly readinessScope?: DoctorScope;
|
||||
readonly dryRun?: boolean;
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
withClientServers,
|
||||
type ClientConfigurationDocument,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { copyFile, lstat, realpath, rm } from "node:fs/promises";
|
||||
import { copyFile, lstat, readFile, realpath, rm } from "node:fs/promises";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
||||
import { join } from "node:path";
|
||||
@@ -15,15 +15,9 @@ import writeFileAtomic from "write-file-atomic";
|
||||
import { z } from "zod";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import {
|
||||
isManagedSkillManifest,
|
||||
MANAGED_SKILL_FILES,
|
||||
readSkillFile,
|
||||
skillDestinations,
|
||||
} from "./SetupSkill.js";
|
||||
import { skillDestinations } from "./SetupSkill.js";
|
||||
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
import {
|
||||
manualRegistrationRemediation,
|
||||
supportedClients,
|
||||
@@ -33,32 +27,29 @@ import {
|
||||
interface ManagedPathStats {
|
||||
readonly uid?: number;
|
||||
isFile?(): boolean;
|
||||
isDirectory?(): boolean;
|
||||
isSymbolicLink(): boolean;
|
||||
}
|
||||
|
||||
/** Injectable filesystem operations used to prove uninstall failure recovery. */
|
||||
export interface UninstallFileSystem {
|
||||
readText(path: string): Promise<string>;
|
||||
readRegularText?(path: string): Promise<string>;
|
||||
/** Preserve the first backup; reject an existing destination with EEXIST. */
|
||||
copy(source: string, destination: string): Promise<void>;
|
||||
writeText(path: string, contents: string): Promise<void>;
|
||||
stat(path: string): Promise<ManagedPathStats>;
|
||||
realpath?(path: string): Promise<string>;
|
||||
remove(path: string, recursive?: boolean): Promise<void>;
|
||||
remove(path: string): Promise<void>;
|
||||
}
|
||||
|
||||
const systemFileSystem: UninstallFileSystem = {
|
||||
readText: (path) => readRegularFileText(path),
|
||||
readRegularText: readSkillFile,
|
||||
readText: (path) => readFile(path, "utf8"),
|
||||
copy: (source, destination) =>
|
||||
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
|
||||
writeText: (path, contents) =>
|
||||
writeFileAtomic(path, contents, { encoding: "utf8" }),
|
||||
stat: (path) => lstat(path),
|
||||
realpath: (path) => realpath(path),
|
||||
remove: (path, recursive = true) => rm(path, { recursive }),
|
||||
remove: (path) => rm(path, { recursive: true }),
|
||||
};
|
||||
|
||||
/** One explicitly classified uninstall action. */
|
||||
@@ -431,7 +422,7 @@ const removeManagedSkills = async (
|
||||
const results = await Promise.all(
|
||||
skillDestinations(home, undefined, environment, platform).map(
|
||||
({ client, path }) =>
|
||||
removeManagedSkill(
|
||||
removeManagedPath(
|
||||
path,
|
||||
client === "claude_code" ? "Claude Code skill" : "skill",
|
||||
fileSystem,
|
||||
@@ -466,146 +457,6 @@ const removeManagedSkills = async (
|
||||
);
|
||||
};
|
||||
|
||||
const removeManagedSkill = async (
|
||||
path: string,
|
||||
name: string,
|
||||
fileSystem: UninstallFileSystem,
|
||||
): Promise<UninstallItem> => {
|
||||
let root: ManagedPathStats;
|
||||
try {
|
||||
root = await fileSystem.stat(path);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", "Skill directory does not exist.")
|
||||
: item(name, "failed", "Skill directory could not be inspected.");
|
||||
}
|
||||
if (root.isSymbolicLink())
|
||||
return item(name, "retained", "Skill directory is a symbolic link.");
|
||||
if (root.isDirectory?.() !== true)
|
||||
return item(name, "retained", "Skill path is not a directory.");
|
||||
|
||||
const manifestPath = join(path, "SKILL.md");
|
||||
try {
|
||||
const manifest = await fileSystem.stat(manifestPath);
|
||||
if (manifest.isSymbolicLink())
|
||||
return item(name, "retained", "Skill manifest is a symbolic link.");
|
||||
if (manifest.isFile?.() !== true)
|
||||
return item(name, "retained", "Skill manifest is not a regular file.");
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "retained", "Skill ownership could not be established.")
|
||||
: item(name, "failed", "Skill manifest could not be inspected.");
|
||||
}
|
||||
let content: string;
|
||||
try {
|
||||
content = await (fileSystem.readRegularText?.(manifestPath) ??
|
||||
fileSystem.readText(manifestPath));
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "retained", "Skill ownership could not be established.")
|
||||
: item(name, "failed", "Skill manifest could not be read.");
|
||||
}
|
||||
if (!isManagedSkillManifest(content))
|
||||
return item(name, "retained", "Skill manifest is not REA-owned.");
|
||||
|
||||
const manifestName = "SKILL.md";
|
||||
const otherFiles = MANAGED_SKILL_FILES.filter(
|
||||
(relativePath) => relativePath !== manifestName,
|
||||
);
|
||||
const results: UninstallItem[] = [];
|
||||
for (const relativePath of otherFiles)
|
||||
results.push(
|
||||
await removeManagedSkillFile(path, relativePath, name, fileSystem),
|
||||
);
|
||||
if (
|
||||
results.some(({ status }) => status === "failed" || status === "retained")
|
||||
)
|
||||
results.push(
|
||||
item(
|
||||
name,
|
||||
"retained",
|
||||
"REA ownership manifest was retained because a managed file could not be safely removed.",
|
||||
),
|
||||
);
|
||||
else
|
||||
results.push(
|
||||
await removeManagedSkillFile(path, manifestName, name, fileSystem),
|
||||
);
|
||||
const failed = results.find(({ status }) => status === "failed");
|
||||
if (failed !== undefined)
|
||||
return item(name, "failed", results.map(({ detail }) => detail).join(" "));
|
||||
const removed = results.some(({ status }) => status === "removed");
|
||||
return item(
|
||||
name,
|
||||
removed ? "removed" : "skipped",
|
||||
"REA-managed skill files were removed; other files were preserved.",
|
||||
);
|
||||
};
|
||||
|
||||
const removeManagedSkillFile = async (
|
||||
root: string,
|
||||
relativePath: string,
|
||||
name: string,
|
||||
fileSystem: UninstallFileSystem,
|
||||
): Promise<UninstallItem> => {
|
||||
const segments = relativePath.split(/[\\/]/u);
|
||||
let parent = root;
|
||||
for (const segment of segments.slice(0, -1)) {
|
||||
parent = join(parent, segment);
|
||||
try {
|
||||
const stats = await fileSystem.stat(parent);
|
||||
if (stats.isSymbolicLink() || stats.isDirectory?.() !== true)
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed path parent was retained: ${parent}.`,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(
|
||||
name,
|
||||
"skipped",
|
||||
`${join(parent, segments.at(-1) ?? "")} does not exist.`,
|
||||
)
|
||||
: item(
|
||||
name,
|
||||
"failed",
|
||||
`Managed path parent could not be inspected: ${parent}.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const target = join(root, relativePath);
|
||||
let stats: ManagedPathStats;
|
||||
try {
|
||||
stats = await fileSystem.stat(target);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", `${target} does not exist.`)
|
||||
: item(name, "failed", `Managed file could not be inspected: ${target}.`);
|
||||
}
|
||||
if (stats.isSymbolicLink())
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed file is a symbolic link: ${target}.`,
|
||||
);
|
||||
if (stats.isFile?.() !== true)
|
||||
return item(
|
||||
name,
|
||||
"retained",
|
||||
`Managed path is not a regular file: ${target}.`,
|
||||
);
|
||||
try {
|
||||
await fileSystem.remove(target, false);
|
||||
return item(name, "removed", `Removed ${target}.`);
|
||||
} catch (cause: unknown) {
|
||||
return isMissing(cause)
|
||||
? item(name, "skipped", `${target} does not exist.`)
|
||||
: item(name, "failed", `Managed file could not be removed: ${target}.`);
|
||||
}
|
||||
};
|
||||
|
||||
const item = (
|
||||
name: string,
|
||||
status: UninstallItem["status"],
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { z } from "zod";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
@@ -10,19 +12,13 @@ import {
|
||||
parseClientConfiguration,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { supportedClients } from "./SupportedClients.js";
|
||||
import { existingSkillDestinations } from "./SetupSkill.js";
|
||||
import type { SetupAction } from "./SetupTypes.js";
|
||||
import type { Result } from "../domain/result.js";
|
||||
import { readRegularFileText } from "./RegularFileRead.js";
|
||||
|
||||
/** Existing REA integrations selected for maintenance, without discovering new targets. */
|
||||
export interface MaintenanceScope {
|
||||
readonly clients: readonly string[];
|
||||
readonly skill: boolean;
|
||||
readonly skillDestinations?: readonly {
|
||||
readonly client: string;
|
||||
readonly path: string;
|
||||
}[];
|
||||
}
|
||||
|
||||
/** Unapplied maintenance evidence produced by the updated setup executable. */
|
||||
@@ -53,7 +49,7 @@ const setupPlanSchema = z.object({
|
||||
),
|
||||
});
|
||||
|
||||
/** Discover owned registrations and REA skills without writes. */
|
||||
/** Discover owned registrations and an existing REA skill without writes. */
|
||||
export const existingMaintenanceScope = async (
|
||||
home: string,
|
||||
entryPoint: string,
|
||||
@@ -73,7 +69,7 @@ export const existingMaintenanceScope = async (
|
||||
const client = supported.find(({ name }) => name === registration.client);
|
||||
if (client === undefined) continue;
|
||||
const parsed = parseClientConfiguration(
|
||||
await readRegularFileText(client.configPath),
|
||||
await readFile(client.configPath, "utf8"),
|
||||
client.format,
|
||||
);
|
||||
const enabled = z
|
||||
@@ -91,12 +87,20 @@ export const existingMaintenanceScope = async (
|
||||
)
|
||||
clients.push(client.name);
|
||||
}
|
||||
const skillDestinations = await existingSkillDestinations(home, environment);
|
||||
return {
|
||||
clients,
|
||||
skill: skillDestinations.length > 0,
|
||||
skillDestinations,
|
||||
};
|
||||
let skill = false;
|
||||
try {
|
||||
const content = await readFile(
|
||||
join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName, "SKILL.md"),
|
||||
"utf8",
|
||||
);
|
||||
skill =
|
||||
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
||||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof Error && "code" in cause && cause.code === "ENOENT"))
|
||||
throw cause;
|
||||
}
|
||||
return { clients, skill };
|
||||
};
|
||||
|
||||
/** Ask the verified new executable to plan only existing REA integrations. */
|
||||
@@ -108,7 +112,6 @@ export const planIntegrationMaintenance = async (
|
||||
): Promise<IntegrationMaintenance> => {
|
||||
try {
|
||||
const scope = await existingMaintenanceScope(home, entryPoint, environment);
|
||||
const skillDestinations = scope.skillDestinations ?? [];
|
||||
if (scope.clients.length === 0 && !scope.skill)
|
||||
return { status: "current", plannedActions: [] };
|
||||
const command = [
|
||||
@@ -116,7 +119,6 @@ export const planIntegrationMaintenance = async (
|
||||
entryPoint,
|
||||
"setup",
|
||||
...scope.clients.flatMap((client) => ["--client", client]),
|
||||
...skillDestinations.flatMap(({ client }) => ["--skill-client", client]),
|
||||
`--skill=${String(scope.skill)}`,
|
||||
];
|
||||
const execution = await execute([...command, "--dry-run", "--json"]);
|
||||
@@ -130,17 +132,11 @@ export const planIntegrationMaintenance = async (
|
||||
};
|
||||
if (plan.data.plannedActions.length === 0)
|
||||
return { status: "current", plannedActions: [] };
|
||||
const allowedIds = new Set(
|
||||
scope.clients.map((client) => `configure_client:${client}`),
|
||||
);
|
||||
const skillTargets = new Set(skillDestinations.map(({ path }) => path));
|
||||
if (
|
||||
plan.data.plannedActions.some((action) =>
|
||||
action.kind === "install_skill"
|
||||
? !skillTargets.has(action.target)
|
||||
: !allowedIds.has(action.id),
|
||||
)
|
||||
)
|
||||
const allowedIds = new Set([
|
||||
...scope.clients.map((client) => `configure_client:${client}`),
|
||||
...(scope.skill ? ["install_skill"] : []),
|
||||
]);
|
||||
if (plan.data.plannedActions.some(({ id }) => !allowedIds.has(id)))
|
||||
return {
|
||||
status: "unavailable",
|
||||
remediation:
|
||||
|
||||
@@ -4,7 +4,6 @@ import { isAbsolute } from "node:path";
|
||||
import { z } from "zod";
|
||||
|
||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import { publishWebScripts } from "../browser/assets/PublishWebScripts.js";
|
||||
import { selectScriptCapture } from "../browser/assets/ScriptCaptureAdapters.js";
|
||||
import {
|
||||
@@ -15,11 +14,9 @@ import {
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { createEvidence, type Evidence } from "../domain/evidence.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { safeParseJson } from "../domain/safeJson.js";
|
||||
import {
|
||||
@@ -29,41 +26,19 @@ import {
|
||||
import { WebScriptExportError } from "../domain/webScriptExportError.js";
|
||||
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
||||
import { WEB_SCRIPT_EXPORT_PROVIDER as PROVIDER } from "./InvestigationProviders.js";
|
||||
import {
|
||||
readRegularFile,
|
||||
RegularFileCleanupFailure,
|
||||
retryRegularFileCleanup,
|
||||
} from "./RegularFileRead.js";
|
||||
import {
|
||||
NonRegularFileReadError,
|
||||
RegularFileChangedError,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import { readRegularFile } from "./RegularFileRead.js";
|
||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||
|
||||
const OPERATION = "export_web_scripts";
|
||||
|
||||
interface CaptureRead {
|
||||
readonly bytes: Buffer;
|
||||
readonly cleanup?: {
|
||||
readonly reason: string;
|
||||
readonly incomplete: boolean;
|
||||
readonly cause: unknown;
|
||||
};
|
||||
}
|
||||
|
||||
interface PreparedCapture {
|
||||
readonly bytes: Buffer;
|
||||
readonly loaded: ReturnType<typeof selectScriptCapture>;
|
||||
}
|
||||
|
||||
/** Export one local capture through the shared CLI/MCP application workflow. */
|
||||
export const exportWebScripts = async (
|
||||
rawInput: unknown,
|
||||
resources: ArtifactResourceScope,
|
||||
options: ExecutionOptions = {},
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
const input = exportWebScriptsInputSchema.safeParse(rawInput);
|
||||
return input.success
|
||||
? exportWebScriptsValidated(input.data, resources, options)
|
||||
? exportWebScriptsValidated(input.data, options)
|
||||
: err(
|
||||
analysisInputErrorFromIssues(OPERATION, input.error.issues, rawInput),
|
||||
);
|
||||
@@ -72,7 +47,6 @@ export const exportWebScripts = async (
|
||||
/** Publish input already parsed by a named adapter contract. */
|
||||
export const exportWebScriptsValidated = async (
|
||||
input: ExportWebScriptsInput,
|
||||
resources: ArtifactResourceScope,
|
||||
options: ExecutionOptions = {},
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
if (!isAbsolute(input.capture_path) || !isAbsolute(input.output_directory))
|
||||
@@ -88,19 +62,19 @@ export const exportWebScriptsValidated = async (
|
||||
);
|
||||
try {
|
||||
options.signal?.throwIfAborted();
|
||||
const prepared = await prepareCapture(
|
||||
input.capture_path,
|
||||
resources,
|
||||
options.signal,
|
||||
);
|
||||
if (!prepared.ok) return prepared;
|
||||
const { bytes, loaded } = prepared.value;
|
||||
const read = await readCapture(input.capture_path, options.signal);
|
||||
if (!read.ok) return read;
|
||||
const bytes = read.value;
|
||||
const loaded = parseCapture(bytes);
|
||||
if (!loaded.ok) return loaded;
|
||||
options.signal?.throwIfAborted();
|
||||
const sha256 = createHash("sha256").update(bytes).digest("hex");
|
||||
const result = await publishWebScripts(input, loaded, sha256, {
|
||||
resources,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
const result = await publishWebScripts(
|
||||
input,
|
||||
loaded.value,
|
||||
sha256,
|
||||
options.signal,
|
||||
);
|
||||
return ok(
|
||||
createEvidence(
|
||||
{ path: input.capture_path, sha256, format: "file" },
|
||||
@@ -119,109 +93,55 @@ export const exportWebScriptsValidated = async (
|
||||
limitations: result.limitations,
|
||||
locations: [{ kind: "artifact-path", path: result.manifest.path }],
|
||||
evidenceLinks:
|
||||
loaded.sourceEvidenceId === null ? [] : [loaded.sourceEvidenceId],
|
||||
loaded.value.sourceEvidenceId === null
|
||||
? []
|
||||
: [loaded.value.sourceEvidenceId],
|
||||
},
|
||||
),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
return err(mapExportFailure(cause, input, options.signal));
|
||||
}
|
||||
};
|
||||
|
||||
const prepareCapture = async (
|
||||
path: string,
|
||||
resources: ArtifactResourceScope,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<PreparedCapture, AnalysisError>> => {
|
||||
const read = await resources.run(() => readCapture(path, resources, signal));
|
||||
if (!read.ok) return read;
|
||||
const cleanup = read.value.cleanup;
|
||||
const loaded = parseCapture(
|
||||
read.value.bytes,
|
||||
cleanup === undefined || cleanup.incomplete
|
||||
? undefined
|
||||
: { cause: cleanup.cause },
|
||||
);
|
||||
if (!loaded.ok)
|
||||
if (cause instanceof WebScriptExportError) return err(cause);
|
||||
if (options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
if (cause instanceof ArtifactReaderFailure)
|
||||
return err(
|
||||
new WebScriptExportError(
|
||||
cause.reason,
|
||||
input.output_directory,
|
||||
cause.message,
|
||||
),
|
||||
);
|
||||
if (cause instanceof z.ZodError)
|
||||
return err(new AnalysisOutputError(OPERATION, cause.message, { cause }));
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
typeof cause.code === "string"
|
||||
)
|
||||
return err(
|
||||
new WebScriptExportError(
|
||||
"io",
|
||||
`${input.capture_path} → ${input.output_directory}`,
|
||||
cause.message,
|
||||
),
|
||||
);
|
||||
return err(
|
||||
cleanup?.incomplete === true
|
||||
? incompleteCaptureCleanup(loaded.error, path, cleanup)
|
||||
: loaded.error,
|
||||
);
|
||||
if (cleanup !== undefined) {
|
||||
const primary = new WebScriptExportError(
|
||||
"io",
|
||||
path,
|
||||
`Capture handle close failed: ${cleanup.reason}`,
|
||||
{ cause: cleanup.cause },
|
||||
);
|
||||
return err(
|
||||
cleanup.incomplete
|
||||
? incompleteCaptureCleanup(primary, path, cleanup)
|
||||
: primary,
|
||||
new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
||||
cause,
|
||||
diagnostics: {
|
||||
capture_path: input.capture_path,
|
||||
output_directory: input.output_directory,
|
||||
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
||||
error_message:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: typeof cause === "string"
|
||||
? cause
|
||||
: "Unknown script export failure",
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
return ok({ bytes: read.value.bytes, loaded: loaded.value });
|
||||
};
|
||||
|
||||
const mapExportFailure = (
|
||||
cause: unknown,
|
||||
input: ExportWebScriptsInput,
|
||||
signal: AbortSignal | undefined,
|
||||
): AnalysisError => {
|
||||
if (cause instanceof WebScriptExportError) return cause;
|
||||
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined)
|
||||
return analysisErrorWithCleanupFailure(
|
||||
new WebScriptExportError(
|
||||
cause.reason,
|
||||
input.output_directory,
|
||||
cause.message,
|
||||
{ cause },
|
||||
),
|
||||
new ProviderCleanupError(
|
||||
PROVIDER.id,
|
||||
cause.cleanup.resources,
|
||||
{ reason: cause.cleanup.reason },
|
||||
{ cause, operation: OPERATION },
|
||||
),
|
||||
OPERATION,
|
||||
);
|
||||
if (signal?.aborted === true)
|
||||
return new AnalysisCancelledError(OPERATION, { cause });
|
||||
if (cause instanceof ArtifactReaderFailure)
|
||||
return new WebScriptExportError(
|
||||
cause.reason,
|
||||
input.output_directory,
|
||||
cause.message,
|
||||
{ cause },
|
||||
);
|
||||
if (cause instanceof z.ZodError)
|
||||
return new AnalysisOutputError(OPERATION, cause.message, { cause });
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
typeof cause.code === "string"
|
||||
)
|
||||
return new WebScriptExportError(
|
||||
"io",
|
||||
`${input.capture_path} → ${input.output_directory}`,
|
||||
cause.message,
|
||||
{ cause },
|
||||
);
|
||||
return new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
||||
cause,
|
||||
diagnostics: {
|
||||
capture_path: input.capture_path,
|
||||
output_directory: input.output_directory,
|
||||
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
||||
error_message:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: typeof cause === "string"
|
||||
? cause
|
||||
: "Unknown script export failure",
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -230,115 +150,44 @@ const mapExportFailure = (
|
||||
*/
|
||||
const readCapture = async (
|
||||
path: string,
|
||||
resources: ArtifactResourceScope,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<CaptureRead, AnalysisError>> => {
|
||||
): Promise<Result<Buffer, AnalysisError>> => {
|
||||
try {
|
||||
return ok({ bytes: await readRegularFile(path, { signal }) });
|
||||
return ok(await readRegularFile(path, signal));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) {
|
||||
const retry = await retryRegularFileCleanup(cause, resources);
|
||||
const cleanup = {
|
||||
reason: retry.cleanup?.reason ?? errorMessage(cause.cleanupCause),
|
||||
incomplete: retry.cleanup !== undefined,
|
||||
cause: cause.cleanupCause,
|
||||
};
|
||||
if (cause.outcome.kind === "completed")
|
||||
return ok({ bytes: cause.outcome.value as Buffer, cleanup });
|
||||
const primary = captureReadFailure(
|
||||
path,
|
||||
cause.outcome.cause,
|
||||
signal,
|
||||
retry.cleanup === undefined ? { cause: cause.cleanupCause } : undefined,
|
||||
);
|
||||
const code =
|
||||
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
||||
if (code === "EACCES" || code === "EPERM")
|
||||
return err(
|
||||
retry.cleanup === undefined
|
||||
? primary
|
||||
: incompleteCaptureCleanup(primary, path, cleanup),
|
||||
new AnalysisAccessDeniedError(OPERATION, path, code, { cause }),
|
||||
);
|
||||
}
|
||||
return err(captureReadFailure(path, cause, signal));
|
||||
if (
|
||||
cause instanceof NonRegularFileReadError ||
|
||||
code === "ENOENT" ||
|
||||
code === "ENOTDIR" ||
|
||||
code === "EISDIR" ||
|
||||
code === "ENXIO"
|
||||
)
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, { cause }, [
|
||||
{
|
||||
path: ["capture_path"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
cause instanceof NonRegularFileReadError && code !== "EISDIR"
|
||||
? cause.message
|
||||
: code === "EISDIR"
|
||||
? `Selected capture is a directory, not a file: ${path}`
|
||||
: `Selected capture could not be read (${code}): ${path}`,
|
||||
},
|
||||
]),
|
||||
);
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const captureReadFailure = (
|
||||
path: string,
|
||||
cause: unknown,
|
||||
signal: AbortSignal | undefined,
|
||||
closeFailure?: { readonly cause: unknown },
|
||||
): AnalysisError => {
|
||||
const causeWithClose =
|
||||
closeFailure === undefined
|
||||
? cause
|
||||
: new AggregateError(
|
||||
[cause, closeFailure.cause],
|
||||
"Capture read and file-handle close both failed",
|
||||
{ cause },
|
||||
);
|
||||
if (signal?.aborted === true)
|
||||
return new AnalysisCancelledError(OPERATION, { cause: causeWithClose });
|
||||
const code =
|
||||
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
||||
if (code === "EACCES" || code === "EPERM")
|
||||
return new AnalysisAccessDeniedError(OPERATION, path, code, {
|
||||
cause: causeWithClose,
|
||||
});
|
||||
if (
|
||||
cause instanceof NonRegularFileReadError ||
|
||||
cause instanceof RegularFileChangedError ||
|
||||
code === "ENOENT" ||
|
||||
code === "ENOTDIR" ||
|
||||
code === "EISDIR" ||
|
||||
code === "ENXIO"
|
||||
)
|
||||
return new AnalysisInputError(OPERATION, { cause: causeWithClose }, [
|
||||
{
|
||||
path: ["capture_path"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
cause instanceof NonRegularFileReadError && code !== "EISDIR"
|
||||
? cause.message
|
||||
: code === "EISDIR"
|
||||
? `Selected capture is a directory, not a file: ${path}`
|
||||
: `Selected capture could not be read (${code}): ${path}`,
|
||||
},
|
||||
]);
|
||||
if (code !== "")
|
||||
return new WebScriptExportError("io", path, errorMessage(cause), {
|
||||
cause: causeWithClose,
|
||||
});
|
||||
return new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
||||
cause: causeWithClose,
|
||||
diagnostics: {
|
||||
capture_path: path,
|
||||
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
||||
error_message: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
const incompleteCaptureCleanup = (
|
||||
primary: AnalysisError,
|
||||
path: string,
|
||||
cleanup: NonNullable<CaptureRead["cleanup"]>,
|
||||
): AnalysisError =>
|
||||
analysisErrorWithCleanupFailure(
|
||||
primary,
|
||||
new ProviderCleanupError(
|
||||
PROVIDER.id,
|
||||
[path],
|
||||
{ reason: cleanup.reason },
|
||||
{ cause: cleanup.cause, operation: OPERATION },
|
||||
),
|
||||
OPERATION,
|
||||
);
|
||||
|
||||
const errorMessage = (cause: unknown): string =>
|
||||
cause instanceof Error ? cause.message : String(cause);
|
||||
|
||||
const parseCapture = (
|
||||
bytes: Buffer,
|
||||
closeFailure?: { readonly cause: unknown },
|
||||
): Result<ReturnType<typeof selectScriptCapture>, AnalysisError> => {
|
||||
try {
|
||||
const text = new TextDecoder("utf-8", {
|
||||
@@ -349,16 +198,8 @@ const parseCapture = (
|
||||
if (!json.ok) throw new TypeError(json.error);
|
||||
return ok(selectScriptCapture(json.value));
|
||||
} catch (cause: unknown) {
|
||||
const causeWithClose =
|
||||
closeFailure === undefined
|
||||
? cause
|
||||
: new AggregateError(
|
||||
[cause, closeFailure.cause],
|
||||
"Capture parse and file-handle close both failed",
|
||||
{ cause },
|
||||
);
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, { cause: causeWithClose }, [
|
||||
new AnalysisInputError(OPERATION, { cause }, [
|
||||
{
|
||||
path: ["capture_path"],
|
||||
reason: "invalid_format",
|
||||
|
||||
@@ -28,7 +28,6 @@ export interface WebSourceLocationArtifactPort {
|
||||
|
||||
/** Replaceable codec receives map data and one explicit generated point. */
|
||||
export interface WebSourceMapPort {
|
||||
close?(): Promise<void>;
|
||||
trace(
|
||||
input: {
|
||||
readonly text: string;
|
||||
|
||||
@@ -59,10 +59,6 @@ export class WebSourceLocationService {
|
||||
readonly decoder: WebSourceMapPort,
|
||||
) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.decoder.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
|
||||
async trace(
|
||||
rawInput: unknown,
|
||||
|
||||
@@ -14,7 +14,7 @@ import type { Result } from "../../domain/result.js";
|
||||
export interface AdbDeviceAnalysisPort {
|
||||
/** Probe the selected adb binary without contacting devices. */
|
||||
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
|
||||
/** Terminally stop admission, drain work, and retry retained package rollback. */
|
||||
/** The provider holds no long-lived resources; safe to call always. */
|
||||
close(): Promise<void>;
|
||||
execute(
|
||||
request: AdbRequest,
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import {
|
||||
inspectAnalysisView,
|
||||
inspectAnalysisViewValidated,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "./AnalysisViewService.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
@@ -59,35 +58,6 @@ it.each(["/fixtures/unknown.exe", ""])(
|
||||
},
|
||||
);
|
||||
|
||||
it("reports inline Evidence identity failures at the complete source path", () => {
|
||||
const parent = analysisViewLayoutEvidence();
|
||||
const input = {
|
||||
source: {
|
||||
kind: "inline",
|
||||
evidence: { ...parent, evidence_id: `ev_${"0".repeat(64)}` },
|
||||
},
|
||||
view: { kind: "summary" },
|
||||
} as const;
|
||||
|
||||
for (const result of [
|
||||
inspectAnalysisView(input),
|
||||
inspectAnalysisViewValidated(input),
|
||||
])
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "AnalysisInputError",
|
||||
issues: [
|
||||
{
|
||||
path: ["source", "evidence", "evidence_id"],
|
||||
reason: "invalid_value",
|
||||
message: "Evidence semantic identifier does not match its record",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
|
||||
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||
const parent = createEvidence(
|
||||
|
||||
@@ -132,11 +132,8 @@ const resolveAnalysisViewSource = (
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
cause.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "evidence", ...issue.path],
|
||||
})),
|
||||
{ source },
|
||||
cause.issues,
|
||||
source.evidence,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
|
||||
@@ -5,8 +5,10 @@ import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
|
||||
import { err } from "../../domain/result.js";
|
||||
import type { AnalysisClient } from "../AnalysisProvider.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
import {
|
||||
analysisErrorWithCleanupFailure,
|
||||
closeAnalysisClient,
|
||||
} from "./AnalysisClientCleanup.js";
|
||||
|
||||
const clientClosingWith = (cause: unknown): AnalysisClient => ({
|
||||
execute: () => Promise.resolve(err(new ProviderCleanupError("x", [], {}))),
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
|
||||
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { err, type Result } from "../../domain/result.js";
|
||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { AnalysisClient, ExecutionOptions } from "../AnalysisProvider.js";
|
||||
@@ -27,3 +29,34 @@ export const closeAnalysisClient = async (
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** Preserve the original lifecycle failure alongside incomplete cleanup. */
|
||||
export const analysisErrorWithCleanupFailure = (
|
||||
primary: AnalysisError,
|
||||
cleanup: AnalysisError,
|
||||
operation = "close_binary",
|
||||
): AnalysisError => {
|
||||
const projected = projectAnalysisError(cleanup);
|
||||
return new ProviderAdapterError(
|
||||
cleanup instanceof ProviderAdapterError ? cleanup.providerId : "rea",
|
||||
operation,
|
||||
{
|
||||
cause: primary,
|
||||
userMessage: projectAnalysisError(primary).message,
|
||||
...(primary.capturedOutput === undefined
|
||||
? {}
|
||||
: { capturedOutput: primary.capturedOutput }),
|
||||
...(primary.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: primary.partialObservation }),
|
||||
cleanup: {
|
||||
reason: projected.message,
|
||||
resources: cleanup.cleanupResources,
|
||||
},
|
||||
diagnostics: {
|
||||
primary_error: projectAnalysisError(primary),
|
||||
cleanup_error: projected,
|
||||
},
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
@@ -44,8 +44,10 @@ import {
|
||||
commitExecutionProfile,
|
||||
prepareSessionExecution,
|
||||
} from "./BinarySessionExecution.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
import {
|
||||
analysisErrorWithCleanupFailure,
|
||||
closeAnalysisClient,
|
||||
} from "./AnalysisClientCleanup.js";
|
||||
const OFFICIAL_OPERATIONS: ReadonlySet<string> = new Set(
|
||||
OFFICIAL_TOOL_CONTRACTS.map(({ name }) => name),
|
||||
);
|
||||
|
||||
@@ -8,7 +8,6 @@ import type { Result } from "../../domain/result.js";
|
||||
|
||||
/** Replaceable offline interface-recovery engine, independent of wallets and chain clients. */
|
||||
export interface EvmInterfacePort {
|
||||
close?(): Promise<void>;
|
||||
inspect(
|
||||
input: InspectEvmInterfaceInput,
|
||||
options?: ExecutionOptions,
|
||||
|
||||
@@ -25,10 +25,6 @@ const diagnosticsSchema = z.object({
|
||||
export class EvmInterfaceService {
|
||||
constructor(readonly provider: EvmInterfacePort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Preserve selected byte identity while clearly labeling recovered ABI candidates as inferences. */
|
||||
async inspect(
|
||||
rawInput: unknown,
|
||||
|
||||
@@ -8,7 +8,6 @@ import type { Result } from "../../domain/result.js";
|
||||
|
||||
/** File-oriented firmware engines; executable admission remains a separate operation. */
|
||||
export interface FirmwareAnalysisPort {
|
||||
close?(): Promise<void>;
|
||||
execute(
|
||||
request: FirmwareRequest,
|
||||
options?: ExecutionOptions,
|
||||
|
||||
@@ -18,10 +18,6 @@ import { err, ok, type Result } from "../../domain/result.js";
|
||||
export class FirmwareAnalysisService {
|
||||
constructor(readonly provider: FirmwareAnalysisPort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
/** Execute an explicit inspection or extraction and preserve its producing identity. */
|
||||
async execute(
|
||||
operation: FirmwareOperation,
|
||||
|
||||
@@ -2,10 +2,9 @@ import type { ApplicationNode } from "../../domain/javascript/javascriptApplicat
|
||||
import { sha256Text } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
||||
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import {
|
||||
sourceNodeFor,
|
||||
type JavaScriptArtifactGraphContext,
|
||||
type JavaScriptArtifactGraphCoverage,
|
||||
import type {
|
||||
JavaScriptArtifactGraphContext,
|
||||
JavaScriptArtifactGraphCoverage,
|
||||
} from "./JavaScriptArtifactGraphContext.js";
|
||||
import {
|
||||
astObservationEvidence,
|
||||
@@ -39,11 +38,10 @@ export const electronFindingSourceNode = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
moduleKey: string | null,
|
||||
runtime?: string,
|
||||
): ApplicationNode | undefined =>
|
||||
moduleKey === null
|
||||
? (context.assetNodes.get(file.path) ?? context.fileNodes.get(file.path))
|
||||
: sourceNodeFor(context, file.path, moduleKey, runtime);
|
||||
: context.moduleNodes.get(`${file.path}\0${moduleKey}`);
|
||||
|
||||
/** Observation-scoped identity shared safely across different artifact files. */
|
||||
export const electronObservationIdentity = (
|
||||
|
||||
@@ -73,7 +73,6 @@ const addIpcRecord = (
|
||||
context,
|
||||
record.file,
|
||||
record.finding.module_key,
|
||||
record.finding.module_runtime,
|
||||
);
|
||||
if (source === undefined) return;
|
||||
state.sources.set(record.key, source);
|
||||
@@ -256,7 +255,6 @@ const addValidationObservations = (state: IpcGraphState): void => {
|
||||
state.context,
|
||||
file,
|
||||
validation.module_key,
|
||||
validation.module_runtime,
|
||||
)
|
||||
: state.handlers.get(handlerRecord.key);
|
||||
if (target !== undefined)
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
} from "./JavaScriptArtifactGraphContext.js";
|
||||
import { astObservationEvidence } from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import { javascriptDisplayText } from "../../domain/javascript/javascriptAstValues.js";
|
||||
import {
|
||||
addElectronInferenceEdge,
|
||||
electronFindingSourceNode,
|
||||
@@ -38,12 +37,7 @@ export const addElectronNativeBoundaries = (
|
||||
|
||||
const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
const { context, file, value, coverage } = input;
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
if (source === undefined) return;
|
||||
const resolved = resolveArtifactPath(
|
||||
value.specifier,
|
||||
@@ -57,27 +51,17 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
identity: electronObservationIdentity(
|
||||
context,
|
||||
"javascript-requested-native-export",
|
||||
compositeKey([
|
||||
file.path,
|
||||
value.specifier,
|
||||
value.binding_kind,
|
||||
value.members,
|
||||
value.namespace_access ?? null,
|
||||
value.dynamic_member_access ?? null,
|
||||
electronRangeKey(value.location),
|
||||
]),
|
||||
// Members are a variable-length source-derived list: compositeKey keeps
|
||||
// ["a\0b"] distinct from ["a", "b"] inside the digested identity.
|
||||
`${file.path}:${value.specifier}:${value.binding_kind}:${compositeKey(value.members)}:${electronRangeKey(value.location)}`,
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
label:
|
||||
value.members.map(javascriptDisplayText).join(", ") ||
|
||||
value.specifier,
|
||||
label: value.members.join(", "),
|
||||
properties: {
|
||||
specifier: value.specifier,
|
||||
binding_kind: value.binding_kind,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
resolved_path: resolved,
|
||||
native_export_verification: "not-performed",
|
||||
},
|
||||
@@ -106,8 +90,6 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
specifier: value.specifier,
|
||||
binding_kind: value.binding_kind,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
},
|
||||
limitations: [
|
||||
"Requested JavaScript members are not verified native exports.",
|
||||
@@ -125,8 +107,6 @@ const addNativeBinding = (input: NativeBindingInput): void => {
|
||||
properties: {
|
||||
resolved_path: resolved,
|
||||
requested_members: value.members,
|
||||
namespace_access: value.namespace_access ?? null,
|
||||
dynamic_member_access: value.dynamic_member_access ?? null,
|
||||
verified: false,
|
||||
},
|
||||
confidence: "medium",
|
||||
|
||||
@@ -39,32 +39,17 @@ export const addElectronWindowBoundaries = (
|
||||
if (javascript === null) continue;
|
||||
const coverage = javascriptAnalysisCoverage(javascript);
|
||||
for (const value of javascript.electron.browser_windows) {
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
if (source !== undefined)
|
||||
addBrowserWindow({ context, file, source, value, coverage });
|
||||
}
|
||||
for (const value of javascript.electron.context_bridge_apis) {
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
if (source !== undefined)
|
||||
addContextBridge({ context, file, source, value, coverage });
|
||||
}
|
||||
for (const value of javascript.electron.utility_processes) {
|
||||
const source = electronFindingSourceNode(
|
||||
context,
|
||||
file,
|
||||
value.module_key,
|
||||
value.module_runtime,
|
||||
);
|
||||
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||
if (source !== undefined)
|
||||
addUtilityProcess({ context, file, source, value, coverage });
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
describe("JavaScript analysis cancellation before publication", () => {
|
||||
it("interrupts semantic graph commitment before result validation", async () => {
|
||||
|
||||
@@ -10,7 +10,7 @@ import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascri
|
||||
import { matchJavaScriptApplicationVersions } from "../../domain/javascript/javascriptApplicationVersionKeys.js";
|
||||
import { findApplicationFeatureSeeds } from "../../domain/javascript/javascriptFeatureSeed.js";
|
||||
import { traceApplicationFeatureInputSchema } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
const analyzeSource = async (
|
||||
source: string,
|
||||
|
||||
@@ -5,7 +5,7 @@ import { expect, it } from "vitest";
|
||||
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
it.each([
|
||||
{
|
||||
|
||||
@@ -4,11 +4,9 @@ import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
||||
import { scanCanonicalArtifactInventory } from "../../../tests/fixtures/artifactInventory.js";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
describe("JavaScript application failure diagnostics", () => {
|
||||
it("identifies the rejected result field in caller-visible diagnostics", async () => {
|
||||
@@ -155,41 +153,3 @@ describe("JavaScript application failure diagnostics", () => {
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("JavaScript application cleanup diagnostics", () => {
|
||||
it("preserves cleanup and a complete inventory when translating reader failures", async () => {
|
||||
const inputPath = await createTestTempDirectory("rea-js-cleanup-failure-");
|
||||
await writeFile(join(inputPath, "main.js"), "export const value = 1;\n");
|
||||
const inventory = await scanCanonicalArtifactInventory(inputPath);
|
||||
const cleanup = {
|
||||
reason: "reader remains open",
|
||||
resources: [`JavaScript artifact reader for ${inputPath}`],
|
||||
};
|
||||
const partialObservation = {
|
||||
kind: "artifact-inventory" as const,
|
||||
inventory,
|
||||
};
|
||||
const failure = new ArtifactReaderFailure("io", "Reader cleanup failed", {
|
||||
cleanup,
|
||||
partialObservation,
|
||||
});
|
||||
|
||||
const result = await analyzeJavaScriptApplication(
|
||||
{ input_path: inputPath, format: "directory" },
|
||||
{
|
||||
progress: {
|
||||
report: async () => {
|
||||
throw failure;
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
if (result.ok) throw new Error("Expected reader failure");
|
||||
expect(result.error).toMatchObject({
|
||||
_tag: "ArtifactOperationError",
|
||||
cleanup,
|
||||
partialObservation,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
||||
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
||||
import {
|
||||
analyzeJavaScriptApplicationInputSchema,
|
||||
parseOwnedJavaScriptApplicationAnalysisSteps,
|
||||
@@ -33,7 +32,6 @@ const OPERATION = "analyze_javascript_application" as const;
|
||||
/** Statically analyze one local JavaScript/Electron application. */
|
||||
export const analyzeJavaScriptApplication = async (
|
||||
rawInput: unknown,
|
||||
scope: ArtifactResourceScope,
|
||||
options: ExecutionOptions = {},
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
const parsed = analyzeJavaScriptApplicationInputSchema.safeParse(rawInput);
|
||||
@@ -41,13 +39,12 @@ export const analyzeJavaScriptApplication = async (
|
||||
return err(
|
||||
analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput),
|
||||
);
|
||||
return analyzeJavaScriptApplicationValidated(parsed.data, scope, options);
|
||||
return analyzeJavaScriptApplicationValidated(parsed.data, options);
|
||||
};
|
||||
|
||||
/** Analyze input already parsed by a trusted adapter boundary. */
|
||||
export const analyzeJavaScriptApplicationValidated = async (
|
||||
input: z.output<typeof analyzeJavaScriptApplicationInputSchema>,
|
||||
scope: ArtifactResourceScope,
|
||||
options: ExecutionOptions = {},
|
||||
): Promise<Result<Evidence, AnalysisError>> => {
|
||||
try {
|
||||
@@ -65,7 +62,6 @@ export const analyzeJavaScriptApplicationValidated = async (
|
||||
format: input.format,
|
||||
integrity_policy: input.integrity_policy,
|
||||
},
|
||||
scope,
|
||||
options.signal,
|
||||
options.progress,
|
||||
);
|
||||
@@ -118,7 +114,14 @@ export const analyzeJavaScriptApplicationValidated = async (
|
||||
)
|
||||
return err(cause);
|
||||
if (cause instanceof ArtifactReaderFailure)
|
||||
return err(artifactFailureToOperationError(cause));
|
||||
return err(
|
||||
new ArtifactOperationError(
|
||||
OPERATION,
|
||||
cause.reason,
|
||||
cause.details,
|
||||
cause.message,
|
||||
),
|
||||
);
|
||||
if (cause instanceof z.ZodError)
|
||||
return err(
|
||||
new AnalysisOutputError(OPERATION, describeResultSchemaFailure(cause), {
|
||||
@@ -150,23 +153,6 @@ export const analyzeJavaScriptApplicationValidated = async (
|
||||
}
|
||||
};
|
||||
|
||||
const artifactFailureToOperationError = (
|
||||
cause: ArtifactReaderFailure,
|
||||
): ArtifactOperationError =>
|
||||
new ArtifactOperationError(
|
||||
OPERATION,
|
||||
cause.reason,
|
||||
cause.details,
|
||||
cause.message,
|
||||
{
|
||||
cause,
|
||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
||||
...(cause.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: cause.partialObservation }),
|
||||
},
|
||||
);
|
||||
|
||||
const describeResultSchemaFailure = (cause: z.ZodError): string => {
|
||||
const issue = cause.issues[0];
|
||||
if (issue === undefined) return "Result schema rejected an unknown issue";
|
||||
|
||||
@@ -7,7 +7,7 @@ import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirect
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { javaScriptSemanticQueryResultSchema } from "../../domain/javascript/javascriptSemanticQuerySchemas.js";
|
||||
import { queryJavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticQuery.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
it.each([
|
||||
{
|
||||
|
||||
@@ -5,7 +5,7 @@ import { expect, it } from "vitest";
|
||||
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
const graphIdentities = async (source: string) => {
|
||||
const inputPath = await createTestTempDirectory("rea-js-template-literal-");
|
||||
@@ -31,16 +31,7 @@ const graphIdentities = async (source: string) => {
|
||||
...(identity.strategy === "artifact-local-key"
|
||||
? {
|
||||
namespace: identity.namespace,
|
||||
// Module-export identity is a JSON tuple whose first entry
|
||||
// binds the whole container; only that digest changes here.
|
||||
key:
|
||||
identity.namespace === "module-export"
|
||||
? JSON.stringify(
|
||||
(JSON.parse(identity.key) as unknown[]).slice(1),
|
||||
)
|
||||
: identity.namespace === "source-module"
|
||||
? identity.key.replace(/^[0-9a-f]{64}:/u, "")
|
||||
: identity.key,
|
||||
key: identity.key.replace(/^[0-9a-f]{64}:/u, ""),
|
||||
}
|
||||
: {}),
|
||||
}))
|
||||
|
||||
@@ -7,7 +7,7 @@ import { canonicalJson } from "../../domain/comparisonSemantics.js";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
type ApplicationAnalysis = ReturnType<
|
||||
typeof javascriptApplicationAnalysisResultSchema.parse
|
||||
|
||||
@@ -7,6 +7,7 @@ import type { ApplicationVersionComparisonResult } from "../../domain/javascript
|
||||
import type { JavaScriptExportShapeComparisonResult } from "../../domain/javascript/javascriptExportShapeComparisonSchemas.js";
|
||||
import type { ApplicationFeatureTraceResult } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
||||
import type { JavaScriptSemanticTraceResult } from "../../domain/javascript/javascriptSemanticTraceSchemas.js";
|
||||
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
||||
import type { SourceToBundleComparisonResult } from "../../domain/javascript/sourceToBundleComparisonSchemas.js";
|
||||
import { JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER } from "../InvestigationProviders.js";
|
||||
|
||||
@@ -19,7 +20,7 @@ export const createApplicationFeatureTraceEvidence = (
|
||||
predicateType: "rea.application-feature-trace",
|
||||
operation: "trace_application_feature",
|
||||
parameters,
|
||||
result,
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "inferred",
|
||||
authority: "analyst-inference",
|
||||
@@ -37,7 +38,7 @@ export const createJavaScriptSemanticTraceEvidence = (
|
||||
predicateType: "rea.javascript-semantic-trace",
|
||||
operation: "trace_javascript_semantics",
|
||||
parameters,
|
||||
result,
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "inferred",
|
||||
authority: "analyst-inference",
|
||||
@@ -55,7 +56,7 @@ export const createApplicationVersionComparisonEvidence = (
|
||||
predicateType: "rea.application-version-comparison",
|
||||
operation: "compare_application_versions",
|
||||
parameters,
|
||||
result,
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "inferred",
|
||||
authority: "analyst-inference",
|
||||
@@ -73,7 +74,7 @@ export const createSourceToBundleComparisonEvidence = (
|
||||
predicateType: "rea.source-to-bundle-comparison",
|
||||
operation: "compare_source_to_bundle",
|
||||
parameters,
|
||||
result,
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "inferred",
|
||||
authority: "analyst-inference",
|
||||
@@ -91,7 +92,7 @@ export const createJavaScriptExportShapeComparisonEvidence = (
|
||||
predicateType: "rea.javascript-export-shape-comparison",
|
||||
operation: "compare_javascript_export_shapes",
|
||||
parameters,
|
||||
result,
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "inferred",
|
||||
authority: "analyst-inference",
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import type { JavaScriptBundlerRegistration } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import {
|
||||
addAstContainsEdge,
|
||||
artifactLocalIdentity,
|
||||
@@ -81,11 +80,7 @@ const createBundlerChunkNode = (
|
||||
const chunkKey = `${registration.runtime}:${registration.chunk_keys.join(",")}`;
|
||||
return context.accumulator.addNode({
|
||||
kind: "javascript-chunk",
|
||||
identity: artifactLocalIdentity(
|
||||
file.sha256,
|
||||
"bundler-chunk",
|
||||
compositeKey([registration.runtime, registration.chunk_keys]),
|
||||
),
|
||||
identity: artifactLocalIdentity(file.sha256, "bundler-chunk", chunkKey),
|
||||
observations: [
|
||||
{
|
||||
label: chunkKey,
|
||||
@@ -161,7 +156,7 @@ const addBundlerModuleNodes = (
|
||||
],
|
||||
});
|
||||
context.moduleNodes.set(
|
||||
moduleLookupKey(file.path, registration.runtime, moduleValue.module_key),
|
||||
moduleLookupKey(file.path, moduleValue.module_key),
|
||||
module,
|
||||
);
|
||||
addAstContainsEdge(context, {
|
||||
@@ -195,7 +190,7 @@ const addBundlerRuntimeEdges = (
|
||||
const { file, registration, chunk, coverage } = record;
|
||||
for (const moduleKey of registration.entry_module_keys) {
|
||||
const resolved = context.moduleNodes.get(
|
||||
moduleLookupKey(file.path, registration.runtime, moduleKey),
|
||||
moduleLookupKey(file.path, moduleKey),
|
||||
);
|
||||
const target =
|
||||
resolved ?? unresolvedBundlerModuleNode(context, record, moduleKey);
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
import {
|
||||
completeApplicationCoverage,
|
||||
partialApplicationCoverage,
|
||||
@@ -323,18 +322,15 @@ export const selfReferenceOmissions = (
|
||||
];
|
||||
|
||||
/** Deterministic key for one recovered module inside a bundle asset. */
|
||||
export const moduleLookupKey = (
|
||||
path: string,
|
||||
runtime: string,
|
||||
moduleKey: string,
|
||||
): string => compositeKey([path, runtime, moduleKey]);
|
||||
export const moduleLookupKey = (path: string, moduleKey: string): string =>
|
||||
`${path}\0${moduleKey}`;
|
||||
|
||||
/** Deterministic key for one recovered bundler chunk inside a bundle asset. */
|
||||
export const chunkLookupKey = (
|
||||
path: string,
|
||||
runtime: string,
|
||||
chunkKey: string,
|
||||
): string => compositeKey([path, runtime, chunkKey]);
|
||||
): string => `${path}\0${runtime}\0${chunkKey}`;
|
||||
|
||||
/** Resolve a relative static specifier without escaping the artifact root. */
|
||||
export const resolveArtifactPath = (
|
||||
@@ -356,10 +352,7 @@ export const sourceNodeFor = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
path: string,
|
||||
moduleKey: string | null,
|
||||
runtime?: string,
|
||||
): ApplicationNode | undefined =>
|
||||
moduleKey === null
|
||||
? context.sourceModuleNodes.get(path)
|
||||
: runtime === undefined
|
||||
? undefined
|
||||
: context.moduleNodes.get(moduleLookupKey(path, runtime, moduleKey));
|
||||
: context.moduleNodes.get(moduleLookupKey(path, moduleKey));
|
||||
|
||||
@@ -78,12 +78,8 @@ const addReference = (
|
||||
omissions: { selfReferences: number },
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
const resolved =
|
||||
input.value.specifier === null
|
||||
? null
|
||||
@@ -166,12 +162,8 @@ const addEndpoint = (
|
||||
input: FindingInput<StaticEndpoint>,
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
const key = compactStringIdentityKey(input.value.value);
|
||||
const label = `${input.value.kind} endpoint`;
|
||||
const node = context.accumulator.addNode({
|
||||
@@ -212,12 +204,8 @@ const addStorage = (
|
||||
input: FindingInput<StaticStorage>,
|
||||
): void => {
|
||||
const source =
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
const node = context.accumulator.addNode({
|
||||
kind: "storage",
|
||||
identity: artifactLocalIdentity(
|
||||
@@ -274,12 +262,8 @@ const addDiscoveredRole = (
|
||||
coverage: input.coverage,
|
||||
});
|
||||
const source =
|
||||
sourceNodeFor(
|
||||
context,
|
||||
input.file.path,
|
||||
input.value.module_key,
|
||||
input.value.module_runtime,
|
||||
) ?? input.asset;
|
||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||
input.asset;
|
||||
addStaticInferenceEdge(context, {
|
||||
source,
|
||||
target: role,
|
||||
@@ -375,12 +359,9 @@ const resolveReference = (
|
||||
if (specifier === null) return null;
|
||||
const chunk = resolveBundlerChunkReference(context, sourcePath, reference);
|
||||
if (chunk !== null) return chunk;
|
||||
const module =
|
||||
reference.module_runtime === undefined
|
||||
? undefined
|
||||
: context.moduleNodes.get(
|
||||
moduleLookupKey(sourcePath, reference.module_runtime, specifier),
|
||||
);
|
||||
const module = context.moduleNodes.get(
|
||||
moduleLookupKey(sourcePath, specifier),
|
||||
);
|
||||
if (module !== undefined) {
|
||||
const file = context.filesByPath.get(sourcePath);
|
||||
return file === undefined
|
||||
@@ -426,11 +407,10 @@ const resolveBundlerChunkReference = (
|
||||
if (file === undefined) return null;
|
||||
const registrations = context.analysis.files
|
||||
.find(({ file: analyzedFile }) => analyzedFile.path === sourcePath)
|
||||
?.javascript?.bundler_registrations.filter(({ runtime, modules }) =>
|
||||
reference.module_runtime === undefined
|
||||
? reference.module_key === null
|
||||
: runtime === reference.module_runtime &&
|
||||
modules.some(
|
||||
?.javascript?.bundler_registrations.filter(({ modules }) =>
|
||||
reference.module_key === null
|
||||
? true
|
||||
: modules.some(
|
||||
({ module_key: moduleKey }) => moduleKey === reference.module_key,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -115,18 +115,9 @@ export const addJavaScriptArtifactContainers = (
|
||||
export const addJavaScriptArtifactFiles = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): void => {
|
||||
const packagesByPath = indexFirstByPath(context.analysis.packages);
|
||||
const jsonModulesByPath = indexFirstByPath(context.analysis.json_modules);
|
||||
const sourceMapsByPath = indexFirstByPath(context.analysis.source_maps);
|
||||
for (const analyzed of context.analysis.files) {
|
||||
const { file } = analyzed;
|
||||
const jsonValue = jsonModulesByPath.get(file.path);
|
||||
const target = createFileTarget(
|
||||
context,
|
||||
file,
|
||||
analyzed.javascript,
|
||||
jsonValue,
|
||||
);
|
||||
const target = createFileTarget(context, file, analyzed.javascript);
|
||||
context.fileNodes.set(file.path, target);
|
||||
if (file.kind === "javascript") context.assetNodes.set(file.path, target);
|
||||
const entry = createAsarEntry(context, file);
|
||||
@@ -172,7 +163,9 @@ export const addJavaScriptArtifactFiles = (
|
||||
? "JavaScript syntax could not be parsed."
|
||||
: `JavaScript text was unavailable: ${file.text.reason}.`,
|
||||
});
|
||||
const packageValue = packagesByPath.get(file.path);
|
||||
const packageValue = context.analysis.packages.find(
|
||||
({ path }) => path === file.path,
|
||||
);
|
||||
if (packageValue !== undefined && packageValue.status !== "included")
|
||||
addUnavailableStaticParseScope(context, {
|
||||
file,
|
||||
@@ -180,6 +173,9 @@ export const addJavaScriptArtifactFiles = (
|
||||
operation: "parse-package-json",
|
||||
limitation: packageValue.limitation,
|
||||
});
|
||||
const jsonValue = context.analysis.json_modules.find(
|
||||
({ path }) => path === file.path,
|
||||
);
|
||||
if (jsonValue !== undefined && jsonValue.status !== "included")
|
||||
addUnavailableStaticParseScope(context, {
|
||||
file,
|
||||
@@ -187,7 +183,9 @@ export const addJavaScriptArtifactFiles = (
|
||||
operation: "parse-json-module",
|
||||
limitation: jsonValue.limitation,
|
||||
});
|
||||
const sourceMap = sourceMapsByPath.get(file.path);
|
||||
const sourceMap = context.analysis.source_maps.find(
|
||||
({ path }) => path === file.path,
|
||||
);
|
||||
if (sourceMap !== undefined && sourceMap.status === "invalid")
|
||||
addUnavailableStaticParseScope(context, {
|
||||
file,
|
||||
@@ -272,9 +270,11 @@ const createFileTarget = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
javascript: JavaScriptModuleArtifactAnalysis["files"][number]["javascript"],
|
||||
json: JavaScriptModuleArtifactAnalysis["json_modules"][number] | undefined,
|
||||
): ApplicationNode => {
|
||||
const kind = artifactFileNodeKind(file.kind);
|
||||
const json = context.analysis.json_modules.find(
|
||||
({ path }) => path === file.path,
|
||||
);
|
||||
return context.accumulator.addNode({
|
||||
kind,
|
||||
identity: {
|
||||
@@ -309,16 +309,6 @@ const createFileTarget = (
|
||||
});
|
||||
};
|
||||
|
||||
const indexFirstByPath = <Value extends { readonly path: string }>(
|
||||
values: readonly Value[],
|
||||
): ReadonlyMap<string, Value> => {
|
||||
const index = new Map<string, Value>();
|
||||
for (const value of values) {
|
||||
if (!index.has(value.path)) index.set(value.path, value);
|
||||
}
|
||||
return index;
|
||||
};
|
||||
|
||||
const createAsarEntry = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
|
||||
@@ -234,8 +234,7 @@ const resolvePackageSpecifier = (
|
||||
if (rawExports === undefined || rawExports === null) {
|
||||
if (subpath === ".")
|
||||
return resolveDirectory(input, directory, metadata, new Set());
|
||||
// Preserve the trailing slash for a package directory request.
|
||||
const candidate = confineCandidate(`${directory}/${subpath.slice(2)}`);
|
||||
const candidate = confineCandidate(posix.join(directory, subpath.slice(2)));
|
||||
return typeof candidate === "string"
|
||||
? resolveCandidate(input, candidate, new Set())
|
||||
: candidate;
|
||||
|
||||
@@ -12,7 +12,6 @@ import type { JavaScriptApplicationGraph } from "../../domain/javascript/javascr
|
||||
import type { JavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticGraph.js";
|
||||
import type { ElectronBoundarySummary } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import type { ArtifactInventoryPartialObservation } from "../../domain/artifactPartialObservation.js";
|
||||
import { analyzeAndProjectJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
|
||||
import { readJavaScriptArtifactFiles } from "../../artifacts/javascript/JavaScriptArtifactFiles.js";
|
||||
import { buildImmutableJavaScriptArtifactGraphSteps } from "./JavaScriptArtifactGraphBuilder.js";
|
||||
@@ -20,8 +19,7 @@ import {
|
||||
javascriptArtifactReconstructionInputSchema,
|
||||
type JavaScriptArtifactReconstructionInput,
|
||||
} from "./JavaScriptArtifactReconstructionInput.js";
|
||||
import { scanCanonicalArtifactInventoryInScope } from "../../artifacts/inventory/scanCanonical.js";
|
||||
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
||||
import { scanCanonicalArtifactInventory } from "../../artifacts/inventory/scanCanonical.js";
|
||||
import { summarizeElectronBoundaries } from "./ElectronBoundaryAnalysis.js";
|
||||
import { createJavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
|
||||
import type { ProgressReporter } from "../ProgressReporter.js";
|
||||
@@ -58,161 +56,118 @@ export interface JavaScriptArtifactReconstructionResult {
|
||||
/** Reconstruct one local ASAR or extracted directory without executing code. */
|
||||
export const reconstructJavaScriptArtifact = async (
|
||||
rawInput: unknown,
|
||||
scope: ArtifactResourceScope,
|
||||
signal?: AbortSignal,
|
||||
progress?: ProgressReporter,
|
||||
): Promise<JavaScriptArtifactReconstructionResult> =>
|
||||
scope.run(async () => {
|
||||
const reportPhase = async (
|
||||
phase: string,
|
||||
message: string,
|
||||
): Promise<void> => {
|
||||
await progress?.report({ phase, completed: 0, total: 1, message });
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
};
|
||||
const input = javascriptArtifactReconstructionInputSchema.parse(rawInput);
|
||||
abortIfNeeded(signal);
|
||||
const path = await resolveSelectedInput(input.input_path);
|
||||
const format = await resolveFormat(path, input);
|
||||
const snapshot = await scanCanonicalArtifactInventoryInScope(path, {
|
||||
resourceScope: scope,
|
||||
signal,
|
||||
integrity: { mode: input.integrity_policy },
|
||||
});
|
||||
if (snapshot.manifest.root_format !== format)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`Artifact inventory classified ${path} as ${snapshot.manifest.root_format}, not ${format}`,
|
||||
);
|
||||
const reader = createReader(path, format);
|
||||
let outcome:
|
||||
| {
|
||||
readonly kind: "completed";
|
||||
readonly result: JavaScriptArtifactReconstructionResult;
|
||||
}
|
||||
| { readonly kind: "failed"; readonly cause: unknown };
|
||||
try {
|
||||
await reportPhase(
|
||||
"read_javascript_artifacts",
|
||||
"Reading inventoried JavaScript application sources",
|
||||
);
|
||||
const files = await readJavaScriptArtifactFiles(
|
||||
reader,
|
||||
snapshot,
|
||||
scope,
|
||||
signal,
|
||||
);
|
||||
await reportPhase(
|
||||
"parse_javascript_sources",
|
||||
`Parsing and projecting ${String(files.files.length)} application source files`,
|
||||
);
|
||||
const semanticProjection = createJavaScriptSemanticGraphProjection();
|
||||
const analysis = await analyzeAndProjectJavaScriptArtifactFiles(
|
||||
files,
|
||||
semanticProjection.projectFileSteps,
|
||||
async (file, completed, total) => {
|
||||
abortIfNeeded(signal);
|
||||
await progress?.report({
|
||||
phase: "parse_javascript_source",
|
||||
completed: 0,
|
||||
total: 1,
|
||||
message: `Parsing and projecting ${file.path} (${String(completed + 1)}/${String(total)})`,
|
||||
});
|
||||
abortIfNeeded(signal);
|
||||
},
|
||||
signal,
|
||||
);
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
abortIfNeeded(signal);
|
||||
await reportPhase(
|
||||
"build_javascript_application_graph",
|
||||
"Constructing application and Electron boundary relationships",
|
||||
);
|
||||
const applicationGraphSteps = buildImmutableJavaScriptArtifactGraphSteps(
|
||||
snapshot,
|
||||
files,
|
||||
analysis,
|
||||
);
|
||||
await reportPhase(
|
||||
"seal_javascript_application_graph",
|
||||
"Sealing the validated application graph",
|
||||
);
|
||||
const graph = await completeJavaScriptAnalysisSteps(
|
||||
applicationGraphSteps,
|
||||
signal,
|
||||
);
|
||||
await reportPhase(
|
||||
"build_javascript_semantic_graph",
|
||||
"Binding and validating static semantic relationships",
|
||||
);
|
||||
const semanticGraphSteps = semanticProjection.finishImmutableSteps(
|
||||
snapshot.manifest.root_sha256,
|
||||
graph,
|
||||
);
|
||||
await reportPhase(
|
||||
"seal_javascript_semantic_graph",
|
||||
"Validating and sealing the semantic graph",
|
||||
);
|
||||
const semanticGraph = await completeJavaScriptAnalysisSteps(
|
||||
semanticGraphSteps,
|
||||
signal,
|
||||
);
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
outcome = {
|
||||
kind: "completed",
|
||||
result: {
|
||||
input_path: path,
|
||||
format,
|
||||
root_artifact_sha256: snapshot.manifest.root_sha256,
|
||||
inventory_manifest_id: snapshot.manifest.manifest_id,
|
||||
inventory_graph_sha256: snapshot.manifest.graph_sha256,
|
||||
integrity_contradictions: snapshot.integrity_contradictions,
|
||||
graph,
|
||||
semantic_graph: semanticGraph,
|
||||
electron_summary: summarizeElectronBoundaries(analysis),
|
||||
statistics: {
|
||||
relevant_files: files.files.length,
|
||||
nested_asar_containers: files.containers.length,
|
||||
text_bytes_read: files.text_bytes_read,
|
||||
invalid_utf8_files: files.invalid_utf8_files,
|
||||
parsed_javascript_files: analysis.files.filter(
|
||||
({ javascript }) => javascript !== null,
|
||||
).length,
|
||||
visited_ast_nodes: analysis.visited_ast_nodes,
|
||||
findings: analysis.findings,
|
||||
modules: analysis.modules,
|
||||
parse_failures: analysis.parse_failures,
|
||||
},
|
||||
limitations: analysis.limitations,
|
||||
},
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
outcome = { kind: "failed", cause };
|
||||
}
|
||||
const owner = {
|
||||
kind: "reader" as const,
|
||||
reader,
|
||||
resource: `JavaScript artifact reader for ${path}`,
|
||||
};
|
||||
const cleanupAttempt = await scope.release(owner);
|
||||
if (cleanupAttempt.kind === "failed") {
|
||||
const cleanup = ArtifactReaderFailure.cleanupObservation(
|
||||
cleanupAttempt.cause,
|
||||
owner.resource,
|
||||
);
|
||||
const partialObservation: ArtifactInventoryPartialObservation = {
|
||||
kind: "artifact-inventory",
|
||||
inventory: snapshot,
|
||||
};
|
||||
throw ArtifactReaderFailure.withCleanup(
|
||||
outcome.kind === "failed" ? outcome.cause : cleanupAttempt.cause,
|
||||
cleanup,
|
||||
partialObservation,
|
||||
);
|
||||
}
|
||||
if (outcome.kind === "failed") throw outcome.cause;
|
||||
return outcome.result;
|
||||
): Promise<JavaScriptArtifactReconstructionResult> => {
|
||||
const reportPhase = async (phase: string, message: string): Promise<void> => {
|
||||
await progress?.report({ phase, completed: 0, total: 1, message });
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
};
|
||||
const input = javascriptArtifactReconstructionInputSchema.parse(rawInput);
|
||||
abortIfNeeded(signal);
|
||||
const path = await resolveSelectedInput(input.input_path);
|
||||
const format = await resolveFormat(path, input);
|
||||
const snapshot = await scanCanonicalArtifactInventory(path, {
|
||||
signal,
|
||||
integrity: { mode: input.integrity_policy },
|
||||
});
|
||||
if (snapshot.manifest.root_format !== format)
|
||||
throw new ArtifactReaderFailure(
|
||||
"format",
|
||||
`Artifact inventory classified ${path} as ${snapshot.manifest.root_format}, not ${format}`,
|
||||
);
|
||||
const reader = createReader(path, format);
|
||||
try {
|
||||
await reportPhase(
|
||||
"read_javascript_artifacts",
|
||||
"Reading inventoried JavaScript application sources",
|
||||
);
|
||||
const files = await readJavaScriptArtifactFiles(reader, snapshot, signal);
|
||||
await reportPhase(
|
||||
"parse_javascript_sources",
|
||||
`Parsing and projecting ${String(files.files.length)} application source files`,
|
||||
);
|
||||
const semanticProjection = createJavaScriptSemanticGraphProjection();
|
||||
const analysis = await analyzeAndProjectJavaScriptArtifactFiles(
|
||||
files,
|
||||
semanticProjection.projectFileSteps,
|
||||
async (file, completed, total) => {
|
||||
abortIfNeeded(signal);
|
||||
await progress?.report({
|
||||
phase: "parse_javascript_source",
|
||||
completed: 0,
|
||||
total: 1,
|
||||
message: `Parsing and projecting ${file.path} (${String(completed + 1)}/${String(total)})`,
|
||||
});
|
||||
abortIfNeeded(signal);
|
||||
},
|
||||
signal,
|
||||
);
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
abortIfNeeded(signal);
|
||||
await reportPhase(
|
||||
"build_javascript_application_graph",
|
||||
"Constructing application and Electron boundary relationships",
|
||||
);
|
||||
const applicationGraphSteps = buildImmutableJavaScriptArtifactGraphSteps(
|
||||
snapshot,
|
||||
files,
|
||||
analysis,
|
||||
);
|
||||
await reportPhase(
|
||||
"seal_javascript_application_graph",
|
||||
"Sealing the validated application graph",
|
||||
);
|
||||
const graph = await completeJavaScriptAnalysisSteps(
|
||||
applicationGraphSteps,
|
||||
signal,
|
||||
);
|
||||
await reportPhase(
|
||||
"build_javascript_semantic_graph",
|
||||
"Binding and validating static semantic relationships",
|
||||
);
|
||||
const semanticGraphSteps = semanticProjection.finishImmutableSteps(
|
||||
snapshot.manifest.root_sha256,
|
||||
graph,
|
||||
);
|
||||
await reportPhase(
|
||||
"seal_javascript_semantic_graph",
|
||||
"Validating and sealing the semantic graph",
|
||||
);
|
||||
const semanticGraph = await completeJavaScriptAnalysisSteps(
|
||||
semanticGraphSteps,
|
||||
signal,
|
||||
);
|
||||
await checkpointJavaScriptAnalysis(signal);
|
||||
return {
|
||||
input_path: path,
|
||||
format,
|
||||
root_artifact_sha256: snapshot.manifest.root_sha256,
|
||||
inventory_manifest_id: snapshot.manifest.manifest_id,
|
||||
inventory_graph_sha256: snapshot.manifest.graph_sha256,
|
||||
integrity_contradictions: snapshot.integrity_contradictions,
|
||||
graph,
|
||||
semantic_graph: semanticGraph,
|
||||
electron_summary: summarizeElectronBoundaries(analysis),
|
||||
statistics: {
|
||||
relevant_files: files.files.length,
|
||||
nested_asar_containers: files.containers.length,
|
||||
text_bytes_read: files.text_bytes_read,
|
||||
invalid_utf8_files: files.invalid_utf8_files,
|
||||
parsed_javascript_files: analysis.files.filter(
|
||||
({ javascript }) => javascript !== null,
|
||||
).length,
|
||||
visited_ast_nodes: analysis.visited_ast_nodes,
|
||||
findings: analysis.findings,
|
||||
modules: analysis.modules,
|
||||
parse_failures: analysis.parse_failures,
|
||||
},
|
||||
limitations: analysis.limitations,
|
||||
};
|
||||
} finally {
|
||||
await reader.close();
|
||||
}
|
||||
};
|
||||
|
||||
const OPERATION = "analyze_javascript_application";
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ import { describe, expect, it } from "vitest";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { staticPath } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
|
||||
const urlCases = [
|
||||
["./preload%20name.cjs", "preload name.cjs", "main.mjs"],
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
import { posix } from "node:path";
|
||||
|
||||
import {
|
||||
compareUnicodeCodePoints,
|
||||
compositeKey,
|
||||
} from "../../domain/unicodeCodePointOrder.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import {
|
||||
completeApplicationCoverage,
|
||||
@@ -180,7 +177,7 @@ const addExportRelationship = (
|
||||
identity: artifactLocalIdentity(
|
||||
file.sha256,
|
||||
"module-export",
|
||||
compositeKey([file.container_sha256, file.path, link.exportedName]),
|
||||
`${file.container_sha256}:${file.path}:${link.exportedName}`,
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user