Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9d4dce1263 | ||
|
|
1e034a80fe | ||
|
|
e28762a5c7 | ||
|
|
c1f1570a0c | ||
|
|
bcc3ad6014 | ||
|
|
258ecd361b |
@@ -2,7 +2,7 @@
|
|||||||
name: reverse-engineer-anything
|
name: reverse-engineer-anything
|
||||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||||
metadata:
|
metadata:
|
||||||
version: "37"
|
version: "36"
|
||||||
---
|
---
|
||||||
|
|
||||||
# REA
|
# REA
|
||||||
@@ -25,18 +25,57 @@ a skill installed from repository main may describe capabilities absent from
|
|||||||
an older npm release. Keep complete inline Evidence when that server does not
|
an older npm release. Keep complete inline Evidence when that server does not
|
||||||
advertise retained references.
|
advertise retained references.
|
||||||
|
|
||||||
If tools are absent, a call is blocked by the client, or a required provider is
|
When tools are absent or registration is stale:
|
||||||
unavailable, read [connection and recovery](references/connection-and-recovery.md).
|
|
||||||
Diagnose the observed blocker, continue useful supported work, and resume the
|
|
||||||
original task after repair. Client approval, shell PATH, registration and engine
|
|
||||||
availability have different remedies; avoid repeating setup for all of them.
|
|
||||||
|
|
||||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
1. Diagnose without changing files. For Codex, run
|
||||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||||
skill replacement before applying authorized changes. Preserve existing approval
|
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
`commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`, `hermes`, or `grok_bot`. `grok_bot` has no local
|
||||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Binwalk or Unblob.
|
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||||
|
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||||
|
inspect its registration results before choosing a setup scope.
|
||||||
|
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||||
|
scoped configuration repair. An aligned registration with no tools in the
|
||||||
|
active session needs a restart/reconnection; doctor cannot prove that the
|
||||||
|
current agent has connected. A missing provider affects only tasks requiring
|
||||||
|
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||||
|
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||||
|
3. For a configuration repair, prepare the read-only plan:
|
||||||
|
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||||
|
Use the current client's ID, show its exact proposed paths, backups, and
|
||||||
|
changes, and obtain approval before setup writes configuration or installs
|
||||||
|
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||||
|
replacement in the reviewed plan. After approval, apply the same scope with
|
||||||
|
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||||
|
only if that separate installation was needed and explicitly approved.
|
||||||
|
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||||
|
in the session, then resume the original investigation. If they remain
|
||||||
|
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||||
|
|
||||||
|
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||||
|
JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||||
|
software to repair
|
||||||
|
MCP registration. For an unsupported client, use manual stdio registration
|
||||||
|
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||||
|
|
||||||
|
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||||
|
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||||
|
npx -y rea-agents@latest inspect-analysis-view app-view.json
|
||||||
|
```
|
||||||
|
|
||||||
|
No MCP registration or native engine is required. The CLI returns the complete
|
||||||
|
Evidence record directly, and a real application can produce hundreds of
|
||||||
|
megabytes, so save it to a file rather than reading it into context. Start from
|
||||||
|
the summary view, then request a module page
|
||||||
|
(`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}`) or one
|
||||||
|
module the same way, or pass the saved Evidence inline to
|
||||||
|
`trace-application-feature`. Read graph, limitations, and unknowns with the
|
||||||
|
same care as an MCP result. This fallback does not establish that MCP is
|
||||||
|
configured. Native CLI tasks still require their selected engine.
|
||||||
|
|
||||||
## Route the target first
|
## Route the target first
|
||||||
|
|
||||||
@@ -101,11 +140,9 @@ after opening; IDA does not supply `binary_overview`. Consult session availabili
|
|||||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||||
facets remain unknown; live IDA results are not replayed from snapshots.
|
facets remain unknown; live IDA results are not replayed from snapshots.
|
||||||
|
|
||||||
If the user says only "my app" without a name, path, or target already selected
|
If the app is missing, ask which app to inspect. Resolve a human-readable app
|
||||||
for this task, ask which app before searching for artifacts. Paths in unrelated
|
name to one clear installed artifact when possible; ask only when matches are
|
||||||
saved reports and nearby test fixtures do not select the user's target. Resolve
|
ambiguous. Never choose an example app on the user's behalf.
|
||||||
an actual human-readable app name to one clear installed artifact when possible;
|
|
||||||
ask when matches are ambiguous. Never choose an example app on the user's behalf.
|
|
||||||
|
|
||||||
In a target-free session, use `open_binary` to bind any archive/package or
|
In a target-free session, use `open_binary` to bind any archive/package or
|
||||||
native target whose analysis tool operates on the active target. Do not call an
|
native target whose analysis tool operates on the active target. Do not call an
|
||||||
@@ -120,11 +157,7 @@ For a JavaScript application or ELF layout whose size is unknown or large,
|
|||||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||||
`inspect_analysis_view`.
|
`inspect_analysis_view`.
|
||||||
Page module identities and select the returned `node_id` before requesting an
|
Do not repeat an identical tool call. Make a focused follow-up only when the
|
||||||
item; a single file path can represent several graph roles. Do not repeat an
|
|
||||||
identical analysis or rejected call without a changed input or resolved blocker.
|
|
||||||
Recheck session state when a lifecycle change makes that status useful.
|
|
||||||
Make a focused follow-up only when the
|
|
||||||
returned result leaves a specific question unanswered.
|
returned result leaves a specific question unanswered.
|
||||||
|
|
||||||
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
||||||
@@ -135,14 +168,6 @@ session to a selected path. The analysis remains complete in the ledger; avoid
|
|||||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||||
output also streams without a single MCP frame.
|
output also streams without a single MCP frame.
|
||||||
|
|
||||||
A client can also truncate a successful tool result without a REA transport
|
|
||||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
|
||||||
views or export when the full record is needed. Do not parse the preview as
|
|
||||||
canonical JSON or repeat the producer. For JavaScript, read the
|
|
||||||
[module-view and trace recipes](references/javascript-applications.md) before
|
|
||||||
drilling into a large graph; a broad bidirectional trace can expand unrelated
|
|
||||||
containment relationships even after summary analysis.
|
|
||||||
|
|
||||||
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
||||||
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
||||||
that static analysis observed execution. Runtime requests execute the declared
|
that static analysis observed execution. Runtime requests execute the declared
|
||||||
@@ -150,9 +175,7 @@ target and lifecycle; do not broaden the target or action beyond those fields.
|
|||||||
|
|
||||||
Within the user's requested investigation, call available analysis tools
|
Within the user's requested investigation, call available analysis tools
|
||||||
directly. REA does not require permission grants or per-call approval flags.
|
directly. REA does not require permission grants or per-call approval flags.
|
||||||
The agent client can independently require approval. Follow the declared request
|
Follow the declared request scope and the host's actual access requirements.
|
||||||
scope and the host's actual access requirements; use the recovery guide for an
|
|
||||||
observed client-policy rejection.
|
|
||||||
|
|
||||||
## Plan broader investigations
|
## Plan broader investigations
|
||||||
|
|
||||||
|
|||||||
@@ -1,127 +0,0 @@
|
|||||||
# Connection and task recovery
|
|
||||||
|
|
||||||
Read this guide only for an observed connection, command, approval or provider
|
|
||||||
blocker. Keep the user's target and question in view. Explain what prevented the
|
|
||||||
operation and the next useful step; after repair, continue the investigation.
|
|
||||||
|
|
||||||
Commands below use an installed `rea`. If none is available, use the package
|
|
||||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
|
||||||
server's schemas and the installed CLI's help rather than assuming repository
|
|
||||||
main and an older release expose the same arguments.
|
|
||||||
|
|
||||||
## Choose the remedy from the observed failure
|
|
||||||
|
|
||||||
| Observation | Meaning and next step |
|
|
||||||
| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
|
||||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
|
||||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
|
||||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
|
||||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Resolve only the prerequisite needed for the task; a complete tool catalog does not imply every operation is available. |
|
|
||||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
|
||||||
|
|
||||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
|
||||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
|
||||||
CODEX_HOME is under `/tmp`; use an authorized profile outside the OS temporary
|
|
||||||
directory. REA setup cannot repair the client's executor. Preserve the sandbox
|
|
||||||
instead of silently disabling it to make an analysis command work.
|
|
||||||
|
|
||||||
Static JavaScript and ordinary ZIP inventory need neither Hopper nor Ghidra.
|
|
||||||
HAR inspection is offline and does not require a live browser. Native
|
|
||||||
decompilation, Android analysis and firmware extraction have their own
|
|
||||||
bring-your-own prerequisites. Use supported inventory/static evidence when it
|
|
||||||
answers part of the question, and name the unavailable evidence separately.
|
|
||||||
Do not substitute another target. Label observations from ordinary shell tools
|
|
||||||
as such; only actual REA output supplies REA Evidence.
|
|
||||||
|
|
||||||
## Repair registration when needed
|
|
||||||
|
|
||||||
For Codex:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
rea doctor --client codex --json
|
|
||||||
rea setup --client codex --dry-run --json
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
|
||||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
|
||||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`,
|
|
||||||
`hermes`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
|
||||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
|
||||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
|
||||||
|
|
||||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
|
||||||
Obtain approval for writes when that concrete scope is not already authorized,
|
|
||||||
then apply the same scope with `rea setup --client codex --yes`. Add
|
|
||||||
`--install-hopper` only when separately needed and explicitly approved. Do not
|
|
||||||
install or upgrade unrelated prerequisites. Restart/reconnect, verify tools
|
|
||||||
actually appear, and resume the original task. If they remain absent, inspect
|
|
||||||
the launch failure rather than looping through setup.
|
|
||||||
|
|
||||||
## Codex tool approval is separate from REA readiness
|
|
||||||
|
|
||||||
REA tools have no permission-grant handshake. Codex can still require approval
|
|
||||||
because an operation creates owned resources, launches a helper or has cleanup
|
|
||||||
effects. An advertised tool or successful doctor result does not override that
|
|
||||||
client policy. `workspace-write` alone does not resolve an MCP approval
|
|
||||||
rejection, and `approval_policy = "never"` does not mean every MCP tool is
|
|
||||||
preapproved.
|
|
||||||
|
|
||||||
When interactive approval is available, use the client's approval flow for the
|
|
||||||
requested operation. In an unattended run, continue through an authorized CLI
|
|
||||||
workflow if the execution environment permits it, or report the blocked
|
|
||||||
operation and the approval needed to proceed. Do not retry the same rejected
|
|
||||||
call, silently broaden approvals, or rewrite MCP effects as read-only.
|
|
||||||
|
|
||||||
For an operator who explicitly wants to preapprove named operations in a
|
|
||||||
disposable Codex profile, current Codex supports per-tool configuration. For
|
|
||||||
example, opening and closing an analysis session can be scoped independently:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[mcp_servers.rea.tools.open_binary]
|
|
||||||
approval_mode = "approve"
|
|
||||||
|
|
||||||
[mcp_servers.rea.tools.close_binary]
|
|
||||||
approval_mode = "approve"
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the actual server ID and only the operations whose effects were approved;
|
|
||||||
this example does not approve inspection, extraction or runtime execution.
|
|
||||||
Show the proposed config edit and preserve unrelated entries and backups before
|
|
||||||
writing. Managed policy can constrain client settings. Consult the
|
|
||||||
[Codex configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
|
||||||
for the installed client's supported settings. This is an optional recovery
|
|
||||||
choice, not a prerequisite for every REA task.
|
|
||||||
|
|
||||||
## Continue through an installed CLI
|
|
||||||
|
|
||||||
Check the command environment only if the launcher fails:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
command -v rea
|
|
||||||
command -v node
|
|
||||||
node --version
|
|
||||||
rea --version
|
|
||||||
```
|
|
||||||
|
|
||||||
Use the public launcher at its known installation path with a supported existing
|
|
||||||
Node on PATH when the child shell loses a prefix installation. Avoid internal
|
|
||||||
`dist/cli.js` entrypoints or global toolchain changes. MCP may have its own valid
|
|
||||||
launcher even when a shell cannot resolve `rea`.
|
|
||||||
|
|
||||||
For a user-supplied JavaScript tree or ASAR, select output files in the authorized
|
|
||||||
workspace and save complete Evidence before reading a view:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
|
||||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
|
||||||
rea inspect-analysis-view app-view.json
|
|
||||||
```
|
|
||||||
|
|
||||||
These commands do not register MCP or install an analysis engine. CLI JavaScript
|
|
||||||
JSON is complete and can be very large; redirect it rather than loading it into
|
|
||||||
context. Use an existing JSON processor if `jq` is absent, without installing
|
|
||||||
one just for this recipe. Inspect a module page or item using the same inline
|
|
||||||
source and the [JavaScript recipes](javascript-applications.md). Preserve graph
|
|
||||||
coverage, limitations and unknowns. Native CLI analysis still requires its
|
|
||||||
selected engine, and all CLI operations retain the host's execution permissions.
|
|
||||||
@@ -1,106 +1,20 @@
|
|||||||
# JavaScript and Electron application artifacts
|
# JavaScript and Electron application artifacts
|
||||||
|
|
||||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||||
extracted tree. When size is unknown or large and the connected schema advertises
|
extracted tree. The complete result, graph, and Evidence context are returned
|
||||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
inline when they fit the MCP frame. Source-map contents are part of the static analysis. Start from its
|
||||||
Evidence record with a distinct ID and a `normalized_result.parent_evidence_id`
|
findings, coverage, unknowns, and graph context, then make a focused follow-up
|
||||||
for the complete retained application analysis. Source-map contents are part of
|
only when a specific question remains unanswered.
|
||||||
that static analysis. Read findings, coverage, limitations and unknowns, then
|
|
||||||
follow a specific unanswered question. Older servers without `detail` return
|
|
||||||
the complete application Evidence inline when it fits.
|
|
||||||
|
|
||||||
When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
|
|
||||||
these select the isolated source analyzer's old-space heap and per-source
|
|
||||||
deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
|
|
||||||
process heap. Resource-limited analyses preserve completed static facts and
|
|
||||||
report partial semantic coverage with specific unknown scopes; this does not
|
|
||||||
mean the source is malformed. Inspect those limits before retrying a source.
|
|
||||||
When expanded relationships exceed the parent's available memory, inspect the
|
|
||||||
file's modules item view for retained `static_analysis` and
|
|
||||||
`semantic_module_analysis`. Their facts remain available even when graph links
|
|
||||||
are unknown. Cumulative result exhaustion can stop analysis before every file
|
|
||||||
is examined; the inventory and remaining unknown scopes distinguish that case.
|
|
||||||
Timeout or cancellation after source analysis begins retains partial Evidence
|
|
||||||
in the same session. A returned `error.details.partial_observation` reference
|
|
||||||
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
|
|
||||||
the Evidence bundle before inspecting the partial analysis.
|
|
||||||
|
|
||||||
## Read a module without repeating analysis
|
|
||||||
|
|
||||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
|
||||||
The following IDs are placeholders to replace with returned values. When the
|
|
||||||
module's node ID is not already known, page identities before selecting an item.
|
|
||||||
Pages return only node IDs, paths and roles; choose a useful page size and follow
|
|
||||||
`coverage.next_offset` when another page matters:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"source": {
|
|
||||||
"kind": "retained-evidence",
|
|
||||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
|
||||||
},
|
|
||||||
"view": { "kind": "page", "collection": "modules", "offset": 0, "limit": 16 }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Select the relevant page entry by its exact `node_id` and recorded role:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"source": {
|
|
||||||
"kind": "retained-evidence",
|
|
||||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
|
||||||
},
|
|
||||||
"view": {
|
|
||||||
"kind": "item",
|
|
||||||
"collection": "modules",
|
|
||||||
"selector": { "node_id": "jag_node_ACTUAL_NODE_ID" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Module pages include asset, bundled and source module roles, so one path can
|
|
||||||
identify several entries. Use `selector.path` only when uniqueness is established;
|
|
||||||
the returned node ID avoids needing a complete inventory to prove uniqueness. On ambiguity,
|
|
||||||
select the matching node ID from the page or the error's candidates. Follow
|
|
||||||
`next_offset` when another page matters. An item exposes that module node's
|
|
||||||
recorded observations, identity, hashes, exports and source ranges; it does not
|
|
||||||
include every child syntax observation, related edge or original source file.
|
|
||||||
An absent item fact is not evidence of absence from the whole application.
|
|
||||||
|
|
||||||
## Trace only the remaining question
|
|
||||||
|
|
||||||
Use `trace_application_feature` on the parent application Evidence for one
|
|
||||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
|
||||||
Select `incoming` for relationships leading to the seed, `outgoing` for those
|
|
||||||
leaving it, and `both` when the question needs both. Trace traversal includes
|
|
||||||
containment relationships, so even a narrow seed can expand many nodes; a
|
|
||||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
|
||||||
is preferable when its observations already answer the question. Do not treat
|
|
||||||
all reached nodes as participants in the feature.
|
|
||||||
|
|
||||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
|
||||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
|
||||||
matches exactly.
|
|
||||||
|
|
||||||
## Recover an oversized display
|
|
||||||
|
|
||||||
An oversized response reports `resource_constraint` and
|
An oversized response reports `resource_constraint` and
|
||||||
`details.resource: "transport"`. Reuse
|
`details.resource: "transport"`. Reuse
|
||||||
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
||||||
summary, one module, or a stable page of module identities.
|
summary, one module, or a stable page of module identities. Use
|
||||||
|
`trace_application_feature` once a module, route, or string seed is known.
|
||||||
`export_evidence_bundle` writes the complete canonical session to a
|
`export_evidence_bundle` writes the complete canonical session to a
|
||||||
caller-selected file. These workflows keep the original Evidence and coverage;
|
caller-selected file. These workflows keep the original Evidence and coverage;
|
||||||
a broad follow-up can also exceed framing.
|
a broad follow-up can also exceed framing.
|
||||||
|
|
||||||
A client can truncate a successful result inside its text envelope without a
|
|
||||||
REA `resource_constraint`. Treat a truncation notice or incomplete JSON as a
|
|
||||||
display limitation; do not claim the full record was examined. Reuse the known
|
|
||||||
parent ID for smaller views, or export complete Evidence to a selected file
|
|
||||||
while the session remains open. If the parent ID was lost from the preview,
|
|
||||||
use the prior analysis summary or explicit ledger entries to recover it; do
|
|
||||||
not invent an ID or rerun analysis just to reproduce the same display.
|
|
||||||
|
|
||||||
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
||||||
registrations, utility processes, and native binding requests are static syntax
|
registrations, utility processes, and native binding requests are static syntax
|
||||||
observations. Only a unique exact literal IPC channel match supports an inferred
|
observations. Only a unique exact literal IPC channel match supports an inferred
|
||||||
@@ -108,6 +22,13 @@ pairing. Dynamic or ambiguous channels remain unresolved. A requested `.node`
|
|||||||
member is not a verified native export. Never claim runtime reachability,
|
member is not a verified native export. Never claim runtime reachability,
|
||||||
registration, defaults, or policy enforcement from static analysis.
|
registration, defaults, or policy enforcement from static analysis.
|
||||||
|
|
||||||
|
Use `trace_application_feature` on existing application Evidence for one
|
||||||
|
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||||
|
Choose a direction and reuse the returned application Evidence ID when the
|
||||||
|
connected server advertises retained references, or supply complete Evidence
|
||||||
|
inline. Include
|
||||||
|
Hopper or Ghidra Evidence only when its artifact digest matches exactly.
|
||||||
|
|
||||||
For version comparison, analyze each version once, then call
|
For version comparison, analyze each version once, then call
|
||||||
`compare_application_versions`. Accept only its digest, source-map, structural
|
`compare_application_versions`. Accept only its digest, source-map, structural
|
||||||
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
||||||
|
|||||||
@@ -152,7 +152,7 @@ context. PSP/stack/device state remains unmodeled. See [DOS guide](https://githu
|
|||||||
|
|
||||||
### Function annotations in Ghidra
|
### Function annotations in Ghidra
|
||||||
|
|
||||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations.
|
||||||
|
|
||||||
## Offline PE resource inventory
|
## Offline PE resource inventory
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"_comment": "Test files are intentionally NOT excluded: the gate used to report zero clones because it skipped **/*.test.ts, hiding all test-tree duplication. Current actual is ~2.47% lines across src+ and tests/, down from 3.4% when tests were included. The 2.6% limit leaves modest headroom for ordinary changes; lower it as duplication is removed.",
|
"_comment": "Test files are intentionally NOT excluded: the gate used to report zero clones because it skipped **/*.test.ts, hiding all test-tree duplication. Current actual is ~2.4% lines across src+ and tests/, down from 3.4% when tests were included. Lower this as duplication is removed.",
|
||||||
"threshold": 2.6,
|
"threshold": 2.5,
|
||||||
"reporters": ["consoleFull"],
|
"reporters": ["consoleFull"],
|
||||||
"ignore": [
|
"ignore": [
|
||||||
"node_modules/**",
|
"node_modules/**",
|
||||||
|
|||||||
@@ -120,8 +120,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
);
|
);
|
||||||
private static final String[] CAPABILITIES = {
|
private static final String[] CAPABILITIES = {
|
||||||
"annotate_native_function",
|
"annotate_native_function",
|
||||||
"set_address_name",
|
|
||||||
"set_addresses_names",
|
|
||||||
"inspect_native_load_image",
|
"inspect_native_load_image",
|
||||||
"read_bytes",
|
"read_bytes",
|
||||||
"address_to_file_offset",
|
"address_to_file_offset",
|
||||||
@@ -376,12 +374,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
case "resolve_native_call_targets" -> resolveNativeCallTargets(request.params);
|
case "resolve_native_call_targets" -> resolveNativeCallTargets(request.params);
|
||||||
case "xrefs" -> xrefs(request.params);
|
case "xrefs" -> xrefs(request.params);
|
||||||
case "analyze_function" -> analyzeFunction(request.params);
|
case "analyze_function" -> analyzeFunction(request.params);
|
||||||
case "set_address_name", "set_addresses_names" -> {
|
|
||||||
if (!descriptor.transport.equals("unix-socket"))
|
|
||||||
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
|
||||||
yield request.method.equals("set_address_name")
|
|
||||||
? setAddressName(request.params) : setAddressesNames(request.params);
|
|
||||||
}
|
|
||||||
case "annotate_native_function" -> {
|
case "annotate_native_function" -> {
|
||||||
if (!descriptor.transport.equals("unix-socket"))
|
if (!descriptor.transport.equals("unix-socket"))
|
||||||
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
||||||
@@ -425,8 +417,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
result.addProperty("analysis_timed_out", timedOut);
|
result.addProperty("analysis_timed_out", timedOut);
|
||||||
result.add("capabilities", GSON.toJsonTree(
|
result.add("capabilities", GSON.toJsonTree(
|
||||||
java.util.Arrays.stream(CAPABILITIES)
|
java.util.Arrays.stream(CAPABILITIES)
|
||||||
.filter(value -> !readOnly || !(value.equals("annotate_native_function")
|
.filter(value -> !readOnly || !value.equals("annotate_native_function"))
|
||||||
|| value.equals("set_address_name") || value.equals("set_addresses_names")))
|
|
||||||
.toArray(String[]::new)
|
.toArray(String[]::new)
|
||||||
));
|
));
|
||||||
result.add("target", target);
|
result.add("target", target);
|
||||||
@@ -1036,11 +1027,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
for (String field : List.of("name", "comment", "inline_comment")) {
|
for (String field : List.of("name", "comment", "inline_comment")) {
|
||||||
if (params.has(field)) validateAnnotationText(requireText(params, field), field);
|
if (params.has(field)) validateAnnotationText(requireText(params, field), field);
|
||||||
}
|
}
|
||||||
|
String leafName = params.has("name")
|
||||||
|
? annotationLeafName(function, requireString(params, "name")) : null;
|
||||||
int transaction = currentProgram.startTransaction("REA function annotations");
|
int transaction = currentProgram.startTransaction("REA function annotations");
|
||||||
boolean commit = false;
|
boolean commit = false;
|
||||||
try {
|
try {
|
||||||
String leafName = params.has("name")
|
|
||||||
? symbolLeafName(function.getSymbol(), requireString(params, "name")) : null;
|
|
||||||
// Validate edits through Ghidra's own setters, including name rules.
|
// Validate edits through Ghidra's own setters, including name rules.
|
||||||
// A later invalid name must roll back earlier comment writes.
|
// A later invalid name must roll back earlier comment writes.
|
||||||
for (String field : List.of("comment", "inline_comment")) {
|
for (String field : List.of("comment", "inline_comment")) {
|
||||||
@@ -1089,18 +1080,18 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private String symbolLeafName(Symbol symbol, String requested) throws ghidra.util.exception.InvalidInputException {
|
private String annotationLeafName(Function function, String requested) {
|
||||||
// Fully qualified readback must be usable as an idempotent rename input.
|
// Fully qualified readback must be usable as an idempotent rename input.
|
||||||
if (requested.equals(symbol.getName(true))) return symbol.getName();
|
if (requested.equals(procedureName(function))) return function.getName();
|
||||||
String namespace = symbol.getParentNamespace().isGlobal()
|
String namespace = function.getParentNamespace().isGlobal()
|
||||||
? "" : symbol.getParentNamespace().getName(true);
|
? "" : function.getParentNamespace().getName(true);
|
||||||
String prefix = namespace + "::";
|
String prefix = namespace + "::";
|
||||||
if (!namespace.isEmpty() && requested.startsWith(prefix)) {
|
if (!namespace.isEmpty() && requested.startsWith(prefix)) {
|
||||||
String leaf = requested.substring(prefix.length());
|
String leaf = requested.substring(prefix.length());
|
||||||
if (!leaf.isEmpty()) return leaf;
|
if (!leaf.isEmpty()) return leaf;
|
||||||
throw new ghidra.util.exception.InvalidInputException(
|
throw new RequestFailure("invalid_function_name",
|
||||||
"Symbol name has an empty leaf within namespace " + namespace + " at " +
|
"Function name has an empty leaf within namespace " + namespace + " at " +
|
||||||
canonicalAddress(symbol.getAddress()) + ": " + requested);
|
canonicalAddress(function.getEntryPoint()) + ": " + requested);
|
||||||
}
|
}
|
||||||
return requested;
|
return requested;
|
||||||
}
|
}
|
||||||
@@ -1425,68 +1416,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||||||
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
return symbol == null ? JsonNull.INSTANCE : GSON.toJsonTree(symbol.getName(true));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Analyst naming: a function entry renames the function, other addresses get a primary label.
|
|
||||||
private boolean applyAddressName(Address address, String name) throws Exception {
|
|
||||||
if (!currentProgram.getMemory().contains(address))
|
|
||||||
throw new RequestFailure("invalid_request", "Address is not mapped: " + canonicalAddress(address));
|
|
||||||
validateAnnotationText(name, "name");
|
|
||||||
int transaction = currentProgram.startTransaction("REA address name");
|
|
||||||
boolean commit = false;
|
|
||||||
try {
|
|
||||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
|
||||||
String expected = name;
|
|
||||||
if (function != null) {
|
|
||||||
expected = symbolLeafName(function.getSymbol(), name);
|
|
||||||
function.setName(expected, SourceType.USER_DEFINED);
|
|
||||||
} else {
|
|
||||||
var table = currentProgram.getSymbolTable();
|
|
||||||
Symbol primary = table.getPrimarySymbol(address);
|
|
||||||
if (primary != null && !primary.isDynamic()) {
|
|
||||||
expected = symbolLeafName(primary, name);
|
|
||||||
primary.setName(expected, SourceType.USER_DEFINED);
|
|
||||||
} else {
|
|
||||||
Symbol created = table.createLabel(address, name, SourceType.USER_DEFINED);
|
|
||||||
created.setPrimary();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
invalidateAnalysisCaches();
|
|
||||||
Symbol readback = currentProgram.getSymbolTable().getPrimarySymbol(address);
|
|
||||||
commit = readback != null && expected.equals(readback.getName());
|
|
||||||
return commit;
|
|
||||||
}
|
|
||||||
catch (ghidra.util.exception.InvalidInputException | ghidra.util.exception.DuplicateNameException exception) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
currentProgram.endTransaction(transaction, commit);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private JsonElement setAddressName(JsonObject params) throws Exception {
|
|
||||||
requireKeys(params, Set.of("document", "address", "name"));
|
|
||||||
requireDocument(params);
|
|
||||||
return GSON.toJsonTree(applyAddressName(requireAddress(params, "address"), requireString(params, "name")));
|
|
||||||
}
|
|
||||||
|
|
||||||
private JsonElement setAddressesNames(JsonObject params) throws Exception {
|
|
||||||
requireKeys(params, Set.of("document", "names"));
|
|
||||||
requireDocument(params);
|
|
||||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
|
||||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
|
||||||
JsonObject result = new JsonObject();
|
|
||||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
|
||||||
boolean ok;
|
|
||||||
try {
|
|
||||||
ok = applyAddressName(parseReaAddress(entry.getKey()), entry.getValue().getAsString());
|
|
||||||
}
|
|
||||||
catch (RequestFailure failure) {
|
|
||||||
ok = false;
|
|
||||||
}
|
|
||||||
result.addProperty(entry.getKey(), ok);
|
|
||||||
}
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
private JsonArray listDocuments(JsonObject params) {
|
private JsonArray listDocuments(JsonObject params) {
|
||||||
requireKeys(params, Set.of());
|
requireKeys(params, Set.of());
|
||||||
JsonArray result = new JsonArray();
|
JsonArray result = new JsonArray();
|
||||||
|
|||||||
@@ -136,12 +136,6 @@ existing analysis families directly, for example
|
|||||||
- Pull and capture outputs are additive: the destination must not already
|
- Pull and capture outputs are additive: the destination must not already
|
||||||
exist. Re-running a completed pull refuses the existing directory instead
|
exist. Re-running a completed pull refuses the existing directory instead
|
||||||
of overwriting evidence.
|
of overwriting evidence.
|
||||||
- Cancellation during a package pull preserves caller-selected output. The
|
|
||||||
error includes partial observations identifying completed artifacts and the
|
|
||||||
interrupted local path; remaining APKs were not attempted.
|
|
||||||
- Failed package acquisition rollback retains its directory identity and retries
|
|
||||||
cleanup before later work and during shutdown. Cleanup refuses a replacement
|
|
||||||
directory. Terminal shutdown stops admission and drains active operations.
|
|
||||||
- The provider holds no long-lived processes; every invocation is one
|
- The provider holds no long-lived processes; every invocation is one
|
||||||
bounded adb run. The adb server belongs to the caller.
|
bounded adb run. The adb server belongs to the caller.
|
||||||
|
|
||||||
|
|||||||
+3
-10
@@ -166,16 +166,9 @@ rea import-reference-source /absolute/path/to/source
|
|||||||
```
|
```
|
||||||
|
|
||||||
The import records hashes and metadata for the supplied files, separately
|
The import records hashes and metadata for the supplied files, separately
|
||||||
from observations of the current app. It honors the source tree's `.gitignore`
|
from observations of the current app. File names do not automatically exclude
|
||||||
and applies default exclusions for common generated and dependency paths such
|
files. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
patterns when you want to exclude selected paths.
|
||||||
matched pattern and whether it came from project, default, caller, or sensitive
|
|
||||||
path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
|
||||||
patterns when you want to mark selected paths as sensitive; those patterns are
|
|
||||||
reported separately as configured-secret exclusions.
|
|
||||||
|
|
||||||
Re-import saved graphs whose pattern-based exclusions lack `pattern` before
|
|
||||||
using them in source-to-bundle comparisons.
|
|
||||||
|
|
||||||
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
||||||
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
||||||
|
|||||||
@@ -29,13 +29,6 @@ section names or the record being inside the initial 4 KiB header probe.
|
|||||||
Program and section declarations must agree; duplicate, conflicting, truncated
|
Program and section declarations must agree; duplicate, conflicting, truncated
|
||||||
or out-of-file records are rejected. Stripped section tables and section-only
|
or out-of-file records are rejected. Stripped section tables and section-only
|
||||||
records are supported. Extended table numbering is explicitly unsupported.
|
records are supported. Extended table numbering is explicitly unsupported.
|
||||||
For carved ELFs whose section table extends beyond the captured file, REA
|
|
||||||
examines every complete captured section header and records the unavailable
|
|
||||||
headers as a limitation. A valid program-header ABI record can still establish
|
|
||||||
the supported declaration. Available section declarations must agree with it;
|
|
||||||
their duplicate, conflicting or unreadable records remain errors. A missing
|
|
||||||
program-header record does not make ABI declarations in uncaptured sections
|
|
||||||
absent or supply a default ABI.
|
|
||||||
The field layout follows
|
The field layout follows
|
||||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||||
|
|
||||||
@@ -99,11 +92,6 @@ observations and does not relabel the upstream result as verified.
|
|||||||
|
|
||||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||||
identity, snapshot round-trip and final process cleanup remain covered.
|
identity, snapshot round-trip and final process cleanup remain covered.
|
||||||
For both byte orders, the lane also carves away the section table while retaining
|
|
||||||
all program-header payloads. It checks the actual Ghidra language, entry-address
|
|
||||||
decompilation, distinct artifact digest and missing-section limitations through
|
|
||||||
CLI Evidence and an MCP-produced snapshot. Caller-selected Ghidra heap limits
|
|
||||||
are passed to these child workflows.
|
|
||||||
Pseudocode presence is a liveness check, not proof of semantic equivalence.
|
Pseudocode presence is a liveness check, not proof of semantic equivalence.
|
||||||
Instruction decoding and reader agreement likewise do not establish runtime,
|
Instruction decoding and reader agreement likewise do not establish runtime,
|
||||||
delay-slot execution or complete floating-point semantics. Assertion regressions
|
delay-slot execution or complete floating-point semantics. Assertion regressions
|
||||||
|
|||||||
+4
-45
@@ -68,15 +68,7 @@ selects the npm release, and persistent MCP registrations are pinned to the
|
|||||||
version that performed setup. Installing newer instructions does not update a
|
version that performed setup. Installing newer instructions does not update a
|
||||||
running server or its registration.
|
running server or its registration.
|
||||||
|
|
||||||
The npm release checked on **2026-10-10** was **6.3.0**, published from
|
The release checked on **2026-10-07** was **5.0.0** (133 MCP tools), published
|
||||||
[`74cf9e14`](https://github.com/morluto/rea/commit/74cf9e1401fcd89482fe39c27cc898f5858e9b72),
|
|
||||||
with bundled skill version 34. It does not include automatic Pi or Hermes
|
|
||||||
registration: `setup --client pi` and `setup --client hermes` reject those IDs
|
|
||||||
before writing configuration. Their setup workflows below require a source build
|
|
||||||
or a later release containing those integrations; `@latest` alone cannot select
|
|
||||||
unpublished changes.
|
|
||||||
|
|
||||||
The earlier release checked on **2026-10-07** was **5.0.0** (133 MCP tools), published
|
|
||||||
from the fixed checkpoint
|
from the fixed checkpoint
|
||||||
[`b33236ec`](https://github.com/morluto/rea/releases/tag/rea-agents-5.0.0).
|
[`b33236ec`](https://github.com/morluto/rea/releases/tag/rea-agents-5.0.0).
|
||||||
The public CLI, MCP catalog and target-free session, and isolated update from
|
The public CLI, MCP catalog and target-free session, and isolated update from
|
||||||
@@ -150,29 +142,7 @@ listed after the table because its connector is not one of these files:
|
|||||||
| Pi | `pi` |
|
| Pi | `pi` |
|
||||||
| Hermes | `hermes` |
|
| Hermes | `hermes` |
|
||||||
|
|
||||||
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
|
|
||||||
chat with REA's full input-schema profile before sending a model request
|
|
||||||
(`compaction_static_context_blocked`; [#1554](https://github.com/morluto/rea/issues/1554)).
|
|
||||||
For the native offline OpenAI-compatible BYOK adapter, full-profile chat and
|
|
||||||
analysis also passed with a distinct `COPILOT_PROVIDER_MODEL_ID` such as
|
|
||||||
`rea-gpt-4.1`, `COPILOT_PROVIDER_WIRE_MODEL=gpt-4.1`, and that distinct ID selected
|
|
||||||
through `copilot --model rea-gpt-4.1`. The local fixture requested
|
|
||||||
`COPILOT_PROVIDER_MAX_PROMPT_TOKENS=1000000`; configure the limit supported by
|
|
||||||
your backend. This keeps every full schema and the actual wire model intact.
|
|
||||||
It does not establish the effective client budget or fix the built-in-ID path.
|
|
||||||
The existing compact profile passed native chat and analysis with the complete
|
|
||||||
tool inventory on Linux using a loopback model fixture. After setup, add
|
|
||||||
`"REA_MCP_INPUT_SCHEMA_PROFILE": "compact"` to the `env` object of the
|
|
||||||
`mcpServers.rea` entry in Copilot's `mcp-config.json`, preserving its other
|
|
||||||
settings, then restart the client. Compact advertisements can reduce nested
|
|
||||||
schema constraints; REA still validates complete canonical inputs. Full-profile
|
|
||||||
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
|
|
||||||
Windows remain unverified; see [native client verification](testing.md).
|
|
||||||
|
|
||||||
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
|
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
|
||||||
automatic setup requires a build containing the integration; npm 6.3.0 does not
|
|
||||||
include it. See [released package and main](#released-package-and-main).
|
|
||||||
In a supported build,
|
|
||||||
setup writes a `type: "stdio"` entry under `mcpServers.rea` in the user-level
|
setup writes a `type: "stdio"` entry under `mcpServers.rea` in the user-level
|
||||||
`~/.pi/agent/mcp.json` and installs the bundled skill under `~/.agents/skills`.
|
`~/.pi/agent/mcp.json` and installs the bundled skill under `~/.agents/skills`.
|
||||||
Use `rea setup --client pi --dry-run --json` to inspect the plan, then
|
Use `rea setup --client pi --dry-run --json` to inspect the plan, then
|
||||||
@@ -231,10 +201,7 @@ registration. Doctor treats an `enabled: false` entry as active when
|
|||||||
`enabledServers` lists `rea`, as OMP does, unless `disabledServers` also lists
|
`enabledServers` lists `rea`, as OMP does, unless `disabledServers` also lists
|
||||||
it. Run setup under each profile that should load REA.
|
it. Run setup under each profile that should load REA.
|
||||||
|
|
||||||
For Hermes, automatic setup also requires a build containing the integration;
|
For Hermes, setup writes a `mcp_servers.rea` entry to `config.yaml`, preserving
|
||||||
npm 6.3.0 does not include it. See
|
|
||||||
[released package and main](#released-package-and-main).
|
|
||||||
In a supported build, setup writes a `mcp_servers.rea` entry to `config.yaml`, preserving
|
|
||||||
comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
|
||||||
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
|
||||||
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
|
||||||
@@ -367,12 +334,6 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no
|
|||||||
changes. Agent configuration writes preserve unrelated entries and comments,
|
changes. Agent configuration writes preserve unrelated entries and comments,
|
||||||
create backups, use atomic replacement, and verify their result. Setup and
|
create backups, use atomic replacement, and verify their result. Setup and
|
||||||
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
|
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
|
||||||
Setup also retains valid environment overrides in REA's existing server entry,
|
|
||||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
|
|
||||||
Detected provider settings replace only matching environment keys.
|
|
||||||
Setup and doctor report a malformed REA environment map or invalid runtime
|
|
||||||
setting before setup writes files. Repair the named setting in the client
|
|
||||||
configuration; setup does not silently discard the rest of the map.
|
|
||||||
|
|
||||||
After setup, REA reports which agents, analysis tools, and workflow files passed
|
After setup, REA reports which agents, analysis tools, and workflow files passed
|
||||||
its final checks. Restart any agent named in the completion message, then begin
|
its final checks. Restart any agent named in the completion message, then begin
|
||||||
@@ -637,10 +598,8 @@ dossiers, instructions, recovered data types, measured load mappings, loaded
|
|||||||
memory bytes, and observed file offsets. Independent load-image attestation
|
memory bytes, and observed file offsets. Independent load-image attestation
|
||||||
supports DOS MZ and explicitly selected COM; PE returns its measurements with that limitation.
|
supports DOS MZ and explicitly selected COM; PE returns its measurements with that limitation.
|
||||||
On Linux and macOS, `annotate_native_function` also edits a function name and/or
|
On Linux and macOS, `annotate_native_function` also edits a function name and/or
|
||||||
entry comments atomically and returns refreshed analysis, and `set_address_name`
|
entry comments atomically and returns refreshed analysis. These session metadata
|
||||||
and `set_addresses_names` name a function entry or any other mapped address
|
edits leave executable bytes unchanged and are discarded on close. GUI controls
|
||||||
(creating or renaming its primary label) with per-address success results. These
|
|
||||||
session metadata edits leave executable bytes unchanged and are discarded on close. GUI controls
|
|
||||||
require Hopper; Windows P0 remains read-only.
|
require Hopper; Windows P0 remains read-only.
|
||||||
|
|
||||||
Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
|
Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
|
||||||
|
|||||||
@@ -89,9 +89,9 @@ queries use per-file range indexes, while call sites use exact-range lookups.
|
|||||||
Existing node budgets and coverage reporting remain in effect.
|
Existing node budgets and coverage reporting remain in effect.
|
||||||
|
|
||||||
Progress identifies the source file being parsed and projected. Cancellation is
|
Progress identifies the source file being parsed and projected. Cancellation is
|
||||||
checked between files and throughout result transfer. The owned analyzer can
|
checked between files, with event-loop yields that allow notifications and
|
||||||
be stopped while its synchronous parser or semantic extractor is active;
|
completed file-local allocations to be released. A single file's synchronous
|
||||||
the CLI/MCP process remains available for control requests.
|
parser and semantic extraction still need to finish before that boundary.
|
||||||
|
|
||||||
Graph and Evidence identifiers hash canonical JSON incrementally, without
|
Graph and Evidence identifiers hash canonical JSON incrementally, without
|
||||||
assembling a single string for the whole graph. The canonical bytes and existing
|
assembling a single string for the whole graph. The canonical bytes and existing
|
||||||
@@ -113,46 +113,6 @@ smaller view, for example `--format json --filter-output
|
|||||||
evidence_id,normalized_result.statistics`. Streaming output does not bound the
|
evidence_id,normalized_result.statistics`. Streaming output does not bound the
|
||||||
memory needed to construct the analysis graph itself.
|
memory needed to construct the analysis graph itself.
|
||||||
|
|
||||||
JavaScript source parsing, static analysis and semantic projection run in one
|
|
||||||
owned analysis process, reused sequentially across files. Target code is never
|
|
||||||
executed. The analyzer defaults to 1024 MiB of V8 old-space and a 300000 ms
|
|
||||||
deadline per source, including startup and fact transfer. Select
|
|
||||||
`max_heap_mb` and `analysis_timeout_ms` through MCP, or `--max-heap-mb` and
|
|
||||||
`--analysis-timeout-ms` on the dedicated CLI command. These choices control the
|
|
||||||
analyzer; the caller's `NODE_OPTIONS` still controls the CLI/MCP process heap.
|
|
||||||
V8 old-space limits do not impose a process RSS limit.
|
|
||||||
|
|
||||||
A source that exhausts the analyzer heap does not terminate the CLI/MCP
|
|
||||||
process. Completed static facts are retained, other files can still be analyzed,
|
|
||||||
and unexamined semantic relationships remain explicit unknowns. The parent
|
|
||||||
also checks transfer expansion and accumulated results against its remaining
|
|
||||||
heap before accepting semantic facts. A resource-limited result has partial
|
|
||||||
coverage, the observed limits and an `analyze-javascript-source` unknown scope
|
|
||||||
with the specific failure, including captured diagnostics when available.
|
|
||||||
Resource failures do not increment syntax parse failures. There is no fixed
|
|
||||||
source-length cutoff.
|
|
||||||
|
|
||||||
Static observations also need memory when expanded into nodes, relationships
|
|
||||||
and Evidence. If that expansion does not fit, the file node retains
|
|
||||||
`static_analysis` and any admitted `semantic_module_analysis` observations,
|
|
||||||
including their exact values and source locations. Use the public modules item
|
|
||||||
view to read them. The graph marks those unexpanded relationships as unknown.
|
|
||||||
If cumulative result capacity is exhausted, analysis stops; inventoried but
|
|
||||||
unexamined files remain present with explicit unknown scopes.
|
|
||||||
|
|
||||||
A timeout stops the remaining analysis and returns typed failure details with
|
|
||||||
completed partial Evidence. MCP failure details return
|
|
||||||
`partial_observation: { kind: "retained-evidence", evidence_id: "..." }`
|
|
||||||
after recording succeeds, so `inspect_analysis_view` can open that analysis
|
|
||||||
directly without serializing the entire partial graph into the error response.
|
|
||||||
CLI failure output retains the complete partial Evidence.
|
|
||||||
Cancellation after source analysis begins likewise
|
|
||||||
retains completed facts; an SDK client may reject its cancelled request before
|
|
||||||
receiving that reply. On the same MCP connection, inspect the Evidence bundle
|
|
||||||
and use the retained analysis ID with `inspect_analysis_view`. Owned analyzer
|
|
||||||
cleanup is verified before its private files are released. Cleanup uncertainty
|
|
||||||
is reported separately and retains ownership for retry.
|
|
||||||
|
|
||||||
CLI workflows parse larger JSON input files incrementally from one verified
|
CLI workflows parse larger JSON input files incrementally from one verified
|
||||||
regular-file handle, without constructing a document-sized string. Smaller
|
regular-file handle, without constructing a document-sized string. Smaller
|
||||||
files retain native JSON parsing. Both paths require strict JSON and valid
|
files retain native JSON parsing. Both paths require strict JSON and valid
|
||||||
|
|||||||
@@ -397,12 +397,6 @@ unpaired Unicode surrogate; a rejection identifies the field and UTF-16 index
|
|||||||
and leaves every annotation unchanged. CRLF, supplementary Unicode characters,
|
and leaves every annotation unchanged. CRLF, supplementary Unicode characters,
|
||||||
and combining characters are preserved. Windows P0 remains read-only. Ghidra has no
|
and combining characters are preserved. Windows P0 remains read-only. Ghidra has no
|
||||||
GUI authority, and REA never falls back automatically to Hopper.
|
GUI authority, and REA never falls back automatically to Hopper.
|
||||||
`set_address_name` and `set_addresses_names` give Ghidra sessions the same
|
|
||||||
compact naming results as Hopper: a local function entry renames that function,
|
|
||||||
and any other mapped address, such as a global, receives a new or renamed primary
|
|
||||||
label. A batch applies each address separately and reports one success boolean
|
|
||||||
per address. Use them to name many globals and helpers cheaply; use
|
|
||||||
`annotate_native_function` when you also want comments and a refreshed dossier.
|
|
||||||
Ghidra name edits preserve the existing namespace. Supply either a leaf name
|
Ghidra name edits preserve the existing namespace. Supply either a leaf name
|
||||||
such as `renamed` or a fully qualified name in that namespace, such as
|
such as `renamed` or a fully qualified name in that namespace, such as
|
||||||
`alpha::renamed`. The returned qualified name can be reused as an idempotent
|
`alpha::renamed`. The returned qualified name can be reused as an idempotent
|
||||||
|
|||||||
@@ -114,12 +114,6 @@ from launch, must be ordered, and must fall within `timeout_ms`. Inputs marked
|
|||||||
Environment overrides are recorded in the scenario commitment; inherited
|
Environment overrides are recorded in the scenario commitment; inherited
|
||||||
values are not copied into Evidence.
|
values are not copied into Evidence.
|
||||||
|
|
||||||
Terminal dimensions must satisfy `columns × (rows + scrollback) ≤ 1,000,000`.
|
|
||||||
Every scheduled resize uses the same selected scrollback and must fit this
|
|
||||||
working-buffer budget. REA rejects oversized scenarios before launching the
|
|
||||||
command; it never clamps the requested dimensions. This limit is independent
|
|
||||||
of `limits.output_bytes`, which bounds retained terminal observations.
|
|
||||||
|
|
||||||
The capture contains raw PTY output chunks and rendered terminal states,
|
The capture contains raw PTY output chunks and rendered terminal states,
|
||||||
interaction dispatch outcomes, exit reason, sampled process-tree observations,
|
interaction dispatch outcomes, exit reason, sampled process-tree observations,
|
||||||
and settlement status. When filesystem paths are selected, REA records
|
and settlement status. When filesystem paths are selected, REA records
|
||||||
|
|||||||
+10
-89
@@ -99,10 +99,9 @@ result that its client offloads, and Pi exercises default codemode execution.
|
|||||||
Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package.
|
Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package.
|
||||||
Receipts retain client versions, result digests, host coverage and owned-process
|
Receipts retain client versions, result digests, host coverage and owned-process
|
||||||
cleanup. These POSIX lanes use a deterministic loopback model, so they do not
|
cleanup. These POSIX lanes use a deterministic loopback model, so they do not
|
||||||
prove live model-provider or native Windows compatibility. Qwen, Pi and Hermes
|
prove live model-provider or native Windows compatibility. Qwen and Pi use
|
||||||
bind both `HOME` and `USERPROFILE` to the disposable account. Qwen and Pi discover
|
caller-configured skill directories to isolate the fixture account from the OS
|
||||||
the shared personal skill through their default search paths without an explicit
|
home; this does not prove their default home-directory discovery.
|
||||||
skill setting; Hermes discovers it through the selected `HERMES_HOME`.
|
|
||||||
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
||||||
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
||||||
|
|
||||||
@@ -160,39 +159,11 @@ Use `-- chat` for ordinary
|
|||||||
chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
|
chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
|
||||||
The native offline BYOK adapter uses a loopback OpenAI completions/SSE fixture
|
The native offline BYOK adapter uses a loopback OpenAI completions/SSE fixture
|
||||||
and `gpt-5.4` model metadata; model inference and token usage are synthetic.
|
and `gpt-5.4` model metadata; model inference and token usage are synthetic.
|
||||||
Set `REA_VERIFY_COPILOT_MODEL` to examine another client model ID and
|
Set `REA_VERIFY_COPILOT_MODEL` to examine another model configuration. In the
|
||||||
`REA_VERIFY_COPILOT_WIRE_MODEL` for its backend model name (defaults to the client
|
verified client, `gpt-4.1` blocks the complete catalog before HTTP with
|
||||||
ID). Every actual HTTP request's model is checked and recorded separately.
|
|
||||||
Both `HOME` and `USERPROFILE` point to the disposable account; native `skill add`
|
|
||||||
still selects the installed skill directory explicitly.
|
|
||||||
|
|
||||||
The same native client passed full-profile chat and analysis with the actual
|
|
||||||
wire model `gpt-4.1` under a distinct BYOK model ID:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- chat
|
|
||||||
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- call
|
|
||||||
```
|
|
||||||
|
|
||||||
These runs retain full canonical input/output schemas, all tools, installed-skill
|
|
||||||
loading and complete Evidence readback. They use the fixture's requested
|
|
||||||
one-million-token BYOK prompt setting; effective capacity and live API acceptance
|
|
||||||
remain unknown. With the built-in `gpt-4.1` model ID, the verified client still
|
|
||||||
blocks the full input-schema profile before HTTP with
|
|
||||||
`compaction_static_context_blocked`, even when requesting a larger BYOK prompt
|
`compaction_static_context_blocked`, even when requesting a larger BYOK prompt
|
||||||
capacity. The effective capacity is unknown. The default verifier preserves
|
capacity. The effective capacity is unknown; this lane does not establish a
|
||||||
the complete catalog and full schemas. To verify the existing compact profile:
|
fix for that client/model limit. It preserves the complete catalog and schemas.
|
||||||
|
|
||||||
```bash
|
|
||||||
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- chat
|
|
||||||
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- call
|
|
||||||
```
|
|
||||||
|
|
||||||
Both native workflows passed with the complete tool inventory. The verifier selects the
|
|
||||||
profile only in its disposable Copilot registration and records it in the
|
|
||||||
receipt. Compact advertisements can reduce nested validation structure; the
|
|
||||||
server still validates complete canonical inputs. This is a tested alternate
|
|
||||||
workflow, not a fix for the full-profile client/model limit tracked in #1554.
|
|
||||||
Live model APIs, native Windows and default OS-home skill discovery remain
|
Live model APIs, native Windows and default OS-home skill discovery remain
|
||||||
unverified.
|
unverified.
|
||||||
|
|
||||||
@@ -261,9 +232,8 @@ unverified; skill discovery uses an explicit isolated custom directory.
|
|||||||
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
|
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
|
||||||
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
|
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
|
||||||
installs the skill through `rea setup --skill`, writes the native Cordis MCP
|
installs the skill through `rea setup --skill`, writes the native Cordis MCP
|
||||||
patch into an isolated `DSH_HOME`, and binds `HOME` and `USERPROFILE` to the
|
patch into an isolated `DSH_HOME`, and uses `DSH_AGENTS_HOME` for native personal
|
||||||
disposable account. The native default `~/.agents/skills` search path discovers
|
skill discovery. A separate Git root prevents inherited project skills from
|
||||||
the installed skill without a `DSH_AGENTS_HOME` override. A separate Git root prevents inherited project skills from
|
|
||||||
masking a missing installation. The default `call` mode checks skill loading,
|
masking a missing installation. The default `call` mode checks skill loading,
|
||||||
complete catalog discovery, the actual forwarded input schemas and regexes,
|
complete catalog discovery, the actual forwarded input schemas and regexes,
|
||||||
and full JavaScript Evidence for a Unicode path. Use `-- chat` for ordinary
|
and full JavaScript Evidence for a Unicode path. Use `-- chat` for ordinary
|
||||||
@@ -1083,39 +1053,6 @@ real local Codex CLI with:
|
|||||||
npm run verify:agent
|
npm run verify:agent
|
||||||
```
|
```
|
||||||
|
|
||||||
The lane runs scenarios sequentially in standalone Codex (`--no-daemon`) with a
|
|
||||||
disposable HOME and CODEX_HOME under `~/.cache/rea-agent-evaluations/`. Set
|
|
||||||
`REA_AGENT_EVAL_ROOT` to choose another parent directory outside the OS temporary
|
|
||||||
directory: current Codex refuses helper aliases under `/tmp`, preventing shell
|
|
||||||
commands and skill reads even if MCP works. Each owned run directory is removed
|
|
||||||
after verification unless fixture retention is requested. The lane
|
|
||||||
installs the packaged skill and MCP registration through public `rea setup`,
|
|
||||||
rather than copying a skill into a project that also inherits the caller's
|
|
||||||
skills and configuration. Only authentication is copied, when present, and it
|
|
||||||
is removed even with `REA_AGENT_EVAL_KEEP_FIXTURES=true`. No user configuration,
|
|
||||||
rules, plugins or history is copied; shell snapshots and additional agents are
|
|
||||||
disabled. Existing API-key environment authentication remains available.
|
|
||||||
The native missing-provider scenario preapproves only `open_binary` and
|
|
||||||
`close_binary` in the disposable client so it reaches REA's actual provider
|
|
||||||
boundary instead of stopping at Codex approval. This does not validate a real
|
|
||||||
native engine or authorize arbitrary runtime execution.
|
|
||||||
|
|
||||||
Select a smaller run with `REA_AGENT_EVAL_SCENARIOS`, a comma-separated list of
|
|
||||||
scenario IDs. `asar-zh` exercises the packaged desktop rubric from a Chinese
|
|
||||||
request. `javascript-module-view` exercises summary analysis, module paging and
|
|
||||||
item inspection in a tree with unrelated modules; its routing and text checks
|
|
||||||
remain heuristic, so review the retained transcript for selector choice and
|
|
||||||
coverage claims. Set `REA_CODEX_CLI` to an installed Codex executable and
|
|
||||||
`REA_AGENT_EVAL_MODEL` to an account-supported model. Use an external resource
|
|
||||||
limit for the whole process tree when testing on a constrained host; a Node
|
|
||||||
heap limit alone does not bound client and child-process memory.
|
|
||||||
|
|
||||||
`missing-target` leaves desktop fixtures visible without selecting one in the
|
|
||||||
request. It requires zero REA calls and a structured request for an app name or
|
|
||||||
artifact path. Its `expectedFirstTool` is `null`, and
|
|
||||||
`targetClarificationPassed` grades that limited routing outcome independently
|
|
||||||
of analysis prose heuristics. Analyzing a nearby example cannot pass this case.
|
|
||||||
|
|
||||||
The packaged desktop and parser-comparison scenarios now assess a closed set of
|
The packaged desktop and parser-comparison scenarios now assess a closed set of
|
||||||
known fixture claims. Each final answer must be a strict JSON manifest containing
|
known fixture claims. Each final answer must be a strict JSON manifest containing
|
||||||
exactly the requested claim IDs, values, Evidence IDs, and Evidence authority and
|
exactly the requested claim IDs, values, Evidence IDs, and Evidence authority and
|
||||||
@@ -1140,21 +1077,9 @@ navigation-context, and address-context scenarios have no factual rubric and
|
|||||||
remain `not_assessed`; the managed workflow requires both artifact and member
|
remain `not_assessed`; the managed workflow requires both artifact and member
|
||||||
inspection to answer its type and entry-point question.
|
inspection to answer its type and entry-point question.
|
||||||
|
|
||||||
The closed factual scenarios request complete producer Evidence on the initial
|
|
||||||
analysis so their source selectors can authenticate all configured facts.
|
|
||||||
Summary/page/item workflows are exercised separately by `javascript-module-view`;
|
|
||||||
its heuristic gate does not establish full-producer factual correctness.
|
|
||||||
|
|
||||||
All scenarios retain routing, workflow, validation, repetition, process-exit,
|
All scenarios retain routing, workflow, validation, repetition, process-exit,
|
||||||
and token-use gates. Configured factual scenarios additionally require a factual
|
and token-use gates. Configured factual scenarios additionally require a factual
|
||||||
pass. Their answer-text heuristics are diagnostic and do not affect the gate.
|
pass. Their answer-text heuristics are diagnostic and do not affect the gate.
|
||||||
Validation counts include complete text-only REA `invalid_request` envelopes
|
|
||||||
and failed SDK calls whose arguments violate the named current REA contract.
|
|
||||||
Client approval rejections and provider unavailability remain distinct failures;
|
|
||||||
truncated JSON previews are not parsed into invented error codes.
|
|
||||||
Repeated `binary_session` observations separated by a successful `open_binary`
|
|
||||||
or `close_binary` are treated as lifecycle verification. An unchanged retry,
|
|
||||||
including one after a failed lifecycle call, still counts as repetition.
|
|
||||||
Scenarios without a rubric retain the legacy text-heuristic gate.
|
Scenarios without a rubric retain the legacy text-heuristic gate.
|
||||||
`answerTermCoverageMet` checks case-insensitive substrings,
|
`answerTermCoverageMet` checks case-insensitive substrings,
|
||||||
`epistemicCuePresent` checks keywords, and `finalCitesEvidence` checks only an ID's
|
`epistemicCuePresent` checks keywords, and `finalCitesEvidence` checks only an ID's
|
||||||
@@ -1162,11 +1087,7 @@ presence. These metrics can still accept fabricated prose and must not be read
|
|||||||
as factual assessments. Set `REA_AGENT_EVAL_TRANSCRIPT_DIR` to retain complete
|
as factual assessments. Set `REA_AGENT_EVAL_TRANSCRIPT_DIR` to retain complete
|
||||||
tool results and final answers for review.
|
tool results and final answers for review.
|
||||||
|
|
||||||
Report schema version 4 adds negative routing scenarios with nullable
|
Report schema version 3 changes the top-level `factualCorrectness` from a constant
|
||||||
`expectedFirstTool` and a `targetClarificationPassed` outcome. Consumers must
|
|
||||||
allow an expected absence of REA calls; this does not make an analysis scenario
|
|
||||||
pass without its required tools. Version 3 changed the top-level
|
|
||||||
`factualCorrectness` from a constant
|
|
||||||
string to an assessment summary with status, scope, and assessed/passed/failed/
|
string to an assessment summary with status, scope, and assessed/passed/failed/
|
||||||
not-assessed scenario counts. Scenario records include the factual assessment and
|
not-assessed scenario counts. Scenario records include the factual assessment and
|
||||||
configured claim IDs. The evaluation scope is
|
configured claim IDs. The evaluation scope is
|
||||||
|
|||||||
+4
-7
@@ -85,22 +85,19 @@ if [[ "$version" =~ \+ ]]; then
|
|||||||
fail "npm cannot install an exact version with build metadata. Pass a version without build metadata, then retry."
|
fail "npm cannot install an exact version with build metadata. Pass a version without build metadata, then retry."
|
||||||
fi
|
fi
|
||||||
|
|
||||||
install_args=(install --global)
|
prefix_args=()
|
||||||
if [[ "$platform" == "Linux" ]]; then
|
if [[ "$platform" == "Linux" ]]; then
|
||||||
install_args+=(--prefix "$HOME/.local")
|
prefix_args=(--prefix "$HOME/.local")
|
||||||
install_bin="$HOME/.local/bin/rea"
|
install_bin="$HOME/.local/bin/rea"
|
||||||
else
|
else
|
||||||
npm_prefix="$(npm prefix --global)" || fail "the npm global prefix could not be read. Repair the npm configuration, then retry."
|
npm_prefix="$(npm prefix --global)" || fail "the npm global prefix could not be read. Repair the npm configuration, then retry."
|
||||||
install_bin="$npm_prefix/bin/rea"
|
install_bin="$npm_prefix/bin/rea"
|
||||||
fi
|
fi
|
||||||
install_args+=("$PACKAGE@$version")
|
|
||||||
|
|
||||||
printf 'REA install plan\n'
|
printf 'REA install plan\n'
|
||||||
printf ' Version: %s\n' "$version"
|
printf ' Version: %s\n' "$version"
|
||||||
printf ' Runtime: Node.js %s\n' "$node_version"
|
printf ' Runtime: Node.js %s\n' "$node_version"
|
||||||
printf ' Command:'
|
printf ' Command: npm install --global %s@%s\n' "$PACKAGE" "$version"
|
||||||
printf ' %q' npm "${install_args[@]}"
|
|
||||||
printf '\n'
|
|
||||||
printf ' Binary: %s\n' "$install_bin"
|
printf ' Binary: %s\n' "$install_bin"
|
||||||
printf ' Setup: %s\n' "$([[ "$start_setup" == true ]] && printf 'start when a terminal is available' || printf 'skipped')"
|
printf ' Setup: %s\n' "$([[ "$start_setup" == true ]] && printf 'start when a terminal is available' || printf 'skipped')"
|
||||||
|
|
||||||
@@ -110,7 +107,7 @@ if [[ "$dry_run" == true ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
|
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
|
||||||
npm "${install_args[@]}" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
|
npm install --global ${prefix_args[@]+"${prefix_args[@]}"} "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
|
||||||
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
|
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
|
||||||
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
|
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
|
||||||
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
|
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
|
||||||
|
|||||||
@@ -3,7 +3,6 @@
|
|||||||
"entry": [
|
"entry": [
|
||||||
"src/main.ts",
|
"src/main.ts",
|
||||||
"src/cli.ts",
|
"src/cli.ts",
|
||||||
"src/composition/javascriptAnalysisWorkerProcess.ts",
|
|
||||||
"src/contracts/errorSchemas.ts",
|
"src/contracts/errorSchemas.ts",
|
||||||
"src/evaluation/AgentEvaluationFixtures.ts",
|
"src/evaluation/AgentEvaluationFixtures.ts",
|
||||||
"src/application/RealHopperAssertions.ts",
|
"src/application/RealHopperAssertions.ts",
|
||||||
|
|||||||
@@ -30,9 +30,6 @@
|
|||||||
"access": "public"
|
"access": "public"
|
||||||
},
|
},
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"imports": {
|
|
||||||
"#javascript-analysis-worker": "./dist/composition/javascriptAnalysisWorkerProcess.js"
|
|
||||||
},
|
|
||||||
"bin": {
|
"bin": {
|
||||||
"rea": "scripts/rea.mjs",
|
"rea": "scripts/rea.mjs",
|
||||||
"rea-agents": "scripts/rea.mjs"
|
"rea-agents": "scripts/rea.mjs"
|
||||||
|
|||||||
@@ -1,28 +0,0 @@
|
|||||||
import { analyzeJavaScriptApplication } from "../../dist/application/javascript/JavaScriptApplicationService.js";
|
|
||||||
import { ArtifactResourceScope } from "../../dist/artifacts/ArtifactResourceScope.js";
|
|
||||||
|
|
||||||
/** Analyze one JavaScript application and close its caller-owned resources. */
|
|
||||||
export const analyzeJavaScriptApplicationWithOwnedScope = async (input) => {
|
|
||||||
const scope = new ArtifactResourceScope();
|
|
||||||
let outcome;
|
|
||||||
try {
|
|
||||||
const result = await analyzeJavaScriptApplication(input, scope);
|
|
||||||
outcome = result.ok
|
|
||||||
? { kind: "completed", value: result.value }
|
|
||||||
: { kind: "failed", cause: result.error };
|
|
||||||
} catch (cause) {
|
|
||||||
outcome = { kind: "failed", cause };
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
await scope.close();
|
|
||||||
} catch (cleanupCause) {
|
|
||||||
if (outcome.kind === "failed")
|
|
||||||
throw new AggregateError(
|
|
||||||
[outcome.cause, cleanupCause],
|
|
||||||
"JavaScript analysis and artifact cleanup both failed",
|
|
||||||
);
|
|
||||||
throw cleanupCause;
|
|
||||||
}
|
|
||||||
if (outcome.kind === "failed") throw outcome.cause;
|
|
||||||
return outcome.value;
|
|
||||||
};
|
|
||||||
@@ -21,6 +21,7 @@ export async function verifyGhidraEntryAliases({
|
|||||||
new URL("../../tests/conformance/ghidra/entry-aliases.c", import.meta.url),
|
new URL("../../tests/conformance/ghidra/entry-aliases.c", import.meta.url),
|
||||||
);
|
);
|
||||||
const path = join(workspace, "entry-aliases");
|
const path = join(workspace, "entry-aliases");
|
||||||
|
const prefix = process.platform === "darwin" ? "_" : "";
|
||||||
try {
|
try {
|
||||||
await exec(compiler, ["-O0", "-g", "-fno-inline", source, "-o", path], {
|
await exec(compiler, ["-O0", "-g", "-fno-inline", source, "-o", path], {
|
||||||
env,
|
env,
|
||||||
@@ -28,8 +29,7 @@ export async function verifyGhidraEntryAliases({
|
|||||||
});
|
});
|
||||||
const bytes = await readFile(path);
|
const bytes = await readFile(path);
|
||||||
await call("close_binary");
|
await call("close_binary");
|
||||||
const imported = await call("open_binary", { path, provider_id: "ghidra" });
|
await call("open_binary", { path, provider_id: "ghidra" });
|
||||||
const prefix = imported.format === "mach-o" ? "_" : "";
|
|
||||||
const names = await call("list_names");
|
const names = await call("list_names");
|
||||||
const alias = names.find(
|
const alias = names.find(
|
||||||
({ value }) => value === `${prefix}rea_entry_alias`,
|
({ value }) => value === `${prefix}rea_entry_alias`,
|
||||||
@@ -46,10 +46,8 @@ export async function verifyGhidraEntryAliases({
|
|||||||
);
|
);
|
||||||
assert.ok(canonical, "Fixture alias is not at a function entry");
|
assert.ok(canonical, "Fixture alias is not at a function entry");
|
||||||
const xrefsSelectors = [];
|
const xrefsSelectors = [];
|
||||||
// CLI xrefs keeps bare hexadecimal selectors as addresses. A Mach-O
|
|
||||||
// primary label such as "dead" is tested below as a procedure name instead.
|
|
||||||
for (const value of [
|
for (const value of [
|
||||||
`${prefix}rea_alias_target`,
|
canonical.value,
|
||||||
alias.value,
|
alias.value,
|
||||||
`${prefix}rea_xrefs_data`,
|
`${prefix}rea_xrefs_data`,
|
||||||
`${prefix}rea_xrefs_unreferenced`,
|
`${prefix}rea_xrefs_unreferenced`,
|
||||||
@@ -91,18 +89,6 @@ export async function verifyGhidraEntryAliases({
|
|||||||
);
|
);
|
||||||
assert.ok(interior);
|
assert.ok(interior);
|
||||||
assert.notEqual(interior.address, alias.address);
|
assert.notEqual(interior.address, alias.address);
|
||||||
assert.equal(
|
|
||||||
interior.symbol.type,
|
|
||||||
"label",
|
|
||||||
"Fixture interior control must remain an imported label",
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
(await call("list_procedures")).some(
|
|
||||||
({ address }) => address === interior.address,
|
|
||||||
),
|
|
||||||
false,
|
|
||||||
"Fixture interior control must not be a Ghidra procedure entry",
|
|
||||||
);
|
|
||||||
const failure = await reject("procedure_address", {
|
const failure = await reject("procedure_address", {
|
||||||
procedure: interior.value,
|
procedure: interior.value,
|
||||||
});
|
});
|
||||||
@@ -167,11 +153,7 @@ export async function verifyGhidraEntryAliases({
|
|||||||
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
|
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
|
||||||
);
|
);
|
||||||
const evidence = JSON.parse(stdout);
|
const evidence = JSON.parse(stdout);
|
||||||
assert.deepEqual(
|
assert.deepEqual(evidence.normalized_result, expected);
|
||||||
evidence.normalized_result,
|
|
||||||
expected,
|
|
||||||
`CLI xrefs mismatch for ${JSON.stringify(selector)}`,
|
|
||||||
);
|
|
||||||
assert.equal(evidence.parameters.address, selector.at(-1));
|
assert.equal(evidence.parameters.address, selector.at(-1));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -54,14 +54,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
|||||||
);
|
);
|
||||||
return { namespace, address: procedure.address };
|
return { namespace, address: procedure.address };
|
||||||
});
|
});
|
||||||
const names = await call("list_names");
|
|
||||||
const globals = ["alpha", "outer::inner"].map((namespace) => {
|
|
||||||
const symbol = names.find(
|
|
||||||
(item) => item.value === `${namespace}::value` && item.symbol.primary,
|
|
||||||
);
|
|
||||||
assert.ok(symbol, `Missing demangled fixture global ${namespace}::value`);
|
|
||||||
return { namespace, address: symbol.address };
|
|
||||||
});
|
|
||||||
const ambiguous = await reject("procedure_address", { procedure: "same" });
|
const ambiguous = await reject("procedure_address", { procedure: "same" });
|
||||||
assert.equal(ambiguous.code, "invalid_request");
|
assert.equal(ambiguous.code, "invalid_request");
|
||||||
for (const { address } of entries)
|
for (const { address } of entries)
|
||||||
@@ -133,16 +125,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
|||||||
});
|
});
|
||||||
assert.deepEqual(unchanged.annotations, qualified.annotations);
|
assert.deepEqual(unchanged.annotations, qualified.annotations);
|
||||||
}
|
}
|
||||||
await verifyAddressNames(call, [entries[0], entries[2], ...globals]);
|
|
||||||
for (const { namespace, address } of entries) {
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", {
|
|
||||||
address,
|
|
||||||
name: `${namespace}::qualified`,
|
|
||||||
}),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const { stdout } = await exec(
|
const { stdout } = await exec(
|
||||||
process.execPath,
|
process.execPath,
|
||||||
[
|
[
|
||||||
@@ -177,96 +159,6 @@ export async function verifyGhidraNamespaceAnnotations({
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async function verifyAddressNames(call, symbols) {
|
|
||||||
for (const { namespace, address } of symbols) {
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: "address_renamed" }),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
const qualified = `${namespace}::address_qualified`;
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: qualified }),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
assert.equal(await call("address_name", { address }), qualified);
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: "other::literal" }),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
const literal = await call("address_name", { address });
|
|
||||||
assert.equal(literal, `${namespace}::other::literal`);
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: literal }),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
assert.equal(await call("address_name", { address }), literal);
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: qualified }),
|
|
||||||
true,
|
|
||||||
);
|
|
||||||
for (let repeat = 0; repeat < 2; repeat++) {
|
|
||||||
const readback = await call("address_name", { address });
|
|
||||||
assert.deepEqual(
|
|
||||||
await call("set_addresses_names", { names: { [address]: readback } }),
|
|
||||||
{ [address]: true },
|
|
||||||
);
|
|
||||||
assert.equal(await call("address_name", { address }), qualified);
|
|
||||||
}
|
|
||||||
assert.equal(
|
|
||||||
await call("set_address_name", { address, name: `${namespace}::` }),
|
|
||||||
false,
|
|
||||||
);
|
|
||||||
const session = await call("binary_session");
|
|
||||||
for (const operation of [
|
|
||||||
"set_address_name",
|
|
||||||
"set_addresses_names",
|
|
||||||
"address_name",
|
|
||||||
"list_names",
|
|
||||||
])
|
|
||||||
assert.equal(
|
|
||||||
session.capabilities.find((item) => item.operation === operation)
|
|
||||||
?.available,
|
|
||||||
true,
|
|
||||||
`An invalid name must not mark ${operation} unavailable`,
|
|
||||||
);
|
|
||||||
assert.equal(await call("address_name", { address }), qualified);
|
|
||||||
}
|
|
||||||
const [valid, invalid] = symbols;
|
|
||||||
assert.deepEqual(
|
|
||||||
await call("set_addresses_names", {
|
|
||||||
names: {
|
|
||||||
[valid.address]: `${valid.namespace}::batch_renamed`,
|
|
||||||
[invalid.address]: `${invalid.namespace}::`,
|
|
||||||
},
|
|
||||||
}),
|
|
||||||
{ [valid.address]: true, [invalid.address]: false },
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
await call("address_name", { address: valid.address }),
|
|
||||||
`${valid.namespace}::batch_renamed`,
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
await call("address_name", { address: invalid.address }),
|
|
||||||
`${invalid.namespace}::address_qualified`,
|
|
||||||
);
|
|
||||||
const names = await call("list_names");
|
|
||||||
for (const { namespace, address } of symbols) {
|
|
||||||
const expected =
|
|
||||||
address === valid.address
|
|
||||||
? `${namespace}::batch_renamed`
|
|
||||||
: `${namespace}::address_qualified`;
|
|
||||||
assert.ok(
|
|
||||||
names.some(
|
|
||||||
(item) =>
|
|
||||||
item.address === address &&
|
|
||||||
item.value === expected &&
|
|
||||||
item.symbol.primary,
|
|
||||||
),
|
|
||||||
`Missing refreshed primary symbol ${expected}`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function requireNamespaceDemangler(env) {
|
async function requireNamespaceDemangler(env) {
|
||||||
const inspection = inspectGhidraInstallation({
|
const inspection = inspectGhidraInstallation({
|
||||||
environment: process.env,
|
environment: process.env,
|
||||||
|
|||||||
@@ -1,18 +1,9 @@
|
|||||||
import { spawn } from "node:child_process";
|
import { spawn } from "node:child_process";
|
||||||
import {
|
import { cp, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||||
chmod,
|
import { tmpdir } from "node:os";
|
||||||
copyFile,
|
import { dirname, join, resolve } from "node:path";
|
||||||
mkdir,
|
|
||||||
mkdtemp,
|
|
||||||
readFile,
|
|
||||||
rm,
|
|
||||||
writeFile,
|
|
||||||
} from "node:fs/promises";
|
|
||||||
import { homedir } from "node:os";
|
|
||||||
import { delimiter, dirname, join, resolve } from "node:path";
|
|
||||||
import { createInterface } from "node:readline";
|
import { createInterface } from "node:readline";
|
||||||
import { fileURLToPath } from "node:url";
|
import { fileURLToPath } from "node:url";
|
||||||
import { z } from "zod";
|
|
||||||
|
|
||||||
import { evaluateCodexEvents } from "../dist/evaluation/CodexAgentEval.js";
|
import { evaluateCodexEvents } from "../dist/evaluation/CodexAgentEval.js";
|
||||||
import {
|
import {
|
||||||
@@ -24,95 +15,32 @@ import {
|
|||||||
agentEvaluationPassed,
|
agentEvaluationPassed,
|
||||||
summarizeFactualCorrectness,
|
summarizeFactualCorrectness,
|
||||||
} from "../dist/evaluation/AgentEvaluationReport.js";
|
} from "../dist/evaluation/AgentEvaluationReport.js";
|
||||||
import { PRODUCT_IDENTITY } from "../dist/identity.js";
|
import { MCP_STARTUP_POLICY } from "../dist/mcpStartupPolicy.js";
|
||||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||||
|
|
||||||
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||||
const verifierRun = createVerifierRun();
|
const verifierRun = createVerifierRun();
|
||||||
|
const evaluationRoot = await mkdtemp(join(tmpdir(), "rea-agent-eval-"));
|
||||||
|
const fixtureRoot = join(evaluationRoot, "targets");
|
||||||
|
const skillDestination = join(
|
||||||
|
evaluationRoot,
|
||||||
|
".agents/skills/reverse-engineer-anything",
|
||||||
|
);
|
||||||
const timeoutMs = Number(process.env.REA_AGENT_EVAL_TIMEOUT_MS ?? 480_000);
|
const timeoutMs = Number(process.env.REA_AGENT_EVAL_TIMEOUT_MS ?? 480_000);
|
||||||
const codex = process.env.REA_CODEX_CLI ?? "codex";
|
const codex = process.env.REA_CODEX_CLI ?? "codex";
|
||||||
const optionalModel = process.env.REA_AGENT_EVAL_MODEL;
|
const optionalModel = process.env.REA_AGENT_EVAL_MODEL;
|
||||||
|
|
||||||
if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 10_000)
|
if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 10_000)
|
||||||
throw new Error("REA_AGENT_EVAL_TIMEOUT_MS must be an integer >= 10000");
|
throw new Error("REA_AGENT_EVAL_TIMEOUT_MS must be an integer >= 10000");
|
||||||
// Codex refuses helper aliases when CODEX_HOME is under the OS temp directory.
|
const codexVersion = await readCommandVersion(codex);
|
||||||
// Keep the disposable account in a private cache, then remove the owned run.
|
|
||||||
const evaluationParent = resolve(
|
|
||||||
process.env.REA_AGENT_EVAL_ROOT ??
|
|
||||||
join(homedir(), ".cache", "rea-agent-evaluations"),
|
|
||||||
);
|
|
||||||
await mkdir(evaluationParent, { recursive: true, mode: 0o700 });
|
|
||||||
const evaluationRoot = await mkdtemp(join(evaluationParent, "run-"));
|
|
||||||
const fixtureRoot = join(evaluationRoot, "targets");
|
|
||||||
const account = join(evaluationRoot, "account");
|
|
||||||
const codexHome = join(evaluationRoot, "codex");
|
|
||||||
const authPath = join(codexHome, "auth.json");
|
|
||||||
const skillDestination = join(
|
|
||||||
account,
|
|
||||||
".agents/skills/reverse-engineer-anything",
|
|
||||||
);
|
|
||||||
const evaluationEnvironment = {
|
|
||||||
...Object.fromEntries(
|
|
||||||
Object.entries(process.env).filter(
|
|
||||||
([name]) => !/^(?:REA_|CODEX_|GHIDRA_|HOPPER_|IDA_)/u.test(name),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
HOME: account,
|
|
||||||
USERPROFILE: account,
|
|
||||||
CODEX_HOME: codexHome,
|
|
||||||
XDG_CONFIG_HOME: join(account, ".config"),
|
|
||||||
XDG_DATA_HOME: join(account, ".local/share"),
|
|
||||||
XDG_CACHE_HOME: join(account, ".cache"),
|
|
||||||
REA_PROCESS_RUN_ID: verifierRun.run_id,
|
|
||||||
PATH: `${dirname(process.execPath)}${delimiter}${process.env.PATH ?? ""}`,
|
|
||||||
};
|
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const codexVersion = await readCommandVersion(codex);
|
await mkdir(fixtureRoot, { recursive: true });
|
||||||
await Promise.all([
|
await cp(
|
||||||
mkdir(fixtureRoot, { recursive: true }),
|
join(repositoryRoot, "skills/reverse-engineer-anything"),
|
||||||
mkdir(account),
|
skillDestination,
|
||||||
mkdir(codexHome, { mode: 0o700 }),
|
{ recursive: true },
|
||||||
]);
|
|
||||||
// Reuse authentication only. User config, plugins, rules, history and skills
|
|
||||||
// must not make a fresh installed-skill trial pass accidentally.
|
|
||||||
try {
|
|
||||||
await copyFile(
|
|
||||||
join(process.env.CODEX_HOME ?? join(homedir(), ".codex"), "auth.json"),
|
|
||||||
authPath,
|
|
||||||
);
|
|
||||||
await chmod(authPath, 0o600);
|
|
||||||
} catch (cause) {
|
|
||||||
if (cause?.code !== "ENOENT") throw cause;
|
|
||||||
// Environment/API-key authentication can work without a local auth file.
|
|
||||||
}
|
|
||||||
await writeFile(
|
|
||||||
join(codexHome, "config.toml"),
|
|
||||||
'approval_policy = "never"\nsandbox_mode = "read-only"\nweb_search = "disabled"\n[features]\nmulti_agent = false\nshell_snapshot = false\n[history]\npersistence = "none"\n',
|
|
||||||
{ mode: 0o600 },
|
|
||||||
);
|
);
|
||||||
for (const approval of ["--dry-run", "--yes"]) {
|
|
||||||
await runProcess(
|
|
||||||
process.execPath,
|
|
||||||
[
|
|
||||||
join(repositoryRoot, "scripts/rea.mjs"),
|
|
||||||
"setup",
|
|
||||||
"--client",
|
|
||||||
"codex",
|
|
||||||
approval,
|
|
||||||
"--json",
|
|
||||||
],
|
|
||||||
evaluationRoot,
|
|
||||||
120_000,
|
|
||||||
evaluationEnvironment,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const installedSkill = await readFile(
|
|
||||||
join(skillDestination, "SKILL.md"),
|
|
||||||
"utf8",
|
|
||||||
);
|
|
||||||
if (!installedSkill.includes(` version: "${PRODUCT_IDENTITY.skillVersion}"`))
|
|
||||||
throw new Error("Setup did not install the current packaged REA skill");
|
|
||||||
const requestedScenarioIds = new Set(
|
const requestedScenarioIds = new Set(
|
||||||
(process.env.REA_AGENT_EVAL_SCENARIOS ?? "")
|
(process.env.REA_AGENT_EVAL_SCENARIOS ?? "")
|
||||||
.split(",")
|
.split(",")
|
||||||
@@ -129,11 +57,7 @@ try {
|
|||||||
expectedFirstTool: "open_binary",
|
expectedFirstTool: "open_binary",
|
||||||
requiresEvidence: false,
|
requiresEvidence: false,
|
||||||
requiredAnswerTermGroups: [["unavailable", "could not", "provider"]],
|
requiredAnswerTermGroups: [["unavailable", "could not", "provider"]],
|
||||||
// The verifier authorizes these named session operations, not arbitrary
|
prompt: `Explain what the shipped native program at ${targets.native} does. Base the answer on artifact analysis, and clearly state any facts that remain unavailable.`,
|
||||||
// writes or runtime execution. Missing-engine behavior stays observable.
|
|
||||||
approvedTools: ["open_binary", "close_binary"],
|
|
||||||
requiredToolSubsequence: ["open_binary", "close_binary"],
|
|
||||||
prompt: `Explain what static artifact evidence establishes about the shipped native program at ${targets.native}. Do not execute the target or install tools. Clearly state unavailable provider evidence, and close any analysis session you opened before finishing.`,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "asar",
|
id: "asar",
|
||||||
@@ -146,38 +70,6 @@ try {
|
|||||||
],
|
],
|
||||||
prompt: `Explain how the desktop application at ${targets.javascript} exposes APIs to its renderer. Base the answer on the shipped application artifact, and state what remains unknown.`,
|
prompt: `Explain how the desktop application at ${targets.javascript} exposes APIs to its renderer. Base the answer on the shipped application artifact, and state what remains unknown.`,
|
||||||
},
|
},
|
||||||
{
|
|
||||||
id: "asar-zh",
|
|
||||||
fixtureScenarioId: "asar",
|
|
||||||
expectedFirstTool: "analyze_javascript_application",
|
|
||||||
requiresEvidence: true,
|
|
||||||
requiredAnswerTermGroups: [["profileapi"], ["profile:read"]],
|
|
||||||
prompt: `請靜態分析已交付的 Electron 程式 ${targets.javascript},說明 renderer 如何透過 preload 讀取 profile,以及 main handler 的關係。不要執行目標或安裝分析引擎;引用完整的產出 Evidence,保留 runtime 未知事項。`,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "javascript-module-view",
|
|
||||||
expectedFirstTool: "analyze_javascript_application",
|
|
||||||
requiresEvidence: true,
|
|
||||||
requiredToolSubsequence: [
|
|
||||||
"analyze_javascript_application",
|
|
||||||
"inspect_analysis_view",
|
|
||||||
"inspect_analysis_view",
|
|
||||||
],
|
|
||||||
requiredAnswerTermGroups: [
|
|
||||||
["main.js"],
|
|
||||||
["node", "identity"],
|
|
||||||
["observations", "recorded"],
|
|
||||||
],
|
|
||||||
prompt: `Inspect the recorded module identity, exports and observations for main.js in the shipped JavaScript tree ${targets.largeJavascript}. Use REA without directly reading target files or executing the target. Keep results focused on that module, cite Evidence, and state the limits of what its module view establishes.`,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "missing-target",
|
|
||||||
expectedFirstTool: null,
|
|
||||||
requiresEvidence: false,
|
|
||||||
requiredAnswerTermGroups: [],
|
|
||||||
prompt:
|
|
||||||
'Reverse engineer my desktop app and explain how it saves user settings. Use static analysis; do not install software. End with one JSON object containing status ("needs_target" or "analyzed"), requested_input ("app_name_or_artifact_path" or null), and question (a string or null).',
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
id: "javascript-export-shape",
|
id: "javascript-export-shape",
|
||||||
expectedFirstTool: "analyze_javascript_application",
|
expectedFirstTool: "analyze_javascript_application",
|
||||||
@@ -253,14 +145,12 @@ try {
|
|||||||
const results = [];
|
const results = [];
|
||||||
for (const scenario of scenarios) {
|
for (const scenario of scenarios) {
|
||||||
process.stderr.write(`Running Codex agent evaluation: ${scenario.id}\n`);
|
process.stderr.write(`Running Codex agent evaluation: ${scenario.id}\n`);
|
||||||
const fixtureId = scenario.fixtureScenarioId ?? scenario.id;
|
const fixtureClaims = agentFixtureClaims(scenario.id, targets);
|
||||||
const fixtureClaims = agentFixtureClaims(fixtureId, targets);
|
const answerInstructions = agentFixtureAnswerInstructions(scenario.id);
|
||||||
const answerInstructions = agentFixtureAnswerInstructions(fixtureId);
|
|
||||||
const execution = await runCodex(
|
const execution = await runCodex(
|
||||||
answerInstructions.length === 0
|
answerInstructions.length === 0
|
||||||
? scenario.prompt
|
? scenario.prompt
|
||||||
: `${scenario.prompt}\n\nThis closed factual rubric needs complete producer Evidence in the transcript. Request complete detail on each initial application analysis when supported; summary or view records alone are not accepted by this rubric.\n\n${answerInstructions}`,
|
: `${scenario.prompt}\n\n${answerInstructions}`,
|
||||||
scenario.approvedTools ?? [],
|
|
||||||
);
|
);
|
||||||
const transcriptDirectory = process.env.REA_AGENT_EVAL_TRANSCRIPT_DIR;
|
const transcriptDirectory = process.env.REA_AGENT_EVAL_TRANSCRIPT_DIR;
|
||||||
if (transcriptDirectory !== undefined) {
|
if (transcriptDirectory !== undefined) {
|
||||||
@@ -285,12 +175,6 @@ try {
|
|||||||
...(fixtureClaims === undefined ? {} : { fixtureClaims }),
|
...(fixtureClaims === undefined ? {} : { fixtureClaims }),
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
// Fixtures are deliberately visible beside prior investigation artifacts.
|
|
||||||
// Their existence does not identify the unnamed app in this request.
|
|
||||||
const targetClarificationPassed =
|
|
||||||
scenario.expectedFirstTool === null
|
|
||||||
? validTargetClarification(metrics.finalMessage)
|
|
||||||
: undefined;
|
|
||||||
const result = {
|
const result = {
|
||||||
id: scenario.id,
|
id: scenario.id,
|
||||||
expectedFirstTool: scenario.expectedFirstTool,
|
expectedFirstTool: scenario.expectedFirstTool,
|
||||||
@@ -301,9 +185,6 @@ try {
|
|||||||
exitCode: execution.exitCode,
|
exitCode: execution.exitCode,
|
||||||
stderr: execution.stderr,
|
stderr: execution.stderr,
|
||||||
...metrics,
|
...metrics,
|
||||||
...(targetClarificationPassed === undefined
|
|
||||||
? {}
|
|
||||||
: { targetClarificationPassed }),
|
|
||||||
};
|
};
|
||||||
results.push(result);
|
results.push(result);
|
||||||
process.stderr.write(
|
process.stderr.write(
|
||||||
@@ -313,20 +194,13 @@ try {
|
|||||||
|
|
||||||
const factualCorrectness = summarizeFactualCorrectness(results);
|
const factualCorrectness = summarizeFactualCorrectness(results);
|
||||||
const summary = {
|
const summary = {
|
||||||
schemaVersion: 4,
|
schemaVersion: 3,
|
||||||
evaluationScope: "routing_workflow_and_configured_fixture_claims",
|
evaluationScope: "routing_workflow_and_configured_fixture_claims",
|
||||||
factualCorrectness,
|
factualCorrectness,
|
||||||
verifier_run: await completeVerifierRun(verifierRun),
|
verifier_run: await completeVerifierRun(verifierRun),
|
||||||
codex,
|
codex,
|
||||||
codexVersion,
|
codexVersion,
|
||||||
model: optionalModel ?? null,
|
model: optionalModel ?? null,
|
||||||
installation: {
|
|
||||||
method: "rea_setup_codex",
|
|
||||||
isolatedHome: true,
|
|
||||||
isolatedCodexHome: true,
|
|
||||||
skillVersion: PRODUCT_IDENTITY.skillVersion,
|
|
||||||
authentication: "caller_auth_only_removed_after_run",
|
|
||||||
},
|
|
||||||
scenarios: results,
|
scenarios: results,
|
||||||
totals: {
|
totals: {
|
||||||
scenarios: results.length,
|
scenarios: results.length,
|
||||||
@@ -379,32 +253,12 @@ try {
|
|||||||
`Agent routing/workflow/answer checks failed: ${failed.map(({ id }) => id).join(", ")}`,
|
`Agent routing/workflow/answer checks failed: ${failed.map(({ id }) => id).join(", ")}`,
|
||||||
);
|
);
|
||||||
} finally {
|
} finally {
|
||||||
// Even a deliberately retained fixture directory must never retain auth.
|
|
||||||
await rm(authPath, { force: true });
|
|
||||||
if (process.env.REA_AGENT_EVAL_KEEP_FIXTURES !== "true")
|
if (process.env.REA_AGENT_EVAL_KEEP_FIXTURES !== "true")
|
||||||
await rm(evaluationRoot, { recursive: true, force: true });
|
await rm(evaluationRoot, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
|
|
||||||
async function createTargets(root, includeManaged) {
|
async function createTargets(root, includeManaged) {
|
||||||
const fixtures = await createAgentEvaluationFixtures(root, repositoryRoot);
|
const fixtures = await createAgentEvaluationFixtures(root, repositoryRoot);
|
||||||
const largeJavascript = join(root, "large-app");
|
|
||||||
await mkdir(join(largeJavascript, "lib"), { recursive: true });
|
|
||||||
await Promise.all([
|
|
||||||
writeFile(
|
|
||||||
join(largeJavascript, "package.json"),
|
|
||||||
'{"name":"large-app","main":"main.js"}\n',
|
|
||||||
),
|
|
||||||
writeFile(
|
|
||||||
join(largeJavascript, "main.js"),
|
|
||||||
'const { ipcMain } = require("electron");\nipcMain.handle("profile:read", (_event, id) => ({ id }));\n',
|
|
||||||
),
|
|
||||||
...Array.from({ length: 180 }, (_, index) =>
|
|
||||||
writeFile(
|
|
||||||
join(largeJavascript, "lib", `formatter-${String(index)}.js`),
|
|
||||||
`export const format = (value) => String(value).trim();\n`,
|
|
||||||
),
|
|
||||||
),
|
|
||||||
]);
|
|
||||||
|
|
||||||
const managedOutput = join(root, "managed-output");
|
const managedOutput = join(root, "managed-output");
|
||||||
if (includeManaged) {
|
if (includeManaged) {
|
||||||
@@ -438,29 +292,15 @@ async function createTargets(root, includeManaged) {
|
|||||||
return {
|
return {
|
||||||
native: "/bin/true",
|
native: "/bin/true",
|
||||||
...fixtures,
|
...fixtures,
|
||||||
largeJavascript,
|
|
||||||
managed: join(managedOutput, "AgentEval.dll"),
|
managed: join(managedOutput, "AgentEval.dll"),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function validTargetClarification(message) {
|
async function runCodex(prompt) {
|
||||||
const schema = z.strictObject({
|
|
||||||
status: z.literal("needs_target"),
|
|
||||||
requested_input: z.literal("app_name_or_artifact_path"),
|
|
||||||
question: z.string().refine((value) => value.trim().length > 0),
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
return schema.safeParse(JSON.parse(message)).success;
|
|
||||||
} catch (cause) {
|
|
||||||
if (cause instanceof SyntaxError) return false;
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function runCodex(prompt, approvedTools) {
|
|
||||||
const arguments_ = [
|
const arguments_ = [
|
||||||
"--no-daemon",
|
|
||||||
"exec",
|
"exec",
|
||||||
|
"--ignore-user-config",
|
||||||
|
"--ignore-rules",
|
||||||
"--ephemeral",
|
"--ephemeral",
|
||||||
"--skip-git-repo-check",
|
"--skip-git-repo-check",
|
||||||
"--sandbox",
|
"--sandbox",
|
||||||
@@ -472,16 +312,18 @@ async function runCodex(prompt, approvedTools) {
|
|||||||
evaluationRoot,
|
evaluationRoot,
|
||||||
"-c",
|
"-c",
|
||||||
'approval_policy="never"',
|
'approval_policy="never"',
|
||||||
...approvedTools.flatMap((name) => [
|
"-c",
|
||||||
"-c",
|
`mcp_servers.rea.command=${JSON.stringify(process.execPath)}`,
|
||||||
`mcp_servers.rea.tools.${name}.approval_mode="approve"`,
|
"-c",
|
||||||
]),
|
`mcp_servers.rea.args=${JSON.stringify([join(repositoryRoot, "scripts/rea.mjs"), "mcp"])}`,
|
||||||
|
"-c",
|
||||||
|
`mcp_servers.rea.startup_timeout_sec=${String(MCP_STARTUP_POLICY.codexStartupTimeoutSeconds)}`,
|
||||||
...(optionalModel === undefined ? [] : ["--model", optionalModel]),
|
...(optionalModel === undefined ? [] : ["--model", optionalModel]),
|
||||||
prompt,
|
prompt,
|
||||||
];
|
];
|
||||||
const child = spawn(codex, arguments_, {
|
const child = spawn(codex, arguments_, {
|
||||||
cwd: evaluationRoot,
|
cwd: evaluationRoot,
|
||||||
env: evaluationEnvironment,
|
env: process.env,
|
||||||
stdio: ["ignore", "pipe", "pipe"],
|
stdio: ["ignore", "pipe", "pipe"],
|
||||||
});
|
});
|
||||||
const events = [];
|
const events = [];
|
||||||
@@ -520,16 +362,9 @@ async function runCodex(prompt, approvedTools) {
|
|||||||
return { events, exitCode, stderr: stderr.join("").slice(0, 32_768) };
|
return { events, exitCode, stderr: stderr.join("").slice(0, 32_768) };
|
||||||
}
|
}
|
||||||
|
|
||||||
async function runProcess(
|
async function runProcess(command, arguments_, cwd, timeout) {
|
||||||
command,
|
|
||||||
arguments_,
|
|
||||||
cwd,
|
|
||||||
timeout,
|
|
||||||
env = process.env,
|
|
||||||
) {
|
|
||||||
const child = spawn(command, arguments_, {
|
const child = spawn(command, arguments_, {
|
||||||
cwd,
|
cwd,
|
||||||
env,
|
|
||||||
stdio: ["ignore", "ignore", "pipe"],
|
stdio: ["ignore", "ignore", "pipe"],
|
||||||
});
|
});
|
||||||
let stderr = "";
|
let stderr = "";
|
||||||
|
|||||||
@@ -22,13 +22,6 @@ const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
|||||||
const command = process.env.REA_VERIFY_COPILOT_COMMAND ?? "copilot";
|
const command = process.env.REA_VERIFY_COPILOT_COMMAND ?? "copilot";
|
||||||
const mode = process.argv[2] ?? "call";
|
const mode = process.argv[2] ?? "call";
|
||||||
const model = process.env.REA_VERIFY_COPILOT_MODEL ?? "gpt-5.4";
|
const model = process.env.REA_VERIFY_COPILOT_MODEL ?? "gpt-5.4";
|
||||||
const wireModel = process.env.REA_VERIFY_COPILOT_WIRE_MODEL ?? model;
|
|
||||||
const inputSchemaProfile =
|
|
||||||
process.env.REA_VERIFY_COPILOT_SCHEMA_PROFILE ?? "full";
|
|
||||||
assert(
|
|
||||||
["full", "compact"].includes(inputSchemaProfile),
|
|
||||||
"REA_VERIFY_COPILOT_SCHEMA_PROFILE must be full or compact",
|
|
||||||
);
|
|
||||||
assert(
|
assert(
|
||||||
["chat", "call"].includes(mode),
|
["chat", "call"].includes(mode),
|
||||||
"Usage: verify-copilot-client.mjs [chat|call]",
|
"Usage: verify-copilot-client.mjs [chat|call]",
|
||||||
@@ -51,7 +44,6 @@ await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
|||||||
const environment = {
|
const environment = {
|
||||||
PATH: process.env.PATH,
|
PATH: process.env.PATH,
|
||||||
NODE_OPTIONS: process.env.NODE_OPTIONS,
|
NODE_OPTIONS: process.env.NODE_OPTIONS,
|
||||||
HOME: account,
|
|
||||||
USERPROFILE: account,
|
USERPROFILE: account,
|
||||||
COPILOT_HOME: profile,
|
COPILOT_HOME: profile,
|
||||||
COPILOT_OFFLINE: "true",
|
COPILOT_OFFLINE: "true",
|
||||||
@@ -62,7 +54,7 @@ const environment = {
|
|||||||
COPILOT_PROVIDER_BEARER_TOKEN: "",
|
COPILOT_PROVIDER_BEARER_TOKEN: "",
|
||||||
COPILOT_PROVIDER_HEADERS: "",
|
COPILOT_PROVIDER_HEADERS: "",
|
||||||
COPILOT_PROVIDER_MODEL_ID: model,
|
COPILOT_PROVIDER_MODEL_ID: model,
|
||||||
COPILOT_PROVIDER_WIRE_MODEL: wireModel,
|
COPILOT_PROVIDER_WIRE_MODEL: model,
|
||||||
COPILOT_PROVIDER_MAX_PROMPT_TOKENS: "1000000",
|
COPILOT_PROVIDER_MAX_PROMPT_TOKENS: "1000000",
|
||||||
COPILOT_PROVIDER_MAX_OUTPUT_TOKENS: "4096",
|
COPILOT_PROVIDER_MAX_OUTPUT_TOKENS: "4096",
|
||||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||||
@@ -120,17 +112,10 @@ const extractEvidence = (value) => {
|
|||||||
}
|
}
|
||||||
return undefined;
|
return undefined;
|
||||||
};
|
};
|
||||||
const wireModels = [];
|
|
||||||
const fixture = await createOpenAiModelFixture({
|
const fixture = await createOpenAiModelFixture({
|
||||||
directory: lab,
|
directory: lab,
|
||||||
prefix: "copilot",
|
prefix: "copilot",
|
||||||
onRequest: async ({ body, tools, requestIndex }) => {
|
onRequest: async ({ body, tools, requestIndex }) => {
|
||||||
assert.equal(
|
|
||||||
body.model,
|
|
||||||
wireModel,
|
|
||||||
"Copilot must send the selected wire model",
|
|
||||||
);
|
|
||||||
wireModels.push(body.model);
|
|
||||||
// Auxiliary requests without tools cannot execute the analyst's workflow.
|
// Auxiliary requests without tools cannot execute the analyst's workflow.
|
||||||
if (tools.length === 0)
|
if (tools.length === 0)
|
||||||
return { role: "assistant", content: "REA fixture title" };
|
return { role: "assistant", content: "REA fixture title" };
|
||||||
@@ -326,16 +311,6 @@ try {
|
|||||||
assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []);
|
assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []);
|
||||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||||
if (inputSchemaProfile === "compact") {
|
|
||||||
// Select the existing presentation in this disposable native-client profile.
|
|
||||||
const customized = JSON.parse(configured);
|
|
||||||
assert(customized.mcpServers.rea);
|
|
||||||
customized.mcpServers.rea.env = {
|
|
||||||
...customized.mcpServers.rea.env,
|
|
||||||
REA_MCP_INPUT_SCHEMA_PROFILE: inputSchemaProfile,
|
|
||||||
};
|
|
||||||
await writeFile(configPath, JSON.stringify(customized, null, 2));
|
|
||||||
}
|
|
||||||
const installedSkill = await readFile(
|
const installedSkill = await readFile(
|
||||||
join(skillDirectory, "SKILL.md"),
|
join(skillDirectory, "SKILL.md"),
|
||||||
"utf8",
|
"utf8",
|
||||||
@@ -433,8 +408,6 @@ try {
|
|||||||
clientVersion,
|
clientVersion,
|
||||||
mode,
|
mode,
|
||||||
model,
|
model,
|
||||||
wireModel,
|
|
||||||
inputSchemaProfile,
|
|
||||||
passed: failure === undefined,
|
passed: failure === undefined,
|
||||||
artifacts: lab,
|
artifacts: lab,
|
||||||
runtimeRoot,
|
runtimeRoot,
|
||||||
@@ -448,9 +421,7 @@ try {
|
|||||||
skillDiscovery:
|
skillDiscovery:
|
||||||
"native COPILOT_HOME MCP configuration and caller-added isolated skill directory; default OS-home discovery unverified",
|
"native COPILOT_HOME MCP configuration and caller-added isolated skill directory; default OS-home discovery unverified",
|
||||||
capacityBoundary:
|
capacityBoundary:
|
||||||
inputSchemaProfile === "compact"
|
"gpt-4.1 model configuration blocks the full catalog before HTTP even with a ten-million-token requested prompt override; gpt-5.4 is the verified local fixture configuration",
|
||||||
? "Explicit compact profile: complete inventory and canonical server validation; advertised schemas may be reduced. This does not fix the full-profile gpt-4.1 context boundary."
|
|
||||||
: "Complete canonical input and output schemas. Client model ID and wire model are recorded separately; effective token capacity and live provider compatibility remain unknown.",
|
|
||||||
catalogSize,
|
catalogSize,
|
||||||
schemasChecked,
|
schemasChecked,
|
||||||
spillReadRequested,
|
spillReadRequested,
|
||||||
@@ -458,7 +429,6 @@ try {
|
|||||||
skillLoaded,
|
skillLoaded,
|
||||||
artifactDigest,
|
artifactDigest,
|
||||||
finalSeen,
|
finalSeen,
|
||||||
wireModels,
|
|
||||||
requests: fixture.requests,
|
requests: fixture.requests,
|
||||||
probes: fixture.probes,
|
probes: fixture.probes,
|
||||||
fixtureFailure: fixture.failure,
|
fixtureFailure: fixture.failure,
|
||||||
|
|||||||
@@ -40,9 +40,9 @@ await Promise.all([
|
|||||||
]);
|
]);
|
||||||
const environment = {
|
const environment = {
|
||||||
PATH: process.env.PATH ?? "",
|
PATH: process.env.PATH ?? "",
|
||||||
HOME: account,
|
|
||||||
USERPROFILE: account,
|
USERPROFILE: account,
|
||||||
DSH_HOME: profile,
|
DSH_HOME: profile,
|
||||||
|
DSH_AGENTS_HOME: join(account, ".agents"),
|
||||||
OPENAI_API_KEY: "local-fixture-key",
|
OPENAI_API_KEY: "local-fixture-key",
|
||||||
REA_PROCESS_RUN_ID: verifier.run_id,
|
REA_PROCESS_RUN_ID: verifier.run_id,
|
||||||
};
|
};
|
||||||
@@ -328,7 +328,7 @@ try {
|
|||||||
"loopback custom OpenAI-compatible adapter; no live DeepSeek provider claim",
|
"loopback custom OpenAI-compatible adapter; no live DeepSeek provider claim",
|
||||||
modelContextWindow: 1000000,
|
modelContextWindow: 1000000,
|
||||||
skillDiscovery:
|
skillDiscovery:
|
||||||
"native HOME shared personal directory; isolated project root",
|
"native DSH_AGENTS_HOME personal directory; isolated project root",
|
||||||
catalogSize: catalog?.length,
|
catalogSize: catalog?.length,
|
||||||
patternCount,
|
patternCount,
|
||||||
uniquePatterns: patterns.size,
|
uniquePatterns: patterns.size,
|
||||||
|
|||||||
@@ -54,7 +54,6 @@ await writeFile(
|
|||||||
);
|
);
|
||||||
const environment = {
|
const environment = {
|
||||||
PATH: process.env.PATH ?? "",
|
PATH: process.env.PATH ?? "",
|
||||||
HOME: account,
|
|
||||||
USERPROFILE: account,
|
USERPROFILE: account,
|
||||||
[client === "pi" ? "PI_CODING_AGENT_DIR" : "HERMES_HOME"]: profile,
|
[client === "pi" ? "PI_CODING_AGENT_DIR" : "HERMES_HOME"]: profile,
|
||||||
PI_OFFLINE: "1",
|
PI_OFFLINE: "1",
|
||||||
@@ -72,6 +71,13 @@ const original =
|
|||||||
})
|
})
|
||||||
: "# Keep caller settings.\nmcp_servers:\n other:\n command: unrelated-server\n enabled: false\n";
|
: "# Keep caller settings.\nmcp_servers:\n other:\n command: unrelated-server\n enabled: false\n";
|
||||||
await writeFile(configPath, original);
|
await writeFile(configPath, original);
|
||||||
|
if (client === "pi") {
|
||||||
|
// The CLI fixture account and native os.homedir() differ; use Pi's caller-owned skill setting.
|
||||||
|
await writeFile(
|
||||||
|
join(profile, "settings.json"),
|
||||||
|
JSON.stringify({ skills: [join(account, ".agents", "skills")] }),
|
||||||
|
);
|
||||||
|
}
|
||||||
const clientVersion = (
|
const clientVersion = (
|
||||||
await exec(command, ["--version"], { env: environment, timeout: 30_000 })
|
await exec(command, ["--version"], { env: environment, timeout: 30_000 })
|
||||||
).stdout.trim();
|
).stdout.trim();
|
||||||
@@ -442,7 +448,7 @@ try {
|
|||||||
"loopback deterministic OpenAI-compatible endpoint; no live provider claim",
|
"loopback deterministic OpenAI-compatible endpoint; no live provider claim",
|
||||||
skillDiscovery:
|
skillDiscovery:
|
||||||
client === "pi"
|
client === "pi"
|
||||||
? "native HOME shared personal directory; no explicit skill setting"
|
? "caller-configured isolated shared directory; default OS home discovery unverified"
|
||||||
: "native HERMES_HOME personal directory",
|
: "native HERMES_HOME personal directory",
|
||||||
stickyHermes,
|
stickyHermes,
|
||||||
catalogSize: catalog?.length,
|
catalogSize: catalog?.length,
|
||||||
|
|||||||
@@ -45,7 +45,6 @@ await Promise.all([
|
|||||||
]);
|
]);
|
||||||
const baseEnvironment = {
|
const baseEnvironment = {
|
||||||
PATH: process.env.PATH ?? "",
|
PATH: process.env.PATH ?? "",
|
||||||
HOME: account,
|
|
||||||
USERPROFILE: account,
|
USERPROFILE: account,
|
||||||
QWEN_HOME: profile,
|
QWEN_HOME: profile,
|
||||||
QWEN_RUNTIME_DIR: runtime,
|
QWEN_RUNTIME_DIR: runtime,
|
||||||
@@ -72,6 +71,9 @@ const original = JSON.stringify(
|
|||||||
},
|
},
|
||||||
telemetry: { enabled: false },
|
telemetry: { enabled: false },
|
||||||
model: { name: "rea-client-fixture" },
|
model: { name: "rea-client-fixture" },
|
||||||
|
// Qwen uses os.homedir(), while REA's isolated CLI account uses USERPROFILE.
|
||||||
|
// This caller-owned setting lets the real client load the REA-installed bundle.
|
||||||
|
skills: { directories: [sharedSkills] },
|
||||||
mcpServers: { other: { command: "unrelated-server", disabled: true } },
|
mcpServers: { other: { command: "unrelated-server", disabled: true } },
|
||||||
},
|
},
|
||||||
null,
|
null,
|
||||||
@@ -110,7 +112,7 @@ const configured = JSON.parse(
|
|||||||
await readFile(join(profile, "settings.json"), "utf8"),
|
await readFile(join(profile, "settings.json"), "utf8"),
|
||||||
);
|
);
|
||||||
assert.deepEqual(configured.model, { name: "rea-client-fixture" });
|
assert.deepEqual(configured.model, { name: "rea-client-fixture" });
|
||||||
assert.equal(configured.skills, undefined);
|
assert.deepEqual(configured.skills, { directories: [sharedSkills] });
|
||||||
assert.deepEqual(configured.mcpServers.other, {
|
assert.deepEqual(configured.mcpServers.other, {
|
||||||
command: "unrelated-server",
|
command: "unrelated-server",
|
||||||
disabled: true,
|
disabled: true,
|
||||||
@@ -382,7 +384,7 @@ try {
|
|||||||
modelFixture:
|
modelFixture:
|
||||||
"loopback deterministic OpenAI-compatible endpoint; no live model-provider claim",
|
"loopback deterministic OpenAI-compatible endpoint; no live model-provider claim",
|
||||||
skillDiscovery:
|
skillDiscovery:
|
||||||
"native HOME shared personal directory; no explicit skill setting",
|
"caller-configured isolated shared directory; default OS home discovery unverified",
|
||||||
catalogSize: TOOL_CONTRACTS.length,
|
catalogSize: TOOL_CONTRACTS.length,
|
||||||
requests: fixture.requests,
|
requests: fixture.requests,
|
||||||
probes: fixture.probes,
|
probes: fixture.probes,
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { execFile } from "node:child_process";
|
import { execFile } from "node:child_process";
|
||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
import { mkdtemp, mkdir, readFile, rm } from "node:fs/promises";
|
||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { fileURLToPath } from "node:url";
|
import { fileURLToPath } from "node:url";
|
||||||
@@ -60,9 +60,6 @@ const env = {
|
|||||||
JAVA_HOME: installation.javaHome,
|
JAVA_HOME: installation.javaHome,
|
||||||
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper",
|
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper",
|
||||||
};
|
};
|
||||||
for (const setting of ["GHIDRA_HEADLESS_MAXMEM", "GHIDRA_MAXMEM"]) {
|
|
||||||
if (process.env[setting] !== undefined) env[setting] = process.env[setting];
|
|
||||||
}
|
|
||||||
const reports = [];
|
const reports = [];
|
||||||
let primaryFailure;
|
let primaryFailure;
|
||||||
try {
|
try {
|
||||||
@@ -230,103 +227,6 @@ try {
|
|||||||
fixture.sha256,
|
fixture.sha256,
|
||||||
"Analysis modified the source executable",
|
"Analysis modified the source executable",
|
||||||
);
|
);
|
||||||
// Carved firmware keeps all program headers/load bytes but loses sections.
|
|
||||||
const read16 = (offset) =>
|
|
||||||
byteOrder === "little"
|
|
||||||
? sourceBytes.readUInt16LE(offset)
|
|
||||||
: sourceBytes.readUInt16BE(offset);
|
|
||||||
const read32 = (offset) =>
|
|
||||||
byteOrder === "little"
|
|
||||||
? sourceBytes.readUInt32LE(offset)
|
|
||||||
: sourceBytes.readUInt32BE(offset);
|
|
||||||
const sectionOffset = read32(32);
|
|
||||||
const cut = sectionOffset - 1;
|
|
||||||
const programOffset = read32(28);
|
|
||||||
const programCount = read16(44);
|
|
||||||
assert.equal(read16(42), 32);
|
|
||||||
assert.ok(cut >= programOffset + programCount * 32);
|
|
||||||
for (let i = 0; i < programCount; i += 1) {
|
|
||||||
const offset = programOffset + i * 32;
|
|
||||||
assert.ok(read32(offset + 4) + read32(offset + 16) <= cut);
|
|
||||||
}
|
|
||||||
const carvedBytes = sourceBytes.subarray(0, cut);
|
|
||||||
const carvedPath = join(workspace, `${byteOrder}-carved.elf`);
|
|
||||||
const carvedSha256 = createHash("sha256")
|
|
||||||
.update(carvedBytes)
|
|
||||||
.digest("hex");
|
|
||||||
await writeFile(carvedPath, carvedBytes);
|
|
||||||
const carvedResolved = await parseBinaryTarget(carvedPath);
|
|
||||||
if (!carvedResolved.ok) throw carvedResolved.error;
|
|
||||||
assert.deepEqual(
|
|
||||||
carvedResolved.value.mips.abiFlags,
|
|
||||||
independent.mips.abiFlags,
|
|
||||||
);
|
|
||||||
assert.match(
|
|
||||||
carvedResolved.value.mips.limitations.join(" "),
|
|
||||||
/section header table/u,
|
|
||||||
);
|
|
||||||
const carvedTarget = await call("open_binary", {
|
|
||||||
path: carvedPath,
|
|
||||||
provider_id: "ghidra",
|
|
||||||
});
|
|
||||||
assert.equal(carvedTarget.sha256, carvedSha256);
|
|
||||||
const carvedImage = await call("inspect_native_load_image");
|
|
||||||
assertMipsLoadedImage(carvedImage.observations, byteOrder, carvedSha256);
|
|
||||||
const carvedProcedures = await call("list_procedures");
|
|
||||||
assert.ok(
|
|
||||||
carvedProcedures.some(
|
|
||||||
(p) =>
|
|
||||||
BigInt(p.address) === BigInt(independent.symbols.rea_mips_entry),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
const carvedDossier = functionDossierSchema.parse(
|
|
||||||
await call("analyze_function", {
|
|
||||||
procedure: independent.symbols.rea_mips_entry,
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
assert.ok(carvedDossier.pseudocode.trim().length > 0);
|
|
||||||
const carvedSnapshotPath = join(
|
|
||||||
workspace,
|
|
||||||
`${byteOrder}-carved.snapshot.json`,
|
|
||||||
);
|
|
||||||
await call("close_binary", { snapshot_path: carvedSnapshotPath });
|
|
||||||
const carvedSnapshot = parseAnalysisSnapshot(
|
|
||||||
JSON.parse(await readFile(carvedSnapshotPath, "utf8")),
|
|
||||||
);
|
|
||||||
assert.equal(carvedSnapshot.target.sha256, carvedSha256);
|
|
||||||
assert.ok(
|
|
||||||
carvedSnapshot.evidence_bundle.records.some((record) =>
|
|
||||||
parseEvidence(record).limitations.some((item) =>
|
|
||||||
item.includes("section header table"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
const carvedCli = await execute(
|
|
||||||
process.execPath,
|
|
||||||
[
|
|
||||||
entrypoint,
|
|
||||||
"function",
|
|
||||||
carvedPath,
|
|
||||||
independent.symbols.rea_mips_entry,
|
|
||||||
"--provider",
|
|
||||||
"ghidra",
|
|
||||||
"--json",
|
|
||||||
],
|
|
||||||
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
|
|
||||||
);
|
|
||||||
const carvedEvidence = parseEvidence(JSON.parse(carvedCli.stdout));
|
|
||||||
assert.equal(carvedEvidence.subject?.digest.sha256, carvedSha256);
|
|
||||||
assert.ok(
|
|
||||||
carvedEvidence.limitations.some((item) =>
|
|
||||||
item.includes("section header table"),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
assert.equal(
|
|
||||||
functionDossierSchema.parse(carvedEvidence.normalized_result).procedure
|
|
||||||
.address,
|
|
||||||
carvedDossier.procedure.address,
|
|
||||||
);
|
|
||||||
assert.deepEqual(await readFile(carvedPath), carvedBytes);
|
|
||||||
reports.push({
|
reports.push({
|
||||||
...fixture,
|
...fixture,
|
||||||
independent_reader: independent,
|
independent_reader: independent,
|
||||||
@@ -341,14 +241,6 @@ try {
|
|||||||
"CLI/MCP and snapshot identity",
|
"CLI/MCP and snapshot identity",
|
||||||
],
|
],
|
||||||
procedures: procedures.length,
|
procedures: procedures.length,
|
||||||
carved: {
|
|
||||||
sha256: carvedSha256,
|
|
||||||
source_bytes: carvedBytes.length,
|
|
||||||
section_header_offset: sectionOffset,
|
|
||||||
effective_language: carvedImage.observations.language_id,
|
|
||||||
limitations: carvedResolved.value.mips.limitations,
|
|
||||||
procedures: carvedProcedures.length,
|
|
||||||
},
|
|
||||||
status: "passed",
|
status: "passed",
|
||||||
});
|
});
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
import { realpath } from "node:fs/promises";
|
import { realpath } from "node:fs/promises";
|
||||||
import { parseArgs } from "node:util";
|
import { parseArgs } from "node:util";
|
||||||
|
|
||||||
import { analyzeJavaScriptApplicationWithOwnedScope } from "../../lib/javascript-application-analysis.mjs";
|
import { analyzeJavaScriptApplication } from "../../../dist/application/javascript/JavaScriptApplicationService.js";
|
||||||
import {
|
import {
|
||||||
compareApplicationVersionsEvidence,
|
compareApplicationVersionsEvidence,
|
||||||
compareJavaScriptExportShapesEvidence,
|
compareJavaScriptExportShapesEvidence,
|
||||||
@@ -199,5 +199,9 @@ function selectorKey(selector) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function analyze(path) {
|
async function analyze(path) {
|
||||||
return analyzeJavaScriptApplicationWithOwnedScope({ input_path: path });
|
const result = await analyzeJavaScriptApplication({
|
||||||
|
input_path: path,
|
||||||
|
});
|
||||||
|
if (!result.ok) throw result.error;
|
||||||
|
return result.value;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,8 +3,8 @@
|
|||||||
import { readFile, realpath } from "node:fs/promises";
|
import { readFile, realpath } from "node:fs/promises";
|
||||||
import { parseArgs } from "node:util";
|
import { parseArgs } from "node:util";
|
||||||
|
|
||||||
|
import { analyzeJavaScriptApplication } from "../../../dist/application/javascript/JavaScriptApplicationService.js";
|
||||||
import { reconcileJavaScriptRuntimeEvidence } from "../../../dist/application/javascript/JavaScriptRuntimeReconciliationService.js";
|
import { reconcileJavaScriptRuntimeEvidence } from "../../../dist/application/javascript/JavaScriptRuntimeReconciliationService.js";
|
||||||
import { analyzeJavaScriptApplicationWithOwnedScope } from "../../lib/javascript-application-analysis.mjs";
|
|
||||||
import {
|
import {
|
||||||
completeVerifierRun,
|
completeVerifierRun,
|
||||||
createVerifierRun,
|
createVerifierRun,
|
||||||
@@ -44,13 +44,14 @@ const canonicalLayers = await Promise.all(
|
|||||||
);
|
);
|
||||||
const staticLayers = [];
|
const staticLayers = [];
|
||||||
for (const layer of canonicalLayers) {
|
for (const layer of canonicalLayers) {
|
||||||
const analyzed = await analyzeJavaScriptApplicationWithOwnedScope({
|
const analyzed = await analyzeJavaScriptApplication({
|
||||||
input_path: layer.path,
|
input_path: layer.path,
|
||||||
format: "auto",
|
format: "auto",
|
||||||
});
|
});
|
||||||
|
if (!analyzed.ok) throw analyzed.error;
|
||||||
staticLayers.push({
|
staticLayers.push({
|
||||||
role: layer.role,
|
role: layer.role,
|
||||||
analysis: analyzed,
|
analysis: analyzed.value,
|
||||||
runtime_mappings: [],
|
runtime_mappings: [],
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
+46
-348
@@ -1,6 +1,6 @@
|
|||||||
import { execFile } from "node:child_process";
|
import { execFile } from "node:child_process";
|
||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import { constants as fsConstants } from "node:fs";
|
import { createReadStream, constants as fsConstants } from "node:fs";
|
||||||
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
|
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
|
||||||
import { basename, join, resolve } from "node:path";
|
import { basename, join, resolve } from "node:path";
|
||||||
|
|
||||||
@@ -24,11 +24,8 @@ import {
|
|||||||
AnalysisTimeoutError,
|
AnalysisTimeoutError,
|
||||||
AnalysisUnsupportedTargetError,
|
AnalysisUnsupportedTargetError,
|
||||||
} from "../domain/analysisErrorCore.js";
|
} from "../domain/analysisErrorCore.js";
|
||||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
|
||||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
|
||||||
import {
|
import {
|
||||||
execFileOutput,
|
execFileOutput,
|
||||||
execFileOutputFailure,
|
execFileOutputFailure,
|
||||||
@@ -100,18 +97,11 @@ const STDERR_EXCERPT_BYTES = 512;
|
|||||||
const UNPARSED_LINES_REPORTED = 10;
|
const UNPARSED_LINES_REPORTED = 10;
|
||||||
const STOP_GRACE_MS = 5_000;
|
const STOP_GRACE_MS = 5_000;
|
||||||
|
|
||||||
const hashAdbFile = async (
|
/** Stream a local file digest without retaining large files in memory. */
|
||||||
path: string,
|
const sha256File = async (path: string): Promise<string> => {
|
||||||
operation: string,
|
const hash = createHash("sha256");
|
||||||
signal: AbortSignal | undefined,
|
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||||
) => {
|
return hash.digest("hex");
|
||||||
try {
|
|
||||||
return await hashStableFile(path, signal);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (signal?.aborted === true)
|
|
||||||
throw new AnalysisCancelledError(operation, { cause });
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/** adb joins shell operands into a remote POSIX command; quote at that boundary. */
|
/** adb joins shell operands into a remote POSIX command; quote at that boundary. */
|
||||||
@@ -345,48 +335,6 @@ export interface AdbProviderOptions {
|
|||||||
readonly environment?: Readonly<Record<string, string | undefined>>;
|
readonly environment?: Readonly<Record<string, string | undefined>>;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface PendingPackageDirectoryCleanup {
|
|
||||||
readonly path: string;
|
|
||||||
readonly identity: PackageDirectoryIdentity | undefined;
|
|
||||||
readonly primary: AnalysisError;
|
|
||||||
readonly remove: () => Promise<void>;
|
|
||||||
}
|
|
||||||
|
|
||||||
interface PackageDirectoryIdentity {
|
|
||||||
readonly dev: number;
|
|
||||||
readonly ino: number;
|
|
||||||
readonly mode: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
const removeOwnedPackageDirectory = async (
|
|
||||||
path: string,
|
|
||||||
identity: PackageDirectoryIdentity | undefined,
|
|
||||||
): Promise<void> => {
|
|
||||||
if (identity === undefined)
|
|
||||||
throw new Error(
|
|
||||||
`Cannot verify the created package directory identity; refusing to remove ${path}`,
|
|
||||||
);
|
|
||||||
let current: Awaited<ReturnType<typeof lstat>>;
|
|
||||||
try {
|
|
||||||
current = await lstat(path);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
|
|
||||||
return;
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
if (
|
|
||||||
!current.isDirectory() ||
|
|
||||||
current.isSymbolicLink() ||
|
|
||||||
current.dev !== identity.dev ||
|
|
||||||
current.ino !== identity.ino ||
|
|
||||||
current.mode !== identity.mode
|
|
||||||
)
|
|
||||||
throw new Error(
|
|
||||||
`The created package directory was replaced or changed; refusing to remove ${path}`,
|
|
||||||
);
|
|
||||||
await rm(path, { recursive: true, force: true });
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* ADB device inspection and acquisition over a caller-selected adb binary.
|
* ADB device inspection and acquisition over a caller-selected adb binary.
|
||||||
*
|
*
|
||||||
@@ -398,12 +346,6 @@ const removeOwnedPackageDirectory = async (
|
|||||||
*/
|
*/
|
||||||
export class AdbProvider {
|
export class AdbProvider {
|
||||||
private readonly environment: Readonly<Record<string, string | undefined>>;
|
private readonly environment: Readonly<Record<string, string | undefined>>;
|
||||||
readonly #pendingPackageDirectoryCleanup =
|
|
||||||
new Set<PendingPackageDirectoryCleanup>();
|
|
||||||
readonly #activeOperations = new Set<Promise<void>>();
|
|
||||||
#closing = false;
|
|
||||||
#closePromise: Promise<void> | undefined;
|
|
||||||
#packageCleanupPromise: Promise<void> | undefined;
|
|
||||||
|
|
||||||
constructor(options: AdbProviderOptions = {}) {
|
constructor(options: AdbProviderOptions = {}) {
|
||||||
this.environment = options.environment ?? process.env;
|
this.environment = options.environment ?? process.env;
|
||||||
@@ -436,121 +378,12 @@ export class AdbProvider {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Retry removal of package directories retained after failed rollback. */
|
/** The provider holds no long-lived resources; the adb server is caller-owned. */
|
||||||
close(): Promise<void> {
|
async close(): Promise<void> {}
|
||||||
this.#closing = true;
|
|
||||||
if (this.#closePromise !== undefined) return this.#closePromise;
|
|
||||||
const closing = (async () => {
|
|
||||||
await Promise.all(this.#activeOperations);
|
|
||||||
await this.#retryPackageDirectoryCleanup();
|
|
||||||
})().finally(() => {
|
|
||||||
if (this.#closePromise === closing) this.#closePromise = undefined;
|
|
||||||
});
|
|
||||||
this.#closePromise = closing;
|
|
||||||
return this.#closePromise;
|
|
||||||
}
|
|
||||||
|
|
||||||
#retryPackageDirectoryCleanup(): Promise<void> {
|
|
||||||
if (this.#packageCleanupPromise !== undefined)
|
|
||||||
return this.#packageCleanupPromise;
|
|
||||||
const cleanup = this.#performPackageDirectoryCleanup().finally(() => {
|
|
||||||
if (this.#packageCleanupPromise === cleanup)
|
|
||||||
this.#packageCleanupPromise = undefined;
|
|
||||||
});
|
|
||||||
this.#packageCleanupPromise = cleanup;
|
|
||||||
return cleanup;
|
|
||||||
}
|
|
||||||
|
|
||||||
async #performPackageDirectoryCleanup(): Promise<void> {
|
|
||||||
const failures: AnalysisError[] = [];
|
|
||||||
for (const owner of this.#pendingPackageDirectoryCleanup) {
|
|
||||||
try {
|
|
||||||
await owner.remove();
|
|
||||||
this.#pendingPackageDirectoryCleanup.delete(owner);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
const cleanup = new ProviderCleanupError(
|
|
||||||
ADB_PROVIDER_IDENTITY.id,
|
|
||||||
[owner.path],
|
|
||||||
{
|
|
||||||
reason: cause instanceof Error ? cause.message : String(cause),
|
|
||||||
},
|
|
||||||
{ operation: "pull_adb_package", cause },
|
|
||||||
);
|
|
||||||
failures.push(
|
|
||||||
analysisErrorWithCleanupFailure(
|
|
||||||
owner.primary,
|
|
||||||
cleanup,
|
|
||||||
"pull_adb_package",
|
|
||||||
),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (failures.length === 1) throw failures[0];
|
|
||||||
if (failures.length > 1)
|
|
||||||
throw new ProviderCleanupError(
|
|
||||||
ADB_PROVIDER_IDENTITY.id,
|
|
||||||
[...this.#pendingPackageDirectoryCleanup].map(({ path }) => path),
|
|
||||||
{
|
|
||||||
reason:
|
|
||||||
"Failed package output directories remain after cleanup retry",
|
|
||||||
failures: failures.map(({ message, cleanupResources }) => ({
|
|
||||||
message,
|
|
||||||
resources: [...cleanupResources],
|
|
||||||
})),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
operation: "pull_adb_package",
|
|
||||||
cause: new AggregateError(
|
|
||||||
failures,
|
|
||||||
"Multiple ADB package rollback cleanups failed",
|
|
||||||
),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
async execute(
|
async execute(
|
||||||
request: AdbRequest,
|
request: AdbRequest,
|
||||||
options?: ExecutionOptions,
|
options?: ExecutionOptions,
|
||||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
|
||||||
if (this.#closing)
|
|
||||||
return err(new AnalysisCancelledError(request.operation));
|
|
||||||
let release!: () => void;
|
|
||||||
const active = new Promise<void>((resolve) => {
|
|
||||||
release = resolve;
|
|
||||||
});
|
|
||||||
this.#activeOperations.add(active);
|
|
||||||
try {
|
|
||||||
if (this.#pendingPackageDirectoryCleanup.size > 0) {
|
|
||||||
try {
|
|
||||||
await this.#retryPackageDirectoryCleanup();
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
return err(
|
|
||||||
cause instanceof AnalysisError
|
|
||||||
? cause
|
|
||||||
: new ProviderCleanupError(
|
|
||||||
ADB_PROVIDER_IDENTITY.id,
|
|
||||||
[...this.#pendingPackageDirectoryCleanup].map(
|
|
||||||
({ path }) => path,
|
|
||||||
),
|
|
||||||
{
|
|
||||||
reason:
|
|
||||||
cause instanceof Error ? cause.message : String(cause),
|
|
||||||
},
|
|
||||||
{ operation: request.operation, cause },
|
|
||||||
),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return await this.#executeOperation(request, options);
|
|
||||||
} finally {
|
|
||||||
this.#activeOperations.delete(active);
|
|
||||||
release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async #executeOperation(
|
|
||||||
request: AdbRequest,
|
|
||||||
options?: ExecutionOptions,
|
|
||||||
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
): Promise<Result<AnalysisExecution, AnalysisError>> {
|
||||||
const signal = options?.signal;
|
const signal = options?.signal;
|
||||||
if (signal?.aborted === true)
|
if (signal?.aborted === true)
|
||||||
@@ -1177,11 +1010,9 @@ export class AdbProvider {
|
|||||||
);
|
);
|
||||||
if (!directory.ok) return directory;
|
if (!directory.ok) return directory;
|
||||||
const packageDirectory = join(directory.value, packageName);
|
const packageDirectory = join(directory.value, packageName);
|
||||||
let packageDirectoryCreated = false;
|
|
||||||
let packageDirectoryIdentity: PackageDirectoryIdentity | undefined;
|
|
||||||
try {
|
try {
|
||||||
if (
|
if (
|
||||||
await lstat(packageDirectory).then(
|
await stat(packageDirectory).then(
|
||||||
() => true,
|
() => true,
|
||||||
() => false,
|
() => false,
|
||||||
)
|
)
|
||||||
@@ -1190,44 +1021,14 @@ export class AdbProvider {
|
|||||||
`The package output directory already exists: ${packageDirectory}`,
|
`The package output directory already exists: ${packageDirectory}`,
|
||||||
);
|
);
|
||||||
await mkdir(packageDirectory);
|
await mkdir(packageDirectory);
|
||||||
packageDirectoryCreated = true;
|
|
||||||
const created = await lstat(packageDirectory);
|
|
||||||
if (!created.isDirectory() || created.isSymbolicLink())
|
|
||||||
throw new Error(
|
|
||||||
`The created package output is not a regular directory: ${packageDirectory}`,
|
|
||||||
);
|
|
||||||
packageDirectoryIdentity = {
|
|
||||||
dev: created.dev,
|
|
||||||
ino: created.ino,
|
|
||||||
mode: created.mode,
|
|
||||||
};
|
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
const primary = new AnalysisOutputError(
|
return err(
|
||||||
operation,
|
new AnalysisOutputError(
|
||||||
cause instanceof Error ? cause.message : String(cause),
|
operation,
|
||||||
{ cause },
|
cause instanceof Error ? cause.message : String(cause),
|
||||||
|
{ cause },
|
||||||
|
),
|
||||||
);
|
);
|
||||||
if (!packageDirectoryCreated) return err(primary);
|
|
||||||
const owner: PendingPackageDirectoryCleanup = {
|
|
||||||
path: packageDirectory,
|
|
||||||
identity: packageDirectoryIdentity,
|
|
||||||
primary,
|
|
||||||
remove: () =>
|
|
||||||
removeOwnedPackageDirectory(
|
|
||||||
packageDirectory,
|
|
||||||
packageDirectoryIdentity,
|
|
||||||
),
|
|
||||||
};
|
|
||||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
|
||||||
const cleanup = new ProviderCleanupError(
|
|
||||||
ADB_PROVIDER_IDENTITY.id,
|
|
||||||
[packageDirectory],
|
|
||||||
{
|
|
||||||
reason: "Could not establish an owned package directory identity",
|
|
||||||
},
|
|
||||||
{ operation, cause },
|
|
||||||
);
|
|
||||||
return err(analysisErrorWithCleanupFailure(primary, cleanup, operation));
|
|
||||||
}
|
}
|
||||||
const artifacts: {
|
const artifacts: {
|
||||||
device_path: string;
|
device_path: string;
|
||||||
@@ -1243,37 +1044,6 @@ export class AdbProvider {
|
|||||||
stage: "resolve_paths" | "pull" | "digest";
|
stage: "resolve_paths" | "pull" | "digest";
|
||||||
message: string;
|
message: string;
|
||||||
}[] = [];
|
}[] = [];
|
||||||
const cancelledPackagePull = (
|
|
||||||
error: AnalysisCancelledError,
|
|
||||||
localPath: string,
|
|
||||||
devicePath: string,
|
|
||||||
stage: "pull" | "digest",
|
|
||||||
) => {
|
|
||||||
error.retainPartialObservation({
|
|
||||||
provider_id: "adb",
|
|
||||||
operation,
|
|
||||||
result: {
|
|
||||||
client,
|
|
||||||
serial,
|
|
||||||
package_name: packageName,
|
|
||||||
output_directory: directory.value,
|
|
||||||
artifacts,
|
|
||||||
failures: [
|
|
||||||
...failures,
|
|
||||||
{
|
|
||||||
device_path: devicePath,
|
|
||||||
stage,
|
|
||||||
message:
|
|
||||||
stage === "pull"
|
|
||||||
? `Pull cancelled; any incomplete file is retained at ${localPath}. Remaining APKs were not attempted.`
|
|
||||||
: `Digesting the pulled file was cancelled; the file is retained at ${localPath}. Remaining APKs were not attempted.`,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
coverage: "partial",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
return err(error);
|
|
||||||
};
|
|
||||||
const seenFileNames = new Set<string>();
|
const seenFileNames = new Set<string>();
|
||||||
for (const entry of parsed.paths) {
|
for (const entry of parsed.paths) {
|
||||||
if (seenFileNames.has(entry.file_name)) {
|
if (seenFileNames.has(entry.file_name)) {
|
||||||
@@ -1295,13 +1065,7 @@ export class AdbProvider {
|
|||||||
signal,
|
signal,
|
||||||
);
|
);
|
||||||
if (!pulled.ok) {
|
if (!pulled.ok) {
|
||||||
if (pulled.error instanceof AnalysisCancelledError)
|
if (pulled.error instanceof AnalysisCancelledError) return pulled;
|
||||||
return cancelledPackagePull(
|
|
||||||
pulled.error,
|
|
||||||
localPath,
|
|
||||||
entry.device_path,
|
|
||||||
"pull",
|
|
||||||
);
|
|
||||||
failures.push({
|
failures.push({
|
||||||
device_path: entry.device_path,
|
device_path: entry.device_path,
|
||||||
stage: "pull",
|
stage: "pull",
|
||||||
@@ -1310,24 +1074,18 @@ export class AdbProvider {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const digest = await hashAdbFile(localPath, operation, signal);
|
const info = await stat(localPath);
|
||||||
|
const sha256 = await sha256File(localPath);
|
||||||
artifacts.push({
|
artifacts.push({
|
||||||
device_path: entry.device_path,
|
device_path: entry.device_path,
|
||||||
file_name: entry.file_name,
|
file_name: entry.file_name,
|
||||||
local_path: localPath,
|
local_path: localPath,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
role: entry.role,
|
role: entry.role,
|
||||||
role_basis: "file_name",
|
role_basis: "file_name",
|
||||||
});
|
});
|
||||||
} catch (cause: unknown) {
|
} catch {
|
||||||
if (cause instanceof AnalysisCancelledError)
|
|
||||||
return cancelledPackagePull(
|
|
||||||
cause,
|
|
||||||
localPath,
|
|
||||||
entry.device_path,
|
|
||||||
"digest",
|
|
||||||
);
|
|
||||||
failures.push({
|
failures.push({
|
||||||
device_path: entry.device_path,
|
device_path: entry.device_path,
|
||||||
stage: "digest",
|
stage: "digest",
|
||||||
@@ -1336,36 +1094,14 @@ export class AdbProvider {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (artifacts.length === 0 && failures.length > 0) {
|
if (artifacts.length === 0 && failures.length > 0) {
|
||||||
const primary = new AnalysisProtocolError(
|
await rm(packageDirectory, { recursive: true, force: true }).catch(
|
||||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
() => undefined,
|
||||||
|
);
|
||||||
|
return err(
|
||||||
|
new AnalysisProtocolError(
|
||||||
|
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
const owner: PendingPackageDirectoryCleanup = {
|
|
||||||
path: packageDirectory,
|
|
||||||
identity: packageDirectoryIdentity,
|
|
||||||
primary,
|
|
||||||
remove: () =>
|
|
||||||
removeOwnedPackageDirectory(
|
|
||||||
packageDirectory,
|
|
||||||
packageDirectoryIdentity,
|
|
||||||
),
|
|
||||||
};
|
|
||||||
try {
|
|
||||||
await owner.remove();
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
|
||||||
const cleanup = new ProviderCleanupError(
|
|
||||||
ADB_PROVIDER_IDENTITY.id,
|
|
||||||
[packageDirectory],
|
|
||||||
{
|
|
||||||
reason: cause instanceof Error ? cause.message : String(cause),
|
|
||||||
},
|
|
||||||
{ operation: operation, cause },
|
|
||||||
);
|
|
||||||
return err(
|
|
||||||
analysisErrorWithCleanupFailure(primary, cleanup, operation),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
return err(primary);
|
|
||||||
}
|
}
|
||||||
return ok(
|
return ok(
|
||||||
createAnalysisExecution(
|
createAnalysisExecution(
|
||||||
@@ -1420,7 +1156,8 @@ export class AdbProvider {
|
|||||||
);
|
);
|
||||||
if (!pulled.ok) return pulled;
|
if (!pulled.ok) return pulled;
|
||||||
try {
|
try {
|
||||||
const digest = await hashAdbFile(localPath, operation, signal);
|
const info = await stat(localPath);
|
||||||
|
const sha256 = await sha256File(localPath);
|
||||||
return ok(
|
return ok(
|
||||||
createAnalysisExecution(
|
createAnalysisExecution(
|
||||||
{
|
{
|
||||||
@@ -1428,8 +1165,8 @@ export class AdbProvider {
|
|||||||
serial: input.serial,
|
serial: input.serial,
|
||||||
device_path: input.device_path,
|
device_path: input.device_path,
|
||||||
local_path: localPath,
|
local_path: localPath,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
},
|
},
|
||||||
provider,
|
provider,
|
||||||
{
|
{
|
||||||
@@ -1440,20 +1177,6 @@ export class AdbProvider {
|
|||||||
),
|
),
|
||||||
);
|
);
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
if (cause instanceof AnalysisCancelledError) {
|
|
||||||
cause.retainPartialObservation({
|
|
||||||
provider_id: "adb",
|
|
||||||
operation,
|
|
||||||
result: {
|
|
||||||
client,
|
|
||||||
serial: input.serial,
|
|
||||||
local_path: localPath,
|
|
||||||
device_path: input.device_path,
|
|
||||||
digest_status: "unknown",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
return err(cause);
|
|
||||||
}
|
|
||||||
return err(
|
return err(
|
||||||
new AnalysisOutputError(
|
new AnalysisOutputError(
|
||||||
operation,
|
operation,
|
||||||
@@ -1488,13 +1211,7 @@ export class AdbProvider {
|
|||||||
cause: new Error(`The local file does not exist: ${localPath}`),
|
cause: new Error(`The local file does not exist: ${localPath}`),
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
const sha256 = await sha256File(localPath);
|
||||||
try {
|
|
||||||
digest = await hashAdbFile(localPath, operation, signal);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
const existsProbe = await runAdb(
|
const existsProbe = await runAdb(
|
||||||
operation,
|
operation,
|
||||||
input.device_path,
|
input.device_path,
|
||||||
@@ -1540,7 +1257,7 @@ export class AdbProvider {
|
|||||||
);
|
);
|
||||||
if (deviceDigest.ok) {
|
if (deviceDigest.ok) {
|
||||||
const digestMatch = /^([a-f0-9]{64})\s/u.exec(deviceDigest.value.stdout);
|
const digestMatch = /^([a-f0-9]{64})\s/u.exec(deviceDigest.value.stdout);
|
||||||
if (digestMatch !== null && digestMatch[1] !== digest.sha256)
|
if (digestMatch !== null && digestMatch[1] !== sha256)
|
||||||
return err(
|
return err(
|
||||||
new AnalysisArtifactChangedError(
|
new AnalysisArtifactChangedError(
|
||||||
operation,
|
operation,
|
||||||
@@ -1555,8 +1272,8 @@ export class AdbProvider {
|
|||||||
serial: input.serial,
|
serial: input.serial,
|
||||||
local_path: localPath,
|
local_path: localPath,
|
||||||
device_path: input.device_path,
|
device_path: input.device_path,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
device_sha256: digestMatch?.[1] ?? null,
|
device_sha256: digestMatch?.[1] ?? null,
|
||||||
overwritten: deviceExists,
|
overwritten: deviceExists,
|
||||||
device_digest_source: "device_sha256sum",
|
device_digest_source: "device_sha256sum",
|
||||||
@@ -1577,8 +1294,8 @@ export class AdbProvider {
|
|||||||
serial: input.serial,
|
serial: input.serial,
|
||||||
local_path: localPath,
|
local_path: localPath,
|
||||||
device_path: input.device_path,
|
device_path: input.device_path,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
device_sha256: null,
|
device_sha256: null,
|
||||||
overwritten: deviceExists,
|
overwritten: deviceExists,
|
||||||
device_digest_source: "unavailable",
|
device_digest_source: "unavailable",
|
||||||
@@ -1617,13 +1334,7 @@ export class AdbProvider {
|
|||||||
cause: new Error(`The local APK does not exist: ${apkPath}`),
|
cause: new Error(`The local APK does not exist: ${apkPath}`),
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
let digest: Awaited<ReturnType<typeof hashAdbFile>>;
|
const sha256 = await sha256File(apkPath);
|
||||||
try {
|
|
||||||
digest = await hashAdbFile(apkPath, operation, signal);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof AnalysisCancelledError) return err(cause);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
const installed = await runAdb(
|
const installed = await runAdb(
|
||||||
operation,
|
operation,
|
||||||
input.serial,
|
input.serial,
|
||||||
@@ -1652,8 +1363,8 @@ export class AdbProvider {
|
|||||||
client,
|
client,
|
||||||
serial: input.serial,
|
serial: input.serial,
|
||||||
apk_path: apkPath,
|
apk_path: apkPath,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
replaced: input.replace,
|
replaced: input.replace,
|
||||||
},
|
},
|
||||||
provider,
|
provider,
|
||||||
@@ -2023,15 +1734,16 @@ export class AdbProvider {
|
|||||||
),
|
),
|
||||||
);
|
);
|
||||||
try {
|
try {
|
||||||
const digest = await hashAdbFile(reportedPath, operation, signal);
|
const info = await stat(reportedPath);
|
||||||
|
const sha256 = await sha256File(reportedPath);
|
||||||
return ok(
|
return ok(
|
||||||
createAnalysisExecution(
|
createAnalysisExecution(
|
||||||
{
|
{
|
||||||
client,
|
client,
|
||||||
serial: input.serial,
|
serial: input.serial,
|
||||||
local_path: reportedPath,
|
local_path: reportedPath,
|
||||||
bytes: digest.bytes,
|
bytes: info.size,
|
||||||
sha256: digest.sha256,
|
sha256,
|
||||||
adb_reported_path: reportedPath,
|
adb_reported_path: reportedPath,
|
||||||
},
|
},
|
||||||
provider,
|
provider,
|
||||||
@@ -2043,20 +1755,6 @@ export class AdbProvider {
|
|||||||
),
|
),
|
||||||
);
|
);
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
if (cause instanceof AnalysisCancelledError) {
|
|
||||||
cause.retainPartialObservation({
|
|
||||||
provider_id: "adb",
|
|
||||||
operation,
|
|
||||||
result: {
|
|
||||||
client,
|
|
||||||
serial: input.serial,
|
|
||||||
local_path: reportedPath,
|
|
||||||
adb_reported_path: reportedPath,
|
|
||||||
digest_status: "unknown",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
return err(cause);
|
|
||||||
}
|
|
||||||
return err(
|
return err(
|
||||||
new AnalysisOutputError(
|
new AnalysisOutputError(
|
||||||
operation,
|
operation,
|
||||||
|
|||||||
@@ -1,43 +1,40 @@
|
|||||||
|
import { constants, createReadStream } from "node:fs";
|
||||||
|
import { chmod, copyFile } from "node:fs/promises";
|
||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import type { ReadStream, Stats, WriteStream } from "node:fs";
|
|
||||||
import { chmod, open, rm, stat, type FileHandle } from "node:fs/promises";
|
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { Transform, type TransformCallback } from "node:stream";
|
|
||||||
import { pipeline } from "node:stream/promises";
|
|
||||||
import {
|
import {
|
||||||
AnalysisInputError,
|
AnalysisInputError,
|
||||||
AnalysisCapabilityUnavailableError,
|
AnalysisCapabilityUnavailableError,
|
||||||
} from "../domain/analysisErrorCore.js";
|
} from "../domain/analysisErrorCore.js";
|
||||||
import {
|
|
||||||
openRegularFile,
|
|
||||||
sameRegularFileState,
|
|
||||||
} from "../filesystem/RegularFile.js";
|
|
||||||
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
||||||
|
|
||||||
|
/** Stream a local file digest without retaining the APK or JAR in memory. */
|
||||||
|
export const hashAndroidFile = async (path: string): Promise<string> => {
|
||||||
|
const hash = createHash("sha256");
|
||||||
|
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||||
|
return hash.digest("hex");
|
||||||
|
};
|
||||||
|
|
||||||
/** Bind decompiler input to the admitted artifact, even if the original changes. */
|
/** Bind decompiler input to the admitted artifact, even if the original changes. */
|
||||||
export const snapshotAndroidTarget = async (
|
export const snapshotAndroidTarget = async (
|
||||||
path: string,
|
path: string,
|
||||||
sha256: string,
|
sha256: string,
|
||||||
root: string,
|
root: string,
|
||||||
operation: AndroidOperation,
|
operation: AndroidOperation,
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<string> => {
|
): Promise<string> => {
|
||||||
const snapshot = join(root, "target.apk");
|
const snapshot = join(root, "target.apk");
|
||||||
try {
|
await copyFile(path, snapshot, constants.COPYFILE_EXCL);
|
||||||
await copyAndroidSnapshot(path, snapshot, sha256, signal);
|
await chmod(snapshot, 0o400);
|
||||||
signal?.throwIfAborted();
|
if ((await hashAndroidFile(snapshot)) !== sha256)
|
||||||
await chmod(snapshot, 0o400);
|
throw new AnalysisInputError(operation, undefined, [
|
||||||
return snapshot;
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
|
||||||
throw new AnalysisInputError(operation, { cause }, [
|
|
||||||
{
|
{
|
||||||
path: ["path"],
|
path: ["path"],
|
||||||
reason: "invalid_value",
|
reason: "invalid_value",
|
||||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
message:
|
||||||
|
"APK bytes changed after admission; retry against a stable file.",
|
||||||
},
|
},
|
||||||
]);
|
]);
|
||||||
}
|
return snapshot;
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Fingerprint the actual immutable engine bytes executed for this observation. */
|
/** Fingerprint the actual immutable engine bytes executed for this observation. */
|
||||||
@@ -46,120 +43,15 @@ export const snapshotAndroidEngine = async (
|
|||||||
sha256: string,
|
sha256: string,
|
||||||
root: string,
|
root: string,
|
||||||
operation: AndroidOperation,
|
operation: AndroidOperation,
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<{ path: string; sha256: string }> => {
|
): Promise<{ path: string; sha256: string }> => {
|
||||||
const path = join(root, "engine.jar");
|
const path = join(root, "engine.jar");
|
||||||
try {
|
await copyFile(source, path, constants.COPYFILE_EXCL);
|
||||||
const observed = await copyAndroidSnapshot(source, path, sha256, signal);
|
await chmod(path, 0o400);
|
||||||
signal?.throwIfAborted();
|
if ((await hashAndroidFile(path)) !== sha256)
|
||||||
await chmod(path, 0o400);
|
|
||||||
return { path, sha256: observed };
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof AndroidSnapshotChangedError)) throw cause;
|
|
||||||
throw new AnalysisCapabilityUnavailableError(
|
throw new AnalysisCapabilityUnavailableError(
|
||||||
"jadx",
|
"jadx",
|
||||||
operation,
|
operation,
|
||||||
`REA_JADX_MCP_JAR bytes changed during admission at ${source}; retry with a stable engine file.`,
|
"REA_JADX_MCP_JAR bytes changed during admission; retry with a stable engine file.",
|
||||||
{ cause },
|
|
||||||
);
|
);
|
||||||
}
|
return { path, sha256 };
|
||||||
};
|
};
|
||||||
|
|
||||||
const copyAndroidSnapshot = async (
|
|
||||||
sourcePath: string,
|
|
||||||
snapshotPath: string,
|
|
||||||
expectedSha256: string,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<string> => {
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
const source = await openRegularFile(sourcePath, {
|
|
||||||
symlinks: "follow",
|
|
||||||
signal,
|
|
||||||
});
|
|
||||||
let destination: FileHandle | undefined;
|
|
||||||
let inputStream: ReadStream | undefined;
|
|
||||||
let outputStream: WriteStream | undefined;
|
|
||||||
let removeSnapshot = false;
|
|
||||||
let copiedBytes = 0;
|
|
||||||
const hash = createHash("sha256");
|
|
||||||
try {
|
|
||||||
const initial = await source.stat();
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
const digest = new Transform({
|
|
||||||
transform(
|
|
||||||
chunk: Buffer,
|
|
||||||
_encoding: BufferEncoding,
|
|
||||||
callback: TransformCallback,
|
|
||||||
) {
|
|
||||||
copiedBytes += chunk.length;
|
|
||||||
if (copiedBytes > initial.size) {
|
|
||||||
callback(
|
|
||||||
new AndroidSnapshotChangedError(
|
|
||||||
`Android input grew while being copied: ${sourcePath}`,
|
|
||||||
),
|
|
||||||
);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
hash.update(chunk);
|
|
||||||
callback(null, chunk);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
destination = await open(snapshotPath, "wx", 0o600);
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
inputStream = source.createReadStream({
|
|
||||||
start: 0,
|
|
||||||
end: initial.size,
|
|
||||||
autoClose: false,
|
|
||||||
});
|
|
||||||
outputStream = destination.createWriteStream({ autoClose: false });
|
|
||||||
await pipeline(inputStream, digest, outputStream, { signal });
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
await verifySourceState(source, sourcePath, initial, copiedBytes);
|
|
||||||
const observedSha256 = hash.digest("hex");
|
|
||||||
if (observedSha256 !== expectedSha256)
|
|
||||||
throw new AndroidSnapshotChangedError(
|
|
||||||
`Android snapshot digest mismatch for ${sourcePath}`,
|
|
||||||
);
|
|
||||||
return observedSha256;
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
removeSnapshot = destination !== undefined;
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
} finally {
|
|
||||||
// Non-auto-closing streams retain a reference to their FileHandle even
|
|
||||||
// after pipeline completion. Release those references before awaiting
|
|
||||||
// descriptor closure, after the admitted source state has been verified.
|
|
||||||
inputStream?.destroy();
|
|
||||||
outputStream?.destroy();
|
|
||||||
try {
|
|
||||||
await Promise.all([source.close(), destination?.close()]);
|
|
||||||
} finally {
|
|
||||||
if (removeSnapshot)
|
|
||||||
await rm(snapshotPath, { force: true }).catch(() => undefined);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const verifySourceState = async (
|
|
||||||
handle: Awaited<ReturnType<typeof openRegularFile>>,
|
|
||||||
path: string,
|
|
||||||
initial: Stats,
|
|
||||||
observedBytes: number,
|
|
||||||
): Promise<void> => {
|
|
||||||
const [opened, currentPath] = await Promise.all([handle.stat(), stat(path)]);
|
|
||||||
if (
|
|
||||||
observedBytes !== initial.size ||
|
|
||||||
!sameRegularFileState(initial, opened) ||
|
|
||||||
!sameRegularFileState(initial, currentPath)
|
|
||||||
)
|
|
||||||
throw new AndroidSnapshotChangedError(
|
|
||||||
`Android input changed while being read: ${path}`,
|
|
||||||
);
|
|
||||||
};
|
|
||||||
|
|
||||||
class AndroidSnapshotChangedError extends Error {
|
|
||||||
constructor(message: string, options?: ErrorOptions) {
|
|
||||||
super(message, options);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -52,11 +52,7 @@ export const inspectJadxAvailability = async (
|
|||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
): Promise<ProviderAvailability> => {
|
): Promise<ProviderAvailability> => {
|
||||||
try {
|
try {
|
||||||
const configuration = await readJadxConfigurationInputs(
|
const configuration = await readJadxConfiguration(environment, signal);
|
||||||
environment,
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
|
||||||
return {
|
return {
|
||||||
status: "available",
|
status: "available",
|
||||||
code: null,
|
code: null,
|
||||||
@@ -81,83 +77,52 @@ export const inspectJadxAvailability = async (
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Resolve caller-selected paths and JVM options without probing their contents. */
|
/** Resolve caller-selected tools and preserve the same prerequisite diagnostics. */
|
||||||
export const resolveJadxConfigurationInputs = async (
|
export const resolveJadxConfiguration = async (
|
||||||
environment: Readonly<Record<string, string | undefined>>,
|
environment: Readonly<Record<string, string | undefined>>,
|
||||||
operation: AndroidOperation,
|
operation: AndroidOperation,
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
): Promise<JadxConfiguration> => {
|
): Promise<JadxConfiguration> => {
|
||||||
try {
|
try {
|
||||||
return await readJadxConfigurationInputs(environment, signal);
|
return await readJadxConfiguration(environment, signal);
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
if (signal?.aborted === true) throw cause;
|
if (signal?.aborted === true) throw cause;
|
||||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
||||||
throw projectConfigurationFailure(operation, cause);
|
const output = cause.diagnostics;
|
||||||
|
throw new AnalysisCapabilityUnavailableError(
|
||||||
|
"jadx",
|
||||||
|
operation,
|
||||||
|
cause.message,
|
||||||
|
{
|
||||||
|
cause,
|
||||||
|
userMessage: cause.message,
|
||||||
|
...(typeof output.stdout === "string" &&
|
||||||
|
typeof output.stderr === "string"
|
||||||
|
? {
|
||||||
|
capturedOutput: {
|
||||||
|
stdout: output.stdout,
|
||||||
|
stderr: output.stderr,
|
||||||
|
truncated: output.output_truncated === true,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
: {}),
|
||||||
|
},
|
||||||
|
);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Probe Java and archive prerequisites before starting a new engine session. */
|
const readJadxConfiguration = async (
|
||||||
export const validateJadxConfiguration = async (
|
|
||||||
configuration: JadxConfiguration,
|
|
||||||
environment: Readonly<Record<string, string | undefined>>,
|
|
||||||
operation: AndroidOperation,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<void> => {
|
|
||||||
try {
|
|
||||||
await validateJadxConfigurationContents(configuration, environment, signal);
|
|
||||||
} catch (cause) {
|
|
||||||
if (signal?.aborted === true) throw cause;
|
|
||||||
if (!(cause instanceof JadxConfigurationFailure)) throw cause;
|
|
||||||
throw projectConfigurationFailure(operation, cause);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const readJadxConfigurationInputs = async (
|
|
||||||
environment: Readonly<Record<string, string | undefined>>,
|
environment: Readonly<Record<string, string | undefined>>,
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
): Promise<JadxConfiguration> => {
|
): Promise<JadxConfiguration> => {
|
||||||
signal?.throwIfAborted();
|
|
||||||
requireSupportedHost();
|
requireSupportedHost();
|
||||||
const jar = await resolveJar(environment);
|
const jar = await resolveJar(environment);
|
||||||
const java = await resolveJava(environment, signal);
|
const java = await resolveJava(environment, signal);
|
||||||
await requireJavaHomeExecutable(environment.JAVA_HOME, java);
|
await requireJavaHomeExecutable(environment.JAVA_HOME, java);
|
||||||
signal?.throwIfAborted();
|
const jvmArguments = resolveJvmArguments(environment);
|
||||||
return { jar, java, jvmArguments: resolveJvmArguments(environment) };
|
await inspectJavaRuntime(java, environment, signal);
|
||||||
};
|
await requireEngineArchive(jar, signal);
|
||||||
|
return { jar, java, jvmArguments };
|
||||||
const validateJadxConfigurationContents = async (
|
|
||||||
configuration: JadxConfiguration,
|
|
||||||
environment: Readonly<Record<string, string | undefined>>,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<void> => {
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
await inspectJavaRuntime(configuration.java, environment, signal);
|
|
||||||
await requireEngineArchive(configuration.jar, signal);
|
|
||||||
};
|
|
||||||
|
|
||||||
const projectConfigurationFailure = (
|
|
||||||
operation: AndroidOperation,
|
|
||||||
cause: JadxConfigurationFailure,
|
|
||||||
): AnalysisCapabilityUnavailableError => {
|
|
||||||
const output = cause.diagnostics;
|
|
||||||
return new AnalysisCapabilityUnavailableError(
|
|
||||||
"jadx",
|
|
||||||
operation,
|
|
||||||
cause.message,
|
|
||||||
{
|
|
||||||
cause,
|
|
||||||
userMessage: cause.message,
|
|
||||||
...(typeof output.stdout === "string" && typeof output.stderr === "string"
|
|
||||||
? {
|
|
||||||
capturedOutput: {
|
|
||||||
stdout: output.stdout,
|
|
||||||
stderr: output.stderr,
|
|
||||||
truncated: output.output_truncated === true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
: {}),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const requireEngineArchive = async (
|
const requireEngineArchive = async (
|
||||||
|
|||||||
+28
-98
@@ -21,14 +21,12 @@ import { err, ok, type Result } from "../domain/result.js";
|
|||||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||||
import {
|
import {
|
||||||
snapshotAndroidEngine,
|
snapshotAndroidEngine,
|
||||||
|
hashAndroidFile,
|
||||||
snapshotAndroidTarget,
|
snapshotAndroidTarget,
|
||||||
} from "./AndroidTargetSnapshot.js";
|
} from "./AndroidTargetSnapshot.js";
|
||||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
|
||||||
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
|
|
||||||
import {
|
import {
|
||||||
inspectJadxAvailability,
|
inspectJadxAvailability,
|
||||||
resolveJadxConfigurationInputs,
|
resolveJadxConfiguration,
|
||||||
validateJadxConfiguration,
|
|
||||||
} from "./JadxConfiguration.js";
|
} from "./JadxConfiguration.js";
|
||||||
import type { JadxLauncher } from "./JadxMcpTransport.js";
|
import type { JadxLauncher } from "./JadxMcpTransport.js";
|
||||||
|
|
||||||
@@ -106,50 +104,6 @@ const executionError = (context: {
|
|||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
const androidTargetChanged = (
|
|
||||||
operation: string,
|
|
||||||
path: string,
|
|
||||||
cause?: unknown,
|
|
||||||
): AnalysisInputError =>
|
|
||||||
new AnalysisInputError(
|
|
||||||
operation,
|
|
||||||
cause === undefined ? undefined : { cause },
|
|
||||||
[
|
|
||||||
{
|
|
||||||
path: ["path"],
|
|
||||||
reason: "invalid_value",
|
|
||||||
message: `APK bytes changed after admission at ${path}; retry against a stable file.`,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
);
|
|
||||||
|
|
||||||
const androidSourceChanged = (
|
|
||||||
operation: string,
|
|
||||||
source: string,
|
|
||||||
cause?: unknown,
|
|
||||||
): AnalysisCapabilityUnavailableError =>
|
|
||||||
new AnalysisCapabilityUnavailableError(
|
|
||||||
"jadx",
|
|
||||||
operation,
|
|
||||||
`Selected JADX input bytes changed during admission at ${source}; retry with a stable installation.`,
|
|
||||||
cause === undefined ? undefined : { cause },
|
|
||||||
);
|
|
||||||
|
|
||||||
const hashAndroidSource = async (
|
|
||||||
path: string,
|
|
||||||
operation: string,
|
|
||||||
signal: AbortSignal,
|
|
||||||
): Promise<string> => {
|
|
||||||
try {
|
|
||||||
return (await hashStableFile(path, signal)).sha256;
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (signal.aborted) throw cause;
|
|
||||||
if (cause instanceof RegularFileChangedError)
|
|
||||||
throw androidSourceChanged(operation, path, cause);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
interface PendingCleanup {
|
interface PendingCleanup {
|
||||||
session: JadxSession | undefined;
|
session: JadxSession | undefined;
|
||||||
root: PrivateRuntimeRoot | undefined;
|
root: PrivateRuntimeRoot | undefined;
|
||||||
@@ -193,18 +147,17 @@ const cleanup = async (
|
|||||||
await root?.close();
|
await root?.close();
|
||||||
resources.root = undefined;
|
resources.root = undefined;
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
const error = new ProviderCleanupError(
|
return err(
|
||||||
"jadx",
|
new ProviderCleanupError(
|
||||||
[root?.path ?? "unknown"],
|
"jadx",
|
||||||
{
|
[root?.path ?? "unknown"],
|
||||||
...diagnostics,
|
{
|
||||||
reason: cause instanceof Error ? cause.message : String(cause),
|
...diagnostics,
|
||||||
},
|
reason: cause instanceof Error ? cause.message : String(cause),
|
||||||
{ cause: previousError ?? cause },
|
},
|
||||||
|
{ cause: previousError ?? cause },
|
||||||
|
),
|
||||||
);
|
);
|
||||||
if (previousError?.partialObservation !== undefined)
|
|
||||||
error.retainPartialObservation(previousError.partialObservation);
|
|
||||||
return err(error);
|
|
||||||
}
|
}
|
||||||
return ok(undefined);
|
return ok(undefined);
|
||||||
};
|
};
|
||||||
@@ -334,33 +287,23 @@ export class JadxProvider implements AndroidAnalysisPort {
|
|||||||
request.operation,
|
request.operation,
|
||||||
`Target ${target.path} is ${target.format}; provide one standalone APK.`,
|
`Target ${target.path} is ${target.format}; provide one standalone APK.`,
|
||||||
);
|
);
|
||||||
const configuration = await resolveJadxConfigurationInputs(
|
const configuration = await resolveJadxConfiguration(
|
||||||
this.environment,
|
this.environment,
|
||||||
request.operation,
|
request.operation,
|
||||||
signal,
|
signal,
|
||||||
);
|
);
|
||||||
signal.throwIfAborted();
|
signal.throwIfAborted();
|
||||||
const jarHash = await hashAndroidSource(
|
const jarHash = await hashAndroidFile(configuration.jar);
|
||||||
configuration.jar,
|
const bridgeHash = await hashAndroidFile(BRIDGE_SOURCE);
|
||||||
request.operation,
|
if ((await hashAndroidFile(target.path)) !== target.sha256)
|
||||||
signal,
|
throw new AnalysisInputError(request.operation, undefined, [
|
||||||
);
|
{
|
||||||
const bridgeHash = await hashAndroidSource(
|
path: ["path"],
|
||||||
BRIDGE_SOURCE,
|
reason: "invalid_value",
|
||||||
request.operation,
|
message:
|
||||||
signal,
|
"APK bytes changed after admission; retry against a stable file.",
|
||||||
);
|
},
|
||||||
let targetHash: string;
|
]);
|
||||||
try {
|
|
||||||
targetHash = (await hashStableFile(target.path, signal)).sha256;
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (signal.aborted) throw cause;
|
|
||||||
if (cause instanceof RegularFileChangedError)
|
|
||||||
throw androidTargetChanged(request.operation, target.path, cause);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
if (targetHash !== target.sha256)
|
|
||||||
throw androidTargetChanged(request.operation, target.path);
|
|
||||||
const key = JSON.stringify({
|
const key = JSON.stringify({
|
||||||
path: target.path,
|
path: target.path,
|
||||||
sha256: target.sha256,
|
sha256: target.sha256,
|
||||||
@@ -383,40 +326,27 @@ export class JadxProvider implements AndroidAnalysisPort {
|
|||||||
}
|
}
|
||||||
let retained = this.#retained;
|
let retained = this.#retained;
|
||||||
if (retained === undefined) {
|
if (retained === undefined) {
|
||||||
await validateJadxConfiguration(
|
|
||||||
configuration,
|
|
||||||
this.environment,
|
|
||||||
request.operation,
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" });
|
root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" });
|
||||||
const engine = await snapshotAndroidEngine(
|
const engine = await snapshotAndroidEngine(
|
||||||
configuration.jar,
|
configuration.jar,
|
||||||
jarHash,
|
jarHash,
|
||||||
root.path,
|
root.path,
|
||||||
request.operation,
|
request.operation,
|
||||||
signal,
|
|
||||||
);
|
);
|
||||||
const snapshot = await snapshotAndroidTarget(
|
const snapshot = await snapshotAndroidTarget(
|
||||||
target.path,
|
target.path,
|
||||||
target.sha256,
|
target.sha256,
|
||||||
root.path,
|
root.path,
|
||||||
request.operation,
|
request.operation,
|
||||||
signal,
|
|
||||||
);
|
);
|
||||||
const bridge = join(root.path, "ReaJadxBridge.java");
|
const bridge = join(root.path, "ReaJadxBridge.java");
|
||||||
await copyFile(BRIDGE_SOURCE, bridge);
|
await copyFile(BRIDGE_SOURCE, bridge);
|
||||||
await chmod(bridge, 0o400);
|
await chmod(bridge, 0o400);
|
||||||
const copiedBridgeHash = await hashAndroidSource(
|
if ((await hashAndroidFile(bridge)) !== bridgeHash)
|
||||||
bridge,
|
throw new AnalysisCapabilityUnavailableError(
|
||||||
request.operation,
|
"jadx",
|
||||||
signal,
|
|
||||||
);
|
|
||||||
if (copiedBridgeHash !== bridgeHash)
|
|
||||||
throw androidSourceChanged(
|
|
||||||
request.operation,
|
request.operation,
|
||||||
BRIDGE_SOURCE,
|
"REA Android bridge bytes changed during admission; retry with a stable installation.",
|
||||||
undefined,
|
|
||||||
);
|
);
|
||||||
signal.throwIfAborted();
|
signal.throwIfAborted();
|
||||||
session = new JadxSession(
|
session = new JadxSession(
|
||||||
|
|||||||
@@ -373,13 +373,7 @@ const readExecutableMetadata = async (
|
|||||||
if (!abiFlags.ok) return abiFlags;
|
if (!abiFlags.ok) return abiFlags;
|
||||||
return ok({
|
return ok({
|
||||||
...parsed.value,
|
...parsed.value,
|
||||||
mips: {
|
mips: { ...parsed.value.mips, abiFlags: abiFlags.value },
|
||||||
...parsed.value.mips,
|
|
||||||
abiFlags: abiFlags.value.abiFlags,
|
|
||||||
...(abiFlags.value.limitations.length === 0
|
|
||||||
? {}
|
|
||||||
: { limitations: abiFlags.value.limitations }),
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -1,22 +1,17 @@
|
|||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import {
|
import { readFile } from "node:fs/promises";
|
||||||
RegularFileCleanupFailure,
|
|
||||||
readRegularFileText,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
import {
|
import {
|
||||||
effectiveClientServer,
|
effectiveClientServer,
|
||||||
parseClientConfiguration,
|
parseClientConfiguration,
|
||||||
type ClientConfigurationDocument,
|
type ClientConfigurationDocument,
|
||||||
} from "./ClientConfigurationDocument.js";
|
} from "./ClientConfigurationDocument.js";
|
||||||
import type { SetupClient } from "./SupportedClients.js";
|
import type { SetupClient } from "./SupportedClients.js";
|
||||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
|
||||||
|
|
||||||
/** A named source failure that setup can explain without losing the bad path. */
|
/** A named source failure that setup can explain without losing the bad path. */
|
||||||
export interface ClientConfigurationFileError {
|
export interface ClientConfigurationFileError {
|
||||||
readonly kind: "unreadable" | "malformed";
|
readonly kind: "unreadable" | "malformed";
|
||||||
readonly path: string;
|
readonly path: string;
|
||||||
readonly detail: string;
|
readonly detail: string;
|
||||||
readonly cleanupFailure?: RegularFileCleanupFailure;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Preserve named read failures and distinguish a missing source from an empty file. */
|
/** Preserve named read failures and distinguish a missing source from an empty file. */
|
||||||
@@ -24,7 +19,7 @@ export const readConfigurationText = async (
|
|||||||
path: string,
|
path: string,
|
||||||
): Promise<Result<string | undefined, ClientConfigurationFileError>> => {
|
): Promise<Result<string | undefined, ClientConfigurationFileError>> => {
|
||||||
try {
|
try {
|
||||||
return ok(await readRegularFileText(path));
|
return ok(await readFile(path, "utf8"));
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
|
||||||
return ok(undefined);
|
return ok(undefined);
|
||||||
@@ -32,9 +27,6 @@ export const readConfigurationText = async (
|
|||||||
kind: "unreadable",
|
kind: "unreadable",
|
||||||
path,
|
path,
|
||||||
detail: `Configuration ${path} could not be read: ${cause instanceof Error ? cause.message : String(cause)}`,
|
detail: `Configuration ${path} could not be read: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||||
...(cause instanceof RegularFileCleanupFailure
|
|
||||||
? { cleanupFailure: cause }
|
|
||||||
: {}),
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
@@ -71,16 +63,7 @@ export const readClientConfigurationFiles = async (
|
|||||||
for (const path of client.configPaths ?? [client.configPath]) {
|
for (const path of client.configPaths ?? [client.configPath]) {
|
||||||
const document = await readConfigurationFile(path, client.format);
|
const document = await readConfigurationFile(path, client.format);
|
||||||
if (!document.ok) return document;
|
if (!document.ok) return document;
|
||||||
if (document.value !== undefined) {
|
if (document.value !== undefined) documents.push(document.value);
|
||||||
const environment = clientRegistrationEnvironment(document.value);
|
|
||||||
if (!environment.ok)
|
|
||||||
return err({
|
|
||||||
kind: "malformed",
|
|
||||||
path,
|
|
||||||
detail: `Configuration ${path} has an invalid REA server environment: ${environment.error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
|
|
||||||
});
|
|
||||||
documents.push(document.value);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return ok(documents);
|
return ok(documents);
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,54 +0,0 @@
|
|||||||
import { z } from "zod";
|
|
||||||
|
|
||||||
import { parseEnvironment } from "../config/environment.js";
|
|
||||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
|
||||||
import {
|
|
||||||
effectiveClientServer,
|
|
||||||
type ClientConfigurationDocument,
|
|
||||||
} from "./ClientConfigurationDocument.js";
|
|
||||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
|
||||||
|
|
||||||
/** Read explicit REA overrides without dropping peers when one value is invalid. */
|
|
||||||
export const clientRegistrationEnvironment = (
|
|
||||||
parsed: ClientConfigurationDocument | undefined,
|
|
||||||
overrides: Readonly<Record<string, string>> = {},
|
|
||||||
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
|
|
||||||
const key =
|
|
||||||
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
|
|
||||||
? "environment"
|
|
||||||
: "env";
|
|
||||||
const registration =
|
|
||||||
parsed === undefined
|
|
||||||
? undefined
|
|
||||||
: effectiveClientServer(parsed, PRODUCT_IDENTITY.mcpServerKey);
|
|
||||||
const raw =
|
|
||||||
typeof registration === "object" && registration !== null
|
|
||||||
? Reflect.get(registration, key)
|
|
||||||
: undefined;
|
|
||||||
const existing = z
|
|
||||||
.record(z.string(), z.string())
|
|
||||||
.safeParse(raw === undefined ? {} : raw);
|
|
||||||
if (!existing.success)
|
|
||||||
return err(
|
|
||||||
new ConfigurationError("Invalid REA server environment", {
|
|
||||||
settings: existing.error.issues.map((issue) => ({
|
|
||||||
setting: [key, ...issue.path].join("."),
|
|
||||||
constraint:
|
|
||||||
"Server environment must be an object containing only string values",
|
|
||||||
})),
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
// Validate known settings with the runtime's parser, but retain the original
|
|
||||||
// string overrides, including unknown client-specific keys and no defaults.
|
|
||||||
const environment = { ...existing.data, ...overrides };
|
|
||||||
const validated = parseEnvironment(environment);
|
|
||||||
if (!validated.ok) return validated;
|
|
||||||
return ok(
|
|
||||||
Object.fromEntries(
|
|
||||||
Object.entries(environment).sort(([left], [right]) =>
|
|
||||||
left.localeCompare(right),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
};
|
|
||||||
@@ -5,22 +5,27 @@ import type { JsonValue } from "../domain/jsonValue.js";
|
|||||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import { compareBundles } from "../domain/bundleComparison.js";
|
import { compareBundles } from "../domain/bundleComparison.js";
|
||||||
|
import { EvidenceLedger } from "./investigation/EvidenceLedger.js";
|
||||||
import {
|
import {
|
||||||
readEvidenceBundle,
|
readEvidenceBundle,
|
||||||
writeEvidenceBundle,
|
writeEvidenceBundle,
|
||||||
} from "./EvidenceBundleFiles.js";
|
} from "./EvidenceBundleFiles.js";
|
||||||
|
|
||||||
/** Validate a standalone bundle and report its complete record/revision counts. */
|
/** Validate and merge one bundle using the session Evidence ledger. */
|
||||||
export const importEvidenceBundleCommand = async (
|
export const importEvidenceBundleCommand = async (
|
||||||
path: string,
|
path: string,
|
||||||
): Promise<Result<JsonValue, AnalysisError>> => {
|
): Promise<Result<JsonValue, AnalysisError>> => {
|
||||||
const loaded = await readEvidenceBundle(path);
|
const loaded = await readEvidenceBundle(path);
|
||||||
if (!loaded.ok) return loaded;
|
if (!loaded.ok) return loaded;
|
||||||
return ok({
|
const ledger = createLedger();
|
||||||
imported: loaded.value.records.length,
|
const imported = ledger.import(loaded.value);
|
||||||
unknowns_added: loaded.value.unknowns.length,
|
return imported.ok
|
||||||
total: loaded.value.records.length,
|
? ok({
|
||||||
});
|
imported: imported.value.recordsAdded,
|
||||||
|
unknowns_added: imported.value.unknownsAdded,
|
||||||
|
total: ledger.export().records.length,
|
||||||
|
})
|
||||||
|
: imported;
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Validate a source bundle and atomically export canonical bytes. */
|
/** Validate a source bundle and atomically export canonical bytes. */
|
||||||
@@ -59,6 +64,8 @@ export const compareEvidenceBundlesCommand = async (input: {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const createLedger = (): EvidenceLedger => new EvidenceLedger();
|
||||||
|
|
||||||
const projectWrite = (
|
const projectWrite = (
|
||||||
bundle: EvidenceBundle,
|
bundle: EvidenceBundle,
|
||||||
written: Awaited<ReturnType<typeof writeEvidenceBundle>>,
|
written: Awaited<ReturnType<typeof writeEvidenceBundle>>,
|
||||||
|
|||||||
@@ -16,7 +16,8 @@ import {
|
|||||||
AnalysisResourceConstraintError,
|
AnalysisResourceConstraintError,
|
||||||
} from "../domain/analysisErrorCore.js";
|
} from "../domain/analysisErrorCore.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import { readJsonInputFile } from "./JsonInputFile.js";
|
import { parseUtf8Json } from "./Utf8JsonInput.js";
|
||||||
|
import { readRegularFile } from "./RegularFileRead.js";
|
||||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||||
|
|
||||||
/** Request control and its owning operation for an interruptible atomic write. */
|
/** Request control and its owning operation for an interruptible atomic write. */
|
||||||
@@ -36,20 +37,12 @@ export const readJsonFile = async (
|
|||||||
> => {
|
> => {
|
||||||
const requestedPath = resolve(path);
|
const requestedPath = resolve(path);
|
||||||
try {
|
try {
|
||||||
const decoded = await readJsonInputFile(
|
const encoded = await readRegularFile(requestedPath);
|
||||||
requestedPath,
|
const decoded = parseUtf8Json(encoded, "read_evidence_file", requestedPath);
|
||||||
"read_evidence_file",
|
|
||||||
);
|
|
||||||
if (!decoded.ok) {
|
if (!decoded.ok) {
|
||||||
if (decoded.error instanceof AnalysisResourceConstraintError)
|
|
||||||
return err(decoded.error);
|
|
||||||
return err(
|
return err(
|
||||||
new EvidenceFileError("read", "invalid-json", {
|
new EvidenceFileError("read", "invalid-json", {
|
||||||
cause:
|
cause: decoded.cause ?? new SyntaxError(decoded.error),
|
||||||
decoded.error.cause ??
|
|
||||||
new SyntaxError(
|
|
||||||
decoded.error.issues[0]?.message ?? decoded.error.message,
|
|
||||||
),
|
|
||||||
path: requestedPath,
|
path: requestedPath,
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -5,12 +5,6 @@ import type {
|
|||||||
} from "../domain/binaryTargetTypes.js";
|
} from "../domain/binaryTargetTypes.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
|
|
||||||
/** Observed ABI declarations and coverage of their source tables. */
|
|
||||||
export interface MipsAbiFlagsInspection {
|
|
||||||
readonly abiFlags: MipsAbiFlags | null;
|
|
||||||
readonly limitations: readonly string[];
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Read ELF32 MIPS ABI declarations from the caller's already-open file.
|
* Read ELF32 MIPS ABI declarations from the caller's already-open file.
|
||||||
* Inspect both typed tables (not section names), including records beyond the
|
* Inspect both typed tables (not section names), including records beyond the
|
||||||
@@ -21,7 +15,7 @@ export const readMipsElfAbiFlags = async (
|
|||||||
handle: FileHandle,
|
handle: FileHandle,
|
||||||
metadata: MipsElfMetadata,
|
metadata: MipsElfMetadata,
|
||||||
checkCancelled: () => void,
|
checkCancelled: () => void,
|
||||||
): Promise<Result<MipsAbiFlagsInspection, string>> => {
|
): Promise<Result<MipsAbiFlags | null, string>> => {
|
||||||
checkCancelled();
|
checkCancelled();
|
||||||
if (metadata.elfClass !== 32)
|
if (metadata.elfClass !== 32)
|
||||||
return err("MIPS ABI record inspection currently supports ELF32 only");
|
return err("MIPS ABI record inspection currently supports ELF32 only");
|
||||||
@@ -87,7 +81,6 @@ export const readMipsElfAbiFlags = async (
|
|||||||
dataOffset: 4,
|
dataOffset: 4,
|
||||||
sizeOffset: 16,
|
sizeOffset: 16,
|
||||||
recordType: 0x70000003, // PT_MIPS_ABIFLAGS
|
recordType: 0x70000003, // PT_MIPS_ABIFLAGS
|
||||||
optional: false,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
label: "section header table",
|
label: "section header table",
|
||||||
@@ -99,7 +92,6 @@ export const readMipsElfAbiFlags = async (
|
|||||||
dataOffset: 16,
|
dataOffset: 16,
|
||||||
sizeOffset: 20,
|
sizeOffset: 20,
|
||||||
recordType: 0x7000002a, // SHT_MIPS_ABIFLAGS
|
recordType: 0x7000002a, // SHT_MIPS_ABIFLAGS
|
||||||
optional: true,
|
|
||||||
},
|
},
|
||||||
];
|
];
|
||||||
// Extended numbering needs a separate section-zero reader. Refuse it rather
|
// Extended numbering needs a separate section-zero reader. Refuse it rather
|
||||||
@@ -107,7 +99,6 @@ export const readMipsElfAbiFlags = async (
|
|||||||
if (u16(bytes, 44) === 0xffff || u16(bytes, 48) >= 0xff00)
|
if (u16(bytes, 44) === 0xffff || u16(bytes, 48) >= 0xff00)
|
||||||
return err("extended or reserved MIPS ELF table numbering is unsupported");
|
return err("extended or reserved MIPS ELF table numbering is unsupported");
|
||||||
let record: Buffer | undefined;
|
let record: Buffer | undefined;
|
||||||
const limitations: string[] = [];
|
|
||||||
for (const table of tables) {
|
for (const table of tables) {
|
||||||
if (table.count === 0) {
|
if (table.count === 0) {
|
||||||
if (table.offset !== 0)
|
if (table.offset !== 0)
|
||||||
@@ -116,27 +107,16 @@ export const readMipsElfAbiFlags = async (
|
|||||||
}
|
}
|
||||||
if (table.offset < 52 || table.stride !== table.entrySize)
|
if (table.offset < 52 || table.stride !== table.entrySize)
|
||||||
return err(`invalid MIPS ${table.label} offset or entry size`);
|
return err(`invalid MIPS ${table.label} offset or entry size`);
|
||||||
const count = table.optional
|
|
||||||
? Math.min(
|
|
||||||
table.count,
|
|
||||||
Math.max(0, Math.floor((fileSize - table.offset) / table.entrySize)),
|
|
||||||
)
|
|
||||||
: table.count;
|
|
||||||
if (count < table.count)
|
|
||||||
limitations.push(
|
|
||||||
`MIPS ${table.label} at file offset ${table.offset} extends beyond the captured file: examined ${count} of ${table.count} complete headers. ABI declarations in unavailable headers remain unknown.`,
|
|
||||||
);
|
|
||||||
if (count === 0) continue;
|
|
||||||
// Counts are bounded by the ELF16 fields and strides by their exact ELF32
|
// Counts are bounded by the ELF16 fields and strides by their exact ELF32
|
||||||
// structure sizes. Never allocate according to an untrusted section size.
|
// structure sizes. Never allocate according to an untrusted section size.
|
||||||
const contents = await read(
|
const contents = await read(
|
||||||
table.offset,
|
table.offset,
|
||||||
count * table.entrySize,
|
table.count * table.entrySize,
|
||||||
table.label,
|
table.label,
|
||||||
);
|
);
|
||||||
if (!contents.ok) return contents;
|
if (!contents.ok) return contents;
|
||||||
let found = false;
|
let found = false;
|
||||||
for (let i = 0; i < count; i += 1) {
|
for (let i = 0; i < table.count; i += 1) {
|
||||||
checkCancelled();
|
checkCancelled();
|
||||||
const position = i * table.entrySize;
|
const position = i * table.entrySize;
|
||||||
if (u32(contents.value, position + table.typeOffset) !== table.recordType)
|
if (u32(contents.value, position + table.typeOffset) !== table.recordType)
|
||||||
@@ -154,10 +134,10 @@ export const readMipsElfAbiFlags = async (
|
|||||||
record = candidate.value;
|
record = candidate.value;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (record === undefined) return ok({ abiFlags: null, limitations });
|
if (record === undefined) return ok(null);
|
||||||
// Preserve declarations verbatim, including unknown versions/values. Provider
|
// Preserve declarations verbatim, including unknown versions/values. Provider
|
||||||
// admission, not parsing, decides which interpretations have been verified.
|
// admission, not parsing, decides which interpretations have been verified.
|
||||||
const abiFlags: MipsAbiFlags = {
|
return ok({
|
||||||
version: u16(record, 0),
|
version: u16(record, 0),
|
||||||
isaLevel: record.readUInt8(2),
|
isaLevel: record.readUInt8(2),
|
||||||
isaRevision: record.readUInt8(3),
|
isaRevision: record.readUInt8(3),
|
||||||
@@ -169,6 +149,5 @@ export const readMipsElfAbiFlags = async (
|
|||||||
ases: u32(record, 12),
|
ases: u32(record, 12),
|
||||||
flags1: u32(record, 16),
|
flags1: u32(record, 16),
|
||||||
flags2: u32(record, 20),
|
flags2: u32(record, 20),
|
||||||
};
|
});
|
||||||
return ok({ abiFlags, limitations });
|
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,11 +1,3 @@
|
|||||||
import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js";
|
|
||||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
|
||||||
import {
|
|
||||||
createJavaScriptSemanticGraph,
|
|
||||||
createJavaScriptSemanticGraphNode,
|
|
||||||
JavaScriptSemanticEvidenceContextRegistry,
|
|
||||||
} from "../domain/javascript/javascriptSemanticGraph.js";
|
|
||||||
import { proofEvidence } from "./ReconstructionObligationLedger.fixture.js";
|
|
||||||
import { describe, expect, it } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
import { createEvidence } from "../domain/evidence.js";
|
import { createEvidence } from "../domain/evidence.js";
|
||||||
@@ -179,115 +171,3 @@ describe("reconstruction obligation candidates", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps each semantic candidate bound to its own file and authority context", () => {
|
|
||||||
const contexts = new JavaScriptSemanticEvidenceContextRegistry();
|
|
||||||
const base = javascriptApplicationAnalysisResultSchema.parse(
|
|
||||||
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
|
|
||||||
);
|
|
||||||
const sourceRecords = [
|
|
||||||
proofEvidence("first-file"),
|
|
||||||
proofEvidence("second-file"),
|
|
||||||
];
|
|
||||||
const module = createJavaScriptSemanticGraphNode(
|
|
||||||
{
|
|
||||||
kind: "module",
|
|
||||||
identity: {
|
|
||||||
artifact_sha256: base.root_artifact_sha256,
|
|
||||||
module_path: "app.js",
|
|
||||||
source_range: null,
|
|
||||||
role_key: "module",
|
|
||||||
},
|
|
||||||
function_node_id: null,
|
|
||||||
application_node_ids: [],
|
|
||||||
label: "app.js",
|
|
||||||
properties: {},
|
|
||||||
evidence: artifactEvidence(base.root_artifact_sha256, "app.js"),
|
|
||||||
},
|
|
||||||
contexts,
|
|
||||||
);
|
|
||||||
const requests = sourceRecords.flatMap((source, fileIndex) =>
|
|
||||||
[1, 2].map((line) => {
|
|
||||||
const sha256 = String(fileIndex + 4).repeat(64);
|
|
||||||
const path = `file-${String(fileIndex)}.js`;
|
|
||||||
return createJavaScriptSemanticGraphNode(
|
|
||||||
{
|
|
||||||
kind: "request",
|
|
||||||
identity: {
|
|
||||||
artifact_sha256: sha256,
|
|
||||||
module_path: path,
|
|
||||||
source_range: {
|
|
||||||
start: { line, column: 0 },
|
|
||||||
end: { line, column: 8 },
|
|
||||||
},
|
|
||||||
role_key: `request-${String(line)}`,
|
|
||||||
},
|
|
||||||
function_node_id: null,
|
|
||||||
application_node_ids: [],
|
|
||||||
label: `${path}:${String(line)}`,
|
|
||||||
properties: {},
|
|
||||||
evidence: {
|
|
||||||
...artifactEvidence(sha256, path, "ast-static-analysis"),
|
|
||||||
evidence_ids: [source.evidence_id],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
contexts,
|
|
||||||
);
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
const { graph_id: _graphId, ...semantic } = base.semantic_graph;
|
|
||||||
const semanticGraph = createJavaScriptSemanticGraph({
|
|
||||||
...semantic,
|
|
||||||
root_node_ids: [module.node_id],
|
|
||||||
nodes: [module, ...requests],
|
|
||||||
evidence_contexts: contexts.contexts,
|
|
||||||
});
|
|
||||||
const example = JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE;
|
|
||||||
const application = createEvidence(
|
|
||||||
{
|
|
||||||
path: base.input_path,
|
|
||||||
sha256: base.root_artifact_sha256,
|
|
||||||
format: base.format,
|
|
||||||
},
|
|
||||||
example.provider,
|
|
||||||
{
|
|
||||||
predicateType: example.predicate_type,
|
|
||||||
operation: example.operation,
|
|
||||||
parameters: example.parameters,
|
|
||||||
result: jsonValueSchema.parse({ ...base, semantic_graph: semanticGraph }),
|
|
||||||
confidence: "derived",
|
|
||||||
authority: "shipped-artifact",
|
|
||||||
evidenceLinks: sourceRecords.map(({ evidence_id }) => evidence_id),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
const generated = deriveReconstructionObligationCandidates(
|
|
||||||
createEvidenceBundle([...sourceRecords, application]),
|
|
||||||
[],
|
|
||||||
);
|
|
||||||
expect(generated.limitations).toEqual([]);
|
|
||||||
expect(generated.candidates).toHaveLength(requests.length);
|
|
||||||
for (const [index, node] of requests.entries()) {
|
|
||||||
const source = sourceRecords[Math.floor(index / 2)];
|
|
||||||
expect(
|
|
||||||
generated.candidates.find(
|
|
||||||
({ target }) => target.semantic_node_id === node.node_id,
|
|
||||||
),
|
|
||||||
).toMatchObject({
|
|
||||||
title: node.label,
|
|
||||||
family: "request",
|
|
||||||
source_state: "candidate",
|
|
||||||
target: {
|
|
||||||
artifact_sha256: node.identity.artifact_sha256,
|
|
||||||
semantic_node_id: node.node_id,
|
|
||||||
},
|
|
||||||
authority_references: [
|
|
||||||
{
|
|
||||||
evidence_id: source?.evidence_id,
|
|
||||||
authority: "controlled-replay",
|
|
||||||
state: "candidate",
|
|
||||||
location: `${semanticGraph.graph_id}/node/${node.node_id}`,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import type {
|
|||||||
ReconstructionObligation,
|
ReconstructionObligation,
|
||||||
ReviewedReconstructionObligation,
|
ReviewedReconstructionObligation,
|
||||||
} from "../domain/reconstructionObligationLedgerSchemas.js";
|
} from "../domain/reconstructionObligationLedgerSchemas.js";
|
||||||
|
import { resolveJavaScriptSemanticEvidence } from "../domain/javascript/javascriptSemanticGraph.js";
|
||||||
import { parseApplicationGraphEvidence } from "./javascript/JavaScriptApplicationEvidenceGraph.js";
|
import { parseApplicationGraphEvidence } from "./javascript/JavaScriptApplicationEvidenceGraph.js";
|
||||||
import {
|
import {
|
||||||
applicationObligationPolicy,
|
applicationObligationPolicy,
|
||||||
@@ -114,21 +115,14 @@ const deriveApplicationCandidates = (
|
|||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
if (source.semanticGraph !== null) {
|
if (source.semanticGraph !== null)
|
||||||
const evidenceContexts = new Map(
|
|
||||||
source.semanticGraph.evidence_contexts.map((context) => [
|
|
||||||
context.context_id,
|
|
||||||
context,
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
for (const node of source.semanticGraph.nodes) {
|
for (const node of source.semanticGraph.nodes) {
|
||||||
const candidatePolicy = semanticObligationPolicy(node.kind);
|
const candidatePolicy = semanticObligationPolicy(node.kind);
|
||||||
if (candidatePolicy === undefined) continue;
|
if (candidatePolicy === undefined) continue;
|
||||||
const evidenceContext = evidenceContexts.get(node.evidence.context_id);
|
const evidenceContext = resolveJavaScriptSemanticEvidence(
|
||||||
if (evidenceContext === undefined)
|
source.semanticGraph,
|
||||||
throw new TypeError(
|
node.evidence,
|
||||||
"Semantic evidence reference names an absent context",
|
);
|
||||||
);
|
|
||||||
addCandidate(
|
addCandidate(
|
||||||
candidates,
|
candidates,
|
||||||
generatedCandidate({
|
generatedCandidate({
|
||||||
@@ -150,7 +144,6 @@ const deriveApplicationCandidates = (
|
|||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
|
||||||
if (source.graph.coverage.status !== "complete")
|
if (source.graph.coverage.status !== "complete")
|
||||||
limitations.add(
|
limitations.add(
|
||||||
`Application graph ${source.graph.graph_id} coverage is ${source.graph.coverage.status}; omitted candidates remain unknown.`,
|
`Application graph ${source.graph.graph_id} coverage is ${source.graph.coverage.status}; omitted candidates remain unknown.`,
|
||||||
|
|||||||
@@ -8,24 +8,14 @@ import {
|
|||||||
type HistoricalSourceGraphInput,
|
type HistoricalSourceGraphInput,
|
||||||
} from "../domain/referenceSourceGraph.js";
|
} from "../domain/referenceSourceGraph.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
|
||||||
import { readReferenceSource } from "../reference/ReferenceSourceReader.js";
|
import { readReferenceSource } from "../reference/ReferenceSourceReader.js";
|
||||||
import type {
|
|
||||||
ReferenceSourceEntryKind,
|
|
||||||
ReferenceSourceRead,
|
|
||||||
ReferenceSourceReaderError,
|
|
||||||
} from "../reference/ReferenceSourceReaderTypes.js";
|
|
||||||
import { parseReferenceSourceEntries } from "./ReferenceSourceImportEntries.js";
|
import { parseReferenceSourceEntries } from "./ReferenceSourceImportEntries.js";
|
||||||
import { readReferenceSourceVcs } from "./ReferenceSourceVcsAdapter.js";
|
import { readReferenceSourceVcs } from "./ReferenceSourceVcsAdapter.js";
|
||||||
import {
|
import {
|
||||||
type ReferenceSourceImportError,
|
type ReferenceSourceImportError,
|
||||||
type ReferenceSourceImportOptions,
|
type ReferenceSourceImportOptions,
|
||||||
} from "./ReferenceSourceImportTypes.js";
|
} from "./ReferenceSourceImportTypes.js";
|
||||||
import {
|
import { prepareReferenceSourceImport } from "./ReferenceSourceImportPolicy.js";
|
||||||
prepareReferenceSourceImport,
|
|
||||||
type PreparedReferenceSourceImport,
|
|
||||||
} from "./ReferenceSourceImportPolicy.js";
|
|
||||||
|
|
||||||
const failure = (
|
const failure = (
|
||||||
code: ReferenceSourceImportError["code"],
|
code: ReferenceSourceImportError["code"],
|
||||||
@@ -36,12 +26,8 @@ const failure = (
|
|||||||
message,
|
message,
|
||||||
});
|
});
|
||||||
|
|
||||||
const cancelled = (
|
const cancelled = (): ReferenceSourceImportError =>
|
||||||
partial?: ReferenceSourceRead,
|
failure("cancelled", "Reference source import cancelled");
|
||||||
): ReferenceSourceImportError => ({
|
|
||||||
...failure("cancelled", "Reference source import cancelled"),
|
|
||||||
...(partial === undefined ? {} : { partial }),
|
|
||||||
});
|
|
||||||
|
|
||||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
||||||
|
|
||||||
@@ -129,48 +115,9 @@ const sortExclusions = (
|
|||||||
[...exclusions].sort((left, right) => {
|
[...exclusions].sort((left, right) => {
|
||||||
const byPath = compareUnicodeCodePoints(left.path, right.path);
|
const byPath = compareUnicodeCodePoints(left.path, right.path);
|
||||||
if (byPath !== 0) return byPath;
|
if (byPath !== 0) return byPath;
|
||||||
const byReason = compareUnicodeCodePoints(left.reason, right.reason);
|
return compareUnicodeCodePoints(left.reason, right.reason);
|
||||||
if (byReason !== 0) return byReason;
|
|
||||||
return compareUnicodeCodePoints(
|
|
||||||
"pattern" in left ? left.pattern : "",
|
|
||||||
"pattern" in right ? right.pattern : "",
|
|
||||||
);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
const createShouldExclude =
|
|
||||||
(
|
|
||||||
exclusions: HistoricalSourceGraphInput["exclusions"],
|
|
||||||
secrets: PreparedReferenceSourceImport["secrets"],
|
|
||||||
ignored: PreparedReferenceSourceImport["ignored"],
|
|
||||||
): ((path: string, kind: ReferenceSourceEntryKind) => boolean) =>
|
|
||||||
(path, kind) => {
|
|
||||||
const patternPath = kind === "directory" ? `${path}/` : path;
|
|
||||||
const secretMatch = secrets.test(patternPath);
|
|
||||||
if (secretMatch.ignored) {
|
|
||||||
if (!secretMatch.rule)
|
|
||||||
throw new Error(`Ignored secret path has no matching rule: ${path}`);
|
|
||||||
exclusions.push({
|
|
||||||
path,
|
|
||||||
reason: "configured-secret",
|
|
||||||
pattern: secretMatch.rule.pattern,
|
|
||||||
});
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
const match = ignored.test(patternPath);
|
|
||||||
if (!match.ignored) return false;
|
|
||||||
if (!match.rule)
|
|
||||||
throw new Error(`Ignored path has no matching rule: ${path}`);
|
|
||||||
const reason = match.rule.mark;
|
|
||||||
if (
|
|
||||||
reason !== "project-ignored" &&
|
|
||||||
reason !== "default-ignored" &&
|
|
||||||
reason !== "caller-excluded"
|
|
||||||
)
|
|
||||||
throw new Error(`Ignored path has unknown rule origin: ${path}`);
|
|
||||||
exclusions.push({ path, reason, pattern: match.rule.pattern });
|
|
||||||
return true;
|
|
||||||
};
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Import a reference source directory into a committed historical source graph.
|
* Import a reference source directory into a committed historical source graph.
|
||||||
*
|
*
|
||||||
@@ -180,41 +127,43 @@ const createShouldExclude =
|
|||||||
*/
|
*/
|
||||||
export const importReferenceSource = async (
|
export const importReferenceSource = async (
|
||||||
options: ReferenceSourceImportOptions,
|
options: ReferenceSourceImportOptions,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
): Promise<Result<HistoricalSourceGraph, ReferenceSourceImportError>> => {
|
): Promise<Result<HistoricalSourceGraph, ReferenceSourceImportError>> => {
|
||||||
if (isAborted(options.signal)) return err(cancelled());
|
if (isAborted(options.signal)) return err(cancelled());
|
||||||
const prepared = await prepareReferenceSourceImport(options, resources);
|
const prepared = await prepareReferenceSourceImport(options);
|
||||||
if (!prepared.ok) return prepared;
|
if (!prepared.ok) return prepared;
|
||||||
const { ignored, root, secrets } = prepared.value;
|
const { ignored, root, secrets } = prepared.value;
|
||||||
if (isAborted(options.signal)) return err(cancelled());
|
if (isAborted(options.signal)) return err(cancelled());
|
||||||
|
|
||||||
const exclusions: HistoricalSourceGraphInput["exclusions"] = [];
|
const exclusions: HistoricalSourceGraphInput["exclusions"] = [];
|
||||||
const shouldExclude = createShouldExclude(exclusions, secrets, ignored);
|
const shouldExclude = (path: string): boolean => {
|
||||||
|
if (secrets.ignores(path)) {
|
||||||
|
exclusions.push({ path, reason: "configured-secret" });
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (ignored.ignores(path)) {
|
||||||
|
exclusions.push({ path, reason: "caller-excluded" });
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
|
||||||
const [readResult, vcsResult] = await Promise.all([
|
const [readResult, vcs] = await Promise.all([
|
||||||
readReferenceSource(root, resources, {
|
readReferenceSource(root, {
|
||||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||||
shouldExclude,
|
shouldExclude,
|
||||||
}),
|
}),
|
||||||
readReferenceSourceVcs(root, resources, options.signal),
|
readReferenceSourceVcs(root, options.signal),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
if (!readResult.ok)
|
if (!readResult.ok) {
|
||||||
return err(
|
const error = readResult.error;
|
||||||
combineReadFailures(
|
if (error.code === "cancelled") return err(cancelled());
|
||||||
readResult.error,
|
if (error.code === "unsupported")
|
||||||
vcsResult.ok ? undefined : vcsResult.error,
|
return err(failure("unsupported", error.message));
|
||||||
readResult.error.partial,
|
return err(failure("io", error.message));
|
||||||
),
|
}
|
||||||
);
|
|
||||||
if (!vcsResult.ok)
|
|
||||||
return err(
|
|
||||||
combineReadFailures(vcsResult.error, undefined, readResult.value),
|
|
||||||
);
|
|
||||||
|
|
||||||
const vcs = vcsResult.value;
|
if (isAborted(options.signal)) return err(cancelled());
|
||||||
|
|
||||||
if (isAborted(options.signal)) return err(cancelled(readResult.value));
|
|
||||||
|
|
||||||
const read = readResult.value;
|
const read = readResult.value;
|
||||||
const filePaths = new Set(
|
const filePaths = new Set(
|
||||||
@@ -226,7 +175,7 @@ export const importReferenceSource = async (
|
|||||||
const { entries, relationships, parseFailures, limitations } =
|
const { entries, relationships, parseFailures, limitations } =
|
||||||
parseReferenceSourceEntries(read, filePaths, options.signal);
|
parseReferenceSourceEntries(read, filePaths, options.signal);
|
||||||
|
|
||||||
if (isAborted(options.signal)) return err(cancelled(read));
|
if (isAborted(options.signal)) return err(cancelled());
|
||||||
|
|
||||||
const uniqueRelationships = deduplicateRelationships(relationships);
|
const uniqueRelationships = deduplicateRelationships(relationships);
|
||||||
const uniqueFailures = normalizeHistoricalSourceParseFailures(parseFailures);
|
const uniqueFailures = normalizeHistoricalSourceParseFailures(parseFailures);
|
||||||
@@ -272,45 +221,3 @@ export const importReferenceSource = async (
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const combineReadFailures = (
|
|
||||||
primary: ReferenceSourceReaderError,
|
|
||||||
other: ReferenceSourceReaderError | undefined,
|
|
||||||
partial: ReferenceSourceRead | undefined,
|
|
||||||
): ReferenceSourceImportError => {
|
|
||||||
const code =
|
|
||||||
primary.code === "cancelled" || other?.code === "cancelled"
|
|
||||||
? "cancelled"
|
|
||||||
: primary.code;
|
|
||||||
const cleanup = mergeCleanup(primary.cleanup, other?.cleanup);
|
|
||||||
const message =
|
|
||||||
other === undefined
|
|
||||||
? primary.message
|
|
||||||
: `${primary.message}; additional reference metadata failure: ${other.message}`;
|
|
||||||
const cause =
|
|
||||||
other === undefined
|
|
||||||
? primary.cause
|
|
||||||
: new AggregateError(
|
|
||||||
[primary, other],
|
|
||||||
"Reference source inventory and metadata both failed",
|
|
||||||
{ cause: primary },
|
|
||||||
);
|
|
||||||
return {
|
|
||||||
...failure(code, message),
|
|
||||||
...(cleanup === undefined ? {} : { cleanup }),
|
|
||||||
...(partial === undefined ? {} : { partial }),
|
|
||||||
...(cause === undefined ? {} : { cause }),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
const mergeCleanup = (
|
|
||||||
first: AnalysisCleanupObservation | undefined,
|
|
||||||
second: AnalysisCleanupObservation | undefined,
|
|
||||||
): AnalysisCleanupObservation | undefined => {
|
|
||||||
if (first === undefined) return second;
|
|
||||||
if (second === undefined) return first;
|
|
||||||
return {
|
|
||||||
reason: `${first.reason}; ${second.reason}`,
|
|
||||||
resources: [...new Set([...first.resources, ...second.resources])],
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|||||||
@@ -1,22 +1,14 @@
|
|||||||
import { lstat, realpath, stat } from "node:fs/promises";
|
import { readFile, realpath, stat } from "node:fs/promises";
|
||||||
import { join, resolve } from "node:path";
|
import { join, resolve } from "node:path";
|
||||||
|
|
||||||
import ignore from "ignore";
|
import ignore from "ignore";
|
||||||
import {
|
|
||||||
readRegularFileText,
|
|
||||||
RegularFileCleanupFailure,
|
|
||||||
retryRegularFileCleanup,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
|
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
|
||||||
import {
|
import {
|
||||||
DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS,
|
DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS,
|
||||||
type ReferenceSourceImportError,
|
type ReferenceSourceImportError,
|
||||||
type ReferenceSourceImportOptions,
|
type ReferenceSourceImportOptions,
|
||||||
} from "./ReferenceSourceImportTypes.js";
|
} from "./ReferenceSourceImportTypes.js";
|
||||||
import { rootFilesystemFailure } from "../reference/ReferenceSourceReaderErrors.js";
|
|
||||||
|
|
||||||
/** Validated inputs ready for filesystem traversal. */
|
/** Validated inputs ready for filesystem traversal. */
|
||||||
export interface PreparedReferenceSourceImport {
|
export interface PreparedReferenceSourceImport {
|
||||||
@@ -34,158 +26,54 @@ const failure = (
|
|||||||
message,
|
message,
|
||||||
});
|
});
|
||||||
|
|
||||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
|
||||||
|
|
||||||
const addMarkedPatterns = (
|
|
||||||
matcher: ReturnType<typeof ignore>,
|
|
||||||
patterns: readonly string[],
|
|
||||||
mark: "project-ignored" | "default-ignored" | "caller-excluded",
|
|
||||||
): void => {
|
|
||||||
for (const pattern of patterns)
|
|
||||||
for (const line of pattern.split(/\r?\n/u))
|
|
||||||
matcher.add({ pattern: line, mark });
|
|
||||||
};
|
|
||||||
|
|
||||||
const resolveRoot = async (
|
const resolveRoot = async (
|
||||||
requestedRoot: string,
|
requestedRoot: string,
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<Result<string, ReferenceSourceImportError>> => {
|
): Promise<Result<string, ReferenceSourceImportError>> => {
|
||||||
if (isAborted(signal))
|
|
||||||
return err(failure("cancelled", "Reference source import cancelled"));
|
|
||||||
try {
|
try {
|
||||||
const metadata = await stat(requestedRoot);
|
if (!(await stat(requestedRoot)).isDirectory())
|
||||||
if (isAborted(signal))
|
|
||||||
return err(failure("cancelled", "Reference source import cancelled"));
|
|
||||||
if (!metadata.isDirectory())
|
|
||||||
return err(
|
return err(
|
||||||
failure(
|
failure("invalid-root", "Reference source root is not a directory"),
|
||||||
"invalid-root",
|
|
||||||
`Reference source root is not a directory: ${requestedRoot}`,
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
const canonicalRoot = await realpath(resolve(requestedRoot));
|
const canonicalRoot = await realpath(resolve(requestedRoot));
|
||||||
if (isAborted(signal))
|
|
||||||
return err(failure("cancelled", "Reference source import cancelled"));
|
|
||||||
return ok(canonicalRoot);
|
return ok(canonicalRoot);
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (isAborted(signal))
|
void cause;
|
||||||
return err(failure("cancelled", "Reference source import cancelled"));
|
return err(
|
||||||
const rootFailure = rootFilesystemFailure(
|
failure("invalid-root", "Reference source root could not be resolved"),
|
||||||
cause,
|
|
||||||
"Reference source root could not be resolved",
|
|
||||||
);
|
);
|
||||||
if (rootFailure === undefined) throw cause;
|
|
||||||
return err({ ...failure(rootFailure.code, rootFailure.message), cause });
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const buildIgnored = async (
|
const buildIgnored = async (
|
||||||
root: string,
|
root: string,
|
||||||
excludePaths: readonly string[],
|
excludePaths: readonly string[],
|
||||||
resources: ArtifactResourceScope,
|
): Promise<ReturnType<typeof ignore>> => {
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<Result<ReturnType<typeof ignore>, ReferenceSourceImportError>> => {
|
|
||||||
const ignored = ignore();
|
const ignored = ignore();
|
||||||
const policyPath = join(root, ".gitignore");
|
|
||||||
let present = true;
|
|
||||||
try {
|
try {
|
||||||
await lstat(policyPath);
|
ignored.add(await readFile(join(root, ".gitignore"), "utf8"));
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (
|
// Missing or unreadable ignore file does not authorize broader access.
|
||||||
!(
|
void cause;
|
||||||
typeof cause === "object" &&
|
|
||||||
cause !== null &&
|
|
||||||
"code" in cause &&
|
|
||||||
cause.code === "ENOENT"
|
|
||||||
)
|
|
||||||
)
|
|
||||||
throw cause;
|
|
||||||
present = false;
|
|
||||||
}
|
}
|
||||||
signal?.throwIfAborted();
|
ignored.add([...DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS]);
|
||||||
if (present) {
|
|
||||||
let text: string;
|
|
||||||
try {
|
|
||||||
text = await readRegularFileText(policyPath, { signal });
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
|
||||||
const retry = await retryRegularFileCleanup(cause, resources);
|
|
||||||
if (retry.cleanup !== undefined) {
|
|
||||||
const primary =
|
|
||||||
cause.outcome.kind === "failed"
|
|
||||||
? cause.outcome.cause
|
|
||||||
: cause.cleanupCause;
|
|
||||||
return err({
|
|
||||||
...failure(
|
|
||||||
signal?.aborted === true ? "cancelled" : "io",
|
|
||||||
`Reference source ignore policy could not be read at ${policyPath}: ${errorMessage(primary)}`,
|
|
||||||
),
|
|
||||||
cleanup: retry.cleanup,
|
|
||||||
cause: primary,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
if (retry.outcome.kind === "failed") throw retry.outcome.cause;
|
|
||||||
text = retry.outcome.value;
|
|
||||||
}
|
|
||||||
addMarkedPatterns(ignored, [text], "project-ignored");
|
|
||||||
}
|
|
||||||
addMarkedPatterns(
|
|
||||||
ignored,
|
|
||||||
DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS,
|
|
||||||
"default-ignored",
|
|
||||||
);
|
|
||||||
for (const path of excludePaths) {
|
for (const path of excludePaths) {
|
||||||
addMarkedPatterns(ignored, [path, `${path}/`], "caller-excluded");
|
ignored.add(path);
|
||||||
|
ignored.add(`${path}/`);
|
||||||
}
|
}
|
||||||
return ok(ignored);
|
return ignored;
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Resolve the caller-selected directory and build path filters. */
|
/** Resolve the caller-selected directory and build path filters. */
|
||||||
export const prepareReferenceSourceImport = async (
|
export const prepareReferenceSourceImport = async (
|
||||||
options: ReferenceSourceImportOptions,
|
options: ReferenceSourceImportOptions,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
): Promise<
|
): Promise<
|
||||||
Result<PreparedReferenceSourceImport, ReferenceSourceImportError>
|
Result<PreparedReferenceSourceImport, ReferenceSourceImportError>
|
||||||
> => {
|
> => {
|
||||||
try {
|
const root = await resolveRoot(options.root);
|
||||||
return await resources.run(async () => {
|
if (!root.ok) return root;
|
||||||
const root = await resolveRoot(options.root, options.signal);
|
return ok({
|
||||||
if (!root.ok) return root;
|
root: root.value,
|
||||||
try {
|
ignored: await buildIgnored(root.value, options.excludePaths ?? []),
|
||||||
const ignored = await buildIgnored(
|
secrets: ignore().add([...options.policy.secretPatterns]),
|
||||||
root.value,
|
});
|
||||||
options.excludePaths ?? [],
|
|
||||||
resources,
|
|
||||||
options.signal,
|
|
||||||
);
|
|
||||||
if (!ignored.ok) return ignored;
|
|
||||||
return ok({
|
|
||||||
root: root.value,
|
|
||||||
ignored: ignored.value,
|
|
||||||
secrets: ignore().add([...options.policy.secretPatterns]),
|
|
||||||
});
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
return err({
|
|
||||||
...failure(
|
|
||||||
options.signal?.aborted === true ? "cancelled" : "io",
|
|
||||||
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${errorMessage(cause)}`,
|
|
||||||
),
|
|
||||||
cause,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof ArtifactReaderFailure)) throw cause;
|
|
||||||
return err({
|
|
||||||
...failure(
|
|
||||||
cause.reason === "cancelled" ? "cancelled" : "io",
|
|
||||||
cause.message,
|
|
||||||
),
|
|
||||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
|
||||||
cause,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const errorMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|||||||
@@ -1,80 +0,0 @@
|
|||||||
import { describe, expect, it } from "vitest";
|
|
||||||
|
|
||||||
import {
|
|
||||||
projectReferenceSourceImportError,
|
|
||||||
type ReferenceSourceImportError,
|
|
||||||
} from "./ReferenceSourceImportTypes.js";
|
|
||||||
import type { ReferenceSourceRead } from "../reference/ReferenceSourceReaderTypes.js";
|
|
||||||
|
|
||||||
describe("reference-source import error projection", () => {
|
|
||||||
it("preserves primary evidence with category-specific recovery guidance", () => {
|
|
||||||
const expectedCategories = {
|
|
||||||
cancelled: "cancelled",
|
|
||||||
"invalid-root": "invalid_input",
|
|
||||||
unsupported: "unsupported_host",
|
|
||||||
io: "execution_failure",
|
|
||||||
parse: "execution_failure",
|
|
||||||
} as const;
|
|
||||||
for (const [code, category] of Object.entries(expectedCategories)) {
|
|
||||||
const projected = projectReferenceSourceImportError({
|
|
||||||
tag: "reference-source-import",
|
|
||||||
code: code as ReferenceSourceImportError["code"],
|
|
||||||
message: "EIO while reading /selected/source/src/main.ts",
|
|
||||||
});
|
|
||||||
expect(projected.category).toBe(category);
|
|
||||||
expect(projected.message).toContain(
|
|
||||||
"EIO while reading /selected/source/src/main.ts",
|
|
||||||
);
|
|
||||||
expect(projected.message).toMatch(
|
|
||||||
/try again|when ready|Check that|REA on Linux/u,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it("preserves cancellation, partial facts, and cleanup diagnostics", () => {
|
|
||||||
const bytes = Buffer.from("export const observed = true;\n");
|
|
||||||
const partial: ReferenceSourceRead = {
|
|
||||||
root: "/selected/source",
|
|
||||||
entries: [
|
|
||||||
{
|
|
||||||
status: "read",
|
|
||||||
kind: "file",
|
|
||||||
path: "src/main.ts",
|
|
||||||
bytes,
|
|
||||||
size: bytes.length,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
bytesRead: bytes.length,
|
|
||||||
limitations: [],
|
|
||||||
};
|
|
||||||
const projected = projectReferenceSourceImportError({
|
|
||||||
tag: "reference-source-import",
|
|
||||||
code: "cancelled",
|
|
||||||
message: "Traversal cancelled after reading src/main.ts",
|
|
||||||
cleanup: {
|
|
||||||
reason: "directory close rejected",
|
|
||||||
resources: ["/selected/source"],
|
|
||||||
},
|
|
||||||
partial,
|
|
||||||
});
|
|
||||||
|
|
||||||
expect(projected).toMatchObject({
|
|
||||||
category: "cancelled",
|
|
||||||
message: expect.stringContaining(
|
|
||||||
"Traversal cancelled after reading src/main.ts",
|
|
||||||
),
|
|
||||||
cleanup: {
|
|
||||||
reason: "directory close rejected",
|
|
||||||
resources: ["/selected/source"],
|
|
||||||
},
|
|
||||||
partial: {
|
|
||||||
root: "/selected/source",
|
|
||||||
entries: [{ path: "src/main.ts", bytes }],
|
|
||||||
},
|
|
||||||
});
|
|
||||||
expect(projected.message).toContain("Cleanup failed during import");
|
|
||||||
expect(projected.message).toContain("directory close rejected");
|
|
||||||
expect(projected.message).toContain("/selected/source");
|
|
||||||
expect(projected.message).not.toContain("remains unconfirmed");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,67 +1,44 @@
|
|||||||
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
|
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
|
||||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
|
||||||
import type { ReferenceSourceRead } from "../reference/ReferenceSourceReaderTypes.js";
|
|
||||||
|
|
||||||
/** Typed expected failure returned by historical-source imports. */
|
/** Typed expected failure returned by historical-source imports. */
|
||||||
export interface ReferenceSourceImportError {
|
export interface ReferenceSourceImportError {
|
||||||
readonly tag: "reference-source-import";
|
readonly tag: "reference-source-import";
|
||||||
readonly code: "cancelled" | "invalid-root" | "unsupported" | "io" | "parse";
|
readonly code: "cancelled" | "invalid-root" | "unsupported" | "io" | "parse";
|
||||||
readonly message: string;
|
readonly message: string;
|
||||||
readonly cleanup?: AnalysisCleanupObservation;
|
|
||||||
readonly partial?: ReferenceSourceRead;
|
|
||||||
readonly cause?: unknown;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const IMPORT_FAILURE_GUIDANCE = {
|
|
||||||
cancelled: {
|
|
||||||
category: "cancelled",
|
|
||||||
message:
|
|
||||||
"Reference source import was cancelled. Start it again when ready.",
|
|
||||||
},
|
|
||||||
"invalid-root": {
|
|
||||||
category: "invalid_input",
|
|
||||||
message:
|
|
||||||
"Reference source directory could not be opened. Check that the path exists, is readable, and points to a directory.",
|
|
||||||
},
|
|
||||||
unsupported: {
|
|
||||||
category: "unsupported_host",
|
|
||||||
message:
|
|
||||||
"Safe no-follow file opens are unavailable on this host. Import the source tree with REA on Linux (including WSL) or macOS.",
|
|
||||||
},
|
|
||||||
io: {
|
|
||||||
category: "execution_failure",
|
|
||||||
message:
|
|
||||||
"Reference source files could not be read. Check directory permissions and try again.",
|
|
||||||
},
|
|
||||||
parse: {
|
|
||||||
category: "execution_failure",
|
|
||||||
message:
|
|
||||||
"Reference source could not be indexed. Check that the source tree is readable, then try again.",
|
|
||||||
},
|
|
||||||
} satisfies Record<
|
|
||||||
ReferenceSourceImportError["code"],
|
|
||||||
{ readonly category: string; readonly message: string }
|
|
||||||
>;
|
|
||||||
|
|
||||||
/** Safe CLI projection for a historical-source import failure. */
|
/** Safe CLI projection for a historical-source import failure. */
|
||||||
export const projectReferenceSourceImportError = (
|
export const projectReferenceSourceImportError = (
|
||||||
error: ReferenceSourceImportError,
|
error: ReferenceSourceImportError,
|
||||||
): Readonly<{
|
): Readonly<{ category: string; message: string }> => {
|
||||||
category: string;
|
if (error.code === "cancelled")
|
||||||
message: string;
|
return {
|
||||||
cleanup?: AnalysisCleanupObservation;
|
category: "cancelled",
|
||||||
partial?: ReferenceSourceRead;
|
message:
|
||||||
}> => {
|
"Reference source import was cancelled. Start it again when ready.",
|
||||||
const primary = IMPORT_FAILURE_GUIDANCE[error.code];
|
};
|
||||||
const cleanupMessage =
|
if (error.code === "invalid-root")
|
||||||
error.cleanup === undefined
|
return {
|
||||||
? ""
|
category: "invalid_input",
|
||||||
: ` Cleanup failed during import for ${error.cleanup.resources.join(", ")}: ${error.cleanup.reason}.`;
|
message:
|
||||||
|
"Reference source directory could not be opened. Check that the path exists, is readable, and points to a directory.",
|
||||||
|
};
|
||||||
|
if (error.code === "unsupported")
|
||||||
|
return {
|
||||||
|
category: "unsupported_host",
|
||||||
|
message:
|
||||||
|
"Safe no-follow file opens are unavailable on this host. Import the source tree with REA on Linux (including WSL) or macOS.",
|
||||||
|
};
|
||||||
|
if (error.code === "io")
|
||||||
|
return {
|
||||||
|
category: "execution_failure",
|
||||||
|
message:
|
||||||
|
"Reference source files could not be read. Check directory permissions and try again.",
|
||||||
|
};
|
||||||
return {
|
return {
|
||||||
category: primary.category,
|
category: "execution_failure",
|
||||||
message: `${error.message} ${primary.message}${cleanupMessage}`,
|
message:
|
||||||
...(error.cleanup === undefined ? {} : { cleanup: error.cleanup }),
|
"Reference source could not be indexed. Check that the source tree is readable, then try again.",
|
||||||
...(error.partial === undefined ? {} : { partial: error.partial }),
|
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -1,15 +1,6 @@
|
|||||||
import fs from "node:fs";
|
import fs from "node:fs";
|
||||||
import { lstat } from "node:fs/promises";
|
import { lstat, readFile } from "node:fs/promises";
|
||||||
import { join, resolve } from "node:path";
|
import { join, resolve } from "node:path";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
|
||||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
|
||||||
import {
|
|
||||||
readRegularFile,
|
|
||||||
RegularFileCleanupFailure,
|
|
||||||
retryRegularFileCleanup,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
import type { ReferenceSourceReaderError } from "../reference/ReferenceSourceReaderTypes.js";
|
|
||||||
|
|
||||||
export type ReferenceSourceVcsInfo =
|
export type ReferenceSourceVcsInfo =
|
||||||
| {
|
| {
|
||||||
@@ -27,174 +18,34 @@ export type ReferenceSourceVcsInfo =
|
|||||||
*/
|
*/
|
||||||
export const readReferenceSourceVcs = async (
|
export const readReferenceSourceVcs = async (
|
||||||
root: string,
|
root: string,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
): Promise<Result<ReferenceSourceVcsInfo, ReferenceSourceReaderError>> => {
|
): Promise<ReferenceSourceVcsInfo> => {
|
||||||
|
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
|
||||||
try {
|
try {
|
||||||
return await resources.run(async () => {
|
await lstat(join(root, ".git"));
|
||||||
if (isAborted(signal)) return ok(unknownVcs());
|
|
||||||
try {
|
|
||||||
await lstat(join(root, ".git"));
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
return ok(
|
|
||||||
errorCode(cause) === "ENOENT"
|
|
||||||
? { kind: "none", head: null, dirty: null }
|
|
||||||
: unknownVcs(),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const head = await resolveSourceHead(root, resources, signal);
|
|
||||||
if (isAborted(signal)) return ok(unknownVcs());
|
|
||||||
return ok({ kind: "git", head, dirty: null });
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof ArtifactReaderFailure) throw cause;
|
|
||||||
return ok(unknownVcs());
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (!(cause instanceof ArtifactReaderFailure)) throw cause;
|
return errorCode(cause) === "ENOENT"
|
||||||
return err({
|
? { kind: "none", head: null, dirty: null }
|
||||||
tag: "reference-source-reader",
|
: { kind: "unknown", head: null, dirty: null };
|
||||||
code: cause.reason === "cancelled" ? "cancelled" : "io",
|
}
|
||||||
message: cause.message,
|
try {
|
||||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
const head = await resolveSourceHead(root);
|
||||||
cause,
|
if (isAborted(signal)) return { kind: "unknown", head: null, dirty: null };
|
||||||
});
|
return { kind: "git", head, dirty: null };
|
||||||
|
} catch (cause: unknown) {
|
||||||
|
// Unresolvable refs mean VCS state is unknown, not absent.
|
||||||
|
void cause;
|
||||||
|
return { kind: "unknown", head: null, dirty: null };
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const unknownVcs = (): ReferenceSourceVcsInfo => ({
|
const resolveSourceHead = async (root: string): Promise<string> => {
|
||||||
kind: "unknown",
|
|
||||||
head: null,
|
|
||||||
dirty: null,
|
|
||||||
});
|
|
||||||
|
|
||||||
const resolveSourceHead = async (
|
|
||||||
root: string,
|
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<string> => {
|
|
||||||
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
|
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
|
||||||
const { resolveRef } = await import("isomorphic-git");
|
const { resolveRef } = await import("isomorphic-git");
|
||||||
let failedRead: { readonly cause: unknown } | undefined;
|
|
||||||
const cleanupFailures: ArtifactReaderFailure[] = [];
|
|
||||||
const readMetadataBytes = async (path: string): Promise<Buffer> => {
|
|
||||||
try {
|
|
||||||
return await readRegularFile(path, { signal });
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
|
||||||
const retry = await retryRegularFileCleanup(cause, resources);
|
|
||||||
if (
|
|
||||||
retry.outcome.kind === "failed" &&
|
|
||||||
errorCode(retry.outcome.cause) !== "ENOENT"
|
|
||||||
)
|
|
||||||
failedRead ??= { cause: retry.outcome.cause };
|
|
||||||
if (
|
|
||||||
retry.cleanup === undefined &&
|
|
||||||
retry.outcome.kind === "failed" &&
|
|
||||||
!isAborted(signal)
|
|
||||||
)
|
|
||||||
throw retry.outcome.cause;
|
|
||||||
const primary =
|
|
||||||
retry.outcome.kind === "failed"
|
|
||||||
? retry.outcome.cause
|
|
||||||
: cause.cleanupCause;
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
isAborted(signal) ? "cancelled" : "io",
|
|
||||||
`Git metadata could not be read safely: ${errorMessage(primary)}`,
|
|
||||||
{
|
|
||||||
cause: primary,
|
|
||||||
...(retry.cleanup === undefined ? {} : { cleanup: retry.cleanup }),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
const readMetadataText = async (path: string): Promise<string> =>
|
|
||||||
(await readMetadataBytes(path)).toString("utf8");
|
|
||||||
const throwCleanupFailures = (primary: unknown): never => {
|
|
||||||
const observations = cleanupFailures.flatMap((failure) =>
|
|
||||||
failure.cleanup === undefined ? [] : [failure.cleanup],
|
|
||||||
);
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
isAborted(signal) ? "cancelled" : "io",
|
|
||||||
`Git metadata could not be read safely: ${errorMessage(primary)}`,
|
|
||||||
{
|
|
||||||
cause: new AggregateError(
|
|
||||||
[primary, ...cleanupFailures],
|
|
||||||
"Git metadata reads and cleanup failed",
|
|
||||||
{ cause: primary },
|
|
||||||
),
|
|
||||||
...(observations.length === 0
|
|
||||||
? {}
|
|
||||||
: {
|
|
||||||
cleanup: {
|
|
||||||
reason: observations.map(({ reason }) => reason).join("; "),
|
|
||||||
resources: [
|
|
||||||
...new Set(
|
|
||||||
observations.flatMap(({ resources }) => resources),
|
|
||||||
),
|
|
||||||
],
|
|
||||||
},
|
|
||||||
}),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
};
|
|
||||||
const referenceFs = {
|
|
||||||
...fs,
|
|
||||||
promises: {
|
|
||||||
...fs.promises,
|
|
||||||
// Let isomorphic-git continue from a failed loose ref to packed-refs,
|
|
||||||
// then report the original read failure after resolution. This preserves
|
|
||||||
// both observations when packed metadata also fails cleanup.
|
|
||||||
readFile: async (
|
|
||||||
path: string | undefined,
|
|
||||||
options:
|
|
||||||
| BufferEncoding
|
|
||||||
| { readonly encoding?: BufferEncoding | null }
|
|
||||||
| null = null,
|
|
||||||
) => {
|
|
||||||
// The library probes readFile() without a path to detect promise APIs.
|
|
||||||
// Reject that invocation without attributing it to a source read.
|
|
||||||
if (path === undefined)
|
|
||||||
throw new TypeError("Git filesystem read requires a path");
|
|
||||||
try {
|
|
||||||
const bytes = await readMetadataBytes(path);
|
|
||||||
const encoding =
|
|
||||||
typeof options === "string" ? options : options?.encoding;
|
|
||||||
return encoding === undefined || encoding === null
|
|
||||||
? bytes
|
|
||||||
: bytes.toString(encoding);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof ArtifactReaderFailure) {
|
|
||||||
cleanupFailures.push(cause);
|
|
||||||
} else if (errorCode(cause) !== "ENOENT") failedRead ??= { cause };
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const readRef = async (options: {
|
|
||||||
readonly gitdir?: string;
|
|
||||||
readonly ref: string;
|
|
||||||
}) => {
|
|
||||||
let head: string;
|
|
||||||
try {
|
|
||||||
head = await resolveRef({ fs: referenceFs, dir: root, ...options });
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cleanupFailures.length > 0)
|
|
||||||
throwCleanupFailures(failedRead?.cause ?? cause);
|
|
||||||
if (failedRead !== undefined) throw failedRead.cause;
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
if (cleanupFailures.length > 0)
|
|
||||||
throwCleanupFailures(failedRead?.cause ?? cleanupFailures[0]?.cause);
|
|
||||||
signal?.throwIfAborted();
|
|
||||||
if (failedRead !== undefined) throw failedRead.cause;
|
|
||||||
return head;
|
|
||||||
};
|
|
||||||
const dotgit = join(root, ".git");
|
const dotgit = join(root, ".git");
|
||||||
if (!(await lstat(dotgit)).isFile()) return readRef({ ref: "HEAD" });
|
if (!(await lstat(dotgit)).isFile())
|
||||||
const pointer = (await readMetadataText(dotgit)).replace(/\r?\n$/u, "");
|
return resolveRef({ fs, dir: root, ref: "HEAD" });
|
||||||
|
const pointer = (await readFile(dotgit, "utf8")).replace(/\r?\n$/u, "");
|
||||||
if (!pointer.startsWith("gitdir: ") || pointer.length === 8)
|
if (!pointer.startsWith("gitdir: ") || pointer.length === 8)
|
||||||
throw new Error("Invalid Git directory pointer");
|
throw new Error("Invalid Git directory pointer");
|
||||||
const gitdir = resolve(root, pointer.slice(8));
|
const gitdir = resolve(root, pointer.slice(8));
|
||||||
@@ -202,22 +53,25 @@ const resolveSourceHead = async (
|
|||||||
try {
|
try {
|
||||||
commonDirectory = resolve(
|
commonDirectory = resolve(
|
||||||
gitdir,
|
gitdir,
|
||||||
(await readMetadataText(join(gitdir, "commondir"))).replace(
|
(await readFile(join(gitdir, "commondir"), "utf8")).replace(
|
||||||
/\r?\n$/u,
|
/\r?\n$/u,
|
||||||
"",
|
"",
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (errorCode(cause) !== "ENOENT") throw cause;
|
if (errorCode(cause) !== "ENOENT") throw cause;
|
||||||
return readRef({ gitdir, ref: "HEAD" });
|
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||||
}
|
}
|
||||||
const head = (await readMetadataText(join(gitdir, "HEAD"))).trim();
|
const head = (await readFile(join(gitdir, "HEAD"), "utf8")).trim();
|
||||||
if (!head.startsWith("ref: ")) return readRef({ gitdir, ref: "HEAD" });
|
if (!head.startsWith("ref: "))
|
||||||
|
return resolveRef({ fs, dir: root, gitdir, ref: "HEAD" });
|
||||||
const ref = head.slice(5);
|
const ref = head.slice(5);
|
||||||
const privateRef = ["refs/bisect/", "refs/rewritten/", "refs/worktree/"].some(
|
const privateRef = ["refs/bisect/", "refs/rewritten/", "refs/worktree/"].some(
|
||||||
(prefix) => ref.startsWith(prefix),
|
(prefix) => ref.startsWith(prefix),
|
||||||
);
|
);
|
||||||
return readRef({
|
return resolveRef({
|
||||||
|
fs,
|
||||||
|
dir: root,
|
||||||
gitdir: privateRef ? gitdir : commonDirectory,
|
gitdir: privateRef ? gitdir : commonDirectory,
|
||||||
ref,
|
ref,
|
||||||
});
|
});
|
||||||
@@ -228,7 +82,4 @@ const errorCode = (cause: unknown): string | undefined =>
|
|||||||
? String(cause.code)
|
? String(cause.code)
|
||||||
: undefined;
|
: undefined;
|
||||||
|
|
||||||
const errorMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|
||||||
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
|
||||||
|
|||||||
@@ -1,158 +1,29 @@
|
|||||||
import type { Stats } from "node:fs";
|
import type { Stats } from "node:fs";
|
||||||
import { lstat, stat, type FileHandle } from "node:fs/promises";
|
import type { FileHandle } from "node:fs/promises";
|
||||||
|
|
||||||
import {
|
import { openRegularFile } from "../filesystem/RegularFile.js";
|
||||||
RegularFileAdmissionFailure,
|
|
||||||
openRegularFile,
|
|
||||||
sameRegularFileState,
|
|
||||||
RegularFileChangedError,
|
|
||||||
} from "../filesystem/RegularFile.js";
|
|
||||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
|
||||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
|
||||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
|
||||||
|
|
||||||
/** Filesystem admission policy for one selected regular input. */
|
|
||||||
export interface RegularFileReadOptions {
|
|
||||||
readonly signal?: AbortSignal | undefined;
|
|
||||||
readonly symlinks?: "follow" | "reject";
|
|
||||||
}
|
|
||||||
|
|
||||||
export type RegularFileReadOutcome<Value> =
|
|
||||||
| { readonly kind: "completed"; readonly value: Value }
|
|
||||||
| { readonly kind: "failed"; readonly cause: unknown };
|
|
||||||
|
|
||||||
/** A close failed after reading; this error transfers the exact handle owner. */
|
|
||||||
export class RegularFileCleanupFailure<Value = unknown> extends Error {
|
|
||||||
constructor(
|
|
||||||
readonly path: string,
|
|
||||||
readonly owner: OwnedFileHandle,
|
|
||||||
readonly outcome: RegularFileReadOutcome<Value>,
|
|
||||||
readonly cleanupCause: unknown,
|
|
||||||
) {
|
|
||||||
super(`Regular file cleanup failed: ${path}`, {
|
|
||||||
cause: outcome.kind === "failed" ? outcome.cause : cleanupCause,
|
|
||||||
});
|
|
||||||
this.name = "RegularFileCleanupFailure";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Retry a failed close through its retained owner and report any remaining cleanup. */
|
|
||||||
export const retryRegularFileCleanup = async <Value>(
|
|
||||||
failure: RegularFileCleanupFailure<Value>,
|
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
): Promise<{
|
|
||||||
readonly outcome: RegularFileReadOutcome<Value>;
|
|
||||||
readonly cleanup?: AnalysisCleanupObservation;
|
|
||||||
}> => {
|
|
||||||
const released = await resources.release({
|
|
||||||
kind: "file-handle",
|
|
||||||
handle: failure.owner,
|
|
||||||
resource: failure.path,
|
|
||||||
});
|
|
||||||
if (released.kind === "released") return { outcome: failure.outcome };
|
|
||||||
return {
|
|
||||||
outcome: failure.outcome,
|
|
||||||
cleanup: {
|
|
||||||
reason: `${failureMessage(failure.cleanupCause)}; retry: ${failureMessage(released.cause)}`,
|
|
||||||
resources: [failure.path],
|
|
||||||
},
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
const failureMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|
||||||
/** Admit one regular-file handle, retain it through the consumer, and always close it. */
|
/** Admit one regular-file handle, retain it through the consumer, and always close it. */
|
||||||
export const withRegularFile = async <Value>(
|
export const withRegularFile = async <Value>(
|
||||||
path: string,
|
path: string,
|
||||||
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
|
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
|
||||||
options: RegularFileReadOptions = {},
|
signal?: AbortSignal,
|
||||||
): Promise<Value> => {
|
): Promise<Value> => {
|
||||||
const { signal } = options;
|
const handle = await openRegularFile(path, { symlinks: "follow", signal });
|
||||||
let handle: FileHandle;
|
|
||||||
try {
|
|
||||||
handle = await openRegularFile(path, {
|
|
||||||
symlinks: options.symlinks ?? "follow",
|
|
||||||
signal,
|
|
||||||
});
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof RegularFileAdmissionFailure)) throw cause;
|
|
||||||
throw new RegularFileCleanupFailure(
|
|
||||||
path,
|
|
||||||
cause.owner,
|
|
||||||
{ kind: "failed", cause: cause.cause },
|
|
||||||
cause.cleanupCause,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const owner = new OwnedFileHandle(handle);
|
|
||||||
let outcome: RegularFileReadOutcome<Value>;
|
|
||||||
try {
|
try {
|
||||||
const stats = await handle.stat();
|
const stats = await handle.stat();
|
||||||
signal?.throwIfAborted();
|
signal?.throwIfAborted();
|
||||||
const value = await read(handle, stats);
|
const value = await read(handle, stats);
|
||||||
signal?.throwIfAborted();
|
signal?.throwIfAborted();
|
||||||
const [opened, currentPath] = await Promise.all([
|
return value;
|
||||||
handle.stat(),
|
} finally {
|
||||||
options.symlinks === "reject" ? lstat(path) : stat(path),
|
await handle.close();
|
||||||
]);
|
|
||||||
if (
|
|
||||||
!sameRegularFileState(stats, opened) ||
|
|
||||||
!sameRegularFileState(stats, currentPath)
|
|
||||||
)
|
|
||||||
throw new RegularFileChangedError(path);
|
|
||||||
outcome = { kind: "completed", value };
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
outcome = { kind: "failed", cause };
|
|
||||||
}
|
}
|
||||||
try {
|
|
||||||
await owner.close();
|
|
||||||
} catch (cleanupCause: unknown) {
|
|
||||||
throw new RegularFileCleanupFailure(path, owner, outcome, cleanupCause);
|
|
||||||
}
|
|
||||||
if (outcome.kind === "failed") throw outcome.cause;
|
|
||||||
return outcome.value;
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Admit a regular file without waiting for a pipe, then read its verified handle. */
|
/** Admit a regular file without waiting for a pipe, then read its verified handle. */
|
||||||
export const readRegularFile = (
|
export const readRegularFile = (
|
||||||
path: string,
|
path: string,
|
||||||
options: RegularFileReadOptions = {},
|
signal?: AbortSignal,
|
||||||
): Promise<Buffer> =>
|
): Promise<Buffer> =>
|
||||||
withRegularFile(
|
withRegularFile(path, (handle) => handle.readFile({ signal }), signal);
|
||||||
path,
|
|
||||||
async (handle, stats) => {
|
|
||||||
const bytes = await readBoundedFileBytes(
|
|
||||||
handle,
|
|
||||||
stats.size,
|
|
||||||
options.signal,
|
|
||||||
);
|
|
||||||
if (bytes === undefined || bytes.length !== stats.size)
|
|
||||||
throw new RegularFileChangedError(path);
|
|
||||||
return bytes;
|
|
||||||
},
|
|
||||||
options,
|
|
||||||
);
|
|
||||||
|
|
||||||
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
|
|
||||||
export const readRegularFileText = async (
|
|
||||||
path: string,
|
|
||||||
options: RegularFileReadOptions = {},
|
|
||||||
): Promise<string> => {
|
|
||||||
try {
|
|
||||||
return (await readRegularFile(path, options)).toString("utf8");
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (
|
|
||||||
cause instanceof RegularFileCleanupFailure &&
|
|
||||||
cause.outcome.kind === "completed" &&
|
|
||||||
Buffer.isBuffer(cause.outcome.value)
|
|
||||||
)
|
|
||||||
throw new RegularFileCleanupFailure(
|
|
||||||
cause.path,
|
|
||||||
cause.owner,
|
|
||||||
{ kind: "completed", value: cause.outcome.value.toString("utf8") },
|
|
||||||
cause.cleanupCause,
|
|
||||||
);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|||||||
@@ -1,8 +1,5 @@
|
|||||||
import { describe, expect, it } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
|
||||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
|
||||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
|
||||||
import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
|
import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
|
||||||
import { FakeSetupHost, options } from "./Setup.fixture.js";
|
import { FakeSetupHost, options } from "./Setup.fixture.js";
|
||||||
import { runSetup } from "./Setup.js";
|
import { runSetup } from "./Setup.js";
|
||||||
@@ -110,76 +107,6 @@ describe("setup workflow", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("setup failure cleanup projection", () => {
|
|
||||||
it("projects a configuration failure together with failed host cleanup", async () => {
|
|
||||||
const configPath = "/fixture/home/vscode/settings.json";
|
|
||||||
const primary = new ConfigurationError("Client config could not be read", {
|
|
||||||
settings: [
|
|
||||||
{
|
|
||||||
setting: configPath,
|
|
||||||
constraint: "The selected config could not be read.",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
cleanup: {
|
|
||||||
reason: "The admitted config handle remains open.",
|
|
||||||
resources: [configPath],
|
|
||||||
},
|
|
||||||
});
|
|
||||||
class SetupHostWithCleanupFailure extends FakeSetupHost {
|
|
||||||
close = async (): Promise<string> =>
|
|
||||||
"Setup could not release its client configuration resources.";
|
|
||||||
}
|
|
||||||
const host = new SetupHostWithCleanupFailure();
|
|
||||||
host.clients = [{ name: "codex", configPath }];
|
|
||||||
host.inspectClientConfiguration = async () => {
|
|
||||||
throw primary;
|
|
||||||
};
|
|
||||||
|
|
||||||
const failure = await runSetup(
|
|
||||||
{ ...options(true), clientIds: ["codex"] },
|
|
||||||
host,
|
|
||||||
).then(
|
|
||||||
() => undefined,
|
|
||||||
(cause: unknown) => cause,
|
|
||||||
);
|
|
||||||
expect(failure).toBeInstanceOf(AnalysisError);
|
|
||||||
if (!(failure instanceof AnalysisError))
|
|
||||||
throw new Error("Expected setup to preserve a typed analysis failure");
|
|
||||||
|
|
||||||
const projected = projectAnalysisError(failure);
|
|
||||||
expect(projected).toMatchObject({
|
|
||||||
code: "cleanup_incomplete",
|
|
||||||
details: {
|
|
||||||
cleanup: "incomplete",
|
|
||||||
resources: [configPath],
|
|
||||||
diagnostics: {
|
|
||||||
primary_error: {
|
|
||||||
code: "cleanup_incomplete",
|
|
||||||
details: { resources: [configPath] },
|
|
||||||
},
|
|
||||||
cleanup_error: {
|
|
||||||
code: "cleanup_incomplete",
|
|
||||||
details: { resources: [configPath] },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
expect(projected.details).toMatchObject({
|
|
||||||
diagnostics: {
|
|
||||||
primary_error: { message: expect.stringContaining("selected config") },
|
|
||||||
cleanup_error: {
|
|
||||||
details: {
|
|
||||||
diagnostics: {
|
|
||||||
reason:
|
|
||||||
"Setup could not release its client configuration resources.",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("setup scoped readiness", () => {
|
describe("setup scoped readiness", () => {
|
||||||
it("limits Hopper host checks to workflows that request Hopper", async () => {
|
it("limits Hopper host checks to workflows that request Hopper", async () => {
|
||||||
const platformHost = new FakeSetupHost("win32");
|
const platformHost = new FakeSetupHost("win32");
|
||||||
|
|||||||
@@ -23,9 +23,6 @@ import {
|
|||||||
hostRemediation,
|
hostRemediation,
|
||||||
systemSetupHost,
|
systemSetupHost,
|
||||||
} from "./SetupHost.js";
|
} from "./SetupHost.js";
|
||||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
|
||||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
|
||||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
|
||||||
|
|
||||||
/** Discover, approve, and apply setup actions idempotently. */
|
/** Discover, approve, and apply setup actions idempotently. */
|
||||||
export const runSetup = async (
|
export const runSetup = async (
|
||||||
@@ -51,17 +48,6 @@ export const runSetup = async (
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (threw) {
|
if (threw) {
|
||||||
if (cleanupFailure !== undefined && primaryFailure instanceof AnalysisError)
|
|
||||||
throw analysisErrorWithCleanupFailure(
|
|
||||||
primaryFailure,
|
|
||||||
new ProviderCleanupError(
|
|
||||||
"setup",
|
|
||||||
primaryFailure.cleanupResources,
|
|
||||||
{ reason: cleanupFailure },
|
|
||||||
{ operation: "setup" },
|
|
||||||
),
|
|
||||||
"setup",
|
|
||||||
);
|
|
||||||
if (cleanupFailure !== undefined)
|
if (cleanupFailure !== undefined)
|
||||||
throw new AggregateError(
|
throw new AggregateError(
|
||||||
[primaryFailure, new Error(cleanupFailure)],
|
[primaryFailure, new Error(cleanupFailure)],
|
||||||
|
|||||||
@@ -11,22 +11,17 @@ import {
|
|||||||
serializeClientConfiguration,
|
serializeClientConfiguration,
|
||||||
withClientServers,
|
withClientServers,
|
||||||
type ClientConfigurationDocument,
|
type ClientConfigurationDocument,
|
||||||
|
type ClientRegistrationDialect,
|
||||||
} from "./ClientConfigurationDocument.js";
|
} from "./ClientConfigurationDocument.js";
|
||||||
import { constants as fsConstants } from "node:fs";
|
import { constants as fsConstants } from "node:fs";
|
||||||
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
|
import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||||
import { dirname } from "node:path";
|
import { dirname } from "node:path";
|
||||||
import writeFileAtomic from "write-file-atomic";
|
import writeFileAtomic from "write-file-atomic";
|
||||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
|
||||||
|
|
||||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||||
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||||
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
||||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||||
import {
|
|
||||||
readRegularFileText,
|
|
||||||
RegularFileCleanupFailure,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
import type { ClientConfigurationFileError } from "./ClientConfigurationFiles.js";
|
|
||||||
import type {
|
import type {
|
||||||
ClientConfigurationInspection,
|
ClientConfigurationInspection,
|
||||||
ClientConfigurationResult,
|
ClientConfigurationResult,
|
||||||
@@ -42,37 +37,25 @@ export const configureClientConfiguration = (
|
|||||||
): Promise<ClientConfigurationResult> => {
|
): Promise<ClientConfigurationResult> => {
|
||||||
if (client.configPathError !== undefined)
|
if (client.configPathError !== undefined)
|
||||||
return Promise.resolve({ status: "failed", reason: "path" });
|
return Promise.resolve({ status: "failed", reason: "path" });
|
||||||
return configureClientDocument(client, environment, command);
|
if (client.format === undefined || client.format === "unsupported")
|
||||||
|
return Promise.resolve({ status: "failed", reason: "readback" });
|
||||||
|
return configureClientDocument(client, environment, command, client.format);
|
||||||
};
|
};
|
||||||
|
|
||||||
const configureClientDocument = async (
|
const configureClientDocument = async (
|
||||||
client: SetupClient,
|
client: SetupClient,
|
||||||
environment: SetupProviderEnvironment,
|
environment: SetupProviderEnvironment,
|
||||||
command: readonly string[],
|
command: readonly string[],
|
||||||
|
format: NonNullable<SetupClient["format"]>,
|
||||||
): Promise<ClientConfigurationResult> => {
|
): Promise<ClientConfigurationResult> => {
|
||||||
const format = client.format;
|
|
||||||
if (format === undefined || format === "unsupported")
|
|
||||||
return { status: "failed", reason: "readback" };
|
|
||||||
const files = await readClientConfigurationFiles(client);
|
const files = await readClientConfigurationFiles(client);
|
||||||
if (!files.ok) {
|
if (!files.ok) return { status: "failed", reason: "readback" };
|
||||||
throwCleanupFailure(files.error);
|
|
||||||
return { status: "failed", reason: "readback" };
|
|
||||||
}
|
|
||||||
const policy = await readClientPolicyBlock(
|
const policy = await readClientPolicyBlock(
|
||||||
client,
|
client,
|
||||||
undefined,
|
undefined,
|
||||||
clientConfigurationDesired(
|
clientConfigurationDesired(client, environment, command, client.format),
|
||||||
client,
|
|
||||||
environment,
|
|
||||||
command,
|
|
||||||
files.value.at(-1),
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
if (!policy.ok) {
|
if (!policy.ok || policy.value !== undefined)
|
||||||
throwCleanupFailure(policy.error);
|
|
||||||
return { status: "failed", reason: "readback" };
|
|
||||||
}
|
|
||||||
if (policy.value !== undefined)
|
|
||||||
return { status: "failed", reason: "readback" };
|
return { status: "failed", reason: "readback" };
|
||||||
const transactionPath = await resolveClientConfigTransactionPath(
|
const transactionPath = await resolveClientConfigTransactionPath(
|
||||||
client.configPath,
|
client.configPath,
|
||||||
@@ -81,9 +64,8 @@ const configureClientDocument = async (
|
|||||||
return { status: "failed", reason: "path" };
|
return { status: "failed", reason: "path" };
|
||||||
let original: string | undefined;
|
let original: string | undefined;
|
||||||
try {
|
try {
|
||||||
original = await readRegularFileText(transactionPath);
|
original = await readFile(transactionPath, "utf8");
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
if (!isMissing(cause)) return { status: "failed", reason: "readback" };
|
if (!isMissing(cause)) return { status: "failed", reason: "readback" };
|
||||||
}
|
}
|
||||||
let parsed: ClientConfigurationDocument;
|
let parsed: ClientConfigurationDocument;
|
||||||
@@ -95,16 +77,11 @@ const configureClientDocument = async (
|
|||||||
const policy = await readClientPolicyBlock(
|
const policy = await readClientPolicyBlock(
|
||||||
client,
|
client,
|
||||||
parsed,
|
parsed,
|
||||||
clientConfigurationDesired(client, environment, command, parsed),
|
clientConfigurationDesired(client, environment, command, parsed.dialect),
|
||||||
);
|
);
|
||||||
if (!policy.ok) {
|
if (!policy.ok || policy.value !== undefined)
|
||||||
throwCleanupFailure(policy.error);
|
|
||||||
return { status: "failed", reason: "readback" };
|
|
||||||
}
|
|
||||||
if (policy.value !== undefined)
|
|
||||||
return { status: "failed", reason: "readback" };
|
return { status: "failed", reason: "readback" };
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
// Malformed existing configuration fails the readback gate.
|
// Malformed existing configuration fails the readback gate.
|
||||||
void cause;
|
void cause;
|
||||||
return { status: "failed", reason: "readback" };
|
return { status: "failed", reason: "readback" };
|
||||||
@@ -113,36 +90,9 @@ const configureClientDocument = async (
|
|||||||
client,
|
client,
|
||||||
environment,
|
environment,
|
||||||
command,
|
command,
|
||||||
parsed,
|
parsed.dialect,
|
||||||
);
|
);
|
||||||
if (registrationCurrent(parsed, desired)) return { status: "unchanged" };
|
if (registrationCurrent(parsed, desired)) return { status: "unchanged" };
|
||||||
return persistClientConfiguration({
|
|
||||||
client,
|
|
||||||
transactionPath,
|
|
||||||
format,
|
|
||||||
original,
|
|
||||||
parsed,
|
|
||||||
desired,
|
|
||||||
});
|
|
||||||
};
|
|
||||||
|
|
||||||
interface ClientConfigurationUpdate {
|
|
||||||
readonly client: SetupClient;
|
|
||||||
readonly transactionPath: string;
|
|
||||||
readonly format: NonNullable<SetupClient["format"]>;
|
|
||||||
readonly original: string | undefined;
|
|
||||||
readonly parsed: ClientConfigurationDocument;
|
|
||||||
readonly desired: unknown;
|
|
||||||
}
|
|
||||||
|
|
||||||
const persistClientConfiguration = async ({
|
|
||||||
client,
|
|
||||||
transactionPath,
|
|
||||||
format,
|
|
||||||
original,
|
|
||||||
parsed,
|
|
||||||
desired,
|
|
||||||
}: ClientConfigurationUpdate): Promise<ClientConfigurationResult> => {
|
|
||||||
const backupPath =
|
const backupPath =
|
||||||
original === undefined ? undefined : `${client.configPath}.rea.backup`;
|
original === undefined ? undefined : `${client.configPath}.rea.backup`;
|
||||||
if (
|
if (
|
||||||
@@ -199,7 +149,7 @@ const persistClientConfiguration = async ({
|
|||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const readback = parseClientConfiguration(
|
const readback = parseClientConfiguration(
|
||||||
await readRegularFileText(transactionPath),
|
await readFile(transactionPath, "utf8"),
|
||||||
format,
|
format,
|
||||||
);
|
);
|
||||||
if (!registrationCurrent(readback, desired)) {
|
if (!registrationCurrent(readback, desired)) {
|
||||||
@@ -207,7 +157,6 @@ const persistClientConfiguration = async ({
|
|||||||
return { status: "failed", reason: "readback" };
|
return { status: "failed", reason: "readback" };
|
||||||
}
|
}
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
// Readback failure restores the transaction before reporting.
|
// Readback failure restores the transaction before reporting.
|
||||||
void cause;
|
void cause;
|
||||||
await restoreConfig(transactionPath, original);
|
await restoreConfig(transactionPath, original);
|
||||||
@@ -228,22 +177,21 @@ export const clientConfigurationAligned = async (
|
|||||||
if (client.configPathError !== undefined) return false;
|
if (client.configPathError !== undefined) return false;
|
||||||
try {
|
try {
|
||||||
const files = await readClientConfigurationFiles(client);
|
const files = await readClientConfigurationFiles(client);
|
||||||
if (!files.ok) {
|
if (!files.ok) return false;
|
||||||
throwCleanupFailure(files.error);
|
const original = await readFile(client.configPath, "utf8");
|
||||||
return false;
|
|
||||||
}
|
|
||||||
const original = await readRegularFileText(client.configPath);
|
|
||||||
const parsed = parseClientConfiguration(original, client.format);
|
const parsed = parseClientConfiguration(original, client.format);
|
||||||
const policy = await readClientPolicyBlock(
|
const policy = await readClientPolicyBlock(
|
||||||
client,
|
client,
|
||||||
parsed,
|
parsed,
|
||||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
clientConfigurationDesired(
|
||||||
|
client,
|
||||||
|
providerEnvironment,
|
||||||
|
command,
|
||||||
|
parsed.dialect,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
if (!policy.ok) {
|
|
||||||
throwCleanupFailure(policy.error);
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
return (
|
return (
|
||||||
|
policy.ok &&
|
||||||
policy.value === undefined &&
|
policy.value === undefined &&
|
||||||
registrationCurrent(
|
registrationCurrent(
|
||||||
parsed,
|
parsed,
|
||||||
@@ -251,12 +199,11 @@ export const clientConfigurationAligned = async (
|
|||||||
client,
|
client,
|
||||||
providerEnvironment,
|
providerEnvironment,
|
||||||
command,
|
command,
|
||||||
parsed,
|
parsed.dialect,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
);
|
);
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
// Unreadable configuration is treated as not aligned so setup repairs it.
|
// Unreadable configuration is treated as not aligned so setup repairs it.
|
||||||
void cause;
|
void cause;
|
||||||
return false;
|
return false;
|
||||||
@@ -273,13 +220,7 @@ export const inspectClientConfiguration = async (
|
|||||||
return { status: "invalid", remediation: client.configPathError };
|
return { status: "invalid", remediation: client.configPathError };
|
||||||
if (client.format === "unsupported") return { status: "already_current" };
|
if (client.format === "unsupported") return { status: "already_current" };
|
||||||
const files = await readClientConfigurationFiles(client);
|
const files = await readClientConfigurationFiles(client);
|
||||||
if (!files.ok) {
|
if (!files.ok) return { status: "invalid", remediation: files.error.detail };
|
||||||
throwCleanupFailure(files.error);
|
|
||||||
return {
|
|
||||||
status: "invalid",
|
|
||||||
remediation: files.error.detail,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const policy = await readClientPolicyBlock(
|
const policy = await readClientPolicyBlock(
|
||||||
client,
|
client,
|
||||||
undefined,
|
undefined,
|
||||||
@@ -287,16 +228,11 @@ export const inspectClientConfiguration = async (
|
|||||||
client,
|
client,
|
||||||
providerEnvironment,
|
providerEnvironment,
|
||||||
command,
|
command,
|
||||||
files.value.at(-1),
|
client.format,
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
if (!policy.ok) {
|
if (!policy.ok)
|
||||||
throwCleanupFailure(policy.error);
|
return { status: "invalid", remediation: policy.error.detail };
|
||||||
return {
|
|
||||||
status: "invalid",
|
|
||||||
remediation: policy.error.detail,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (policy.value !== undefined)
|
if (policy.value !== undefined)
|
||||||
return { status: "invalid", remediation: policy.value };
|
return { status: "invalid", remediation: policy.value };
|
||||||
const transactionPath = await resolveClientConfigTransactionPath(
|
const transactionPath = await resolveClientConfigTransactionPath(
|
||||||
@@ -310,13 +246,13 @@ export const inspectClientConfiguration = async (
|
|||||||
};
|
};
|
||||||
let original: string;
|
let original: string;
|
||||||
try {
|
try {
|
||||||
original = await readRegularFileText(transactionPath);
|
original = await readFile(transactionPath, "utf8");
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
if (isMissing(cause)) return { status: "create" };
|
if (isMissing(cause)) return { status: "create" };
|
||||||
return {
|
return {
|
||||||
status: "invalid",
|
status: "invalid",
|
||||||
remediation: `The configuration file could not be read. Check its permissions before rerunning setup. ${cause instanceof Error ? cause.message : String(cause)}`,
|
remediation:
|
||||||
|
"The configuration file could not be read. Check its permissions before rerunning setup.",
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
@@ -324,27 +260,26 @@ export const inspectClientConfiguration = async (
|
|||||||
const policy = await readClientPolicyBlock(
|
const policy = await readClientPolicyBlock(
|
||||||
client,
|
client,
|
||||||
parsed,
|
parsed,
|
||||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
clientConfigurationDesired(
|
||||||
|
client,
|
||||||
|
providerEnvironment,
|
||||||
|
command,
|
||||||
|
parsed.dialect,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
if (!policy.ok) {
|
if (!policy.ok)
|
||||||
throwCleanupFailure(policy.error);
|
return { status: "invalid", remediation: policy.error.detail };
|
||||||
return {
|
|
||||||
status: "invalid",
|
|
||||||
remediation: policy.error.detail,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if (policy.value !== undefined)
|
if (policy.value !== undefined)
|
||||||
return { status: "invalid", remediation: policy.value };
|
return { status: "invalid", remediation: policy.value };
|
||||||
const desired = clientConfigurationDesired(
|
const desired = clientConfigurationDesired(
|
||||||
client,
|
client,
|
||||||
providerEnvironment,
|
providerEnvironment,
|
||||||
command,
|
command,
|
||||||
parsed,
|
parsed.dialect,
|
||||||
);
|
);
|
||||||
if (registrationCurrent(parsed, desired))
|
if (registrationCurrent(parsed, desired))
|
||||||
return { status: "already_current" };
|
return { status: "already_current" };
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
|
||||||
// Malformed configuration is reported with the invalid remediation.
|
// Malformed configuration is reported with the invalid remediation.
|
||||||
void cause;
|
void cause;
|
||||||
return {
|
return {
|
||||||
@@ -361,11 +296,6 @@ export const inspectClientConfiguration = async (
|
|||||||
|
|
||||||
const isMissing = (cause: unknown): boolean =>
|
const isMissing = (cause: unknown): boolean =>
|
||||||
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
|
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
|
||||||
|
|
||||||
const throwCleanupFailure = (error: ClientConfigurationFileError): void => {
|
|
||||||
if (error.cleanupFailure !== undefined) throw error.cleanupFailure;
|
|
||||||
};
|
|
||||||
|
|
||||||
const preserveConfigBackup = async (
|
const preserveConfigBackup = async (
|
||||||
source: string,
|
source: string,
|
||||||
destination: string,
|
destination: string,
|
||||||
@@ -407,20 +337,17 @@ const clientConfigurationDesired = (
|
|||||||
client: SetupClient,
|
client: SetupClient,
|
||||||
providerEnvironment: SetupProviderEnvironment,
|
providerEnvironment: SetupProviderEnvironment,
|
||||||
command: readonly string[],
|
command: readonly string[],
|
||||||
parsed?: ClientConfigurationDocument,
|
format: ClientRegistrationDialect | undefined,
|
||||||
) => {
|
) => {
|
||||||
const format = parsed?.dialect ?? client.format ?? "json";
|
const environment = Object.fromEntries(
|
||||||
// Preserve explicitly configured server settings, never the ambient process
|
Object.entries(providerEnvironment).sort(([left], [right]) =>
|
||||||
// environment. Newly discovered provider paths win only for their own keys.
|
left.localeCompare(right),
|
||||||
const environment = clientRegistrationEnvironment(
|
),
|
||||||
parsed,
|
|
||||||
providerEnvironment,
|
|
||||||
);
|
);
|
||||||
if (!environment.ok) throw environment.error;
|
|
||||||
const registration = clientRegistrationEntry(
|
const registration = clientRegistrationEntry(
|
||||||
format,
|
format ?? "json",
|
||||||
command.length === 0 ? [PRODUCT_IDENTITY.cliBinary, "mcp"] : command,
|
command.length === 0 ? [PRODUCT_IDENTITY.cliBinary, "mcp"] : command,
|
||||||
environment.value,
|
environment,
|
||||||
);
|
);
|
||||||
return {
|
return {
|
||||||
...registration,
|
...registration,
|
||||||
|
|||||||
+12
-110
@@ -1,9 +1,6 @@
|
|||||||
import { access } from "node:fs/promises";
|
import { access } from "node:fs/promises";
|
||||||
import { resolve } from "node:path";
|
import { resolve } from "node:path";
|
||||||
|
|
||||||
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
|
||||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
|
||||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||||
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||||
import {
|
import {
|
||||||
@@ -33,10 +30,6 @@ import {
|
|||||||
inspectClientConfiguration,
|
inspectClientConfiguration,
|
||||||
} from "./SetupClientConfiguration.js";
|
} from "./SetupClientConfiguration.js";
|
||||||
import { setupInstallFailure } from "./SetupInstallFailure.js";
|
import { setupInstallFailure } from "./SetupInstallFailure.js";
|
||||||
import {
|
|
||||||
RegularFileCleanupFailure,
|
|
||||||
retryRegularFileCleanup,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
import { providerRegistrationEnvironment } from "./SetupRegistrationEnvironment.js";
|
import { providerRegistrationEnvironment } from "./SetupRegistrationEnvironment.js";
|
||||||
import type {
|
import type {
|
||||||
SetupHost,
|
SetupHost,
|
||||||
@@ -105,7 +98,6 @@ export const systemSetupHost = (
|
|||||||
platform === "darwin"
|
platform === "darwin"
|
||||||
? systemMacHopperInstallHost(homeDirectory, environment)
|
? systemMacHopperInstallHost(homeDirectory, environment)
|
||||||
: undefined;
|
: undefined;
|
||||||
const resources = new ArtifactResourceScope();
|
|
||||||
return {
|
return {
|
||||||
platform,
|
platform,
|
||||||
homeDirectory,
|
homeDirectory,
|
||||||
@@ -118,7 +110,7 @@ export const systemSetupHost = (
|
|||||||
nodeVersion: process.versions.node,
|
nodeVersion: process.versions.node,
|
||||||
macosVersion: () => doctorHost.macosVersion(),
|
macosVersion: () => doctorHost.macosVersion(),
|
||||||
linuxDistribution: readLinuxDistribution,
|
linuxDistribution: readLinuxDistribution,
|
||||||
close: () => closeSetupResources(resources, macHopperHost),
|
close: () => macHopperHost?.close() ?? Promise.resolve(undefined),
|
||||||
initialSetupState: async (
|
initialSetupState: async (
|
||||||
scope?: DoctorScope,
|
scope?: DoctorScope,
|
||||||
): Promise<SetupInitialState> => {
|
): Promise<SetupInitialState> => {
|
||||||
@@ -170,7 +162,17 @@ export const systemSetupHost = (
|
|||||||
detectedClients: () => detectClients(homeDirectory, platform, environment),
|
detectedClients: () => detectClients(homeDirectory, platform, environment),
|
||||||
supportedClients: () =>
|
supportedClients: () =>
|
||||||
Promise.resolve(supportedClients(homeDirectory, platform, environment)),
|
Promise.resolve(supportedClients(homeDirectory, platform, environment)),
|
||||||
...clientConfigurationOperations(resources),
|
configureClient: (client, providerEnvironment, command) =>
|
||||||
|
client.format === "unsupported"
|
||||||
|
? Promise.resolve({ status: "skipped" })
|
||||||
|
: configureClientConfiguration(client, providerEnvironment, command),
|
||||||
|
clientNeedsConfigure: (client, providerEnvironment, command) =>
|
||||||
|
client.format === "unsupported"
|
||||||
|
? Promise.resolve(false)
|
||||||
|
: clientConfigurationAligned(client, providerEnvironment, command).then(
|
||||||
|
(aligned) => !aligned,
|
||||||
|
),
|
||||||
|
inspectClientConfiguration: inspectClientConfiguration,
|
||||||
skillNeedsInstall: (clientIds) =>
|
skillNeedsInstall: (clientIds) =>
|
||||||
canonicalSkillNeedsInstall(
|
canonicalSkillNeedsInstall(
|
||||||
homeDirectory,
|
homeDirectory,
|
||||||
@@ -184,106 +186,6 @@ export const systemSetupHost = (
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
|
|
||||||
configureClient: (
|
|
||||||
client: SetupClient,
|
|
||||||
providerEnvironment: SetupProviderEnvironment,
|
|
||||||
command: readonly string[],
|
|
||||||
) =>
|
|
||||||
runClientConfigurationOperation(resources, async () =>
|
|
||||||
client.format === "unsupported"
|
|
||||||
? { status: "skipped" as const }
|
|
||||||
: configureClientConfiguration(client, providerEnvironment, command),
|
|
||||||
),
|
|
||||||
clientNeedsConfigure: (
|
|
||||||
client: SetupClient,
|
|
||||||
providerEnvironment: SetupProviderEnvironment,
|
|
||||||
command: readonly string[],
|
|
||||||
) =>
|
|
||||||
runClientConfigurationOperation(resources, async () =>
|
|
||||||
client.format === "unsupported"
|
|
||||||
? false
|
|
||||||
: !(await clientConfigurationAligned(
|
|
||||||
client,
|
|
||||||
providerEnvironment,
|
|
||||||
command,
|
|
||||||
)),
|
|
||||||
),
|
|
||||||
inspectClientConfiguration: (
|
|
||||||
client: SetupClient,
|
|
||||||
providerEnvironment: SetupProviderEnvironment,
|
|
||||||
command: readonly string[],
|
|
||||||
) =>
|
|
||||||
runClientConfigurationOperation(resources, () =>
|
|
||||||
inspectClientConfiguration(client, providerEnvironment, command),
|
|
||||||
),
|
|
||||||
});
|
|
||||||
|
|
||||||
const closeSetupResources = async (
|
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
macHopperHost: ReturnType<typeof systemMacHopperInstallHost> | undefined,
|
|
||||||
): Promise<string | undefined> => {
|
|
||||||
const [resourceCleanup, hopperCleanup] = await Promise.allSettled([
|
|
||||||
resources.close(),
|
|
||||||
Promise.resolve().then(() => macHopperHost?.close()),
|
|
||||||
]);
|
|
||||||
const failures: string[] = [];
|
|
||||||
if (hopperCleanup.status === "rejected")
|
|
||||||
failures.push(
|
|
||||||
`Hopper host cleanup failed: ${errorMessage(hopperCleanup.reason)}`,
|
|
||||||
);
|
|
||||||
else if (hopperCleanup.value !== undefined)
|
|
||||||
failures.push(hopperCleanup.value);
|
|
||||||
if (resourceCleanup.status === "rejected")
|
|
||||||
failures.push(errorMessage(resourceCleanup.reason));
|
|
||||||
return failures.length === 0 ? undefined : failures.join("; ");
|
|
||||||
};
|
|
||||||
|
|
||||||
const runClientConfigurationOperation = async <Value>(
|
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
operation: () => Promise<Value>,
|
|
||||||
): Promise<Value> => {
|
|
||||||
try {
|
|
||||||
return await resources.run(async () => {
|
|
||||||
try {
|
|
||||||
return await operation();
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
|
|
||||||
const cleanup = await retryRegularFileCleanup(cause, resources);
|
|
||||||
const primary =
|
|
||||||
cleanup.outcome.kind === "failed"
|
|
||||||
? cleanup.outcome.cause
|
|
||||||
: cause.cleanupCause;
|
|
||||||
throw configurationCleanupError(
|
|
||||||
`Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}`,
|
|
||||||
cause,
|
|
||||||
cleanup.cleanup,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (cause instanceof ArtifactReaderFailure)
|
|
||||||
throw configurationCleanupError(cause.message, cause, cause.cleanup);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const configurationCleanupError = (
|
|
||||||
message: string,
|
|
||||||
cause: unknown,
|
|
||||||
cleanup:
|
|
||||||
| { readonly reason: string; readonly resources: readonly string[] }
|
|
||||||
| undefined,
|
|
||||||
): ConfigurationError =>
|
|
||||||
new ConfigurationError(message, {
|
|
||||||
cause,
|
|
||||||
settings: [{ setting: "client configuration", constraint: message }],
|
|
||||||
...(cleanup === undefined ? {} : { cleanup }),
|
|
||||||
});
|
|
||||||
|
|
||||||
const errorMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|
||||||
/** Detect supported agents from their config files or stable installation markers. */
|
/** Detect supported agents from their config files or stable installation markers. */
|
||||||
export const detectClients = async (
|
export const detectClients = async (
|
||||||
home: string,
|
home: string,
|
||||||
|
|||||||
@@ -312,8 +312,8 @@ const clientConfigurationDetail = (
|
|||||||
.map(([name, value]) => `${name}=${value}`);
|
.map(([name, value]) => `${name}=${value}`);
|
||||||
const environmentDetail =
|
const environmentDetail =
|
||||||
entries.length === 0
|
entries.length === 0
|
||||||
? " No additional provider environment variables. Existing server environment overrides are retained."
|
? " No additional environment variables."
|
||||||
: ` Detected provider environment: ${entries.join(", ")}. Existing server environment overrides are retained; detected values replace matching keys.`;
|
: ` Environment: ${entries.join(", ")}.`;
|
||||||
const hopperDetail = installHopper
|
const hopperDetail = installHopper
|
||||||
? environment.HOPPER_LAUNCHER_PATH === undefined
|
? environment.HOPPER_LAUNCHER_PATH === undefined
|
||||||
? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation."
|
? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation."
|
||||||
|
|||||||
@@ -4,13 +4,14 @@ import { dirname, join } from "node:path";
|
|||||||
import writeFileAtomic from "write-file-atomic";
|
import writeFileAtomic from "write-file-atomic";
|
||||||
|
|
||||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||||
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
import {
|
||||||
import { readRegularFileText } from "./RegularFileRead.js";
|
NonRegularFileReadError,
|
||||||
|
openRegularFile,
|
||||||
|
} from "../filesystem/RegularFile.js";
|
||||||
import { clientSkillDirectories } from "./SupportedClients.js";
|
import { clientSkillDirectories } from "./SupportedClients.js";
|
||||||
|
|
||||||
export const MANAGED_SKILL_FILES = [
|
export const MANAGED_SKILL_FILES = [
|
||||||
"SKILL.md",
|
"SKILL.md",
|
||||||
"references/connection-and-recovery.md",
|
|
||||||
"references/native-and-artifacts.md",
|
"references/native-and-artifacts.md",
|
||||||
"references/javascript-applications.md",
|
"references/javascript-applications.md",
|
||||||
"references/android-applications.md",
|
"references/android-applications.md",
|
||||||
@@ -22,8 +23,14 @@ export const isManagedSkillManifest = (content: string): boolean =>
|
|||||||
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
content.startsWith(`---\nname: ${PRODUCT_IDENTITY.skillName}\n`) ||
|
||||||
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
content.startsWith(`---\r\nname: ${PRODUCT_IDENTITY.skillName}\r\n`);
|
||||||
|
|
||||||
export const readSkillFile = (path: string): Promise<string> =>
|
export const readSkillFile = async (path: string): Promise<string> => {
|
||||||
readRegularFileText(path, { symlinks: "reject" });
|
const handle = await openRegularFile(path, { symlinks: "reject" });
|
||||||
|
try {
|
||||||
|
return await handle.readFile("utf8");
|
||||||
|
} finally {
|
||||||
|
await handle.close();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
interface CanonicalSkillFile {
|
interface CanonicalSkillFile {
|
||||||
readonly content: string;
|
readonly content: string;
|
||||||
@@ -192,7 +199,7 @@ export const installCanonicalSkill = async (
|
|||||||
for (const { destination, content } of changed)
|
for (const { destination, content } of changed)
|
||||||
await writeText(destination, content);
|
await writeText(destination, content);
|
||||||
for (const { destination, content } of changed)
|
for (const { destination, content } of changed)
|
||||||
if ((await readSkillFile(destination)) !== content)
|
if ((await readFile(destination, "utf8")) !== content)
|
||||||
throw new Error(`skill readback mismatch: ${destination}`);
|
throw new Error(`skill readback mismatch: ${destination}`);
|
||||||
return "installed";
|
return "installed";
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
@@ -224,7 +231,7 @@ export const readInstalledSkillIdentity = async (
|
|||||||
let content: string | undefined;
|
let content: string | undefined;
|
||||||
for (const destination of destinations) {
|
for (const destination of destinations) {
|
||||||
try {
|
try {
|
||||||
content = await readSkillFile(join(destination.path, "SKILL.md"));
|
content = await readFile(join(destination.path, "SKILL.md"), "utf8");
|
||||||
break;
|
break;
|
||||||
} catch {
|
} catch {
|
||||||
// Another selected copy may provide metadata; canonical validation below
|
// Another selected copy may provide metadata; canonical validation below
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ import {
|
|||||||
withClientServers,
|
withClientServers,
|
||||||
type ClientConfigurationDocument,
|
type ClientConfigurationDocument,
|
||||||
} from "./ClientConfigurationDocument.js";
|
} from "./ClientConfigurationDocument.js";
|
||||||
import { copyFile, lstat, realpath, rm } from "node:fs/promises";
|
import { copyFile, lstat, readFile, realpath, rm } from "node:fs/promises";
|
||||||
import { constants as fsConstants } from "node:fs";
|
import { constants as fsConstants } from "node:fs";
|
||||||
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
@@ -23,7 +23,6 @@ import {
|
|||||||
} from "./SetupSkill.js";
|
} from "./SetupSkill.js";
|
||||||
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
|
||||||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||||
import { readRegularFileText } from "./RegularFileRead.js";
|
|
||||||
import {
|
import {
|
||||||
manualRegistrationRemediation,
|
manualRegistrationRemediation,
|
||||||
supportedClients,
|
supportedClients,
|
||||||
@@ -50,7 +49,7 @@ export interface UninstallFileSystem {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const systemFileSystem: UninstallFileSystem = {
|
const systemFileSystem: UninstallFileSystem = {
|
||||||
readText: (path) => readRegularFileText(path),
|
readText: (path) => readFile(path, "utf8"),
|
||||||
readRegularText: readSkillFile,
|
readRegularText: readSkillFile,
|
||||||
copy: (source, destination) =>
|
copy: (source, destination) =>
|
||||||
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
|
copyFile(source, destination, fsConstants.COPYFILE_EXCL),
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { readFile } from "node:fs/promises";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||||
@@ -13,7 +14,6 @@ import { supportedClients } from "./SupportedClients.js";
|
|||||||
import { existingSkillDestinations } from "./SetupSkill.js";
|
import { existingSkillDestinations } from "./SetupSkill.js";
|
||||||
import type { SetupAction } from "./SetupTypes.js";
|
import type { SetupAction } from "./SetupTypes.js";
|
||||||
import type { Result } from "../domain/result.js";
|
import type { Result } from "../domain/result.js";
|
||||||
import { readRegularFileText } from "./RegularFileRead.js";
|
|
||||||
|
|
||||||
/** Existing REA integrations selected for maintenance, without discovering new targets. */
|
/** Existing REA integrations selected for maintenance, without discovering new targets. */
|
||||||
export interface MaintenanceScope {
|
export interface MaintenanceScope {
|
||||||
@@ -73,7 +73,7 @@ export const existingMaintenanceScope = async (
|
|||||||
const client = supported.find(({ name }) => name === registration.client);
|
const client = supported.find(({ name }) => name === registration.client);
|
||||||
if (client === undefined) continue;
|
if (client === undefined) continue;
|
||||||
const parsed = parseClientConfiguration(
|
const parsed = parseClientConfiguration(
|
||||||
await readRegularFileText(client.configPath),
|
await readFile(client.configPath, "utf8"),
|
||||||
client.format,
|
client.format,
|
||||||
);
|
);
|
||||||
const enabled = z
|
const enabled = z
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ import { isAbsolute } from "node:path";
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
|
||||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
|
||||||
import { publishWebScripts } from "../browser/assets/PublishWebScripts.js";
|
import { publishWebScripts } from "../browser/assets/PublishWebScripts.js";
|
||||||
import { selectScriptCapture } from "../browser/assets/ScriptCaptureAdapters.js";
|
import { selectScriptCapture } from "../browser/assets/ScriptCaptureAdapters.js";
|
||||||
import {
|
import {
|
||||||
@@ -15,11 +14,9 @@ import {
|
|||||||
} from "../domain/analysisErrorCore.js";
|
} from "../domain/analysisErrorCore.js";
|
||||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||||
import { createEvidence, type Evidence } from "../domain/evidence.js";
|
import { createEvidence, type Evidence } from "../domain/evidence.js";
|
||||||
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
|
|
||||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import { safeParseJson } from "../domain/safeJson.js";
|
import { safeParseJson } from "../domain/safeJson.js";
|
||||||
import {
|
import {
|
||||||
@@ -29,41 +26,19 @@ import {
|
|||||||
import { WebScriptExportError } from "../domain/webScriptExportError.js";
|
import { WebScriptExportError } from "../domain/webScriptExportError.js";
|
||||||
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
||||||
import { WEB_SCRIPT_EXPORT_PROVIDER as PROVIDER } from "./InvestigationProviders.js";
|
import { WEB_SCRIPT_EXPORT_PROVIDER as PROVIDER } from "./InvestigationProviders.js";
|
||||||
import {
|
import { readRegularFile } from "./RegularFileRead.js";
|
||||||
readRegularFile,
|
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
|
||||||
RegularFileCleanupFailure,
|
|
||||||
retryRegularFileCleanup,
|
|
||||||
} from "./RegularFileRead.js";
|
|
||||||
import {
|
|
||||||
NonRegularFileReadError,
|
|
||||||
RegularFileChangedError,
|
|
||||||
} from "../filesystem/RegularFile.js";
|
|
||||||
|
|
||||||
const OPERATION = "export_web_scripts";
|
const OPERATION = "export_web_scripts";
|
||||||
|
|
||||||
interface CaptureRead {
|
|
||||||
readonly bytes: Buffer;
|
|
||||||
readonly cleanup?: {
|
|
||||||
readonly reason: string;
|
|
||||||
readonly incomplete: boolean;
|
|
||||||
readonly cause: unknown;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
interface PreparedCapture {
|
|
||||||
readonly bytes: Buffer;
|
|
||||||
readonly loaded: ReturnType<typeof selectScriptCapture>;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Export one local capture through the shared CLI/MCP application workflow. */
|
/** Export one local capture through the shared CLI/MCP application workflow. */
|
||||||
export const exportWebScripts = async (
|
export const exportWebScripts = async (
|
||||||
rawInput: unknown,
|
rawInput: unknown,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
options: ExecutionOptions = {},
|
options: ExecutionOptions = {},
|
||||||
): Promise<Result<Evidence, AnalysisError>> => {
|
): Promise<Result<Evidence, AnalysisError>> => {
|
||||||
const input = exportWebScriptsInputSchema.safeParse(rawInput);
|
const input = exportWebScriptsInputSchema.safeParse(rawInput);
|
||||||
return input.success
|
return input.success
|
||||||
? exportWebScriptsValidated(input.data, resources, options)
|
? exportWebScriptsValidated(input.data, options)
|
||||||
: err(
|
: err(
|
||||||
analysisInputErrorFromIssues(OPERATION, input.error.issues, rawInput),
|
analysisInputErrorFromIssues(OPERATION, input.error.issues, rawInput),
|
||||||
);
|
);
|
||||||
@@ -72,7 +47,6 @@ export const exportWebScripts = async (
|
|||||||
/** Publish input already parsed by a named adapter contract. */
|
/** Publish input already parsed by a named adapter contract. */
|
||||||
export const exportWebScriptsValidated = async (
|
export const exportWebScriptsValidated = async (
|
||||||
input: ExportWebScriptsInput,
|
input: ExportWebScriptsInput,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
options: ExecutionOptions = {},
|
options: ExecutionOptions = {},
|
||||||
): Promise<Result<Evidence, AnalysisError>> => {
|
): Promise<Result<Evidence, AnalysisError>> => {
|
||||||
if (!isAbsolute(input.capture_path) || !isAbsolute(input.output_directory))
|
if (!isAbsolute(input.capture_path) || !isAbsolute(input.output_directory))
|
||||||
@@ -88,19 +62,19 @@ export const exportWebScriptsValidated = async (
|
|||||||
);
|
);
|
||||||
try {
|
try {
|
||||||
options.signal?.throwIfAborted();
|
options.signal?.throwIfAborted();
|
||||||
const prepared = await prepareCapture(
|
const read = await readCapture(input.capture_path, options.signal);
|
||||||
input.capture_path,
|
if (!read.ok) return read;
|
||||||
resources,
|
const bytes = read.value;
|
||||||
options.signal,
|
const loaded = parseCapture(bytes);
|
||||||
);
|
if (!loaded.ok) return loaded;
|
||||||
if (!prepared.ok) return prepared;
|
|
||||||
const { bytes, loaded } = prepared.value;
|
|
||||||
options.signal?.throwIfAborted();
|
options.signal?.throwIfAborted();
|
||||||
const sha256 = createHash("sha256").update(bytes).digest("hex");
|
const sha256 = createHash("sha256").update(bytes).digest("hex");
|
||||||
const result = await publishWebScripts(input, loaded, sha256, {
|
const result = await publishWebScripts(
|
||||||
resources,
|
input,
|
||||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
loaded.value,
|
||||||
});
|
sha256,
|
||||||
|
options.signal,
|
||||||
|
);
|
||||||
return ok(
|
return ok(
|
||||||
createEvidence(
|
createEvidence(
|
||||||
{ path: input.capture_path, sha256, format: "file" },
|
{ path: input.capture_path, sha256, format: "file" },
|
||||||
@@ -119,109 +93,55 @@ export const exportWebScriptsValidated = async (
|
|||||||
limitations: result.limitations,
|
limitations: result.limitations,
|
||||||
locations: [{ kind: "artifact-path", path: result.manifest.path }],
|
locations: [{ kind: "artifact-path", path: result.manifest.path }],
|
||||||
evidenceLinks:
|
evidenceLinks:
|
||||||
loaded.sourceEvidenceId === null ? [] : [loaded.sourceEvidenceId],
|
loaded.value.sourceEvidenceId === null
|
||||||
|
? []
|
||||||
|
: [loaded.value.sourceEvidenceId],
|
||||||
},
|
},
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
return err(mapExportFailure(cause, input, options.signal));
|
if (cause instanceof WebScriptExportError) return err(cause);
|
||||||
}
|
if (options.signal?.aborted === true)
|
||||||
};
|
return err(new AnalysisCancelledError(OPERATION));
|
||||||
|
if (cause instanceof ArtifactReaderFailure)
|
||||||
const prepareCapture = async (
|
return err(
|
||||||
path: string,
|
new WebScriptExportError(
|
||||||
resources: ArtifactResourceScope,
|
cause.reason,
|
||||||
signal: AbortSignal | undefined,
|
input.output_directory,
|
||||||
): Promise<Result<PreparedCapture, AnalysisError>> => {
|
cause.message,
|
||||||
const read = await resources.run(() => readCapture(path, resources, signal));
|
),
|
||||||
if (!read.ok) return read;
|
);
|
||||||
const cleanup = read.value.cleanup;
|
if (cause instanceof z.ZodError)
|
||||||
const loaded = parseCapture(
|
return err(new AnalysisOutputError(OPERATION, cause.message, { cause }));
|
||||||
read.value.bytes,
|
if (
|
||||||
cleanup === undefined || cleanup.incomplete
|
cause instanceof Error &&
|
||||||
? undefined
|
"code" in cause &&
|
||||||
: { cause: cleanup.cause },
|
typeof cause.code === "string"
|
||||||
);
|
)
|
||||||
if (!loaded.ok)
|
return err(
|
||||||
|
new WebScriptExportError(
|
||||||
|
"io",
|
||||||
|
`${input.capture_path} → ${input.output_directory}`,
|
||||||
|
cause.message,
|
||||||
|
),
|
||||||
|
);
|
||||||
return err(
|
return err(
|
||||||
cleanup?.incomplete === true
|
new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
||||||
? incompleteCaptureCleanup(loaded.error, path, cleanup)
|
cause,
|
||||||
: loaded.error,
|
diagnostics: {
|
||||||
);
|
capture_path: input.capture_path,
|
||||||
if (cleanup !== undefined) {
|
output_directory: input.output_directory,
|
||||||
const primary = new WebScriptExportError(
|
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
||||||
"io",
|
error_message:
|
||||||
path,
|
cause instanceof Error
|
||||||
`Capture handle close failed: ${cleanup.reason}`,
|
? cause.message
|
||||||
{ cause: cleanup.cause },
|
: typeof cause === "string"
|
||||||
);
|
? cause
|
||||||
return err(
|
: "Unknown script export failure",
|
||||||
cleanup.incomplete
|
},
|
||||||
? incompleteCaptureCleanup(primary, path, cleanup)
|
}),
|
||||||
: primary,
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
return ok({ bytes: read.value.bytes, loaded: loaded.value });
|
|
||||||
};
|
|
||||||
|
|
||||||
const mapExportFailure = (
|
|
||||||
cause: unknown,
|
|
||||||
input: ExportWebScriptsInput,
|
|
||||||
signal: AbortSignal | undefined,
|
|
||||||
): AnalysisError => {
|
|
||||||
if (cause instanceof WebScriptExportError) return cause;
|
|
||||||
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined)
|
|
||||||
return analysisErrorWithCleanupFailure(
|
|
||||||
new WebScriptExportError(
|
|
||||||
cause.reason,
|
|
||||||
input.output_directory,
|
|
||||||
cause.message,
|
|
||||||
{ cause },
|
|
||||||
),
|
|
||||||
new ProviderCleanupError(
|
|
||||||
PROVIDER.id,
|
|
||||||
cause.cleanup.resources,
|
|
||||||
{ reason: cause.cleanup.reason },
|
|
||||||
{ cause, operation: OPERATION },
|
|
||||||
),
|
|
||||||
OPERATION,
|
|
||||||
);
|
|
||||||
if (signal?.aborted === true)
|
|
||||||
return new AnalysisCancelledError(OPERATION, { cause });
|
|
||||||
if (cause instanceof ArtifactReaderFailure)
|
|
||||||
return new WebScriptExportError(
|
|
||||||
cause.reason,
|
|
||||||
input.output_directory,
|
|
||||||
cause.message,
|
|
||||||
{ cause },
|
|
||||||
);
|
|
||||||
if (cause instanceof z.ZodError)
|
|
||||||
return new AnalysisOutputError(OPERATION, cause.message, { cause });
|
|
||||||
if (
|
|
||||||
cause instanceof Error &&
|
|
||||||
"code" in cause &&
|
|
||||||
typeof cause.code === "string"
|
|
||||||
)
|
|
||||||
return new WebScriptExportError(
|
|
||||||
"io",
|
|
||||||
`${input.capture_path} → ${input.output_directory}`,
|
|
||||||
cause.message,
|
|
||||||
{ cause },
|
|
||||||
);
|
|
||||||
return new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
|
||||||
cause,
|
|
||||||
diagnostics: {
|
|
||||||
capture_path: input.capture_path,
|
|
||||||
output_directory: input.output_directory,
|
|
||||||
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
|
||||||
error_message:
|
|
||||||
cause instanceof Error
|
|
||||||
? cause.message
|
|
||||||
: typeof cause === "string"
|
|
||||||
? cause
|
|
||||||
: "Unknown script export failure",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -230,115 +150,44 @@ const mapExportFailure = (
|
|||||||
*/
|
*/
|
||||||
const readCapture = async (
|
const readCapture = async (
|
||||||
path: string,
|
path: string,
|
||||||
resources: ArtifactResourceScope,
|
|
||||||
signal: AbortSignal | undefined,
|
signal: AbortSignal | undefined,
|
||||||
): Promise<Result<CaptureRead, AnalysisError>> => {
|
): Promise<Result<Buffer, AnalysisError>> => {
|
||||||
try {
|
try {
|
||||||
return ok({ bytes: await readRegularFile(path, { signal }) });
|
return ok(await readRegularFile(path, signal));
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
if (cause instanceof RegularFileCleanupFailure) {
|
const code =
|
||||||
const retry = await retryRegularFileCleanup(cause, resources);
|
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
||||||
const cleanup = {
|
if (code === "EACCES" || code === "EPERM")
|
||||||
reason: retry.cleanup?.reason ?? errorMessage(cause.cleanupCause),
|
|
||||||
incomplete: retry.cleanup !== undefined,
|
|
||||||
cause: cause.cleanupCause,
|
|
||||||
};
|
|
||||||
if (cause.outcome.kind === "completed")
|
|
||||||
return ok({ bytes: cause.outcome.value as Buffer, cleanup });
|
|
||||||
const primary = captureReadFailure(
|
|
||||||
path,
|
|
||||||
cause.outcome.cause,
|
|
||||||
signal,
|
|
||||||
retry.cleanup === undefined ? { cause: cause.cleanupCause } : undefined,
|
|
||||||
);
|
|
||||||
return err(
|
return err(
|
||||||
retry.cleanup === undefined
|
new AnalysisAccessDeniedError(OPERATION, path, code, { cause }),
|
||||||
? primary
|
|
||||||
: incompleteCaptureCleanup(primary, path, cleanup),
|
|
||||||
);
|
);
|
||||||
}
|
if (
|
||||||
return err(captureReadFailure(path, cause, signal));
|
cause instanceof NonRegularFileReadError ||
|
||||||
|
code === "ENOENT" ||
|
||||||
|
code === "ENOTDIR" ||
|
||||||
|
code === "EISDIR" ||
|
||||||
|
code === "ENXIO"
|
||||||
|
)
|
||||||
|
return err(
|
||||||
|
new AnalysisInputError(OPERATION, { cause }, [
|
||||||
|
{
|
||||||
|
path: ["capture_path"],
|
||||||
|
reason: "invalid_value",
|
||||||
|
message:
|
||||||
|
cause instanceof NonRegularFileReadError && code !== "EISDIR"
|
||||||
|
? cause.message
|
||||||
|
: code === "EISDIR"
|
||||||
|
? `Selected capture is a directory, not a file: ${path}`
|
||||||
|
: `Selected capture could not be read (${code}): ${path}`,
|
||||||
|
},
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
throw cause;
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const captureReadFailure = (
|
|
||||||
path: string,
|
|
||||||
cause: unknown,
|
|
||||||
signal: AbortSignal | undefined,
|
|
||||||
closeFailure?: { readonly cause: unknown },
|
|
||||||
): AnalysisError => {
|
|
||||||
const causeWithClose =
|
|
||||||
closeFailure === undefined
|
|
||||||
? cause
|
|
||||||
: new AggregateError(
|
|
||||||
[cause, closeFailure.cause],
|
|
||||||
"Capture read and file-handle close both failed",
|
|
||||||
{ cause },
|
|
||||||
);
|
|
||||||
if (signal?.aborted === true)
|
|
||||||
return new AnalysisCancelledError(OPERATION, { cause: causeWithClose });
|
|
||||||
const code =
|
|
||||||
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
|
||||||
if (code === "EACCES" || code === "EPERM")
|
|
||||||
return new AnalysisAccessDeniedError(OPERATION, path, code, {
|
|
||||||
cause: causeWithClose,
|
|
||||||
});
|
|
||||||
if (
|
|
||||||
cause instanceof NonRegularFileReadError ||
|
|
||||||
cause instanceof RegularFileChangedError ||
|
|
||||||
code === "ENOENT" ||
|
|
||||||
code === "ENOTDIR" ||
|
|
||||||
code === "EISDIR" ||
|
|
||||||
code === "ENXIO"
|
|
||||||
)
|
|
||||||
return new AnalysisInputError(OPERATION, { cause: causeWithClose }, [
|
|
||||||
{
|
|
||||||
path: ["capture_path"],
|
|
||||||
reason: "invalid_value",
|
|
||||||
message:
|
|
||||||
cause instanceof NonRegularFileReadError && code !== "EISDIR"
|
|
||||||
? cause.message
|
|
||||||
: code === "EISDIR"
|
|
||||||
? `Selected capture is a directory, not a file: ${path}`
|
|
||||||
: `Selected capture could not be read (${code}): ${path}`,
|
|
||||||
},
|
|
||||||
]);
|
|
||||||
if (code !== "")
|
|
||||||
return new WebScriptExportError("io", path, errorMessage(cause), {
|
|
||||||
cause: causeWithClose,
|
|
||||||
});
|
|
||||||
return new ProviderAdapterError(PROVIDER.id, OPERATION, {
|
|
||||||
cause: causeWithClose,
|
|
||||||
diagnostics: {
|
|
||||||
capture_path: path,
|
|
||||||
error_name: cause instanceof Error ? cause.name : "UnknownError",
|
|
||||||
error_message: cause instanceof Error ? cause.message : String(cause),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
};
|
|
||||||
|
|
||||||
const incompleteCaptureCleanup = (
|
|
||||||
primary: AnalysisError,
|
|
||||||
path: string,
|
|
||||||
cleanup: NonNullable<CaptureRead["cleanup"]>,
|
|
||||||
): AnalysisError =>
|
|
||||||
analysisErrorWithCleanupFailure(
|
|
||||||
primary,
|
|
||||||
new ProviderCleanupError(
|
|
||||||
PROVIDER.id,
|
|
||||||
[path],
|
|
||||||
{ reason: cleanup.reason },
|
|
||||||
{ cause: cleanup.cause, operation: OPERATION },
|
|
||||||
),
|
|
||||||
OPERATION,
|
|
||||||
);
|
|
||||||
|
|
||||||
const errorMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|
||||||
const parseCapture = (
|
const parseCapture = (
|
||||||
bytes: Buffer,
|
bytes: Buffer,
|
||||||
closeFailure?: { readonly cause: unknown },
|
|
||||||
): Result<ReturnType<typeof selectScriptCapture>, AnalysisError> => {
|
): Result<ReturnType<typeof selectScriptCapture>, AnalysisError> => {
|
||||||
try {
|
try {
|
||||||
const text = new TextDecoder("utf-8", {
|
const text = new TextDecoder("utf-8", {
|
||||||
@@ -349,16 +198,8 @@ const parseCapture = (
|
|||||||
if (!json.ok) throw new TypeError(json.error);
|
if (!json.ok) throw new TypeError(json.error);
|
||||||
return ok(selectScriptCapture(json.value));
|
return ok(selectScriptCapture(json.value));
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
const causeWithClose =
|
|
||||||
closeFailure === undefined
|
|
||||||
? cause
|
|
||||||
: new AggregateError(
|
|
||||||
[cause, closeFailure.cause],
|
|
||||||
"Capture parse and file-handle close both failed",
|
|
||||||
{ cause },
|
|
||||||
);
|
|
||||||
return err(
|
return err(
|
||||||
new AnalysisInputError(OPERATION, { cause: causeWithClose }, [
|
new AnalysisInputError(OPERATION, { cause }, [
|
||||||
{
|
{
|
||||||
path: ["capture_path"],
|
path: ["capture_path"],
|
||||||
reason: "invalid_format",
|
reason: "invalid_format",
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import type { Result } from "../../domain/result.js";
|
|||||||
export interface AdbDeviceAnalysisPort {
|
export interface AdbDeviceAnalysisPort {
|
||||||
/** Probe the selected adb binary without contacting devices. */
|
/** Probe the selected adb binary without contacting devices. */
|
||||||
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
|
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
|
||||||
/** Terminally stop admission, drain work, and retry retained package rollback. */
|
/** The provider holds no long-lived resources; safe to call always. */
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
execute(
|
execute(
|
||||||
request: AdbRequest,
|
request: AdbRequest,
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import {
|
|||||||
} from "../../../tests/fixtures/analysisView.js";
|
} from "../../../tests/fixtures/analysisView.js";
|
||||||
import {
|
import {
|
||||||
inspectAnalysisView,
|
inspectAnalysisView,
|
||||||
inspectAnalysisViewValidated,
|
|
||||||
summarizeRetainedAnalysis,
|
summarizeRetainedAnalysis,
|
||||||
} from "./AnalysisViewService.js";
|
} from "./AnalysisViewService.js";
|
||||||
import type { Evidence } from "../../domain/evidence.js";
|
import type { Evidence } from "../../domain/evidence.js";
|
||||||
@@ -59,35 +58,6 @@ it.each(["/fixtures/unknown.exe", ""])(
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
it("reports inline Evidence identity failures at the complete source path", () => {
|
|
||||||
const parent = analysisViewLayoutEvidence();
|
|
||||||
const input = {
|
|
||||||
source: {
|
|
||||||
kind: "inline",
|
|
||||||
evidence: { ...parent, evidence_id: `ev_${"0".repeat(64)}` },
|
|
||||||
},
|
|
||||||
view: { kind: "summary" },
|
|
||||||
} as const;
|
|
||||||
|
|
||||||
for (const result of [
|
|
||||||
inspectAnalysisView(input),
|
|
||||||
inspectAnalysisViewValidated(input),
|
|
||||||
])
|
|
||||||
expect(result).toMatchObject({
|
|
||||||
ok: false,
|
|
||||||
error: {
|
|
||||||
_tag: "AnalysisInputError",
|
|
||||||
issues: [
|
|
||||||
{
|
|
||||||
path: ["source", "evidence", "evidence_id"],
|
|
||||||
reason: "invalid_value",
|
|
||||||
message: "Evidence semantic identifier does not match its record",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
},
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
|
it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
|
||||||
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||||
const parent = createEvidence(
|
const parent = createEvidence(
|
||||||
|
|||||||
@@ -132,11 +132,8 @@ const resolveAnalysisViewSource = (
|
|||||||
return err(
|
return err(
|
||||||
analysisInputErrorFromIssues(
|
analysisInputErrorFromIssues(
|
||||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||||
cause.issues.map((issue) => ({
|
cause.issues,
|
||||||
...issue,
|
source.evidence,
|
||||||
path: ["source", "evidence", ...issue.path],
|
|
||||||
})),
|
|
||||||
{ source },
|
|
||||||
{ cause },
|
{ cause },
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -2,10 +2,9 @@ import type { ApplicationNode } from "../../domain/javascript/javascriptApplicat
|
|||||||
import { sha256Text } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
import { sha256Text } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
||||||
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||||
import {
|
import type {
|
||||||
sourceNodeFor,
|
JavaScriptArtifactGraphContext,
|
||||||
type JavaScriptArtifactGraphContext,
|
JavaScriptArtifactGraphCoverage,
|
||||||
type JavaScriptArtifactGraphCoverage,
|
|
||||||
} from "./JavaScriptArtifactGraphContext.js";
|
} from "./JavaScriptArtifactGraphContext.js";
|
||||||
import {
|
import {
|
||||||
astObservationEvidence,
|
astObservationEvidence,
|
||||||
@@ -39,11 +38,10 @@ export const electronFindingSourceNode = (
|
|||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
moduleKey: string | null,
|
moduleKey: string | null,
|
||||||
runtime?: string,
|
|
||||||
): ApplicationNode | undefined =>
|
): ApplicationNode | undefined =>
|
||||||
moduleKey === null
|
moduleKey === null
|
||||||
? (context.assetNodes.get(file.path) ?? context.fileNodes.get(file.path))
|
? (context.assetNodes.get(file.path) ?? context.fileNodes.get(file.path))
|
||||||
: sourceNodeFor(context, file.path, moduleKey, runtime);
|
: context.moduleNodes.get(`${file.path}\0${moduleKey}`);
|
||||||
|
|
||||||
/** Observation-scoped identity shared safely across different artifact files. */
|
/** Observation-scoped identity shared safely across different artifact files. */
|
||||||
export const electronObservationIdentity = (
|
export const electronObservationIdentity = (
|
||||||
|
|||||||
@@ -73,7 +73,6 @@ const addIpcRecord = (
|
|||||||
context,
|
context,
|
||||||
record.file,
|
record.file,
|
||||||
record.finding.module_key,
|
record.finding.module_key,
|
||||||
record.finding.module_runtime,
|
|
||||||
);
|
);
|
||||||
if (source === undefined) return;
|
if (source === undefined) return;
|
||||||
state.sources.set(record.key, source);
|
state.sources.set(record.key, source);
|
||||||
@@ -256,7 +255,6 @@ const addValidationObservations = (state: IpcGraphState): void => {
|
|||||||
state.context,
|
state.context,
|
||||||
file,
|
file,
|
||||||
validation.module_key,
|
validation.module_key,
|
||||||
validation.module_runtime,
|
|
||||||
)
|
)
|
||||||
: state.handlers.get(handlerRecord.key);
|
: state.handlers.get(handlerRecord.key);
|
||||||
if (target !== undefined)
|
if (target !== undefined)
|
||||||
|
|||||||
@@ -38,12 +38,7 @@ export const addElectronNativeBoundaries = (
|
|||||||
|
|
||||||
const addNativeBinding = (input: NativeBindingInput): void => {
|
const addNativeBinding = (input: NativeBindingInput): void => {
|
||||||
const { context, file, value, coverage } = input;
|
const { context, file, value, coverage } = input;
|
||||||
const source = electronFindingSourceNode(
|
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||||
context,
|
|
||||||
file,
|
|
||||||
value.module_key,
|
|
||||||
value.module_runtime,
|
|
||||||
);
|
|
||||||
if (source === undefined) return;
|
if (source === undefined) return;
|
||||||
const resolved = resolveArtifactPath(
|
const resolved = resolveArtifactPath(
|
||||||
value.specifier,
|
value.specifier,
|
||||||
|
|||||||
@@ -39,32 +39,17 @@ export const addElectronWindowBoundaries = (
|
|||||||
if (javascript === null) continue;
|
if (javascript === null) continue;
|
||||||
const coverage = javascriptAnalysisCoverage(javascript);
|
const coverage = javascriptAnalysisCoverage(javascript);
|
||||||
for (const value of javascript.electron.browser_windows) {
|
for (const value of javascript.electron.browser_windows) {
|
||||||
const source = electronFindingSourceNode(
|
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||||
context,
|
|
||||||
file,
|
|
||||||
value.module_key,
|
|
||||||
value.module_runtime,
|
|
||||||
);
|
|
||||||
if (source !== undefined)
|
if (source !== undefined)
|
||||||
addBrowserWindow({ context, file, source, value, coverage });
|
addBrowserWindow({ context, file, source, value, coverage });
|
||||||
}
|
}
|
||||||
for (const value of javascript.electron.context_bridge_apis) {
|
for (const value of javascript.electron.context_bridge_apis) {
|
||||||
const source = electronFindingSourceNode(
|
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||||
context,
|
|
||||||
file,
|
|
||||||
value.module_key,
|
|
||||||
value.module_runtime,
|
|
||||||
);
|
|
||||||
if (source !== undefined)
|
if (source !== undefined)
|
||||||
addContextBridge({ context, file, source, value, coverage });
|
addContextBridge({ context, file, source, value, coverage });
|
||||||
}
|
}
|
||||||
for (const value of javascript.electron.utility_processes) {
|
for (const value of javascript.electron.utility_processes) {
|
||||||
const source = electronFindingSourceNode(
|
const source = electronFindingSourceNode(context, file, value.module_key);
|
||||||
context,
|
|
||||||
file,
|
|
||||||
value.module_key,
|
|
||||||
value.module_runtime,
|
|
||||||
);
|
|
||||||
if (source !== undefined)
|
if (source !== undefined)
|
||||||
addUtilityProcess({ context, file, source, value, coverage });
|
addUtilityProcess({ context, file, source, value, coverage });
|
||||||
}
|
}
|
||||||
|
|||||||
+13
-24
@@ -2,11 +2,11 @@ import { setImmediate } from "node:timers";
|
|||||||
import { writeFile } from "node:fs/promises";
|
import { writeFile } from "node:fs/promises";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { describe, expect, it } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
describe("JavaScript application cancellation", () => {
|
describe("JavaScript analysis cancellation before publication", () => {
|
||||||
it("interrupts semantic graph commitment before result validation", async () => {
|
it("interrupts semantic graph commitment before result validation", async () => {
|
||||||
const root = await createTestTempDirectory("rea-js-cancel-semantic-");
|
const root = await createTestTempDirectory("rea-js-cancel-semantic-");
|
||||||
await writeFile(
|
await writeFile(
|
||||||
@@ -33,9 +33,7 @@ describe("JavaScript application cancellation", () => {
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
if (result.ok)
|
if (result.ok)
|
||||||
throw new Error(
|
throw new Error("Cancelled commitment must not publish Evidence");
|
||||||
"Cancelled commitment must not publish a successful analysis",
|
|
||||||
);
|
|
||||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||||
code: "cancelled",
|
code: "cancelled",
|
||||||
details: { reason: "cancelled" },
|
details: { reason: "cancelled" },
|
||||||
@@ -44,7 +42,7 @@ describe("JavaScript application cancellation", () => {
|
|||||||
expect(phases).not.toContain("validate_javascript_application_result");
|
expect(phases).not.toContain("validate_javascript_application_result");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("honors control turns and cancellation while preparing one large source for owned analysis (#1462)", async () => {
|
it("handles control turns and cancellation inside one large source file (#1462)", async () => {
|
||||||
const root = await createTestTempDirectory("rea-js-cancel-in-file-");
|
const root = await createTestTempDirectory("rea-js-cancel-in-file-");
|
||||||
await writeFile(
|
await writeFile(
|
||||||
join(root, "index.js"),
|
join(root, "index.js"),
|
||||||
@@ -76,7 +74,7 @@ describe("JavaScript application cancellation", () => {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
if (result.ok) throw new Error("Cancelled analysis must not succeed");
|
if (result.ok) throw new Error("Cancelled analysis must not publish");
|
||||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||||
code: "cancelled",
|
code: "cancelled",
|
||||||
});
|
});
|
||||||
@@ -120,21 +118,12 @@ describe("JavaScript application cancellation", () => {
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
expect(requested).toBe(true);
|
expect(requested).toBe(true);
|
||||||
if (result.ok) throw new Error("Cancelled analysis must return a failure");
|
if (result.ok)
|
||||||
const failure = projectAnalysisError(result.error);
|
throw new Error("Cancelled analysis must not publish Evidence");
|
||||||
expect(failure).toMatchObject({ code: "cancelled" });
|
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||||
if (phase === "parse_javascript_source")
|
code: "cancelled",
|
||||||
expect(failure).toMatchObject({
|
details: { reason: "cancelled" },
|
||||||
details: {
|
});
|
||||||
operation: "analyze_javascript_application",
|
|
||||||
cleanup: "complete",
|
|
||||||
partial_observation: {
|
|
||||||
normalized_result: {
|
|
||||||
statistics: { relevant_files: 1, parse_failures: 0 },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
expect(terminal).toEqual([]);
|
expect(terminal).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -10,7 +10,7 @@ import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascri
|
|||||||
import { matchJavaScriptApplicationVersions } from "../../domain/javascript/javascriptApplicationVersionKeys.js";
|
import { matchJavaScriptApplicationVersions } from "../../domain/javascript/javascriptApplicationVersionKeys.js";
|
||||||
import { findApplicationFeatureSeeds } from "../../domain/javascript/javascriptFeatureSeed.js";
|
import { findApplicationFeatureSeeds } from "../../domain/javascript/javascriptFeatureSeed.js";
|
||||||
import { traceApplicationFeatureInputSchema } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
import { traceApplicationFeatureInputSchema } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
const analyzeSource = async (
|
const analyzeSource = async (
|
||||||
source: string,
|
source: string,
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ import { expect, it } from "vitest";
|
|||||||
|
|
||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
it.each([
|
it.each([
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -4,11 +4,9 @@ import { join } from "node:path";
|
|||||||
import { describe, expect, it } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
|
||||||
import { scanCanonicalArtifactInventory } from "../../../tests/fixtures/artifactInventory.js";
|
|
||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
describe("JavaScript application failure diagnostics", () => {
|
describe("JavaScript application failure diagnostics", () => {
|
||||||
it("identifies the rejected result field in caller-visible diagnostics", async () => {
|
it("identifies the rejected result field in caller-visible diagnostics", async () => {
|
||||||
@@ -155,41 +153,3 @@ describe("JavaScript application failure diagnostics", () => {
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("JavaScript application cleanup diagnostics", () => {
|
|
||||||
it("preserves cleanup and a complete inventory when translating reader failures", async () => {
|
|
||||||
const inputPath = await createTestTempDirectory("rea-js-cleanup-failure-");
|
|
||||||
await writeFile(join(inputPath, "main.js"), "export const value = 1;\n");
|
|
||||||
const inventory = await scanCanonicalArtifactInventory(inputPath);
|
|
||||||
const cleanup = {
|
|
||||||
reason: "reader remains open",
|
|
||||||
resources: [`JavaScript artifact reader for ${inputPath}`],
|
|
||||||
};
|
|
||||||
const partialObservation = {
|
|
||||||
kind: "artifact-inventory" as const,
|
|
||||||
inventory,
|
|
||||||
};
|
|
||||||
const failure = new ArtifactReaderFailure("io", "Reader cleanup failed", {
|
|
||||||
cleanup,
|
|
||||||
partialObservation,
|
|
||||||
});
|
|
||||||
|
|
||||||
const result = await analyzeJavaScriptApplication(
|
|
||||||
{ input_path: inputPath, format: "directory" },
|
|
||||||
{
|
|
||||||
progress: {
|
|
||||||
report: async () => {
|
|
||||||
throw failure;
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
);
|
|
||||||
|
|
||||||
if (result.ok) throw new Error("Expected reader failure");
|
|
||||||
expect(result.error).toMatchObject({
|
|
||||||
_tag: "ArtifactOperationError",
|
|
||||||
cleanup,
|
|
||||||
partialObservation,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|||||||
@@ -1,14 +1,18 @@
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
||||||
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
|
||||||
import {
|
import {
|
||||||
analyzeJavaScriptApplicationInputSchema,
|
analyzeJavaScriptApplicationInputSchema,
|
||||||
parseOwnedJavaScriptApplicationAnalysisSteps,
|
parseOwnedJavaScriptApplicationAnalysisSteps,
|
||||||
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
|
import {
|
||||||
|
AnalysisAccessDeniedError,
|
||||||
|
AnalysisInputError,
|
||||||
|
AnalysisOutputError,
|
||||||
|
AnalysisUnsupportedTargetError,
|
||||||
|
} from "../../domain/analysisErrorCore.js";
|
||||||
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
|
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
|
||||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
import { type AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||||
import type { Evidence } from "../../domain/evidence.js";
|
import type { Evidence } from "../../domain/evidence.js";
|
||||||
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
||||||
import { err, ok, type Result } from "../../domain/result.js";
|
import { err, ok, type Result } from "../../domain/result.js";
|
||||||
@@ -28,7 +32,6 @@ const OPERATION = "analyze_javascript_application" as const;
|
|||||||
/** Statically analyze one local JavaScript/Electron application. */
|
/** Statically analyze one local JavaScript/Electron application. */
|
||||||
export const analyzeJavaScriptApplication = async (
|
export const analyzeJavaScriptApplication = async (
|
||||||
rawInput: unknown,
|
rawInput: unknown,
|
||||||
scope: ArtifactResourceScope,
|
|
||||||
options: ExecutionOptions = {},
|
options: ExecutionOptions = {},
|
||||||
): Promise<Result<Evidence, AnalysisError>> => {
|
): Promise<Result<Evidence, AnalysisError>> => {
|
||||||
const parsed = analyzeJavaScriptApplicationInputSchema.safeParse(rawInput);
|
const parsed = analyzeJavaScriptApplicationInputSchema.safeParse(rawInput);
|
||||||
@@ -36,13 +39,12 @@ export const analyzeJavaScriptApplication = async (
|
|||||||
return err(
|
return err(
|
||||||
analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput),
|
analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput),
|
||||||
);
|
);
|
||||||
return analyzeJavaScriptApplicationValidated(parsed.data, scope, options);
|
return analyzeJavaScriptApplicationValidated(parsed.data, options);
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Analyze input already parsed by a trusted adapter boundary. */
|
/** Analyze input already parsed by a trusted adapter boundary. */
|
||||||
export const analyzeJavaScriptApplicationValidated = async (
|
export const analyzeJavaScriptApplicationValidated = async (
|
||||||
input: z.output<typeof analyzeJavaScriptApplicationInputSchema>,
|
input: z.output<typeof analyzeJavaScriptApplicationInputSchema>,
|
||||||
scope: ArtifactResourceScope,
|
|
||||||
options: ExecutionOptions = {},
|
options: ExecutionOptions = {},
|
||||||
): Promise<Result<Evidence, AnalysisError>> => {
|
): Promise<Result<Evidence, AnalysisError>> => {
|
||||||
try {
|
try {
|
||||||
@@ -59,14 +61,7 @@ export const analyzeJavaScriptApplicationValidated = async (
|
|||||||
input_path: input.input_path,
|
input_path: input.input_path,
|
||||||
format: input.format,
|
format: input.format,
|
||||||
integrity_policy: input.integrity_policy,
|
integrity_policy: input.integrity_policy,
|
||||||
...(input.max_heap_mb === undefined
|
|
||||||
? {}
|
|
||||||
: { max_heap_mb: input.max_heap_mb }),
|
|
||||||
...(input.analysis_timeout_ms === undefined
|
|
||||||
? {}
|
|
||||||
: { analysis_timeout_ms: input.analysis_timeout_ms }),
|
|
||||||
},
|
},
|
||||||
scope,
|
|
||||||
options.signal,
|
options.signal,
|
||||||
options.progress,
|
options.progress,
|
||||||
);
|
);
|
||||||
@@ -111,10 +106,22 @@ export const analyzeJavaScriptApplicationValidated = async (
|
|||||||
assertJavaScriptAnalysisActive(options.signal);
|
assertJavaScriptAnalysisActive(options.signal);
|
||||||
return ok(evidence);
|
return ok(evidence);
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
// Preserve execution constraints and their completed partial Evidence.
|
// Input selection failures are already typed by their cause.
|
||||||
if (cause instanceof AnalysisError) return err(cause);
|
if (
|
||||||
|
cause instanceof AnalysisInputError ||
|
||||||
|
cause instanceof AnalysisUnsupportedTargetError ||
|
||||||
|
cause instanceof AnalysisAccessDeniedError
|
||||||
|
)
|
||||||
|
return err(cause);
|
||||||
if (cause instanceof ArtifactReaderFailure)
|
if (cause instanceof ArtifactReaderFailure)
|
||||||
return err(artifactFailureToOperationError(cause));
|
return err(
|
||||||
|
new ArtifactOperationError(
|
||||||
|
OPERATION,
|
||||||
|
cause.reason,
|
||||||
|
cause.details,
|
||||||
|
cause.message,
|
||||||
|
),
|
||||||
|
);
|
||||||
if (cause instanceof z.ZodError)
|
if (cause instanceof z.ZodError)
|
||||||
return err(
|
return err(
|
||||||
new AnalysisOutputError(OPERATION, describeResultSchemaFailure(cause), {
|
new AnalysisOutputError(OPERATION, describeResultSchemaFailure(cause), {
|
||||||
@@ -146,23 +153,6 @@ export const analyzeJavaScriptApplicationValidated = async (
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const artifactFailureToOperationError = (
|
|
||||||
cause: ArtifactReaderFailure,
|
|
||||||
): ArtifactOperationError =>
|
|
||||||
new ArtifactOperationError(
|
|
||||||
OPERATION,
|
|
||||||
cause.reason,
|
|
||||||
cause.details,
|
|
||||||
cause.message,
|
|
||||||
{
|
|
||||||
cause,
|
|
||||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
|
||||||
...(cause.partialObservation === undefined
|
|
||||||
? {}
|
|
||||||
: { partialObservation: cause.partialObservation }),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
|
|
||||||
const describeResultSchemaFailure = (cause: z.ZodError): string => {
|
const describeResultSchemaFailure = (cause: z.ZodError): string => {
|
||||||
const issue = cause.issues[0];
|
const issue = cause.issues[0];
|
||||||
if (issue === undefined) return "Result schema rejected an unknown issue";
|
if (issue === undefined) return "Result schema rejected an unknown issue";
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirect
|
|||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { javaScriptSemanticQueryResultSchema } from "../../domain/javascript/javascriptSemanticQuerySchemas.js";
|
import { javaScriptSemanticQueryResultSchema } from "../../domain/javascript/javascriptSemanticQuerySchemas.js";
|
||||||
import { queryJavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticQuery.js";
|
import { queryJavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticQuery.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
it.each([
|
it.each([
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ import { expect, it } from "vitest";
|
|||||||
|
|
||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
const graphIdentities = async (source: string) => {
|
const graphIdentities = async (source: string) => {
|
||||||
const inputPath = await createTestTempDirectory("rea-js-template-literal-");
|
const inputPath = await createTestTempDirectory("rea-js-template-literal-");
|
||||||
@@ -31,16 +31,7 @@ const graphIdentities = async (source: string) => {
|
|||||||
...(identity.strategy === "artifact-local-key"
|
...(identity.strategy === "artifact-local-key"
|
||||||
? {
|
? {
|
||||||
namespace: identity.namespace,
|
namespace: identity.namespace,
|
||||||
// Module-export identity is a JSON tuple whose first entry
|
key: identity.key.replace(/^[0-9a-f]{64}:/u, ""),
|
||||||
// binds the whole container; only that digest changes here.
|
|
||||||
key:
|
|
||||||
identity.namespace === "module-export"
|
|
||||||
? JSON.stringify(
|
|
||||||
(JSON.parse(identity.key) as unknown[]).slice(1),
|
|
||||||
)
|
|
||||||
: identity.namespace === "source-module"
|
|
||||||
? identity.key.replace(/^[0-9a-f]{64}:/u, "")
|
|
||||||
: identity.key,
|
|
||||||
}
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}))
|
}))
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import { canonicalJson } from "../../domain/comparisonSemantics.js";
|
|||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
|
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
type ApplicationAnalysis = ReturnType<
|
type ApplicationAnalysis = ReturnType<
|
||||||
typeof javascriptApplicationAnalysisResultSchema.parse
|
typeof javascriptApplicationAnalysisResultSchema.parse
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import type { ApplicationVersionComparisonResult } from "../../domain/javascript
|
|||||||
import type { JavaScriptExportShapeComparisonResult } from "../../domain/javascript/javascriptExportShapeComparisonSchemas.js";
|
import type { JavaScriptExportShapeComparisonResult } from "../../domain/javascript/javascriptExportShapeComparisonSchemas.js";
|
||||||
import type { ApplicationFeatureTraceResult } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
import type { ApplicationFeatureTraceResult } from "../../domain/javascript/javascriptFeatureTraceSchemas.js";
|
||||||
import type { JavaScriptSemanticTraceResult } from "../../domain/javascript/javascriptSemanticTraceSchemas.js";
|
import type { JavaScriptSemanticTraceResult } from "../../domain/javascript/javascriptSemanticTraceSchemas.js";
|
||||||
|
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
||||||
import type { SourceToBundleComparisonResult } from "../../domain/javascript/sourceToBundleComparisonSchemas.js";
|
import type { SourceToBundleComparisonResult } from "../../domain/javascript/sourceToBundleComparisonSchemas.js";
|
||||||
import { JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER } from "../InvestigationProviders.js";
|
import { JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER } from "../InvestigationProviders.js";
|
||||||
|
|
||||||
@@ -19,7 +20,7 @@ export const createApplicationFeatureTraceEvidence = (
|
|||||||
predicateType: "rea.application-feature-trace",
|
predicateType: "rea.application-feature-trace",
|
||||||
operation: "trace_application_feature",
|
operation: "trace_application_feature",
|
||||||
parameters,
|
parameters,
|
||||||
result,
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult: null,
|
rawResult: null,
|
||||||
confidence: "inferred",
|
confidence: "inferred",
|
||||||
authority: "analyst-inference",
|
authority: "analyst-inference",
|
||||||
@@ -37,7 +38,7 @@ export const createJavaScriptSemanticTraceEvidence = (
|
|||||||
predicateType: "rea.javascript-semantic-trace",
|
predicateType: "rea.javascript-semantic-trace",
|
||||||
operation: "trace_javascript_semantics",
|
operation: "trace_javascript_semantics",
|
||||||
parameters,
|
parameters,
|
||||||
result,
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult: null,
|
rawResult: null,
|
||||||
confidence: "inferred",
|
confidence: "inferred",
|
||||||
authority: "analyst-inference",
|
authority: "analyst-inference",
|
||||||
@@ -55,7 +56,7 @@ export const createApplicationVersionComparisonEvidence = (
|
|||||||
predicateType: "rea.application-version-comparison",
|
predicateType: "rea.application-version-comparison",
|
||||||
operation: "compare_application_versions",
|
operation: "compare_application_versions",
|
||||||
parameters,
|
parameters,
|
||||||
result,
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult: null,
|
rawResult: null,
|
||||||
confidence: "inferred",
|
confidence: "inferred",
|
||||||
authority: "analyst-inference",
|
authority: "analyst-inference",
|
||||||
@@ -73,7 +74,7 @@ export const createSourceToBundleComparisonEvidence = (
|
|||||||
predicateType: "rea.source-to-bundle-comparison",
|
predicateType: "rea.source-to-bundle-comparison",
|
||||||
operation: "compare_source_to_bundle",
|
operation: "compare_source_to_bundle",
|
||||||
parameters,
|
parameters,
|
||||||
result,
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult: null,
|
rawResult: null,
|
||||||
confidence: "inferred",
|
confidence: "inferred",
|
||||||
authority: "analyst-inference",
|
authority: "analyst-inference",
|
||||||
@@ -91,7 +92,7 @@ export const createJavaScriptExportShapeComparisonEvidence = (
|
|||||||
predicateType: "rea.javascript-export-shape-comparison",
|
predicateType: "rea.javascript-export-shape-comparison",
|
||||||
operation: "compare_javascript_export_shapes",
|
operation: "compare_javascript_export_shapes",
|
||||||
parameters,
|
parameters,
|
||||||
result,
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult: null,
|
rawResult: null,
|
||||||
confidence: "inferred",
|
confidence: "inferred",
|
||||||
authority: "analyst-inference",
|
authority: "analyst-inference",
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ import {
|
|||||||
classifyParsedJavaScriptOpenReceiversSteps,
|
classifyParsedJavaScriptOpenReceiversSteps,
|
||||||
} from "../../domain/javascript/javascriptSemanticAnalysis.js";
|
} from "../../domain/javascript/javascriptSemanticAnalysis.js";
|
||||||
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
|
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
|
||||||
import { projectJavaScriptModuleSemantics } from "../../domain/javascript/javascriptModuleSemanticIr.js";
|
|
||||||
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
|
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
|
||||||
import { hasValidSourceMapContents } from "../../domain/sourceMapContents.js";
|
import { hasValidSourceMapContents } from "../../domain/sourceMapContents.js";
|
||||||
import { flattenSourceMapLeaves } from "../../domain/sourceMapEnvelope.js";
|
import { flattenSourceMapLeaves } from "../../domain/sourceMapEnvelope.js";
|
||||||
@@ -36,15 +35,6 @@ import type {
|
|||||||
import { analyzeJavaScriptJsonModule } from "./JavaScriptJsonModules.js";
|
import { analyzeJavaScriptJsonModule } from "./JavaScriptJsonModules.js";
|
||||||
import { completeJavaScriptAnalysisSteps } from "./JavaScriptAnalysisControl.js";
|
import { completeJavaScriptAnalysisSteps } from "./JavaScriptAnalysisControl.js";
|
||||||
import { htmlArtifactReferences } from "../../domain/javascript/htmlArtifactReferences.js";
|
import { htmlArtifactReferences } from "../../domain/javascript/htmlArtifactReferences.js";
|
||||||
import type { JavaScriptSourceAnalysisPort } from "../../domain/javascript/javascriptSourceAnalysis.js";
|
|
||||||
import type { JavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
|
|
||||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
|
||||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
|
||||||
import {
|
|
||||||
AnalysisCancelledError,
|
|
||||||
AnalysisResourceConstraintError,
|
|
||||||
AnalysisTimeoutError,
|
|
||||||
} from "../../domain/analysisErrorCore.js";
|
|
||||||
|
|
||||||
interface MutableArtifactAnalysis<
|
interface MutableArtifactAnalysis<
|
||||||
SemanticIr extends JavaScriptModuleSemanticIr,
|
SemanticIr extends JavaScriptModuleSemanticIr,
|
||||||
@@ -105,7 +95,27 @@ export const analyzeAndProjectJavaScriptArtifactFiles = async (
|
|||||||
): Generator<void, JavaScriptModuleSemanticIr> {
|
): Generator<void, JavaScriptModuleSemanticIr> {
|
||||||
const projection = projectSemantics(file, ir);
|
const projection = projectSemantics(file, ir);
|
||||||
if (projection !== undefined) while (projection.next().done !== true) yield;
|
if (projection !== undefined) while (projection.next().done !== true) yield;
|
||||||
return projectJavaScriptModuleSemantics(ir);
|
const programScopes = ir.scopes.filter(({ kind }) => kind === "program");
|
||||||
|
const programScopeId = programScopes[0]?.scopeId;
|
||||||
|
const localNames = new Set(
|
||||||
|
ir.moduleLinks.map(({ localName }) => localName),
|
||||||
|
);
|
||||||
|
const callableIds = new Set(
|
||||||
|
ir.moduleLinks.map(({ callableId }) => callableId),
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
scopes: programScopes,
|
||||||
|
bindings: ir.bindings.filter(
|
||||||
|
({ name, scopeId }) =>
|
||||||
|
scopeId === programScopeId && localNames.has(name),
|
||||||
|
),
|
||||||
|
callables: ir.callables.filter(({ callableId }) =>
|
||||||
|
callableIds.has(callableId),
|
||||||
|
),
|
||||||
|
moduleLinks: ir.moduleLinks,
|
||||||
|
coverage: ir.coverage,
|
||||||
|
limitations: ir.limitations,
|
||||||
|
};
|
||||||
};
|
};
|
||||||
const context = { state, projectSemantics: projectFileSemantics };
|
const context = { state, projectSemantics: projectFileSemantics };
|
||||||
for (const [index, file] of fileSet.files.entries()) {
|
for (const [index, file] of fileSet.files.entries()) {
|
||||||
@@ -120,174 +130,6 @@ export const analyzeAndProjectJavaScriptArtifactFiles = async (
|
|||||||
return finalizeArtifactAnalysis(state);
|
return finalizeArtifactAnalysis(state);
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Analyze sources behind a hard heap boundary, retaining collected facts on interruption. */
|
|
||||||
export const analyzeAndAdoptJavaScriptArtifactFiles = async (
|
|
||||||
fileSet: JavaScriptArtifactFileSet,
|
|
||||||
worker: Pick<JavaScriptSourceAnalysisPort, "analyze">,
|
|
||||||
projection: JavaScriptSemanticGraphProjection,
|
|
||||||
beforeFile: (
|
|
||||||
file: JavaScriptArtifactFile,
|
|
||||||
completed: number,
|
|
||||||
total: number,
|
|
||||||
) => Promise<void>,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<{
|
|
||||||
readonly analysis: JavaScriptModuleArtifactAnalysis;
|
|
||||||
readonly interruption: AnalysisError | null;
|
|
||||||
}> => {
|
|
||||||
const state = emptyArtifactAnalysis<JavaScriptModuleSemanticIr>();
|
|
||||||
let interruption: AnalysisError | null = null;
|
|
||||||
for (const [index, file] of fileSet.files.entries()) {
|
|
||||||
try {
|
|
||||||
await setImmediate();
|
|
||||||
await beforeFile(file, index, fileSet.files.length);
|
|
||||||
if (signal?.aborted === true)
|
|
||||||
throw new AnalysisCancelledError("analyze_javascript_application");
|
|
||||||
if (file.kind !== "javascript" || !file.text.included) {
|
|
||||||
const steps = analyzeArtifactFileSteps(file, {
|
|
||||||
state,
|
|
||||||
projectSemantics: function* (_file, ir) {
|
|
||||||
yield;
|
|
||||||
return projectJavaScriptModuleSemantics(ir);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
await completeJavaScriptAnalysisSteps(steps, signal);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const result = await worker.analyze(
|
|
||||||
file,
|
|
||||||
projection.remainingFileNodeBudget(),
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
const facts = result.ok ? result.value : result.error;
|
|
||||||
const applicationProjectionFailure = facts.applicationProjectionFailure;
|
|
||||||
if (facts.projection !== null)
|
|
||||||
projection.adoptFileProjection(facts.projection);
|
|
||||||
const error = result.ok ? null : result.error.error;
|
|
||||||
const limits =
|
|
||||||
error instanceof AnalysisResourceConstraintError
|
|
||||||
? error.reportedLimits
|
|
||||||
: error instanceof AnalysisTimeoutError
|
|
||||||
? { worker_timeout_ms: error.timeoutMs }
|
|
||||||
: null;
|
|
||||||
const failure =
|
|
||||||
error === null
|
|
||||||
? undefined
|
|
||||||
: {
|
|
||||||
reason: `${file.path}: ${error.message}`,
|
|
||||||
error: projectAnalysisError(error),
|
|
||||||
limits,
|
|
||||||
};
|
|
||||||
state.files.push({
|
|
||||||
file,
|
|
||||||
javascript: facts.javascript,
|
|
||||||
semantic: facts.module === null ? null : { ir: facts.module },
|
|
||||||
...(failure === undefined ? {} : { analysis_failure: failure }),
|
|
||||||
...(applicationProjectionFailure === undefined
|
|
||||||
? {}
|
|
||||||
: {
|
|
||||||
application_projection_failure: {
|
|
||||||
reason: applicationProjectionFailure.message,
|
|
||||||
error: projectAnalysisError(applicationProjectionFailure),
|
|
||||||
limits: applicationProjectionFailure.reportedLimits,
|
|
||||||
},
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
if (failure !== undefined && facts.projection === null)
|
|
||||||
projection.recordUnavailableFile(
|
|
||||||
file,
|
|
||||||
failure.reason,
|
|
||||||
failure.limits,
|
|
||||||
error instanceof AnalysisResourceConstraintError ||
|
|
||||||
error instanceof AnalysisTimeoutError
|
|
||||||
? "resource-limit"
|
|
||||||
: "incomplete-module",
|
|
||||||
);
|
|
||||||
if (facts.javascript !== null) {
|
|
||||||
state.visitedNodes += facts.javascript.visited_ast_nodes;
|
|
||||||
state.findings +=
|
|
||||||
findingCount(facts.javascript) +
|
|
||||||
(facts.module?.moduleLinks.length ?? 0);
|
|
||||||
state.modules += facts.javascript.bundler_registrations.reduce(
|
|
||||||
(count, item) => count + item.modules.length,
|
|
||||||
0,
|
|
||||||
);
|
|
||||||
if (facts.javascript.parse_status === "failed")
|
|
||||||
state.parseFailures += 1;
|
|
||||||
}
|
|
||||||
if (
|
|
||||||
error !== null &&
|
|
||||||
(!(error instanceof AnalysisResourceConstraintError) ||
|
|
||||||
error.reportedLimits?.parent_result_budget_exhausted === true)
|
|
||||||
) {
|
|
||||||
interruption = error;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (signal?.aborted !== true && !(cause instanceof AnalysisError))
|
|
||||||
throw cause;
|
|
||||||
interruption =
|
|
||||||
signal?.aborted === true
|
|
||||||
? new AnalysisCancelledError("analyze_javascript_application")
|
|
||||||
: cause instanceof AnalysisError
|
|
||||||
? cause
|
|
||||||
: null;
|
|
||||||
if (interruption !== null) {
|
|
||||||
state.files.push({
|
|
||||||
file,
|
|
||||||
javascript: null,
|
|
||||||
semantic: null,
|
|
||||||
analysis_failure: {
|
|
||||||
reason: `${file.path}: ${interruption.message}`,
|
|
||||||
error: projectAnalysisError(interruption),
|
|
||||||
limits: null,
|
|
||||||
},
|
|
||||||
});
|
|
||||||
projection.recordUnavailableFile(
|
|
||||||
file,
|
|
||||||
interruption.message,
|
|
||||||
null,
|
|
||||||
"incomplete-module",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (interruption !== null) {
|
|
||||||
const analyzedPaths = new Set(state.files.map(({ file }) => file.path));
|
|
||||||
const failure = {
|
|
||||||
reason: `Source analysis was interrupted: ${interruption.message}`,
|
|
||||||
error: projectAnalysisError(interruption),
|
|
||||||
limits:
|
|
||||||
interruption instanceof AnalysisResourceConstraintError
|
|
||||||
? interruption.reportedLimits
|
|
||||||
: interruption instanceof AnalysisTimeoutError
|
|
||||||
? { worker_timeout_ms: interruption.timeoutMs }
|
|
||||||
: null,
|
|
||||||
};
|
|
||||||
for (const file of fileSet.files) {
|
|
||||||
if (analyzedPaths.has(file.path)) continue;
|
|
||||||
state.files.push({
|
|
||||||
file,
|
|
||||||
javascript: null,
|
|
||||||
semantic: null,
|
|
||||||
analysis_failure: failure,
|
|
||||||
});
|
|
||||||
if (file.kind === "javascript")
|
|
||||||
projection.recordUnavailableFile(
|
|
||||||
file,
|
|
||||||
failure.reason,
|
|
||||||
failure.limits,
|
|
||||||
interruption instanceof AnalysisResourceConstraintError ||
|
|
||||||
interruption instanceof AnalysisTimeoutError
|
|
||||||
? "resource-limit"
|
|
||||||
: "incomplete-module",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return { analysis: finalizeArtifactAnalysis(state), interruption };
|
|
||||||
};
|
|
||||||
|
|
||||||
const analyzeArtifactFiles = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
const analyzeArtifactFiles = <SemanticIr extends JavaScriptModuleSemanticIr>(
|
||||||
fileSet: JavaScriptArtifactFileSet,
|
fileSet: JavaScriptArtifactFileSet,
|
||||||
projectSemantics: ArtifactAnalysisContext<SemanticIr>["projectSemantics"],
|
projectSemantics: ArtifactAnalysisContext<SemanticIr>["projectSemantics"],
|
||||||
@@ -319,12 +161,6 @@ const finalizeArtifactAnalysis = <
|
|||||||
limitations: [
|
limitations: [
|
||||||
"JavaScript and HTML were parsed as inert text; bundle bootstrap code was never executed.",
|
"JavaScript and HTML were parsed as inert text; bundle bootstrap code was never executed.",
|
||||||
"Static paths and relationships may remain unresolved when expressions are dynamic or obfuscated.",
|
"Static paths and relationships may remain unresolved when expressions are dynamic or obfuscated.",
|
||||||
...state.files.flatMap(
|
|
||||||
({ analysis_failure, application_projection_failure }) =>
|
|
||||||
[analysis_failure, application_projection_failure].flatMap(
|
|
||||||
(failure) => (failure === undefined ? [] : [failure.reason]),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
],
|
],
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -4,9 +4,7 @@ import type {
|
|||||||
JavaScriptStaticAnalysis,
|
JavaScriptStaticAnalysis,
|
||||||
} from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
} from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||||
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
|
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
|
||||||
import type { JavaScriptModuleSemanticIr } from "../../domain/javascript/javascriptModuleSemanticIr.js";
|
|
||||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
|
||||||
|
|
||||||
interface JavaScriptStructuredObservation {
|
interface JavaScriptStructuredObservation {
|
||||||
readonly path: string;
|
readonly path: string;
|
||||||
@@ -91,7 +89,15 @@ export type JavaScriptJsonModuleObservation = JavaScriptStructuredObservation &
|
|||||||
);
|
);
|
||||||
|
|
||||||
/** Module facts needed after file-local semantic projection has finished. */
|
/** Module facts needed after file-local semantic projection has finished. */
|
||||||
export type { JavaScriptModuleSemanticIr } from "../../domain/javascript/javascriptModuleSemanticIr.js";
|
export type JavaScriptModuleSemanticIr = Pick<
|
||||||
|
JavaScriptSemanticIr,
|
||||||
|
| "scopes"
|
||||||
|
| "bindings"
|
||||||
|
| "callables"
|
||||||
|
| "moduleLinks"
|
||||||
|
| "coverage"
|
||||||
|
| "limitations"
|
||||||
|
>;
|
||||||
|
|
||||||
/** One relevant file plus optional AST-only JavaScript facts. */
|
/** One relevant file plus optional AST-only JavaScript facts. */
|
||||||
export interface AnalyzedJavaScriptArtifactFile<
|
export interface AnalyzedJavaScriptArtifactFile<
|
||||||
@@ -102,18 +108,6 @@ export interface AnalyzedJavaScriptArtifactFile<
|
|||||||
readonly semantic: {
|
readonly semantic: {
|
||||||
readonly ir: SemanticIr;
|
readonly ir: SemanticIr;
|
||||||
} | null;
|
} | null;
|
||||||
/** Operational incompleteness, distinct from a malformed source file. */
|
|
||||||
readonly analysis_failure?: {
|
|
||||||
readonly reason: string;
|
|
||||||
readonly error: JsonValue;
|
|
||||||
readonly limits: Readonly<Record<string, JsonValue>> | null;
|
|
||||||
};
|
|
||||||
/** Facts retained inline when their expanded graph exceeds available memory. */
|
|
||||||
readonly application_projection_failure?: {
|
|
||||||
readonly reason: string;
|
|
||||||
readonly error: JsonValue;
|
|
||||||
readonly limits: Readonly<Record<string, JsonValue>> | null;
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Complete static-analysis projection before graph construction. */
|
/** Complete static-analysis projection before graph construction. */
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ export class JavaScriptArtifactGraphAccumulator {
|
|||||||
readonly #nodes = new Map<string, ApplicationNode>();
|
readonly #nodes = new Map<string, ApplicationNode>();
|
||||||
readonly #edges = new Map<string, ApplicationEdge>();
|
readonly #edges = new Map<string, ApplicationEdge>();
|
||||||
|
|
||||||
/** Create or merge one canonical node, retaining a stable live construction reference. */
|
/** Create or merge one canonical node. */
|
||||||
addNode<Input extends DisplayableNodeInput>(input: Input): ApplicationNode {
|
addNode<Input extends DisplayableNodeInput>(input: Input): ApplicationNode {
|
||||||
const created = createJavaScriptApplicationNode(displayableNode(input));
|
const created = createJavaScriptApplicationNode(displayableNode(input));
|
||||||
const existing = this.#nodes.get(created.node_id);
|
const existing = this.#nodes.get(created.node_id);
|
||||||
@@ -23,23 +23,19 @@ export class JavaScriptArtifactGraphAccumulator {
|
|||||||
const bySemanticId = new Map(
|
const bySemanticId = new Map(
|
||||||
[...existing.observations, ...created.observations].map((observation) => [
|
[...existing.observations, ...created.observations].map((observation) => [
|
||||||
observation.observation_id,
|
observation.observation_id,
|
||||||
observation,
|
observationInput(observation),
|
||||||
]),
|
]),
|
||||||
);
|
);
|
||||||
const observations = [...bySemanticId.values()].sort((left, right) =>
|
const observations = [...bySemanticId.entries()]
|
||||||
left.observation_id < right.observation_id
|
.sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0))
|
||||||
? -1
|
.map(([, observation]) => observation);
|
||||||
: left.observation_id > right.observation_id
|
const merged = createJavaScriptApplicationNode({
|
||||||
? 1
|
kind: existing.kind,
|
||||||
: 0,
|
identity: existing.identity,
|
||||||
);
|
observations,
|
||||||
// Both factories already validated and identified these observations.
|
});
|
||||||
// Replacing the node retained every previous observation array through
|
this.#nodes.set(merged.node_id, merged);
|
||||||
// path/chunk/IPC indexes when multiple paths shared an entity identity.
|
return merged;
|
||||||
existing.observations.length = 0;
|
|
||||||
for (const observation of observations)
|
|
||||||
existing.observations.push(observation);
|
|
||||||
return existing;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Create or replace one canonical edge by semantic identifier. */
|
/** Create or replace one canonical edge by semantic identifier. */
|
||||||
@@ -60,6 +56,17 @@ export class JavaScriptArtifactGraphAccumulator {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const observationInput = (
|
||||||
|
observation: ApplicationNode["observations"][number],
|
||||||
|
) => {
|
||||||
|
const {
|
||||||
|
observation_id: _observationId,
|
||||||
|
identifier_strategy: _strategy,
|
||||||
|
...input
|
||||||
|
} = observation;
|
||||||
|
return input;
|
||||||
|
};
|
||||||
|
|
||||||
/** Node fields whose display form is normalized before validation. */
|
/** Node fields whose display form is normalized before validation. */
|
||||||
interface DisplayableNodeInput {
|
interface DisplayableNodeInput {
|
||||||
readonly observations: readonly { readonly label: string | null }[];
|
readonly observations: readonly { readonly label: string | null }[];
|
||||||
|
|||||||
@@ -107,33 +107,19 @@ function* buildJavaScriptArtifactGraphInputSteps(
|
|||||||
yield;
|
yield;
|
||||||
addJavaScriptArtifactFiles(context);
|
addJavaScriptArtifactFiles(context);
|
||||||
yield;
|
yield;
|
||||||
// Raw observations remain on the file nodes. Only admitted expansions feed
|
const packageRoots = addJavaScriptPackageNodes(context);
|
||||||
// the relationship builders, including their Electron and bundler callers.
|
|
||||||
const projectionContext = {
|
|
||||||
...context,
|
|
||||||
analysis: {
|
|
||||||
...analysis,
|
|
||||||
files: analysis.files.map((file) =>
|
|
||||||
file.application_projection_failure === undefined
|
|
||||||
? file
|
|
||||||
: { ...file, javascript: null, semantic: null },
|
|
||||||
),
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const packageRoots = addJavaScriptPackageNodes(projectionContext);
|
|
||||||
yield;
|
yield;
|
||||||
addJavaScriptSourceModules(projectionContext);
|
addJavaScriptSourceModules(context);
|
||||||
yield;
|
yield;
|
||||||
const bundlerLimitations = addJavaScriptBundlerNodes(projectionContext);
|
const bundlerLimitations = addJavaScriptBundlerNodes(context);
|
||||||
const relationshipOmissions =
|
const relationshipOmissions =
|
||||||
yield* addJavaScriptModuleRelationshipsSteps(projectionContext);
|
yield* addJavaScriptModuleRelationshipsSteps(context);
|
||||||
const findingLimitations =
|
const findingLimitations = yield* addJavaScriptStaticFindingsSteps(context);
|
||||||
yield* addJavaScriptStaticFindingsSteps(projectionContext);
|
addElectronBoundaries(context);
|
||||||
addElectronBoundaries(projectionContext);
|
|
||||||
yield;
|
yield;
|
||||||
addJavaScriptHtmlRoles(projectionContext);
|
addJavaScriptHtmlRoles(context);
|
||||||
yield;
|
yield;
|
||||||
addJavaScriptSourceMapOriginals(projectionContext);
|
addJavaScriptSourceMapOriginals(context);
|
||||||
yield;
|
yield;
|
||||||
const coverage = graphCoverage(context);
|
const coverage = graphCoverage(context);
|
||||||
return {
|
return {
|
||||||
@@ -167,34 +153,14 @@ const graphCoverage = (context: JavaScriptArtifactGraphContext) => {
|
|||||||
javascript !== null && javascript.parse_status === "partial",
|
javascript !== null && javascript.parse_status === "partial",
|
||||||
);
|
);
|
||||||
const unknownGap =
|
const unknownGap =
|
||||||
context.analysis.files.some(
|
|
||||||
({ analysis_failure, application_projection_failure }) =>
|
|
||||||
analysis_failure !== undefined ||
|
|
||||||
application_projection_failure !== undefined,
|
|
||||||
) ||
|
|
||||||
context.analysis.parse_failures > 0 ||
|
context.analysis.parse_failures > 0 ||
|
||||||
context.fileSet.invalid_utf8_files > 0 ||
|
context.fileSet.invalid_utf8_files > 0 ||
|
||||||
sourceMapPolicyGap ||
|
sourceMapPolicyGap ||
|
||||||
malformedStructuredData ||
|
malformedStructuredData ||
|
||||||
partialJavaScript;
|
partialJavaScript;
|
||||||
const executionLimits = context.analysis.files.flatMap(
|
if (resourceLimits.length > 0)
|
||||||
({ analysis_failure, application_projection_failure }) =>
|
|
||||||
[analysis_failure, application_projection_failure].flatMap((failure) =>
|
|
||||||
failure === undefined
|
|
||||||
? []
|
|
||||||
: javaScriptAnalysisCoverageLimits(failure.limits),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
if (resourceLimits.length > 0 || executionLimits.length > 0)
|
|
||||||
return partialApplicationCoverage(
|
return partialApplicationCoverage(
|
||||||
[
|
semanticResourceLimitCoverage(resourceLimits),
|
||||||
...new Map(
|
|
||||||
[
|
|
||||||
...semanticResourceLimitCoverage(resourceLimits),
|
|
||||||
...executionLimits,
|
|
||||||
].map((limit) => [JSON.stringify(limit), limit]),
|
|
||||||
).values(),
|
|
||||||
],
|
|
||||||
null,
|
null,
|
||||||
);
|
);
|
||||||
if (unknownGap) return partialApplicationCoverage([], null);
|
if (unknownGap) return partialApplicationCoverage([], null);
|
||||||
@@ -294,4 +260,3 @@ const graphLimitations = (
|
|||||||
]),
|
]),
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
import { javaScriptAnalysisCoverageLimits } from "../../domain/javascript/javascriptAnalysisResourceControls.js";
|
|
||||||
|
|||||||
@@ -161,7 +161,7 @@ const addBundlerModuleNodes = (
|
|||||||
],
|
],
|
||||||
});
|
});
|
||||||
context.moduleNodes.set(
|
context.moduleNodes.set(
|
||||||
moduleLookupKey(file.path, registration.runtime, moduleValue.module_key),
|
moduleLookupKey(file.path, moduleValue.module_key),
|
||||||
module,
|
module,
|
||||||
);
|
);
|
||||||
addAstContainsEdge(context, {
|
addAstContainsEdge(context, {
|
||||||
@@ -195,7 +195,7 @@ const addBundlerRuntimeEdges = (
|
|||||||
const { file, registration, chunk, coverage } = record;
|
const { file, registration, chunk, coverage } = record;
|
||||||
for (const moduleKey of registration.entry_module_keys) {
|
for (const moduleKey of registration.entry_module_keys) {
|
||||||
const resolved = context.moduleNodes.get(
|
const resolved = context.moduleNodes.get(
|
||||||
moduleLookupKey(file.path, registration.runtime, moduleKey),
|
moduleLookupKey(file.path, moduleKey),
|
||||||
);
|
);
|
||||||
const target =
|
const target =
|
||||||
resolved ?? unresolvedBundlerModuleNode(context, record, moduleKey);
|
resolved ?? unresolvedBundlerModuleNode(context, record, moduleKey);
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
|
||||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||||
import {
|
import {
|
||||||
completeApplicationCoverage,
|
completeApplicationCoverage,
|
||||||
@@ -100,8 +99,6 @@ export interface UnavailableParseInput {
|
|||||||
readonly asset: ApplicationNode;
|
readonly asset: ApplicationNode;
|
||||||
readonly operation: string;
|
readonly operation: string;
|
||||||
readonly limitation: string;
|
readonly limitation: string;
|
||||||
readonly failure?: JsonValue;
|
|
||||||
readonly limits?: JavaScriptArtifactGraphCoverage["limits"];
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Add a direct artifact-containment edge. */
|
/** Add a direct artifact-containment edge. */
|
||||||
@@ -185,15 +182,12 @@ export const addUnavailableStaticParseScope = (
|
|||||||
path: input.file.path,
|
path: input.file.path,
|
||||||
operation: input.operation,
|
operation: input.operation,
|
||||||
limitation: input.limitation,
|
limitation: input.limitation,
|
||||||
...(input.failure === undefined
|
|
||||||
? {}
|
|
||||||
: { analysis_failure: input.failure }),
|
|
||||||
},
|
},
|
||||||
evidence: unavailableAstEvidence({
|
evidence: unavailableAstEvidence({
|
||||||
sha256: input.file.sha256,
|
sha256: input.file.sha256,
|
||||||
path: input.file.path,
|
path: input.file.path,
|
||||||
operation: input.operation,
|
operation: input.operation,
|
||||||
coverage: partialApplicationCoverage(input.limits ?? [], null),
|
coverage: partialApplicationCoverage([], null),
|
||||||
limitation: input.limitation,
|
limitation: input.limitation,
|
||||||
}),
|
}),
|
||||||
},
|
},
|
||||||
@@ -329,11 +323,8 @@ export const selfReferenceOmissions = (
|
|||||||
];
|
];
|
||||||
|
|
||||||
/** Deterministic key for one recovered module inside a bundle asset. */
|
/** Deterministic key for one recovered module inside a bundle asset. */
|
||||||
export const moduleLookupKey = (
|
export const moduleLookupKey = (path: string, moduleKey: string): string =>
|
||||||
path: string,
|
`${path}\0${moduleKey}`;
|
||||||
runtime: string,
|
|
||||||
moduleKey: string,
|
|
||||||
): string => compositeKey([path, runtime, moduleKey]);
|
|
||||||
|
|
||||||
/** Deterministic key for one recovered bundler chunk inside a bundle asset. */
|
/** Deterministic key for one recovered bundler chunk inside a bundle asset. */
|
||||||
export const chunkLookupKey = (
|
export const chunkLookupKey = (
|
||||||
@@ -362,10 +353,7 @@ export const sourceNodeFor = (
|
|||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
path: string,
|
path: string,
|
||||||
moduleKey: string | null,
|
moduleKey: string | null,
|
||||||
runtime?: string,
|
|
||||||
): ApplicationNode | undefined =>
|
): ApplicationNode | undefined =>
|
||||||
moduleKey === null
|
moduleKey === null
|
||||||
? context.sourceModuleNodes.get(path)
|
? context.sourceModuleNodes.get(path)
|
||||||
: runtime === undefined
|
: context.moduleNodes.get(moduleLookupKey(path, moduleKey));
|
||||||
? undefined
|
|
||||||
: context.moduleNodes.get(moduleLookupKey(path, runtime, moduleKey));
|
|
||||||
|
|||||||
@@ -78,12 +78,8 @@ const addReference = (
|
|||||||
omissions: { selfReferences: number },
|
omissions: { selfReferences: number },
|
||||||
): void => {
|
): void => {
|
||||||
const source =
|
const source =
|
||||||
sourceNodeFor(
|
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||||
context,
|
input.asset;
|
||||||
input.file.path,
|
|
||||||
input.value.module_key,
|
|
||||||
input.value.module_runtime,
|
|
||||||
) ?? input.asset;
|
|
||||||
const resolved =
|
const resolved =
|
||||||
input.value.specifier === null
|
input.value.specifier === null
|
||||||
? null
|
? null
|
||||||
@@ -166,12 +162,8 @@ const addEndpoint = (
|
|||||||
input: FindingInput<StaticEndpoint>,
|
input: FindingInput<StaticEndpoint>,
|
||||||
): void => {
|
): void => {
|
||||||
const source =
|
const source =
|
||||||
sourceNodeFor(
|
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||||
context,
|
input.asset;
|
||||||
input.file.path,
|
|
||||||
input.value.module_key,
|
|
||||||
input.value.module_runtime,
|
|
||||||
) ?? input.asset;
|
|
||||||
const key = compactStringIdentityKey(input.value.value);
|
const key = compactStringIdentityKey(input.value.value);
|
||||||
const label = `${input.value.kind} endpoint`;
|
const label = `${input.value.kind} endpoint`;
|
||||||
const node = context.accumulator.addNode({
|
const node = context.accumulator.addNode({
|
||||||
@@ -212,12 +204,8 @@ const addStorage = (
|
|||||||
input: FindingInput<StaticStorage>,
|
input: FindingInput<StaticStorage>,
|
||||||
): void => {
|
): void => {
|
||||||
const source =
|
const source =
|
||||||
sourceNodeFor(
|
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||||
context,
|
input.asset;
|
||||||
input.file.path,
|
|
||||||
input.value.module_key,
|
|
||||||
input.value.module_runtime,
|
|
||||||
) ?? input.asset;
|
|
||||||
const node = context.accumulator.addNode({
|
const node = context.accumulator.addNode({
|
||||||
kind: "storage",
|
kind: "storage",
|
||||||
identity: artifactLocalIdentity(
|
identity: artifactLocalIdentity(
|
||||||
@@ -274,12 +262,8 @@ const addDiscoveredRole = (
|
|||||||
coverage: input.coverage,
|
coverage: input.coverage,
|
||||||
});
|
});
|
||||||
const source =
|
const source =
|
||||||
sourceNodeFor(
|
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||||
context,
|
input.asset;
|
||||||
input.file.path,
|
|
||||||
input.value.module_key,
|
|
||||||
input.value.module_runtime,
|
|
||||||
) ?? input.asset;
|
|
||||||
addStaticInferenceEdge(context, {
|
addStaticInferenceEdge(context, {
|
||||||
source,
|
source,
|
||||||
target: role,
|
target: role,
|
||||||
@@ -375,12 +359,9 @@ const resolveReference = (
|
|||||||
if (specifier === null) return null;
|
if (specifier === null) return null;
|
||||||
const chunk = resolveBundlerChunkReference(context, sourcePath, reference);
|
const chunk = resolveBundlerChunkReference(context, sourcePath, reference);
|
||||||
if (chunk !== null) return chunk;
|
if (chunk !== null) return chunk;
|
||||||
const module =
|
const module = context.moduleNodes.get(
|
||||||
reference.module_runtime === undefined
|
moduleLookupKey(sourcePath, specifier),
|
||||||
? undefined
|
);
|
||||||
: context.moduleNodes.get(
|
|
||||||
moduleLookupKey(sourcePath, reference.module_runtime, specifier),
|
|
||||||
);
|
|
||||||
if (module !== undefined) {
|
if (module !== undefined) {
|
||||||
const file = context.filesByPath.get(sourcePath);
|
const file = context.filesByPath.get(sourcePath);
|
||||||
return file === undefined
|
return file === undefined
|
||||||
@@ -426,11 +407,10 @@ const resolveBundlerChunkReference = (
|
|||||||
if (file === undefined) return null;
|
if (file === undefined) return null;
|
||||||
const registrations = context.analysis.files
|
const registrations = context.analysis.files
|
||||||
.find(({ file: analyzedFile }) => analyzedFile.path === sourcePath)
|
.find(({ file: analyzedFile }) => analyzedFile.path === sourcePath)
|
||||||
?.javascript?.bundler_registrations.filter(({ runtime, modules }) =>
|
?.javascript?.bundler_registrations.filter(({ modules }) =>
|
||||||
reference.module_runtime === undefined
|
reference.module_key === null
|
||||||
? reference.module_key === null
|
? true
|
||||||
: runtime === reference.module_runtime &&
|
: modules.some(
|
||||||
modules.some(
|
|
||||||
({ module_key: moduleKey }) => moduleKey === reference.module_key,
|
({ module_key: moduleKey }) => moduleKey === reference.module_key,
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -2,9 +2,6 @@ import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySn
|
|||||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||||
import { completeApplicationCoverage } from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
import { completeApplicationCoverage } from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
||||||
import type { JavaScriptModuleArtifactAnalysis } from "./JavaScriptArtifactAnalysisTypes.js";
|
import type { JavaScriptModuleArtifactAnalysis } from "./JavaScriptArtifactAnalysisTypes.js";
|
||||||
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
|
||||||
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
|
|
||||||
import type { JavaScriptSourceRange } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
|
||||||
import type {
|
import type {
|
||||||
JavaScriptArtifactContainer,
|
JavaScriptArtifactContainer,
|
||||||
JavaScriptArtifactFile,
|
JavaScriptArtifactFile,
|
||||||
@@ -15,14 +12,12 @@ import {
|
|||||||
addUnavailableStaticParseScope,
|
addUnavailableStaticParseScope,
|
||||||
artifactFileNodeKind,
|
artifactFileNodeKind,
|
||||||
artifactLocalIdentity,
|
artifactLocalIdentity,
|
||||||
javascriptAnalysisCoverage,
|
|
||||||
createElectronRoleNode,
|
createElectronRoleNode,
|
||||||
linkElectronRoleToAsset,
|
linkElectronRoleToAsset,
|
||||||
type JavaScriptArtifactGraphContext,
|
type JavaScriptArtifactGraphContext,
|
||||||
} from "./JavaScriptArtifactGraphContext.js";
|
} from "./JavaScriptArtifactGraphContext.js";
|
||||||
import {
|
import {
|
||||||
artifactObservationEvidence,
|
artifactObservationEvidence,
|
||||||
astObservationEvidence,
|
|
||||||
staticInferenceEvidence,
|
staticInferenceEvidence,
|
||||||
} from "./JavaScriptArtifactGraphEvidence.js";
|
} from "./JavaScriptArtifactGraphEvidence.js";
|
||||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||||
@@ -120,52 +115,9 @@ export const addJavaScriptArtifactContainers = (
|
|||||||
export const addJavaScriptArtifactFiles = (
|
export const addJavaScriptArtifactFiles = (
|
||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
): void => {
|
): void => {
|
||||||
const packagesByPath = indexFirstByPath(context.analysis.packages);
|
|
||||||
const jsonModulesByPath = indexFirstByPath(context.analysis.json_modules);
|
|
||||||
const sourceMapsByPath = indexFirstByPath(context.analysis.source_maps);
|
|
||||||
for (const analyzed of context.analysis.files) {
|
for (const analyzed of context.analysis.files) {
|
||||||
const { file } = analyzed;
|
const { file } = analyzed;
|
||||||
const jsonValue = jsonModulesByPath.get(file.path);
|
const target = createFileTarget(context, file, analyzed.javascript);
|
||||||
const target = createFileTarget(
|
|
||||||
context,
|
|
||||||
file,
|
|
||||||
analyzed.javascript,
|
|
||||||
jsonValue,
|
|
||||||
);
|
|
||||||
if (
|
|
||||||
analyzed.application_projection_failure !== undefined &&
|
|
||||||
analyzed.javascript !== null
|
|
||||||
)
|
|
||||||
context.accumulator.addNode({
|
|
||||||
kind: target.kind,
|
|
||||||
identity: target.identity,
|
|
||||||
observations: [
|
|
||||||
{
|
|
||||||
label: file.path,
|
|
||||||
properties: {
|
|
||||||
static_analysis: freezeJsonSnapshot(
|
|
||||||
jsonValueSchema.parse(analyzed.javascript),
|
|
||||||
),
|
|
||||||
semantic_module_analysis:
|
|
||||||
analyzed.semantic === null
|
|
||||||
? null
|
|
||||||
: freezeJsonSnapshot(
|
|
||||||
jsonValueSchema.parse(analyzed.semantic.ir),
|
|
||||||
),
|
|
||||||
application_projection_failure:
|
|
||||||
analyzed.application_projection_failure.error,
|
|
||||||
},
|
|
||||||
evidence: astObservationEvidence({
|
|
||||||
sha256: file.sha256,
|
|
||||||
path: file.path,
|
|
||||||
operation: "retain-unexpanded-static-findings",
|
|
||||||
range: sourceFileRange(file),
|
|
||||||
coverage: javascriptAnalysisCoverage(analyzed.javascript),
|
|
||||||
limitations: [analyzed.application_projection_failure.reason],
|
|
||||||
}),
|
|
||||||
},
|
|
||||||
],
|
|
||||||
});
|
|
||||||
context.fileNodes.set(file.path, target);
|
context.fileNodes.set(file.path, target);
|
||||||
if (file.kind === "javascript") context.assetNodes.set(file.path, target);
|
if (file.kind === "javascript") context.assetNodes.set(file.path, target);
|
||||||
const entry = createAsarEntry(context, file);
|
const entry = createAsarEntry(context, file);
|
||||||
@@ -198,18 +150,7 @@ export const addJavaScriptArtifactFiles = (
|
|||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
if (analyzed.analysis_failure !== undefined)
|
if (
|
||||||
addUnavailableStaticParseScope(context, {
|
|
||||||
file,
|
|
||||||
asset: target,
|
|
||||||
operation: "analyze-javascript-source",
|
|
||||||
limitation: analyzed.analysis_failure.reason,
|
|
||||||
failure: analyzed.analysis_failure.error,
|
|
||||||
limits: javaScriptAnalysisCoverageLimits(
|
|
||||||
analyzed.analysis_failure.limits,
|
|
||||||
),
|
|
||||||
});
|
|
||||||
else if (
|
|
||||||
file.kind === "javascript" &&
|
file.kind === "javascript" &&
|
||||||
(analyzed.javascript === null ||
|
(analyzed.javascript === null ||
|
||||||
analyzed.javascript.parse_status === "failed")
|
analyzed.javascript.parse_status === "failed")
|
||||||
@@ -222,18 +163,9 @@ export const addJavaScriptArtifactFiles = (
|
|||||||
? "JavaScript syntax could not be parsed."
|
? "JavaScript syntax could not be parsed."
|
||||||
: `JavaScript text was unavailable: ${file.text.reason}.`,
|
: `JavaScript text was unavailable: ${file.text.reason}.`,
|
||||||
});
|
});
|
||||||
if (analyzed.application_projection_failure !== undefined)
|
const packageValue = context.analysis.packages.find(
|
||||||
addUnavailableStaticParseScope(context, {
|
({ path }) => path === file.path,
|
||||||
file,
|
);
|
||||||
asset: target,
|
|
||||||
operation: "project-javascript-application",
|
|
||||||
limitation: analyzed.application_projection_failure.reason,
|
|
||||||
failure: analyzed.application_projection_failure.error,
|
|
||||||
limits: javaScriptAnalysisCoverageLimits(
|
|
||||||
analyzed.application_projection_failure.limits,
|
|
||||||
),
|
|
||||||
});
|
|
||||||
const packageValue = packagesByPath.get(file.path);
|
|
||||||
if (packageValue !== undefined && packageValue.status !== "included")
|
if (packageValue !== undefined && packageValue.status !== "included")
|
||||||
addUnavailableStaticParseScope(context, {
|
addUnavailableStaticParseScope(context, {
|
||||||
file,
|
file,
|
||||||
@@ -241,6 +173,9 @@ export const addJavaScriptArtifactFiles = (
|
|||||||
operation: "parse-package-json",
|
operation: "parse-package-json",
|
||||||
limitation: packageValue.limitation,
|
limitation: packageValue.limitation,
|
||||||
});
|
});
|
||||||
|
const jsonValue = context.analysis.json_modules.find(
|
||||||
|
({ path }) => path === file.path,
|
||||||
|
);
|
||||||
if (jsonValue !== undefined && jsonValue.status !== "included")
|
if (jsonValue !== undefined && jsonValue.status !== "included")
|
||||||
addUnavailableStaticParseScope(context, {
|
addUnavailableStaticParseScope(context, {
|
||||||
file,
|
file,
|
||||||
@@ -248,7 +183,9 @@ export const addJavaScriptArtifactFiles = (
|
|||||||
operation: "parse-json-module",
|
operation: "parse-json-module",
|
||||||
limitation: jsonValue.limitation,
|
limitation: jsonValue.limitation,
|
||||||
});
|
});
|
||||||
const sourceMap = sourceMapsByPath.get(file.path);
|
const sourceMap = context.analysis.source_maps.find(
|
||||||
|
({ path }) => path === file.path,
|
||||||
|
);
|
||||||
if (sourceMap !== undefined && sourceMap.status === "invalid")
|
if (sourceMap !== undefined && sourceMap.status === "invalid")
|
||||||
addUnavailableStaticParseScope(context, {
|
addUnavailableStaticParseScope(context, {
|
||||||
file,
|
file,
|
||||||
@@ -259,23 +196,6 @@ export const addJavaScriptArtifactFiles = (
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const sourceFileRange = (
|
|
||||||
file: JavaScriptArtifactFile,
|
|
||||||
): JavaScriptSourceRange => {
|
|
||||||
if (!file.text.included)
|
|
||||||
throw new TypeError("Retained static facts require their source text");
|
|
||||||
let line = 1;
|
|
||||||
let lastLineStart = 0;
|
|
||||||
for (const match of file.text.value.matchAll(/\r\n|[\n\r\u2028\u2029]/gu)) {
|
|
||||||
line += 1;
|
|
||||||
lastLineStart = match.index + match[0].length;
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
start: { line: 1, column: 0 },
|
|
||||||
end: { line, column: file.text.value.length - lastLineStart },
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
/** Project package metadata and its declared Electron roles. */
|
/** Project package metadata and its declared Electron roles. */
|
||||||
export const addJavaScriptPackageNodes = (
|
export const addJavaScriptPackageNodes = (
|
||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
@@ -350,9 +270,11 @@ const createFileTarget = (
|
|||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
javascript: JavaScriptModuleArtifactAnalysis["files"][number]["javascript"],
|
javascript: JavaScriptModuleArtifactAnalysis["files"][number]["javascript"],
|
||||||
json: JavaScriptModuleArtifactAnalysis["json_modules"][number] | undefined,
|
|
||||||
): ApplicationNode => {
|
): ApplicationNode => {
|
||||||
const kind = artifactFileNodeKind(file.kind);
|
const kind = artifactFileNodeKind(file.kind);
|
||||||
|
const json = context.analysis.json_modules.find(
|
||||||
|
({ path }) => path === file.path,
|
||||||
|
);
|
||||||
return context.accumulator.addNode({
|
return context.accumulator.addNode({
|
||||||
kind,
|
kind,
|
||||||
identity: {
|
identity: {
|
||||||
@@ -387,16 +309,6 @@ const createFileTarget = (
|
|||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
const indexFirstByPath = <Value extends { readonly path: string }>(
|
|
||||||
values: readonly Value[],
|
|
||||||
): ReadonlyMap<string, Value> => {
|
|
||||||
const index = new Map<string, Value>();
|
|
||||||
for (const value of values) {
|
|
||||||
if (!index.has(value.path)) index.set(value.path, value);
|
|
||||||
}
|
|
||||||
return index;
|
|
||||||
};
|
|
||||||
|
|
||||||
const createAsarEntry = (
|
const createAsarEntry = (
|
||||||
context: JavaScriptArtifactGraphContext,
|
context: JavaScriptArtifactGraphContext,
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
@@ -485,4 +397,3 @@ const containerProperties = (container: JavaScriptArtifactContainer) => ({
|
|||||||
bytes: container.bytes,
|
bytes: container.bytes,
|
||||||
inventory_artifact_id: container.inventory_artifact_id,
|
inventory_artifact_id: container.inventory_artifact_id,
|
||||||
});
|
});
|
||||||
import { javaScriptAnalysisCoverageLimits } from "../../domain/javascript/javascriptAnalysisResourceControls.js";
|
|
||||||
|
|||||||
@@ -234,8 +234,7 @@ const resolvePackageSpecifier = (
|
|||||||
if (rawExports === undefined || rawExports === null) {
|
if (rawExports === undefined || rawExports === null) {
|
||||||
if (subpath === ".")
|
if (subpath === ".")
|
||||||
return resolveDirectory(input, directory, metadata, new Set());
|
return resolveDirectory(input, directory, metadata, new Set());
|
||||||
// Preserve the trailing slash for a package directory request.
|
const candidate = confineCandidate(posix.join(directory, subpath.slice(2)));
|
||||||
const candidate = confineCandidate(`${directory}/${subpath.slice(2)}`);
|
|
||||||
return typeof candidate === "string"
|
return typeof candidate === "string"
|
||||||
? resolveCandidate(input, candidate, new Set())
|
? resolveCandidate(input, candidate, new Set())
|
||||||
: candidate;
|
: candidate;
|
||||||
|
|||||||
@@ -11,22 +11,15 @@ import { createJavaScriptArtifactReader as createReader } from "../../artifacts/
|
|||||||
import type { JavaScriptApplicationGraph } from "../../domain/javascript/javascriptApplicationGraph.js";
|
import type { JavaScriptApplicationGraph } from "../../domain/javascript/javascriptApplicationGraph.js";
|
||||||
import type { JavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticGraph.js";
|
import type { JavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticGraph.js";
|
||||||
import type { ElectronBoundarySummary } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import type { ElectronBoundarySummary } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { parseOwnedJavaScriptApplicationAnalysisSteps } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
|
||||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
|
||||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||||
import type { ArtifactInventoryPartialObservation } from "../../domain/artifactPartialObservation.js";
|
import { analyzeAndProjectJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
|
||||||
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
|
||||||
import { analyzeAndAdoptJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
|
|
||||||
import { createJavaScriptSourceAnalysis } from "../../artifacts/javascript/JavaScriptSourceAnalysis.js";
|
|
||||||
import { JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS } from "../../domain/javascript/javascriptAnalysisResourceControls.js";
|
|
||||||
import { createOwnedJavaScriptApplicationEvidenceCooperatively } from "./JavaScriptApplicationEvidence.js";
|
|
||||||
import { readJavaScriptArtifactFiles } from "../../artifacts/javascript/JavaScriptArtifactFiles.js";
|
import { readJavaScriptArtifactFiles } from "../../artifacts/javascript/JavaScriptArtifactFiles.js";
|
||||||
import { buildImmutableJavaScriptArtifactGraphSteps } from "./JavaScriptArtifactGraphBuilder.js";
|
import { buildImmutableJavaScriptArtifactGraphSteps } from "./JavaScriptArtifactGraphBuilder.js";
|
||||||
import {
|
import {
|
||||||
javascriptArtifactReconstructionInputSchema,
|
javascriptArtifactReconstructionInputSchema,
|
||||||
type JavaScriptArtifactReconstructionInput,
|
type JavaScriptArtifactReconstructionInput,
|
||||||
} from "./JavaScriptArtifactReconstructionInput.js";
|
} from "./JavaScriptArtifactReconstructionInput.js";
|
||||||
import { scanCanonicalArtifactInventoryInScope } from "../../artifacts/inventory/scanCanonical.js";
|
import { scanCanonicalArtifactInventory } from "../../artifacts/inventory/scanCanonical.js";
|
||||||
import { summarizeElectronBoundaries } from "./ElectronBoundaryAnalysis.js";
|
import { summarizeElectronBoundaries } from "./ElectronBoundaryAnalysis.js";
|
||||||
import { createJavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
|
import { createJavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
|
||||||
import type { ProgressReporter } from "../ProgressReporter.js";
|
import type { ProgressReporter } from "../ProgressReporter.js";
|
||||||
@@ -63,241 +56,118 @@ export interface JavaScriptArtifactReconstructionResult {
|
|||||||
/** Reconstruct one local ASAR or extracted directory without executing code. */
|
/** Reconstruct one local ASAR or extracted directory without executing code. */
|
||||||
export const reconstructJavaScriptArtifact = async (
|
export const reconstructJavaScriptArtifact = async (
|
||||||
rawInput: unknown,
|
rawInput: unknown,
|
||||||
scope: ArtifactResourceScope,
|
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
progress?: ProgressReporter,
|
progress?: ProgressReporter,
|
||||||
): Promise<JavaScriptArtifactReconstructionResult> =>
|
): Promise<JavaScriptArtifactReconstructionResult> => {
|
||||||
scope.run(async () => {
|
const reportPhase = async (phase: string, message: string): Promise<void> => {
|
||||||
let effectiveSignal = signal;
|
await progress?.report({ phase, completed: 0, total: 1, message });
|
||||||
let effectiveProgress = progress;
|
await checkpointJavaScriptAnalysis(signal);
|
||||||
const reportPhase = async (
|
};
|
||||||
phase: string,
|
const input = javascriptArtifactReconstructionInputSchema.parse(rawInput);
|
||||||
message: string,
|
abortIfNeeded(signal);
|
||||||
): Promise<void> => {
|
const path = await resolveSelectedInput(input.input_path);
|
||||||
await effectiveProgress?.report({
|
const format = await resolveFormat(path, input);
|
||||||
phase,
|
const snapshot = await scanCanonicalArtifactInventory(path, {
|
||||||
completed: 0,
|
signal,
|
||||||
total: 1,
|
integrity: { mode: input.integrity_policy },
|
||||||
message,
|
|
||||||
});
|
|
||||||
await checkpointJavaScriptAnalysis(effectiveSignal);
|
|
||||||
};
|
|
||||||
const input = javascriptArtifactReconstructionInputSchema.parse(rawInput);
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
const path = await resolveSelectedInput(input.input_path);
|
|
||||||
const format = await resolveFormat(path, input);
|
|
||||||
const snapshot = await scanCanonicalArtifactInventoryInScope(path, {
|
|
||||||
resourceScope: scope,
|
|
||||||
signal,
|
|
||||||
integrity: { mode: input.integrity_policy },
|
|
||||||
});
|
|
||||||
if (snapshot.manifest.root_format !== format)
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"format",
|
|
||||||
`Artifact inventory classified ${path} as ${snapshot.manifest.root_format}, not ${format}`,
|
|
||||||
);
|
|
||||||
const reader = createReader(path, format);
|
|
||||||
const readerOwner = {
|
|
||||||
kind: "reader" as const,
|
|
||||||
reader,
|
|
||||||
resource: `JavaScript artifact reader for ${path}`,
|
|
||||||
};
|
|
||||||
let readerOpen = true;
|
|
||||||
let reconstructionOutcome:
|
|
||||||
| {
|
|
||||||
readonly kind: "completed";
|
|
||||||
readonly result: JavaScriptArtifactReconstructionResult;
|
|
||||||
}
|
|
||||||
| { readonly kind: "failed"; readonly cause: unknown };
|
|
||||||
try {
|
|
||||||
await reportPhase(
|
|
||||||
"read_javascript_artifacts",
|
|
||||||
"Reading inventoried JavaScript application sources",
|
|
||||||
);
|
|
||||||
const files = await readJavaScriptArtifactFiles(
|
|
||||||
reader,
|
|
||||||
snapshot,
|
|
||||||
scope,
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
// Release acquisition before analysis consumes these owned file facts.
|
|
||||||
const earlyCleanup = await scope.release(readerOwner);
|
|
||||||
readerOpen = false;
|
|
||||||
if (earlyCleanup.kind === "failed")
|
|
||||||
throw ArtifactReaderFailure.withCleanup(
|
|
||||||
earlyCleanup.cause,
|
|
||||||
ArtifactReaderFailure.cleanupObservation(
|
|
||||||
earlyCleanup.cause,
|
|
||||||
readerOwner.resource,
|
|
||||||
),
|
|
||||||
{ kind: "artifact-inventory", inventory: snapshot },
|
|
||||||
);
|
|
||||||
await reportPhase(
|
|
||||||
"parse_javascript_sources",
|
|
||||||
`Parsing and projecting ${String(files.files.length)} application source files`,
|
|
||||||
);
|
|
||||||
const semanticProjection = createJavaScriptSemanticGraphProjection();
|
|
||||||
const worker = createJavaScriptSourceAnalysis(
|
|
||||||
{
|
|
||||||
heapMb:
|
|
||||||
input.max_heap_mb ?? JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.heapMb,
|
|
||||||
timeoutMs:
|
|
||||||
input.analysis_timeout_ms ??
|
|
||||||
JAVASCRIPT_ANALYSIS_RESOURCE_DEFAULTS.timeoutMs,
|
|
||||||
},
|
|
||||||
files.files.filter(
|
|
||||||
(file) => file.kind === "javascript" && file.text.included,
|
|
||||||
).length,
|
|
||||||
);
|
|
||||||
let outcome;
|
|
||||||
let cleanupFailure: AnalysisError | undefined;
|
|
||||||
try {
|
|
||||||
outcome = await analyzeAndAdoptJavaScriptArtifactFiles(
|
|
||||||
files,
|
|
||||||
worker,
|
|
||||||
semanticProjection,
|
|
||||||
async (file, completed, total) => {
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
await progress?.report({
|
|
||||||
phase: "parse_javascript_source",
|
|
||||||
completed: 0,
|
|
||||||
total: 1,
|
|
||||||
message: `Parsing and projecting ${file.path} (${String(completed + 1)}/${String(total)})`,
|
|
||||||
});
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
},
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
} finally {
|
|
||||||
try {
|
|
||||||
const cleanup = await scope.release({
|
|
||||||
kind: "javascript-source-analysis",
|
|
||||||
analysis: worker,
|
|
||||||
resource: `JavaScript source analyzer for ${path}`,
|
|
||||||
});
|
|
||||||
if (cleanup.kind === "failed") throw cleanup.cause;
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (!(cause instanceof AnalysisError)) throw cause;
|
|
||||||
cleanupFailure = cause;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const analysis = outcome.analysis;
|
|
||||||
const interruption = cleanupFailure ?? outcome.interruption;
|
|
||||||
if (interruption !== null && interruption !== undefined) {
|
|
||||||
effectiveSignal = undefined;
|
|
||||||
effectiveProgress = undefined;
|
|
||||||
}
|
|
||||||
await checkpointJavaScriptAnalysis(effectiveSignal);
|
|
||||||
abortIfNeeded(effectiveSignal);
|
|
||||||
await reportPhase(
|
|
||||||
"build_javascript_application_graph",
|
|
||||||
"Constructing application and Electron boundary relationships",
|
|
||||||
);
|
|
||||||
const applicationGraphSteps = buildImmutableJavaScriptArtifactGraphSteps(
|
|
||||||
snapshot,
|
|
||||||
files,
|
|
||||||
analysis,
|
|
||||||
);
|
|
||||||
await reportPhase(
|
|
||||||
"seal_javascript_application_graph",
|
|
||||||
"Sealing the validated application graph",
|
|
||||||
);
|
|
||||||
const graph = await completeJavaScriptAnalysisSteps(
|
|
||||||
applicationGraphSteps,
|
|
||||||
effectiveSignal,
|
|
||||||
);
|
|
||||||
await reportPhase(
|
|
||||||
"build_javascript_semantic_graph",
|
|
||||||
"Binding and validating static semantic relationships",
|
|
||||||
);
|
|
||||||
const semanticGraphSteps = semanticProjection.finishImmutableSteps(
|
|
||||||
snapshot.manifest.root_sha256,
|
|
||||||
graph,
|
|
||||||
);
|
|
||||||
await reportPhase(
|
|
||||||
"seal_javascript_semantic_graph",
|
|
||||||
"Validating and sealing the semantic graph",
|
|
||||||
);
|
|
||||||
const semanticGraph = await completeJavaScriptAnalysisSteps(
|
|
||||||
semanticGraphSteps,
|
|
||||||
effectiveSignal,
|
|
||||||
);
|
|
||||||
await checkpointJavaScriptAnalysis(effectiveSignal);
|
|
||||||
const completed = {
|
|
||||||
input_path: path,
|
|
||||||
format,
|
|
||||||
root_artifact_sha256: snapshot.manifest.root_sha256,
|
|
||||||
inventory_manifest_id: snapshot.manifest.manifest_id,
|
|
||||||
inventory_graph_sha256: snapshot.manifest.graph_sha256,
|
|
||||||
integrity_contradictions: snapshot.integrity_contradictions,
|
|
||||||
graph,
|
|
||||||
semantic_graph: semanticGraph,
|
|
||||||
electron_summary: summarizeElectronBoundaries(analysis),
|
|
||||||
statistics: {
|
|
||||||
relevant_files: files.files.length,
|
|
||||||
nested_asar_containers: files.containers.length,
|
|
||||||
text_bytes_read: files.text_bytes_read,
|
|
||||||
invalid_utf8_files: files.invalid_utf8_files,
|
|
||||||
parsed_javascript_files: analysis.files.filter(
|
|
||||||
({ javascript }) => javascript !== null,
|
|
||||||
).length,
|
|
||||||
visited_ast_nodes: analysis.visited_ast_nodes,
|
|
||||||
findings: analysis.findings,
|
|
||||||
modules: analysis.modules,
|
|
||||||
parse_failures: analysis.parse_failures,
|
|
||||||
},
|
|
||||||
limitations: analysis.limitations,
|
|
||||||
};
|
|
||||||
if (interruption !== null && interruption !== undefined) {
|
|
||||||
const { electron_summary: summary, ...application } = completed;
|
|
||||||
const partial = await completeJavaScriptAnalysisSteps(
|
|
||||||
parseOwnedJavaScriptApplicationAnalysisSteps({
|
|
||||||
...application,
|
|
||||||
summary,
|
|
||||||
limitations: completed.graph.limitations,
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
const { max_heap_mb, analysis_timeout_ms, ...coreInput } = input;
|
|
||||||
const parameters = {
|
|
||||||
...coreInput,
|
|
||||||
...(max_heap_mb === undefined ? {} : { max_heap_mb }),
|
|
||||||
...(analysis_timeout_ms === undefined ? {} : { analysis_timeout_ms }),
|
|
||||||
};
|
|
||||||
const evidence =
|
|
||||||
await createOwnedJavaScriptApplicationEvidenceCooperatively(
|
|
||||||
parameters,
|
|
||||||
partial,
|
|
||||||
);
|
|
||||||
throw interruption.retainPartialObservation(evidence);
|
|
||||||
}
|
|
||||||
reconstructionOutcome = { kind: "completed", result: completed };
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
reconstructionOutcome = { kind: "failed", cause };
|
|
||||||
}
|
|
||||||
if (readerOpen) {
|
|
||||||
const cleanupAttempt = await scope.release(readerOwner);
|
|
||||||
if (cleanupAttempt.kind === "failed") {
|
|
||||||
const cleanup = ArtifactReaderFailure.cleanupObservation(
|
|
||||||
cleanupAttempt.cause,
|
|
||||||
readerOwner.resource,
|
|
||||||
);
|
|
||||||
const partialObservation: ArtifactInventoryPartialObservation = {
|
|
||||||
kind: "artifact-inventory",
|
|
||||||
inventory: snapshot,
|
|
||||||
};
|
|
||||||
throw ArtifactReaderFailure.withCleanup(
|
|
||||||
reconstructionOutcome.kind === "failed"
|
|
||||||
? reconstructionOutcome.cause
|
|
||||||
: cleanupAttempt.cause,
|
|
||||||
cleanup,
|
|
||||||
partialObservation,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (reconstructionOutcome.kind === "failed")
|
|
||||||
throw reconstructionOutcome.cause;
|
|
||||||
return reconstructionOutcome.result;
|
|
||||||
});
|
});
|
||||||
|
if (snapshot.manifest.root_format !== format)
|
||||||
|
throw new ArtifactReaderFailure(
|
||||||
|
"format",
|
||||||
|
`Artifact inventory classified ${path} as ${snapshot.manifest.root_format}, not ${format}`,
|
||||||
|
);
|
||||||
|
const reader = createReader(path, format);
|
||||||
|
try {
|
||||||
|
await reportPhase(
|
||||||
|
"read_javascript_artifacts",
|
||||||
|
"Reading inventoried JavaScript application sources",
|
||||||
|
);
|
||||||
|
const files = await readJavaScriptArtifactFiles(reader, snapshot, signal);
|
||||||
|
await reportPhase(
|
||||||
|
"parse_javascript_sources",
|
||||||
|
`Parsing and projecting ${String(files.files.length)} application source files`,
|
||||||
|
);
|
||||||
|
const semanticProjection = createJavaScriptSemanticGraphProjection();
|
||||||
|
const analysis = await analyzeAndProjectJavaScriptArtifactFiles(
|
||||||
|
files,
|
||||||
|
semanticProjection.projectFileSteps,
|
||||||
|
async (file, completed, total) => {
|
||||||
|
abortIfNeeded(signal);
|
||||||
|
await progress?.report({
|
||||||
|
phase: "parse_javascript_source",
|
||||||
|
completed: 0,
|
||||||
|
total: 1,
|
||||||
|
message: `Parsing and projecting ${file.path} (${String(completed + 1)}/${String(total)})`,
|
||||||
|
});
|
||||||
|
abortIfNeeded(signal);
|
||||||
|
},
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
await checkpointJavaScriptAnalysis(signal);
|
||||||
|
abortIfNeeded(signal);
|
||||||
|
await reportPhase(
|
||||||
|
"build_javascript_application_graph",
|
||||||
|
"Constructing application and Electron boundary relationships",
|
||||||
|
);
|
||||||
|
const applicationGraphSteps = buildImmutableJavaScriptArtifactGraphSteps(
|
||||||
|
snapshot,
|
||||||
|
files,
|
||||||
|
analysis,
|
||||||
|
);
|
||||||
|
await reportPhase(
|
||||||
|
"seal_javascript_application_graph",
|
||||||
|
"Sealing the validated application graph",
|
||||||
|
);
|
||||||
|
const graph = await completeJavaScriptAnalysisSteps(
|
||||||
|
applicationGraphSteps,
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
await reportPhase(
|
||||||
|
"build_javascript_semantic_graph",
|
||||||
|
"Binding and validating static semantic relationships",
|
||||||
|
);
|
||||||
|
const semanticGraphSteps = semanticProjection.finishImmutableSteps(
|
||||||
|
snapshot.manifest.root_sha256,
|
||||||
|
graph,
|
||||||
|
);
|
||||||
|
await reportPhase(
|
||||||
|
"seal_javascript_semantic_graph",
|
||||||
|
"Validating and sealing the semantic graph",
|
||||||
|
);
|
||||||
|
const semanticGraph = await completeJavaScriptAnalysisSteps(
|
||||||
|
semanticGraphSteps,
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
await checkpointJavaScriptAnalysis(signal);
|
||||||
|
return {
|
||||||
|
input_path: path,
|
||||||
|
format,
|
||||||
|
root_artifact_sha256: snapshot.manifest.root_sha256,
|
||||||
|
inventory_manifest_id: snapshot.manifest.manifest_id,
|
||||||
|
inventory_graph_sha256: snapshot.manifest.graph_sha256,
|
||||||
|
integrity_contradictions: snapshot.integrity_contradictions,
|
||||||
|
graph,
|
||||||
|
semantic_graph: semanticGraph,
|
||||||
|
electron_summary: summarizeElectronBoundaries(analysis),
|
||||||
|
statistics: {
|
||||||
|
relevant_files: files.files.length,
|
||||||
|
nested_asar_containers: files.containers.length,
|
||||||
|
text_bytes_read: files.text_bytes_read,
|
||||||
|
invalid_utf8_files: files.invalid_utf8_files,
|
||||||
|
parsed_javascript_files: analysis.files.filter(
|
||||||
|
({ javascript }) => javascript !== null,
|
||||||
|
).length,
|
||||||
|
visited_ast_nodes: analysis.visited_ast_nodes,
|
||||||
|
findings: analysis.findings,
|
||||||
|
modules: analysis.modules,
|
||||||
|
parse_failures: analysis.parse_failures,
|
||||||
|
},
|
||||||
|
limitations: analysis.limitations,
|
||||||
|
};
|
||||||
|
} finally {
|
||||||
|
await reader.close();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
const OPERATION = "analyze_javascript_application";
|
const OPERATION = "analyze_javascript_application";
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,11 @@
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import { artifactIntegrityPolicySchema } from "../../domain/artifactIntegrityPolicy.js";
|
import { artifactIntegrityPolicySchema } from "../../domain/artifactIntegrityPolicy.js";
|
||||||
import { javaScriptAnalysisResourceControlsSchema } from "../../domain/javascript/javascriptAnalysisResourceControls.js";
|
|
||||||
|
|
||||||
/** Local ASAR/directory reconstruction request. */
|
/** Local ASAR/directory reconstruction request. */
|
||||||
export const javascriptArtifactReconstructionInputSchema = z.strictObject({
|
export const javascriptArtifactReconstructionInputSchema = z.strictObject({
|
||||||
input_path: z.string().min(1),
|
input_path: z.string().min(1),
|
||||||
format: z.enum(["auto", "asar", "directory"]).default("auto"),
|
format: z.enum(["auto", "asar", "directory"]).default("auto"),
|
||||||
integrity_policy: artifactIntegrityPolicySchema,
|
integrity_policy: artifactIntegrityPolicySchema,
|
||||||
...javaScriptAnalysisResourceControlsSchema.shape,
|
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Parsed local reconstruction request. */
|
/** Parsed local reconstruction request. */
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ import { describe, expect, it } from "vitest";
|
|||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { staticPath } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
import { staticPath } from "../../domain/javascript/javascriptStaticAnalysisHelpers.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
const urlCases = [
|
const urlCases = [
|
||||||
["./preload%20name.cjs", "preload name.cjs", "main.mjs"],
|
["./preload%20name.cjs", "preload name.cjs", "main.mjs"],
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import { promisify } from "node:util";
|
|||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
|
|
||||||
const filesFor = (paths: readonly string[]) =>
|
const filesFor = (paths: readonly string[]) =>
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ import { expect, it } from "vitest";
|
|||||||
|
|
||||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||||
import { analyzeJavaScriptApplication } from "../../../tests/support/javascriptApplicationScope.js";
|
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||||
|
|
||||||
it("classifies overloaded open calls from lexical receiver facts", async () => {
|
it("classifies overloaded open calls from lexical receiver facts", async () => {
|
||||||
const inputPath = await createTestTempDirectory(
|
const inputPath = await createTestTempDirectory(
|
||||||
|
|||||||
@@ -355,7 +355,7 @@ const semanticNodeFor = (roleKey: string) => {
|
|||||||
unpacked: false,
|
unpacked: false,
|
||||||
text: { included: true, value: "" },
|
text: { included: true, value: "" },
|
||||||
};
|
};
|
||||||
const state = createSemanticGraphProjectionState();
|
const state = createSemanticGraphProjectionState({ nodes: [] });
|
||||||
return constructSemanticGraphNode(
|
return constructSemanticGraphNode(
|
||||||
file,
|
file,
|
||||||
{
|
{
|
||||||
@@ -565,7 +565,7 @@ it("does not mark a file truncated when it exactly fills its budget", () => {
|
|||||||
it("only records node loss when the budget rejects a new identity", () => {
|
it("only records node loss when the budget rejects a new identity", () => {
|
||||||
// The truncation signal must record blocked node creation, not a zero
|
// The truncation signal must record blocked node creation, not a zero
|
||||||
// remaining budget: a file using exactly its share drops nothing.
|
// remaining budget: a file using exactly its share drops nothing.
|
||||||
const state = createSemanticGraphProjectionState();
|
const state = createSemanticGraphProjectionState({ nodes: [] });
|
||||||
state.fileNodeBudget = 2;
|
state.fileNodeBudget = 2;
|
||||||
state.fileNodesDropped = false;
|
state.fileNodesDropped = false;
|
||||||
const first = addSemanticGraphNode(state, semanticNodeFor("first"));
|
const first = addSemanticGraphNode(state, semanticNodeFor("first"));
|
||||||
|
|||||||
@@ -1,6 +1,5 @@
|
|||||||
import {
|
import {
|
||||||
createJavaScriptSemanticGraph,
|
createJavaScriptSemanticGraph,
|
||||||
createJavaScriptSemanticGraphUnknown,
|
|
||||||
sealTransferredJavaScriptSemanticGraphSteps,
|
sealTransferredJavaScriptSemanticGraphSteps,
|
||||||
type JavaScriptSemanticGraph,
|
type JavaScriptSemanticGraph,
|
||||||
} from "../../domain/javascript/javascriptSemanticGraph.js";
|
} from "../../domain/javascript/javascriptSemanticGraph.js";
|
||||||
@@ -11,7 +10,6 @@ import type {
|
|||||||
import {
|
import {
|
||||||
JAVASCRIPT_SEMANTIC_RELATION_FAMILIES,
|
JAVASCRIPT_SEMANTIC_RELATION_FAMILIES,
|
||||||
JAVASCRIPT_SEMANTIC_RELATION_FAMILY,
|
JAVASCRIPT_SEMANTIC_RELATION_FAMILY,
|
||||||
JAVASCRIPT_SEMANTIC_RELATIONS,
|
|
||||||
} from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
|
} from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
|
||||||
import type {
|
import type {
|
||||||
JavaScriptSemanticCallArgument,
|
JavaScriptSemanticCallArgument,
|
||||||
@@ -35,13 +33,6 @@ import { projectSemanticResources } from "./JavaScriptSemanticGraphResourceProje
|
|||||||
import { projectSemanticObjects } from "./JavaScriptSemanticGraphObjectProjection.js";
|
import { projectSemanticObjects } from "./JavaScriptSemanticGraphObjectProjection.js";
|
||||||
import { projectSemanticValues } from "./JavaScriptSemanticGraphValueProjection.js";
|
import { projectSemanticValues } from "./JavaScriptSemanticGraphValueProjection.js";
|
||||||
import { inferredSemanticEvidenceAt } from "./JavaScriptSemanticGraphEvidence.js";
|
import { inferredSemanticEvidenceAt } from "./JavaScriptSemanticGraphEvidence.js";
|
||||||
import { observedSemanticEvidence } from "./JavaScriptSemanticGraphEvidence.js";
|
|
||||||
import {
|
|
||||||
partialApplicationCoverage,
|
|
||||||
type ApplicationCoverage,
|
|
||||||
} from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
|
||||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
|
||||||
import { javaScriptAnalysisCoverageLimits } from "../../domain/javascript/javascriptAnalysisResourceControls.js";
|
|
||||||
import { projectSemanticFunctionFingerprints } from "./JavaScriptSemanticGraphFingerprintProjection.js";
|
import { projectSemanticFunctionFingerprints } from "./JavaScriptSemanticGraphFingerprintProjection.js";
|
||||||
import {
|
import {
|
||||||
projectSemanticClosureCaptures,
|
projectSemanticClosureCaptures,
|
||||||
@@ -64,7 +55,6 @@ import {
|
|||||||
bindSemanticGraphApplicationNodes,
|
bindSemanticGraphApplicationNodes,
|
||||||
type SemanticGraphProjectionState as BuilderState,
|
type SemanticGraphProjectionState as BuilderState,
|
||||||
} from "./JavaScriptSemanticGraphConstruction.js";
|
} from "./JavaScriptSemanticGraphConstruction.js";
|
||||||
import type { JavaScriptSemanticFileProjection } from "../../domain/javascript/javascriptSemanticFileProjection.js";
|
|
||||||
|
|
||||||
interface BuilderInput {
|
interface BuilderInput {
|
||||||
readonly rootArtifactSha256: string;
|
readonly rootArtifactSha256: string;
|
||||||
@@ -136,19 +126,6 @@ export const buildJavaScriptSemanticGraph = ({
|
|||||||
|
|
||||||
/** File-local semantic projection that does not retain consumed source IR. */
|
/** File-local semantic projection that does not retain consumed source IR. */
|
||||||
export interface JavaScriptSemanticGraphProjection {
|
export interface JavaScriptSemanticGraphProjection {
|
||||||
/** Node capacity offered to the next independently analyzed source file. */
|
|
||||||
readonly remainingFileNodeBudget: () => number;
|
|
||||||
/** Adopt decoded file-local facts; final validation and proof remain in this realm. */
|
|
||||||
readonly adoptFileProjection: (
|
|
||||||
projection: JavaScriptSemanticFileProjection,
|
|
||||||
) => void;
|
|
||||||
/** Preserve source-bound unknowns for an execution or result-retention failure. */
|
|
||||||
readonly recordUnavailableFile: (
|
|
||||||
file: JavaScriptArtifactFile,
|
|
||||||
reason: string,
|
|
||||||
limits: Readonly<Record<string, JsonValue>> | null,
|
|
||||||
unknownReason: "resource-limit" | "incomplete-module",
|
|
||||||
) => void;
|
|
||||||
readonly projectFile: (
|
readonly projectFile: (
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
ir: JavaScriptSemanticIr,
|
ir: JavaScriptSemanticIr,
|
||||||
@@ -171,15 +148,9 @@ export interface JavaScriptSemanticGraphProjection {
|
|||||||
/** Accumulate semantic nodes while each source IR is still file-local. */
|
/** Accumulate semantic nodes while each source IR is still file-local. */
|
||||||
export const createJavaScriptSemanticGraphProjection =
|
export const createJavaScriptSemanticGraphProjection =
|
||||||
(): JavaScriptSemanticGraphProjection => {
|
(): JavaScriptSemanticGraphProjection => {
|
||||||
const state = emptyState();
|
const state = emptyState({ nodes: [] });
|
||||||
const fingerprints: JavaScriptSemanticFingerprint[] = [];
|
const fingerprints: JavaScriptSemanticFingerprint[] = [];
|
||||||
let truncatedFiles = 0;
|
let truncatedFiles = 0;
|
||||||
let unavailableFiles = 0;
|
|
||||||
let resourceTruncatedFiles = 0;
|
|
||||||
const executionLimits = new Map<
|
|
||||||
string,
|
|
||||||
ApplicationCoverage["limits"][number]
|
|
||||||
>();
|
|
||||||
const projectSourceSteps = function* (
|
const projectSourceSteps = function* (
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
ir: JavaScriptSemanticIr,
|
ir: JavaScriptSemanticIr,
|
||||||
@@ -236,15 +207,12 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
fingerprints: [...fingerprints],
|
fingerprints: [...fingerprints],
|
||||||
unknowns,
|
unknowns,
|
||||||
coverage: {
|
coverage: {
|
||||||
status:
|
status: truncatedFiles > 0 ? "partial" : "unknown",
|
||||||
truncatedFiles > 0 || unavailableFiles > 0 ? "partial" : "unknown",
|
truncated: truncatedFiles > 0,
|
||||||
truncated: truncatedFiles > 0 || resourceTruncatedFiles > 0,
|
omitted_nodes: truncatedFiles > 0 ? null : 0,
|
||||||
omitted_nodes: truncatedFiles > 0 || unavailableFiles > 0 ? null : 0,
|
omitted_relations: truncatedFiles > 0 ? null : 0,
|
||||||
omitted_relations:
|
limits:
|
||||||
truncatedFiles > 0 || unavailableFiles > 0 ? null : 0,
|
truncatedFiles > 0
|
||||||
limits: [
|
|
||||||
...executionLimits.values(),
|
|
||||||
...(truncatedFiles > 0
|
|
||||||
? [
|
? [
|
||||||
{
|
{
|
||||||
name: "semantic_graph_node_ceiling",
|
name: "semantic_graph_node_ceiling",
|
||||||
@@ -252,14 +220,12 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
unit: "items" as const,
|
unit: "items" as const,
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
: []),
|
: [],
|
||||||
],
|
|
||||||
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
|
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
|
||||||
family,
|
family,
|
||||||
status: semanticFamilyStatus(family),
|
status: semanticFamilyStatus(family),
|
||||||
retained_relations: retainedByFamily.get(family) ?? 0,
|
retained_relations: retainedByFamily.get(family) ?? 0,
|
||||||
omitted_relations:
|
omitted_relations: truncatedFiles > 0 ? null : 0,
|
||||||
truncatedFiles > 0 || unavailableFiles > 0 ? null : 0,
|
|
||||||
unknown_ids: unknownIdsByFamily.get(family) ?? [],
|
unknown_ids: unknownIdsByFamily.get(family) ?? [],
|
||||||
})),
|
})),
|
||||||
},
|
},
|
||||||
@@ -286,87 +252,6 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
return graph;
|
return graph;
|
||||||
};
|
};
|
||||||
return {
|
return {
|
||||||
remainingFileNodeBudget: () =>
|
|
||||||
Math.min(
|
|
||||||
SEMANTIC_GRAPH_FILE_NODE_CEILING,
|
|
||||||
Math.max(0, SEMANTIC_GRAPH_NODE_CEILING - state.nodes.size),
|
|
||||||
),
|
|
||||||
adoptFileProjection: (projection) => {
|
|
||||||
const newNodes = projection.nodes.filter(
|
|
||||||
({ node_id }) => !state.nodes.has(node_id),
|
|
||||||
);
|
|
||||||
if (
|
|
||||||
newNodes.length >
|
|
||||||
Math.min(
|
|
||||||
SEMANTIC_GRAPH_FILE_NODE_CEILING,
|
|
||||||
SEMANTIC_GRAPH_NODE_CEILING - state.nodes.size,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
throw new RangeError(
|
|
||||||
"Transferred semantic nodes exceed the offered tree capacity",
|
|
||||||
);
|
|
||||||
for (const context of projection.evidenceContexts)
|
|
||||||
state.evidenceContexts.adopt(context);
|
|
||||||
for (const node of newNodes) state.nodes.set(node.node_id, node);
|
|
||||||
for (const relation of projection.relations)
|
|
||||||
state.relations.set(relation.relation_id, relation);
|
|
||||||
for (const unknown of projection.unknowns)
|
|
||||||
state.unknowns.set(unknown.unknown_id, unknown);
|
|
||||||
for (const root of projection.roots) state.roots.add(root);
|
|
||||||
fingerprints.push(...projection.fingerprints);
|
|
||||||
if (projection.truncated) truncatedFiles += 1;
|
|
||||||
},
|
|
||||||
recordUnavailableFile: (file, reason, limits, unknownReason) => {
|
|
||||||
const retainedLimits = javaScriptAnalysisCoverageLimits(limits);
|
|
||||||
for (const limit of retainedLimits) {
|
|
||||||
executionLimits.set(
|
|
||||||
JSON.stringify([limit.name, limit.value, limit.unit]),
|
|
||||||
limit,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
state.fileNodeBudget = Math.min(
|
|
||||||
SEMANTIC_GRAPH_FILE_NODE_CEILING,
|
|
||||||
Math.max(0, SEMANTIC_GRAPH_NODE_CEILING - state.nodes.size),
|
|
||||||
);
|
|
||||||
const module = retainNode(state, file, {
|
|
||||||
kind: "module",
|
|
||||||
roleKey: "module",
|
|
||||||
location: null,
|
|
||||||
label: file.path,
|
|
||||||
functionNodeId: null,
|
|
||||||
});
|
|
||||||
state.fileNodeBudget = null;
|
|
||||||
if (module !== null) state.roots.add(module.node_id);
|
|
||||||
const evidence = {
|
|
||||||
...observedSemanticEvidence(file, null),
|
|
||||||
authority: "unknown" as const,
|
|
||||||
state: "unknown" as const,
|
|
||||||
confidence: "unknown" as const,
|
|
||||||
coverage: partialApplicationCoverage(retainedLimits, null),
|
|
||||||
limitations: [reason],
|
|
||||||
};
|
|
||||||
for (const family of JAVASCRIPT_SEMANTIC_RELATION_FAMILIES) {
|
|
||||||
const unknown = createJavaScriptSemanticGraphUnknown(
|
|
||||||
{
|
|
||||||
node_id: module?.node_id ?? null,
|
|
||||||
family,
|
|
||||||
relation_kinds: JAVASCRIPT_SEMANTIC_RELATIONS.filter(
|
|
||||||
(relation) =>
|
|
||||||
JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation] === family,
|
|
||||||
),
|
|
||||||
reason: unknownReason,
|
|
||||||
detail: reason,
|
|
||||||
candidate_node_ids: [],
|
|
||||||
evidence,
|
|
||||||
},
|
|
||||||
state.evidenceContexts,
|
|
||||||
);
|
|
||||||
state.unknowns.set(unknown.unknown_id, unknown);
|
|
||||||
}
|
|
||||||
unavailableFiles += 1;
|
|
||||||
if (unknownReason === "resource-limit" && retainedLimits.length > 0)
|
|
||||||
resourceTruncatedFiles += 1;
|
|
||||||
},
|
|
||||||
projectFile: (file, ir) => {
|
projectFile: (file, ir) => {
|
||||||
const steps = projectSourceSteps(file, ir);
|
const steps = projectSourceSteps(file, ir);
|
||||||
while (steps.next().done !== true);
|
while (steps.next().done !== true);
|
||||||
@@ -387,35 +272,6 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Project one source with the parent's remaining node capacity, retaining no full IR. */
|
|
||||||
export function* projectJavaScriptSemanticFileSteps(
|
|
||||||
file: JavaScriptArtifactFile,
|
|
||||||
ir: JavaScriptSemanticIr,
|
|
||||||
nodeBudget: number,
|
|
||||||
): Generator<void, JavaScriptSemanticFileProjection> {
|
|
||||||
if (
|
|
||||||
!Number.isSafeInteger(nodeBudget) ||
|
|
||||||
nodeBudget < 0 ||
|
|
||||||
nodeBudget > SEMANTIC_GRAPH_FILE_NODE_CEILING
|
|
||||||
)
|
|
||||||
throw new RangeError(
|
|
||||||
"File semantic projection requires a valid offered node capacity",
|
|
||||||
);
|
|
||||||
const state = emptyState();
|
|
||||||
state.fileNodeBudget = nodeBudget;
|
|
||||||
state.fileNodesDropped = false;
|
|
||||||
const fingerprints = yield* projectFileSteps(file, ir, state);
|
|
||||||
return {
|
|
||||||
roots: [...state.roots],
|
|
||||||
evidenceContexts: state.evidenceContexts.contexts,
|
|
||||||
nodes: [...state.nodes.values()],
|
|
||||||
relations: [...state.relations.values()],
|
|
||||||
unknowns: [...state.unknowns.values()],
|
|
||||||
fingerprints,
|
|
||||||
truncated: Boolean(state.fileNodesDropped),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
function* projectFileSteps(
|
function* projectFileSteps(
|
||||||
file: JavaScriptArtifactFile,
|
file: JavaScriptArtifactFile,
|
||||||
ir: JavaScriptSemanticIr,
|
ir: JavaScriptSemanticIr,
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ export interface SemanticGraphProjectionState {
|
|||||||
readonly unknowns: Map<string, JavaScriptSemanticGraphUnknown>;
|
readonly unknowns: Map<string, JavaScriptSemanticGraphUnknown>;
|
||||||
readonly evidenceContexts: JavaScriptSemanticEvidenceContextRegistry;
|
readonly evidenceContexts: JavaScriptSemanticEvidenceContextRegistry;
|
||||||
readonly roots: Set<string>;
|
readonly roots: Set<string>;
|
||||||
|
readonly applicationNodeIdsByLocation: ReadonlyMap<string, readonly string[]>;
|
||||||
/**
|
/**
|
||||||
* Remaining node budget for the file currently being projected.
|
* Remaining node budget for the file currently being projected.
|
||||||
*
|
*
|
||||||
@@ -68,29 +69,32 @@ export interface SemanticRelationConstructionInput {
|
|||||||
readonly properties?: Readonly<Record<string, JsonValue>>;
|
readonly properties?: Readonly<Record<string, JsonValue>>;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Create empty projection state; application-node links are bound at finish. */
|
/** Create empty projection state with structural application-node mappings. */
|
||||||
export const createSemanticGraphProjectionState =
|
export const createSemanticGraphProjectionState = (
|
||||||
(): SemanticGraphProjectionState => ({
|
applicationGraph: Pick<JavaScriptApplicationGraph, "nodes">,
|
||||||
|
): SemanticGraphProjectionState => {
|
||||||
|
const index = indexApplicationNodes(applicationGraph.nodes);
|
||||||
|
return {
|
||||||
nodes: new Map(),
|
nodes: new Map(),
|
||||||
relations: new Map(),
|
relations: new Map(),
|
||||||
unknowns: new Map(),
|
unknowns: new Map(),
|
||||||
evidenceContexts: new JavaScriptSemanticEvidenceContextRegistry(),
|
evidenceContexts: new JavaScriptSemanticEvidenceContextRegistry(),
|
||||||
roots: new Set(),
|
roots: new Set(),
|
||||||
});
|
applicationNodeIdsByLocation: index.identifiers,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
/** Bind retained semantic identities after all application observations exist. */
|
/** Bind retained semantic identities after all application observations exist. */
|
||||||
export const bindSemanticGraphApplicationNodes = (
|
export const bindSemanticGraphApplicationNodes = (
|
||||||
state: SemanticGraphProjectionState,
|
state: SemanticGraphProjectionState,
|
||||||
applicationGraph: Pick<JavaScriptApplicationGraph, "nodes">,
|
applicationGraph: Pick<JavaScriptApplicationGraph, "nodes">,
|
||||||
): void => {
|
): void => {
|
||||||
const applicationNodeIdsByLocation = indexApplicationNodes(
|
const index = indexApplicationNodes(applicationGraph.nodes).identifiers;
|
||||||
applicationGraph.nodes,
|
|
||||||
);
|
|
||||||
for (const [identifier, node] of state.nodes) {
|
for (const [identifier, node] of state.nodes) {
|
||||||
state.nodes.set(identifier, {
|
state.nodes.set(identifier, {
|
||||||
...node,
|
...node,
|
||||||
application_node_ids: [
|
application_node_ids: [
|
||||||
...(applicationNodeIdsByLocation.get(
|
...(index.get(
|
||||||
applicationLocationKey(
|
applicationLocationKey(
|
||||||
node.identity.artifact_sha256,
|
node.identity.artifact_sha256,
|
||||||
node.identity.module_path,
|
node.identity.module_path,
|
||||||
@@ -118,7 +122,7 @@ export const constructSemanticGraphNode = (
|
|||||||
role_key: input.roleKey,
|
role_key: input.roleKey,
|
||||||
},
|
},
|
||||||
function_node_id: input.functionNodeId,
|
function_node_id: input.functionNodeId,
|
||||||
application_node_ids: [],
|
application_node_ids: matchingApplicationNodeIds(file, input, state),
|
||||||
label: input.label === null ? null : javascriptDisplayText(input.label),
|
label: input.label === null ? null : javascriptDisplayText(input.label),
|
||||||
properties: input.properties ?? {},
|
properties: input.properties ?? {},
|
||||||
evidence: observedSemanticEvidence(file, input.location),
|
evidence: observedSemanticEvidence(file, input.location),
|
||||||
@@ -243,7 +247,9 @@ export const addSemanticFallbackRoot = (
|
|||||||
|
|
||||||
const indexApplicationNodes = (
|
const indexApplicationNodes = (
|
||||||
nodes: JavaScriptApplicationGraph["nodes"],
|
nodes: JavaScriptApplicationGraph["nodes"],
|
||||||
): ReadonlyMap<string, readonly string[]> => {
|
): {
|
||||||
|
readonly identifiers: ReadonlyMap<string, readonly string[]>;
|
||||||
|
} => {
|
||||||
const identifiers = new Map<string, Set<string>>();
|
const identifiers = new Map<string, Set<string>>();
|
||||||
for (const node of nodes) {
|
for (const node of nodes) {
|
||||||
for (const observation of node.observations) {
|
for (const observation of node.observations) {
|
||||||
@@ -274,9 +280,19 @@ const indexApplicationNodes = (
|
|||||||
return [key, sorted];
|
return [key, sorted];
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
return indexed;
|
return { identifiers: indexed };
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const matchingApplicationNodeIds = (
|
||||||
|
file: JavaScriptArtifactFile,
|
||||||
|
input: SemanticNodeConstructionInput,
|
||||||
|
state: SemanticGraphProjectionState,
|
||||||
|
): string[] => [
|
||||||
|
...(state.applicationNodeIdsByLocation.get(
|
||||||
|
applicationLocationKey(file.sha256, file.path, input.location),
|
||||||
|
) ?? []),
|
||||||
|
];
|
||||||
|
|
||||||
const addApplicationNodeIdentifier = (
|
const addApplicationNodeIdentifier = (
|
||||||
identifiers: Map<string, Set<string>>,
|
identifiers: Map<string, Set<string>>,
|
||||||
key: string,
|
key: string,
|
||||||
|
|||||||
@@ -325,7 +325,8 @@ const resolveUiAction = (
|
|||||||
(node) =>
|
(node) =>
|
||||||
["control", "view", "view_controller", "scene"].includes(node.kind) &&
|
["control", "view", "view_controller", "scene"].includes(node.kind) &&
|
||||||
(node.id === query ||
|
(node.id === query ||
|
||||||
node.attributes.interface_builder_object_id === query),
|
node.attributes.interface_builder_object_id === query ||
|
||||||
|
node.id.endsWith(`:object:${query}`)),
|
||||||
);
|
);
|
||||||
const actionIds = new Set(
|
const actionIds = new Set(
|
||||||
graph.edges.flatMap((edge) =>
|
graph.edges.flatMap((edge) =>
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ import {
|
|||||||
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
|
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
|
||||||
import { functionDossierSchema } from "../../domain/hopperValues.js";
|
import { functionDossierSchema } from "../../domain/hopperValues.js";
|
||||||
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
||||||
import { nativeValueTraceSchema } from "../../domain/native/nativeValueTrace.js";
|
|
||||||
import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js";
|
import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js";
|
||||||
import { ok, err } from "../../domain/result.js";
|
import { ok, err } from "../../domain/result.js";
|
||||||
|
|
||||||
@@ -103,132 +102,6 @@ const analysis: AnalysisOperationPort = {
|
|||||||
);
|
);
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
describe("native value parameter-use node membership", () => {
|
|
||||||
it("retains only parameter uses whose parameter nodes fit the node budget", async () => {
|
|
||||||
const operationCount = 3_000;
|
|
||||||
const parameterUseCount = 12_000;
|
|
||||||
const base = functionDossierSchema.parse(ghidraFunctionDossier());
|
|
||||||
const procedureAddress = base.procedure.address;
|
|
||||||
const largeFlow = {
|
|
||||||
available: true as const,
|
|
||||||
provenance: "ghidra-high-pcode" as const,
|
|
||||||
operations: Array.from({ length: operationCount }, (_, index) => ({
|
|
||||||
...node(`operation-${String(index)}`, procedureAddress, "MULTIEQUAL"),
|
|
||||||
inputs: [scalar, scalar, scalar, scalar],
|
|
||||||
})),
|
|
||||||
def_use: [],
|
|
||||||
effects: [],
|
|
||||||
parameters: [{ ordinal: 0, name: "value", data_type: "/int" }],
|
|
||||||
parameter_uses: Array.from({ length: parameterUseCount }, (_, index) => ({
|
|
||||||
ordinal: 0,
|
|
||||||
use: `operation-${String(index % operationCount)}`,
|
|
||||||
input_index: Math.floor(index / operationCount),
|
|
||||||
})),
|
|
||||||
truncated: false,
|
|
||||||
omitted_operations_lower_bound: 0,
|
|
||||||
known_omitted_inputs: 0,
|
|
||||||
known_omitted_edges: 0,
|
|
||||||
limitations: [],
|
|
||||||
};
|
|
||||||
const largeDossier = {
|
|
||||||
...base,
|
|
||||||
native_value_flow: largeFlow,
|
|
||||||
};
|
|
||||||
const largeAnalysis: AnalysisOperationPort = {
|
|
||||||
execute: async () =>
|
|
||||||
ok(createAnalysisExecution(largeDossier, provider, { subject })),
|
|
||||||
};
|
|
||||||
|
|
||||||
const includedParameter = await traceNativeValues(largeAnalysis, {
|
|
||||||
procedure: procedureAddress,
|
|
||||||
max_nodes: 20_000,
|
|
||||||
max_edges: 40_000,
|
|
||||||
offset: operationCount,
|
|
||||||
limit: 1,
|
|
||||||
});
|
|
||||||
if (!includedParameter.ok) throw includedParameter.error;
|
|
||||||
const includedResult = nativeValueTraceSchema.parse(
|
|
||||||
includedParameter.value,
|
|
||||||
);
|
|
||||||
expect(includedResult).toMatchObject({
|
|
||||||
total_nodes: operationCount + 1,
|
|
||||||
total_edges: parameterUseCount,
|
|
||||||
nodes: [
|
|
||||||
expect.objectContaining({
|
|
||||||
id: `${procedureAddress}/parameter:0`,
|
|
||||||
kind: "parameter",
|
|
||||||
}),
|
|
||||||
],
|
|
||||||
truncated: true,
|
|
||||||
});
|
|
||||||
expect(includedResult.edges).toHaveLength(parameterUseCount);
|
|
||||||
|
|
||||||
const omittedParameter = await traceNativeValues(largeAnalysis, {
|
|
||||||
procedure: procedureAddress,
|
|
||||||
max_nodes: operationCount,
|
|
||||||
max_edges: 40_000,
|
|
||||||
offset: operationCount - 1,
|
|
||||||
limit: 1,
|
|
||||||
});
|
|
||||||
if (!omittedParameter.ok) throw omittedParameter.error;
|
|
||||||
const omittedResult = nativeValueTraceSchema.parse(omittedParameter.value);
|
|
||||||
expect(omittedResult).toMatchObject({
|
|
||||||
total_nodes: operationCount,
|
|
||||||
total_edges: 0,
|
|
||||||
nodes: [expect.objectContaining({ kind: "operation" })],
|
|
||||||
edges: [],
|
|
||||||
truncated: true,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
it("preserves operation IDs that collide with generated parameter IDs", async () => {
|
|
||||||
const base = functionDossierSchema.parse(ghidraFunctionDossier());
|
|
||||||
const address = base.procedure.address;
|
|
||||||
const execution = createAnalysisExecution(
|
|
||||||
{
|
|
||||||
...base,
|
|
||||||
native_value_flow: {
|
|
||||||
available: true,
|
|
||||||
provenance: "ghidra-high-pcode",
|
|
||||||
operations: [
|
|
||||||
node("parameter:0", address, "COPY"),
|
|
||||||
node("use", address, "COPY"),
|
|
||||||
],
|
|
||||||
def_use: [],
|
|
||||||
effects: [],
|
|
||||||
parameters: [{ ordinal: 0, name: "value", data_type: "/int" }],
|
|
||||||
parameter_uses: [{ ordinal: 0, use: "use", input_index: 1 }],
|
|
||||||
truncated: false,
|
|
||||||
omitted_operations_lower_bound: 0,
|
|
||||||
known_omitted_inputs: 0,
|
|
||||||
known_omitted_edges: 0,
|
|
||||||
limitations: [],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
provider,
|
|
||||||
{ subject },
|
|
||||||
);
|
|
||||||
const result = await traceNativeValues(
|
|
||||||
{ execute: async () => ok(execution) },
|
|
||||||
{ procedure: address, max_nodes: 2 },
|
|
||||||
);
|
|
||||||
|
|
||||||
if (!result.ok) throw result.error;
|
|
||||||
const parsedResult = nativeValueTraceSchema.parse(result.value);
|
|
||||||
expect(parsedResult).toMatchObject({
|
|
||||||
total_nodes: 2,
|
|
||||||
total_edges: 1,
|
|
||||||
edges: [
|
|
||||||
{
|
|
||||||
source: `${address}/parameter:0`,
|
|
||||||
target: `${address}/use`,
|
|
||||||
kind: "parameter-use",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("bounded native value dependency composition", () => {
|
describe("bounded native value dependency composition", () => {
|
||||||
it("reports work truncation and stable pagination without traversing ambiguous calls", async () => {
|
it("reports work truncation and stable pagination without traversing ambiguous calls", async () => {
|
||||||
const bounded = await traceNativeValues(analysis, {
|
const bounded = await traceNativeValues(analysis, {
|
||||||
|
|||||||
@@ -21,7 +21,6 @@ export const traceNativeValues = async (
|
|||||||
): EnhancedResult => {
|
): EnhancedResult => {
|
||||||
const input = nativeValueTraceInputSchema.parse(parameters);
|
const input = nativeValueTraceInputSchema.parse(parameters);
|
||||||
const nodes: typeof nativeValueTraceSchema._output.nodes = [];
|
const nodes: typeof nativeValueTraceSchema._output.nodes = [];
|
||||||
const nodeIds = new Set<string>();
|
|
||||||
const edges: typeof nativeValueTraceSchema._output.edges = [];
|
const edges: typeof nativeValueTraceSchema._output.edges = [];
|
||||||
const procedures: typeof nativeValueTraceSchema._output.procedures = [];
|
const procedures: typeof nativeValueTraceSchema._output.procedures = [];
|
||||||
const unknowns: typeof nativeValueTraceSchema._output.unknowns = [];
|
const unknowns: typeof nativeValueTraceSchema._output.unknowns = [];
|
||||||
@@ -155,28 +154,25 @@ export const traceNativeValues = async (
|
|||||||
bytes += size;
|
bytes += size;
|
||||||
ids.add(operation.id);
|
ids.add(operation.id);
|
||||||
nodes.push(node);
|
nodes.push(node);
|
||||||
nodeIds.add(node.id);
|
|
||||||
}
|
}
|
||||||
for (const parameter of flow.parameters) {
|
for (const parameter of flow.parameters) {
|
||||||
if (nodes.length >= input.max_nodes) {
|
if (nodes.length >= input.max_nodes) {
|
||||||
truncated = true;
|
truncated = true;
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
const id = `${address}/parameter:${parameter.ordinal}`;
|
|
||||||
nodes.push({
|
nodes.push({
|
||||||
id,
|
id: `${address}/parameter:${parameter.ordinal}`,
|
||||||
procedure: address,
|
procedure: address,
|
||||||
evidence_id: evidenceId,
|
evidence_id: evidenceId,
|
||||||
kind: "parameter",
|
kind: "parameter",
|
||||||
operation: null,
|
operation: null,
|
||||||
parameter,
|
parameter,
|
||||||
});
|
});
|
||||||
nodeIds.add(id);
|
|
||||||
}
|
}
|
||||||
for (const use of flow.parameter_uses) {
|
for (const use of flow.parameter_uses) {
|
||||||
if (
|
if (
|
||||||
!ids.has(use.use) ||
|
!ids.has(use.use) ||
|
||||||
!nodeIds.has(`${address}/parameter:${use.ordinal}`)
|
!nodes.some((node) => node.id === `${address}/parameter:${use.ordinal}`)
|
||||||
)
|
)
|
||||||
continue;
|
continue;
|
||||||
if (edges.length >= input.max_edges) {
|
if (edges.length >= input.max_edges) {
|
||||||
|
|||||||
@@ -60,30 +60,6 @@ describe("English Unicode case collisions", () => {
|
|||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("retains every spelling in a high-cardinality folded sibling group", () => {
|
|
||||||
const count = 8_192;
|
|
||||||
const variants = caseVariants("abcdefghijklm", count);
|
|
||||||
const { limitation, byPath } = annotate(variants);
|
|
||||||
const ordered = [...variants].sort();
|
|
||||||
const first = ordered[0];
|
|
||||||
const second = ordered[1];
|
|
||||||
const original = variants[0];
|
|
||||||
if (first === undefined || second === undefined || original === undefined) {
|
|
||||||
throw new Error("Expected at least two case variants");
|
|
||||||
}
|
|
||||||
expect(limitation).toBe(ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION);
|
|
||||||
expect(byPath.size).toBe(count);
|
|
||||||
expect(byPath.get(first)).toEqual([
|
|
||||||
`Logical path ${first} collides under English (en-US) Unicode case folding with ${second} and ${String(count - 2)} other spellings; ${NOTE}`,
|
|
||||||
]);
|
|
||||||
expect(byPath.get(original)?.[0]).toContain(
|
|
||||||
`with ${first} and ${String(count - 2)} other spellings;`,
|
|
||||||
);
|
|
||||||
expect(
|
|
||||||
[...byPath.values()].every((limitations) => limitations.length === 1),
|
|
||||||
).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("stays linear for large case-variant subtrees", () => {
|
it("stays linear for large case-variant subtrees", () => {
|
||||||
const count = 5_000;
|
const count = 5_000;
|
||||||
const paths = Array.from({ length: count }, (_, index) => [
|
const paths = Array.from({ length: count }, (_, index) => [
|
||||||
@@ -105,12 +81,3 @@ describe("English Unicode case collisions", () => {
|
|||||||
expect(byPath.get(deep)).toEqual([]);
|
expect(byPath.get(deep)).toEqual([]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
const caseVariants = (lowercase: string, count: number): string[] =>
|
|
||||||
Array.from({ length: count }, (_, mask) =>
|
|
||||||
[...lowercase]
|
|
||||||
.map((character, index) =>
|
|
||||||
(mask & (1 << index)) === 0 ? character : character.toUpperCase(),
|
|
||||||
)
|
|
||||||
.join(""),
|
|
||||||
);
|
|
||||||
|
|||||||
@@ -177,12 +177,12 @@ const englishUnicodeCaseCollisions = (
|
|||||||
if (path.length === 0 || path === ".") continue;
|
if (path.length === 0 || path === ".") continue;
|
||||||
let node = root;
|
let node = root;
|
||||||
for (const part of path.split("/")) {
|
for (const part of path.split("/")) {
|
||||||
|
const folded = englishCaseFold(part);
|
||||||
|
const spellings = node.spellingsByFold.get(folded);
|
||||||
|
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
|
||||||
|
else if (!spellings.includes(part)) spellings.push(part);
|
||||||
let child = node.bySpelling.get(part);
|
let child = node.bySpelling.get(part);
|
||||||
if (child === undefined) {
|
if (child === undefined) {
|
||||||
const folded = englishCaseFold(part);
|
|
||||||
const spellings = node.spellingsByFold.get(folded);
|
|
||||||
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
|
|
||||||
else spellings.push(part);
|
|
||||||
child = emptyCaseNode();
|
child = emptyCaseNode();
|
||||||
node.bySpelling.set(part, child);
|
node.bySpelling.set(part, child);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -33,12 +33,10 @@ import type { JsonValue } from "../domain/jsonValue.js";
|
|||||||
import { interfaceBuilderLimitsSchema } from "../domain/apple/interfaceBuilderGraph.js";
|
import { interfaceBuilderLimitsSchema } from "../domain/apple/interfaceBuilderGraph.js";
|
||||||
import { err, ok, type Result } from "../domain/result.js";
|
import { err, ok, type Result } from "../domain/result.js";
|
||||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||||
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
|
|
||||||
import { artifactCapabilities } from "./ArtifactProviderMetadata.js";
|
import { artifactCapabilities } from "./ArtifactProviderMetadata.js";
|
||||||
import { ARTIFACT_GRAPH_PROVIDER } from "../application/InvestigationProviders.js";
|
import { ARTIFACT_GRAPH_PROVIDER } from "../application/InvestigationProviders.js";
|
||||||
import { createEvidence } from "../domain/evidence.js";
|
import { createEvidence } from "../domain/evidence.js";
|
||||||
import { createArtifactInspection } from "../domain/artifactInspection.js";
|
import { createArtifactInspection } from "../domain/artifactInspection.js";
|
||||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
|
||||||
|
|
||||||
/** Read-only inventory and exclusively owned extraction provider. */
|
/** Read-only inventory and exclusively owned extraction provider. */
|
||||||
export class ArtifactProvider implements AnalysisProvider {
|
export class ArtifactProvider implements AnalysisProvider {
|
||||||
@@ -69,8 +67,6 @@ export class ArtifactProvider implements AnalysisProvider {
|
|||||||
}
|
}
|
||||||
|
|
||||||
class ArtifactClient implements AnalysisClient {
|
class ArtifactClient implements AnalysisClient {
|
||||||
readonly #resourceScope = new ArtifactResourceScope();
|
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private readonly target: BinaryTarget,
|
private readonly target: BinaryTarget,
|
||||||
private readonly environment: Readonly<NodeJS.ProcessEnv>,
|
private readonly environment: Readonly<NodeJS.ProcessEnv>,
|
||||||
@@ -188,7 +184,6 @@ class ArtifactClient implements AnalysisClient {
|
|||||||
outputRoot: parsed.output_root,
|
outputRoot: parsed.output_root,
|
||||||
environment: this.environment,
|
environment: this.environment,
|
||||||
integrityPolicy: parsed.integrity_policy,
|
integrityPolicy: parsed.integrity_policy,
|
||||||
resourceScope: this.#resourceScope,
|
|
||||||
},
|
},
|
||||||
options?.signal,
|
options?.signal,
|
||||||
);
|
);
|
||||||
@@ -225,25 +220,7 @@ class ArtifactClient implements AnalysisClient {
|
|||||||
}
|
}
|
||||||
|
|
||||||
close(): Promise<Result<null, AnalysisError>> {
|
close(): Promise<Result<null, AnalysisError>> {
|
||||||
return this.#resourceScope.close().then(
|
return Promise.resolve(ok(null));
|
||||||
() => ok(null),
|
|
||||||
(cause: unknown) => {
|
|
||||||
const failure =
|
|
||||||
cause instanceof ArtifactReaderFailure ? cause : undefined;
|
|
||||||
return err(
|
|
||||||
new ProviderCleanupError(
|
|
||||||
ARTIFACT_GRAPH_PROVIDER.id,
|
|
||||||
failure?.cleanup?.resources ?? ["artifact resources"],
|
|
||||||
{
|
|
||||||
reason:
|
|
||||||
failure?.cleanup?.reason ??
|
|
||||||
(cause instanceof Error ? cause.message : String(cause)),
|
|
||||||
},
|
|
||||||
{ cause, operation: "close_binary" },
|
|
||||||
),
|
|
||||||
);
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The active target's kind is outside this operation's supported targets. */
|
/** The active target's kind is outside this operation's supported targets. */
|
||||||
@@ -376,7 +353,6 @@ class ArtifactClient implements AnalysisClient {
|
|||||||
options?: ExecutionOptions,
|
options?: ExecutionOptions,
|
||||||
) {
|
) {
|
||||||
return inventoryArtifact(this.target.sourcePath ?? this.target.path, {
|
return inventoryArtifact(this.target.sourcePath ?? this.target.path, {
|
||||||
resourceScope: this.#resourceScope,
|
|
||||||
environment: this.environment,
|
environment: this.environment,
|
||||||
...(options?.signal === undefined ? {} : { signal: options.signal }),
|
...(options?.signal === undefined ? {} : { signal: options.signal }),
|
||||||
integrity: { mode: parsed.integrity_policy },
|
integrity: { mode: parsed.integrity_policy },
|
||||||
|
|||||||
@@ -31,37 +31,6 @@ export interface ArtifactEntry {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Snapshot mutable entry metadata before the producer later validates open(). */
|
|
||||||
export const copyArtifactEntry = (entry: ArtifactEntry): ArtifactEntry => ({
|
|
||||||
...entry,
|
|
||||||
limitations: [...entry.limitations],
|
|
||||||
...(entry.sourceIdentity === undefined
|
|
||||||
? {}
|
|
||||||
: { sourceIdentity: { ...entry.sourceIdentity } }),
|
|
||||||
});
|
|
||||||
|
|
||||||
/** Compare every portable and adapter-owned fact supplied with an entry. */
|
|
||||||
export const sameArtifactEntry = (
|
|
||||||
entry: ArtifactEntry,
|
|
||||||
produced: ArtifactEntry,
|
|
||||||
): boolean =>
|
|
||||||
entry.path === produced.path &&
|
|
||||||
entry.kind === produced.kind &&
|
|
||||||
entry.declaredSize === produced.declaredSize &&
|
|
||||||
entry.compressedSize === produced.compressedSize &&
|
|
||||||
entry.executable === produced.executable &&
|
|
||||||
entry.encrypted === produced.encrypted &&
|
|
||||||
entry.byteOffset === produced.byteOffset &&
|
|
||||||
entry.declaredSha256 === produced.declaredSha256 &&
|
|
||||||
entry.unpacked === produced.unpacked &&
|
|
||||||
entry.adapterKey === produced.adapterKey &&
|
|
||||||
entry.limitations.length === produced.limitations.length &&
|
|
||||||
entry.limitations.every(
|
|
||||||
(limitation, index) => limitation === produced.limitations[index],
|
|
||||||
) &&
|
|
||||||
entry.sourceIdentity?.device === produced.sourceIdentity?.device &&
|
|
||||||
entry.sourceIdentity?.inode === produced.sourceIdentity?.inode;
|
|
||||||
|
|
||||||
/** Read-only adapter over one directory, archive, or virtual container. */
|
/** Read-only adapter over one directory, archive, or virtual container. */
|
||||||
export interface ArtifactReader {
|
export interface ArtifactReader {
|
||||||
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
|
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
|
||||||
|
|||||||
@@ -1,114 +0,0 @@
|
|||||||
import { expect, it } from "vitest";
|
|
||||||
|
|
||||||
import { err } from "../domain/result.js";
|
|
||||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
|
||||||
import type { ArtifactReader } from "./ArtifactReader.js";
|
|
||||||
import { ArtifactResourceScope } from "./ArtifactResourceScope.js";
|
|
||||||
|
|
||||||
const deferred = <T>() => {
|
|
||||||
let resolve: (value: T) => void = () => {};
|
|
||||||
const promise = new Promise<T>((resolvePromise) => {
|
|
||||||
resolve = resolvePromise;
|
|
||||||
});
|
|
||||||
return { promise, resolve };
|
|
||||||
};
|
|
||||||
|
|
||||||
it("waits for admitted operations and shares concurrent close", async () => {
|
|
||||||
const scope = new ArtifactResourceScope();
|
|
||||||
const started = deferred<void>();
|
|
||||||
const finish = deferred<void>();
|
|
||||||
const operation = scope.run(async () => {
|
|
||||||
started.resolve();
|
|
||||||
await finish.promise;
|
|
||||||
});
|
|
||||||
await started.promise;
|
|
||||||
|
|
||||||
const close = scope.close();
|
|
||||||
expect(scope.close()).toBe(close);
|
|
||||||
await expect(scope.run(async () => undefined)).rejects.toMatchObject({
|
|
||||||
reason: "unavailable",
|
|
||||||
message: "Artifact resource scope is closed",
|
|
||||||
});
|
|
||||||
let closed = false;
|
|
||||||
void close.then(() => {
|
|
||||||
closed = true;
|
|
||||||
});
|
|
||||||
await Promise.resolve();
|
|
||||||
expect(closed).toBe(false);
|
|
||||||
|
|
||||||
finish.resolve();
|
|
||||||
await operation;
|
|
||||||
await close;
|
|
||||||
expect(closed).toBe(true);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("retains the same failed reader owner and retries before admitting work", async () => {
|
|
||||||
const scope = new ArtifactResourceScope();
|
|
||||||
let closeAttempts = 0;
|
|
||||||
const reader = {
|
|
||||||
async close() {
|
|
||||||
closeAttempts += 1;
|
|
||||||
if (closeAttempts < 4) throw new Error(`close attempt ${closeAttempts}`);
|
|
||||||
},
|
|
||||||
} as unknown as ArtifactReader;
|
|
||||||
const owner = { kind: "reader" as const, resource: "fixture reader", reader };
|
|
||||||
|
|
||||||
expect(await scope.release(owner)).toMatchObject({ kind: "failed" });
|
|
||||||
expect(closeAttempts).toBe(1);
|
|
||||||
|
|
||||||
await expect(scope.run(async () => "must not run")).rejects.toMatchObject({
|
|
||||||
reason: "unavailable",
|
|
||||||
message: "Artifact cleanup remains incomplete: fixture reader",
|
|
||||||
});
|
|
||||||
expect(closeAttempts).toBe(2);
|
|
||||||
await expect(scope.close()).rejects.toMatchObject({
|
|
||||||
reason: "unavailable",
|
|
||||||
});
|
|
||||||
expect(closeAttempts).toBe(3);
|
|
||||||
await scope.close();
|
|
||||||
expect(closeAttempts).toBe(4);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("retains a failed source analyzer until shutdown can verify its release", async () => {
|
|
||||||
const scope = new ArtifactResourceScope();
|
|
||||||
let releaseAllowed = false;
|
|
||||||
const failure = new AnalysisOutputError(
|
|
||||||
"analyze_javascript_application",
|
|
||||||
"Owned worker is still running",
|
|
||||||
);
|
|
||||||
const owner = {
|
|
||||||
kind: "javascript-source-analysis" as const,
|
|
||||||
resource: "fixture source analyzer",
|
|
||||||
analysis: {
|
|
||||||
async analyze() {
|
|
||||||
return err({
|
|
||||||
error: failure,
|
|
||||||
javascript: null,
|
|
||||||
module: null,
|
|
||||||
projection: null,
|
|
||||||
});
|
|
||||||
},
|
|
||||||
async close() {
|
|
||||||
if (!releaseAllowed) throw failure;
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
expect(await scope.release(owner)).toEqual({
|
|
||||||
kind: "failed",
|
|
||||||
cause: failure,
|
|
||||||
});
|
|
||||||
let admitted = false;
|
|
||||||
await expect(
|
|
||||||
scope.run(async () => {
|
|
||||||
admitted = true;
|
|
||||||
}),
|
|
||||||
).rejects.toMatchObject({
|
|
||||||
cleanup: { resources: ["fixture source analyzer"] },
|
|
||||||
});
|
|
||||||
expect(admitted).toBe(false);
|
|
||||||
await expect(scope.close()).rejects.toMatchObject({
|
|
||||||
cleanup: { resources: ["fixture source analyzer"] },
|
|
||||||
});
|
|
||||||
releaseAllowed = true;
|
|
||||||
await scope.close();
|
|
||||||
});
|
|
||||||
@@ -1,156 +0,0 @@
|
|||||||
import type { JavaScriptSourceAnalysisPort } from "../domain/javascript/javascriptSourceAnalysis.js";
|
|
||||||
import type { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
|
||||||
import type { OwnedDirectoryHandle } from "../filesystem/OwnedDirectoryHandle.js";
|
|
||||||
|
|
||||||
import type { ArtifactReader } from "./ArtifactReader.js";
|
|
||||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
|
||||||
import type { SafeOutputTree } from "./SafeOutputTree.js";
|
|
||||||
|
|
||||||
/** One artifact resource and its format-specific cleanup owner. */
|
|
||||||
export type ArtifactResourceOwner =
|
|
||||||
| {
|
|
||||||
readonly kind: "javascript-source-analysis";
|
|
||||||
readonly resource: string;
|
|
||||||
readonly analysis: JavaScriptSourceAnalysisPort;
|
|
||||||
}
|
|
||||||
| {
|
|
||||||
readonly kind: "directory-handle";
|
|
||||||
readonly resource: string;
|
|
||||||
readonly handle: OwnedDirectoryHandle;
|
|
||||||
}
|
|
||||||
| {
|
|
||||||
readonly kind: "reader";
|
|
||||||
readonly resource: string;
|
|
||||||
readonly reader: ArtifactReader;
|
|
||||||
}
|
|
||||||
| {
|
|
||||||
readonly kind: "file-handle";
|
|
||||||
readonly resource: string;
|
|
||||||
readonly handle: OwnedFileHandle;
|
|
||||||
}
|
|
||||||
| {
|
|
||||||
readonly kind: "output-tree";
|
|
||||||
readonly resource: string;
|
|
||||||
readonly tree: SafeOutputTree;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type ArtifactCleanupAttempt =
|
|
||||||
| { readonly kind: "released" }
|
|
||||||
| { readonly kind: "failed"; readonly cause: unknown };
|
|
||||||
|
|
||||||
/** Caller-owned scope that retains failed artifact cleanup for a later close. */
|
|
||||||
export class ArtifactResourceScope {
|
|
||||||
readonly #pending = new Set<ArtifactResourceOwner>();
|
|
||||||
readonly #active = new Set<Promise<unknown>>();
|
|
||||||
#draining: Promise<void> | undefined;
|
|
||||||
#closed = false;
|
|
||||||
|
|
||||||
/** Admit a resource operation while the scope remains open. */
|
|
||||||
async run<T>(operation: () => Promise<T>): Promise<T> {
|
|
||||||
while (true) {
|
|
||||||
if (this.#closed)
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"unavailable",
|
|
||||||
"Artifact resource scope is closed",
|
|
||||||
);
|
|
||||||
if (this.#draining !== undefined) await this.#draining;
|
|
||||||
if (this.#closed)
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"unavailable",
|
|
||||||
"Artifact resource scope is closed",
|
|
||||||
);
|
|
||||||
if (this.#pending.size > 0) {
|
|
||||||
await this.#retryCleanup();
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const running = Promise.resolve().then(operation);
|
|
||||||
this.#active.add(running);
|
|
||||||
try {
|
|
||||||
return await running;
|
|
||||||
} finally {
|
|
||||||
this.#active.delete(running);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Attempt cleanup now and retain the same owner if it fails. */
|
|
||||||
async release(owner: ArtifactResourceOwner): Promise<ArtifactCleanupAttempt> {
|
|
||||||
try {
|
|
||||||
await cleanup(owner);
|
|
||||||
this.#pending.delete(owner);
|
|
||||||
return { kind: "released" };
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
this.#pending.add(owner);
|
|
||||||
return { kind: "failed", cause };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Retry retained owners without closing resource admission. */
|
|
||||||
#retryCleanup(): Promise<void> {
|
|
||||||
if (this.#draining !== undefined) return this.#draining;
|
|
||||||
this.#draining = this.#drainAndRetry().finally(() => {
|
|
||||||
this.#draining = undefined;
|
|
||||||
});
|
|
||||||
return this.#draining;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Stop admission, wait for active operations, and retry every retained owner. */
|
|
||||||
close(): Promise<void> {
|
|
||||||
this.#closed = true;
|
|
||||||
return this.#retryCleanup();
|
|
||||||
}
|
|
||||||
|
|
||||||
async #drainAndRetry(): Promise<void> {
|
|
||||||
while (this.#active.size > 0) await Promise.allSettled(this.#active);
|
|
||||||
const failures: {
|
|
||||||
readonly owner: ArtifactResourceOwner;
|
|
||||||
readonly cause: unknown;
|
|
||||||
}[] = [];
|
|
||||||
for (const owner of this.#pending) {
|
|
||||||
try {
|
|
||||||
await cleanup(owner);
|
|
||||||
this.#pending.delete(owner);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
failures.push({ owner, cause });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (failures.length === 0) return;
|
|
||||||
const observations = failures.map(({ owner, cause }) =>
|
|
||||||
ArtifactReaderFailure.cleanupObservation(cause, owner.resource),
|
|
||||||
);
|
|
||||||
const resources = [
|
|
||||||
...new Set(observations.flatMap(({ resources }) => resources)),
|
|
||||||
];
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"unavailable",
|
|
||||||
`Artifact cleanup remains incomplete: ${resources.join(", ")}`,
|
|
||||||
{
|
|
||||||
cause: new AggregateError(
|
|
||||||
failures.map(({ cause }) => cause),
|
|
||||||
"One or more artifact resources remain owned",
|
|
||||||
),
|
|
||||||
cleanup: {
|
|
||||||
reason: observations.map(({ reason }) => reason).join("; "),
|
|
||||||
resources,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const cleanup = async (owner: ArtifactResourceOwner): Promise<void> => {
|
|
||||||
switch (owner.kind) {
|
|
||||||
case "javascript-source-analysis":
|
|
||||||
await owner.analysis.close();
|
|
||||||
return;
|
|
||||||
case "reader":
|
|
||||||
await owner.reader.close();
|
|
||||||
return;
|
|
||||||
case "file-handle":
|
|
||||||
case "directory-handle":
|
|
||||||
await owner.handle.close();
|
|
||||||
return;
|
|
||||||
case "output-tree":
|
|
||||||
await owner.tree.rollback();
|
|
||||||
}
|
|
||||||
};
|
|
||||||
@@ -19,19 +19,15 @@ import { isAbsolute, join, relative, sep } from "node:path";
|
|||||||
import { Readable, Transform } from "node:stream";
|
import { Readable, Transform } from "node:stream";
|
||||||
import { pipeline } from "node:stream/promises";
|
import { pipeline } from "node:stream/promises";
|
||||||
|
|
||||||
import { listPackage, statFile, uncache } from "@electron/asar";
|
import { getRawHeader, listPackage, statFile, uncache } from "@electron/asar";
|
||||||
|
|
||||||
import {
|
import {
|
||||||
ArtifactReaderFailure,
|
ArtifactReaderFailure,
|
||||||
copyArtifactEntry,
|
|
||||||
sameArtifactEntry,
|
|
||||||
type ArtifactEntry,
|
type ArtifactEntry,
|
||||||
type ArtifactReader,
|
type ArtifactReader,
|
||||||
} from "./ArtifactReader.js";
|
} from "./ArtifactReader.js";
|
||||||
import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js";
|
import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js";
|
||||||
import { admitAsarHeader } from "./AsarHeader.js";
|
import { admitAsarHeader } from "./AsarHeader.js";
|
||||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
|
||||||
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Official Electron ASAR adapter with range-streamed member reads.
|
* Official Electron ASAR adapter with range-streamed member reads.
|
||||||
@@ -45,13 +41,6 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
#archiveSize: number | undefined;
|
#archiveSize: number | undefined;
|
||||||
#headerSize: number | undefined;
|
#headerSize: number | undefined;
|
||||||
readonly #entries = new Map<string, AsarEntryState>();
|
readonly #entries = new Map<string, AsarEntryState>();
|
||||||
readonly #unpackedHandles = new Map<
|
|
||||||
UnpackedFileHandle,
|
|
||||||
{
|
|
||||||
readonly path: string;
|
|
||||||
readonly owner: OwnedFileHandle<UnpackedFileHandle>;
|
|
||||||
}
|
|
||||||
>();
|
|
||||||
#snapshot: { readonly path: string; readonly sha256: string } | undefined;
|
#snapshot: { readonly path: string; readonly sha256: string } | undefined;
|
||||||
#snapshotRoot: string | undefined;
|
#snapshotRoot: string | undefined;
|
||||||
|
|
||||||
@@ -76,7 +65,6 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
try {
|
try {
|
||||||
const admitted = await admitAsarHeader(headerHandle);
|
const admitted = await admitAsarHeader(headerHandle);
|
||||||
this.#archiveSize = admitted.archiveBytes;
|
this.#archiveSize = admitted.archiveBytes;
|
||||||
this.#headerSize = admitted.headerBytes;
|
|
||||||
} finally {
|
} finally {
|
||||||
await headerHandle.close();
|
await headerHandle.close();
|
||||||
}
|
}
|
||||||
@@ -84,6 +72,7 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
paths = listPackage(snapshotPath, { isPack: false }).sort((left, right) =>
|
paths = listPackage(snapshotPath, { isPack: false }).sort((left, right) =>
|
||||||
left.localeCompare(right, "en"),
|
left.localeCompare(right, "en"),
|
||||||
);
|
);
|
||||||
|
this.#headerSize = getRawHeader(snapshotPath).headerSize;
|
||||||
const archiveMetadata = await lstat(snapshotPath);
|
const archiveMetadata = await lstat(snapshotPath);
|
||||||
if (!archiveMetadata.isFile() || archiveMetadata.isSymbolicLink())
|
if (!archiveMetadata.isFile() || archiveMetadata.isSymbolicLink())
|
||||||
throw new ArtifactReaderFailure(
|
throw new ArtifactReaderFailure(
|
||||||
@@ -167,8 +156,32 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
);
|
);
|
||||||
const { metadata } = state;
|
const { metadata } = state;
|
||||||
const producedEntry = state.entry;
|
const producedEntry = state.entry;
|
||||||
if (producedEntry.unpacked)
|
if (producedEntry.unpacked) {
|
||||||
return this.#openUnpackedStream(producedEntry, signal);
|
const handle = await this.#openUnpackedEntry(producedEntry);
|
||||||
|
try {
|
||||||
|
const observed = await handle.stat();
|
||||||
|
abortIfNeeded(signal);
|
||||||
|
if (!observed.isFile())
|
||||||
|
throw new ArtifactReaderFailure(
|
||||||
|
"path",
|
||||||
|
`ASAR unpacked entry is not a regular file: ${producedEntry.path}`,
|
||||||
|
);
|
||||||
|
} catch (cause: unknown) {
|
||||||
|
await handle.close().catch(() => undefined);
|
||||||
|
throw asarFailure(this.path, `read ${producedEntry.path}`, cause);
|
||||||
|
}
|
||||||
|
const source = handle.createReadStream({
|
||||||
|
start: 0,
|
||||||
|
autoClose: true,
|
||||||
|
});
|
||||||
|
return readValidatedAsarEntry(
|
||||||
|
source,
|
||||||
|
undefined,
|
||||||
|
producedEntry.path,
|
||||||
|
this.path,
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
}
|
||||||
const archiveSize = this.#archiveSize;
|
const archiveSize = this.#archiveSize;
|
||||||
const headerSize = this.#headerSize;
|
const headerSize = this.#headerSize;
|
||||||
const offset =
|
const offset =
|
||||||
@@ -302,39 +315,22 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async close(): Promise<void> {
|
async close(): Promise<void> {
|
||||||
let cleanupFailure: ArtifactReaderFailure | undefined;
|
|
||||||
for (const [handle, { path }] of this.#unpackedHandles) {
|
|
||||||
try {
|
|
||||||
await this.#closeUnpackedHandle(handle);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
|
||||||
cleanupFailure ?? cause,
|
|
||||||
ArtifactReaderFailure.cleanupObservation(cause, path),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const root = this.#snapshotRoot;
|
const root = this.#snapshotRoot;
|
||||||
if (root !== undefined) {
|
if (root !== undefined) {
|
||||||
try {
|
try {
|
||||||
await this.removeSnapshot(root);
|
await this.removeSnapshot(root);
|
||||||
if (this.#snapshotRoot === root) this.#snapshotRoot = undefined;
|
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
throw ArtifactReaderFailure.withCleanup(cause, {
|
||||||
cleanupFailure ?? cause,
|
reason: `Could not remove ASAR snapshot: ${root}`,
|
||||||
{
|
resources: [root],
|
||||||
reason: `Could not remove ASAR snapshot: ${root}`,
|
});
|
||||||
resources: [root],
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
this.#snapshotRoot = undefined;
|
||||||
}
|
}
|
||||||
if (root === undefined || this.#snapshotRoot === undefined) {
|
if (this.#snapshot !== undefined) uncache(this.#snapshot.path);
|
||||||
if (this.#snapshot !== undefined) uncache(this.#snapshot.path);
|
this.#snapshot = undefined;
|
||||||
this.#snapshot = undefined;
|
this.#resetArchiveState();
|
||||||
this.#resetArchiveState();
|
this.#entries.clear();
|
||||||
this.#entries.clear();
|
|
||||||
}
|
|
||||||
if (cleanupFailure !== undefined) throw cleanupFailure;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#snapshotPath(): string {
|
#snapshotPath(): string {
|
||||||
@@ -350,12 +346,8 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
return [];
|
return [];
|
||||||
}
|
}
|
||||||
|
|
||||||
async #openUnpackedStream(
|
async #openUnpackedEntry(entry: ArtifactEntry): Promise<UnpackedFileHandle> {
|
||||||
entry: ArtifactEntry,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): Promise<Readable> {
|
|
||||||
const unpackedRoot = `${this.path}.unpacked`;
|
const unpackedRoot = `${this.path}.unpacked`;
|
||||||
let handle: UnpackedFileHandle | undefined;
|
|
||||||
try {
|
try {
|
||||||
const rootMetadata = await lstat(unpackedRoot);
|
const rootMetadata = await lstat(unpackedRoot);
|
||||||
if (!rootMetadata.isDirectory() || rootMetadata.isSymbolicLink())
|
if (!rootMetadata.isDirectory() || rootMetadata.isSymbolicLink())
|
||||||
@@ -382,21 +374,22 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
"path",
|
"path",
|
||||||
`ASAR unpacked entry is not a regular file: ${entry.path}`,
|
`ASAR unpacked entry is not a regular file: ${entry.path}`,
|
||||||
);
|
);
|
||||||
handle = await this.openUnpackedFile(
|
const handle = await this.openUnpackedFile(
|
||||||
canonical,
|
canonical,
|
||||||
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
||||||
);
|
);
|
||||||
const openedHandle = handle;
|
const openedMetadata = await handle
|
||||||
this.#unpackedHandles.set(openedHandle, {
|
.stat()
|
||||||
path: canonical,
|
.catch(async (cause: unknown) => {
|
||||||
owner: new OwnedFileHandle(openedHandle),
|
await handle.close().catch(() => undefined);
|
||||||
});
|
throw cause;
|
||||||
const openedMetadata = await openedHandle.stat();
|
});
|
||||||
if (
|
if (
|
||||||
!openedMetadata.isFile() ||
|
!openedMetadata.isFile() ||
|
||||||
openedMetadata.dev !== pathMetadata.dev ||
|
openedMetadata.dev !== pathMetadata.dev ||
|
||||||
openedMetadata.ino !== pathMetadata.ino
|
openedMetadata.ino !== pathMetadata.ino
|
||||||
) {
|
) {
|
||||||
|
await handle.close();
|
||||||
throw new ArtifactReaderFailure(
|
throw new ArtifactReaderFailure(
|
||||||
"integrity",
|
"integrity",
|
||||||
`ASAR unpacked entry changed before read: ${entry.path}`,
|
`ASAR unpacked entry changed before read: ${entry.path}`,
|
||||||
@@ -409,46 +402,25 @@ export class AsarArtifactReader implements ArtifactReader {
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
abortIfNeeded(signal);
|
return handle;
|
||||||
const source = readFileHandleChunks(openedHandle, {
|
|
||||||
start: 0,
|
|
||||||
});
|
|
||||||
source.once("close", () => {
|
|
||||||
void this.#closeUnpackedHandle(openedHandle).catch(() => undefined);
|
|
||||||
});
|
|
||||||
return readValidatedAsarEntry(
|
|
||||||
source,
|
|
||||||
undefined,
|
|
||||||
entry.path,
|
|
||||||
this.path,
|
|
||||||
signal,
|
|
||||||
);
|
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
let failure = asarUnpackedEntryFailure(this.path, entry, cause);
|
if (cause instanceof ArtifactReaderFailure) throw cause;
|
||||||
if (handle !== undefined && this.#unpackedHandles.has(handle)) {
|
if (entry.unpacked && isMissingFile(cause))
|
||||||
try {
|
throw new ArtifactReaderFailure(
|
||||||
await this.#closeUnpackedHandle(handle);
|
"unavailable",
|
||||||
} catch (cleanupCause: unknown) {
|
`ASAR unpacked entry bytes are unavailable: ${entry.path}`,
|
||||||
failure = ArtifactReaderFailure.withCleanup(
|
{ cause },
|
||||||
failure,
|
{
|
||||||
ArtifactReaderFailure.cleanupObservation(
|
logicalPath: entry.path,
|
||||||
cleanupCause,
|
declaredSha256: entry.declaredSha256,
|
||||||
this.#unpackedHandles.get(handle)?.path ?? entry.path,
|
calculatedSha256: null,
|
||||||
),
|
unpacked: true,
|
||||||
);
|
},
|
||||||
}
|
);
|
||||||
}
|
throw asarFailure(this.path, `read ${entry.path}`, cause);
|
||||||
throw failure;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async #closeUnpackedHandle(handle: UnpackedFileHandle): Promise<void> {
|
|
||||||
const owned = this.#unpackedHandles.get(handle);
|
|
||||||
if (owned === undefined) return;
|
|
||||||
await owned.owner.close();
|
|
||||||
this.#unpackedHandles.delete(handle);
|
|
||||||
}
|
|
||||||
|
|
||||||
#resetArchiveState(): void {
|
#resetArchiveState(): void {
|
||||||
this.#archiveSize = undefined;
|
this.#archiveSize = undefined;
|
||||||
this.#headerSize = undefined;
|
this.#headerSize = undefined;
|
||||||
@@ -461,36 +433,43 @@ type AsarEntryState = {
|
|||||||
readonly entry: ArtifactEntry;
|
readonly entry: ArtifactEntry;
|
||||||
};
|
};
|
||||||
type UnpackedFileHandle = {
|
type UnpackedFileHandle = {
|
||||||
readonly fd: number;
|
|
||||||
stat(): Promise<Stats>;
|
stat(): Promise<Stats>;
|
||||||
close(): Promise<void>;
|
close(): Promise<void>;
|
||||||
read: FileHandle["read"];
|
createReadStream: FileHandle["createReadStream"];
|
||||||
};
|
};
|
||||||
type OpenAsarUnpackedFile = (
|
type OpenAsarUnpackedFile = (
|
||||||
path: string,
|
path: string,
|
||||||
flags: number,
|
flags: number,
|
||||||
) => Promise<UnpackedFileHandle>;
|
) => Promise<UnpackedFileHandle>;
|
||||||
|
|
||||||
const asarUnpackedEntryFailure = (
|
const copyArtifactEntry = (entry: ArtifactEntry): ArtifactEntry => ({
|
||||||
archivePath: string,
|
...entry,
|
||||||
|
limitations: [...entry.limitations],
|
||||||
|
...(entry.sourceIdentity === undefined
|
||||||
|
? {}
|
||||||
|
: { sourceIdentity: { ...entry.sourceIdentity } }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const sameArtifactEntry = (
|
||||||
entry: ArtifactEntry,
|
entry: ArtifactEntry,
|
||||||
cause: unknown,
|
produced: ArtifactEntry,
|
||||||
): ArtifactReaderFailure => {
|
): boolean =>
|
||||||
if (cause instanceof ArtifactReaderFailure) return cause;
|
entry.path === produced.path &&
|
||||||
if (entry.unpacked && isMissingFile(cause))
|
entry.kind === produced.kind &&
|
||||||
return new ArtifactReaderFailure(
|
entry.declaredSize === produced.declaredSize &&
|
||||||
"unavailable",
|
entry.compressedSize === produced.compressedSize &&
|
||||||
`ASAR unpacked entry bytes are unavailable: ${entry.path}`,
|
entry.executable === produced.executable &&
|
||||||
{ cause },
|
entry.encrypted === produced.encrypted &&
|
||||||
{
|
entry.byteOffset === produced.byteOffset &&
|
||||||
logicalPath: entry.path,
|
entry.declaredSha256 === produced.declaredSha256 &&
|
||||||
declaredSha256: entry.declaredSha256,
|
entry.unpacked === produced.unpacked &&
|
||||||
calculatedSha256: null,
|
entry.adapterKey === produced.adapterKey &&
|
||||||
unpacked: true,
|
entry.limitations.length === produced.limitations.length &&
|
||||||
},
|
entry.limitations.every(
|
||||||
);
|
(limitation, index) => limitation === produced.limitations[index],
|
||||||
return asarFailure(archivePath, `read ${entry.path}`, cause);
|
) &&
|
||||||
};
|
entry.sourceIdentity?.device === produced.sourceIdentity?.device &&
|
||||||
|
entry.sourceIdentity?.inode === produced.sourceIdentity?.inode;
|
||||||
|
|
||||||
const isAsarFileMetadata = (
|
const isAsarFileMetadata = (
|
||||||
metadata: ReturnType<typeof statFile>,
|
metadata: ReturnType<typeof statFile>,
|
||||||
|
|||||||
@@ -1,33 +1,23 @@
|
|||||||
import { constants, type Stats } from "node:fs";
|
import { constants } from "node:fs";
|
||||||
import {
|
import { lstat, open, opendir, realpath } from "node:fs/promises";
|
||||||
lstat,
|
|
||||||
open,
|
|
||||||
opendir,
|
|
||||||
realpath,
|
|
||||||
type FileHandle,
|
|
||||||
} from "node:fs/promises";
|
|
||||||
import { isAbsolute, join, relative, sep } from "node:path";
|
import { isAbsolute, join, relative, sep } from "node:path";
|
||||||
import { Readable } from "node:stream";
|
import type { Readable } from "node:stream";
|
||||||
|
|
||||||
import {
|
import {
|
||||||
ArtifactReaderFailure,
|
ArtifactReaderFailure,
|
||||||
type ArtifactEntry,
|
type ArtifactEntry,
|
||||||
type ArtifactReader,
|
type ArtifactReader,
|
||||||
} from "./ArtifactReader.js";
|
} from "./ArtifactReader.js";
|
||||||
import { OwnedDirectoryHandle } from "../filesystem/OwnedDirectoryHandle.js";
|
|
||||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
|
||||||
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
|
|
||||||
|
|
||||||
/** Non-symlink-following directory reader. */
|
/** Non-symlink-following directory reader. */
|
||||||
export class DirectoryArtifactReader implements ArtifactReader {
|
export class DirectoryArtifactReader implements ArtifactReader {
|
||||||
readonly format = "directory" as const;
|
readonly format = "directory" as const;
|
||||||
#rootPromise: Promise<string> | undefined;
|
readonly #rootPromise: Promise<string>;
|
||||||
readonly #rootPath: string;
|
readonly #rootPath: string;
|
||||||
readonly #directories = new Set<OwnedDirectoryHandle>();
|
|
||||||
readonly #files = new Map<OwnedFileHandle, string>();
|
|
||||||
|
|
||||||
constructor(root: string) {
|
constructor(root: string) {
|
||||||
this.#rootPath = root;
|
this.#rootPath = root;
|
||||||
|
this.#rootPromise = realpath(root);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Enumerate entries, optionally restricting descent while retaining directories. */
|
/** Enumerate entries, optionally restricting descent while retaining directories. */
|
||||||
@@ -35,7 +25,9 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
|||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
descendInto?: (path: string) => boolean,
|
descendInto?: (path: string) => boolean,
|
||||||
): AsyncIterable<ArtifactEntry> {
|
): AsyncIterable<ArtifactEntry> {
|
||||||
const root = await this.#resolveRoot();
|
const root = await this.#rootPromise.catch((cause: unknown) => {
|
||||||
|
throw directoryIoFailure("resolve artifact root", this.#rootPath, cause);
|
||||||
|
});
|
||||||
const pending = [root];
|
const pending = [root];
|
||||||
while (pending.length > 0) {
|
while (pending.length > 0) {
|
||||||
abortIfNeeded(signal);
|
abortIfNeeded(signal);
|
||||||
@@ -44,100 +36,62 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
|||||||
await withDirectoryIoContext("inspect directory", directory, () =>
|
await withDirectoryIoContext("inspect directory", directory, () =>
|
||||||
assertContainedDirectory(root, directory),
|
assertContainedDirectory(root, directory),
|
||||||
);
|
);
|
||||||
for await (const entry of this.#readDirectory(
|
const handle = await opendir(directory).catch((cause: unknown) => {
|
||||||
directory,
|
throw directoryIoFailure("open directory", directory, cause);
|
||||||
pending,
|
});
|
||||||
signal,
|
const directories: string[] = [];
|
||||||
descendInto,
|
try {
|
||||||
))
|
for await (const child of handle) {
|
||||||
yield entry;
|
abortIfNeeded(signal);
|
||||||
|
const absolute = join(directory, child.name);
|
||||||
|
const metadata = await lstat(absolute).catch((cause: unknown) => {
|
||||||
|
throw directoryIoFailure("inspect entry", absolute, cause);
|
||||||
|
});
|
||||||
|
const path = relative(root, absolute).split(sep).join("/");
|
||||||
|
const kind = metadata.isSymbolicLink()
|
||||||
|
? "symlink"
|
||||||
|
: metadata.isDirectory()
|
||||||
|
? "directory"
|
||||||
|
: metadata.isFile()
|
||||||
|
? "file"
|
||||||
|
: undefined;
|
||||||
|
if (kind === undefined) continue;
|
||||||
|
yield {
|
||||||
|
path,
|
||||||
|
kind,
|
||||||
|
declaredSize: kind === "file" ? metadata.size : null,
|
||||||
|
compressedSize: null,
|
||||||
|
executable: (metadata.mode & 0o111) !== 0,
|
||||||
|
encrypted: false,
|
||||||
|
byteOffset: null,
|
||||||
|
declaredSha256: null,
|
||||||
|
unpacked: false,
|
||||||
|
limitations:
|
||||||
|
kind === "symlink"
|
||||||
|
? ["Symlink target was not followed or disclosed."]
|
||||||
|
: [],
|
||||||
|
adapterKey: absolute,
|
||||||
|
...(kind === "file"
|
||||||
|
? {
|
||||||
|
sourceIdentity: { device: metadata.dev, inode: metadata.ino },
|
||||||
|
}
|
||||||
|
: {}),
|
||||||
|
};
|
||||||
|
if (kind === "directory" && (descendInto?.(path) ?? true))
|
||||||
|
directories.push(absolute);
|
||||||
|
}
|
||||||
|
} catch (cause: unknown) {
|
||||||
|
if (cause instanceof ArtifactReaderFailure) throw cause;
|
||||||
|
throw directoryIoFailure("read directory", directory, cause);
|
||||||
|
} finally {
|
||||||
|
// best-effort cleanup: directory-handle close must not mask traversal.
|
||||||
|
await handle.close().catch(() => undefined);
|
||||||
|
}
|
||||||
await withDirectoryIoContext("inspect directory", directory, () =>
|
await withDirectoryIoContext("inspect directory", directory, () =>
|
||||||
assertContainedDirectory(root, directory),
|
assertContainedDirectory(root, directory),
|
||||||
);
|
);
|
||||||
}
|
directories.reverse();
|
||||||
}
|
pending.push(...directories);
|
||||||
|
|
||||||
async #resolveRoot(): Promise<string> {
|
|
||||||
return (this.#rootPromise ??= realpath(this.#rootPath)).catch(
|
|
||||||
(cause: unknown) => {
|
|
||||||
throw directoryIoFailure(
|
|
||||||
"resolve artifact root",
|
|
||||||
this.#rootPath,
|
|
||||||
cause,
|
|
||||||
);
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
async *#readDirectory(
|
|
||||||
directory: string,
|
|
||||||
pending: string[],
|
|
||||||
signal?: AbortSignal,
|
|
||||||
descendInto?: (path: string) => boolean,
|
|
||||||
): AsyncIterable<ArtifactEntry> {
|
|
||||||
const root = await this.#resolveRoot();
|
|
||||||
const handle = new OwnedDirectoryHandle(
|
|
||||||
await opendir(directory).catch((cause: unknown) => {
|
|
||||||
throw directoryIoFailure("open directory", directory, cause);
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
this.#directories.add(handle);
|
|
||||||
const childDirectories: string[] = [];
|
|
||||||
let traversal:
|
|
||||||
| { readonly kind: "complete" }
|
|
||||||
| { readonly kind: "failed"; readonly cause: unknown } = {
|
|
||||||
kind: "complete",
|
|
||||||
};
|
|
||||||
try {
|
|
||||||
for (;;) {
|
|
||||||
const child = await handle.handle.read();
|
|
||||||
if (child === null) break;
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
const absolute = join(directory, child.name);
|
|
||||||
const metadata = await lstat(absolute).catch((cause: unknown) => {
|
|
||||||
throw directoryIoFailure("inspect entry", absolute, cause);
|
|
||||||
});
|
|
||||||
const path = relative(root, absolute).split(sep).join("/");
|
|
||||||
const entry = artifactEntry(path, absolute, metadata);
|
|
||||||
if (entry === undefined) continue;
|
|
||||||
yield entry;
|
|
||||||
if (entry.kind === "directory" && (descendInto?.(path) ?? true))
|
|
||||||
childDirectories.push(absolute);
|
|
||||||
}
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
traversal = {
|
|
||||||
kind: "failed",
|
|
||||||
cause:
|
|
||||||
cause instanceof ArtifactReaderFailure
|
|
||||||
? cause
|
|
||||||
: directoryIoFailure("read directory", directory, cause),
|
|
||||||
};
|
|
||||||
} finally {
|
|
||||||
await this.#finishDirectory(handle, directory, traversal);
|
|
||||||
}
|
|
||||||
if (traversal.kind === "failed") throw traversal.cause;
|
|
||||||
childDirectories.reverse();
|
|
||||||
pending.push(...childDirectories);
|
|
||||||
}
|
|
||||||
|
|
||||||
async #finishDirectory(
|
|
||||||
handle: OwnedDirectoryHandle,
|
|
||||||
path: string,
|
|
||||||
traversal:
|
|
||||||
| { readonly kind: "complete" }
|
|
||||||
| { readonly kind: "failed"; readonly cause: unknown },
|
|
||||||
): Promise<void> {
|
|
||||||
try {
|
|
||||||
await handle.close();
|
|
||||||
this.#directories.delete(handle);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
const closeFailure = directoryIoFailure("close directory", path, cause);
|
|
||||||
if (traversal.kind === "failed")
|
|
||||||
throw ArtifactReaderFailure.withCleanup(
|
|
||||||
traversal.cause,
|
|
||||||
ArtifactReaderFailure.cleanupObservation(closeFailure, path),
|
|
||||||
);
|
|
||||||
throw closeFailure;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -148,16 +102,14 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
|||||||
"format",
|
"format",
|
||||||
"Only regular files can be opened",
|
"Only regular files can be opened",
|
||||||
);
|
);
|
||||||
const fileHandle = await open(
|
const handle = await open(
|
||||||
entry.adapterKey,
|
entry.adapterKey,
|
||||||
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
|
||||||
).catch((cause: unknown) => {
|
).catch((cause: unknown) => {
|
||||||
throw directoryIoFailure("open entry", entry.adapterKey, cause);
|
throw directoryIoFailure("open entry", entry.adapterKey, cause);
|
||||||
});
|
});
|
||||||
const owner = new OwnedFileHandle(fileHandle);
|
|
||||||
this.#files.set(owner, entry.adapterKey);
|
|
||||||
try {
|
try {
|
||||||
const observed = await owner.handle.stat().catch((cause: unknown) => {
|
const observed = await handle.stat().catch((cause: unknown) => {
|
||||||
throw directoryIoFailure(
|
throw directoryIoFailure(
|
||||||
"inspect opened entry",
|
"inspect opened entry",
|
||||||
entry.adapterKey,
|
entry.adapterKey,
|
||||||
@@ -175,76 +127,16 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
|||||||
`Directory entry changed before open: ${entry.path}`,
|
`Directory entry changed before open: ${entry.path}`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
abortIfNeeded(signal);
|
return handle.createReadStream({ autoClose: true });
|
||||||
const source = Readable.from(
|
|
||||||
mapFileReadCancellation(owner.handle, signal),
|
|
||||||
{ objectMode: false },
|
|
||||||
);
|
|
||||||
source.once("close", () => {
|
|
||||||
void this.#closeFile(owner).catch(() => undefined);
|
|
||||||
});
|
|
||||||
return source;
|
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
try {
|
// best-effort cleanup: file-handle close must not mask the open failure.
|
||||||
await this.#closeFile(owner);
|
await handle.close().catch(() => undefined);
|
||||||
} catch (closeCause: unknown) {
|
|
||||||
throw ArtifactReaderFailure.withCleanup(
|
|
||||||
cause,
|
|
||||||
ArtifactReaderFailure.cleanupObservation(
|
|
||||||
directoryIoFailure(
|
|
||||||
"close opened entry",
|
|
||||||
entry.adapterKey,
|
|
||||||
closeCause,
|
|
||||||
),
|
|
||||||
entry.adapterKey,
|
|
||||||
),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
throw cause;
|
throw cause;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async close(): Promise<void> {
|
close(): Promise<void> {
|
||||||
const failures: unknown[] = [];
|
return Promise.resolve();
|
||||||
for (const [owner, path] of this.#files) {
|
|
||||||
try {
|
|
||||||
await this.#closeFile(owner);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
failures.push(directoryIoFailure("close opened entry", path, cause));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for (const owner of this.#directories) {
|
|
||||||
try {
|
|
||||||
await owner.close();
|
|
||||||
this.#directories.delete(owner);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
failures.push(
|
|
||||||
directoryIoFailure("close directory", owner.handle.path, cause),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (failures.length > 0) {
|
|
||||||
const reason = failures.map(failureMessage).join("; ");
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"io",
|
|
||||||
`Directory artifact cleanup failed: ${reason}`,
|
|
||||||
{
|
|
||||||
cause: new AggregateError(failures),
|
|
||||||
cleanup: {
|
|
||||||
reason,
|
|
||||||
resources: [
|
|
||||||
...this.#files.values(),
|
|
||||||
...[...this.#directories].map((owner) => owner.handle.path),
|
|
||||||
],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async #closeFile(owner: OwnedFileHandle<FileHandle>): Promise<void> {
|
|
||||||
await owner.close();
|
|
||||||
this.#files.delete(owner);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
provenance(): readonly [] {
|
provenance(): readonly [] {
|
||||||
@@ -252,42 +144,6 @@ export class DirectoryArtifactReader implements ArtifactReader {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const artifactEntry = (
|
|
||||||
path: string,
|
|
||||||
absolute: string,
|
|
||||||
metadata: Stats,
|
|
||||||
): ArtifactEntry | undefined => {
|
|
||||||
const kind = metadata.isSymbolicLink()
|
|
||||||
? "symlink"
|
|
||||||
: metadata.isDirectory()
|
|
||||||
? "directory"
|
|
||||||
: metadata.isFile()
|
|
||||||
? "file"
|
|
||||||
: undefined;
|
|
||||||
if (kind === undefined) return undefined;
|
|
||||||
return {
|
|
||||||
path,
|
|
||||||
kind,
|
|
||||||
declaredSize: kind === "file" ? metadata.size : null,
|
|
||||||
compressedSize: null,
|
|
||||||
executable: (metadata.mode & 0o111) !== 0,
|
|
||||||
encrypted: false,
|
|
||||||
byteOffset: null,
|
|
||||||
declaredSha256: null,
|
|
||||||
unpacked: false,
|
|
||||||
limitations:
|
|
||||||
kind === "symlink"
|
|
||||||
? ["Symlink target was not followed or disclosed."]
|
|
||||||
: [],
|
|
||||||
adapterKey: absolute,
|
|
||||||
...(kind === "file"
|
|
||||||
? {
|
|
||||||
sourceIdentity: { device: metadata.dev, inode: metadata.ino },
|
|
||||||
}
|
|
||||||
: {}),
|
|
||||||
};
|
|
||||||
};
|
|
||||||
|
|
||||||
const directoryIoFailure = (
|
const directoryIoFailure = (
|
||||||
operation: string,
|
operation: string,
|
||||||
path: string,
|
path: string,
|
||||||
@@ -301,38 +157,13 @@ const directoryIoFailure = (
|
|||||||
typeof cause.code === "string"
|
typeof cause.code === "string"
|
||||||
? ` (${cause.code})`
|
? ` (${cause.code})`
|
||||||
: "";
|
: "";
|
||||||
const detail = failureMessage(cause);
|
|
||||||
return new ArtifactReaderFailure(
|
return new ArtifactReaderFailure(
|
||||||
"io",
|
"io",
|
||||||
`Could not ${operation} at ${path}${code}: ${detail}`,
|
`Could not ${operation} at ${path}${code}`,
|
||||||
{ cause },
|
{ cause },
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|
||||||
const failureMessage = (cause: unknown): string =>
|
|
||||||
cause instanceof Error ? cause.message : String(cause);
|
|
||||||
|
|
||||||
const mapFileReadCancellation = async function* (
|
|
||||||
handle: FileHandle,
|
|
||||||
signal?: AbortSignal,
|
|
||||||
): AsyncGenerator<Buffer> {
|
|
||||||
try {
|
|
||||||
for await (const chunk of readFileHandleChunks(handle, { signal })) {
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
yield chunk;
|
|
||||||
}
|
|
||||||
abortIfNeeded(signal);
|
|
||||||
} catch (cause: unknown) {
|
|
||||||
if (signal?.aborted === true)
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"cancelled",
|
|
||||||
"Directory artifact read cancelled",
|
|
||||||
{ cause },
|
|
||||||
);
|
|
||||||
throw cause;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const withDirectoryIoContext = async <T>(
|
const withDirectoryIoContext = async <T>(
|
||||||
operation: string,
|
operation: string,
|
||||||
path: string,
|
path: string,
|
||||||
|
|||||||
@@ -1,84 +1,41 @@
|
|||||||
import { writeFile } from "node:fs/promises";
|
import { readFile } from "node:fs/promises";
|
||||||
import { join } from "node:path";
|
|
||||||
|
|
||||||
import { describe, expect, it, onTestFinished } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
|
|
||||||
import { CPU, fatImage, machoImage } from "./apple/MachoImage.fixture.js";
|
|
||||||
import { MachOSliceArtifactReader } from "./MachOSliceArtifactReader.js";
|
import { MachOSliceArtifactReader } from "./MachOSliceArtifactReader.js";
|
||||||
import { ok } from "../domain/result.js";
|
|
||||||
import { openRegularFile } from "../filesystem/RegularFile.js";
|
|
||||||
import type { NativeCommandRunner } from "../native/CommandRunner.js";
|
|
||||||
|
|
||||||
describe("MachOSliceArtifactReader", () => {
|
/**
|
||||||
it("keeps a borrowed source usable after a cancelled slice read", async () => {
|
* Read a 32-bit FAT table directly. The host's /bin/ls changes between macOS
|
||||||
const root = await createTestTempDirectory("rea-slices-cancelled-");
|
* releases, so the expected slice layout comes from the file itself.
|
||||||
const binary = join(root, "fat");
|
*/
|
||||||
const x86 = machoImage({ cpu: CPU.x86_64 });
|
const fatSlices = async (
|
||||||
const arm = Buffer.concat([
|
path: string,
|
||||||
Buffer.from(machoImage({ cpu: CPU.arm64 })),
|
): Promise<readonly (readonly [number, number])[]> => {
|
||||||
Buffer.alloc(160_000, 0x5a),
|
const bytes = await readFile(path);
|
||||||
]);
|
expect(bytes.readUInt32BE(0)).toBe(0xca_fe_ba_be);
|
||||||
await writeFile(
|
return Array.from({ length: bytes.readUInt32BE(4) }, (_, index) => {
|
||||||
binary,
|
const entry = 8 + index * 20;
|
||||||
fatImage([
|
return [bytes.readUInt32BE(entry + 8), bytes.readUInt32BE(entry + 12)];
|
||||||
{ cpu: CPU.x86_64, bytes: x86 },
|
|
||||||
{ cpu: CPU.arm64, bytes: arm },
|
|
||||||
]),
|
|
||||||
);
|
|
||||||
const handle = await openRegularFile(binary, { symlinks: "reject" });
|
|
||||||
const source = { handle, initial: await handle.stat() };
|
|
||||||
const runner: NativeCommandRunner = {
|
|
||||||
run: () =>
|
|
||||||
Promise.resolve(
|
|
||||||
ok({
|
|
||||||
tool: "lipo",
|
|
||||||
executable: "/usr/bin/lipo",
|
|
||||||
executableSha256: "1".repeat(64),
|
|
||||||
toolVersion: null,
|
|
||||||
versionReason: "fixture",
|
|
||||||
arguments: ["-detailed_info", binary],
|
|
||||||
stdout: `architecture x86_64\n cputype 16777223\n cpusubtype 3\n offset 4096\n size ${String(x86.length)}\n align 2^12 (4096)\narchitecture arm64\n cputype 16777228\n cpusubtype 0\n offset 8192\n size ${String(arm.length)}\n align 2^12 (4096)\n`,
|
|
||||||
stderr: "",
|
|
||||||
stdoutBytes: 1,
|
|
||||||
stderrBytes: 0,
|
|
||||||
exitCode: 0,
|
|
||||||
signal: null,
|
|
||||||
}),
|
|
||||||
),
|
|
||||||
};
|
|
||||||
const reader = new MachOSliceArtifactReader(binary, {}, runner, source);
|
|
||||||
onTestFinished(async () => {
|
|
||||||
await reader.close();
|
|
||||||
await handle.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
const entries = [];
|
|
||||||
for await (const entry of reader.entries()) entries.push(entry);
|
|
||||||
const armEntry = entries.find((entry) => entry.path === "slices/arm64");
|
|
||||||
if (armEntry === undefined) throw new Error("Expected an arm64 slice");
|
|
||||||
|
|
||||||
const controller = new AbortController();
|
|
||||||
const cancelledRead = await reader.open(armEntry, controller.signal);
|
|
||||||
const iterator = cancelledRead[Symbol.asyncIterator]();
|
|
||||||
const firstChunk = await iterator.next();
|
|
||||||
expect(firstChunk.value).toHaveLength(64 * 1024);
|
|
||||||
controller.abort();
|
|
||||||
await expect(iterator.next()).rejects.toMatchObject({
|
|
||||||
reason: "cancelled",
|
|
||||||
});
|
|
||||||
|
|
||||||
const verifiedEntries = [];
|
|
||||||
for await (const entry of reader.entries()) verifiedEntries.push(entry);
|
|
||||||
const verifiedArmEntry = verifiedEntries.find(
|
|
||||||
(entry) => entry.path === "slices/arm64",
|
|
||||||
);
|
|
||||||
if (verifiedArmEntry === undefined)
|
|
||||||
throw new Error("Expected arm64 after re-verifying the borrowed source");
|
|
||||||
const verifiedChunks: Buffer[] = [];
|
|
||||||
const verifiedRead = await reader.open(verifiedArmEntry);
|
|
||||||
for await (const chunk of verifiedRead)
|
|
||||||
verifiedChunks.push(Buffer.from(chunk));
|
|
||||||
expect(Buffer.concat(verifiedChunks)).toEqual(arm);
|
|
||||||
});
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
describe("Mach-O slice reader with system lipo", () => {
|
||||||
|
it.skipIf(process.platform !== "darwin")(
|
||||||
|
"checks real /bin/ls lipo CPU symbols and ptrauth capabilities against the FAT table",
|
||||||
|
async () => {
|
||||||
|
const reader = new MachOSliceArtifactReader("/bin/ls", {});
|
||||||
|
const entries = [];
|
||||||
|
for await (const entry of reader.entries()) entries.push(entry);
|
||||||
|
expect(entries.map(({ path }) => path)).toEqual([
|
||||||
|
"slices/x86_64",
|
||||||
|
"slices/arm64e",
|
||||||
|
]);
|
||||||
|
expect(
|
||||||
|
entries.map(({ byteOffset, declaredSize }) => [
|
||||||
|
byteOffset,
|
||||||
|
declaredSize,
|
||||||
|
]),
|
||||||
|
).toEqual(await fatSlices("/bin/ls"));
|
||||||
|
},
|
||||||
|
);
|
||||||
});
|
});
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user