docs: organize canonical documentation layout (URGENTE)
Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering. Co-Authored-By: Codex <noreply@openai.com>
This commit is contained in:
@@ -0,0 +1,128 @@
|
||||
# Electron file-page observation
|
||||
|
||||
REA can attach to an Electron/Chromium CDP endpoint and inspect existing `file://` renderer pages without evaluating JavaScript or invoking Electron APIs. The request supplies the endpoint directly; REA accepts only literal-port loopback HTTP endpoints. Selecting an endpoint exposes every Electron target it serves, including local file paths and page metadata.
|
||||
|
||||
This passive runtime surface is distinct from the target-free static
|
||||
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
|
||||
workflow. Static analysis reads a local ASAR or extracted directory without
|
||||
executing application code. Passive observation inspects an existing Electron
|
||||
page. The results keep static inferences and runtime observations distinct.
|
||||
Use the separate
|
||||
[`reconcile_javascript_runtime`](javascript-runtime-reconciliation.md) workflow
|
||||
when both Evidence sets already exist.
|
||||
|
||||
## Target boundary
|
||||
|
||||
The request supplies a literal-port loopback endpoint and, for inspection, a
|
||||
target ID. REA accepts local hostless `file://` URLs that resolve to regular
|
||||
files, including pages a packaged application serves from inside `app.asar`.
|
||||
For those, the first `.asar` path component that is a regular file is the
|
||||
archive, the remainder must name a regular file in its header, and the reported
|
||||
path is the canonical archive path joined with that member. Remote hosts,
|
||||
encoded path separators, ASAR directories or links, and nonexistent paths are
|
||||
rejected. The endpoint exposes every eligible target and its local path
|
||||
metadata.
|
||||
|
||||
## Workflow
|
||||
|
||||
`list_electron_targets` returns every eligible target served by the supplied endpoint in one inline array.
|
||||
|
||||
For the MCP follow-up call, pass the same literal-loopback endpoint and the selected target ID to `inspect_electron_page`. REA rediscovers targets and validates the ID against the live endpoint before inspection.
|
||||
|
||||
```bash
|
||||
rea list-electron-targets http://127.0.0.1:9223 --json
|
||||
rea inspect-electron-page http://127.0.0.1:9223 TARGET_ID \
|
||||
--observation-ms 100 --json
|
||||
```
|
||||
|
||||
Script content is excluded by default. Setting `include_script_sources` in the
|
||||
request returns the selected target's script sources:
|
||||
|
||||
```bash
|
||||
rea inspect-electron-page http://127.0.0.1:9223 TARGET_ID \
|
||||
--include-script-sources \
|
||||
--json
|
||||
```
|
||||
|
||||
The normalized result contains canonical local paths, frame and DOM
|
||||
structure, resource metadata, stable script/resource identities, explicit
|
||||
completeness, and content-addressed script artifacts. Script metadata
|
||||
retains its execution-context frame ID when CDP supplies one. The capture also
|
||||
inventories worker, service-worker, and shared-worker targets with
|
||||
validated opener-target and parent-frame IDs. Worker discovery uses passive
|
||||
target metadata; REA does not attach to or execute code in those targets.
|
||||
Collection counts and aggregate script-source bytes are not capped. Like every
|
||||
target, frame, script, and resource, a worker URL must resolve to a local
|
||||
file before it is retained. Relationship IDs improve
|
||||
attribution but do not prove which static module started a worker or that its
|
||||
work completed.
|
||||
|
||||
Inspection does not retain DOM values, execute renderer code, navigate, click,
|
||||
invoke Electron IPC, close a target, or terminate the application.
|
||||
|
||||
## Active Electron scenarios
|
||||
|
||||
The scenario request supplies the Electron executable and application entry
|
||||
point, plus any arguments and actions. The application root defaults to the
|
||||
entry point's parent directory and can be supplied when the app uses a different
|
||||
root. REA launches that executable through
|
||||
the official Playwright Electron API and owns its lifetime. It accepts click/wait actions,
|
||||
window-targeted renderer reload/crash actions, and synthetic `open-url` or
|
||||
`second-instance` deep-link delivery. It records capture-scoped window and
|
||||
WebContents identities, process metrics, and IPC channel and value-shape metadata.
|
||||
Payload values are not retained.
|
||||
|
||||
Run the real fixture verifier with an operator-provided Electron runtime:
|
||||
|
||||
```bash
|
||||
REA_ELECTRON_EXECUTABLE=/absolute/path/to/electron npm run verify:electron
|
||||
```
|
||||
|
||||
This operation actively launches and interacts with the selected target. It is
|
||||
not a passive CDP observation. The owned process keeps normal host filesystem
|
||||
and network privileges, so lifecycle ownership does not make it a sandbox.
|
||||
|
||||
The CLI and MCP surfaces accept the same schema. For a JSON request file:
|
||||
|
||||
```bash
|
||||
rea capture-electron-scenario scenario.json --json
|
||||
```
|
||||
|
||||
The result records action status and targets, correlated app/window/WebContents,
|
||||
preload, session, navigation, shell, permission, popup, download, protocol,
|
||||
native-addon, process, and IPC timeline events. IPC channels and argument-shape
|
||||
metadata retain their observed lengths; payload values are never retained.
|
||||
Renderer crash/restart and deep-link actions are synthetic
|
||||
scenario controls; their attempted and observed outcomes remain in the
|
||||
timeline. The active hook blocks and records external shell/navigation,
|
||||
permission, download, popup, updater, and OS-integration effects. The timeline
|
||||
is explicitly partial when attachment starts after application activity. The
|
||||
startup has a 60-second deadline. Actions wait until completion or request
|
||||
cancellation; a click whose target never appears remains pending until cancelled.
|
||||
CLI SIGINT/SIGTERM requests cancellation and awaits owned-process cleanup.
|
||||
Window IDs, URLs, and cached titles come from one main-process Electron snapshot,
|
||||
so a crashed renderer does not require a successful renderer-side title query.
|
||||
Window instrumentation uses Electron's window-created event, including runtimes
|
||||
with a read-only `BrowserWindow` export. Preload configuration comes from the
|
||||
window's runtime metadata, including Electron's optional internal preload
|
||||
descriptor when web preferences omit it. This descriptor is verified on
|
||||
Electron 44.7.0; runtimes that expose neither representation retain an unavailable
|
||||
path. A configured path or working IPC handler does not prove preload execution.
|
||||
If cleanup cannot be verified, the error retains collected observations in
|
||||
`details.partial_observation` and marks their application cleanup `unverified`.
|
||||
Cleanup uncertainty does not authorize signaling unrelated processes.
|
||||
|
||||
The real verifier exercises compiled CLI and stdio MCP discovery, page/script
|
||||
inspection, static/runtime reconciliation, active observation, and CLI SIGTERM
|
||||
cleanup before checking the IPC, renderer crash/reload, utility/child-process,
|
||||
deep-link, and blocked-effect fixture. A cleanup error keeps the full lane failed
|
||||
even when the retained capture contains useful observations.
|
||||
|
||||
Active Electron Evidence can also be supplied to
|
||||
[`reconcile_javascript_runtime`](javascript-runtime-reconciliation.md). That
|
||||
projection is intentionally target-only and partial: it binds the selected
|
||||
application path and capture outcome to the static graph, while frames, scripts,
|
||||
workers, and execution claims remain unavailable until a separate passive runtime
|
||||
capture provides them.
|
||||
|
||||
ASAR observation reads the header through one open file handle, validates its declared lengths against the archive size, and limits encoded JSON headers to 16 MiB before parsing. The header cache also retains at most 16 MiB of encoded headers across eight archives. Oversized headers produce a scope refusal.
|
||||
@@ -0,0 +1,193 @@
|
||||
# JavaScript Application Graph
|
||||
|
||||
JavaScript Application Graph is REA's provider-neutral domain model for
|
||||
connecting shipped application bytes, JavaScript structure, Electron process
|
||||
boundaries, passive browser observations, and native add-ons. It is a durable,
|
||||
canonical data contract; it is not an extractor or a new CLI/MCP tool.
|
||||
|
||||
The implementation is pure and side-effect free. It validates caller-supplied
|
||||
facts, derives semantic identifiers, enforces graph integrity, and serializes
|
||||
verified graphs as RFC 8785 canonical JSON. It performs no filesystem or CDP
|
||||
I/O, does not evaluate JavaScript, and has no application-specific node kinds.
|
||||
|
||||
## Layer boundary
|
||||
|
||||
The graph complements existing REA models instead of replacing them:
|
||||
|
||||
| Existing model | Responsibility | Application Graph use |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
|
||||
| Artifact graph | Content-addressed containers, byte occurrences, and extraction boundaries | Supplies artifact identities and exact paths |
|
||||
| Web bundle analysis | Bounded AST, chunk, route, endpoint, and source-map observations | Supplies static nodes and relationships |
|
||||
| Browser/Electron observation | Authorized passive CDP metadata and runtime captures | Supplies runtime instances and `observed_as` relationships |
|
||||
| Native analysis | Provider-qualified binary functions, symbols, and references | Supplies native add-on/export observations |
|
||||
| JavaScript Application Graph | Cross-layer entity identity, evidence-bearing relationships, and canonical commitments | Preserves the combined application model |
|
||||
|
||||
The shipped [JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
||||
service projects bounded directory/ASAR, package, bundle, source-map,
|
||||
BrowserWindow, preload, contextBridge, IPC, utility-process, and JavaScript-side
|
||||
native binding facts into this graph. The separate
|
||||
[static/runtime reconciliation](javascript-runtime-reconciliation.md) service
|
||||
adds capture-scoped targets, frames, scripts, workers, and inferred
|
||||
`observed_as` edges from already-produced Evidence. Native binary export
|
||||
verification remains a separate later layer. The domain graph itself never
|
||||
reaches outward to obtain any of those facts.
|
||||
|
||||
## Stored record
|
||||
|
||||
A stored graph has this top-level shape:
|
||||
|
||||
```text
|
||||
schema: "JavaScriptApplicationGraph"
|
||||
graph_id: jag_<sha256 of normalized graph semantics>
|
||||
root_node_ids: jag_node_<sha256>[]
|
||||
nodes: application entities with one or more observations
|
||||
edges: directed evidence-bearing relationships
|
||||
coverage: complete, partial, unknown, or unavailable
|
||||
limitations: explicit graph-wide constraints
|
||||
```
|
||||
|
||||
Parsing rejects unknown fields, non-canonical ordering, duplicate IDs, stale
|
||||
semantic commitments, missing roots, dangling endpoints, self-edges, and
|
||||
`changed_from` edges between unlike node kinds. Constructors normalize set-like
|
||||
arrays before deriving their IDs.
|
||||
|
||||
## Entities and relationships
|
||||
|
||||
The node vocabulary covers:
|
||||
|
||||
- package, installer, and artifact;
|
||||
- ASAR entry;
|
||||
- Electron main, preload, renderer, and utility process code;
|
||||
- JavaScript asset, chunk, and module;
|
||||
- source map and original source module;
|
||||
- BrowserWindow, frame, and target;
|
||||
- contextBridge API, IPC channel, and IPC handler;
|
||||
- worker and service worker;
|
||||
- endpoint and storage;
|
||||
- native add-on and native export;
|
||||
- runtime script instance;
|
||||
- an explicit `unknown` kind when classification is not justified.
|
||||
|
||||
The directed relation vocabulary is `contains`, `loads`, `imports`, `maps_to`,
|
||||
`exposes`, `sends`, `invokes`, `handles`, `calls`, `persists_to`, `observed_as`,
|
||||
and `changed_from`. Provider names are deliberately absent from both
|
||||
vocabularies.
|
||||
|
||||
For a `maps_to` edge targeting a source-map node, the edge carries
|
||||
`resolved_path` and `declared_url_sha256`. The target node's reference
|
||||
observation carries that same digest and the exact `declared_url`; match the
|
||||
digest to select the URL belonging to that edge when several references share
|
||||
one physical source map. The full URL is kept out of node identity, display
|
||||
label, and edge properties, so large inline data URLs are stored once per
|
||||
source-map reference.
|
||||
|
||||
Consumers previously reading `edge.properties.declared_url` should follow
|
||||
`target_node_id` and select the observation by `declared_url_sha256` instead.
|
||||
Node labels serve display; the observation preserves the original URL.
|
||||
|
||||
For example, a source-owned synthetic fixture proves this path without using a
|
||||
third-party application:
|
||||
|
||||
```text
|
||||
package contains app.asar
|
||||
main asset contains BrowserWindow
|
||||
BrowserWindow loads preload role
|
||||
preload asset exposes contextBridge API
|
||||
preload asset invokes uniquely matched main-process IPC handler
|
||||
main-process IPC handler handles literal IPC channel
|
||||
JavaScript asset imports requested native binding
|
||||
resolved native add-on exposes requested binding (not a verified binary export)
|
||||
```
|
||||
|
||||
The `loads`, `exposes`, `invokes`, `handles`, and `imports` relationships remain
|
||||
static inferences. Dynamic or ambiguous IPC is retained without a pairing edge.
|
||||
Runtime entities retain passive-CDP authority, while a successful static/runtime
|
||||
correspondence is a separately identified cross-layer inference.
|
||||
|
||||
## Evidence and authority
|
||||
|
||||
Every observation and edge carries all of the following:
|
||||
|
||||
- an authority, such as artifact bytes, AST analysis, static relationship
|
||||
inference, passive CDP runtime, and cross-layer reconciliation,
|
||||
native analysis, historical reference, or user assertion;
|
||||
- an epistemic state: `observed`, `inferred`, `unknown`, or `unavailable`;
|
||||
- confidence independently bounded as exact, high, medium, low, or unknown;
|
||||
- an explicit content-addressed artifact reference or an unavailable reason;
|
||||
- an explicit artifact, source, URL, runtime, native, or package-export
|
||||
location, or an unavailable reason;
|
||||
- extractor name, version, operation, and optional executable digest;
|
||||
- coverage status, truncation, omitted count, and named limits;
|
||||
- limitations and links to existing Evidence records;
|
||||
- a content-derived identifier and its declared stability strategy.
|
||||
|
||||
Authority and epistemic state are separate fields. An AST observation does not
|
||||
become a runtime fact, and a passive runtime fact does not prove which static
|
||||
module produced it. In particular:
|
||||
|
||||
- `static-relationship-inference` can only produce `inferred` facts;
|
||||
- `cross-layer-reconciliation` can only produce `inferred` facts and must link
|
||||
both static and runtime Evidence;
|
||||
- unknown or unavailable facts require unknown confidence and a limitation;
|
||||
- inferred facts require an explicit limitation;
|
||||
- exact confidence is reserved for observations;
|
||||
- all observations require an actionable location, and artifact-backed facts
|
||||
require a digest;
|
||||
- static/native authorities cannot claim runtime locations;
|
||||
- truncated coverage must be partial, name the limiting bound, and not claim
|
||||
zero omissions;
|
||||
- complete coverage cannot be truncated or omit items.
|
||||
|
||||
Unavailable values are represented explicitly rather than as missing fields,
|
||||
so absence of evidence cannot silently become evidence of absence.
|
||||
|
||||
## Entity identity
|
||||
|
||||
Nodes separate stable entity identity from individual observations. Changing a
|
||||
label, extractor, or evidence record changes the observation ID without
|
||||
necessarily changing the node ID.
|
||||
|
||||
| Strategy | Stability scope | Intended use |
|
||||
| ------------------------- | -------------------------------- | ----------------------------------------------------------------- |
|
||||
| `content-digest` | Globally exact bytes | Artifacts and byte-identical assets |
|
||||
| `source-map-original` | One source-map commitment | Recovered original source modules |
|
||||
| `canonical-path` | One artifact version | ASAR entries and assets at exact normalized paths |
|
||||
| `artifact-local-key` | One artifact version | IPC channels, handlers, exports, and other named entities |
|
||||
| `structural-fingerprint` | Explicit cross-version inference | Modules whose relationship is hypothesized from bounded structure |
|
||||
| `runtime-instance` | One capture only | Targets, frames, runtime scripts, and workers |
|
||||
| `observation-fingerprint` | One observation scope | Facts with no stronger justified identity |
|
||||
|
||||
Every strategy states its stability scope. Runtime identities commit the
|
||||
capture digest and runtime key; canonical-path and artifact-local identities
|
||||
commit the containing artifact digest. Structural fingerprints require an
|
||||
inferred supporting observation and never claim exact cross-version identity.
|
||||
Content-digest and source-map identities require an observation of the same
|
||||
artifact digest.
|
||||
|
||||
Observation IDs use `semantic-content-sha256` with `observation-exact`
|
||||
stability. Edge IDs use the same derivation with `relationship-exact`
|
||||
stability. Node, observation, edge, and graph IDs are recomputed during parsing,
|
||||
so callers cannot preserve a stale ID after changing semantic content.
|
||||
|
||||
## Structure
|
||||
|
||||
The graph does not impose aggregate caps on roots, nodes, edges, observations,
|
||||
or JSON properties. Properties must be valid JSON; field-specific validation
|
||||
still applies to paths, identifiers, and evidence values. Extractors report
|
||||
actual omissions in coverage instead of silently dropping a prefix.
|
||||
|
||||
## Domain API
|
||||
|
||||
The public domain functions are:
|
||||
|
||||
- `createJavaScriptApplicationNode` to normalize an entity and derive node and
|
||||
observation IDs;
|
||||
- `createJavaScriptApplicationEdge` to normalize a relationship and derive its
|
||||
ID;
|
||||
- `createJavaScriptApplicationGraph` to sort complete graph content and derive
|
||||
its graph ID;
|
||||
- `parseJavaScriptApplicationGraph` to verify a stored graph and every
|
||||
commitment;
|
||||
- `serializeJavaScriptApplicationGraph` for canonical JSON;
|
||||
- `computeJavaScriptApplicationGraphSha256` for a byte-stable verified graph
|
||||
digest.
|
||||
@@ -0,0 +1,296 @@
|
||||
# Cross-layer JavaScript application workflows
|
||||
|
||||
REA derives a complete reachable feature trace from one authenticated JavaScript
|
||||
Application Graph and compares two authenticated graph versions. The MCP tools
|
||||
are `inspect_analysis_view`, `trace_application_feature`,
|
||||
`trace_javascript_semantics`, `compare_application_versions`,
|
||||
`compare_source_to_bundle`, and `compare_javascript_export_shapes`; their CLI
|
||||
equivalents use the same names with hyphens. `inspect_analysis_view` projects a
|
||||
summary, module page, or one module from already completed application
|
||||
Evidence; it does not walk relationships.
|
||||
|
||||
`inspect_analysis_view` also projects retained `inspect_binary_layout`
|
||||
Evidence. The remaining workflows consume Evidence produced by
|
||||
`analyze_javascript_application` or `reconcile_javascript_runtime`. They do not
|
||||
read an artifact, execute application code, attach to a process, or open a
|
||||
native-analysis provider. Static artifact observations, passive runtime
|
||||
observations, relationship inferences, and unknown or unavailable facts retain
|
||||
their original graph authority.
|
||||
|
||||
`analyze_javascript_application` Evidence retains the structural JavaScript
|
||||
Application Graph and a separate semantic relation graph bound to the same root
|
||||
artifact digest and structural graph ID. Semantic tracing requires the semantic
|
||||
relation graph and reports when it is unavailable rather than treating missing
|
||||
data as an empty graph.
|
||||
|
||||
The semantic graph wire format stores shared non-location provenance once in
|
||||
its required `evidence_contexts` table. Each node, relation, fingerprint, and
|
||||
unknown retains its exact `evidence.location` and names the owning context with
|
||||
`evidence.context_id`. This is a breaking representation change: consumers
|
||||
reading fields such as `evidence.authority` must look up the context by ID and
|
||||
combine its fields with the fact's location. Semantic trace results include the
|
||||
canonical context subset referenced by their returned facts, so those facts
|
||||
remain self-contained in the response.
|
||||
|
||||
## Feature tracing
|
||||
|
||||
Select one literal seed kind: node ID, route, string, API, IPC channel, module,
|
||||
or native export. Matching is exact or literal substring matching; regular
|
||||
expressions and executable predicates are not accepted. The trace includes all
|
||||
matching seeds and the complete graph reachable in the selected direction.
|
||||
|
||||
The result contains the matching basis, complete reachable subgraph, terminal
|
||||
paths, authority summaries, and native handoffs. A handoff binds
|
||||
the exact native artifact digest and requested exports from the application
|
||||
graph. Existing Hopper or Ghidra Evidence is linked only when its subject digest
|
||||
matches exactly. Otherwise the result recommends provider-neutral follow-up
|
||||
tools and reports `requires-provider-analysis`; it never starts or switches a
|
||||
provider implicitly.
|
||||
|
||||
## Semantic relation tracing
|
||||
|
||||
`trace_javascript_semantics` queries the authenticated companion graph without
|
||||
rereading mutable files. A seed may identify a semantic or application node,
|
||||
literal, function fingerprint, property, endpoint, event, or boundary field.
|
||||
The query declares backward provenance, forward influence, callers, or
|
||||
ownership and can restrict admitted relation kinds. REA returns every matching
|
||||
seed and the complete reachable subgraph; callers do not choose node, relation,
|
||||
depth, function, module, seed-match, or page budgets.
|
||||
|
||||
The extractor covers lexical definitions and reads, literal seeds, static
|
||||
property slots, object reads/writes/spreads/destructuring, closure captures,
|
||||
uniquely resolved local calls, argument/return flow, and explicit Promise
|
||||
construction, chaining, aggregation, await, return, assignment, and
|
||||
detachment. It also recovers static candidates for EventEmitter
|
||||
registrations/removals and dispatch, Node timers and cancellation handles, asynchronous
|
||||
`node:child_process` creation and ownership, configuration sources and direct
|
||||
defaults, request construction and response consumers, parse/coercion/
|
||||
validation boundaries, and built-in resource acquisition/release.
|
||||
|
||||
Mutation tracking covers explicit member assignments, updates, deletion, and
|
||||
loop assignment targets, including the supported local aliases and shared
|
||||
nested references. Calls conservatively invalidate exposed mutable references;
|
||||
callee bodies and built-ins such as `Object.assign`, `Reflect.set`, and
|
||||
`Reflect.deleteProperty` are not interpreted to establish exact post-call
|
||||
values. This uncertainty does not prove that a call actually mutated a value.
|
||||
Interprocedural aliasing beyond the supported call-site references remains
|
||||
outside this mutation pass.
|
||||
|
||||
Function fingerprints commit normalized syntax, control-flow shape, relation
|
||||
shape, literal sets, arity, and detected effects without using local names or
|
||||
source offsets. Equal duplicate fingerprints remain ambiguous. Dynamic
|
||||
calls/properties/names, parser recovery, unresolved handles, and unsupported API
|
||||
shapes stay explicit unknowns; a missing relation is never reported as proven
|
||||
behavioral absence.
|
||||
|
||||
Semantic relations use static-inference authority. `resolved` means the static
|
||||
target identity was unique within admitted analysis; it does not mean the path
|
||||
executed. Candidate edges are excluded by default, require explicit opt-in, and
|
||||
keep the result ambiguous. Runtime Evidence can later corroborate an exact
|
||||
mapped candidate, but structural reachability, semantic influence, runtime
|
||||
observation, and causal proof remain separate claims.
|
||||
|
||||
Semantic literal nodes keep their complete value in `properties.value`; literal
|
||||
queries match that field. String labels use `string literal` when that is
|
||||
shorter than the complete JSON value. A literal's `identity.role_key` commits
|
||||
its canonical JSON value as `value-sha256:<digest>` when the digest key is
|
||||
shorter; smaller values keep their existing JSON key. This keeps large values
|
||||
out of identity and display metadata without expanding short values or
|
||||
truncating their Evidence.
|
||||
|
||||
New analyses therefore produce different IDs for hashed literals from the
|
||||
earlier payload-bearing role keys. Use IDs returned by the selected parent
|
||||
graph and read literal values from `properties.value`, rather than decoding
|
||||
role keys or labels. Previously stored graphs and their original IDs remain
|
||||
valid inputs.
|
||||
|
||||
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
|
||||
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
|
||||
identity keys use `value-sha256:<digest>` over the canonical JSON string when
|
||||
that is shorter, or when the original value starts with the reserved digest
|
||||
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
|
||||
shorter of the exact value and `<kind> endpoint`; semantic request/response
|
||||
labels use the shorter of the endpoint and operation method. These labels are
|
||||
display metadata, not endpoint lookup values.
|
||||
|
||||
New analyses therefore change hashed endpoint IDs and longer endpoint labels.
|
||||
Use IDs from the selected graph and read the complete endpoint properties;
|
||||
previously stored graphs still support their original IDs. Literal endpoint
|
||||
search, semantic tracing, and version comparison use the preserved values.
|
||||
|
||||
Event and listener nodes likewise preserve the complete event name in
|
||||
`properties.event_name`. Event role keys use the same reserved-prefix string
|
||||
digest rule, scoped to the emitter; repeated registrations, removals, and
|
||||
dispatches keep one event per exact emitter/name pair. Long event labels use
|
||||
`event`, and listener labels use the shorter of `<method>:<name>` and
|
||||
`<method>:listener`. Dynamic names remain `null` with explicit uncertainty. New analyses
|
||||
change hashed event IDs and longer event/listener labels; query the preserved
|
||||
name or select IDs from the parent graph. Previously stored graphs keep their
|
||||
original IDs and remain valid trace inputs.
|
||||
|
||||
## Version comparison
|
||||
|
||||
REA pairs entities only when a tier produces one unique candidate on each side.
|
||||
The tiers are exact content digest, exact module-source digest, exact node
|
||||
identity, source-map identity, structural fingerprint, and a semantic key for
|
||||
non-module entities. Lower tiers never override a higher unique match.
|
||||
|
||||
Module ordinals, stable minified names, and fuzzy text similarity are not
|
||||
persistent identities. Duplicate candidates remain ambiguous. Source-map and
|
||||
structural matches are high/medium-confidence inferences rather than exact
|
||||
facts. Each item is `unchanged`, `added`, `removed`, `changed`, or `unknown`,
|
||||
and the result includes its basis, candidates, changed dimensions, Evidence
|
||||
links, limitations, and a `changed_from` graph containing all compared nodes,
|
||||
their observations, and relationships.
|
||||
|
||||
One-sided absence is `added` or `removed` only when the opposite input graph has
|
||||
complete coverage. With partial, unavailable, or truncated input, the same
|
||||
condition is `unknown`. The comparison retains every classified item and
|
||||
ambiguity candidate. Coverage carries omitted counts and limits from each input
|
||||
graph; comparison adds no caller-selected item, candidate, graph node, or edge
|
||||
budget. Unresolved comparison items are recorded as a residual unknown linked
|
||||
to the comparison Evidence in a live session.
|
||||
|
||||
## Historical source-to-bundle comparison
|
||||
|
||||
`compare_source_to_bundle` compares one cryptographically committed
|
||||
`HistoricalSourceGraph` with authenticated application-graph Evidence. The
|
||||
stable scoring model reports every admitted signal and weight: exact source
|
||||
digest, source-map original path, exact current path, path suffix, basename, and
|
||||
language extension. Exact digest wins over location inference; weak basename
|
||||
or extension evidence never forces a mapping.
|
||||
|
||||
Each historical source file is classified as `unchanged`, `modified`,
|
||||
`removed`, `split`, `merged`, `duplicated`, or `unknown`. Multiple equally
|
||||
scored path mappings form a static split inference. Multiple current
|
||||
nodes with the exact historical digest are duplicated; multiple historical
|
||||
paths with exact identity to one current node are merged. `removed` requires
|
||||
complete historical and application inventories. The comparison analyzes and
|
||||
returns all source files, candidate nodes, and matching signals represented in
|
||||
the supplied graphs. Partial inventories remain unknown where absence cannot
|
||||
be established. These static mappings do not claim runtime loading or semantic
|
||||
equivalence.
|
||||
|
||||
## Export return-shape comparison
|
||||
|
||||
`compare_javascript_export_shapes` selects exactly one module path and export
|
||||
name on each authenticated graph. Missing or duplicate exact selectors return
|
||||
candidate inventory and an unknown result; the tool never chooses a fuzzy
|
||||
match. An export is linked to a callable only through exact lexical/module
|
||||
relationships recovered by the AST analysis.
|
||||
|
||||
Direct return expressions, including expression-bodied arrows, are evaluated
|
||||
through an execution-free value lattice. Literal object fields and direct
|
||||
return sites are represented in the result. Calls, dynamic spreads, computed
|
||||
keys, and parser recovery remain partial or unknown.
|
||||
Objects and arrays passed to calls, constructors, or tagged templates, used
|
||||
as method receivers, or stored through property targets are not assumed
|
||||
unchanged afterward. Property stores conservatively retain reference escape
|
||||
uncertainty rather than pretending to execute later writes. References
|
||||
handed back by local functions, methods, constructors, async functions, and
|
||||
generators also retain escape uncertainty, including when the call is read-only;
|
||||
this does not establish that a mutation occurred. Aliases
|
||||
and shared children in spread and rest copies retain that uncertainty; copied
|
||||
primitive slots and unrelated containing properties remain known. Object rest
|
||||
excludes consumed keys, array rest excludes the consumed prefix for known
|
||||
arrays, and later explicit data properties and methods replace earlier object
|
||||
spread references. A known array prefix does not expose values from a following
|
||||
spread. Accessors, custom or uncertain iteration, unknown computed keys, and
|
||||
unknown spread lengths retain conservative uncertainty.
|
||||
Nested callable returns are not assigned to their parent callable. Projected
|
||||
graph observations carry source ranges but never source text.
|
||||
|
||||
Return variants pair only when a literal discriminant such as `/type` has one
|
||||
unique occurrence on each side and pairing is reciprocal. Source order never
|
||||
implies correspondence. Each retained variant contributes a property inventory
|
||||
of observed JSON Pointer names, including fields whose static values stay
|
||||
unknown, plus that variant's parent-property coverage.
|
||||
|
||||
Changes use JSON Pointer paths with `added`, `removed`, `changed`, or `unknown`
|
||||
status and a separate `presence` object (`present`, `absent`, or
|
||||
`unknown-coverage`) on each side. A name is `added` or `removed` when one paired
|
||||
shape proves presence and the other proves absence. Complete parent-property
|
||||
coverage can establish absence even when field values stay unresolved. Identical literals are
|
||||
omitted. Unresolved values that remain on both sides stay `unknown` when the
|
||||
projections differ, and are omitted when presence is unchanged and the unknown
|
||||
projections match. Incomplete spreads and other partial parent coverage keep
|
||||
one-sided names `unknown` with `unknown-coverage` on the incomplete side.
|
||||
Unpaired variants remain visible as unknown changes and still list their
|
||||
property inventories, each with its own `source_range`. Inventories exclude
|
||||
array holes and slots whose presence was invalidated by mutation; an unresolved
|
||||
value alone does not make an observed property uncertain. `summary.added` and
|
||||
`summary.removed` count
|
||||
presence-level add/remove as well as literal value add/remove;
|
||||
`summary.unknown` does not absorb complete-coverage presence-only gaps.
|
||||
A partial comparison still records an unknown in the MCP session when matching
|
||||
uncertain projections produce no changes and `summary.unknown` is zero.
|
||||
|
||||
The output includes exact selector candidates, omissions, Evidence links,
|
||||
coverage, and limitations; it does not execute JavaScript. When runtime
|
||||
semantics matter, run behavioral probes directly against the relevant
|
||||
application versions and capture them through the available browser, Electron,
|
||||
or process workflows.
|
||||
|
||||
## CLI and verification
|
||||
|
||||
All six CLI commands accept inline JSON or a path to a JSON file. The CLI
|
||||
returns an Evidence record directly. Put the full records in a later CLI input;
|
||||
a separate CLI process has no retained MCP connection state.
|
||||
|
||||
File inputs must be regular files; symlinks to regular files are accepted.
|
||||
Directories, named pipes and device files produce an input error before JSON
|
||||
parsing.
|
||||
|
||||
For a literal string trace, analyze your supplied tree once, then build the
|
||||
input from the saved Evidence (replace the target and seed):
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > application-evidence.json
|
||||
node --input-type=module -e '
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
const application = JSON.parse(readFileSync("application-evidence.json", "utf8"));
|
||||
writeFileSync("trace-input.json", JSON.stringify({
|
||||
application, seed: { kind: "string", value: "search-result" }, direction: "both"
|
||||
}));'
|
||||
rea trace-application-feature ./trace-input.json --json
|
||||
```
|
||||
|
||||
MCP analysis returns the complete Evidence record, with the operation result
|
||||
in `normalized_result` and its identity in `evidence_id`. If the connected server advertises retained references, reuse its
|
||||
exact returned ID as `{"kind":"retained-evidence","evidence_id":"RETURNED_ID"}`
|
||||
in `application`, or `left`/`right` for comparisons. `RETURNED_ID` is a template,
|
||||
not a literal valid ID. Native Evidence arrays still use complete records.
|
||||
References are scoped to one connection; `close_binary` clears them. Export a
|
||||
bundle before closing and import it on another connection, or supply the full
|
||||
inline Evidence there. Versions before 4.1.0 accept full inline Evidence only; installing
|
||||
newer skill instructions does not change that schema. See
|
||||
[MCP Evidence inputs](../../reference/mcp-contracts.md#retained-application-evidence-inputs).
|
||||
|
||||
For two operator-provided directories or ASARs, run:
|
||||
|
||||
```bash
|
||||
npm run verify:application-workflows -- \
|
||||
--left /absolute/path/to/version-a \
|
||||
--right /absolute/path/to/version-b
|
||||
```
|
||||
|
||||
The verifier reconstructs both versions independently, compares them, runs one
|
||||
literal trace when a seed is available, and compares the first common exact
|
||||
export return shape when present. It prints only graph, artifact, and Evidence
|
||||
identifiers plus matching, changes, summary, handoff, and coverage statistics;
|
||||
it does not print source text. Use `--seed-kind` and `--seed-value` to select a
|
||||
specific route, string, API, channel, module, native export, or node ID. Supply
|
||||
all four `--left-module-path`, `--left-export-name`, `--right-module-path`, and
|
||||
`--right-export-name` options to verify an explicit export pair.
|
||||
|
||||
Because source Evidence and derived Evidence are retained by the normal session
|
||||
ledger, evidence bundles and analysis snapshots can carry these records without
|
||||
another persistence format.
|
||||
|
||||
## Runtime evidence
|
||||
|
||||
Static graph workflows do not execute recovered code. When runtime semantics
|
||||
matter, run behavioral probes directly against the relevant application
|
||||
versions and capture them through the available browser, Electron, or process
|
||||
workflows. Those observations do not prove behavior in an unobserved app
|
||||
version or environment.
|
||||
@@ -0,0 +1,621 @@
|
||||
# JavaScript artifact reconstruction
|
||||
|
||||
Inspect a local Electron/JavaScript directory or ASAR without executing its code.
|
||||
`analyze_javascript_application` and the CLI routes below return an Evidence
|
||||
record with the [application graph](javascript-application-graph.md), static
|
||||
Electron observations, statistics, limitations, and coverage.
|
||||
|
||||
Start with the [public workflow](#public-workflow), select sources when useful,
|
||||
then [read the analysis](#read-the-analysis). Use [application workflows](javascript-application-workflows.md)
|
||||
for tracing and comparisons, or [partial recovery](#recover-partial-analysis)
|
||||
when an operation is interrupted.
|
||||
|
||||
## Public workflow
|
||||
|
||||
The tool reads the caller-selected local directory or ASAR path directly:
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/apps/app.asar --json
|
||||
rea analyze-javascript-application /absolute/path/to/apps/app.asar \
|
||||
--integrity-policy record-and-continue --json
|
||||
```
|
||||
|
||||
Generic `rea analyze` selects this static provider for directories and `.asar`
|
||||
paths when neither `--provider` nor `--snapshot` is supplied. The dedicated
|
||||
command and the generic route return their complete results inline. Native
|
||||
targets, single JavaScript files, `.app` bundles, and explicit deep-provider or
|
||||
snapshot requests retain the native deep-analysis route.
|
||||
|
||||
Both CLI routes emit `rea_progress` JSON lines on stderr as inventory, source
|
||||
parsing, graph construction, validation, and Evidence hashing begin. Stdout
|
||||
remains the selected result document. Completion is reported after Evidence
|
||||
creation; a failed analysis retains its typed diagnostic result.
|
||||
|
||||
Configure an MCP client with the ordinary REA setup command:
|
||||
|
||||
```bash
|
||||
rea setup
|
||||
```
|
||||
|
||||
The equivalent MCP input is:
|
||||
|
||||
```json
|
||||
{
|
||||
"input_path": "/absolute/path/to/apps/app.asar",
|
||||
"format": "auto",
|
||||
"integrity_policy": "fail"
|
||||
}
|
||||
```
|
||||
|
||||
`input_path` must be absolute. `format` accepts `auto`, `asar`, or `directory`.
|
||||
`integrity_policy` accepts `fail` or `record-and-continue`; both CLI routes and
|
||||
the MCP tool default to `fail`.
|
||||
The result retains the canonical local path, root artifact digest, artifact
|
||||
manifest and graph commitments, JavaScript Application Graph, static
|
||||
Electron summary, reconstruction statistics, and explicit limitations. It does
|
||||
not require a live Hopper, Ghidra, browser, or Electron process.
|
||||
|
||||
The application graph selects the shallowest `package.json` outside
|
||||
`node_modules` as its package root, breaking equal-depth ties by Unicode
|
||||
code-point order. Dependency manifests remain package observations but do not
|
||||
declare Electron main or renderer entries. If only dependency manifests are
|
||||
present, the inventory artifact remains the graph root.
|
||||
|
||||
## Select sources before analysis
|
||||
|
||||
For a question about known entry sources, supply exact artifact-relative file
|
||||
paths in `source_paths`, or repeat `--source-path` on either CLI route:
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/app.asar \
|
||||
--source-path package.json --source-path main.js --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"input_path": "/absolute/path/to/app.asar",
|
||||
"source_paths": ["package.json", "main.js"],
|
||||
"detail": "summary"
|
||||
}
|
||||
```
|
||||
|
||||
REA inventories and verifies the complete artifact, including unselected bytes,
|
||||
but allocates source text and performs static analysis only for selected files.
|
||||
Include supporting manifests, HTML or source maps explicitly when needed;
|
||||
imports do not expand the selection. Paths must match relevant inventoried
|
||||
files exactly, including nested ASAR prefixes. Unknown paths, aliases and globs
|
||||
return an `invalid_request` correction. Selected files are analyzed in request
|
||||
order before unselected scopes are projected. Repeated paths preserve the
|
||||
caller's request and first-occurrence priority, analyzing the file once.
|
||||
Omit the selection for the existing complete
|
||||
application workflow.
|
||||
|
||||
The result and summary retain `source_selection.requested_paths` and
|
||||
`source_selection.unselected_files`. Every relevant file keeps its original
|
||||
inventory identity; unselected files report `text_status:
|
||||
"outside-analysis-scope"`. Their relationships remain unknown, and application
|
||||
coverage is partial while any file remains outside scope. Selection does not
|
||||
claim whole-application completeness or change integrity policy. Summary detail
|
||||
only reduces delivery; source selection reduces text acquisition and analysis.
|
||||
Explicit native provider or snapshot routing rejects `--source-path` rather
|
||||
than ignoring it.
|
||||
|
||||
## Read the analysis
|
||||
|
||||
`analyze_javascript_application` returns the complete analysis Evidence by
|
||||
default. Select `"detail": "summary"` to retain that complete Evidence in the
|
||||
current session and receive only its `inspect_analysis_view` summary: artifact
|
||||
identity, statistics, Electron surface counts, application and semantic
|
||||
coverage, limitations, and `normalized_result.parent_evidence_id`. Pass that ID
|
||||
to `inspect_analysis_view` for module pages or items, or to the application
|
||||
workflows as a retained reference. The summary does not repeat or truncate the
|
||||
analysis; a server without session retention refuses summary detail with
|
||||
`capability_unavailable`. CLI output is unchanged and always complete.
|
||||
|
||||
With a connected MCP `client` and the caller-selected absolute `inputPath`, this
|
||||
recipe analyzes once and pages module identities using the returned ID:
|
||||
|
||||
```js
|
||||
const analysis = await client.callTool({
|
||||
name: "analyze_javascript_application",
|
||||
arguments: { input_path: inputPath, detail: "summary" },
|
||||
});
|
||||
if (analysis.isError) throw new Error(JSON.stringify(analysis.content));
|
||||
const parentId =
|
||||
analysis.structuredContent.normalized_result.parent_evidence_id;
|
||||
const modules = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: { kind: "page", collection: "modules", offset: 0, limit: 16 },
|
||||
},
|
||||
});
|
||||
if (modules.isError) throw new Error(JSON.stringify(modules.content));
|
||||
```
|
||||
|
||||
Select a returned module's exact `node_id` for an item view. A path can identify
|
||||
several graph roles; module items preserve their recorded properties and source
|
||||
locations. Follow `coverage.next_offset` when another page matters. For feature
|
||||
tracing, version comparison, and retained-input examples, use the
|
||||
[application workflows](javascript-application-workflows.md).
|
||||
|
||||
For supplied passive browser or Electron observations, use
|
||||
[runtime reconciliation](javascript-runtime-reconciliation.md). It compares
|
||||
existing Evidence without rerunning static analysis or contacting CDP.
|
||||
|
||||
## ASAR integrity
|
||||
|
||||
ASAR inventory checks Electron integrity metadata for embedded archive entries
|
||||
and supplied `.asar.unpacked` companion files. An integrity failure identifies
|
||||
the logical path, declared and calculated SHA-256 values, and whether the entry
|
||||
was unpacked. By default, a mismatch is returned as a failure with its artifact
|
||||
context. Application analysis can explicitly select `record-and-continue` to
|
||||
analyze observed bytes while retaining each contradiction's declared and
|
||||
observed hashes and `observed-untrusted` trust. Such results mark application
|
||||
graph coverage partial and include every contradicted path in the limitations.
|
||||
Mismatched nested ASARs stay opaque rather than being expanded.
|
||||
|
||||
An unpacked entry whose companion bytes were not supplied remains
|
||||
`unavailable`. REA continues analyzing embedded JavaScript and records the
|
||||
missing native/resource bytes as unknown. The occurrence keeps
|
||||
`hash_status: unavailable`; no content-addressed child artifact is created for
|
||||
those missing bytes. See [MCP integrity handling](../../reference/mcp-contracts.md#integrity-record-and-continue)
|
||||
for the tool-result contract.
|
||||
|
||||
## Large results
|
||||
|
||||
Use MCP `detail: "summary"` for a large application, then read selected views of
|
||||
the retained analysis. Summary changes delivery only. To reduce source parsing,
|
||||
select exact [source paths](#select-sources-before-analysis).
|
||||
|
||||
If a complete response exceeds the transport budget, use its retained Evidence
|
||||
ID for a summary, module page, or item, or export the bundle. See
|
||||
[MCP tool results](../../reference/mcp-contracts.md#tool-results) for delivery limits, export, and
|
||||
matching server/client buffer settings. Follow-up results also need to fit that
|
||||
budget. [Analyzer limits](#analyzer-and-result-admission) control analysis memory,
|
||||
source deadlines, and semantic capacity separately.
|
||||
|
||||
## Recover partial analysis
|
||||
|
||||
A source timeout stops the remaining analysis and returns completed facts as
|
||||
partial Evidence. Cancellation after source analysis begins can also retain
|
||||
completed facts, including when interrupted during cleanup, graph construction,
|
||||
or Evidence sealing. Unexamined relationships remain unknown.
|
||||
|
||||
MCP returns a `partial_observation` retained reference after recording succeeds;
|
||||
open it with `inspect_analysis_view` on the same connection. If the SDK stopped
|
||||
waiting before receiving the reply, follow [MCP cancellation recovery](../../reference/mcp-contracts.md#progress-and-cancellation)
|
||||
to find the record. CLI failure output contains the complete partial Evidence.
|
||||
Owned analyzer cleanup is verified before private files are released; uncertain
|
||||
cleanup retains ownership for retry and is reported separately.
|
||||
|
||||
## What is reconstructed
|
||||
|
||||
The projector reuses the content-addressed artifact inventory and safe artifact
|
||||
readers. It retains canonical artifact-relative paths, exact byte counts,
|
||||
SHA-256 digests, inventory IDs, ASAR container identity, and `.asar.unpacked`
|
||||
status. Direct ASAR inputs and filesystem-backed ASAR files nested beneath a
|
||||
directory are supported.
|
||||
|
||||
JavaScript sources (`.js`, `.jsx`, `.mjs`, and `.cjs`) and TypeScript sources
|
||||
(`.ts`, `.tsx`, `.mts`, and `.cts`) are parsed as inert text; analysis does not
|
||||
execute them.
|
||||
|
||||
JavaScript and HTML source ranges retain an initial UTF-8 BOM as one UTF-16
|
||||
code unit, matching the original bytes identified by the artifact digest.
|
||||
|
||||
Plain `.ts` artifact sources use TypeScript syntax without JSX, including
|
||||
angle-bracket type assertions. `.tsx` and `.jsx` retain JSX parsing.
|
||||
|
||||
Relative TypeScript imports can name their emitted `.js`, `.mjs`, or `.cjs`
|
||||
counterparts while resolving to inventoried source files. This substitution
|
||||
requires import or require semantics. Literal Electron preload, renderer, and
|
||||
utility-process runtime paths retain the runtime filename selected by the caller.
|
||||
|
||||
Selected bounded text is then parsed as inert data to recover:
|
||||
|
||||
- `package.json` metadata and declared main or renderer entrypoints;
|
||||
- Electron preload paths and renderer files visible in static syntax;
|
||||
- local HTML script entrypoints;
|
||||
- Webpack and Rspack chunk registrations and module factories represented as
|
||||
AST literals;
|
||||
- static imports, dynamic imports, CommonJS `require` calls, workers, and
|
||||
service workers;
|
||||
- route, network endpoint, storage, and vendor-marker observations;
|
||||
- local source-map declarations and original
|
||||
source names and content digests;
|
||||
- explicit BrowserWindow options and `webPreferences`, including statically
|
||||
resolvable preload entrypoints;
|
||||
- `contextBridge.exposeInMainWorld` and `exposeInIsolatedWorld` API keys and
|
||||
bounded literal member paths;
|
||||
- renderer and main-process IPC operations, literal or dynamic channels, exact
|
||||
handler locations, and conservative pairing status;
|
||||
- sender, frame, URL, and origin validation candidates without claiming that a
|
||||
visible check enforces a complete policy;
|
||||
- utility-process entrypoints and native `.node` binding requests without
|
||||
parsing or executing the add-on.
|
||||
|
||||
Overloaded `.open` calls use lexical receiver facts: ambient browser window and
|
||||
document receivers are treated as browsing-context or document operations,
|
||||
while locally shadowed receivers can still contribute network endpoint
|
||||
candidates. A template recovered with a parser error and no cooked value stays
|
||||
dynamic; its raw spelling is not treated as a valid JavaScript string.
|
||||
|
||||
RPC client syntax also contributes network endpoint candidates, in both
|
||||
`analyze_javascript_application` and `analyze_web_bundle`, distinguished by
|
||||
`mechanism`:
|
||||
|
||||
- `trpc:procedure:<method>` records the static member path between the root
|
||||
expression and the method, such as `invoice.list` in
|
||||
`root.invoice.list.useQuery(...)`. tRPC proxies keep these property names
|
||||
through identifier minification. React hooks and TanStack option factories
|
||||
(`useQuery`, `useMutation`, `useInfiniteQuery`, `useSubscription`, suspense
|
||||
and prefetch variants, `queryOptions`, `infiniteQueryOptions`,
|
||||
`mutationOptions`, `subscriptionOptions`) are recognized directly. The
|
||||
vanilla `query` and `mutate` methods are recognized on a root bound to
|
||||
`createTRPCClient` or `createTRPCProxyClient` in the same lexical binding, or on a
|
||||
path of at least two segments in a source containing `TRPCClientError`,
|
||||
`trpc-accept`, or `/trpc`. Factory bindings must have one unconditional
|
||||
initializer and no binding assignments; shadowed locals do not inherit client
|
||||
identity. `subscribe` additionally requires a tRPC observer
|
||||
argument (`onData`, `onError`, `onStarted`, `onStopped`, `onComplete`, or
|
||||
`onConnectionStateChange`). `this` roots, browser globals, and any path
|
||||
containing an RTK Query `endpoints` segment are never procedures; cache-key
|
||||
helpers such as `queryKey` issue no request and are excluded.
|
||||
- `trpc:link:<link>` records the literal `url` option of `httpLink`,
|
||||
`httpBatchLink`, `httpBatchStreamLink`, and `httpSubscriptionLink` when the
|
||||
link's callee name survives bundling, including `(0, module.httpBatchLink)`
|
||||
import-call wrappers.
|
||||
- `graphql:query`, `graphql:mutation`, and `graphql:subscription` record a
|
||||
named operation found at the start of a string or cooked template literal —
|
||||
the name must be followed by variable definitions (`($`), a directive, or a
|
||||
selection set starting with a field name — or in a precompiled
|
||||
`OperationDefinition` document object. A graphql-tag `loc.source.body` copy is
|
||||
not reported twice. Anonymous operations are not reported.
|
||||
|
||||
A procedure path is the server-side procedure name, not a resolved URL; the
|
||||
transport URL depends on the link configuration and batching. When the proxy is
|
||||
reached through a member expression, such as a webpack module binding
|
||||
(`l.S.invoice.list`) or a property holding the proxy (`o.api.post.byId`), the
|
||||
recorded path keeps those leading carrier segments; the procedure path is its
|
||||
suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
|
||||
and links whose callee was renamed by a minifier are not recognized.
|
||||
|
||||
Each recovered bundle module retains the exact factory-source digest. A complete
|
||||
bounded AST also receives a `babel-ast-v1` structural fingerprint that ignores
|
||||
ordinary identifier names while retaining syntax, literals, operators, object
|
||||
keys, and member properties. If fingerprint construction reaches its structural
|
||||
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
|
||||
prefix is never presented as a complete stable fingerprint.
|
||||
|
||||
## Authority and unknowns
|
||||
|
||||
Artifact bytes and AST syntax are observations. Entrypoint resolution, imports,
|
||||
loads, calls, and persistence relationships are static inferences and explicitly
|
||||
say that syntax does not prove runtime execution. A malformed or unavailable
|
||||
JavaScript file, package record, or source map produces an `unknown` graph scope
|
||||
with `state: unavailable`; it is not treated as evidence that modules or source
|
||||
are absent.
|
||||
|
||||
Source-map contents are parsed as local input. The graph stores original source names and optional content digests, not raw `sourcesContent` text.
|
||||
|
||||
Only explicitly present BrowserWindow values are observations. REA does not
|
||||
substitute version-dependent Electron defaults for omitted `webPreferences`.
|
||||
Unresolved option objects, bridge keys, API objects, and IPC channel expressions
|
||||
remain unknown and make coverage partial.
|
||||
|
||||
Recognized literal module-table bootstraps also support bounded lexical
|
||||
substitution through module exports, import helper bodies, local wrapper calls,
|
||||
returned closures, object spreads, and callback parameters. Recovered Electron
|
||||
paths retain source ranges, module identities, literal channels and branch
|
||||
conditions as static candidates. Unknown spreads, reassigned bindings, accessors,
|
||||
dynamic channels and unsupported syntax do not become exact implementations.
|
||||
This pass neither executes the bundle nor proves that runtime guards pass.
|
||||
An ambiguous or unresolved handler retains its channel registration but does
|
||||
not acquire a path to one arbitrarily selected callback. Unknown factory
|
||||
conditions retain both return branches, including fallthrough. Short-circuit
|
||||
selection distinguishes truthiness from nullishness; uncertain `??` paths
|
||||
retain a nullish guard rather than a falsy guard.
|
||||
|
||||
These paths appear as contextual `javascript-expression` nodes. An exact `api`
|
||||
seed such as `desktop.copy` can trace an exposed member through a unique IPC
|
||||
pairing to an external API call. `addListener` is normalized to `on`. Semantic
|
||||
projection retains these boundary paths before bulk function definitions, within
|
||||
the existing per-file node budget; omitted general semantics remain explicitly
|
||||
partial. Wrapper expansion is bounded by one million visits/substitutions and
|
||||
128 nested substitutions, with exhaustion reported as unknown coverage.
|
||||
|
||||
Electron API identity follows proven lexical module aliases. Namespace interop
|
||||
also preserves identity when the complete Rolldown helper operations, lexical
|
||||
roles, live getters, and captured intrinsic origins match, or when a stable
|
||||
helper simply returns its module parameter. Harmless single-statement control
|
||||
blocks preserve recognition; added operations still require proof. Literal
|
||||
member and destructuring projections are retained from proven results.
|
||||
Omitted or literal node-mode arguments preserve named exports; a module-valued
|
||||
default projection requires a truthy literal node mode. Computed or extra
|
||||
arguments remain unresolved.
|
||||
Recognition uses structure rather than emitted names; other helper bodies and ambiguous
|
||||
runtime fallbacks remain unresolved. Captured function identities survive later global-property
|
||||
replacement, including copies through initialized aliases. Reads before an
|
||||
alias initializer remain unresolved. Visible intrinsic writes before capture or
|
||||
conversion are checked in their owning callable scope, including identifier leaves in nested/default/rest
|
||||
destructuring and loop targets. Property keys and shadowed local bindings do not
|
||||
overwrite the global intrinsic identity. Cross-call side effects, including
|
||||
writes inside directly invoked local functions, are not evaluated. Other
|
||||
reassignments, dynamic selections, and visibly overwritten namespace members
|
||||
are unresolved.
|
||||
Known Object/Reflect mutators remain recognizable through literal destructuring
|
||||
of intrinsic objects and stable bound-function aliases. Calls retain bound
|
||||
argument prefixes and the original bound receiver through `call`, `apply`, and
|
||||
`Reflect.apply`, including generic `Function.prototype` invocation and binding
|
||||
creation. Arguments follow each actual callable invocation rather than property
|
||||
chain depth; rebinding retains the original bound receiver.
|
||||
Visibly replaced invocation intrinsics produce unknown effects instead
|
||||
of using their former call semantics; captured function aliases retain their
|
||||
earlier identity. Unexamined targets remain unresolved. `Reflect.set` examines
|
||||
its explicit receiver as well as its target, retaining unknown receiver effects.
|
||||
Creating an unused binding does not itself mutate an intrinsic. Variable
|
||||
redeclarations retain a single unconditional initializer
|
||||
when they introduce no assignment or second initializer.
|
||||
Stable identifier chains to fresh object, array, function,
|
||||
undecorated class or RegExp syntax keep those targets separate from intrinsic
|
||||
objects when initialized before use; opaque, decorated, reassigned or
|
||||
not-yet-initialized targets remain unresolved. Writes during class initialization
|
||||
are still retained. Function declarations are available before their textual
|
||||
position; variable and class initializers must precede use.
|
||||
Direct and destructuring assignment, loop, update, and delete targets are checked
|
||||
through namespace aliases; unrelated member writes do not discard observed API
|
||||
identity. This static check does
|
||||
not evaluate arbitrary call side effects or runtime registration. Missing findings
|
||||
do not establish that an application has no Electron boundary usage.
|
||||
|
||||
IPC pairing is an inference, not an observation. A renderer `invoke` pairs only
|
||||
with one unique `ipcMain.handle`/`handleOnce` candidate on the same exact literal
|
||||
channel; a renderer send pairs only with one unique `ipcMain.on`/`once`/`addListener`
|
||||
candidate. Dynamic channels are never paired, and multiple compatible handlers
|
||||
are reported as ambiguous without adding a caller-to-handler edge. A matching
|
||||
channel still does not prove registration order, reachability, or runtime use.
|
||||
|
||||
For `.node` bindings, member names mean “requested by JavaScript syntax.” A
|
||||
resolved add-on path does not convert them into verified native exports. Native
|
||||
symbol verification remains a separate deep-analysis claim.
|
||||
|
||||
Endpoint observations preserve useful local diagnostics, including query values
|
||||
and fragments, while removing only URL username/password credentials. Artifact
|
||||
paths, digests, parse locations, and analysis metadata remain actionable
|
||||
because REA is local-only.
|
||||
|
||||
## Safety boundary
|
||||
|
||||
The reconstruction path never uses `eval`, `Function`, `vm.runInContext`, a DOM,
|
||||
bundle `push` handlers, or application bootstrap code. The Webpack/Rspack fixture
|
||||
used by the test suite mutates a global and throws if executed; reconstruction
|
||||
recovers its four module factories without triggering either side effect.
|
||||
|
||||
Directory readers do not follow symlinks. Artifact paths pass through the shared
|
||||
normalizer and collision registry, ASAR entry bytes are rechecked against the
|
||||
inventory digest before parsing, and malformed ASAR operations return typed
|
||||
format diagnostics that retain the local container path. Native add-ons are
|
||||
represented by metadata only.
|
||||
|
||||
## Bounds and coverage
|
||||
|
||||
The reconstruction keeps internal resource safeguards for artifact entries,
|
||||
cumulative artifact bytes, bytes per entry, compression ratio, path depth and
|
||||
length, selected text files, text bytes, AST nodes, and cooperative parse time.
|
||||
These safeguards are not caller-selectable output budgets. Source-map parsing
|
||||
reports truncation when its format or parser safety boundary is reached. The
|
||||
application graph has no aggregate node, edge, root, or observation prefix cap.
|
||||
|
||||
Semantic evaluation retains at most 256 distinct primitive candidates per
|
||||
expression and at most 1 MiB of worst-case JSON string bytes across a normalized
|
||||
primitive value's string candidates. The byte budget estimates six JSON bytes
|
||||
per UTF-16 code unit, the expansion bound for escaped strings, and bounds both
|
||||
retained candidate strings and temporary canonical-key serialization. This is
|
||||
an in-memory semantic allocation bound independent of CLI or MCP transport
|
||||
budgets. Source literal bytes remain in the parsed source and artifact evidence;
|
||||
when a normalized semantic value exceeds the bound, its value is unknown and
|
||||
coverage is partial. Expression
|
||||
evaluation and provenance walks stop after 256 nested levels. Union products
|
||||
and string lengths are checked before allocating combinations or concatenated
|
||||
strings. When a boundary is reached, the value stays unknown, semantic coverage
|
||||
becomes partial, and graph evidence retains the expression location and typed
|
||||
limiting reason. The number of unresolved alternatives is unknown; it is not
|
||||
reported as an exact omission count.
|
||||
|
||||
Byte, entry, path, and graph-shape bounds are hard limits. The parse deadline is
|
||||
checked before and between bounded parsing and traversal phases; the synchronous
|
||||
Babel and JSON parser calls cannot be preempted mid-call, so their input byte
|
||||
bounds remain the hard protection for an individual call.
|
||||
|
||||
Byte-identical assets share a content-digest node. Every distinct observation
|
||||
and inventoried containment edge is retained. If parsing or artifact safety
|
||||
boundaries omit input, coverage reports that omission; when an exact omission
|
||||
count is not knowable, `omitted_count` is `null` rather than a guessed value.
|
||||
|
||||
### Analyzer and result admission
|
||||
|
||||
Application analysis projects semantic relationships one source file at a time.
|
||||
After projection, it retains only the lexical module facts and exported return
|
||||
shapes needed to build application relationships. Full source IR does not stay
|
||||
resident for every file together. Callable ownership and contained-reference
|
||||
queries use per-file range indexes, while call sites use exact-range lookups.
|
||||
Existing node budgets and coverage reporting remain in effect.
|
||||
|
||||
Progress identifies the source file being parsed and projected. Cancellation is
|
||||
checked between files and throughout result transfer. The owned analyzer can
|
||||
be stopped while its synchronous parser or semantic extractor is active;
|
||||
the CLI/MCP process remains available for control requests.
|
||||
|
||||
Graph and Evidence identifiers hash canonical JSON incrementally, without
|
||||
assembling a single string for the whole graph. The canonical bytes and existing
|
||||
identifiers remain unchanged. The opt-in regression check is
|
||||
`npm run verify:javascript:digests`;
|
||||
it hashes a value larger than the running Node engine's single-string limit.
|
||||
|
||||
Both application CLI routes stream complete JSON and JSONL output in bounded
|
||||
chunks, including field filters and `--full-output`. They wait for each stdout
|
||||
write instead of constructing or reparsing a document-sized string. The opt-in
|
||||
`npm run verify:javascript:output` checks the actual CLI formatting boundary with
|
||||
a temporary output file larger than the running engine's string limit, verifies
|
||||
its bytes and digest, and removes it. Use `-- jsonl` for the compact JSONL check.
|
||||
Each check needs space for one output file plus a 1 GiB free-space reserve.
|
||||
|
||||
Other CLI formats and `--token-count` still assemble whole strings.
|
||||
Field selection remains useful when the caller needs a
|
||||
smaller view, for example `--format json --filter-output
|
||||
evidence_id,normalized_result.statistics`. Streaming output does not bound the
|
||||
memory needed to construct the analysis graph itself.
|
||||
|
||||
JavaScript source parsing, static analysis and semantic projection run in one
|
||||
owned analysis process, reused sequentially across files. Target code is never
|
||||
executed. The analyzer defaults to 1024 MiB of V8 old-space and a 300000 ms
|
||||
deadline per source, including startup and fact transfer. Select
|
||||
`max_heap_mb` and `analysis_timeout_ms` through MCP, or `--max-heap-mb` and
|
||||
`--analysis-timeout-ms` on the dedicated CLI command. These choices control the
|
||||
analyzer; the caller's `NODE_OPTIONS` still controls the CLI/MCP process heap.
|
||||
V8 old-space limits do not impose a process RSS limit. When REA runs on Bun,
|
||||
the analyzer ignores the V8 heap flags; resource failures then report
|
||||
`worker_heap_limit_bytes` as null instead of a requested or V8-reported limit.
|
||||
On Windows, the analyzer's owned Job Object instead limits its committed
|
||||
memory to `max_heap_mb` plus a 512 MiB reserve for Bun itself (1536 MiB by
|
||||
default). Reaching it fails the current source with the same memory resource
|
||||
failure and reports `worker_process_memory_limit_bytes`, also a coverage limit;
|
||||
the next source starts a new analyzer. Bun elsewhere enforces no analyzer limit.
|
||||
|
||||
The default semantic capacities are 100000 nodes across the application and
|
||||
20000 per source. Large bundles may explicitly select `max_semantic_nodes` and
|
||||
`max_semantic_nodes_per_file` (CLI: `--max-semantic-nodes` and
|
||||
`--max-semantic-nodes-per-file`), each from 1 to 1000000. The per-file offer is
|
||||
also bounded by the remaining application capacity. Raising these controls does
|
||||
not bypass worker heap, execution deadline or parent result admission. Truncated
|
||||
coverage reports both effective node ceilings; omitted relationships remain
|
||||
unknown.
|
||||
|
||||
While sources remain, the sequential worker reserves one next compact static
|
||||
checkpoint at its 8 MiB bound with the same 6x retention expansion used at
|
||||
admission. Earlier checkpoints are already charged to retained bytes; the
|
||||
remaining inventory does not multiply the reserve. Streamed file transfers keep the default 512 MiB
|
||||
allowance, and can grow with the parent's offered retention budget (8x byte
|
||||
expansion), up to 2 GiB. Individual records remain bounded to 8 MiB. The parent
|
||||
still independently checks actual bytes, record overhead, digests and source
|
||||
identities before adopting results.
|
||||
|
||||
A source that exhausts the analyzer heap does not terminate the CLI/MCP
|
||||
process. Completed static facts are retained, other files can still be analyzed,
|
||||
and unexamined semantic relationships remain explicit unknowns. The parent
|
||||
also checks transfer expansion and accumulated results against its remaining
|
||||
heap before accepting semantic facts. A resource-limited result has partial
|
||||
coverage, the observed limits and an `analyze-javascript-source` unknown scope
|
||||
with the specific failure, including captured diagnostics when available.
|
||||
Resource failures do not increment syntax parse failures. There is no fixed
|
||||
source-length cutoff.
|
||||
|
||||
Static observations also need memory when expanded into nodes, relationships
|
||||
and Evidence. If that expansion does not fit, the file node retains
|
||||
`static_analysis` and any admitted `semantic_module_analysis` observations,
|
||||
including their exact values and source locations. Use the public modules item
|
||||
view to read them. The graph marks those unexpanded relationships as unknown.
|
||||
If cumulative result capacity is exhausted, analysis stops; inventoried but
|
||||
unexamined files remain present with explicit unknown scopes.
|
||||
|
||||
### Large local JSON inputs
|
||||
|
||||
CLI workflows parse larger JSON input files incrementally from one verified
|
||||
regular-file handle, without constructing a document-sized string. Smaller
|
||||
files retain native JSON parsing. Both paths require strict JSON and valid
|
||||
UTF-8, and return one complete value; the assembled object still needs memory.
|
||||
An individual JSON string exceeding the native length limit or the available
|
||||
heap headroom for assembly returns
|
||||
`resource_constraint` with `input_reason: "too-large"`. Supply a smaller valid
|
||||
value; for Evidence workflows, re-analyze a smaller selection of the original
|
||||
target and use its complete Evidence. Splitting JSON text or trimming Evidence
|
||||
fields does not produce a valid workflow input.
|
||||
|
||||
Evidence bundle and analysis snapshot file readers still decode whole
|
||||
documents. Their runtime string-limit failures report a resource constraint
|
||||
with the selected path, observed bytes and UTF-16 limit, rather than malformed
|
||||
JSON.
|
||||
|
||||
## Verification boundary
|
||||
|
||||
The source-owned fixture covers an extracted directory, a direct ASAR with an
|
||||
unpacked native add-on, a direct ASAR whose unpacked native companion is
|
||||
missing, and an ASAR nested beneath a directory. Tests also cover deterministic
|
||||
reruns, parse-not-execute behavior, source-map parsing, global source-map
|
||||
limits, malformed structured data, invalid containers, traversal, symlink
|
||||
escape, oversized text, cancellation, AST truncation, and repeated content
|
||||
identities. A separate synthetic Electron fixture covers explicit safe and
|
||||
unsafe preference values, preload and contextBridge surfaces, literal, dynamic,
|
||||
paired, ambiguous, and unpaired IPC, validation candidates, utility processes,
|
||||
and native binding requests. These fixtures establish parser and artifact-reader
|
||||
claims; they do not replace the later operator-supplied real-application
|
||||
benchmark.
|
||||
|
||||
URI schemes are classified independently of local file names: a reference such
|
||||
as `web3:app.js` remains external even when an artifact has that literal name.
|
||||
Scheme characters may include digits after the initial letter; a relative path
|
||||
such as `./web3:app.js` still names a local artifact.
|
||||
|
||||
Package `exports` fallback arrays are supported both at the top level and under
|
||||
the root `"."` entry. The resolver selects targets in declared order using the
|
||||
same conditional and invalid-entry handling as nested exports arrays.
|
||||
Nonempty arrays containing only inactive conditions remain unmatched, allowing
|
||||
the enclosing condition object to continue to a later active condition.
|
||||
Empty arrays and explicit null targets keep their blocking outcome.
|
||||
Refusal evidence distinguishes those source values and retains condition names
|
||||
from unmatched nested targets rather than reporting array indices.
|
||||
For imports and requires, the active Node conditions include `module-sync`;
|
||||
earlier active conditions retain precedence over later ones.
|
||||
|
||||
Package self-references use the importer's enclosing package scope before
|
||||
installed dependencies. ESM requires the parsed package name to equal the
|
||||
metadata name; a scope-only name such as `@scope` cannot capture `@scope/pkg`.
|
||||
CommonJS retains Node's raw-request name/prefix matching. Both paths preserve
|
||||
the original specifier and select exports under their own loader conditions.
|
||||
|
||||
Exact package-private `#alias` specifiers resolve against the importer's
|
||||
enclosing `package.json` `imports` map — the nearest same-container package
|
||||
scope, stopping at a `node_modules` directory so a nested package cannot inherit
|
||||
an ancestor's aliases. Selected targets are package-local `./` files with no
|
||||
extension or index fallback, using the same condition, array, and null selection
|
||||
as exports subpaths. A missing key, explicit null, or out-of-scope alias keeps
|
||||
an unresolved edge with that reason. Wildcard keys (`#internal/*`), recursive
|
||||
`#` targets, and external-package import targets remain unresolved.
|
||||
|
||||
Exact keys such as `#/x` are literal lookup identities. `#` and names ending
|
||||
in `/` are invalid. Imports arrays skip invalid parent-relative, absolute, and
|
||||
URL targets before selecting a later package-local file; an unsupported bare
|
||||
package or recursive alias keeps its unresolved outcome rather than selecting
|
||||
a later fallback.
|
||||
|
||||
ESM relative module paths, legacy package subpaths without exports, and selected
|
||||
package exports targets use URL suffix handling and one percent-decoding pass.
|
||||
Bare specifiers retain their complete query/fragment text for exports key matching.
|
||||
For dependency lookup, ESM constructs a package metadata URL instead of treating
|
||||
the parsed package name as a literal directory. A root request such as `legacy?x`
|
||||
does not load either `node_modules/legacy?x` or `node_modules/legacy`; an exact
|
||||
self-reference with that metadata name can still select its exports first.
|
||||
Without an exports map, ESM subpaths select exact files; directory imports and
|
||||
extension fallback are refused. CommonJS relative paths and legacy package main
|
||||
fields retain literal filename punctuation. Exports targets must start with
|
||||
`./` and exclude `.`, `..`, and `node_modules` path segments, including their
|
||||
encoded forms, before URL normalization. Arrays skip invalid exports targets
|
||||
in declared order, including URLs that escape the package root after normalization.
|
||||
Selected package targets use the normalized URL path for lookup; tabs and
|
||||
newlines in the URL text are removed by the URL parser. CommonJS package targets
|
||||
reject encoded separators anywhere in the selected URL; ESM checks its pathname.
|
||||
URL decoding and file lookup failures for a selected target do not select a
|
||||
later entry. Rejected references retain the package metadata
|
||||
path, original target, failed constraint, and importing source location.
|
||||
|
||||
HTML script references resolve to exact inventoried files after applying the
|
||||
document base and query/fragment rules. As in a browser, the script URL and base
|
||||
href are read without surrounding whitespace or embedded tabs and newlines, and
|
||||
their percent-encoded path bytes are decoded; encoded dot and separator bytes
|
||||
are still rejected. CommonJS module lookups retain extension and directory
|
||||
resolution.
|
||||
Unresolved HTML references retain their declaration, source range, and resolution
|
||||
reason in the renderer observations.
|
||||
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
|
||||
line endings, preserving UTF-16 columns.
|
||||
@@ -0,0 +1,141 @@
|
||||
# JavaScript source recovery
|
||||
|
||||
`recover_javascript_sources` / `recover-javascript-sources` derives readable
|
||||
modules from explicitly selected UTF-8 JavaScript files or bundles. Use it on
|
||||
files returned by `export_web_scripts` or other local scripts. The operation
|
||||
never executes the selected or recovered application code.
|
||||
|
||||
## Caller-supplied engine
|
||||
|
||||
REA accepts Wakaru **^1.13.0** on Linux x64. The report parser is verified with
|
||||
Wakaru **1.14.0**. Another release in that range is accepted, the observed
|
||||
version is reported, and a limitation records that the parser was not verified
|
||||
against that release. Wakaru changes `--version`, `--json` and `provenance.json`
|
||||
only by addition within a major version
|
||||
([compatibility policy](https://github.com/pionxzh/wakaru/blob/v1.14.0/docs/cli.md#machine-readable-output-compatibility)),
|
||||
so the parser keeps unknown report fields and reported module statuses.
|
||||
|
||||
Set `REA_WAKARU_COMMAND` to the absolute path of an executable supplied by the
|
||||
caller. The adapter uses util-linux `prlimit` (default `/usr/bin/prlimit`;
|
||||
override with an absolute `REA_JAVASCRIPT_PRLIMIT_COMMAND`). REA does not install
|
||||
either tool. Engine configuration is checked only when recovery is requested;
|
||||
other REA capabilities remain available without it.
|
||||
|
||||
Unmodified upstream source is retained in `third_party/wakaru`, pinned to
|
||||
`8219bf5016a063d7fb215101a8c45d5959ec733f` (v1.14.0, Apache-2.0).
|
||||
The result reports the executed binary's exact path, SHA-256 and observed version.
|
||||
The audited source revision is distinct from `executed_source_revision: null`:
|
||||
a version string cannot prove which source produced a caller-supplied binary.
|
||||
|
||||
### Install Wakaru
|
||||
|
||||
Use the official release binary, the same one CI verifies:
|
||||
|
||||
```sh
|
||||
curl -fL https://github.com/pionxzh/wakaru/releases/download/v1.14.0/wakaru-linux-x64.tar.gz | tar -xz wakaru
|
||||
export REA_WAKARU_COMMAND="$PWD/wakaru"
|
||||
```
|
||||
|
||||
Or install it from npm. Point at the platform binary, not the Node.js shim in
|
||||
`node_modules/.bin`:
|
||||
|
||||
```sh
|
||||
npm install wakaru@1.14.0
|
||||
export REA_WAKARU_COMMAND="$(realpath node_modules/@wakaru/cli-linux-x64/wakaru)"
|
||||
```
|
||||
|
||||
Both give the same binary (SHA-256
|
||||
`664fa06782a61eda7ed15885134b52525fd565dd77607e642062ceb6cd927c38`).
|
||||
|
||||
## Recover and analyze
|
||||
|
||||
```sh
|
||||
rea recover-javascript-sources /tmp/web-scripts/app.js /tmp/recovered-app --json
|
||||
rea analyze-javascript-application /tmp/recovered-app/modules --json
|
||||
|
||||
# Recover related code-split chunks together; repeat --additional-path.
|
||||
rea recover-javascript-sources /tmp/web-scripts/main.js /tmp/recovered-chunks \
|
||||
--additional-path /tmp/web-scripts/runtime.js \
|
||||
--additional-path /tmp/web-scripts/shared.js \
|
||||
--additional-path /tmp/web-scripts/checkout.chunk.js --json
|
||||
```
|
||||
|
||||
MCP input:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/tmp/web-scripts/main.js",
|
||||
"additional_paths": [
|
||||
"/tmp/web-scripts/runtime.js",
|
||||
"/tmp/web-scripts/shared.js",
|
||||
"/tmp/web-scripts/checkout.chunk.js"
|
||||
],
|
||||
"output_directory": "/tmp/recovered-chunks"
|
||||
}
|
||||
```
|
||||
|
||||
The output directory must be absent, absolute, and have an existing parent.
|
||||
Each source is snapshotted separately in a private workspace. Multiple inputs
|
||||
retain their relative directory layout and are passed together to one Wakaru
|
||||
invocation, allowing cross-chunk module IDs to resolve. Inputs must be distinct;
|
||||
`additional_paths` accepts up to 16 additional files. The engine writes only to
|
||||
its own staging directory. Reported module paths, input identities, UTF-8 byte
|
||||
ranges, regular files and emitted map representations are validated before
|
||||
publication. REA writes verified bytes to an exclusively owned output tree and
|
||||
rolls it back on failure. It also checks that the original source and engine
|
||||
bytes remain unchanged before committing. The existing output is never overwritten.
|
||||
|
||||
Multi-input relative filenames must be representable unchanged as artifact paths:
|
||||
NFC Unicode spelling and no literal backslashes. Unsupported layouts fail before
|
||||
invoking the engine rather than silently renaming inputs referenced by source maps.
|
||||
|
||||
Published contents:
|
||||
|
||||
- `inputs/bundle.js`: exact original snapshot for a single input. Multiple inputs
|
||||
retain paths relative to their common parent under `inputs/`.
|
||||
- `modules/`: derived modules and any upstream-emitted source maps.
|
||||
- `reports/stdout.json` and `reports/provenance.json`: exact producer reports.
|
||||
- `manifest.json`: identities, modules, transform choices, warnings and limitations.
|
||||
|
||||
The inline result includes complete file paths/digests/sizes, extraction ranges,
|
||||
warnings and `analysis_input`, directly usable as the arguments to
|
||||
`analyze_javascript_application`. `source` identifies the primary input and binds
|
||||
the Evidence subject; `additional_sources` records the other snapshots and their
|
||||
published copies when selected. Each module names its owning snapshot, original
|
||||
path and original digest; all extraction and context ranges refer to that input.
|
||||
Raw Evidence retains selected command arguments,
|
||||
reported input paths, exit status, stderr and reports; temporary paths are recorded
|
||||
as historical execution coordinates after their workspace is removed.
|
||||
|
||||
## Transform choices and interpretation
|
||||
|
||||
`extraction_mode` defaults to `structural` (Wakaru `--unpack=strict`). `heuristic`
|
||||
uses `--unpack=auto`; `inspection` uses `--unpack=inspect` and can return finer,
|
||||
non-executable regions. `rewrite_level` accepts `minimal`, `standard` (default)
|
||||
and `aggressive`. Maps and provenance are requested in every operation.
|
||||
|
||||
Structural extraction can fall back to a single file. In the real fixtures,
|
||||
webpack 5 yields two modules; the esbuild scope-hoisted fixture remains one
|
||||
unknown-format script. Returned module counts and detected formats report those
|
||||
actual outcomes. A valid report with failed transformations returns `partial`
|
||||
with original warnings and files. An unusable report, inconsistent provenance,
|
||||
escaping path, symlink or missing module is an error, with owned cleanup.
|
||||
|
||||
Evidence confidence is `derived`. Extraction ranges are half-open **UTF-8 byte
|
||||
offsets in the original snapshot**, rather than rewritten line positions or a
|
||||
coverage assertion. Source maps are preserved without claiming independently
|
||||
verified mapping accuracy. Original variable names and arbitrary runtime
|
||||
equivalence remain unknown. No network fetching occurs.
|
||||
|
||||
## Resource policy and verification
|
||||
|
||||
Each operation uses one worker, a 1 GiB address-space ceiling and a 120-second
|
||||
deadline; it accepts up to 64 MiB of total input across all selected files and
|
||||
128 MiB/10000 entries of staged
|
||||
output, with 8 MiB per diagnostic/report stream. Exceeding a limit is an explicit
|
||||
failure with rollback. These limits apply per operation and do not claim aggregate
|
||||
host containment across concurrent callers.
|
||||
|
||||
See [testing.md](../../development/testing.md#javascript-source-recovery) for the focused real
|
||||
Wakaru CLI/MCP lane and its source-owned compiler fixtures. Other host platforms
|
||||
and Bun/Vue recovery are not claimed by this adapter.
|
||||
@@ -0,0 +1,152 @@
|
||||
# Passive Node and Electron runtime observation
|
||||
|
||||
REA can attach to an already-running Node.js or Electron V8 Inspector target
|
||||
and retain script-load and execution-context metadata. The
|
||||
`list_javascript_runtime_targets` and `observe_javascript_runtime` MCP tools
|
||||
have equivalent `rea list-javascript-runtime-targets` and
|
||||
`rea observe-javascript-runtime` commands. Successful calls return
|
||||
deterministic Evidence.
|
||||
|
||||
Each request supplies the loopback Inspector endpoint and, for observation, the
|
||||
target ID. This surface is separate from browser CDP, Electron file-page
|
||||
inspection, and Process Capture:
|
||||
|
||||
```bash
|
||||
rea list-javascript-runtime-targets http://127.0.0.1:9229 \
|
||||
--json
|
||||
rea observe-javascript-runtime http://127.0.0.1:9229 TARGET_ID \
|
||||
--runtime-kind node --json
|
||||
```
|
||||
|
||||
## Start a caller-owned Node target
|
||||
|
||||
Run your selected application with Inspector bound to loopback:
|
||||
|
||||
```bash
|
||||
node --inspect=127.0.0.1:9229 ./app.mjs
|
||||
```
|
||||
|
||||
Keep it running while listing targets and observing the selected ID. Use a free
|
||||
port and the same literal endpoint in both REA calls. Stop the application
|
||||
through its normal lifecycle when finished; REA disconnects but does not own or
|
||||
terminate it. Node.js must already be installed.
|
||||
|
||||
Use `--inspect` for this passive workflow. `--inspect-brk` pauses before the
|
||||
application starts, and REA never resumes it or sends
|
||||
`Runtime.runIfWaitingForDebugger`. Resume a paused target with a separate
|
||||
caller-owned debugger before requesting application runtime observations.
|
||||
|
||||
The caller supplies the literal-loopback HTTP endpoint directly. Selecting an endpoint exposes all targets it serves and their local script locations or renderer origins. A target must match the exact requested target ID and expose a same-port WebSocket reported by that endpoint. Targets explicitly marked attached are rejected rather than displacing another debugger. Node's `/json/list` omits attachment state, so REA preserves it as `null` (unknown), reports that limitation, and permits observation unless the endpoint explicitly reports `attached: true`. Local paths are canonicalized after symlink resolution.
|
||||
|
||||
The `attached` field in JavaScript runtime and browser target results is nullable: `true` and `false` are reported only when discovery supplied a boolean; `null` means the endpoint omitted or could not report the state. CDP `TargetInfo` worker records require an `attached` boolean; REA excludes a worker record with a missing or malformed value as an invalid protocol value.
|
||||
|
||||
`list_javascript_runtime_targets` returns every available target in one inline
|
||||
array.
|
||||
|
||||
For the MCP follow-up call, pass the same literal-loopback endpoint and the selected target ID to `observe_javascript_runtime`. REA rediscovers targets and validates the ID against the live endpoint before observation.
|
||||
|
||||
### Discovery file locations
|
||||
|
||||
Node's `/json/list` reports a best-effort pathname prefixed with `file://`,
|
||||
without URI encoding. On POSIX, REA interprets unambiguous paths literally,
|
||||
including `#` and `%` in filenames. Node also replaces backslashes and double
|
||||
quotes with underscores. That transformation cannot be reversed reliably:
|
||||
an underscore may already be part of the original filename.
|
||||
|
||||
Windows paths, ambiguous underscore paths, and local discovery paths that
|
||||
cannot be verified remain eligible by their endpoint and exact target ID.
|
||||
Their target location preserves the reported value:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "unresolved",
|
||||
"reported_url": "file://C:_tools_entry.js",
|
||||
"reason": "unverifiable-file-location"
|
||||
}
|
||||
```
|
||||
|
||||
This location does not claim that a file exists or authorize reading its
|
||||
contents. Listing remains HTTP-only. Observation resolves loaded script
|
||||
locations independently from `Debugger.scriptParsed` file URLs, preserving
|
||||
the existing checks for local files and encoded separators. Unsupported
|
||||
schemes, remote file hosts, and a bare `file://` remain excluded. These Node
|
||||
discovery rules do not apply to page targets or other runtime products.
|
||||
|
||||
Reconciliation retains an unresolved `target_location` object and reports
|
||||
`runtime-location-unresolved` with unknown confidence for that target. It
|
||||
does not derive a static match from the reported pathname; verified loaded
|
||||
script locations remain available for independent matching.
|
||||
|
||||
## Passive protocol boundary
|
||||
|
||||
The provider sends exactly two protocol commands:
|
||||
|
||||
- `Runtime.enable`
|
||||
- `Debugger.enable`
|
||||
|
||||
It never sends `Runtime.evaluate`, reads script source, installs breakpoints,
|
||||
pauses or resumes execution, calls `Runtime.runIfWaitingForDebugger`, launches
|
||||
the target, instruments JavaScript, or invokes Electron APIs. Closing a call
|
||||
closes only REA's WebSocket.
|
||||
|
||||
The direct observations are:
|
||||
|
||||
- `Debugger.scriptParsed`, establishing that a bounded script location was
|
||||
parsed in the observed execution context;
|
||||
- `Debugger.scriptFailedToParse`, retained only as a bounded invalid-script
|
||||
count;
|
||||
- `Runtime.executionContextCreated`,
|
||||
`Runtime.executionContextDestroyed`, and
|
||||
`Runtime.executionContextsCleared`, establishing context lifecycle within
|
||||
the capture window.
|
||||
|
||||
The Inspector protocol does not directly expose require/import caller edges,
|
||||
EventEmitter emissions or listener invocation, Electron IPC messages or
|
||||
handlers, or script-unload events. Those facts remain explicit unknowns.
|
||||
`scriptParsed` proves script presence, not the importing module, feature
|
||||
execution, initialization order, or causality.
|
||||
|
||||
## Node and Electron roles
|
||||
|
||||
The caller declares one role: `node`, `electron-main`, `electron-preload`, or
|
||||
`electron-renderer`. Node and Electron-main declarations require a protocol
|
||||
target of type `node`; preload and renderer declarations require `page`.
|
||||
Inspector does not authenticate the operating-system PID or distinguish an
|
||||
Electron role, so Evidence records the role authority as
|
||||
`caller-declared-unverified`.
|
||||
|
||||
Electron main, preload, and renderer behavior can therefore be observed only
|
||||
as separate Inspector targets. The provider does not infer that two
|
||||
targets belong to the same Electron application.
|
||||
|
||||
## Capture and determinism
|
||||
|
||||
Capture retains valid script and execution-context metadata within an 8 MiB
|
||||
aggregate budget, including conservative estimates for UTF-16 strings, record
|
||||
and collection overhead, and projected URL fields. Each CDP message also has a
|
||||
16 MiB transport limit. A message over that limit fails with `payload_limit`;
|
||||
when the aggregate budget is reached, later metadata events are dropped and
|
||||
the partial result reports `events_dropped`, `truncated: true`, and
|
||||
`retained_metadata_budget_exceeded`. Repeated context transitions update the
|
||||
current context record instead of accumulating duplicate objects. Scripts are
|
||||
validated after capture, deduplicated by stable metadata, and canonically
|
||||
sorted. Wall-clock timestamps and protocol script IDs are excluded from the
|
||||
durable result, so identical inputs and captured metadata produce the same
|
||||
Evidence ID.
|
||||
|
||||
Inspector attach is inherently incomplete. Enabling `Debugger` reports known
|
||||
and uncollected scripts, but scripts collected before attachment may be
|
||||
missing. Absence from the observation window never proves that a script or
|
||||
behavior does not occur.
|
||||
|
||||
## Static correlation
|
||||
|
||||
Pass the resulting observation Evidence and an
|
||||
`analyze_javascript_application` Evidence record to
|
||||
`reconcile_javascript_runtime`. The reconciliation accepts this provider
|
||||
alongside passive web and Electron page captures. Exact file or URL mappings
|
||||
supplied with the request can correlate script presence with JavaScript Application Graph
|
||||
assets. Because this provider never reads source bytes, matches normally use a
|
||||
unique authorized location and remain weaker than captured-byte identity.
|
||||
Builtin `node:` scripts remain runtime-only nodes unless a future explicit
|
||||
static authority models them.
|
||||
@@ -0,0 +1,172 @@
|
||||
# JavaScript static/runtime reconciliation
|
||||
|
||||
REA can combine an existing static JavaScript Application Graph with one or
|
||||
more passive browser, Electron page, or Node/Electron V8 Inspector captures without collapsing their different
|
||||
authorities. The `reconcile_javascript_runtime` MCP tool and
|
||||
`rea reconcile-javascript-runtime` CLI command accept Evidence records and
|
||||
return derived Evidence containing a combined JavaScript Application Graph
|
||||
and explicit match, ambiguity, mismatch, and unknown classifications.
|
||||
|
||||
The operation performs no filesystem, browser, or Electron I/O. It accepts only
|
||||
semantically verified Evidence produced by `analyze_javascript_application`,
|
||||
`inspect_web_page`, `inspect_electron_page`, or
|
||||
`observe_javascript_runtime`; matching operation names alone
|
||||
is insufficient. Provider identity, predicate type, subject digest, target
|
||||
parameters, and normalized-result schemas are checked before reconciliation.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Run `analyze_javascript_application` against exactly one application layer.
|
||||
Optional cache or assets directories can be analyzed independently and
|
||||
supplied as additional layers.
|
||||
2. Run `inspect_web_page`, `inspect_electron_page`, or the attach-only
|
||||
`observe_javascript_runtime`. Capturing script sources
|
||||
is optional, but a source capture provides the strongest byte
|
||||
identity.
|
||||
3. Pass the resulting Evidence records to `reconcile_javascript_runtime`.
|
||||
Browser URLs normally need an explicit URL-prefix mapping. Extracted
|
||||
Electron and V8 Inspector file locations are mapped automatically when runtime files remain
|
||||
beneath the analyzed input root; relocated files, ASAR projections, caches,
|
||||
and asset roots need explicit mappings.
|
||||
|
||||
The MCP input has this shape; the `analysis` and `runtime_observations` values
|
||||
are complete Evidence objects, not only evidence IDs:
|
||||
|
||||
```json
|
||||
{
|
||||
"static_layers": [
|
||||
{
|
||||
"role": "application",
|
||||
"analysis": "ANALYZE_JAVASCRIPT_APPLICATION_EVIDENCE",
|
||||
"runtime_mappings": [
|
||||
{
|
||||
"kind": "url-prefix",
|
||||
"prefix": "https://example.test/assets/",
|
||||
"artifact_prefix": "dist"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"runtime_observations": ["INSPECT_WEB_PAGE_EVIDENCE"]
|
||||
}
|
||||
```
|
||||
|
||||
`static_layers` requires exactly one `application` role and permits additional
|
||||
`cache` and `assets` roles. A `file-root` mapping translates a canonical runtime
|
||||
filesystem root to an artifact-relative prefix. A `url-prefix` mapping does the
|
||||
same for one exact HTTP(S) origin and path prefix. Mappings cannot add CDP
|
||||
origins, broaden Electron roots, read files, or authorize source-map access;
|
||||
they are caller-declared inference inputs applied only to already-authorized
|
||||
locations.
|
||||
|
||||
The one-shot CLI accepts either a JSON file path or the same JSON inline. A
|
||||
file avoids the operating system's command-line size limit for realistic
|
||||
Evidence records:
|
||||
|
||||
```bash
|
||||
rea reconcile-javascript-runtime reconciliation-input.json --json
|
||||
```
|
||||
|
||||
For a repeatable local check from operator-provided paths, build first and run:
|
||||
|
||||
```bash
|
||||
npm run verify:javascript-runtime -- \
|
||||
--application /absolute/path/to/app \
|
||||
--cache /absolute/path/to/cache \
|
||||
--runtime-evidence /absolute/path/to/electron-evidence.json \
|
||||
--mapping '{"layer":1,"kind":"file-root","root":"/runtime/cache","artifact_prefix":""}'
|
||||
```
|
||||
|
||||
The verifier analyzes the supplied static paths, reads bounded Evidence JSON
|
||||
files, and prints graph IDs, evidence IDs, paths, digests, counts, reasons, and
|
||||
coverage. It does not print captured source text.
|
||||
|
||||
## Matching rules
|
||||
|
||||
Runtime targets, frames, scripts, and workers become capture-scoped graph
|
||||
nodes. REA evaluates compatible static candidates conservatively:
|
||||
|
||||
1. A captured-source SHA-256 plus one matching mapped location yields
|
||||
a content-and-location match.
|
||||
2. One unique exact content digest can match even when no location mapping is
|
||||
available. Whole-asset digests and reconstructed module-factory source
|
||||
digests remain distinguishable in the reported basis.
|
||||
3. Without captured bytes, one unique mapped artifact path can produce a
|
||||
medium-confidence location match.
|
||||
4. Multiple candidates remain `ambiguous`; no `observed_as` edge is emitted.
|
||||
`candidate_static_count` reports the full qualified layer/node count, and
|
||||
`candidate_static_nodes` includes every unique candidate reference.
|
||||
5. If captured bytes disagree with the static candidate at the mapped path,
|
||||
the result is an explicit mismatch. A path never overrides a digest
|
||||
disagreement.
|
||||
6. Missing authorization mappings and incomplete or truncated input coverage
|
||||
remain unknown or truncated rather than becoming absence claims.
|
||||
|
||||
Every successful cross-layer `observed_as` edge has
|
||||
`authority: cross-layer-reconciliation` and `state: inferred`. The underlying
|
||||
artifact/AST observations and passive-CDP observations remain unchanged and
|
||||
retain their own Evidence links.
|
||||
|
||||
## Loaded, resident, and not observed
|
||||
|
||||
Static JavaScript assets, chunks, and modules receive a separate load-state
|
||||
classification:
|
||||
|
||||
- `loaded` means an accepted runtime script correspondence was found, using
|
||||
exact bytes or one unique mapped location;
|
||||
- `resident-in-loaded-asset` means a containing asset was observed, but the
|
||||
embedded module was not independently observed or shown to execute;
|
||||
- `not-observed-in-capture` means the entity did not appear in the complete
|
||||
authorized captures supplied to this call and its artifact path falls beneath
|
||||
an exercised automatic or operator-declared mapping prefix;
|
||||
- `unknown` covers incomplete captures, layers outside runtime scope, and
|
||||
unresolved correspondence. Nodes outside a mapped artifact prefix stay
|
||||
unknown even when another prefix in the same static layer was observed.
|
||||
|
||||
None of these states proves feature execution, initialization order,
|
||||
reachability, or causality. In particular, `not-observed-in-capture` never means
|
||||
globally unloaded.
|
||||
|
||||
## Frames, workers, and source maps
|
||||
|
||||
Passive inspection retains the execution-context frame ID supplied by CDP
|
||||
for each accepted script. Electron captures also inventory authorized worker,
|
||||
service-worker, and shared-worker targets, including bounded opener-target and
|
||||
parent-frame relationships. These fields improve attribution while preserving
|
||||
the original exact-origin or canonical-root filter.
|
||||
|
||||
V8 Inspector observations expose execution-context IDs but no authenticated
|
||||
frame relationship, so their scripts attach to the exact runtime target in the
|
||||
combined graph. They never supply source bytes, require/import caller edges,
|
||||
EventEmitter activity, Electron IPC, or script-unload evidence. A unique
|
||||
authorized file or URL location can still correlate script presence with a
|
||||
static asset, but it does not prove module initialization or feature execution.
|
||||
|
||||
Source-map declarations and original-source reads remain static evidence. They
|
||||
are reported in `source_map_authority` but never used
|
||||
as primary runtime byte matches. A source name or `sourcesContent` digest is not
|
||||
silently equated with the generated bytes observed by CDP.
|
||||
|
||||
## Bounds and determinism
|
||||
|
||||
The workflow does not impose aggregate count caps on static layers, passive
|
||||
captures, runtime entities, reconciliation items, static load states, runtime
|
||||
mapping declarations, or candidate references. These collections remain
|
||||
subject to the structural validation applied to each Evidence record and
|
||||
application graph; passive observation returns all in-window collection items
|
||||
that pass per-value validation. Reconciliation reports zero
|
||||
omitted items when it projects the complete supplied inputs. Static JavaScript
|
||||
analysis can still report its own bounded AST coverage as partial or truncated.
|
||||
|
||||
Layers, captures, nodes, edges, matches, and evidence links are canonically
|
||||
ordered. Repeating the same verified Evidence and mappings produces the same
|
||||
reconciliation, graph, and Evidence identifiers.
|
||||
|
||||
## Non-goals
|
||||
|
||||
Reconciliation does not attach to CDP, fetch scripts or source maps, execute
|
||||
JavaScript, invoke Electron IPC, inspect native add-ons, infer arbitrary URL
|
||||
rewrites, or search the filesystem. It does not replace the older
|
||||
`correlate_static_and_runtime` investigation tool: that tool records explicit
|
||||
cross-version static/runtime comparison hypotheses, while this operation maps
|
||||
JavaScript runtime instances to static application-graph entities.
|
||||
Reference in New Issue
Block a user