chore: integrate canonical Evidence changes for MCP error delivery

This commit is contained in:
N0zoM1z0
2026-10-09 12:43:17 +00:00
643 changed files with 12727 additions and 10243 deletions
@@ -252,7 +252,7 @@ No request can cross a concurrent target or provider transition.
canonical path, source digest, generic kind and format, selected architecture,
and available architectures. Hopper loader arguments leave the domain model.
Provider-owned databases use a generic analysis-database classification; their
adapter determines support. A legacy `.hop` target is therefore accepted only
adapter determines support. A `.hop` analysis database is therefore accepted only
by Hopper without teaching the domain how to open it. Ghidra v1 continues to use
private temporary projects and does not open user projects.
@@ -336,19 +336,19 @@ entries, so snapshots must be recaptured before they can be used as a cache.
An embedded Evidence bundle can still be imported separately through the
Evidence import flow; it is never promoted into an analysis cache.
### 8. Add profile commitments to new deep Evidence while preserving legacy records
### 8. Commit concrete or explicitly unknown analysis profiles in Evidence
The optional `analysis_profile` field uses the canonical envelope above. Every
newly created deep-analysis observation must include it, and its provider ID
and version must match the record's provider. The field participates in the
Evidence semantic ID.
Every Evidence record includes `analysis_profile`, either the canonical profile
envelope above or `null` when no profile was observed. Deep-analysis observations
commit the resolved provider ID, version, and effective configuration in this
profile. The field participates in the Evidence semantic ID.
Existing Evidence records without `analysis_profile` remain valid and keep
their existing IDs. Absence on a legacy deep record means the analysis profile
is unknown; REA does not infer or enrich it during import. Such evidence can be
displayed and cited but cannot establish profile compatibility, seed snapshot
replay, or support an "unchanged" conclusion that depends on identical analysis
semantics.
An explicitly unknown profile can be displayed and cited, but cannot establish
profile compatibility, seed snapshot replay, or support an "unchanged"
conclusion that depends on identical analysis semantics. Snapshot query entries
must have matching profile-bound Evidence; imports reject missing or mismatched
observations instead of silently dropping query entries. Both `entries` and
`workflow_entries` are required arrays, including when empty.
Different providers or profile digests are never snapshot-compatible. A
derived comparison may compare cross-provider normalized facts only when its
@@ -407,12 +407,12 @@ means no dynamic provider has supplied a snapshot. Once one or more start,
snapshots are not live inventories, and a point-in-time observation never proves
historical absence.
The existing flat `provider`, `providers`, and `capabilities` status fields stay
during the 1.x compatibility window. Their effective deep-operation entries
come only from the active binding; target-free ambiguity is unavailable rather
than arbitrarily collapsed. The new candidate fields are authoritative for
selection. A future major release may remove the synthetic composite provider
field after callers have migrated.
Status exposes `analysis_provider_candidates` for configured deep providers and
`analysis_provider_binding` for the selected concrete provider and profile.
`capabilities` retains effective operation availability and exact provider
provenance, including auxiliary operation families. There are no synthetic
flat `provider` or `providers` aliases. Provider prompt completion uses the
actual selectable candidates.
Selection failures extend the tagged `ProviderSelectionError` with a stable
reason:
@@ -448,26 +448,20 @@ snapshot's provider.
- Tool names and normalized output contracts remain provider-neutral. REA does
not add `ghidra_decompile` or `hopper_decompile` tools.
## Compatibility and migration
## Provider and snapshot representations
| Existing behavior or data | Decision |
| ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| No provider selector, only Hopper usable | `auto` selects the sole candidate; behavior remains equivalent |
| No deep provider usable | Automatic open may remain unbound for auxiliary operations; deep capabilities explain why they are unavailable |
| Hopper and Ghidra both usable, no preference | Open fails with `ambiguous`; installation order never becomes policy |
| Existing Hopper environment settings | Remain supported as Hopper adapter configuration |
| `open_binary` and CLI structured `loaderArgs` output | Retained as a deprecated Hopper-only compatibility projection for 1.x; the analysis profile is authoritative |
| `BinaryTarget.loaderArgs` | Removed; Hopper derives them inside its adapter and commits normalized semantics in the profile |
| Legacy `.hop` input | Classified generically as an analysis database and accepted only by Hopper |
| Snapshot v1 | Not accepted as the current snapshot format; recapture before cache replay |
| Existing Evidence without a profile | Accepted unchanged; profile compatibility is unknown |
| New deep-analysis Evidence | Must include `analysis_profile`; its semantic ID commits the field |
| Existing status fields | Retained for 1.x with additive binding and candidate fields |
| Provider failure after selection | Returned from the selected provider; no transparent retry through another engine |
The deprecated `loaderArgs` response must never be read to select a provider or
validate a snapshot. It is empty for non-Hopper bindings and may be removed only
with a separately documented major contract change.
| Behavior or data | Contract |
| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| No selector, one usable deep provider | `auto` selects the sole candidate |
| No usable deep provider | Automatic open may remain unbound for auxiliary operations; deep capabilities explain their unavailability |
| Several usable deep providers without a preference | Open fails with `ambiguous`; installation order never becomes policy |
| Hopper environment settings | Adapter configuration committed into the typed analysis profile, including effective loader arguments |
| `open_binary` output | Provider-neutral target identity only; no `loaderArgs` projection |
| `.hop` input | Analysis database accepted only by Hopper |
| Snapshots | Concrete provider/profile binding, required primitive and workflow entry arrays, and matching authenticated Evidence |
| Evidence without an observed profile | Explicit `analysis_profile: null`; ineligible for profile-dependent replay |
| Snapshot save receipt | `primitive_entries`, `workflow_entries`, and `evidence_records` counts without a duplicate `entries` alias |
| Provider failure after selection | Returned from that provider, without transparent retry through another engine |
## Required behavior scenarios
+32 -1
View File
@@ -23,6 +23,23 @@ export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
export JAVA_HOME=/absolute/path/existing-jdk
```
On Windows x64, use a REA package with its matching bundled Windows native
controls **including owned protocol stdin support**. This Windows boundary is
new on repository main; an older released package can still reject this host.
PowerShell accepts paths containing spaces without additional escaping
inside these environment values:
```powershell
$env:REA_JADX_MCP_JAR = 'D:\tools\jadx-headless-mcp-0.7.1-all.jar'
$env:JAVA_HOME = 'D:\tools\jdk-21'
rea inspect-android-package 'D:\targets\Example.apk'
```
REA selects `JAVA_HOME\bin\java.exe`, or resolves `java.exe` on the selected
Windows `PATH`. A missing or mismatched native artifact reports the package
prerequisite rather than attempting an unowned Java launch. Windows arm64 is
outside this provider's current native boundary.
Readiness checks the selected JAR's ZIP directory for the classes consumed by
REA's metadata bridge, as well as Java's compiler and version modules. It does
not load an APK or execute engine code. Invalid archives and missing bridge
@@ -34,7 +51,9 @@ the exhausted read budget and the selected JAR path.
The current metadata bridge is verified on macOS arm64 with OpenJDK 21 and the
public Appium ApiDemos fixture. The POSIX adapter also supports Linux; the new
bridge has not yet undergone real verification on Linux/JDK 25.
Windows is unsupported for this provider's owned stdio process boundary.
Windows x64 uses an owned Job Object with bidirectional stdio, atomic job
assignment and kill-on-owner-close. Real Windows verification uses JDK 21 and
the pinned JADX 0.7.1 JAR; see the real verification lane below.
## Answer an analyst question
@@ -138,6 +157,12 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
server shutdown. One-shot CLI commands always join cleanup before returning.
REA never writes to the original APK or launches target code.
On Windows, stdin writes preserve stream backpressure and run outside the Node
event loop. Cancellation terminates the retained job, including descendants,
before workspace cleanup. An abrupt CLI/server process termination closes the
job and stops Java, but cannot run JavaScript workspace cleanup; temporary
workspace files can remain after that forced exit.
If Java cannot start the bridge, REA reports the selected executable and retained
startup diagnostics. Select a full JDK through `JAVA_HOME` or `PATH`, and use
`java --list-modules` to confirm `jdk.compiler` is present. A runnable JRE alone
@@ -197,6 +222,12 @@ without the compiler module. It checks real Java startup failures, missing
selectors, recovery after failures, and CLI cancellation with owned cleanup.
The original APK and host Java configuration remain unchanged.
The same lane runs on Windows x64 with the bundled native controls and a full
JDK. It checks CLI/MCP results, real MCP cancellation and disconnect, and Java
exit after abrupt CLI owner termination. Windows forced termination is checked
separately from the POSIX CLI SIGTERM handler; it does not claim a structured
cancellation result or workspace removal after the owner is forcibly killed.
The download script verifies fixed SHA-256 values, reuses matching files and
refuses to overwrite a different existing file. APK/JAR files live under ignored
`_reference/` and are excluded from package contents. The lane exercises actual
+8
View File
@@ -31,6 +31,14 @@ overall coverage becomes `partial` and a limitation explains the omission.
Duplicate inventory pages are merged by artifact identity and occurrence path;
their Evidence IDs remain attached as source Evidence.
Content nodes identify bytes by SHA-256 and report only content-derived formats.
Each occurrence reports its own `artifact_kind`, `artifact_format`, and executable
permission. Identical bytes can therefore appear as a script, resource, or native
addon at different paths without overwriting another occurrence's role. Package
formats such as APK and IPA belong to the occurrence; their ZIP bytes share a
content format. Graph and directory identities use locale-independent Unicode
ordering.
For IPA inventories, archive components outside every application root remain
in the component lists. Their JavaScript and native candidates are grouped with
other unrooted components, preserving the inventory projection's path pairing
+1 -2
View File
@@ -110,8 +110,7 @@ cached result. `close_binary` accepts `snapshot_path` and optional
leaves the session open so the caller can resolve the output failure.
The MCP save receipt reports `primitive_entries`, `workflow_entries`, and
`evidence_records` separately; the existing `entries` field remains the
primitive count. Zero primitive bindings can still accompany retained workflow
`evidence_records` separately. Zero primitive bindings can still accompany retained workflow
results and Evidence. These are cached observations, not a saved provider
database: only eligible exact queries can reuse a result, and new or live
queries can still require provider startup. The CLI uses the same snapshot
+3 -2
View File
@@ -515,8 +515,9 @@ import and auto-analysis. The first Ghidra-backed query starts that work lazily.
The provider startup deadline is 330,000 ms by default for import, analysis,
bridge, and health readiness. Large binaries can need more: set
`REA_GHIDRA_STARTUP_TIMEOUT_MS` to an integer between 1 and 2,147,483,647
milliseconds in the server environment. An absent, empty, invalid or out-of-range
value keeps the default. REA parses the supplied configuration environment and
milliseconds in the server environment. An absent setting uses the default;
empty, invalid or out-of-range supplied values fail configuration validation.
REA parses the supplied configuration environment and
passes the deadline to each provider client; a startup failure is returned by the query that triggered it,
without exposing partial analysis. This deadline is separate from MCP transport
initialization and the client's deadline for that individual tool call. A client
+2 -2
View File
@@ -189,8 +189,8 @@ writeFileSync("trace-input.json", JSON.stringify({
rea trace-application-feature ./trace-input.json --json
```
MCP analysis returns an envelope containing `result`, `evidence_id`, and full
`evidence`. If the connected server advertises retained references, reuse its
MCP analysis returns the complete Evidence record, with the operation result
in `normalized_result` and its identity in `evidence_id`. If the connected server advertises retained references, reuse its
exact returned ID as `{"kind":"retained-evidence","evidence_id":"RETURNED_ID"}`
in `application`, or `left`/`right` for comparisons. `RETURNED_ID` is a template,
not a literal valid ID. Native Evidence arrays still use complete records.
+35 -32
View File
@@ -2,7 +2,7 @@
## Generated catalog
Run `npm run build:cached` in a source checkout to generate the machine-readable
Run `npm run docs:generate` in a source checkout to generate the machine-readable
catalog at `docs/public/product-catalog.json`. Documentation deployments serve
the same file at [/rea/product-catalog.json](/rea/product-catalog.json). PR CI retains it with the packaged
skill and portable conformance projections in the `generated-docs` artifact.
@@ -24,28 +24,28 @@ health transition leaves that catalog unchanged and does not emit
`notifications/tools/list_changed`.
Advertised input and output schemas contain no reachable recursive references.
The input compatibility profile limits nesting to ten object, array, and
`anyOf`/`oneOf`/`allOf` levels, following local references per path. Property
maps, reference definitions, and example data do not add schema levels. Tests
check the complete catalog after SDK conversion and its generated counterpart.
This is REA's local compatibility profile; individual model APIs can impose
additional limits.
Schemas share repeated definitions through schema-local references while
preserving complete fields and validation rules. Input properties retain their
descriptions and literal examples. The SDK advertises an object root for
canonical object unions and retains each branch's requirements and exclusions.
Tests validate the schemas after SDK conversion and compare advertised
validation with actual calls. Individual model APIs can impose additional
nesting limits; complete producer captures can exceed ten structural levels.
Input and output schemas share repeated definitions through schema-local
references while preserving their complete fields and validation rules. Input
properties retain their descriptions and literal examples. Root input unions
keep their inline object presentation to avoid redundant branch nesting.
Canonical Zod validation and the input compatibility profile continue to apply.
`compare_web_captures` accepts exactly one of two input shapes:
For passive `compare_web_captures` inputs, pass each complete
`inspect_web_page` result in `before.inspection` or `after.inspection`, with an
optional complete `discover_webmcp_tools` result in the matching `webmcp` field.
The input schema describes these producer-result objects as round-trip payloads.
For scenario comparisons, pass complete `capture_browser_scenario` results in
`before_scenario` and `after_scenario`. REA validates all nested fields with the
original capture schemas; observation output schemas remain complete. Scenario
authoring inputs and comparison normalization options retain their full
advertised structure.
- Passive: `before` and `after` each contain `inspection`, the complete
`normalized_result` from `inspect_web_page`. Each may also contain `webmcp`,
the complete `normalized_result` from `discover_webmcp_tools`, or null.
- Scenario: `before_scenario` and `after_scenario` contain complete
`normalized_result` objects from `capture_browser_scenario`. Optional
`normalization` defaults to `{ "rules": [] }`.
The advertised schema includes complete nested capture fields and rejects
incomplete pairs, mixed comparison families, and structurally malformed
captures. Domain validation additionally checks relationships such as event
sequence references and retained counts; JSON Schema does not express those
cross-field invariants.
Call `binary_session` with `{}` and read `result.tool_availability` to choose a
callable operation for the current target, provider, host, and negotiated client
@@ -128,11 +128,11 @@ auto-analysis, bridge connection, and health readiness before returning analysis
These deadlines have different owners:
| Deadline | Owner and effect |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| MCP initialize | The client bounds transport/REA connection startup, before any target query. |
| Ghidra startup | REA allows 330,000 ms by default; `REA_GHIDRA_STARTUP_TIMEOUT_MS` accepts 1–2,147,483,647 ms. Invalid values keep the default. Startup failure is reported by the first provider query. |
| Individual tool call | The client bounds its wait, including cold engine startup. The pinned client SDK 2.3.1 defaults to 60,000 ms and can cancel earlier than REA's startup deadline. |
| Deadline | Owner and effect |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| MCP initialize | The client bounds transport/REA connection startup, before any target query. |
| Ghidra startup | REA allows 330,000 ms by default; `REA_GHIDRA_STARTUP_TIMEOUT_MS` accepts decimal integers from 1–2,147,483,647 ms. Invalid supplied values fail configuration validation. Startup failure is reported by the first provider query. |
| Individual tool call | The client bounds its wait, including cold engine startup. The pinned client SDK 2.3.1 defaults to 60,000 ms and can cancel earlier than REA's startup deadline. |
For an already connected client using the pinned SDK, request options are the
**second** argument of `callTool`:
@@ -188,18 +188,21 @@ increasing a client deadline alone does not fix those failures.
## Tool results
Evidence-producing tools return `{ result, evidence_id, evidence }` in both
text and structured content. `evidence` is the complete canonical Evidence
record, including `normalized_result`, which equals `result`. The same record
is retained in the session bundle. Read `result` directly, or pass `evidence`
to a compatible comparison tool: `analyze_function` Evidence can be passed
Evidence-producing tools return the complete canonical Evidence record in both
text and structured content. Read `structuredContent.normalized_result` for the
operation result and `structuredContent.evidence_id` for its identity. The same
record is retained in the session bundle. `analysis_profile` is always present:
a concrete profile object or `null`; either value participates in semantic
identity. Records omitting this field are rejected. Pass the returned Evidence
directly to a compatible comparison tool: `analyze_function` Evidence can be passed
directly to `compare_functions`, and `inspect_artifact` Evidence to
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
retained session history or an explicit bundle for transfer.
REA prepares complete MCP results within the pinned stdio client's 10 MiB
receive-buffer budget, including both text and structured representations and
room for the JSON-RPC envelope. If a result cannot fit, REA returns
room for the JSON-RPC envelope. Response budget settings are captured at startup;
restart or recreate the server to apply changes. If a result cannot fit, REA returns
`resource_constraint` with `details.resource: "transport"` before constructing
a document-sized string. Analysis Evidence remains complete in the current
session. Its exact reference is reported in
+30 -7
View File
@@ -1,13 +1,18 @@
# Testing REA
Prefer evidence in this order: full end-to-end workflows with real production
providers, integration tests across data/API boundaries, then golden regressions
from real captured inputs. Keep focused module tests for distinct failure or
semantic cases that these workflows cannot reliably reproduce. A test's path
or suite name does not establish its behavioral depth.
Prefer evidence in this order: full end-to-end workflows through the public
CLI or MCP with real production providers and no mocked dependencies,
integration across production data/API boundaries, then golden regressions
from real captured inputs. Keep a focused module test only for a distinct
failure or semantic case that stronger workflows cannot reliably reproduce.
A test's path, compiled imports, or suite name does not establish its depth.
Avoid tests that mirror getters, enum mappings, fixture helpers, or prescribed
call sequences, or assert cache object identity without a caller-visible contract.
Delete tests that only mirror getters, serialization helpers, enum mappings,
fixture helpers, prescribed call sequences, arbitrary counts or snapshots,
or cache object identity without a caller-visible contract. Do not retain a
weaker test merely because it is fast when a stronger workflow already proves
the same claim. Keep format-specific malformed representations and actual
capacity, cleanup, permission and target-identity regressions.
Prefer representative workflows over a Cartesian matrix when rows cross the
same boundary. Retain combinations that exercise a distinct interaction, such as
filtering within a full-output envelope. Keep distinct command handlers,
@@ -458,6 +463,13 @@ Electron GUI behavior or another engine.
## Android APK analysis
The real Android lane also runs on Windows x64 with its matching bundled native
controls. It verifies bidirectional protocol input, CLI/MCP result parity,
real MCP cancellation/disconnect and Java exit after abrupt CLI owner
termination. The forced Windows exit does not exercise the POSIX SIGTERM
handler and can leave temporary workspace files. `verify:windows-native`
separately checks binary input, backpressure, EOF and pending-write job closure.
`npm run verify:android` requires an existing Java 17+ and an explicit
`REA_JADX_MCP_JAR` for jadx-headless-mcp 0.7.1. Set `REA_ANDROID_TEST_APK` to the
fixed public ApiDemos v6.0.18 fixture. Obtain both with the explicit
@@ -647,6 +659,17 @@ The dry-run option reports the chosen merge base, scope and build prerequisite
without executing tests or building. A missing Git base reports how to fetch
it or select another revision.
Source typechecking needs no compiled runtime or generated test metadata.
`npm run build:cached` compiles the CLI/MCP runtime and its packaged skill.
`npm run test:prepare` also generates the MCP contract test catalog, product
catalog, and portable managed evidence used by the complete suite. Focused
tests prepare those extra outputs only when their selected files consume them.
The MCP test catalog is JSON in `.cache/mcp-tool-catalog.json`; the tracked
test loader supplies its types without making generation a source-check
prerequisite. Run `npm run mcp-catalog:generate` to refresh it independently.
Documentation generation and managed conformance run through their separate
`docs:generate` and `evidence:generate` task graphs.
Changed selection can miss runtime registration, generated data, shell
entrypoints, bridges, or other relationships absent from the import graph.
An empty changed selection means no tests were selected, not verified
+1 -2
View File
@@ -10,8 +10,7 @@ verified manifest into one caller-selected, absent directory.
For passive inspection, select `include_script_sources: true` (CLI:
`--include-script-sources`). For scenarios, select
`capture.network.response_body: true`. Save either the complete Evidence record
or its normalized result as UTF-8 JSON. The compact MCP wrapper's `evidence`
field contains the complete Evidence record.
or its normalized result as UTF-8 JSON. MCP `structuredContent` contains the complete Evidence record.
```sh
rea export-web-scripts /analysis/capture.json /analysis/exported-scripts --json