Compare commits

..
Author SHA1 Message Date
25e0709b2b fix(docs): update browser capture guide link after move (#2820)
Refresh gettext protected-link comments alongside the authored href.

Co-authored-by: Codex <noreply@openai.com>
2026-10-14 06:55:59 +08:00
morluto 8aad04e82c chore(merge): refresh PR #2791 with CI repair checkpoint 2026-10-14 03:10:44 +08:00
morluto 1f5fb6a9c9 chore(merge): refresh PR #2791 with latest main 2026-10-14 02:23:57 +08:00
morluto dbbf649f89 docs: correct verified documentation route counts 2026-10-14 02:11:45 +08:00
morluto 71a0e347e1 chore(merge): refresh PR #2791 with main 2026-10-14 02:10:19 +08:00
morluto 4c66974724 fix(docs): update links after guide relocation 2026-10-14 00:57:21 +08:00
95b53e9584 fix: repair the static-check failures on the default branch (#2895)
* fix: repair the static-check failures on the default branch

The default branch fails its own required `Static checks` job three ways, so
every pull request reports that job as failed regardless of its own change.

- `npm run format:check` reports `src/domain/functionComparison.test.ts` and
  `src/ida/IdaOperationRunner.ts`.
- `npm run lint` reports a duplicate `import { join } from "node:path"` in
  `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17).
- `npm run typecheck` reports `format: "devin"` missing from the client format
  union in `src/application/SupportedClients.ts`, and an unchecked index in
  `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is
  inferred as `string[][]`.

Apply the pinned formatter, drop the duplicate import, list the `"devin"` format
that its own definition already uses, and mark the pair array `as const`.

Validation: `npm run check:fast` and `npm run format:check` pass.

* fix(setup): keep Devin on the JSON configuration dialect

---------

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 00:53:45 +08:00
morluto 1f2e8499dc chore: merge current main into PR #2791 2026-10-13 23:57:24 +08:00
morluto d06bfb12e5 fix(docs): resolve captured main conflicts and review findings 2026-10-13 23:11:33 +08:00
morluto 0b46590f8f chore(docs): merge captured main into canonical guide layout 2026-10-13 21:44:34 +08:00
morluto 2f1f944b6f fix(docs): preserve routes and navigation cache inputs 2026-10-13 21:31:17 +08:00
morluto ce3a418715 chore(docs): merge captured main into documentation continuation 2026-10-13 21:07:53 +08:00
morluto ce6bce4389 fix: reconcile final main for PR #2791 2026-10-13 20:54:03 +08:00
morluto 324afce053 style: format reviewed source changes 2026-10-13 20:36:56 +08:00
morluto 813cb3afc9 fix(docs): honor home layout in all-page route verification 2026-10-13 20:34:23 +08:00
morluto 8c113fa959 chore: merge latest main into documentation continuation 2026-10-13 20:28:40 +08:00
morluto 2b1cc586ab fix(docs): reconcile canonical layout and verify new pages 2026-10-13 20:10:33 +08:00
Matheus BreguêzandCodex 51aa6eeca3 merge: preserve current MCP paging contracts in organized documentation
Retain list_unknowns paging guidance in docs/reference/mcp-contracts.md and verify the public routes and affected MCP regressions.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-12 21:24:41 -03:00
Matheus BreguêzandCodex b8f8d39cb5 merge: reconcile current guides and require published-page regressions
Preserve upstream managed summaries and signal guides; verify public routes, source paths and anchors in documentation CI.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-12 21:14:35 -03:00
Matheus BreguêzandCodex 3bb92dbee8 merge: integrate upstream documentation and preserve published routes
Resolve upstream changes without dropping translations or the signal-recording guide. Add a production HTML regression for stable routes, edit links, assets and fragment targets.

Refs #2401

Co-Authored-By: Codex <noreply@openai.com>
2026-10-12 20:59:35 -03:00
Matheus BreguêzandCodex ea504fac97 docs: organize canonical documentation layout (URGENTE)
Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-12 18:44:55 -03:00
648 changed files with 37196 additions and 103391 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
# REA changelog update
Use this for release-note preparation and contributor attribution. Read
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
publication authority and the existing checkpoint checks. This skill prepares
notes; it does not authorize merging, tagging, or publishing.
+1 -1
View File
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
removed without losing that protection. The outcome may be a verified fix,
an evidence-backed recommendation, or an explicit decision to stop.
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
and pinned toolchain, rather than commands remembered from an earlier audit.
Preserve the requested scope: investigation alone does not authorize edits;
+2 -2
View File
@@ -5,7 +5,7 @@ description: Design or change REA investigation tools, CLI/MCP contracts, provid
# REA tool design
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
contract semantics, provider ownership, and discoverability. Inspect the current
contract and nearest existing tool relevant to the analyst question; source,
documentation, and live catalogs may describe different versions.
@@ -14,7 +14,7 @@ For a design request, return the proposed boundary, input/result shape, Evidence
semantics, nearest alternative, and verification approach. Implement when
requested, preserving the user's scope and existing authorization.
For implementation or verification, consult [testing](../../../docs/testing.md)
For implementation or verification, consult [testing](../../../docs/development/testing.md)
for the affected consumer and provider lanes, and
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
provider/workflow guides only as the task requires.
@@ -31,49 +31,6 @@ Diagnose the observed blocker, continue useful supported work, and resume the
original task after repair. Client approval, shell PATH, registration and engine
availability have different remedies; avoid repeating setup for all of them.
1. Diagnose without changing REA or client configuration. `npx` may download
the package and write to npm's cache. For Codex, run
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
file registration: doctor reports the Grok Bot chat step, and setup does not
write `mcp.json`. If the client is unknown, use `doctor --json` and
inspect its registration results before choosing a setup scope.
2. Distinguish the reason. Missing, malformed, or stale registration needs a
scoped configuration repair. An aligned registration with no tools in the
active session needs a restart/reconnection; doctor cannot prove that the
current agent has connected. A missing provider affects only tasks requiring
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
and Android inspection has separate bring-your-own JADX/Java prerequisites.
3. For a configuration repair, prepare the read-only plan:
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
Use the current client's ID, show its exact proposed paths, backups, and
changes, and obtain approval before setup writes configuration or installs
Hopper. Setup normally installs the matching bundled skill too; include that
replacement in the reviewed plan. After approval, apply the same scope with
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
only if that separate installation was needed and explicitly approved.
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
in the session, then resume the original investigation. If they remain
absent, inspect the client's MCP launch error rather than repeating setup.
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
software to repair
MCP registration. For an unsupported client, use manual stdio registration
with a version-pinned `rea-agents` package or continue through the CLI.
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
```bash
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
```
No MCP registration or native engine is required. Read the returned Evidence,
graph, limitations, and unknowns with the same care as an MCP result; the CLI
returns the Evidence record directly. This fallback does not establish that
MCP is configured. Native CLI tasks still require their selected engine.
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
@@ -99,7 +56,7 @@ explicit path or endpoint and do not need it.
- Firmware image: `inspect_firmware_regions` when advertised. Use
`extract_firmware` when extraction is requested, with a caller-selected new
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
optional metadata recovery and its supported host/layout boundary.
@@ -111,7 +68,7 @@ explicit path or endpoint and do not need it.
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
original record ordinals when useful; inspect producer fields and byte/number
sidecars without fetching recorded URLs or claiming live attribution. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
or establish network loss. Prefer detail summary, then kind qlog trace/stream
@@ -124,7 +81,7 @@ explicit path or endpoint and do not need it.
Query/page the relevant records; registrations do not establish observed traffic.
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
Packet limits count input packets before filtering. Output defaults to 50 matching
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
@@ -132,7 +89,7 @@ explicit path or endpoint and do not need it.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
@@ -140,7 +97,7 @@ explicit path or endpoint and do not need it.
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
Output defaults to 50 literal lines; select annotation classes and page with
`line_offset` / `line_limit`, or explicit null for all remaining output.
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
@@ -149,7 +106,7 @@ explicit path or endpoint and do not need it.
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
Select JavaScript glue and candidate paths explicitly and preserve URL/path
ambiguity. No implicit fetch or target execution; WAT is not original source.
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
`raw` or `hex` encoding. Preserve carrier/decoded digests and treat selectors,
argument strings and mutability as inferred candidates; this performs no chain
@@ -158,21 +115,13 @@ explicit path or endpoint and do not need it.
Evidence stays retained under its original identity. Use the returned
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
Candidate coverage does not establish interface completeness. See the
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
with the explicit local file. This built-in target-free inspection requires no
Go installation or disassembler. Preserve raw module bytes and source ranges;
absent metadata does not prove a binary is not Go, and unparsed text records
remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
- Embedded cargo-auditable Rust dependency metadata: `inspect_rust_binary`
with the explicit local ELF, PE, or thin Mach-O file. This built-in
target-free inspection requires no Rust installation or disassembler. Preserve
package order, dependency indices, and the section file range. A missing
`.dep-v0` section is null metadata, not proof the binary has no dependencies.
A rustc commit hash is an inference from a read-only `/rustc/<40-hex>/` path.
See the
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
@@ -181,17 +130,17 @@ explicit path or endpoint and do not need it.
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
report truncation and have unspecified order. Use a quiescent snapshot rather
than assuming a live database/WAL pair was copied atomically. See the
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
thread, raw note and source-bound register without opening a live target.
Historical PIDs do not select live processes. Request `include_debugger_context`
only when core-only mapping candidates help; see the
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
@@ -205,7 +154,7 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
Proceed directly to `analyze_function`, pseudocode, or function/string searches
after opening; IDA does not supply `binary_overview`. Consult session availability
when composing broader workflows. Modern direct callers and unsupported dossier
@@ -65,7 +65,3 @@ Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
method, origin and path template. Format matches are inferences, not handler
identity or schemas. Group pages cite parent record ordinals; consult record
coverage and exclusions before claiming completeness. No requests are replayed.
When the connected schema advertises `include_query_parameter_names`, select
it in the view for distinct observed names with original record ordinals. Query
values are omitted; the names establish neither required parameters nor a
complete server schema. Keep the original parent Evidence for exact URLs.
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
`additional_paths` when advertised, then analyze the returned `analysis_input`
directory together. Each input retains its own snapshot, digest and byte-range
provenance; inputs and recovered code are never executed. Recovery requires a
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. When the question concerns known source paths and the connected
@@ -11,18 +11,6 @@ unparsed module-text line. A missing record is an unknown, not proof of a
non-Go binary; observed metadata does not authenticate its claims. Function,
source and type recovery remain a separate native-analysis question.
## Rust dependency metadata
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
Preserve artifact SHA-256, package order, dependency indices, and the section
file range. A missing section is null metadata, not proof the binary has no
dependencies. An omitted package kind stays null. A rustc commit hash is
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
## SQLite snapshots
Use `inspect_sqlite_database` with an explicit local database snapshot for its
@@ -36,7 +24,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
unspecified row order explicit. Virtual tables, shadow tables and views are
schema facts, not row-read targets. REA inspects private copies and leaves
the source files untouched; obtain a coherent quiescent database/WAL snapshot
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
## Recorded Linux crashes
@@ -51,7 +39,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
establish current file identity; zero reported flags leave permissions unknown.
Requested unavailable context fails with setup guidance. See
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
for exact upstream profiles, bounds and verification coverage.
## Offline ELF layout
@@ -190,7 +178,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
NativeAOT JAR. These are separate routes. The built-in parser also runs on
Windows Ghidra P0; read frozen literals from its report, because
`search_strings` does not include them. See the
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
for setup, supported layouts, and host restrictions. Preserve binary identity
and distinguish derived metadata from Ghidra database observations. Recovered
types and native pseudocode are not original C# source. Windows Ghidra P0 has
@@ -219,7 +207,7 @@ every executable in the bundle. It also lists missing or weak loads and earlier
provides them.
Use `extract_artifact` when materialized files are needed. It extracts the active
target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
It materializes all regular files into a fresh temporary directory chosen by
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
@@ -234,7 +222,7 @@ separate filesystem-writing operation.
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
`--target-format dos-com`). Inspect the load image before trusting function
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
@@ -244,7 +232,7 @@ that address. Hopper and IDA refuse this interpretation; Ghidra uses
and Cortex-M variants are unsupported. `6502` selects NMOS 6502; the entry and
complete file mapping must fit within `0x0000..0xffff`. 65C02, 65816, cartridge
containers, bank switching, and mapper semantics are unsupported. Windows
Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
### Function annotations in Ghidra
@@ -260,7 +248,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
resolution is explicit; missing references and size mismatch remain visible.
Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
@@ -39,5 +39,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
traffic analysis, connect Everywhere's separate MCP server and use its
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
`AgentCalls`). Preserve existing sessions and filters when inspecting a
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
for the workflow and Telerik's public setup and traffic-debugging skills.
+1 -1
View File
@@ -20,7 +20,7 @@ body:
attributes:
label: Documentation location
description: Link the page, section, command, API, skill, or inline comment.
placeholder: README.md#quick-start or docs/mcp-contracts.md
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
validations:
required: true
-42
View File
@@ -1,42 +0,0 @@
name: Cache npm package downloads
description: Restore npm's download cache and save it only from trusted default-branch events.
outputs:
can-save:
description: Whether this run may update the shared npm download cache.
value: ${{ steps.context.outputs.can-save }}
path:
description: The npm download-cache directory.
value: ${{ steps.context.outputs.path }}
runs:
using: composite
steps:
- id: context
shell: bash
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
LOCKFILES_HASH: ${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}
run: |
cache_path="$(npm config get cache)"
runtime_arch="$(node -p 'process.arch')"
can_save=false
if [[ -n "${DEFAULT_BRANCH}" && "${GITHUB_REF}" == "refs/heads/${DEFAULT_BRANCH}" && ( "${GITHUB_EVENT_NAME}" == "push" || "${GITHUB_EVENT_NAME}" == "schedule" || "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ) ]]; then
can_save=true
fi
cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
{
printf 'can-save=%s\n' "${can_save}"
printf 'path=%s\n' "${cache_path}"
printf 'key=%s\n' "${cache_key}"
} >> "${GITHUB_OUTPUT}"
- name: Restore and save npm package downloads
if: steps.context.outputs.can-save == 'true'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
- name: Restore npm package downloads without saving
if: steps.context.outputs.can-save != 'true'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
+1 -1
View File
@@ -85,7 +85,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Every release increments minor, including breaking changes;
keep breaking markers and migration notes so callers can assess compatibility.
See docs/releasing.md. -->
See docs/development/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
+1 -39
View File
@@ -12,13 +12,12 @@ concurrency:
permissions:
contents: read
actions: read
pull-requests: read
jobs:
report:
name: Collect recent completed workflow attempts
runs-on: ubuntu-latest
timeout-minutes: 10
timeout-minutes: 5
steps:
# Consume only API data and execute batch code from the trusted default branch.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -28,18 +27,6 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
continue-on-error: true
uses: ./.github/actions/npm-cache
- name: Diagnose missing and startup-failed PR CI
continue-on-error: true
env:
GH_TOKEN: ${{ github.token }}
CI_REPOSITORY: ${{ github.repository }}
CI_REPOSITORY_ID: ${{ github.repository_id }}
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: node scripts/ci/pr-ci-health.mjs
- name: Record queue, execution, and matched step timing
continue-on-error: true
env:
@@ -57,28 +44,3 @@ jobs:
if-no-files-found: error
overwrite: true
retention-days: 14
- name: Install dependencies for static failure attribution
id: failure-deps
if: ${{ !cancelled() && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
continue-on-error: true
run: npm ci
- name: Attribute supported static failures against main
if: ${{ !cancelled() && steps.failure-deps.outcome == 'success' && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
continue-on-error: true
env:
GH_TOKEN: ${{ github.token }}
CI_REPOSITORY: ${{ github.repository }}
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: node scripts/ci/failure-report.mjs
- name: Upload CI health and failure diagnostics
if: ${{ !cancelled() }}
continue-on-error: true
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci-health-diagnostics
path: |
.cache/ci-pr-health/
.cache/ci-failures/
include-hidden-files: true
if-no-files-found: warn
retention-days: 14
+25 -128
View File
@@ -42,7 +42,6 @@ jobs:
tier: ${{ steps.plan.outputs.tier }}
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
code: ${{ steps.plan.outputs.code }}
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
sqlite: ${{ steps.plan.outputs.sqlite }}
runtime: ${{ steps.plan.outputs.runtime }}
package: ${{ steps.plan.outputs.package }}
@@ -123,28 +122,11 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Restore static task cache
if: needs.changes.outputs.code == 'true'
id: static-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
# Turbo validates each task's inputs after restoring this snapshot.
restore-keys: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
- name: Record static task source and execution identity
continue-on-error: true
if: needs.changes.outputs.code == 'true'
env:
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: npm run ci:static-context
- if: needs.changes.outputs.code == 'true'
run: npm run check:ci
- if: needs.changes.outputs.code == 'true'
@@ -154,15 +136,6 @@ jobs:
run: npm run test:python
- if: needs.changes.outputs.code != 'true'
run: npm run format:check
- name: Save validated static task cache
if: >-
success() && needs.changes.outputs.code == 'true' &&
steps.npm-cache.outputs.can-save == 'true' &&
steps.static-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: ${{ steps.static-cache.outputs.cache-primary-key }}
- name: Upload complete static diagnostics
if: always() && needs.changes.outputs.code == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -173,34 +146,6 @@ jobs:
if-no-files-found: ignore
retention-days: 7
test-source:
name: Source tests
needs: changes
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Run source tests without waiting for build
run: npm run test:fast -- --reporter=default --reporter=blob
env:
VITEST_BLOB_LABEL: source
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ !cancelled() }}
with:
name: vitest-report-source
path: .vitest/blob/*
include-hidden-files: true
if-no-files-found: error
retention-days: 7
test-shard:
name: Test shard (${{ matrix.shard }})
needs: [changes, build]
@@ -208,21 +153,15 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
# Failed PRs release the remaining matrix slots; qualification retains
# every shard's diagnostics and coverage. Passing PRs run the full suite.
fail-fast: ${{ github.event_name == 'pull_request' && needs.changes.outputs.tier == 'fast' }}
fail-fast: false
matrix:
# Fast feedback spends the same six test jobs on one source group and
# five built groups; full qualification retains six complete shards.
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/5","2/5","3/5","4/5","5/5"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
shard: [1/6, 2/6, 3/6, 4/6, 5/6, 6/6]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -233,42 +172,21 @@ jobs:
run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
env:
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- name: Run built-dependent tests for fast feedback
- name: Run complete portable suite for fast feedback
if: needs.changes.outputs.tier == 'fast'
run: npm run test:built -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
env:
VITEST_BLOB_LABEL: built
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ !cancelled() }}
with:
name: vitest-report-built-${{ strategy.job-index }}
name: vitest-report-${{ strategy.job-index }}
path: .vitest/blob/*
include-hidden-files: true
if-no-files-found: error
overwrite: true
retention-days: 7
reverse-tools-quality:
name: Reverse tools coverage and mutation
needs: [changes, static]
if: needs.changes.outputs.reverse_tools == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Run reverse tools tests with coverage
run: npm run test:coverage:reverse-tools
- name: Report reverse tools mutation results
run: npm run test:mutation:reverse-tools
build:
name: Build package
needs: changes
@@ -294,9 +212,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Reuse portable build outputs across runs
uses: ./.github/actions/portable-build-cache
@@ -355,7 +271,7 @@ jobs:
test:
name: Portable tests
needs: [changes, test-source, test-shard]
needs: [changes, test-shard]
if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }}
runs-on: ubuntu-latest
timeout-minutes: 10
@@ -370,18 +286,13 @@ jobs:
if: needs.changes.outputs.code == 'true' && success()
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
if: needs.changes.outputs.code == 'true' && success()
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- if: needs.changes.outputs.code == 'true' && success()
run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
if: >-
!cancelled() && needs.changes.outputs.code == 'true' &&
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
(needs.changes.outputs.tier == 'fast' &&
(needs.test-source.result == 'success' || needs.test-source.result == 'failure')))
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
with:
path: .vitest/blob
pattern: vitest-report-*
@@ -409,8 +320,7 @@ jobs:
if: >-
!cancelled() && needs.changes.outputs.code == 'true' &&
needs.changes.outputs.tier == 'fast' &&
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
run: >-
npm exec -- vitest --merge-reports
--reporter=default
@@ -423,8 +333,6 @@ jobs:
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-timings.json') != ''
env:
SHARD_RESULT: ${{ needs.test-shard.result }}
SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
SOURCE_RESULT: ${{ needs.test-source.result }}
MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }}
CI_VERIFY_TEST_INVENTORY: "true"
CI_TEST_SELECTION: ${{ needs.changes.outputs.test_selection }}
@@ -457,13 +365,7 @@ jobs:
env:
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
SHARD_RESULT: ${{ needs.test-shard.result }}
SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
SOURCE_RESULT: ${{ needs.test-source.result }}
run: |
if [ "${CODE_REQUIRED}" = "true" ]; then
test "${SHARD_RESULT}" = "success" || exit 1
test "${SOURCE_REQUIRED}" != "true" || test "${SOURCE_RESULT}" = "success"
fi
run: test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
conformance-fixtures:
name: Source-built conformance fixtures
@@ -548,9 +450,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- name: Verify native runtime identity
env:
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
@@ -572,6 +472,11 @@ jobs:
- run: npm run verify:windows-native:artifact
- name: Verify NativeAOT diagnostic applicability on this host
run: npm run test:focused -- tests/evaluation/package/nativeAotDiagnostics.test.ts
- name: Resolve npm cache before verifier HOME isolation
id: npm-cache
run: |
cache_path="$(npm config get cache)"
echo "path=${cache_path}" >> "${GITHUB_OUTPUT}"
- run: npm run verify:package
env:
npm_config_cache: ${{ steps.npm-cache.outputs.path }}
@@ -601,9 +506,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -651,9 +554,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Run the public CLI/MCP reconstruction journey
env:
@@ -682,9 +583,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -804,11 +703,11 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- run: npm run docs:check
- name: Verify published routes, source links and local anchors
run: npm run verify:docs:routes
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -895,9 +794,7 @@ jobs:
changes,
static,
build,
reverse-tools-quality,
test-shard,
test-source,
sqlite-runtime,
test,
conformance-fixtures,
+2 -4
View File
@@ -23,8 +23,6 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- run: npm run docs:build
- run: npm run verify:docs:routes
+1 -3
View File
@@ -32,9 +32,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
+1 -3
View File
@@ -25,9 +25,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
-76
View File
@@ -1,76 +0,0 @@
name: Real upstream Cutter verification
on:
workflow_dispatch:
repository_dispatch:
types: [real-cutter-upstream]
permissions:
contents: read
jobs:
windows-native-artifact:
name: Build Windows bridge security boundary
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Install the cross compiler used by the Windows native lane
run: |
sudo apt-get update
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
- run: npm run build:windows-native
- uses: actions/upload-artifact@v4
with:
name: cutter-windows-native
path: native/windows/build/
if-no-files-found: error
cutter-upstream:
name: Cutter upstream smoke (${{ matrix.platform }})
needs: windows-native-artifact
runs-on: ${{ matrix.runner }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
- platform: linux
runner: [self-hosted, cutter-upstream, linux]
- platform: macOS
runner: [self-hosted, cutter-upstream, macOS]
- platform: Windows
runner: [self-hosted, cutter-upstream, Windows]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- if: matrix.platform == 'Windows'
uses: actions/download-artifact@v4
with:
name: cutter-windows-native
path: native/windows/build/
- if: matrix.platform == 'Windows'
run: npm run verify:windows-native:artifact
- if: matrix.platform == 'Windows'
run: npm run build:cached
- if: matrix.platform == 'Windows'
run: npm run verify:windows-native
- name: Verify host and real Cutter bridge
env:
EXPECTED_PLATFORM: ${{ matrix.platform }}
run: npm run verify:cutter
- uses: actions/upload-artifact@v4
if: always()
with:
name: cutter-upstream-${{ matrix.platform }}
path: cutter-smoke.json
if-no-files-found: ignore
retention-days: 30
+1 -3
View File
@@ -26,9 +26,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
+1 -3
View File
@@ -48,9 +48,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
with:
go-version: "1.26.1"
+2 -6
View File
@@ -19,9 +19,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Pack CLI for setup verification
shell: bash
@@ -42,9 +40,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Install packed CLI in isolation
shell: bash
+1 -3
View File
@@ -19,9 +19,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- name: Install packed CLI in isolation
shell: bash
@@ -27,9 +27,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
+1 -3
View File
@@ -47,9 +47,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci
- run: npm run build:cached
+1 -3
View File
@@ -29,9 +29,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
+1 -3
View File
@@ -40,9 +40,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci
- run: npm run build:cached
@@ -26,9 +26,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- name: Use pinned package manager
run: npm install --global npm@11.16.0
- run: npm ci
+2 -4
View File
@@ -21,14 +21,12 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
- run: npm run verify:browser:runtime
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
+1 -3
View File
@@ -21,9 +21,7 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
+1 -3
View File
@@ -27,9 +27,7 @@ jobs:
with:
node-version: ${{ matrix.node }}
check-latest: true
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
cache: npm
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
-2
View File
@@ -33,8 +33,6 @@ codedb.snapshot
htmlcov/
.coverage
coverage/
.stryker-tmp/
reports/mutation/
.vitest/
.vitest-reports/
test-results.xml
+9 -3
View File
@@ -14,7 +14,7 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
## Architecture and Cleanup
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
@@ -57,6 +57,12 @@ do not establish fewer tokens or faster analysis.
## Development
Documentation ownership and authoring conventions live in
[docs/development/documentation.md](docs/development/documentation.md). A feature
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
Consult the [documentation directory](docs/README.md) before adding a guide.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
- `npm ci`: install locked dependencies.
@@ -72,9 +78,9 @@ Use relevant checks for the change. The complete local gate, `npm run check:pr`,
## Tool and Test Changes
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
+2 -2
View File
@@ -69,7 +69,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
@@ -237,7 +237,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
+11 -7
View File
@@ -11,7 +11,7 @@ incorrect user instructions warrant standalone corrections. Collect other
cosmetic-only suggestions in an issue rather than separate PRs.
For capability organization and provider composition, use the
[architecture map](docs/architecture.mermaid). Production factories in
[architecture map](docs/architecture/architecture.mermaid). Production factories in
`src/composition/` construct fresh providers without starting engines or acquiring
targets; optional adapter failures must preserve successful peers. Run
`npm run verify:test-discovery` after adding or moving tests. Resolved import
@@ -22,7 +22,7 @@ For analysis workflows and agent interfaces, follow the
scope, reuse/cache decisions, and relevant before/after measurements when these
costs change.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
## Contribution scope
@@ -124,7 +124,7 @@ guarantee.
Use the development toolchain pinned in `.nvmrc` and
`package.json#packageManager`. Run `nvm use` before installing dependencies;
installed-package runtime support is documented in
[installation](docs/installation.md#start-setup).
[installation](docs/getting-started/installation.md#start-setup).
```bash
npm ci
@@ -159,6 +159,10 @@ verification prerequisites. Keep the generated addon and manifest uncommitted.
## Documentation website
Use the [documentation directory](docs/README.md) and
[authoring guide](docs/development/documentation.md) for document ownership,
domain-based feature specifications and architectural decision records.
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
for live editing, `npm run docs:build` to check the production build and
links, and `npm run docs:preview` to preview that build at `/rea/`.
@@ -195,7 +199,7 @@ npm run check:fast
`test:local` runs source tests without building. `check:changed` combines cached
static checks with source tests affected since the merge base with `origin/main`.
Import-graph selection can miss runtime registration, bridges, and generated data;
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
for commands, test placement, pruning, and real-provider prerequisites.
Before requesting review, run relevant tests and `npm run check`; add
@@ -262,8 +266,8 @@ Run `npm run verify:hopper` on a supported macOS host or
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
native fixtures and verify the installed engine through CLI/MCP; package or
simulated-provider checks do not establish engine behavior. See
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
for supported hosts and unattended startup.
Use the default `build/conformance/manifest.json` unless explicitly verifying
@@ -272,7 +276,7 @@ Generated fixtures and manifests remain ignored.
## Maintainer release checklist
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
Release Please PR with the next version and changelog. Review its migration
notes, wait for the current CI checks to pass, and merge it into main to publish.
+47 -60
View File
@@ -1,7 +1,6 @@
<div align="center">
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
· [Ελληνικά](docs/readme/README_el.md)
# REA: Reverse Engineer Anything
@@ -10,7 +9,7 @@
**See a feature you like. Understand how it works, down to the binary level.**
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/reference/mcp-contracts.md#generated-catalog)
[![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#what-you-can-analyze)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -24,12 +23,6 @@
<code>npx rea-agents setup</code>
> [!IMPORTANT]
> Get REA from [morluto/rea](https://github.com/morluto/rea),
> [rea.tools](https://rea.tools/), or the [rea-agents npm package](https://www.npmjs.com/package/rea-agents).
> Follow the installation and release links from these official sources. A similar
> repository name or copied README does not establish that a download is official.
<br />
<img src="docs/assets/rea-hopper-analysis.png" alt="REA launching its analysis bridge inside Hopper while inspecting a native binary" width="1200" />
@@ -77,8 +70,8 @@ REA's MCP server and matching workflow instructions, with backups of existing
configuration. Restart your agent afterward.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
[other agents](docs/getting-started/installation.md#supported-agents). See
[installation and setup](docs/getting-started/installation.md) for provider configuration and
manual MCP registration.
### Ask your agent
@@ -109,7 +102,7 @@ rea --help
```
For native analysis, configure a provider first. See the
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
snapshots and scripting.
### Update REA
@@ -157,52 +150,46 @@ CLI commands use the same workflows.
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](https://rea.tools/guides/native/) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](https://rea.tools/guides/javascript/) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](https://rea.tools/guides/browser/) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](https://rea.tools/guides/native/) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](https://rea.tools/guides/javascript/) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](https://rea.tools/guides/browser/) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
using your user permissions; each runtime guide describes its effects.
For offline native-binary inspection, REA also supports caller-supplied GNU
objdump and Rizin. See the [reverse-engineering tools guide](docs/reverse-engineering-tools.md)
for provider setup, CLI commands, and Evidence limitations. REA does not install
or update either provider.
<a id="choosing-a-deep-analysis-provider"></a>
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), and
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), and
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
Check [release availability](docs/installation.md#released-package-and-main)
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
for features added since the latest npm release.
## Showcases
@@ -245,8 +232,8 @@ your question, how REA helped, and what you found.
<summary><strong>Which agents can use REA?</strong></summary>
Any agent that supports local MCP servers. Setup configures the
[supported agents](docs/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/installation.md#mcp-registry).
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
</details>
@@ -255,7 +242,7 @@ Any agent that supports local MCP servers. Setup configures the
Deep native analysis uses one of them. Static JavaScript and .NET inspection
work without a native analysis engine. Setup can install Hopper after approval;
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
Ghidra and IDA use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper).
</details>
@@ -266,7 +253,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
ask you to choose demo mode or activate your license. Complete that prompt in the
same user session before your first CLI/MCP operation or unattended job. Open
Hopper manually if needed. See
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
</details>
@@ -275,7 +262,7 @@ Hopper manually if needed. See
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
to register REA's MCP server and install the matching instructions, then restart
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
</details>
@@ -324,12 +311,12 @@ with your REA version, target type, steps to reproduce and error output.
Start with the website's [worked guides](https://rea.tools/guides/).
For exact options, prerequisites and result contracts:
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -364,8 +351,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
to improve the code or docs.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[testing](docs/testing.md) for verification lanes, and the
[architecture map](docs/architecture.mermaid) for the project structure.
[testing](docs/development/testing.md) for verification lanes, and the
[architecture map](docs/architecture/architecture.mermaid) for the project structure.
## License
+1 -1
View File
@@ -27,7 +27,7 @@ provider-dependent behavior to be reproduced safely.
REA authenticates its Hopper and Ghidra bridges with per-session random
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
the current user and SYSTEM. See [Windows Ghidra P0](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Tokens are transferred through private session descriptors rather than process
arguments or environment variables.
This is not a sandbox and does not protect against malicious processes already
-816
View File
@@ -1,816 +0,0 @@
"""REA's experimental local bridge plugin for Cutter.
Install this file in Cutter's user plugin directory. The bridge is REA-owned
IPC, not an upstream Cutter transport. It binds to loopback and requires a
per-process random token stored in a user-only descriptor file.
"""
from __future__ import annotations
import json
import hashlib
import hmac
import os
import secrets
import socket
import stat
import tempfile
import threading
import time
import uuid
from pathlib import Path
from typing import Any
if os.name == "nt":
import ctypes
from ctypes import wintypes
import cutter
try:
from PySide6.QtCore import QObject, Signal
except ImportError:
from PySide2.QtCore import QObject, Signal
MAX_FRAME_BYTES = 16 * 1024 * 1024
SOCKET_TIMEOUT_SECONDS = 20
AUTH_FRAME_TIMEOUT_SECONDS = 1
MAX_DESCRIPTOR_BYTES = 64 * 1024
MAX_COMPLETION_RECEIPTS = 64
def _windows_api() -> tuple[Any, Any]:
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
advapi32 = ctypes.WinDLL("advapi32", use_last_error=True)
kernel32.GetCurrentProcess.restype = wintypes.HANDLE
kernel32.LocalFree.argtypes = [wintypes.HLOCAL]
kernel32.LocalFree.restype = wintypes.HLOCAL
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
kernel32.CloseHandle.restype = wintypes.BOOL
kernel32.GetFileAttributesW.argtypes = [wintypes.LPCWSTR]
kernel32.GetFileAttributesW.restype = wintypes.DWORD
kernel32.CreateDirectoryW.argtypes = [wintypes.LPCWSTR, wintypes.LPVOID]
kernel32.CreateDirectoryW.restype = wintypes.BOOL
kernel32.CreateFileW.argtypes = [wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD,
wintypes.LPVOID, wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE]
kernel32.CreateFileW.restype = wintypes.HANDLE
kernel32.WriteFile.argtypes = [wintypes.HANDLE, wintypes.LPVOID, wintypes.DWORD,
ctypes.POINTER(wintypes.DWORD), wintypes.LPVOID]
kernel32.WriteFile.restype = wintypes.BOOL
kernel32.FlushFileBuffers.argtypes = [wintypes.HANDLE]
kernel32.FlushFileBuffers.restype = wintypes.BOOL
kernel32.MoveFileExW.argtypes = [wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD]
kernel32.MoveFileExW.restype = wintypes.BOOL
kernel32.DeleteFileW.argtypes = [wintypes.LPCWSTR]
kernel32.DeleteFileW.restype = wintypes.BOOL
advapi32.OpenProcessToken.argtypes = [wintypes.HANDLE, wintypes.DWORD, ctypes.POINTER(wintypes.HANDLE)]
advapi32.OpenProcessToken.restype = wintypes.BOOL
advapi32.GetTokenInformation.argtypes = [wintypes.HANDLE, ctypes.c_int, wintypes.LPVOID,
wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
advapi32.GetTokenInformation.restype = wintypes.BOOL
advapi32.ConvertSidToStringSidW.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.LPWSTR)]
advapi32.ConvertSidToStringSidW.restype = wintypes.BOOL
advapi32.GetNamedSecurityInfoW.argtypes = [wintypes.LPWSTR, ctypes.c_int, wintypes.DWORD,
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
ctypes.POINTER(wintypes.LPVOID)]
advapi32.GetNamedSecurityInfoW.restype = wintypes.DWORD
advapi32.GetSecurityDescriptorControl.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.WORD),
ctypes.POINTER(wintypes.DWORD)]
advapi32.GetSecurityDescriptorControl.restype = wintypes.BOOL
advapi32.EqualSid.argtypes = [wintypes.LPVOID, wintypes.LPVOID]
advapi32.EqualSid.restype = wintypes.BOOL
advapi32.GetAclInformation.argtypes = [wintypes.LPVOID, wintypes.LPVOID, wintypes.DWORD, ctypes.c_int]
advapi32.GetAclInformation.restype = wintypes.BOOL
advapi32.GetAce.argtypes = [wintypes.LPVOID, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID)]
advapi32.GetAce.restype = wintypes.BOOL
advapi32.IsValidSid.argtypes = [wintypes.LPVOID]
advapi32.IsValidSid.restype = wintypes.BOOL
advapi32.GetLengthSid.argtypes = [wintypes.LPVOID]
advapi32.GetLengthSid.restype = wintypes.DWORD
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.argtypes = [
wintypes.LPCWSTR, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.DWORD),
]
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.restype = wintypes.BOOL
return kernel32, advapi32
def _windows_user_sid_text() -> str:
kernel32, advapi32 = _windows_api()
token = wintypes.HANDLE()
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
raise ctypes.WinError(ctypes.get_last_error())
try:
required = wintypes.DWORD()
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
if required.value == 0:
raise ctypes.WinError(ctypes.get_last_error())
buffer = ctypes.create_string_buffer(required.value)
if not advapi32.GetTokenInformation(token, 1, buffer, required, ctypes.byref(required)):
raise ctypes.WinError(ctypes.get_last_error())
class SID_AND_ATTRIBUTES(ctypes.Structure):
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
class TOKEN_USER(ctypes.Structure):
_fields_ = [("User", SID_AND_ATTRIBUTES)]
sid = ctypes.cast(buffer, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
text = wintypes.LPWSTR()
if not advapi32.ConvertSidToStringSidW(sid, ctypes.byref(text)):
raise ctypes.WinError(ctypes.get_last_error())
try:
return text.value
finally:
kernel32.LocalFree(text)
finally:
kernel32.CloseHandle(token)
def _verify_windows_private_path(path: Path, directory: bool) -> None:
kernel32, advapi32 = _windows_api()
attributes = kernel32.GetFileAttributesW(str(path))
if attributes == 0xFFFFFFFF:
raise ctypes.WinError(ctypes.get_last_error())
if attributes & 0x400 or bool(attributes & 0x10) != directory:
raise RuntimeError("Cutter bridge path is a reparse point or has the wrong object type")
class ACL_SIZE_INFORMATION(ctypes.Structure):
_fields_ = [("AceCount", wintypes.DWORD), ("AclBytesInUse", wintypes.DWORD), ("AclBytesFree", wintypes.DWORD)]
class ACE_HEADER(ctypes.Structure):
_fields_ = [("AceType", wintypes.BYTE), ("AceFlags", wintypes.BYTE), ("AceSize", wintypes.WORD)]
class SID_HEADER(ctypes.Structure):
_fields_ = [("Revision", wintypes.BYTE), ("SubAuthorityCount", wintypes.BYTE),
("IdentifierAuthority", wintypes.BYTE * 6)]
class ACCESS_ALLOWED_ACE(ctypes.Structure):
_fields_ = [("Header", ACE_HEADER), ("Mask", wintypes.DWORD), ("SidStart", wintypes.DWORD)]
owner = wintypes.LPVOID()
dacl = wintypes.LPVOID()
descriptor = wintypes.LPVOID()
result = advapi32.GetNamedSecurityInfoW(
str(path), 1, 0x1 | 0x4, ctypes.byref(owner), None, ctypes.byref(dacl), None, ctypes.byref(descriptor),
)
if result != 0:
raise ctypes.WinError(result)
try:
control = wintypes.WORD()
revision = wintypes.DWORD()
if not advapi32.GetSecurityDescriptorControl(descriptor, ctypes.byref(control), ctypes.byref(revision)):
raise ctypes.WinError(ctypes.get_last_error())
if not control.value & 0x1000 or not dacl:
raise RuntimeError("Cutter bridge ACL must be present and protected from inheritance")
token = wintypes.HANDLE()
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
raise ctypes.WinError(ctypes.get_last_error())
try:
required = wintypes.DWORD()
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
token_user = ctypes.create_string_buffer(required.value)
if not advapi32.GetTokenInformation(token, 1, token_user, required, ctypes.byref(required)):
raise ctypes.WinError(ctypes.get_last_error())
class SID_AND_ATTRIBUTES(ctypes.Structure):
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
class TOKEN_USER(ctypes.Structure):
_fields_ = [("User", SID_AND_ATTRIBUTES)]
current_sid = ctypes.cast(token_user, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
if not advapi32.EqualSid(owner, current_sid):
raise RuntimeError("Cutter bridge path owner does not match the current user")
finally:
kernel32.CloseHandle(token)
info = ACL_SIZE_INFORMATION()
if not advapi32.GetAclInformation(dacl, ctypes.byref(info), ctypes.sizeof(info), 2) or info.AceCount != 2:
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
expected_user = _windows_user_sid_text()
user_allowed = False
system_allowed = False
for index in range(info.AceCount):
ace_pointer = wintypes.LPVOID()
if not advapi32.GetAce(dacl, index, ctypes.byref(ace_pointer)):
raise ctypes.WinError(ctypes.get_last_error())
header = ctypes.cast(ace_pointer, ctypes.POINTER(ACE_HEADER)).contents
if header.AceType != 0 or header.AceSize < ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD) + ctypes.sizeof(SID_HEADER):
raise RuntimeError("Cutter bridge ACL contains a malformed or unsupported ACE")
ace_address = ctypes.cast(ace_pointer, ctypes.c_void_p).value
if ace_address is None:
raise RuntimeError("Cutter bridge ACL returned a null ACE")
sid_pointer = ctypes.c_void_p(ace_address + ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD))
sid_header = ctypes.cast(sid_pointer, ctypes.POINTER(SID_HEADER)).contents
sid_size = ctypes.sizeof(SID_HEADER) + sid_header.SubAuthorityCount * ctypes.sizeof(wintypes.DWORD)
if sid_size > header.AceSize - ctypes.sizeof(ACE_HEADER) - ctypes.sizeof(wintypes.DWORD) or not advapi32.IsValidSid(sid_pointer) or advapi32.GetLengthSid(sid_pointer) != sid_size:
raise RuntimeError("Cutter bridge ACL contains an invalid SID")
ace = ctypes.cast(ace_pointer, ctypes.POINTER(ACCESS_ALLOWED_ACE)).contents
sid_text = wintypes.LPWSTR()
if ace.Mask != 0x001F01FF or ace.Header.AceFlags != 0x03:
raise RuntimeError("Cutter bridge ACL contains an unsupported ACE")
if not advapi32.ConvertSidToStringSidW(sid_pointer, ctypes.byref(sid_text)):
raise ctypes.WinError(ctypes.get_last_error())
try:
if sid_text.value == expected_user:
user_allowed = True
elif sid_text.value == "S-1-5-18":
system_allowed = True
else:
raise RuntimeError("Cutter bridge ACL allows an unrelated principal")
finally:
kernel32.LocalFree(sid_text)
if not user_allowed or not system_allowed:
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
finally:
kernel32.LocalFree(descriptor)
def _windows_security_attributes() -> tuple[Any, Any]:
kernel32, advapi32 = _windows_api()
sid = _windows_user_sid_text()
sddl = f"O:{sid}G:{sid}D:P(A;OICI;FA;;;{sid})(A;OICI;FA;;;SY)"
descriptor = wintypes.LPVOID()
if not advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW(sddl, 1, ctypes.byref(descriptor), None):
raise ctypes.WinError(ctypes.get_last_error())
class SECURITY_ATTRIBUTES(ctypes.Structure):
_fields_ = [("nLength", wintypes.DWORD), ("lpSecurityDescriptor", wintypes.LPVOID), ("bInheritHandle", wintypes.BOOL)]
attributes = SECURITY_ATTRIBUTES(ctypes.sizeof(SECURITY_ATTRIBUTES), descriptor, False)
return kernel32, (descriptor, attributes)
def _ensure_windows_bridge_directory(directory: Path) -> None:
directory.parent.mkdir(parents=True, exist_ok=True)
if not directory.exists():
kernel32, (descriptor, attributes) = _windows_security_attributes()
try:
if not kernel32.CreateDirectoryW(str(directory), ctypes.byref(attributes)):
error = ctypes.get_last_error()
if error != 183:
raise ctypes.WinError(error)
finally:
kernel32.LocalFree(descriptor)
_verify_windows_private_path(directory, True)
def _ensure_posix_bridge_directory(directory: Path) -> None:
directory.mkdir(mode=0o700, parents=True, exist_ok=True)
if not hasattr(os, "O_NOFOLLOW") or not hasattr(os, "O_DIRECTORY"):
raise RuntimeError("Cutter bridge directory requires no-follow directory handles")
descriptor = os.open(directory, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
try:
directory_info = os.fstat(descriptor)
if directory_info.st_uid != os.getuid():
raise RuntimeError("Cutter bridge directory must be owned by the current user")
os.fchmod(descriptor, 0o700)
directory_info = os.fstat(descriptor)
if stat.S_IMODE(directory_info.st_mode) != 0o700:
raise RuntimeError("Cutter bridge directory must have mode 0700")
finally:
os.close(descriptor)
def _write_windows_descriptor(directory: Path, destination: Path, payload: bytes) -> None:
if len(payload) > MAX_DESCRIPTOR_BYTES:
raise RuntimeError("Cutter bridge descriptor exceeds its byte limit")
kernel32, (security_descriptor, attributes) = _windows_security_attributes()
temporary = directory / f".{destination.name}.{uuid.uuid4().hex}.tmp"
handle = wintypes.HANDLE()
try:
handle = kernel32.CreateFileW(str(temporary), 0x40000000 | 0x00020000 | 0x00040000,
0x1 | 0x2 | 0x4, ctypes.byref(attributes), 1, 0x80, None)
if handle == wintypes.HANDLE(-1).value:
raise ctypes.WinError(ctypes.get_last_error())
_verify_windows_private_path(temporary, False)
if payload:
data = ctypes.create_string_buffer(payload)
written = wintypes.DWORD()
if not kernel32.WriteFile(handle, data, len(payload), ctypes.byref(written), None) or written.value != len(payload):
raise ctypes.WinError(ctypes.get_last_error())
if not kernel32.FlushFileBuffers(handle):
raise ctypes.WinError(ctypes.get_last_error())
kernel32.CloseHandle(handle)
handle = wintypes.HANDLE()
if not kernel32.MoveFileExW(str(temporary), str(destination), 0x1 | 0x8):
raise ctypes.WinError(ctypes.get_last_error())
finally:
if handle and handle != wintypes.HANDLE(-1).value:
kernel32.CloseHandle(handle)
kernel32.DeleteFileW(str(temporary))
kernel32.LocalFree(security_descriptor)
def _bridge_directory() -> Path:
configured = os.environ.get("REA_CUTTER_BRIDGE_DIR")
if configured:
return Path(configured).expanduser()
local_app_data = os.environ.get("LOCALAPPDATA")
if os.name == "nt" and local_app_data:
return Path(local_app_data) / "REA" / "CutterBridge"
return Path.home() / ".cache" / "rea" / "cutter-bridge"
class _RequestDispatcher(QObject):
dispatch = Signal(object, object)
def __init__(self) -> None:
super().__init__()
self.dispatch.connect(self._handle)
self.generation = 0
self.completed_commands: dict[str, str] = {}
self._receipt_lock = threading.Lock()
self.current_file = self._read_current_file()
try:
self.cutter_version = str(cutter.core().getVersionInformation())
except Exception:
self.cutter_version = None
@staticmethod
def _read_current_file() -> str | None:
try:
value = cutter.core().currentlyOpenFile()
return str(value) if value else None
except Exception:
return None
def _handle(self, request: dict[str, Any], reply: dict[str, Any]) -> None:
kind = request.get("kind")
request_id = request.get("request_id")
if kind in ("command", "ack"):
try:
valid_request_id = (
isinstance(request_id, str)
and str(uuid.UUID(request_id)) == request_id.lower()
)
except (ValueError, AttributeError):
valid_request_id = False
if not valid_request_id:
reply.update({
"ok": False,
"error": "invalid-command-request-id",
"execution_state": "not_started",
})
request["_event"].set()
return
if kind == "ack":
with self._receipt_lock:
acknowledged = self.completed_commands.pop(request_id, None) is not None
reply.update({"ok": True, "acknowledged": acknowledged})
request["_event"].set()
return
if request["_cancelled"].is_set():
reply.update({
"ok": False,
"error": "request-cancelled-before-execution",
"execution_state": "not_started",
})
if kind == "command":
self._record_command_completion(request_id, "not_started")
request["_event"].set()
return
self._refresh_document_identity()
if kind == "status":
with self._receipt_lock:
completed_commands = [
{"request_id": receipt_id, "execution_state": state}
for receipt_id, state in self.completed_commands.items()
]
reply.update({
"ok": True,
"session_id": request.get("session_id"),
"current_file": self.current_file,
"document_generation": self.generation,
"cutter_version": self.cutter_version,
"identity_status": "partial",
"completed_commands": completed_commands,
})
request["_event"].set()
return
if kind != "command":
reply.update({"ok": False, "error": "unsupported-request-kind"})
request["_event"].set()
return
if request.get("expected_generation") != self.generation:
reply.update({
"ok": False,
"error": "active-document-generation-changed",
"execution_state": "not_started",
})
self._record_command_completion(request_id, "not_started")
request["_event"].set()
return
command = request.get("command")
if not isinstance(command, str) or not command.strip():
reply.update({
"ok": False,
"error": "invalid-command",
"execution_state": "not_started",
})
self._record_command_completion(request_id, "not_started")
request["_event"].set()
return
reply["execution_state"] = "running"
try:
output = cutter.cmdj(command) if request.get("json") is True else cutter.cmd(command)
command_error = None
except Exception as error:
output = None
command_error = str(error)
self._refresh_document_identity()
if command_error is None:
reply.update({
"ok": True,
"output": output,
"current_file": self.current_file,
"document_generation": self.generation,
"cutter_version": self.cutter_version,
"identity_status": "partial",
"execution_state": "complete",
})
self._record_command_completion(request_id, "complete")
else:
reply.update({
"ok": False,
"error": command_error,
"current_file": self.current_file,
"document_generation": self.generation,
"cutter_version": self.cutter_version,
"identity_status": "partial",
"execution_state": "unknown",
"message": "The command raised an exception; it may have produced partial effects. Do not retry automatically.",
})
self._record_command_completion(request_id, "failed")
request["_event"].set()
def _record_command_completion(
self, request_id: str, execution_state: str
) -> None:
with self._receipt_lock:
if request_id in self.completed_commands or len(self.completed_commands) < MAX_COMPLETION_RECEIPTS:
self.completed_commands[request_id] = execution_state
def can_accept_command(self) -> bool:
with self._receipt_lock:
return len(self.completed_commands) < MAX_COMPLETION_RECEIPTS
def _refresh_document_identity(self) -> None:
current_file = self._read_current_file()
if current_file != self.current_file:
self.current_file = current_file
self.generation += 1
class _BridgeServer(threading.Thread):
def __init__(self, dispatcher: _RequestDispatcher) -> None:
super().__init__(name="rea-cutter-bridge", daemon=True)
self.dispatcher = dispatcher
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server.bind(("127.0.0.1", 0))
self.server.listen(8)
self.server.settimeout(0.5)
self.port = self.server.getsockname()[1]
self.session_id = str(uuid.uuid4())
self.token = secrets.token_urlsafe(32)
self.poisoned = False
self.connection_ordinal = 0
self.directory = _bridge_directory()
self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
def publish(self) -> None:
if os.name == "nt":
_ensure_windows_bridge_directory(self.directory)
else:
_ensure_posix_bridge_directory(self.directory)
if os.name == "nt" and (self.directory.is_symlink() or not self.directory.is_dir()):
raise RuntimeError("Cutter bridge directory must be a real directory")
payload = {
"session_id": self.session_id,
"pid": os.getpid(),
"host": "127.0.0.1",
"port": self.port,
"token": self.token,
"document_generation": self.dispatcher.generation,
"current_file": self.dispatcher.current_file,
"cutter_version": self.dispatcher.cutter_version,
"identity_status": "partial",
}
if os.name == "nt":
_write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
return
descriptor, temporary_path = tempfile.mkstemp(prefix=f".{self.descriptor.name}.", dir=self.directory)
try:
os.fchmod(descriptor, 0o600)
descriptor_info = os.fstat(descriptor)
if descriptor_info.st_uid != os.getuid() or stat.S_IMODE(descriptor_info.st_mode) & 0o077:
raise RuntimeError("Cutter bridge descriptor must be owned by the current user with mode 0600")
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
stream.write(json.dumps(payload))
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary_path, self.descriptor)
except Exception:
try:
os.close(descriptor)
except OSError:
pass
try:
os.unlink(temporary_path)
except OSError:
pass
raise
def run(self) -> None:
try:
self.publish()
except Exception:
self.server.close()
return
while True:
try:
connection, _ = self.server.accept()
except socket.timeout:
if getattr(self, "stopping", False):
break
continue
except OSError:
break
self.connection_ordinal += 1
connection_ordinal = self.connection_ordinal
with connection:
connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
authenticated = False
try:
hello = self._receive_frame(connection)
if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
self._send(connection, {"ok": False, "error": "authentication-failed"})
continue
client_nonce = hello["nonce"]
server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
authenticated_frame = self._receive_frame(connection)
request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
self._send(connection, {"ok": False, "error": "authentication-failed"})
continue
request = json.loads(request_bytes)
if not isinstance(request, dict) or request.get("session_id") != self.session_id:
self._send(connection, {"ok": False, "error": "authentication-failed"})
continue
authenticated = True
connection.settimeout(SOCKET_TIMEOUT_SECONDS)
if self.poisoned and request.get("kind") not in ("status", "ack"):
self._send_authenticated(connection, {
"ok": False,
"error": "session-poisoned-after-ui-timeout",
"execution_state": "not_started",
}, client_nonce, server_nonce)
continue
if (
request.get("kind") == "command"
and not self.dispatcher.can_accept_command()
):
self._send_authenticated(connection, {
"ok": False,
"error": "completion-receipt-capacity-exceeded",
"execution_state": "not_started",
}, client_nonce, server_nonce)
continue
request["session_id"] = self.session_id
event = threading.Event()
cancelled = threading.Event()
reply: dict[str, Any] = {}
request["_event"] = event
request["_cancelled"] = cancelled
self.dispatcher.dispatch.emit(request, reply)
if not event.wait(SOCKET_TIMEOUT_SECONDS):
cancelled.set()
self.poisoned = True
self._send_authenticated(connection, {
"ok": False,
"error": "ui-thread-request-timeout",
"execution_state": "unknown",
"message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
}, client_nonce, server_nonce)
else:
if request.get("kind") == "status" and reply.get("ok") is True:
reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
self._send_authenticated(connection, reply, client_nonce, server_nonce)
except socket.timeout:
if authenticated:
self._send_authenticated(connection, {
"ok": False,
"error": "request-timeout",
"execution_state": "unknown",
"message": "The command may have produced partial or persistent effects. Do not retry automatically.",
}, client_nonce, server_nonce)
except Exception as error:
self._send(connection, {"ok": False, "error": str(error)})
self.server.close()
try:
self.descriptor.unlink(missing_ok=True)
except OSError:
pass
@staticmethod
def _receive_frame(connection: socket.socket) -> dict[str, Any]:
data = bytearray()
deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
while b"\n" not in data:
if len(data) >= MAX_FRAME_BYTES:
raise ValueError("frame-limit-exceeded")
remaining = deadline - time.monotonic()
if remaining <= 0:
raise socket.timeout("authentication frame timed out")
connection.settimeout(remaining)
part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
if not part:
raise ValueError("incomplete authentication frame")
data.extend(part)
line, remainder = bytes(data).split(b"\n", 1)
if len(line) > MAX_FRAME_BYTES or remainder:
raise ValueError("invalid framed request")
value = json.loads(line)
return value if isinstance(value, dict) else {}
def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
value = self._truncated_response(value)
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
# This fixed-size fallback is checked through the same signed framing path.
value = {
"ok": False,
"error": "response-limit-exceeded",
"execution_state": value.get("execution_state", "unknown"),
"output": None,
"output_truncated": True,
"message": "The result exceeded the bridge frame limit; command completion state is preserved.",
}
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
if encoded is None:
return
try:
connection.sendall(encoded)
except OSError:
pass
def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
if not _within_response_limit(value):
return None
prefix = b'{"response":"'
suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
frame_size = len(prefix) + len(suffix)
response_parts: list[bytes] = []
escaped_parts: list[bytes] = []
for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
raw = fragment.encode("utf-8")
escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
frame_size += len(escaped)
if frame_size > MAX_FRAME_BYTES:
return None
response_parts.append(raw)
escaped_parts.append(escaped)
proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
for part in response_parts:
proof.update(part)
return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
@staticmethod
def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
response: dict[str, Any] = {
"ok": value.get("ok") is True,
"output": None,
"execution_state": value.get("execution_state", "unknown"),
"output_truncated": True,
"message": "The command result exceeded the bridge frame limit; completion state is preserved and output was omitted.",
}
for key in ("current_file", "document_generation", "cutter_version", "identity_status"):
if key in value:
response[key] = value[key]
if value.get("ok") is not True and isinstance(value.get("error"), str):
response["error"] = value["error"]
return response
@staticmethod
def _send(connection: socket.socket, value: dict[str, Any]) -> None:
if not _within_response_limit(value):
value = {
"ok": False,
"error": "response-limit-exceeded",
"execution_state": value.get("execution_state", "unknown"),
"output_truncated": True,
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
}
parts: list[bytes] = []
encoded_size = 0
for part in json.JSONEncoder(ensure_ascii=False).iterencode(value):
encoded_part = part.encode("utf-8")
encoded_size += len(encoded_part)
if encoded_size > MAX_FRAME_BYTES:
fallback = {
"ok": False,
"error": "response-limit-exceeded",
"execution_state": value.get("execution_state", "unknown"),
"output_truncated": True,
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
}
parts = [json.dumps(fallback, separators=(",", ":")).encode("utf-8")]
break
parts.append(encoded_part)
encoded = b"".join(parts)
try:
connection.sendall(encoded + b"\n")
except OSError:
# A disconnected caller must not terminate the bridge thread or
# prevent later commands from being served.
pass
def stop(self) -> None:
self.stopping = True
try:
self.server.close()
except OSError:
pass
class ReaCutterPlugin(cutter.CutterPlugin):
name = "REA Local Bridge"
description = "Local REA analysis bridge for the active Cutter session"
version = "1.0"
author = "REA"
def __init__(self) -> None:
super().__init__()
self._dispatcher: _RequestDispatcher | None = None
self._bridge: _BridgeServer | None = None
def setupPlugin(self) -> None:
return
def setupInterface(self, main: Any) -> None:
self._dispatcher = _RequestDispatcher()
self._bridge = _BridgeServer(self._dispatcher)
self._bridge.start()
def terminate(self) -> None:
if self._bridge is not None:
self._bridge.stop()
self._bridge.join(timeout=2)
self._bridge = None
self._dispatcher = None
def create_cutter_plugin() -> ReaCutterPlugin:
return ReaCutterPlugin()
def _within_response_limit(value: Any) -> bool:
pending = [value]
seen: set[int] = set()
size = 0
while pending:
current = pending.pop()
if isinstance(current, str):
size += 2
if current.isascii():
size += len(current)
size += current.count('"') + current.count("\\")
size += sum(5 for character in current if ord(character) < 0x20)
else:
size += 4 * len(current)
size += current.count('"') + current.count("\\")
size += sum(5 for character in current if ord(character) < 0x20)
elif isinstance(current, dict):
identity = id(current)
if identity in seen:
return False
seen.add(identity)
size += 2 + 4 * len(current)
if size > MAX_FRAME_BYTES:
return False
pending.extend(current.keys())
pending.extend(current.values())
elif isinstance(current, (list, tuple)):
identity = id(current)
if identity in seen:
return False
seen.add(identity)
size += 2 + 2 * len(current)
if size > MAX_FRAME_BYTES:
return False
pending.extend(current)
else:
size += 16
if size > MAX_FRAME_BYTES:
return False
return True
+79 -275
View File
@@ -81,7 +81,6 @@ import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.FunctionIterator;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.listing.InstructionIterator;
import ghidra.program.model.listing.Listing;
import ghidra.program.model.listing.Variable;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
@@ -406,9 +405,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
case "xrefs" -> xrefs(request.params);
case "analyze_function" -> {
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
JsonObject result = functionObservation(observation.dossier(), observation.limitations());
result.add("relationship_coverage", observation.relationshipCoverage());
yield result;
yield functionObservation(observation.dossier(), observation.limitations());
}
case "set_address_name", "set_addresses_names" -> {
if (!descriptor.transport.equals("unix-socket"))
@@ -1019,19 +1016,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Function item : ordered) {
result.add(canonicalAddress(item.getEntryPoint()));
}
if (callers) {
JsonObject observation = functionObservation(result, noReturnCallLimitations(
return functionObservation(
result,
noReturnCallLimitations(
currentProgram.getListing().getInstructions(function.getBody(), true)
));
observation.add("relationship_coverage", callRelationshipCoverage(null,
unrepresentedIncomingCalls(function, observed)));
monitor.checkCancelled();
return observation;
}
CallBoundaryObservation calls = callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), observed, false, null
)
);
return callRelationshipObservation(result, calls);
}
private JsonObject procedureInfo(JsonObject params) throws Exception {
@@ -1081,10 +1071,18 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.addProperty("direction", direction);
result.add("references", edges);
result.addProperty("reference_kinds_available", true);
JsonArray unresolvedCalls = direction.equals("outgoing")
? callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
).unresolvedCalls() : new JsonArray();
JsonArray unresolvedCalls = new JsonArray();
if (direction.equals("outgoing")) for (Instruction instruction : scanInstructions(function, Integer.MAX_VALUE).instructions) {
if (!instruction.getFlowType().isCall()) continue;
boolean resolved = false;
for (Reference reference : instruction.getReferencesFrom()) if (reference.getReferenceType().isCall()) resolved = true;
if (!resolved) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(instruction.getAddress()));
site.addProperty("reason", "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
unresolvedCalls.add(site);
}
}
result.add("unresolved_calls", unresolvedCalls);
return result;
}
@@ -1492,28 +1490,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
DecompileResults decompilation = decompile(function);
String pseudocode = decompiledText(decompilation);
JsonObject availability = new JsonObject();
availability.add("pseudocode", facetAvailability(
pseudocode != null,
function.isExternal() ? "Ghidra does not decompile external functions."
: function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
: "Ghidra completed decompilation without returning pseudocode."
));
if (pseudocode == null) {
pseudocode = "";
}
List<Reference> bodyIncoming = collectReferences(function, "incoming");
List<Reference> incomingReferences = bodyIncoming.stream()
List<Reference> incomingReferences = collectReferences(function, "incoming")
.stream()
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
.toList();
List<Reference> outgoingReferences = collectReferences(function, "outgoing");
JsonArray incoming = referenceEdges(incomingReferences);
JsonArray outgoing = referenceEdges(outgoingReferences);
JsonArray comments = comments(function);
Set<Function> callerFunctions = function.getCallingFunctions(monitor);
JsonArray callers = procedureIdentities(callerFunctions);
Set<Function> calleeFunctions = function.getCalledFunctions(monitor);
JsonArray callees = procedureIdentities(calleeFunctions);
JsonArray comments = comments(scan.instructions);
JsonArray callers = procedureIdentities(function.getCallingFunctions(monitor));
JsonArray callees = procedureIdentities(function.getCalledFunctions(monitor));
JsonArray referencedStrings = referencedStrings(outgoingReferences);
JsonArray referencedNames = referencedNames(outgoingReferences);
JsonArray blocks = basicBlockValues(function);
@@ -1547,15 +1536,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
nativeApiBoundary(function, decompilation)
);
result.add("native_value_flow", nativeValueFlow(decompilation));
// Successful full queries distinguish collected empty sets from missing facts.
// Availability does not claim that Ghidra resolved every runtime relationship.
for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
availability.add(facet, facetAvailability(true, null));
}
availability.add("assembly", facetAvailability(
!scan.truncated(), "The function instruction scan was truncated."
));
result.add("facet_availability", availability);
JsonArray limitations = new JsonArray();
limitations.add(
"Unresolved computed or indirect flows without target addresses are not represented as reference edges."
@@ -1583,10 +1563,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
);
}
CallBoundaryObservation calls = callBoundaryObservation(
function, scan.instructions, calleeFunctions, true, outgoingReferences);
result.add("unresolved_calls", calls.unresolvedCalls());
for (String limitation : calls.limitations()) {
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
for (String limitation : boundaryLimitations) {
limitations.add(limitation);
}
if (decompilation != null && decompilation.getErrorMessage() != null &&
@@ -1594,14 +1572,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
}
result.add("limitations", limitations);
monitor.checkCancelled();
int missingCallers = 0;
for (Reference reference : bodyIncoming) {
monitor.checkCancelled();
if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
}
return new FunctionAnalysisObservation(result, calls.limitations(),
callRelationshipCoverage(calls, missingCallers));
return new FunctionAnalysisObservation(result, boundaryLimitations);
}
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
@@ -3305,200 +3276,52 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonObject facetAvailability(boolean available, String reason) {
JsonObject result = new JsonObject();
result.addProperty("available", available);
if (!available) result.addProperty("reason", reason);
return result;
}
private JsonObject callRelationshipObservation(JsonElement value, CallBoundaryObservation calls)
throws Exception {
monitor.checkCancelled();
JsonObject result = functionObservation(value, calls.limitations());
result.add("relationship_coverage", callRelationshipCoverage(calls, null));
return result;
}
private static JsonObject callRelationshipCoverage(
CallBoundaryObservation calls, Integer missingCallers) {
JsonObject coverage = completeResolvedCallCoverage();
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
JsonArray directions = new JsonArray();
if (missingCallers != null && missingCallers > 0) directions.add("callers");
String callerLimitation = missingCallers != null && missingCallers > 0
? "Ghidra's caller inventory omits sources of " + missingCallers +
" call reference(s) into this function's body." : null;
if ((calls != null && omitted == null) || (calls == null && missingCallers == null)) {
coverage.addProperty("status", "unknown");
String reason = calls == null ? "No call inventory was examined."
: "The callee inventory was not examined alongside decoded call targets.";
coverage.addProperty("reason", reason + (callerLimitation == null ? "" : " " + callerLimitation));
if (calls == null && missingCallers == null) directions.add("callers");
directions.add("callees");
coverage.add("limited_directions", directions);
return coverage;
}
JsonArray limitations = new JsonArray();
if (callerLimitation != null) limitations.add(callerLimitation);
if (omitted != null && omitted > 0) {
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
" call reference(s) or decoded call(s) without destinations.");
directions.add("callees");
}
if (!limitations.isEmpty()) {
coverage.addProperty("status", "partial");
coverage.add("limitations", limitations);
coverage.add("limited_directions", directions);
}
return coverage;
}
private static JsonObject completeResolvedCallCoverage() {
JsonObject coverage = new JsonObject();
coverage.addProperty("status", "complete");
JsonArray diagnostics = new JsonArray();
diagnostics.add(
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals."
);
coverage.add("diagnostics", diagnostics);
return coverage;
}
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
if (!reference.getReferenceType().isCall()) return false;
Function caller = containingFunction(reference.getFromAddress());
return caller == null || !callers.contains(caller);
}
private int unrepresentedIncomingCalls(Function function, Set<Function> callers)
throws Exception {
int[] omitted = { 0 };
visitReferences(function, "incoming", reference -> {
if (unrepresentedCaller(reference, callers)) omitted[0]++;
});
return omitted[0];
}
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
throws Exception {
List<String> limitations = new ArrayList<>();
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
}
return limitations;
}
private CallBoundaryObservation callBoundaryObservation(
Function function, Iterable<Instruction> instructions, Set<Function> returnedCallees,
boolean includeUnresolvedSites, List<Reference> observedReferences)
throws Exception {
List<String> limitations = new ArrayList<>();
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
else for (Reference reference : observedReferences) {
monitor.checkCancelled();
calls.observe(reference);
}
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
destination = true;
break;
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
continue;
}
if (!destination) {
calls.unrepresented++;
calls.unresolved(instruction.getAddress(),
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
}
return new CallBoundaryObservation(List.copyOf(limitations),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private final class CallBoundaryReferences {
private final Set<Function> returnedCallees;
private final TreeMap<Address, String> unresolved;
private int unrepresented;
CallBoundaryReferences(Set<Function> returnedCallees, boolean includeUnresolvedSites) {
this.returnedCallees = returnedCallees;
unresolved = includeUnresolvedSites ? new TreeMap<>() : null;
}
void observe(Reference reference) {
if (!reference.getReferenceType().isCall()) return;
// Reference identity follows the same containing-function owner as referenceEdge.
Function target = containingFunction(reference.getToAddress());
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
unrepresented++;
if (target == null) unresolved(reference.getFromAddress(),
"A call destination has no known containing Ghidra procedure");
}
void unresolved(Address address, String reason) {
if (unresolved != null) unresolved.put(address, reason);
}
JsonArray unresolvedCalls() {
if (unresolved == null) return null;
JsonArray result = new JsonArray();
for (var item : unresolved.entrySet()) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(item.getKey()));
site.addProperty("reason", item.getValue());
result.add(site);
}
return result;
}
}
private String terminalCallLimitation(Instruction instruction) {
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal())
return null;
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
}
}
limitations.add(limitation.toString());
}
return limitation.toString();
return limitations;
}
private static String renderAssembly(List<Instruction> instructions) {
@@ -3524,27 +3347,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List<Reference> collectReferences(
Function function,
String direction) throws Exception {
TreeMap<String, Reference> observed = new TreeMap<>();
visitReferences(function, direction, reference -> addReference(observed, reference));
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
@FunctionalInterface
private interface ReferenceVisitor {
void accept(Reference reference) throws Exception;
}
private void visitReferences(Function function, String direction, ReferenceVisitor visitor)
throws Exception {
// References can target instruction interiors or originate in embedded
// data. Use actual body ownership, including an external's exact entry.
AddressSet scope = functionObservationScope(function);
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
boolean outgoing = direction.equals("outgoing");
AddressIterator addresses = outgoing
? currentProgram.getReferenceManager().getReferenceSourceIterator(scope, true)
: currentProgram.getReferenceManager().getReferenceDestinationIterator(scope, true);
TreeMap<String, Reference> observed = new TreeMap<>();
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
@@ -3552,7 +3363,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Reference reference : currentProgram.getReferenceManager()
.getReferencesFrom(address)) {
monitor.checkCancelled();
if (!reference.isEntryPointReference()) visitor.accept(reference);
addReference(observed, reference);
}
}
else {
@@ -3560,14 +3371,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
.getReferencesTo(address);
while (iterator.hasNext()) {
monitor.checkCancelled();
Reference reference = iterator.next();
if (!reference.isEntryPointReference()) visitor.accept(reference);
addReference(observed, iterator.next());
}
}
}
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
private static void addReference(TreeMap<String, Reference> observed, Reference reference) {
if (reference.isEntryPointReference()) {
return;
}
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
canonicalAddress(reference.getToAddress()) + "\u0000" +
reference.getReferenceType().getName() + "\u0000" +
@@ -3576,12 +3392,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
observed.putIfAbsent(key, reference);
}
private static AddressSet functionObservationScope(Function function) {
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
return scope;
}
private JsonArray referenceEdges(List<Reference> references) {
JsonArray result = new JsonArray();
for (Reference reference : references) {
@@ -3735,28 +3545,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private JsonArray comments(Function function) throws Exception {
Listing listing = currentProgram.getListing();
AddressSet scope = functionObservationScope(function);
AddressIterator addresses = listing.getCommentAddressIterator(scope, true);
private static JsonArray comments(List<Instruction> instructions) {
TreeMap<String, JsonObject> observed = new TreeMap<>();
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
for (Instruction instruction : instructions) {
for (CommentType type : CommentType.values()) {
String text = listing.getComment(type, address);
String text = instruction.getComment(type);
if (text == null || text.isEmpty()) {
continue;
}
String kind = type == CommentType.EOL ? "inline" : "comment";
JsonObject item = new JsonObject();
item.addProperty("address", canonicalAddress(address));
item.addProperty("address", canonicalAddress(instruction.getAddress()));
item.addProperty("kind", kind);
item.addProperty("text", text);
String key = canonicalAddress(address) + "\u0000" +
String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
kind + "\u0000" + type.name() + "\u0000" + text;
observed.put(key, item);
}
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
@@ -4692,8 +4497,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
private record InventoryItem(Address address, String value, JsonObject facts) {}
private record AddressNameRequest(String inputAddress, Address address, String name) {}
private record FunctionEntry(Function function, InventoryItem item) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
+2 -8
View File
@@ -644,7 +644,7 @@ def _analyze_function(document, params):
"end": _hex(_basic_block_end(procedure, block)),
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
})
pseudo = _pseudocode(document, procedure)
pseudo = _pseudocode(document, procedure) or ""
assembly_lines = _assembly(procedure).splitlines()
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
@@ -706,7 +706,7 @@ def _analyze_function(document, params):
"signature": procedure.signatureString(),
"locals": _procedure_locals(procedure),
},
"pseudocode": pseudo or "",
"pseudocode": pseudo,
"assembly": assembly_lines,
"comments": comments,
"callers": callers, "callees": callees,
@@ -718,12 +718,6 @@ def _analyze_function(document, params):
"basic_blocks": blocks,
"native_api": None,
"native_value_flow": None,
"facet_availability": {
"pseudocode": {"available": True} if pseudo is not None else _unavailable("Hopper did not supply pseudocode."),
**{facet: {"available": True} for facet in (
"assembly", "comments", "callers", "callees", "references", "strings_names", "cfg"
)},
},
"limitations": [
"Native CallReference classifications are observed where available; detailed reference flags remain unknown.",
"Hopper's public Python API does not expose equivalent external or thunk classification in this dossier.",
+12 -133
View File
@@ -11,7 +11,6 @@ import math
import re
import resource
from pathlib import Path
from urllib.parse import unquote_to_bytes
from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
@@ -42,113 +41,6 @@ class BoundedReader:
return self.handle.read(count)
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
URL_IGNORABLE_CONTROLS = "\t\n\r"
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
FORBIDDEN_DOMAIN_CODE_POINTS = {chr(code) for code in range(0x21)} | set("#%/:<>?@[\\]^|\x7f")
def authority_start(text, base_scheme=None):
"""Return where an authority begins and whether it is special, or None.
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
special schemes (and scheme-relative references) skip every leading "/"
or "\\" before the authority, so `https://\\user:pass@host` still has
userinfo. File URLs never carry userinfo: an "@" in a file host makes
WHATWG reject the whole URL, so there are no credentials to remove.
"""
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
name = scheme.group(1).lower() if scheme is not None else None
if name == "file" or (name is None and base_scheme == "file"):
return None
if name is not None and name in SPECIAL_SCHEMES:
# A same-scheme Location without two slash-equivalent separators is
# relative to the recorded request, so its @ belongs to the path.
following = text[scheme.end():scheme.end() + 2]
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
return None
start = scheme.end()
else:
if scheme is not None:
if text[scheme.end():scheme.end() + 2] != "//":
return None
return scheme.end() + 2, False
# A non-special base treats backslashes as authority data and only
# recognizes two forward slashes as a network reference.
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
prefix = re.match(r"^[\x00-\x20]*//", text)
return None if prefix is None else (prefix.end(), False)
prefix = re.match(NETWORK_PREFIX, text)
if prefix is None:
return None
start = prefix.end()
while start < len(text) and text[start] in "/\\":
start += 1
return start, True
def special_host_parses(host_port):
"""Whether WHATWG could accept a special authority's host and port.
A rejected host fails the whole URL, so it has no credentials to remove.
This mirrors the forbidden-code-point, empty-host and port rules; IDNA,
IPv4 range and IPv6 syntax failures are not modeled and still redact.
"""
if host_port.startswith("["):
host_end = host_port.find("]") + 1
if host_end == 0:
return False
else:
colon = host_port.find(":")
host_end = len(host_port) if colon < 0 else colon
host = unquote_to_bytes(host_port[:host_end]).decode("utf-8", errors="replace")
if host == "" or any(char in FORBIDDEN_DOMAIN_CODE_POINTS for char in host):
return False
port = host_port[host_end:]
if port == "":
return True
digits = port[1:]
return port[0] == ":" and (digits == "" or (digits.isascii() and digits.isdigit() and int(digits) <= 65535))
def without_userinfo(value, base_scheme=None):
"""Remove an authority's userinfo using the WHATWG authority boundary.
Special schemes and scheme-relative references also end the authority
at a backslash, so later path text containing "@" is not userinfo.
"""
binary = isinstance(value, bytes)
text = value.decode("latin-1") if binary else value
# WHATWG removes these controls before interpreting scheme/separators.
# Keep that parse view separate from the exact source spelling and bytes.
parsed_text = text.translate(URL_CONTROL_TRANSLATION)
located = authority_start(parsed_text, base_scheme)
if located is None:
return value
start, special = located
terminators = "/?#\\" if special else "/?#"
end = start
while end < len(parsed_text) and parsed_text[end] not in terminators:
end += 1
at = parsed_text.rfind("@", start, end)
if at < start or (special and not special_host_parses(parsed_text[at + 1 : end])):
return value
parsed_offset = 0
raw_start = 0
for raw_offset, char in enumerate(text):
if char in URL_IGNORABLE_CONTROLS:
continue
if parsed_offset == start:
raw_start = raw_offset
if parsed_offset == at:
safe = text[:raw_start] + text[raw_offset + 1 :]
return safe.encode("latin-1") if binary else safe
parsed_offset += 1
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
@@ -166,23 +58,16 @@ def project_record(state, secrets):
return None
return text
def transport_value(value, path, transport_url, base_scheme):
def transport_value(value, path, transport_url):
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
return value, False
binary = isinstance(value, bytes)
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
replacement = rb"\1" if binary else r"\1"
separator = b"@" if binary else "@"
if transport_url:
safe_url = without_userinfo(value, base_scheme)
elif path.endswith("/authority"):
safe_url = value.rsplit(separator, 1)[-1]
else:
# Native request.path is origin-form unless it explicitly carries
# a scheme (absolute-form). Leading separators are path evidence,
# not a header URL reference's network authority.
text = value.decode("latin-1") if binary else value
safe_url = without_userinfo(value) if re.match(
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
) else value
safe_url = re.sub(pattern, replacement, value, flags=re.I)
if path.endswith("/authority") and separator in safe_url:
safe_url = safe_url.rsplit(separator, 1)[1]
changed = safe_url != value
if changed:
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -195,7 +80,7 @@ def project_record(state, secrets):
del numbers[before[1]:]
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
def visit(value, path, depth=0, credential=False, transport_url=False):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
@@ -205,7 +90,7 @@ def project_record(state, secrets):
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
return None
if isinstance(value, bytes):
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
safe_bytes, credential_url = transport_value(value, path, transport_url)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
@@ -222,7 +107,7 @@ def project_record(state, secrets):
return None
return safe_text
if isinstance(value, str):
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
safe_text, _ = transport_value(value, path, transport_url)
return redact(safe_text, path, value)
if isinstance(value, bool) or value is None:
return value
@@ -240,16 +125,10 @@ def project_record(state, secrets):
for index, field in enumerate(value):
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
return result
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
if isinstance(value, dict):
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
request = value.get("request")
scheme = request.get("scheme") if isinstance(request, dict) else None
if isinstance(scheme, bytes):
scheme = scheme.decode("latin-1")
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
result = {}
for key, item in value.items():
if not isinstance(key, str):
@@ -260,7 +139,7 @@ def project_record(state, secrets):
validate_omitted(item, pointer(path, key), depth + 1)
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
continue
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
result[key] = visit(item, pointer(path, key), depth + 1)
return result
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
-214
View File
@@ -1,214 +0,0 @@
"""Source-owned tests for the mitmproxy adapter's record projection."""
import importlib.util
import os
import sys
import types
import unittest
from unittest import mock
def load_capture():
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
fake_mitmproxy = types.ModuleType("mitmproxy")
fake_mitmproxy.ctx = types.SimpleNamespace()
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
fake_io = types.ModuleType("mitmproxy.io")
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
stand_ins = {
"mitmproxy": fake_mitmproxy,
"mitmproxy.io": fake_io,
"mitmproxy.io.tnetstring": fake_io.tnetstring,
}
spec = importlib.util.spec_from_file_location(
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
)
module = importlib.util.module_from_spec(spec)
with mock.patch.dict(sys.modules, stand_ins):
spec.loader.exec_module(module)
return module
capture = load_capture()
def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
class TransportUserinfoTests(unittest.TestCase):
def test_ignorable_controls_cannot_hide_authority_credentials(self):
for control in [b"\t", b"\n", b"\r"]:
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertNotIn("user", text)
self.assertNotIn("pass", text)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_mixed_network_separators_remove_credentials(self):
for prefix in [b"\\\\", b"/\\", b"\\/"]:
with self.subTest(prefix=prefix):
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
self.assertEqual(text, prefix.decode("ascii") + "host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
self.assertEqual(text, "//host/x")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
self.assertEqual(text, "/\\invoice@customer")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
for value, scheme, expected in [
(b"https://user:pass@host/path", b"https", "https://host/path"),
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
(b"https:user:pass@host/path", b"http", "https:host/path"),
]:
with self.subTest(value=value, scheme=scheme):
text, sidecar, redactions = location(value, scheme)
self.assertEqual(text, expected)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_backup_location_uses_its_own_request_scheme(self):
record = capture.project_record({
"request": {"scheme": b"http"},
"backup": {
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
def test_backslash_ends_a_special_scheme_authority(self):
text, sidecar, redactions = location(b"https://example.test\\part@tail")
self.assertEqual(text, "https://example.test\\part@tail")
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_userinfo_before_a_backslash_is_still_removed(self):
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
self.assertEqual(text, "https://example.test\\path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(
redactions,
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
)
def test_ordinary_userinfo_is_removed(self):
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
self.assertEqual(text, "https://example.test/a@b")
def test_empty_userinfo_marker_is_removed(self):
self.assertEqual(
capture.without_userinfo("https://@host"), "https://host"
)
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
for value, expected in [
(b"https://\\user:pass@host/path", b"https://\\host/path"),
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
(b"https:user:pass@host", b"https:host"),
(b"//\\user:pass@host", b"//\\host"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
self.assertEqual(text, "https://\\host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
for value in [b"///u@h", b"////u@h"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_file_urls_keep_path_text(self):
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
def test_file_urls_reject_userinfo_and_keep_their_source(self):
# WHATWG rejects an "@" in a file host, so nothing parses as credentials.
self.assertEqual(
capture.without_userinfo("file://user:pass@host/x"), "file://user:pass@host/x"
)
for value in [b"\\\\user:pass@host/x", b"/\\user:pass@host/x", b"//user:pass@host/x"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_rejected_special_hosts_keep_their_source(self):
# A rejected host or port fails the whole URL, so it has no userinfo.
for value in [
"https:user@%",
"https://user@",
"https://user:pass@h^st/x",
"https://user@ho st/x",
"https://user@host:99999/x",
"https://user@host:12a",
"https://user@[::1",
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), value)
text, sidecar, redactions = location(b"https://user:pass@h^st/x", b"https")
self.assertEqual(text, "https://user:pass@h^st/x")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_accepted_special_hosts_still_remove_userinfo(self):
for value, expected in [
("https:user@%41", "https:%41"),
("https://user@[::1]:8443/x", "https://[::1]:8443/x"),
("https://user@host:/x", "https://host:/x"),
("wss://user:pass@host:65535", "wss://host:65535"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
if __name__ == "__main__":
unittest.main()
+88 -64
View File
@@ -1,4 +1,18 @@
import { defineConfig } from "vitepress";
import { dirname, relative, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
documentationRewrites,
documentationSections,
} from "./documentation.js";
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
const rewrites: Record<string, string> = {
...documentationRewrites,
"README.md": "documentation.md",
"specs/README.md": "specs/index.md",
"adr/README.md": "adr/index.md",
};
export default defineConfig({
title: "REA",
@@ -6,82 +20,92 @@ export default defineConfig({
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
lang: "en-US",
base: "/rea/",
rewrites,
markdown: {
config(md) {
// Repository links resolve from their physical source. Render them from
// the stable public route instead, including links across categories.
md.core.ruler.after("inline", "documentation-source-links", (state) => {
const physicalPath =
state.env.realPath ??
(state.env.relativePath
? resolve(documentationRoot, state.env.relativePath)
: null);
if (physicalPath === null) return;
const renderedPath =
state.env.path ??
resolve(
documentationRoot,
rewrites[state.env.relativePath] ?? state.env.relativePath,
);
for (const block of state.tokens) {
for (const token of block.children ?? []) {
const attribute =
token.type === "link_open"
? "href"
: token.type === "image"
? "src"
: null;
if (attribute === null) continue;
const destination = token.attrGet(attribute);
if (
!destination ||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
)
continue;
const separator = destination.search(/[?#]/u);
const path =
separator === -1 ? destination : destination.slice(0, separator);
const suffix = separator === -1 ? "" : destination.slice(separator);
const source = relative(
documentationRoot,
resolve(dirname(physicalPath), path),
)
.split(sep)
.join("/");
const rendered = relative(
dirname(renderedPath),
resolve(
documentationRoot,
source.startsWith("public/")
? source.slice("public/".length)
: (rewrites[source] ?? source),
),
)
.split(sep)
.join("/");
token.attrSet(attribute, rendered + suffix);
}
}
});
},
},
srcExclude: ["readme/**"],
themeConfig: {
nav: [
{ text: "Get started", link: "/installation" },
{ text: "Documentation", link: "/documentation" },
{ text: "Reference", link: "/mcp-contracts" },
{ text: "Roadmap", link: "/roadmap" },
],
sidebar: [
{
text: "Getting started",
text: "Documentation directory",
items: [
{ text: "Installation and setup", link: "/installation" },
{ text: "Client configuration", link: "/client-configuration" },
{ text: "CLI and Evidence", link: "/cli" },
{ text: "Agent prompts", link: "/mcp-prompts" },
{ text: "Roadmap", link: "/roadmap" },
],
},
{
text: "Investigation guides",
items: [
{ text: "Native binaries", link: "/native-investigation" },
{ text: "Go build metadata", link: "/go-binaries" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript workflows",
link: "/javascript-application-workflows",
},
{ text: "Browser observation", link: "/browser-observation" },
{
text: "Web network captures",
link: "/web-network-captures",
},
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{ text: "Android applications", link: "/android-analysis" },
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
],
},
{
text: "Provider guides",
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
{
text: "Reference and development",
collapsed: true,
items: [
{ text: "MCP contracts", link: "/mcp-contracts" },
{ text: "Tool design", link: "/tool-design" },
{ text: "Testing", link: "/testing" },
{ text: "Client verification", link: "/client-verification" },
{
text: "Reconstruction readiness",
link: "/reconstruction-readiness",
},
{ text: "Releasing", link: "/releasing" },
{ text: "All guides", link: "/documentation" },
{ text: "Authoring conventions", link: "/development/documentation" },
{ text: "Feature specifications", link: "/specs/" },
{ text: "Architecture decisions", link: "/adr/" },
],
},
...documentationSections.map((section) => ({
text: section.text,
collapsed: section.text !== "Getting started",
items: section.items.map((item) => ({
text: item.text,
link: item.route,
})),
})),
],
search: { provider: "local" },
socialLinks: [
+384
View File
@@ -0,0 +1,384 @@
// Authored guide ownership: physical sources and stable public routes.
export const documentationSections = [
{
text: "Getting started",
items: [
{
text: "Installation and setup",
source: "getting-started/installation.md",
route: "/installation",
},
{
text: "Client configuration and troubleshooting",
source: "getting-started/client-configuration.md",
route: "/client-configuration",
},
{
text: "CLI and Evidence",
source: "getting-started/cli.md",
route: "/cli",
},
{
text: "Guided MCP prompts and completion",
source: "getting-started/mcp-prompts.md",
route: "/mcp-prompts",
},
],
},
{
text: "Native binaries",
items: [
{
text: "Native binary investigation",
source: "guides/native/native-investigation.md",
route: "/native-investigation",
},
{
text: "Go binary build metadata",
source: "guides/native/go-binaries.md",
route: "/go-binaries",
},
{
text: "Offline binary diagnostics",
source: "guides/native/binary-diagnostics.md",
route: "/binary-diagnostics",
},
{
text: "Recorded crash evidence",
source: "guides/native/recorded-crashes.md",
route: "/recorded-crashes",
},
],
},
{
text: "JavaScript and Electron",
items: [
{
text: "JavaScript Application Graph",
source: "guides/javascript/javascript-application-graph.md",
route: "/javascript-application-graph",
},
{
text: "Cross-layer JavaScript application workflows",
source: "guides/javascript/javascript-application-workflows.md",
route: "/javascript-application-workflows",
},
{
text: "JavaScript artifact reconstruction",
source: "guides/javascript/javascript-artifact-reconstruction.md",
route: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript source recovery",
source: "guides/javascript/javascript-recovery.md",
route: "/javascript-recovery",
},
{
text: "Passive Node and Electron runtime observation",
source: "guides/javascript/javascript-runtime-observation.md",
route: "/javascript-runtime-observation",
},
{
text: "JavaScript static/runtime reconciliation",
source: "guides/javascript/javascript-runtime-reconciliation.md",
route: "/javascript-runtime-reconciliation",
},
{
text: "Electron file-page observation",
source: "guides/javascript/electron-observation.md",
route: "/electron-observation",
},
],
},
{
text: "Browser and captured websites",
items: [
{
text: "Website observation with CDP",
source: "guides/web/browser-observation.md",
route: "/browser-observation",
},
{
text: "Browser scenario contract",
source: "guides/web/browser-scenario-contract.md",
route: "/browser-scenario-contract",
},
{
text: "Website runtime source attribution",
source: "guides/web/web-runtime.md",
route: "/web-runtime",
},
{
text: "Historical web network captures",
source: "guides/web/web-network-captures.md",
route: "/web-network-captures",
},
{
text: "Captured website source locations",
source: "guides/web/web-source-location.md",
route: "/web-source-location",
},
{
text: "Captured website module relationships",
source: "guides/web/website-module-trace.md",
route: "/website-module-trace",
},
{
text: "Captured website scripts and static JavaScript analysis",
source: "guides/web/website-script-export.md",
route: "/website-script-export",
},
],
},
{
text: "Android applications",
items: [
{
text: "Static Android APK analysis",
source: "guides/android/android-analysis.md",
route: "/android-analysis",
},
{
text: "Apktool resource decoding",
source: "guides/android/apktool-resource-analysis.md",
route: "/apktool-resource-analysis",
},
{
text: "ADB device inspection and acquisition",
source: "guides/android/adb-device-analysis.md",
route: "/adb-device-analysis",
},
{
text: "Flutter build identification",
source: "guides/android/flutter-analysis.md",
route: "/flutter-analysis",
},
],
},
{
text: "Apple applications",
items: [
{
text: "Apple Application Inventory Projection",
source: "guides/apple/apple-application-analysis.md",
route: "/apple-application-analysis",
},
],
},
{
text: ".NET assemblies",
items: [
{
text: "Managed-code analysis",
source: "guides/managed/managed-code-analysis.md",
route: "/managed-code-analysis",
},
],
},
{
text: "Firmware",
items: [
{
text: "Firmware analysis",
source: "guides/firmware/firmware-analysis.md",
route: "/firmware-analysis",
},
],
},
{
text: "HarmonyOS packages",
items: [
{
text: "Static HarmonyOS package inventory",
source: "guides/harmony/harmony-analysis.md",
route: "/harmony-analysis",
},
],
},
{
text: "WebAssembly",
items: [
{
text: "Offline WebAssembly artifacts",
source: "guides/wasm/wasm-artifacts.md",
route: "/wasm-artifacts",
},
],
},
{
text: "EVM contracts",
items: [
{
text: "Offline EVM bytecode",
source: "guides/evm/evm-bytecode.md",
route: "/evm-bytecode",
},
],
},
{
text: "Databases and recorded packets",
items: [
{
text: "SQLite database snapshots",
source: "guides/data/sqlite-databases.md",
route: "/sqlite-databases",
},
{
text: "Recorded packet capture inspection",
source: "guides/data/packet-captures.md",
route: "/packet-captures",
},
{
text: "Packet capture summaries",
source: "guides/data/pcap-analysis.md",
route: "/pcap-analysis",
},
],
},
{
text: "Process capture and instrumentation",
items: [
{
text: "Process capture",
source: "guides/runtime/process-capture.md",
route: "/process-capture",
},
{
text: "Frida instrumentation",
source: "guides/runtime/frida-instrumentation.md",
route: "/frida-instrumentation",
},
],
},
{
text: "Provider setup and formats",
items: [
{
text: "Ghidra provider semantics",
source: "providers/ghidra-provider.md",
route: "/ghidra-provider",
},
{
text: "DOS MZ and COM analysis with Ghidra",
source: "providers/ghidra-dos.md",
route: "/ghidra-dos",
},
{
text: "Optional NativeAOT metadata recovery",
source: "providers/ghidra-nativeaot.md",
route: "/ghidra-nativeaot",
},
{
text: "MIPS ELF analysis through Ghidra",
source: "providers/ghidra-mips.md",
route: "/ghidra-mips",
},
{
text: "PSP static ELF analysis through Ghidra",
source: "providers/ghidra-psp.md",
route: "/ghidra-psp",
},
{
text: "RISC-V ELF analysis through Ghidra",
source: "providers/ghidra-riscv.md",
route: "/ghidra-riscv",
},
{
text: "Windows Ghidra P0",
source: "providers/windows-ghidra-p0.md",
route: "/windows-ghidra-p0",
},
{
text: "Headerless raw-binary analysis with Ghidra",
source: "providers/raw-binary.md",
route: "/raw-binary",
},
{
text: "IDA provider",
source: "providers/ida-provider.md",
route: "/ida-provider",
},
{
text: "JEB instrumentation and analysis",
source: "providers/jeb-analysis.md",
route: "/jeb-analysis",
},
],
},
{
text: "Contracts and evidence",
items: [
{
text: "MCP runtime contracts",
source: "reference/mcp-contracts.md",
route: "/mcp-contracts",
},
{
text: "Reconstruction obligation ledgers",
source: "reference/reconstruction-obligation-ledgers.md",
route: "/reconstruction-obligation-ledgers",
},
],
},
{
text: "Development and verification",
items: [
{
text: "MCP tool design",
source: "development/tool-design.md",
route: "/tool-design",
},
{
text: "Testing REA",
source: "development/testing.md",
route: "/testing",
},
{
text: "Client and agent verification",
source: "development/client-verification.md",
route: "/client-verification",
},
{
text: "Reconstruction readiness conformance",
source: "development/reconstruction-readiness.md",
route: "/reconstruction-readiness",
},
{
text: "Releasing REA",
source: "development/releasing.md",
route: "/releasing",
},
{
text: "REA roadmap",
source: "development/roadmap.md",
route: "/roadmap",
},
],
},
{
text: "Release migrations",
items: [
{
text: "Upgrading from REA 6.1 to 6.2",
source: "migrations/migration-6.2.md",
route: "/migration-6.2",
},
],
},
{
text: "Signal recordings",
items: [
{
text: "Recorded signal inspection",
source: "guides/signals/signal-recordings.md",
route: "/signal-recordings",
},
],
},
];
export const documentationRewrites = Object.fromEntries(
documentationSections.flatMap((section) =>
section.items.map((item) => [item.source, `${item.route.slice(1)}.md`]),
),
);
+138
View File
@@ -0,0 +1,138 @@
# Documentation directory
REA documentation is organized by reader task and document owner. This directory
links every authored guide; the [site](https://rea.tools/) also serves separate
product pages. The VitePress documentation source is this `docs/` tree.
## Getting started
- [Installation and setup](getting-started/installation.md)
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
- [CLI and Evidence](getting-started/cli.md)
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
## Native binaries
- [Native binary investigation](guides/native/native-investigation.md)
- [Go binary build metadata](guides/native/go-binaries.md)
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
- [Recorded crash evidence](guides/native/recorded-crashes.md)
## JavaScript and Electron
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
- [Electron file-page observation](guides/javascript/electron-observation.md)
## Browser and captured websites
- [Website observation with CDP](guides/web/browser-observation.md)
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
- [Website runtime source attribution](guides/web/web-runtime.md)
- [Historical web network captures](guides/web/web-network-captures.md)
- [Captured website source locations](guides/web/web-source-location.md)
- [Captured website module relationships](guides/web/website-module-trace.md)
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
## Android applications
- [Static Android APK analysis](guides/android/android-analysis.md)
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
- [Flutter build identification](guides/android/flutter-analysis.md)
## Apple applications
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
## .NET assemblies
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
## Firmware
- [Firmware analysis](guides/firmware/firmware-analysis.md)
## HarmonyOS packages
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
## WebAssembly
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
## EVM contracts
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
## Databases and recorded packets
- [SQLite database snapshots](guides/data/sqlite-databases.md)
- [Recorded packet capture inspection](guides/data/packet-captures.md)
- [Packet capture summaries](guides/data/pcap-analysis.md)
## Process capture and instrumentation
- [Process capture](guides/runtime/process-capture.md)
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
## Signal recordings
- [Recorded signal inspection](guides/signals/signal-recordings.md)
## Provider setup and formats
- [Ghidra provider semantics](providers/ghidra-provider.md)
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
- [IDA provider](providers/ida-provider.md)
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
## Contracts and evidence
- [MCP runtime contracts](reference/mcp-contracts.md)
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
## Development and verification
- [MCP tool design](development/tool-design.md)
- [Testing REA](development/testing.md)
- [Client and agent verification](development/client-verification.md)
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
- [Releasing REA](development/releasing.md)
- [REA roadmap](development/roadmap.md)
## Release migrations
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
## Architecture and authoring
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
- [Documentation ownership and contribution model](development/documentation.md)
- [Feature specifications](specs/README.md)
- [Architectural decision records](adr/README.md)
## Machine-owned documentation
| Path | Owner and update command |
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
| `assets/` | Shared authored images referenced by guides |
Do not edit generated catalogs or evidence to change product facts. Change their
canonical contracts or producer and regenerate. Historical release records retain
the exact commit messages and references from the release they describe.
+9
View File
@@ -0,0 +1,9 @@
# Architectural decision records
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
global sequence number. Use meaningful names and the Michael Nygard sections:
Status, Context, Decision and Consequences. Link each decision from its owning
feature specification.
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
the distinction between specifications, roadmaps and isolated decisions.
@@ -38,10 +38,6 @@ graph TD
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
OBJDUMP["src/objdump/<br/>GNU objdump command profile"]
RIZIN["src/rizin/<br/>Static command profile and persistent debugger sessions"]
GDB["src/gdb/<br/>Persistent GDB/MI sessions<br/>owned process lifecycle and target state"]
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
@@ -54,7 +50,7 @@ graph TD
end
subgraph "Contracts Layer"
CONTRACTS["src/contracts/<br/>Caller-visible schemas for analysis, reverse engineering, managed, browser, Electron, application, and session tools"]
CONTRACTS["src/contracts/<br/>Caller-visible schemas for analysis, managed, browser, Electron, application, and session tools"]
ERRORS["src/contracts/errorSchemas.ts<br/>Shared analysis error schemas<br/>CLI diagnostics, generated error catalog"]
end
@@ -126,14 +122,6 @@ graph TD
INSPECTOR --> DOMAIN
APP --> DOTNET
DOTNET --> DOMAIN
COMPOSITION --> OBJDUMP
OBJDUMP --> DOMAIN
COMPOSITION --> RIZIN
RIZIN --> DOMAIN
CLI --> GDB
SERVER --> GDB
GDB --> PROCESS
GDB --> DOMAIN
HOPPER --> HOPPER_BRIDGE
GHIDRA --> GHIDRA_BRIDGE
HOPPER --> PROCESS
@@ -406,7 +406,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
manifest readback, competing query variants, and resolved relative imports.
It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](website-script-export.md).
both public adapters. See [website script export](../guides/web/website-script-export.md).
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
real Chromium module-loading fixture: import-map scopes, package prefixes,
@@ -414,7 +414,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
lazy and computed unknowns, exact source/map identities and no implicit refetch.
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
package entrypoint. The complete `verify:browser` lane and existing conditional
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
## Agent evaluation and conformance records
+82
View File
@@ -0,0 +1,82 @@
# Documentation ownership and authoring
Use the [documentation directory](../README.md) to find existing material before
adding another guide. Preserve one canonical owner for each contract or workflow;
link to that owner instead of copying its claims into several guides.
## Choose a destination
| Document | Destination |
| -------------------------------------------------------- | -------------------------------------- |
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
| Shared runtime contracts and evidence semantics | `docs/reference/` |
| Development, testing, release and authoring instructions | `docs/development/` |
| Composition diagrams | `docs/architecture/` |
| Compatibility changes for a released version | `docs/migrations/` |
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
unnecessary: different domains can add features independently, and branches do
not compete for the next sequence number. Check the existing domain directory
and linked issues before creating a feature with the same slug.
Guides describe existing behavior and limitations. Specs define one proposed
feature or domain change; an existing guide does not become a specification by
moving it into `specs/`.
## Feature lifecycle
Create the specification before implementation. Each feature directory contains:
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
- `plan.md`: the single implementation plan, contracts and architectural choices.
- `tasks.md`: incremental, testable steps linked to requirements and verification.
Split independent domains or separately reviewable features before writing their
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
and dependencies; requirements and implementation tasks belong to each child.
Cross-repository contracts have one producer-owned specification. Consumers link
to that contract instead of maintaining competing copies.
Record an isolated architectural decision separately in
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
**Decision** and **Consequences**. Link the ADR from affected specifications.
Do not create decision records for routine file moves or invent past decisions.
Specs do not replace the contribution-scope decision required for a new provider
or expanded execution authority. Follow
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
## Links and public routes
Use relative links to physical Markdown files so GitHub readers can follow them.
Link repository files outside `docs/` with their full GitHub URLs. Preserve
existing heading anchors when moving a guide.
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
owns categorized site navigation and the existing guides' stable public routes.
Update it when adding or moving a guide. VitePress rewrites categorized sources
to the existing public paths; its Markdown hook resolves repository links from
the physical source and renders them against those public routes. Keep basenames
and route assignments unique. Edit links use the physical source file.
For a move, search for both the basename and its old full path. Update README
translations, authored skills, verification instructions, documentation-fact
readers and release configuration when they consume that path. Preserve
historical release records and references bound to earlier commits or tags.
## Generated ownership and checks
The [documentation directory](../README.md#machine-owned-documentation) identifies
the machine-owned outputs and their producers. Their existing paths are retained
because scripts, package consumers and release history depend on them. Change
canonical sources and regenerate; do not fix a generated projection by hand.
Run `npm run docs:check` for documentation facts and generated consistency, and
`npm run verify:docs:routes` to build the complete production site and check local
links, anchors and physical edit targets.
When a physical path moves, also run the tests that read it. Follow
[development checks](testing.md#developer-commands) before requesting review.
@@ -84,7 +84,7 @@ registry metadata or changelog disagrees with that version before creating a tag
### Compatibility and version selection
For the 6.2 candidate, review the [6.1 to 6.2 migration guide](migration-6.2.md)
For the 6.2 candidate, review the [6.1 to 6.2 migration guide](../migrations/migration-6.2.md)
alongside the changelog before publication.
REA uses Release Please's `always-bump-minor` strategy: every release increments
@@ -5,8 +5,8 @@ The [platform tracker](https://github.com/morluto/rea/issues/32),
[browser tracker](https://github.com/morluto/rea/issues/39) track capability and
verification work. Available tools and setup are documented in the
[investigation guides](https://rea.tools/guides/),
[installation guide](installation.md), and
[generated catalog](mcp-contracts.md#generated-catalog).
[installation guide](../getting-started/installation.md), and
[generated catalog](../reference/mcp-contracts.md#generated-catalog).
## Priorities
@@ -27,7 +27,7 @@ fixture servers; custom orchestration needs a demonstrated requirement.
Provider and platform claims require their corresponding
[real verification lanes](testing.md#real-toolchain-verification-lanes).
See [tool design](tool-design.md) for choosing a tool boundary and
[Ghidra semantics](ghidra-provider.md) for the existing deep-provider contract.
[Ghidra semantics](../providers/ghidra-provider.md) for the existing deep-provider contract.
Setup remains additive, reuses existing tools, and requires approval for disclosed
configuration and Hopper installation changes. Additional installer choices need
+25 -108
View File
@@ -237,7 +237,7 @@ logic.
reconstruction, native-boundary relationships and application graphs without
executing fixture PE files. Operator-local manifests and actual ILSpy oracles
remain optional, separately reported checks; the real Ghidra NativeAOT lane has
its own toolchain prerequisites. See [the managed guide](managed-code-analysis.md)
its own toolchain prerequisites. See [the managed guide](../guides/managed/managed-code-analysis.md)
for those configurations. Generated completion-ledger checks use the same owning
entrypoint and include its verifier/fixture files in their cache inputs.
@@ -262,33 +262,8 @@ CI owns coverage; aggregate and domain/contract thresholds are maintained in
`vitest.config.ts`. They are glob-specific and never updated automatically.
Coverage does not replace named boundary or real-provider scenarios.
The `reverse-tools` commands run the focused tests and report coverage and
mutation results without an additional percentage requirement. Test failures,
invalid mutation configuration, and runner failures still fail the check.
Mutation targets cover reviewed production behavior, including signal-terminated
commands, Cutter descriptor identity, debugger cleanup ownership, GDB command
exit and cancellation, and Rizin startup, timeout Evidence, and process exit.
A second exhaustive pass runs the focused Cutter and Rizin timeout semantics tests against every compilable
selected mutant to avoid relying on per-test mutation coverage associations;
Stryker reports compile-error mutants separately because those cannot execute
tests. Both passes resolve named syntax owners, scoped declarations, guards,
callbacks, and reviewed expressions from the current TypeScript source; no
hand-maintained line ranges select mutations. Missing, ambiguous, or malformed
source targets stop configuration before mutation generation. The selections
retain their review scope rather than expanding to whole source files. The
focused suite also exercises affected MCP routes and the Python Cutter bridge.
Contract, integration, and real-provider lanes complement these focused tests;
simulated providers do not establish real-provider behavior.
Fast CI starts the source-only projects after classification, independently of
build and static checks, and runs five built-dependent shards after the build.
The six test jobs still select every portable test. Full qualification keeps six
complete Vitest shards with coverage. Neither tier retries tests. Canonical
project ownership lives in `scripts/lib/test-projects.mjs`, shared by Vitest,
CI group execution, and development-test preparation. A test importing compiled
verifier code belongs to the built group even if its directory is conformance.
Source and built blobs have separate labels to prevent merge collisions.
Each test job emits a blob report; the merge job produces JUnit and JSON diagnostics, a module timing
CI runs six native Vitest shards without retries. Each shard emits a blob
report; the merge job produces JUnit and JSON diagnostics, a module timing
report, plus aggregate coverage during full qualification. It retains available
reports even when a shard fails, uploads available diagnostics, and writes the
slowest files to the workflow summary. Partial reports are marked diagnostic
@@ -323,30 +298,6 @@ portable test shard. These limits leave headroom for cold setup and full coverag
while preventing a hung job from holding the active main run for hours. They do
not bound runner queue waiting. Timeouts still fail the required gate.
The separate `CI timings` workflow also diagnoses missing, startup-failed,
queued/running, approval-required, canceled, and failed PR CI through bounded,
read-only API scans. Incomplete pagination or changing identity stays unknown.
It never closes/reopens PRs, reruns workflows, posts comments, or changes gates.
The timing manifest retains each run's event and head branch, including explicit
unknowns, so PR runs cannot consume the eligible-main comparison budget.
Its static failure report compares supported lint/duplication diagnostics with
failed main runs using exact source, ancestry, task hashes, and toolchain proof.
A match describes those diagnostics only; it does not establish that every
failure in a job came from main. Duplication reports retain the actual checker
exit result, so passing duplication checks preserve lint failures and caller
thresholds remain owned by the producer. Missing checker outcomes leave
those duplication diagnostics unclassified. Missing historical receipts remain unknown.
The reporter executes only trusted default-branch code and parses whitelisted
JSON artifact members without executing artifact content. Reports and limits
are retained in `ci-health-diagnostics`; its ten-minute job limit covers the
existing four-minute timing collection plus the bounded diagnostic stages.
Shard-count reduction remains deferred: the available successful timing sample
covers an older inventory, and file phase weights are not job wall time. The
one-source/five-built split retains six test jobs; it makes no hosted speedup
claim. Current complete timings and completed candidate hosted runs are needed
before enabling a smaller adaptive matrix.
The separate `CI timings` workflow batches recent completed CI and Release runs
hourly or on manual dispatch. It uploads `ci-job-timings-batch` with raw API pages and
a normalized point-in-time report for each collected workflow attempt, plus a
@@ -395,17 +346,6 @@ that explicit lane, not the routine iteration requirement.
## Real-toolchain verification lanes
Linux Rizin debugger ownership changes require a separate real-provider gate on
Rizin 0.7.4. Run
`npm run test:focused -- tests/boundary/providers/RizinInferiorTty.test.ts`.
This test verifies the caller-selected TTY against the actual inferior's file
descriptors, confirms `dbg.exitkills=false`, detaches through the `olj`/`o-`
protocol, and checks that the target survives debugger close and writes to the
selected TTY. The suite is included in the reverse-tools Vitest and mutation
typecheck selections; its real-provider assertions skip when Rizin 0.7.4 is
unavailable. Run the focused command above on a host with that provider for the
explicit real-provider verification gate.
Each real-toolchain command must require only the host tools needed to prove
its stated claim. Use a host-native fixture for host/provider acceptance, and
place optional cross-target formats or platform-specific runners in separate
@@ -574,14 +514,14 @@ fixed public ApiDemos v6.0.18 fixture. Obtain both with the explicit
kept under ignored `_reference/`. No Gradle build, Android SDK, emulator or
application execution is required. The lane compares real CLI/MCP package,
class search, class inventory, method decompilation and incoming references.
See [Android analysis](android-analysis.md) for boundaries and resource budgets.
See [Android analysis](../guides/android/android-analysis.md) for boundaries and resource budgets.
`verify:jeb` requires a caller-started JEB client serving MCP at
`REA_JEB_MCP_URL` (default `http://127.0.0.1:8425/mcp`) with a project
already open, and verifies real CLI client inspection, unit listing coverage,
and method decompilation. The script records the engine's exact response to
`open_jeb_project`; JEB 5.48.0 headless instances do not advertise that tool.
REA does not install or launch JEB; see [JEB analysis](jeb-analysis.md) for
REA does not install or launch JEB; see [JEB analysis](../providers/jeb-analysis.md) for
the bring-your-own boundary. Verified against JEB 5.48.0 serving
`jeb-mcp-server` 1.3.0 at four scales: a compiled Java class fixture, a
locally built signed probe APK with a launcher activity (manifest, v1/v2/v3
@@ -595,7 +535,7 @@ CLI and MCP with explicit unit selection, plus the thin Java wrapper and
decoded manifest. The dedicated Dart snapshot processor and `run_script`
remain GUI-only surfaces in JEB 5.48.0 headless.
Authenticated IPA and macOS application inventory projection is documented in
[Apple application analysis](apple-application-analysis.md).
[Apple application analysis](../guides/apple/apple-application-analysis.md).
`npm run verify:harmony` requires no SDK or engine. It projects the pinned
harmony-next-pipeline 1.0.1 release HAP from an MIT-licensed source repository,
@@ -605,7 +545,7 @@ The lane exercises `inspect_artifact`, projection determinism, Stage-model
path hints, exact component sets, signing-entry limitations, and CLI/MCP parity.
The release artifact has no independently established compiler/SDK or
reproducible source build; the default lane does not verify an installed package.
See [HarmonyOS package inventory](harmony-analysis.md) for those proof boundaries.
See [HarmonyOS package inventory](../guides/harmony/harmony-analysis.md) for those proof boundaries.
HarmonyOS `.har` libraries are deliberately not suffix-classified because
`.har` also names HTTP Archive JSON.
@@ -626,7 +566,7 @@ with the reported target identity, apktool.yml metadata projection
(1.2.3, SDK 24–34), manifest package agreement, and both the default and
`de` string tables projected through the locale option. Parser goldens in
`src/apktool/ApktoolDecodeOutput.test.ts` come from the same decode. See
[Apktool resource analysis](apktool-resource-analysis.md) for boundaries and
[Apktool resource analysis](../guides/android/apktool-resource-analysis.md) for boundaries and
budgets.
## Flutter build identification
@@ -656,7 +596,7 @@ documented Gallery APK to assert its pinned digest, hash, exact engine labels
and per-ABI URI counts. The verifier requires complete structural snapshot
coverage, reports bounded pool coverage as partial/unverified, and reports
exact labels/counts on other APKs unverified without an expectations file.
See [Flutter build identification](flutter-analysis.md) for boundaries and
See [Flutter build identification](../guides/android/flutter-analysis.md) for boundaries and
budgets.
## ADB device acquisition
@@ -687,7 +627,7 @@ removal. System-package pulls whose APKs keep non-`base.apk` names report
the `unknown` role with the file-name basis, verified with
`com.android.settings` (single 73.9 MB APK). Parser goldens for
`adb devices -l`, `getprop`, `pm list packages -f`, and `pm path` come from
the same device. See [ADB device analysis](adb-device-analysis.md) for
the same device. See [ADB device analysis](../guides/android/adb-device-analysis.md) for
boundaries and budgets.
## Optional NativeAOT Ghidra analysis
@@ -731,7 +671,7 @@ with `REA_NATIVEAOT_PROOF_CAPTURE_DIR` (absolute paths).
Keep builds/imports sequential on small hosts; scope `GHIDRA_HEADLESS_MAXMEM`
(e.g. `768M`) to this command and use CPU affinity if needed. REA does not install
or upgrade Java, Ghidra, .NET or native toolchains. See
[the supported layout and provenance](ghidra-nativeaot.md).
[the supported layout and provenance](../providers/ghidra-nativeaot.md).
## IDA MCP adapter
@@ -771,11 +711,11 @@ symbols carry a leading underscore). Before the headless session waited out
the transient `busy` health a worker can report right after opening, that lane
failed intermittently on that host. Linux headless, modern attached GUI tools,
other engine versions and architectures remain unverified; see the
[provider guide](ida-provider.md).
[provider guide](../providers/ida-provider.md).
Native IDA MCP verification uses `"protocol": "native"` with an already-running native plugin and requires `--second-procedure other_function --second-marker unique_second_body_text` to exercise GUI function switching and a return to the first function. These options remain optional for other profiles. The literal marker must appear in the second function's pseudocode and be absent from the first. Native cloud/widget failures are reported without hidden retries.
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](../providers/ida-provider.md#optional-native-ida-mcp-bridge).
## DOS Ghidra analysis
@@ -787,7 +727,7 @@ stable CLI/MCP observations, unchanged source bytes, and owned process/project
cleanup. Raw p-code address-space selector tokens are reported separately from
the stable observation comparison. Linux x64 and macOS arm64 are verified;
macOS x64 remains unverified. This lane is separate from host-native and optional cross-format
verification. See [DOS analysis](ghidra-dos.md).
verification. See [DOS analysis](../providers/ghidra-dos.md).
`npm run verify:ghidra:com` uses a generated headerless fixture with no compiler,
DOS emulator or game data. It exercises explicit admission, BinaryLoader entry
@@ -818,7 +758,7 @@ the entry changes without `provider_id`, unchanged source bytes, and owned
cleanup. Six-byte x86 and x86_64 controls also verify supported compiler
specifications, independent mappings, unchanged bytes, and decompilation of a
known return value. AArch64 remains covered by deterministic tests rather than
this real-provider lane. See [raw binary analysis](raw-binary.md).
this real-provider lane. See [raw binary analysis](../providers/raw-binary.md).
`npm run verify:ghidra:address-context` runs on Linux with a caller-installed
Ghidra, full JDK, and an x86-64 ELF GCC toolchain (`CC` selects another compatible
@@ -947,13 +887,8 @@ successful ancestor push run of this CI workflow, rather than the preceding
push: changes from failed or cancelled intermediate runs remain in scope.
Missing, untrusted, or non-ancestor history selects every lane. The lookup is
bounded to the last 100 successful push runs; lookup failures broaden selection.
Every code change still selects all six portable shards, without coverage
instrumentation in this fast tier. Ordinary PR matrices cancel remaining
shards after the first failure; passing PRs execute the complete suite. Main,
`ci:full`, scheduled, manual, and release qualification retain every shard's
results after a failure. Available failed-shard reports remain diagnostic only:
failed, cancelled, skipped, or missing selected shards cannot pass the gate or
publish successful timing history. This deliberately avoids relying on an
Every code change still runs all six portable shards, without coverage
instrumentation in this fast tier. This deliberately avoids relying on an
import graph for generated files, filesystem inputs, subprocesses, or bridges.
Scheduled qualification is admitted hourly at minute 23. A forced full run at
@@ -981,11 +916,10 @@ classification job records its selection and broad-fallback reasons in the run
summary. Representative Git-diff cases live in
`tests/boundary/filesystem/ciChangeScope.test.ts`.
`CI required` checks classification and every selected lane,
`CI required` always runs and checks classification and every selected lane,
rejecting failed, cancelled, or unexpectedly skipped work. It rejects missing or
unknown tiers and requires every canonical scope to be explicitly `true` or
`false`; incomplete classification cannot silently omit verification lanes.
The aggregate remains cancellable so superseded runs release their slots.
Branch protection
should require this aggregate, `Static checks`, `Portable tests`, and
`Conventional PR title`. Title validation runs separately on PR opening,
@@ -1034,7 +968,7 @@ filename cases.
POSIX package verification resolves the runner's configured npm cache before
creating disposable client homes and passes that location to its child installs.
This reuses the download cache restored by the shared npm-cache action while keeping client
This reuses the download cache restored by `setup-node` while keeping client
configuration isolated. Installs retain their existing registry and integrity
checks.
@@ -1064,24 +998,7 @@ thresholds remain enforced in full qualification.
Missing or unusable history falls back to Vitest's default assignment. Only
timing data is reused; live provider results are never cached. Turbo outputs are
restored by task hash, with trusted default-branch runs as cache writers and PRs
as restore-only consumers. The static job also restores its Turbo snapshot;
each task revalidates its own input hash before reusing outputs or diagnostics.
The Node pin and npm install settings are global hash inputs. Knip includes its ignore rules and VitePress
entry, so changes to those inputs cannot reuse an older successful check.
The shared npm-cache action caches downloads, not `node_modules`, and every
consumer still runs `npm ci`. Keys cover the runner OS, actual Node architecture,
root lockfiles, and `.npmrc`. Only push, schedule, and manual events on the
repository's default branch may save shared npm caches; PRs restore only.
Static snapshots use the same write authority and are saved only after the
static job's validation succeeds. Main warms the new cache namespace after
merge; a cold cache or changed lockfile falls back to the normal install.
This prevents each PR from saving another branch-scoped copy of the same
download cache. It does not remove runner assignment waits.
Portable owner-based test selection remains observational. Its reports compare
proposed omissions with the full run; incomplete or skipped observations do not
establish that fewer tests would preserve regression detection.
as restore-only consumers. npm caches hold downloads, not `node_modules`.
The eight automatic browser, binary-layout, EVM, JavaScript-recovery,
recorded-crash, and web provider lanes download `rea-build-cache` from their own
@@ -1133,7 +1050,7 @@ status, and Android logcat on success or failure. The emulator is disposable.
capinfos against a saved capture. Linux verification uses Wireshark 4.6.6
and captures containing DNS, TLS and HTTP traffic. Native Windows and macOS
engine behavior remains unverified; process fixtures do not establish it.
See [Packet capture analysis](pcap-analysis.md) for contracts and budgets.
See [Packet capture analysis](../guides/data/pcap-analysis.md) for contracts and budgets.
## Apple native metadata and UI
@@ -1153,7 +1070,7 @@ conformance/vtable fixture.
allows a host-permission-boundary-only result and explicitly reports
`positive_e2e: false`; it must not be reported as capture/action proof.
Both commands reject a changed executable digest and clean up the fixture
process and helper. These lanes require an interactive macOS desktop. See [native investigation](native-investigation.md)
process and helper. These lanes require an interactive macOS desktop. See [native investigation](../guides/native/native-investigation.md)
`npm run verify:native-calls` needs macOS with Command Line Tools (`clang`,
`lldb`, `codesign`, `nm`) and actual debugger access to the owned fixtures.
@@ -1191,7 +1108,7 @@ unknown chunks, depth limits and extracted child digests. The optional
`REA_FIRMWARE_VERIFY_EXT4=1` lane requires existing mke2fs/debugfs; the separate
`REA_FIRMWARE_VERIFY_GHIDRA=1` lane checks a selected host ELF through real Ghidra.
Neither optional toolchain is a base-lane prerequisite. See
[firmware analysis](firmware-analysis.md) for limits and unverified formats.
[firmware analysis](../guides/firmware/firmware-analysis.md) for limits and unverified formats.
### JavaScript source recovery
@@ -1232,7 +1149,7 @@ npm run verify:browser:source-maps
checks an installed package after building the verifier dependencies. The separate
conditional `real-web-source-map` CI job supplies Chrome and an isolated pinned
fixture compiler; static/unit checks do not acquire a browser. See
[the source location guide](web-source-location.md) for the verified decoder profile.
[the source location guide](../guides/web/web-source-location.md) for the verified decoder profile.
### JavaScript large-output lane
@@ -1259,7 +1176,7 @@ An optional entrypoint argument to `scripts/verify-browser-runtime.mjs` runs the
same checks through an isolated installed package. The conditional
`real-web-runtime` CI job runs only for relevant changes and needs no fixture
compiler. Ordinary unit/static gates acquire no browser. See
[website runtime attribution](web-runtime.md) for effects, resource bounds and
[website runtime attribution](../guides/web/web-runtime.md) for effects, resource bounds and
coverage limits.
### Offline binary layout
@@ -90,5 +90,5 @@ consumers. Real-engine claims require the matching provider lane; see
When discoverability changes, evaluate representative broad, direct, and negative
tasks on the intended host. Record host/model/catalog, selected tools, arguments,
errors, retries, and task outcomes. Schema validity and keyword heuristics do not
establish successful tool selection. See [mcp-contracts.md](mcp-contracts.md) for
establish successful tool selection. See [mcp-contracts.md](../reference/mcp-contracts.md) for
shipped runtime behavior.
+21 -33
View File
@@ -17,7 +17,7 @@ Use an extracted application directory or an ASAR as the target. Static
JavaScript analysis returns the application graph and Evidence directly.
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
The complete application Evidence can be hundreds of megabytes for a real
@@ -34,7 +34,7 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
of about 10 KB. Replace the view with
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
module page. See
[JavaScript application workflows](javascript-application-workflows.md) for
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
## Managed assemblies (.NET)
@@ -59,7 +59,7 @@ assembly-wide inventory, so the output can still be large.
Choose the workflow from the artifact's contents rather than its `.exe` or
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
unpack .NET single-file hosts or decode IL2CPP metadata. NativeAOT method bodies
require native analysis. See the [managed-code guide](managed-code-analysis.md)
require native analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
for supported deployment forms, selection semantics, and coverage limits.
## SQLite snapshots
@@ -67,7 +67,7 @@ for supported deployment forms, selection semantics, and coverage limits.
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
table. This command does not require a native analysis provider. See
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
source-file handling and row-value semantics.
## Native analysis
@@ -80,23 +80,11 @@ rea inspect-go-binary /absolute/path/to/program --json
```
The shared CLI/MCP workflow reports compiler version, modules, replacements,
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
for supported containers and absent or malformed metadata.
Inspect an embedded cargo-auditable Rust dependency list without installing
Rust or configuring a disassembler:
```bash
rea inspect-rust-binary /absolute/path/to/program --json
```
The same workflow reports package names, versions, sources, the root package,
dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](rust-binaries.md).
Configure [Hopper or Ghidra](installation.md#hopper), or the
[IDA adapter](ida-provider.md), before analyzing a native target. Substitute
[IDA adapter](../providers/ida-provider.md), before analyzing a native target. Substitute
your target, search text and function name or address in these examples:
```bash
@@ -113,11 +101,11 @@ a function dossier; `decompile` returns pseudocode. For assembly instructions
alone, use `rea instructions`. A macOS `.app` bundle can be supplied directly.
Run `rea --help` or a command's `--help` for arguments and output options.
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
tools. Provider-specific target support is described in the
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
[raw binary guide](raw-binary.md), and
[Windows Ghidra guide](windows-ghidra-p0.md).
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
[raw binary guide](../providers/raw-binary.md), and
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
## Choose a provider
@@ -154,13 +142,13 @@ or close. Provider failures are returned with their original reason. For an
`ambiguous` selection error, choose from `details.candidate_ids`; for
`provider_unavailable`, run `rea doctor --provider ID --json` to diagnose the
selected engine. See [task readiness](installation.md#check-readiness-for-your-task)
and [provider selection](mcp-contracts.md#identity-and-discovery).
and [provider selection](../reference/mcp-contracts.md#identity-and-discovery).
The session reports work still in progress through `analysis_activity`.
A client timeout can end its wait while the provider continues analyzing.
`cleanup_incomplete` identifies resources whose shutdown or removal could not
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
## Save and reuse analysis snapshots
@@ -216,8 +204,8 @@ rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/righ
Exports preserve an existing destination unless `--overwrite` is explicit.
For application traces and version comparisons, see
[JavaScript application workflows](javascript-application-workflows.md).
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls consume full saved records.
## Import historical source
@@ -277,18 +265,18 @@ does not infer module targets from replacement characters.
Historical-source import requires safe no-follow file opens on Linux or
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
or another supported host. See
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
for mapping identities, inferences and unknowns.
## Capture runtime behavior
Choose the guide for the target and intended interaction:
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
Runtime requests name the target and actions. Launched targets run with your
user permissions; consult the chosen guide for host requirements and effects.
@@ -10,7 +10,7 @@ acceptance is verified only where explicitly recorded.
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
preserving your REA client entry. See the
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
## GitHub Copilot CLI
@@ -31,7 +31,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
settings, then restart the client. Compact advertisements can reduce nested
schema constraints; REA still validates complete canonical inputs. Full-profile
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
Windows remain unverified; see [native client verification](testing.md).
Windows remain unverified; see [native client verification](../development/testing.md).
## Claude Desktop
@@ -265,21 +265,6 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
## WorkBuddy
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
global `~/.workbuddy/mcp.json` (a standard `mcpServers` JSON file at the user
home root on every platform). Entries carry `command`, `args`, and `env`;
stdio is the default transport. WorkBuddy discovers personal skills from
`~/.workbuddy/skills` (`%userprofile%\.workbuddy\skills` on Windows), so
setup installs its skill copy there.
The global MCP path is a plain JSON file; setup preserves every other server
and top-level key when it upserts `rea`. WorkBuddy also reads a project-level
`<project>/.workbuddy/mcp.json`, which setup does not touch: it registers the
user-level file only. Loading this registration and real REA tool calls
through WorkBuddy are verified.
## ZCode
For [ZCode](https://zcode.z.ai/en/docs/mcp-services), setup registers `rea` in the nested `mcp.servers` table of the
@@ -343,7 +328,7 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
discovers that directory, and its native `skill` tool can load
`reverse-engineer-anything`.
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
checks headless chat, skill loading, all tool input schemas and actual analysis
using the installed Harness and a local model fixture. This establishes the
Harness integration, with live DeepSeek API compatibility remaining unverified.
@@ -3,13 +3,13 @@
Run REA, connect your selected agents, and check the tools needed for your task.
Setup previews configuration changes and asks for approval before applying them.
| Task | Section |
| --------------------------------- | ------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), or the [IDA guide](ida-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
| Task | Section |
| --------------------------------- | -------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), or the [IDA guide](../providers/ida-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
## Start setup
@@ -288,12 +288,11 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| Qoder | `qoder` |
| Factory Droid | `droid` |
| Trae | `trae` |
| WorkBuddy | `workbuddy` |
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
check above does not cover them.
`dsh` and `kimi` require a release containing them. The registry check above does
not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
@@ -317,7 +316,7 @@ Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy. REA itself runs on any
Linux distribution with a recognized apt, dnf, or pacman package family, where
an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
and IDA have their own provider-specific host requirements; Windows Ghidra uses
the [experimental P0 boundary](windows-ghidra-p0.md).
the [experimental P0 boundary](../providers/windows-ghidra-p0.md).
On macOS, approved setup downloads the official DMG, checks its published size
and digest, validates the application bundle, and atomically installs it to
@@ -450,7 +449,7 @@ output-stream closure; `output_closed: false` keeps that uncertainty explicit.
Later requests retain the known exit instead of reducing it to socket loss.
An external GUI's helper exit does not establish that the GUI itself exited.
For cursor navigation, string decoding, symbol-name limits, and regex matching,
see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior).
see [Hopper analysis behavior](../guides/native/native-investigation.md#hopper-analysis-behavior).
Closing or switching a target closes its bound Hopper document, shuts down REA's
bridge and removes its temporary socket directory. The macOS Hopper application
@@ -494,13 +493,7 @@ copying license secrets into logs, or killing unrelated Hopper processes.
REA connects to an existing Ghidra installation on Linux x64/arm64, macOS x64/arm64,
or experimental Windows x64 P0.
It accepts Ghidra 12.1.x and the 64-bit full JDK declared by that installation's
`application.java.min` and `application.java.max`. The JDK's reported platform must also
map to the same native decompiler directory as the host: a JDK running under
emulation (for example x86_64 under Rosetta on Apple silicon) or reporting a
non-host `os.name` fails the doctor
even when it is an otherwise accepted 64-bit full JDK. Select a matching JDK
through `JAVA_HOME`; adding another decompiler in the JVM-selected directory
does not satisfy this host-validated admission rule. Current 12.1 releases require
`application.java.min` and `application.java.max`. Current 12.1 releases require
JDK 21 or newer and set no maximum. The bridge is verified with Ghidra 12.1.4
and JDK 21. Each installation must include the native decompiler for the host
architecture in `Ghidra/Features/Decompiler/os/<platform>/` or the corresponding
@@ -527,12 +520,12 @@ session metadata edits leave executable bytes unchanged and are discarded on clo
require Hopper; Windows P0 remains read-only.
Windows P0 admits native x86 and x86-64 PE applications and DLLs, plus explicitly
selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied
selected [raw NMOS 6502 images](../providers/raw-binary.md#nmos-6502) with a caller-supplied
entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain
unsupported on Windows.
The npm package bundles native Job Object ownership, protected private runtime
DACLs, and handle-based path admission; no separate addon installation is needed.
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
See the [Windows Ghidra P0 guide](../providers/windows-ghidra-p0.md) for verified scope and
limitations.
Extract Ghidra and install the JDK outside REA, then export absolute paths:
@@ -558,16 +551,11 @@ REA skill after approval. Direct registrations use Node to launch REA's entry
script; package-runner registrations use the pinned `npx` or `bunx` command. Hopper
installation remains unavailable on Windows. Setup never installs Ghidra,
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
registrations. See [Windows Ghidra P0](../providers/windows-ghidra-p0.md) for provider diagnostics.
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
It also checks that the reported JVM `os.arch` and `os.name` map to the same native decompiler
directory validated for the host; unrecognized reported architectures or a
non-host operating system are refused. LaunchSupport overrides of either
property, including through helper-environment JVM option variables, are
rejected at launch.
When Java is found through `PATH`, setup records its observed JDK home so GUI
MCP clients do not depend on an incidental shell path. Setup shows every exact
environment entry in its plan, writes only after approval, and never downloads,
@@ -604,7 +592,7 @@ passes the deadline to each provider client; a startup failure is returned by th
without exposing partial analysis. This deadline is separate from MCP transport
initialization and the client's deadline for that individual tool call. A client
can time out earlier even when Ghidra would complete within its startup deadline.
See [Ghidra first-query deadlines and recovery](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
See [Ghidra first-query deadlines and recovery](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for client options and the close/reopen recovery flow.
One session contains exactly one imported Program; use `provider_id: "ghidra"`, `--provider ghidra`, or
@@ -765,7 +753,7 @@ contains the launch arguments and `/proc/<jvm-pid>/status` reports
session and retain the resource flags needed for the measurement rather than
copying full command lines or environments into reports. Heap and affinity
settings affect cost and latency; they are not part of REA's semantic analysis
profile identity. See [real-provider testing](testing.md) for bounded
profile identity. See [real-provider testing](../development/testing.md) for bounded
verification workflows.
### Ghidra diagnostics and verification
@@ -776,9 +764,6 @@ errors expose Error names, messages, and available codes under
truncation markers. Darwin ownership-helper compilation failures also retain
the compiler outcome, exit code or signal, and bounded stderr; a timed-out
compiler remains distinguishable from a compiler that exited with an error.
The helper compile deadline defaults to 120,000 ms; set
`REA_SWIFTC_COMPILE_TIMEOUT_MS` to an integer between 1 and 2,147,483,647 when
a busy host needs longer.
Other rejection values retain their primitive value or an explicit type.
Shutdown warnings include the failure
kind, message, and diagnostics, while process and temporary-project cleanup
-90
View File
@@ -1,90 +0,0 @@
# PowerPC ELF analysis through Ghidra
This lane extends native ELF analysis to 32-bit big-endian PowerPC; it does
not execute PowerPC code. It is the first milestone of
[#1741](https://github.com/morluto/rea/issues/1741). GameCube/Wii DOL
executables, PPC64 and other PowerPC variants are separate follow-ups.
## Admission boundary
ELF `EM_PPC` (20) is retained as the provider-neutral `powerpc` family. The
target also retains class, byte order, `e_type` and raw `e_flags`. Recognition
does not mean every provider or PowerPC variant is supported. `EM_PPC64` (21)
is not recognized as this family.
The Ghidra lane admits ELF32 `ET_EXEC`, big endian. ELF64, little-endian,
relocatable and shared objects are refused with a specific unsupported
constraint. `EF_PPC_EMB` declares an embedded variant and is refused; the
remaining `e_flags` bits only declare relocation conventions and do not change
instruction decoding.
## Bound language
Ghidra's ELF loader selects `PowerPC:BE:32:e500:default` for a plain `EM_PPC`
header with `e_flags` 0. e500 is the embedded Book-E variant whose SPE unit
replaces the classic floating-point unit, so it misinterprets ordinary
750-family code. Nothing in the header distinguishes the two.
The lane therefore imports with `PowerPC:BE:32:default` and compiler spec
`default`, commits that choice in the analysis profile as
`language_selection: "provider-pinned-v1"`, and checks the language Ghidra
reports after import. A different language set through
`REA_GHIDRA_LANGUAGE_ID` (and optionally `REA_GHIDRA_COMPILER_SPEC_ID`)
replaces the lane default and is admitted against the installed language
catalog as usual; for example `PowerPC:BE:32:e500` for genuine e500 code.
PowerPC header facts are committed in the analysis profile so interpretation
changes invalidate profile-bound snapshots. The support-lane identity is
versioned independently of file identity. Evidence, lifecycle outputs and saved
snapshots preserve the `powerpc` family. Hopper and IDA adapters do not
implicitly gain PowerPC support. Windows Ghidra P0 remains PE x86/x86-64 only.
## Use
With a caller-installed Ghidra and compatible JDK on a supported Linux/macOS host:
```bash
rea analyze /absolute/path/to/ppc.elf --provider ghidra --json
rea function /absolute/path/to/ppc.elf function_name --provider ghidra --json
```
MCP uses the existing tools, including `open_binary` with an absolute path and
`provider_id: "ghidra"`. No PowerPC-specific tool catalog or emulator is
introduced.
## Verification
The source-owned freestanding fixture is `tests/conformance/c/ppc.c`. The
optional cross-target lane requires a caller-supplied Clang with the PowerPC
target, LLD, GNU `readelf`, Ghidra and its compatible JDK; ordinary host-native
checks do not acquire these tools. `REA_PPC_CLANG` selects Clang and
`REA_PPC_READELF` selects GNU readelf (for example
`/opt/homebrew/opt/binutils/bin/readelf` on macOS). These are test-time
prerequisites, not REA runtime providers.
```bash
GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra npm run verify:ghidra:ppc
```
The lane compiles the fixture for a 750-family core (`-mcpu=750`) and never
executes it. GNU readelf independently supplies header facts and symbol
addresses, compared with the production REA target resolver before provider
startup. Its version, bounded command output and selected tool path are
retained in the verification report. The parser deliberately recognizes only
this fixture's GNU output, rejecting missing, ambiguous or unexpected fields
rather than substituting defaults.
Existing public MCP operations check Ghidra's **observed** language and
compiler specification and loaded-file hashes; an e500 import fails the lane.
The source-owned `ppc-probe.S` supplies fixed `addi` and conditional-branch
encodings: verification checks bytes, length, immediate and direct destination,
independent of decompiler variable names. Global and string bytes are checked at
independently reported symbol addresses.
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
identity, snapshot round-trip and final process cleanup remain covered.
Pseudocode presence is a liveness check, not proof of semantic equivalence.
Instruction decoding and reader agreement do not establish runtime or complete
floating-point semantics. Parser/profile tests are narrower checks, not
substitutes for this real-provider run. Report the actual lanes run and leave
unexecuted verification pending.
@@ -181,4 +181,4 @@ Record: verified against adb 34.0.5 on Linux against an Android 14
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
287 services, 92 features, a real two-APK split set pulled with byte-exact
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
complete package lifecycle; see `docs/testing.md` for the lane summary.
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
@@ -6,7 +6,7 @@ kept as an unmodified, commit-pinned Git submodule; see
application workflow and return artifact-bound Evidence inline.
This family is present on repository main and in npm 4.1.0. Check the
[released package boundary](installation.md#released-package-and-main) and the
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
connected server's tool list before selecting it.
## Optional package delivery
@@ -33,40 +33,10 @@ unverified.
## Supply tools
Use a **full JDK 17 or newer** (including its compiler module) and
[**jadx-headless-mcp 0.7.1**](https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar).
Verify the downloaded JAR's SHA-256 against
[REA's pinned release metadata](https://github.com/morluto/rea/blob/main/src/android/JadxRelease.ts)
before using it. The following download and verification commands work for npm
package consumers as well as repository checkouts; the repository-only
`npm run fixtures:android` helper is not included in the npm package.
On macOS or Linux, choose a destination and verify the pinned digest:
```sh
mkdir -p "$HOME/.local/share/rea/android" &&
cd "$HOME/.local/share/rea/android" &&
curl -fL -o jadx-headless-mcp-0.7.1-all.jar https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar &&
if command -v sha256sum >/dev/null; then
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | sha256sum -c -
else
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | shasum -a 256 -c -
fi &&
export REA_JADX_MCP_JAR="$PWD/jadx-headless-mcp-0.7.1-all.jar"
```
On Windows, run the equivalent in PowerShell:
```powershell
$jar = Join-Path $HOME '.local\share\rea\android\jadx-headless-mcp-0.7.1-all.jar'
New-Item -ItemType Directory -Force (Split-Path $jar) | Out-Null
Invoke-WebRequest -Uri 'https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar' -OutFile $jar
if ((Get-FileHash -Algorithm SHA256 $jar).Hash.ToLowerInvariant() -ne '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75') { throw 'JADX JAR SHA-256 does not match REA pinned metadata' }
$env:REA_JADX_MCP_JAR = $jar
```
REA runs its packaged Java metadata bridge in source-file mode against the
unmodified engine JAR. REA does not install Java, an Android SDK, JADX, a device
service or an emulator. Set the path in the shell or MCP server environment:
**jadx-headless-mcp 0.7.1**. REA runs its packaged Java metadata bridge in
source-file mode against the unmodified engine JAR. REA does not install Java,
an Android SDK, JADX, a device service or an emulator. Set the path in the shell
or MCP server environment:
```sh
export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
@@ -91,4 +91,4 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
locales — launcher version, on-disk digest agreement, apktool.yml metadata
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
tables projected. Parser goldens in the test suite come from the same
decode. See `docs/testing.md` for the lane summary.
decode. See `docs/development/testing.md` for the lane summary.
@@ -124,7 +124,7 @@ Select a verified native child for the existing tools:
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. Generic
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
[MIPS ELF](../../providers/ghidra-mips.md) and [RISC-V ELF](../../providers/ghidra-riscv.md) lanes are separate
documents with their own admission boundaries. This milestone does
not add PPC decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
@@ -294,7 +294,7 @@ References are scoped to one connection; `close_binary` clears them. Export a
bundle before closing and import it on another connection, or supply the full
inline Evidence there. Versions before 4.1.0 accept full inline Evidence only; installing
newer skill instructions does not change that schema. See
[MCP Evidence inputs](mcp-contracts.md#retained-application-evidence-inputs).
[MCP Evidence inputs](../../reference/mcp-contracts.md#retained-application-evidence-inputs).
For two operator-provided directories or ASARs, run:
@@ -159,7 +159,7 @@ An unpacked entry whose companion bytes were not supplied remains
`unavailable`. REA continues analyzing embedded JavaScript and records the
missing native/resource bytes as unknown. The occurrence keeps
`hash_status: unavailable`; no content-addressed child artifact is created for
those missing bytes. See [MCP integrity handling](mcp-contracts.md#integrity-record-and-continue)
those missing bytes. See [MCP integrity handling](../../reference/mcp-contracts.md#integrity-record-and-continue)
for the tool-result contract.
## Large results
@@ -170,7 +170,7 @@ select exact [source paths](#select-sources-before-analysis).
If a complete response exceeds the transport budget, use its retained Evidence
ID for a summary, module page, or item, or export the bundle. See
[MCP tool results](mcp-contracts.md#tool-results) for delivery limits, export, and
[MCP tool results](../../reference/mcp-contracts.md#tool-results) for delivery limits, export, and
matching server/client buffer settings. Follow-up results also need to fit that
budget. [Analyzer limits](#analyzer-and-result-admission) control analysis memory,
source deadlines, and semantic capacity separately.
@@ -184,7 +184,7 @@ or Evidence sealing. Unexamined relationships remain unknown.
MCP returns a `partial_observation` retained reference after recording succeeds;
open it with `inspect_analysis_view` on the same connection. If the SDK stopped
waiting before receiving the reply, follow [MCP cancellation recovery](mcp-contracts.md#progress-and-cancellation)
waiting before receiving the reply, follow [MCP cancellation recovery](../../reference/mcp-contracts.md#progress-and-cancellation)
to find the record. CLI failure output contains the complete partial Evidence.
Owned analyzer cleanup is verified before private files are released; uncertain
cleanup retains ownership for retry and is reported separately.
@@ -136,6 +136,6 @@ output, with 8 MiB per diagnostic/report stream. Exceeding a limit is an explici
failure with rollback. These limits apply per operation and do not claim aggregate
host containment across concurrent callers.
See [testing.md](testing.md#javascript-source-recovery) for the focused real
See [testing.md](../../development/testing.md#javascript-source-recovery) for the focused real
Wakaru CLI/MCP lane and its source-owned compiler fixtures. Other host platforms
and Bun/Vue recovery are not claimed by this adapter.
@@ -24,7 +24,7 @@ Native-body bridge mapping and managed runtime execution are outside the
current tool set.
The canonical tool inventory is the
[build-generated catalog](mcp-contracts.md#generated-catalog).
[build-generated catalog](../../reference/mcp-contracts.md#generated-catalog).
## Inspect an assembly
@@ -139,40 +139,14 @@ The result preserves the requested names in `selection.type_names` and reports
unmatched names separately. If metadata names are unreadable, limitations retain
that uncertainty. With no selection, `selection` is null and whole-assembly
inspection keeps its existing resource policy. Selection restricts types,
fields, methods, call edges, field accesses and user strings; member references
remain an assembly-wide inventory, so output size is not guaranteed to be small.
fields, methods, call edges and field accesses; member references remain an
assembly-wide inventory, so output size is not guaranteed to be small.
Coverage describes the selected members. Comparisons leave members outside the
opposite observation's selection unknown, and application graphs report their
selected scope as partial. Neither makes an absence claim about the whole
assembly. Older member Evidence without `selection` remains readable.
### User strings
`ldstr` anchors keep their build-local `0x70` tokens. `user_strings.literals`
lists each distinct token loaded by an inspected method body once, in token
order, with its `#US` file offset and decoded value. Values keep their stored
UTF-16 code units, including unpaired surrogates. Token `0x70000000` names the
heap's empty entry at offset 0 and reads as `""`; any other item there is
malformed. The terminal flag is checked against ECMA-335 II.24.2.4, and the
length prefix against the width II.23.2 assigns. A difference is reported with
its byte offset as a user-string issue while preserving the readable literal.
Toolchain conventions can differ; this conformance observation does not
establish runtime rejection. A missing `#US` stream is one stream-level issue;
a malformed item is reported against its token. User strings carry their own
coverage: any of these makes `user_strings.state` partial and lists the issue
in `user_strings.issues`, while the member `coverage` and `metadata.status` are
unaffected, and readable literals remain. The state is also partial when an
inspected method's CIL was not fully decoded (a malformed exception section
alone does not count) or a method that may belong to the inspected types was
not inspected (its row could not be read, or its type's name could not be read
under a `type_names` selection), because `ldstr` instructions there were not
observed. Normalized CIL digests do not include decoded values.
`compare_managed_members` does not decode or compare literals: an unchanged
method does not establish unchanged strings, and the comparison says so in its
limitations. Older member Evidence without `user_strings` reads as `null`: not
examined.
### Edge target declaring types
`member_refs[].parent_name`, `call_edges[].target_declaring_type`, and
@@ -209,34 +183,21 @@ native/runtime questions retain their own authority and coverage.
Names and semantic labels do not establish durable member identity. Comparison
preserves observed differences, structural inferences, and unknowns separately.
Cross-version matching first pairs members of the same declared type and name:
methods by exact CIL/signature identity, then by raw signature, then by
structural body shape; fields by raw signature. Exact CIL/signature identity
requires equal raw CIL digests and decoded fingerprints. Equal decoded
fingerprints alone cannot establish an exact match: they omit switch targets
and collapse distinct NaN encodings. Only then may a remap pair a
renamed or moved member across declared types or names: a method by exact
CIL/signature identity or structural body shape, a field by exact decoded
signature. A remap never takes a member whose own declared type and name remain
unpaired on the other side, so a member is never taken from its own changed
counterpart. A member whose declaring type is unresolved could belong to any
type, so it pairs by name only when no other member shares its exact
CIL/signature key, never by structural shape alone, and any remap group it
joins stays ambiguous. The exact-signature key uses names only as part of that
full tuple; names alone never select a pair. Duplicate tuples remain ambiguous
rather than being paired by token order. The raw signature is exact whether or
not REA decoded it, so undecoded signatures pair by that tuple too, but they
never enter structural rounds. An unmatched member is therefore reported as
`unknown`, not added or removed, when its own signature was not decoded or when
the other side has an unpaired member, one-sided or ambiguous, with the same
declared type and name whose signature was not decoded.
Cross-version matching first prefers exact CIL/signature identity. When that
does not match, it pairs an exact declared type, method name, and raw signature
before trying structural body shape. The exact-signature key uses names only
as part of that full tuple; names alone never select a pair. Duplicate tuples
remain ambiguous rather than being paired by token order. Fields follow the
same exact tiers. The raw signature is exact whether or not REA decoded it, so
undecoded signatures pair by that tuple too, but they never enter structural
rounds. An unmatched member is therefore reported as `unknown`, not added or
removed, when its own signature was not decoded or when the other side has an
unpaired member, one-sided or ambiguous, with the same declared type and name
whose signature was not decoded.
For a matched method, an unavailable or partial body makes body-shape facets
unknown while preserving observed signature differences. For complete bodies,
a difference in either the raw CIL digest or the decoded fingerprint is reported
in the `cil` dimension, including reordered switch targets with equal decoded
fingerprints. Structural identity compares normalized signatures, the limited
decoded-CIL tuple fingerprint,
unknown while preserving observed signature differences. Structural identity
compares normalized signatures, the limited decoded-CIL tuple fingerprint,
constants, and bounded shape context. It reports all candidates at the winning
score and remains ambiguous when the evidence does not distinguish them. The
digest does not itself remap metadata tokens.
@@ -267,13 +228,13 @@ describes exported shared-library entry points.
REA offers a built-in read-only snapshot parser for x64 PE/RTR 9.1 metadata and
an optional Linux headless Ghidra adapter for annotations and broader recovery.
The adapter requires a locally built `REA_GHIDRA_NATIVEAOT_JAR`; REA does not
install it or its toolchain. The [NativeAOT guide](ghidra-nativeaot.md) defines
install it or its toolchain. The [NativeAOT guide](../../providers/ghidra-nativeaot.md) defines
these routes, their layout/host restrictions, derived-memory provenance, and
verification coverage.
REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
applications and DLLs for static analysis. For x64 PE NativeAOT, the read-only
parser also runs there; the [NativeAOT guide](ghidra-nativeaot.md) lists what a
parser also runs there; the [NativeAOT guide](../../providers/ghidra-nativeaot.md) lists what a
Windows host covers.
For NativeAOT analysis, preserve the selected native artifact's path and digest
@@ -282,6 +243,7 @@ to native function addresses requires verified
provider evidence; names or vtable similarity alone do not prove that a native
function implements a specific managed method.
<div id="managednative-boundary-rules"></div>
<div id="managed-native-boundaries"></div>
## Managed/native boundary rules
@@ -293,7 +255,7 @@ matches establish declaration links; they do not establish that a call executed.
Metadata tokens are not native addresses. Native body mapping for C++/CLI,
ReadyToRun, NativeAOT, and IL2CPP requires separately verified format and provider
support. The managed static tools do not recover those mappings. The optional
[Ghidra NativeAOT extension](ghidra-nativeaot.md) has its own supported boundary.
[Ghidra NativeAOT extension](../../providers/ghidra-nativeaot.md) has its own supported boundary.
Declared import inventory supports a bounded positive claim. Its absence does
not exclude dynamic resolution, generated code, protected code, native helpers,
@@ -373,8 +335,8 @@ Opcode names retain short/long forms and prefix opcodes. This makes the value a
reproducible decoded-tuple fingerprint, not a canonical semantic CIL identity:
it does not resolve tokens across MVIDs, does not commit full control flow or
exception semantics, and does not replace the raw CIL digest. Cross-build
comparison requires equal raw CIL digests as well for exact CIL/signature
matches and reports ambiguity when structural evidence is insufficient.
comparison uses the value only as one unique-only structural tier and reports
ambiguity when that evidence is insufficient.
For example, a decoded `nop; ret` body serializes exactly as
`[["nop","none",null],["ret","none",null]]` and has digest
@@ -419,7 +381,7 @@ The current lane uses source-owned byte-built fixtures and malformed-input
regressions, with optional operator-supplied applications and ILSpy verification.
Run the deterministic lane with `npm run verify:managed`. See
[testing](testing.md#real-toolchain-verification-lanes) for prerequisites and
[testing](../../development/testing.md#real-toolchain-verification-lanes) for prerequisites and
coverage. Generated outputs belong outside tracked fixture directories and must
not remain after verification or packaging.
@@ -104,7 +104,7 @@ reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
These settings affect delivery, not decoder coverage or limits. The large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
@@ -81,7 +81,7 @@ temporary files, starts no subprocesses and accesses no network endpoint.
Resource guards bound the selected file to 256 MiB, aggregate embedded strings
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
report to 8 MiB. Oversized results fail without partial success. MCP also applies
its [response transport budget](mcp-contracts.md), which counts both text and
its [response transport budget](../../reference/mcp-contracts.md), which counts both text and
structured content; a report within the metadata limit can still require export
from retained Evidence. These limits are not operating-system memory or CPU
isolation.
@@ -8,7 +8,7 @@ observation complement that analysis when they answer a specific question.
## Start with a function
Choose an installed provider from [setup](installation.md#hopper). For Ghidra:
Choose an installed provider from [setup](../../getting-started/installation.md#hopper). For Ghidra:
```bash
rea inspect /absolute/path/to/program --provider ghidra --json
@@ -299,7 +299,7 @@ complete observation.
It reports each `@rpath`, `@loader_path` and `@executable_path` candidate with
its outcome. Paths outside the analyzed root, including shared-cache system
libraries, stay undetermined. See
[Apple application analysis](apple-application-analysis.md#dylib-load-resolution).
[Apple application analysis](../apple/apple-application-analysis.md#dylib-load-resolution).
- `inspect_asset_catalog` / `rea inspect-asset-catalog <app>` reads compiled
`Assets.car` metadata through macOS `assetutil --info`. Catalog digests, raw
rendition fields, pagination and exact UI resource-name matches are returned.
@@ -477,10 +477,10 @@ Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then config
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
the matching native decompiler. Experimental Windows x64 P0 admits native
x86 and x86-64 PE applications and DLLs, plus explicitly selected
[raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied load address,
[raw NMOS 6502 images](../../providers/raw-binary.md#nmos-6502) with a caller-supplied load address,
on fixed local NTFS volumes. Other raw CPU families remain unsupported on Windows.
The boundary uses bundled Job Object ownership, protected runtime DACLs, and
handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and
handle-based path admission. See [Windows Ghidra P0](../../providers/windows-ghidra-p0.md) and
[issue #527](https://github.com/morluto/rea/issues/527).
On Linux and macOS, `annotate_native_function` atomically edits a function name
and entry comments in the ephemeral database, returning refreshed analysis
@@ -561,5 +561,5 @@ Use `start`/`end` to constrain the inclusive address window, and continue with
`next_cursor` until `complete` is true. Inspect `searched` for page coverage and
skipped uninitialized blocks. A missing match does not establish absence from
uninitialized memory or across block boundaries. See
[Ghidra byte search](ghidra-provider.md#masked-byte-search) for mask and coverage
[Ghidra byte search](../../providers/ghidra-provider.md#masked-byte-search) for mask and coverage
semantics. This query never executes or modifies target bytes.
@@ -6,8 +6,8 @@ For remote devices, pass a host with an optional port (`host[:port]`; bracket IP
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. A failed cleanup remains reported rather than treating an unregistered session as fully released.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
@@ -61,24 +61,9 @@ The tool profile preserves upstream release/source identity, resolved executable
paths, executable digests, exact banners and arguments. Executable digests are
checked again after each command. Caller-supplied tools are trusted programs;
version and digest recording are not publisher authentication or an OS sandbox.
Each subprocess has a 30-second deadline and 32 MiB combined output budget.
Inspection output past the budget fails with `resource_constraint`; an oversized
`--version` probe still reports `unreadable_output`. Both streams share the
budget, so a stream that starts after the other has filled it retains nothing.
The captured output keeps up to the first 4,096 bytes that each stream retained,
each stream's retained byte count, and the exit code and signal. Retained text
can end in U+FFFD where the budget cut a multi-byte character.
These excerpts bound the returned diagnostic text.
[PR #2117](https://github.com/morluto/rea/pull/2117) reports elapsed time, CPU
time, and peak process-family memory, including WABT children, for warm Linux
x64 WABT 1.0.42 CLI workflows; those costs did not change outside measurement
noise. Agent token usage and truly cold workflows remain
unmeasured. The underlying error cause still retains the full captured
subprocess snapshot, so the excerpt does not establish lower retained memory
or faster analysis.
All commands use existing owned-process cancellation and cleanup. Workspace
cleanup failures remain explicit errors with resource paths.
Each subprocess has a 30-second deadline and 32 MiB combined output budget;
truncation fails. All commands use existing owned-process cancellation and
cleanup. Workspace cleanup failures remain explicit errors with resource paths.
WAT is a decoded representation, not original source or proof of runtime
behavior. Custom annotations can preserve payloads, but WAT does not establish original
@@ -98,7 +83,7 @@ Candidates are caller-selected files; their identity does not imply validation.
Both CLI and MCP use the same typed port and application service. MCP publishes
SDK-compatible schemas and records Evidence for `get_evidence_bundle` readback.
See [WABT provenance](https://github.com/morluto/rea/blob/main/third_party/wabt/README.md) and
[testing](testing.md#offline-wasm-artifacts) for real-tool verification.
[testing](../../development/testing.md#offline-wasm-artifacts) for real-tool verification.
## Compare core-module interfaces
@@ -51,7 +51,7 @@ Caller-owned JSON/base64 strings, input parsing and validation, transport
buffers, garbage-collector behavior, and other runtime memory are outside this
estimate; the budget does not promise a bound on total process memory.
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](../javascript/electron-observation.md).
## Provider authorities
@@ -65,7 +65,7 @@ vocabulary (`BrowserExclusionReason`): every denial reason in every stack maps
onto it, even where a wire schema keeps a narrower historical bucket.
Existing static application Evidence and passive web/Electron captures can be
combined later through
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
[JavaScript static/runtime reconciliation](../javascript/javascript-runtime-reconciliation.md).
## Start a browser
@@ -243,7 +243,6 @@ REA retains local observation data and removes only URL userinfo credentials, ex
- Console observations with a stack source retain call type, argument types, timestamp, and source location. When console text capture is selected, already-delivered primitive values are retained verbatim; objects, getters, and remote properties are never expanded.
- WebSocket observations retain direction, opcode, and payload byte length. When shape capture is selected, text frames are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw frames are never retained.
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache bodies above 64 KiB and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
- Script metadata is included only when CDP supplies a URL on an allowed origin. Stable keys exclude transient CDP script IDs, and exact transient raw URLs are used only during script/resource reconciliation. URL-less scripts are excluded because their origin cannot be established. When CDP supplies an execution-context association, the accepted script retains its authorized frame ID for later attribution. Script metadata admission bounds retained identity and source-map strings to 8 MiB and 50,000 records. When a source-map URL cannot fit, the script identity remains and source-map coverage reports `resource_budget_exhausted`; rejected script observations are counted without retaining their IDs. Source content is omitted unless explicitly requested and is returned inline as a self-verifying artifact with its SHA-256 digest.
Frames, resources, scripts, events, and workers outside the selected origin scope are excluded. An explicit `allowed_origins` list can include additional origins; excluded target details are counted without being exposed.
@@ -224,22 +224,6 @@ are 0 and 50. The view groups request lines by method, WHATWG URL origin and
path template, sorted by origin, template and method. Query strings, fragments,
credentials, headers and bodies are not copied into groups.
Select `include_query_parameter_names: true` in the view to include distinct
observed query names per group. The default and explicit `false` omit the
`query_parameter_names` field. Selected groups include an array of
`{name, record_ordinals}` entries, sorted by Unicode code point; each name's
ordinals follow the parent's selected record order and occur once per record.
For `/products?category=books&page=1` and
`/products?category=games&sort=price`, names are `category`, `page` and `sort`.
Query values and fragments are not copied into this inventory.
Names use WHATWG `URLSearchParams` form-query decoding: literal `+` becomes
space, `%2B` becomes `+`, percent escapes decode as UTF-8, and empty names are
retained. Empty query pairs are ignored. Original URLs stay in parent Evidence;
name decoding does not normalize the recorded paths used for grouping.
The inventory describes observed names, without inferring required parameters,
value types or a complete API schema. Sensitive-parent refusals still apply.
| Rule | Exact segment format | Placeholder |
| ---------- | ---------------------------------------------------- | ------------ |
| `numeric` | ASCII digits only | `{numeric}` |
@@ -285,16 +269,13 @@ and validation scratch by walking the parent without cloning it. Inline requests
borrow the Evidence value through CLI/MCP envelope validation; the application
admits the entire portable envelope before authenticated parsing or hashing.
The envelope clone, hash scratch and capture clone stay charged during grouping.
Temporary URL/path/MIME/query work and retained state must
Temporary URL/path/MIME work and retained state must
fit together within a 96 MiB projection representation budget. A whole group or
merge is admitted before allocating its retained sets, map entries or array
values. Fixed group storage, distinct text, exclusions and limitations count as
well as matched segments. Per-record reservations are released after each
record; repeated values do not accumulate another retained charge. Only the
requested page is materialized as output groups.
Opt-in query projection reserves parser tuples and the per-record name set
before decoding, then admits distinct retained names and source ordinals before
changing a group. Repeated names within a request retain that ordinal once.
The representation budget uses conservative storage and worst-case JSON-escaping
allowances; it is not an enforced V8 heap or process-memory limit. It is separate
@@ -82,4 +82,4 @@ closes only the owned browser; cleanup failure cannot return a successful trace.
`REA_BROWSER_NO_SANDBOX=true` passes Chromium's corresponding launch option when
the host requires it; it does not enable an REA security boundary.
See [testing](testing.md) for the native loading oracle and CLI/MCP real lane.
See [testing](../../development/testing.md) for the native loading oracle and CLI/MCP real lane.
+21 -18
View File
@@ -41,28 +41,31 @@ For a first result from an extracted JavaScript application or ASAR:
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
```
See [installation and setup](./installation.md) for requirements and configuration.
See [installation and setup](getting-started/installation.md) for requirements and configuration.
For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
For runtime instrumentation, see the [Frida guide](guides/runtime/frida-instrumentation.md).
## Choose a guide
| Task | Guides |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md) |
| Inspect assemblies, packages, or firmware | [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
| Capture and compare a command | [Process capture](process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
The [documentation directory](README.md) lists every guide by domain and explains
where authored references, specifications and generated records belong.
For development, use [testing](testing.md), [client verification](client-verification.md),
[tool design](tool-design.md), and [releasing](releasing.md).
[Reconstruction readiness](reconstruction-readiness.md) describes a development
conformance verifier. [The 6.2 migration](migration-6.2.md) records that release's
compatibility changes; [the roadmap](roadmap.md) tracks future work.
| Task | Guides |
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Inspect native functions, references, or types | [Native binaries](guides/native/native-investigation.md), [Go metadata](guides/native/go-binaries.md), [offline ELF layout](guides/native/binary-diagnostics.md), [recorded crashes](guides/native/recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](providers/ghidra-provider.md), [Windows](providers/windows-ghidra-p0.md), [DOS](providers/ghidra-dos.md), [raw bytes](providers/raw-binary.md), [MIPS](providers/ghidra-mips.md), [PSP](providers/ghidra-psp.md), [NativeAOT](providers/ghidra-nativeaot.md), [IDA](providers/ida-provider.md), [JEB](providers/jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](guides/javascript/javascript-artifact-reconstruction.md), [tracing and comparisons](guides/javascript/javascript-application-workflows.md), [application graph](guides/javascript/javascript-application-graph.md), [source recovery](guides/javascript/javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](guides/javascript/javascript-runtime-observation.md), [reconciliation](guides/javascript/javascript-runtime-reconciliation.md), [Electron observation](guides/javascript/electron-observation.md) |
| Inspect assemblies, packages, or firmware | [.NET](guides/managed/managed-code-analysis.md), [Android](guides/android/android-analysis.md), [Apktool resources](guides/android/apktool-resource-analysis.md), [ADB acquisition](guides/android/adb-device-analysis.md), [Apple](guides/apple/apple-application-analysis.md), [Harmony](guides/harmony/harmony-analysis.md), [firmware](guides/firmware/firmware-analysis.md), [WASM](guides/wasm/wasm-artifacts.md), [EVM](guides/evm/evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](guides/web/browser-observation.md), [scenarios](guides/web/browser-scenario-contract.md), [runtime evidence](guides/web/web-runtime.md), [historical captures](guides/web/web-network-captures.md) |
| Trace captured website source | [Source locations](guides/web/web-source-location.md), [module traces](guides/web/website-module-trace.md), [script export](guides/web/website-script-export.md) |
| Capture and compare a command | [Process capture](guides/runtime/process-capture.md) |
| Use CLI/MCP and retain evidence | [CLI](getting-started/cli.md), [MCP contracts](reference/mcp-contracts.md), [agent prompts](getting-started/mcp-prompts.md), [obligation ledgers](reference/reconstruction-obligation-ledgers.md) |
For development, use [testing](development/testing.md), [client verification](development/client-verification.md),
[tool design](development/tool-design.md), and [releasing](development/releasing.md).
[Reconstruction readiness](development/reconstruction-readiness.md) describes a development
conformance verifier. [The 6.2 migration](migrations/migration-6.2.md) records that release's
compatibility changes; [the roadmap](development/roadmap.md) tracks future work.
![REA inspecting a native binary in Hopper](./assets/rea-hopper-analysis.png)
@@ -38,7 +38,7 @@ if (reply.isError === true) {
Lifecycle and other non-Evidence tools still use their own advertised output
contracts. Do not assume every successful tool returns Evidence. Consult
[`tools/list` and the result guide](mcp-contracts.md#tool-results).
[`tools/list` and the result guide](../reference/mcp-contracts.md#tool-results).
A transport `resource_constraint` error can include a retained
`details.reported_limits.evidence_reference`. Use `inspect_analysis_view` or
@@ -79,7 +79,7 @@ rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/pat
In MCP, open the target without the incompatible `snapshot_path`, run the
required queries, then save to a new path through `close_binary.snapshot_path`.
Do not fabricate missing provider profiles or query bindings. See
[snapshot usage](cli.md#save-and-reuse-analysis-snapshots).
[snapshot usage](../getting-started/cli.md#save-and-reuse-analysis-snapshots).
Artifact inventory nodes identify content bytes. Path-specific role and
permissions now belong to each occurrence:
@@ -8,7 +8,7 @@ labels, functions, or memory bytes. The hydration semantics are adapted from
`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against
.NET's `MethodTable.h` at runtime commit
`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in
[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt).
[`licenses/nativeaot-parser-MIT.txt`](https://github.com/morluto/rea/blob/main/licenses/nativeaot-parser-MIT.txt).
The optional headless Ghidra adapter remains available for database annotations
and broader recovery. It uses the same pinned upstream commit as an opt-in
@@ -132,7 +132,7 @@ does not imply that ordinary native disassembly/decompilation is impossible.
## Verification
See [the dedicated lane](testing.md#optional-nativeaot-ghidra-analysis).
See [the dedicated lane](../development/testing.md#optional-nativeaot-ghidra-analysis).
Fixtures are source-built benign .NET programs with inheritance, an interface,
virtual dispatch, representative arithmetic and frozen strings. Independent
compiler/linker symbols and raw directory bytes provide the oracle. Stripped
@@ -5,18 +5,18 @@ read-only analysis and atomic function-name/entry-comment edits; Windows P0 is
read-only. GUI controls and persistent database mutation are unavailable.
Executable bytes remain unchanged.
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
Use [installation](../getting-started/installation.md#ghidra) for supported hosts, prerequisites,
configuration, startup deadlines, resource controls, and cleanup recovery.
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
snapshot parser and a separate optional annotation extension. Check the
[release boundary](installation.md#released-package-and-main) when using npm.
[release boundary](../getting-started/installation.md#released-package-and-main) when using npm.
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
See [import language and analysis seeds](../getting-started/installation.md#ghidra-import-language-and-analysis-seeds)
to override the header-detected language or seed analysis.
Provider selection and profile identity follow
[MCP contracts](mcp-contracts.md#identity-and-discovery). One deep
[MCP contracts](../reference/mcp-contracts.md#identity-and-discovery). One deep
provider stays bound to the target; runtime failure never selects another
provider automatically. The contracts below describe Ghidra observations,
not equivalence with another engine's pseudocode or analysis database.
@@ -73,7 +73,7 @@ convention beyond the ABI the pinned language models.
Ghidra 12.1.4 PUBLIC with a 64-bit JDK 21+ is required, and on macOS the release
tree ships no host decompiler binary, so `Decompiler/os/<platform>/decompile` and
`sleigh` must come from your own build; see
[installation](installation.md#ghidra).
[installation](../getting-started/installation.md#ghidra).
Run `npm run verify:ghidra:riscv` with an existing Ghidra 12.1.4 installation,
JDK 21+, and a RISC-V-capable Clang, LLD and LLVM readelf. The verifier acquires
@@ -147,7 +147,7 @@ Complete body ranges, typed reference edges, CFG, referenced data, source locals
## Verification
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](testing.md) for the distinction between adapter tests and real-provider evidence.
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](../development/testing.md) for the distinction between adapter tests and real-provider evidence.
Native verification requires `--second-procedure other_function --second-marker unique_second_body_text`; these options remain optional for the other profiles. The marker must occur in the second function's pseudocode and be absent from the first; the verifier also checks that returning to the first restores its text.

Some files were not shown because too many files have changed in this diff Show More