Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 N0zoM1z0
|
1784ca940a | ||
|
N0zoM1z0
|
36eca26b38 |
@@ -1,134 +0,0 @@
|
||||
---
|
||||
name: rea-changelog-update
|
||||
description: Prepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization.
|
||||
---
|
||||
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
## Inventory before prose
|
||||
|
||||
Fetch complete history and tags. Select the published baseline tag from
|
||||
`.release-please-manifest.json` on the release PR's **base branch**, and pin that
|
||||
branch's exact SHA as the editorial target. REA tags are `rea-agents-VERSION`.
|
||||
Use the proposed version, including its prerelease suffix; do not change the
|
||||
`always-bump-minor` strategy to imitate OpenClaw's calendar versions.
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs inventory \
|
||||
--repository morluto/rea --base rea-agents-PREVIOUS \
|
||||
--target FULL_BASE_BRANCH_SHA --version VERSION \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--summary \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
Read the inventory, not just the existing bot draft. It includes side-branch
|
||||
commits, GitHub-associated PRs whose merge commits are in the range, commit
|
||||
authors/co-authors, and confirmed closing issues with their reporters. GitHub
|
||||
connections are paginated. Network errors stop generation; rerun into a new
|
||||
output file after recovery. Existing output files are never overwritten.
|
||||
|
||||
Unresolved commit identities are retained as names/emails with `user: null`;
|
||||
never guess a handle or thank the merger as a substitute. The report preserves
|
||||
unknowns. This is a reviewed metadata snapshot, not independent proof that a
|
||||
human authored a patch. For unattributed identities, unusual merge strategies,
|
||||
backports, reversions, or references that GitHub did not associate, inspect
|
||||
the source PRs and diffs and report the limitation. The current collector uses
|
||||
confirmed PR-closing relationships for issue-reporter credit; unrelated issue
|
||||
mentions are not automatically credited. Do not claim that merely inventoried
|
||||
work still ships after a revert.
|
||||
|
||||
## Write for users and retain credit
|
||||
|
||||
Rewrite only the newest `## [VERSION]` section in `CHANGELOG.md`, preserving its
|
||||
version/date/link heading. Use single-line Markdown list items under:
|
||||
|
||||
- `### Highlights`: a few release-defining user outcomes; no quota for small releases.
|
||||
- `### Changes`: capabilities and meaningful behavior changes.
|
||||
- `### Fixes`: reliability, correctness, compatibility, and safety fixes.
|
||||
- `### ⚠ BREAKING CHANGES`: retain when applicable, with migrations and original
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior in short, concrete entries. Highlights summarize the release; avoid
|
||||
repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
|
||||
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Credit each represented contribution once in the public notes with its link
|
||||
and `Thanks @...` for its verified human author, known co-authors, and confirmed
|
||||
closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
|
||||
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
|
||||
type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `record --summary` provenance link verbatim at the end
|
||||
of this release. The linked inventory accounts for every discovered in-range
|
||||
PR and direct contribution, including internal work omitted from the prose.
|
||||
Do not paste the full `record` output into public notes: it duplicates thanks
|
||||
and obscures the user-facing changes. Full records remain available for review.
|
||||
Do not hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
Before changing the live release PR, have authorization for that PR update.
|
||||
Local preparation and validation can proceed independently.
|
||||
|
||||
1. Add `rea:release-notes-finalized` to the selected open release PR **before**
|
||||
editing its branch/body. Main proposal pushes respect this label. This is
|
||||
our workflow guard, not Release Please's `autorelease: snooze` mechanism.
|
||||
A proposal already running must finish before editing. Do not dispatch
|
||||
manual preparation over finalized notes.
|
||||
2. On that release branch, save the reviewed inventory as
|
||||
`docs/releases/VERSION.contributions.json`. This is durable public release
|
||||
provenance, not a temporary planning file. Do not add credentials or local
|
||||
environment dumps. Keep the existing version artifacts aligned.
|
||||
3. Run the existing checkpoint validator and the new checker:
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs check \
|
||||
--manifest docs/releases/VERSION.contributions.json
|
||||
node scripts/release-notes.mjs render \
|
||||
--manifest docs/releases/VERSION.contributions.json \
|
||||
--output /tmp/rea-VERSION.notes.md
|
||||
```
|
||||
|
||||
The checker supports inline PR bodies, including their surrounding metadata,
|
||||
up to GitHub's 65,536-character limit. It fails on larger notes rather than
|
||||
truncating credit; Release Please's overflow-link bodies are not integrated.
|
||||
|
||||
4. Update the release PR body with the rendered section while preserving its
|
||||
Release Please header/footer and component/version structure. Use a body
|
||||
file for `gh pr edit --body-file`; changing only `CHANGELOG.md` does not
|
||||
change Release Please's release body. Push only authorized changes.
|
||||
5. Freshly fetch the PR body to a file and run `check --pr-body FILE`. Review
|
||||
the complete diff and wait for current CI. Publication checks the body
|
||||
again before release creation. Avoid further body edits while publishing.
|
||||
|
||||
The finalization label freezes bot refreshes, not main. New product changes
|
||||
after the inventory target require refreshing the notes before publication.
|
||||
The workflow checks target ancestry and allows only release metadata paths
|
||||
after that target. It also requires the inventory target to equal the reviewed
|
||||
merge's first parent, so intervening main commits require a fresh inventory.
|
||||
Review diffs within the allowed release metadata paths too: the path guard does
|
||||
not prove that package dependency/script edits are version-only changes.
|
||||
To resume automatic proposals, remove the finalization label and let the bot
|
||||
regenerate; expect editorial changes to be replaced.
|
||||
|
||||
Existing releases without a contribution inventory retain their current
|
||||
publication path. Never retag or republish a package to correct release prose.
|
||||
@@ -1,112 +0,0 @@
|
||||
---
|
||||
name: rea-ci-tests-audit
|
||||
description: Audit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.
|
||||
---
|
||||
|
||||
# REA CI and tests audit
|
||||
|
||||
Establish which checks still catch real regressions and which costs can be
|
||||
removed without losing that protection. The outcome may be a verified fix,
|
||||
an evidence-backed recommendation, or an explicit decision to stop.
|
||||
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
|
||||
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
|
||||
and pinned toolchain, rather than commands remembered from an earlier audit.
|
||||
Preserve the requested scope: investigation alone does not authorize edits;
|
||||
this skill does not grant permission to publish issues, push, or merge. Existing
|
||||
user authorization still applies. Keep audit notes and raw results outside the
|
||||
tracked tree, normally in a task-specific `/tmp` directory.
|
||||
|
||||
## Establish the executed baseline
|
||||
|
||||
Fetch when the user asks for latest main. Pin the baseline SHA and inspected
|
||||
workflow revision; use an isolated worktree when changing or comparing code.
|
||||
Do not modify another task's checkout. Record dirty changes and the exact
|
||||
command, toolchain, host, selected tests, instrumentation, and prerequisites.
|
||||
An old run is evidence for its executed revision, even if main advances.
|
||||
|
||||
Trace workflow triggers, changed-path selection, job dependencies, reusable
|
||||
actions, package scripts, test discovery, artifact producers, and report gates
|
||||
through to the commands that actually execute. Include scripts, generated data,
|
||||
runtime registration, and subprocess boundaries missing from import graphs.
|
||||
Check positive and negative selection paths: required files must be discovered
|
||||
once per intended test configuration; deliberate platform/configuration repeats
|
||||
are separate coverage. A missing explicit test or missing required report must
|
||||
not turn green. Optional lanes may skip; preserve their stated prerequisites.
|
||||
|
||||
Use existing timing and failure artifacts to select suspicious families. Run
|
||||
relevant checks to establish a baseline; reserve a full local suite for broad
|
||||
CI diagnosis or uncertainty that focused checks cannot resolve. Do not rebuild
|
||||
or rerun an unchanged complete suite after every documentation edit.
|
||||
|
||||
## Prove the failure or protection
|
||||
|
||||
For each candidate, state the expected observable behavior and its owner,
|
||||
then construct the smallest reproduction against production behavior. Follow
|
||||
the producer and affected consumers before deciding where to fix it.
|
||||
|
||||
- For drift, compare fixtures with current admission requirements, protocol
|
||||
replies, named contracts, tool versions, and actual public entrypoints. A
|
||||
plausible-looking mock can fail before the behavior its test claims to cover.
|
||||
Search the whole affected fixture family, including other version strings.
|
||||
Preserve intentionally malformed fixtures and test their intended rejection.
|
||||
- For a suspicious green test, perturb the relevant behavior in an isolated
|
||||
copy or inject a controlled failure. Verify that the existing assertion or
|
||||
gate detects it. A test may merely repeat its mock, import abandoned code,
|
||||
or hide a subprocess/checker failure. Restore mutations after the experiment.
|
||||
- For shared setup or pruning, map distinct assertions to their retained
|
||||
scenarios. Preserve malformed inputs, cancellation, cleanup failure,
|
||||
permissions, capacity, identity, and meaningful concurrency interactions.
|
||||
A faster run that skips a case or weakens its oracle is not an improvement.
|
||||
- For provider coverage, separate fake-producer, boundary, and actual-engine
|
||||
evidence. Inspect named skips and prerequisites; fixtures do not establish
|
||||
real Ghidra, LLDB, browser, or other provider behavior.
|
||||
|
||||
Classify findings as reproduced defects, supported costs, hypotheses, or
|
||||
unavailable evidence. Record the baseline command, observed output/exit status,
|
||||
expected result, affected scope, and competing explanation. A failure on main
|
||||
may be pre-existing; that does not by itself make it irrelevant to a CI audit.
|
||||
Attribute PR failures using the executed revision and a clean baseline, not
|
||||
just their resemblance to a previous failure. Retry only to test a specific
|
||||
transient explanation; retain the original failure and stop when repetition
|
||||
adds no evidence. A passing retry does not establish a root cause or a fix.
|
||||
|
||||
## Assess efficiency without losing the gate
|
||||
|
||||
Read [measurement and CI checks](references/measurement-and-ci.md) when
|
||||
investigating slowness, caching, runner waits, shared artifacts, or log noise.
|
||||
Compare equivalent completed work before claiming a speedup. Distinguish
|
||||
setup/build time, test execution, coverage instrumentation, dependency waits,
|
||||
and runner scheduling; state which costs remain unmeasured.
|
||||
|
||||
Prefer removing proven duplicate preparation, reusing immutable setup, and
|
||||
reducing repeated subprocess work before shrinking correctness coverage.
|
||||
Preserve independent sessions where lifecycle or concurrency is the oracle.
|
||||
Keep live tests fresh; a cached previous pass cannot validate current host,
|
||||
installed provider contents, permissions, or external state.
|
||||
|
||||
## Fix, verify, and hand off within scope
|
||||
|
||||
If implementation is authorized, make the smallest structural change at the
|
||||
owning boundary and add a regression that fails on the baseline for the right
|
||||
reason. Retest affected consumers and meaningful negative paths; verify cache
|
||||
invalidation or discovery when those mechanisms change. Read the installed
|
||||
Turbo documentation before changing its configuration or commands.
|
||||
|
||||
Run the repository's applicable checks and inspect complete current-head CI
|
||||
results before an authorized merge. Preserve required coverage and discovery
|
||||
gates. Do not suppress a failure, add blanket retries, raise timeouts, or delete
|
||||
distinct assertions solely to make CI green. Changes to these controls need
|
||||
their own demonstrated reason and retained failure detection.
|
||||
|
||||
Report each confirmed issue with exact revision, reproduction, consequence,
|
||||
proposed correction, and validation limits. Report performance with workload
|
||||
and measurements, not just a percentage. Use English issue/PR records when
|
||||
requested; follow the user's assignment, commit identity, review, and merge
|
||||
preferences without assuming them for other users. Open ready-for-review PRs
|
||||
only when that lifecycle is authorized. Resolve necessary review findings and
|
||||
check final main against the tested tree after merging.
|
||||
|
||||
Stop when the authorized work is verified, remaining benefits are unproven,
|
||||
or further changes would trade correctness for speculative speed. List genuine
|
||||
uncertainties without turning them into additional optimization rounds.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "REA CI and tests audit"
|
||||
short_description: "Audit CI/test correctness and measured performance"
|
||||
default_prompt: "Use $rea-ci-tests-audit to investigate CI and test correctness, drift, and performance before proposing changes."
|
||||
@@ -1,65 +0,0 @@
|
||||
# Measurement and CI checks
|
||||
|
||||
## Equivalent completed work
|
||||
|
||||
Record the source/workflow SHA, command, test identities and counts, skips,
|
||||
assertions or output contract, exit status, coverage scope and thresholds,
|
||||
toolchain/provider versions, host resources, and prepared artifacts. Inspect
|
||||
both the child result and its report: a profiling wrapper may exit successfully
|
||||
after the measured command failed. Focused tests can pass while global coverage
|
||||
thresholds fail; describe both rather than calling the command green.
|
||||
|
||||
Measure wall time, user/system CPU, and peak process-family memory for relevant
|
||||
completed tasks. Parent-only RSS misses workers and their children. On Linux,
|
||||
summed RSS can double-count shared pages; report RSS/PSS semantics, sampling
|
||||
interval, missing samples, and observed cleanup limits. Use equivalent host
|
||||
tools elsewhere. State unknown costs, including agent tokens; fewer log bytes
|
||||
alone prove neither fewer tokens nor faster tests.
|
||||
|
||||
Separate cold execution, warm build/file caches, and warm runtime caches.
|
||||
Use repeated or alternating runs when variance affects the decision, with
|
||||
enough observations to distinguish the proposed benefit from noise. Keep
|
||||
workload, instrumentation, and host conditions comparable; record interference.
|
||||
A run with fewer tests, early rejection, lost coverage, or a different runner
|
||||
does not prove suite acceleration. CI-wide claims need end-to-end measurements,
|
||||
not an extrapolation from one improved family.
|
||||
|
||||
## Workflow, caches, and shared artifacts
|
||||
|
||||
Trace each reused result from producer to consumers and identify its lifetime.
|
||||
Check key/input coverage against the actual file inventory and result-affecting
|
||||
configuration, environment, generated data, tool versions, and host state.
|
||||
Exercise an affected input change to prove invalidation; exercise an irrelevant
|
||||
change to support reuse where useful. Cache deterministic preparation and
|
||||
static analysis only when their inputs fully describe the result.
|
||||
|
||||
Verify downloaded artifacts match the expected revision, platform, runtime,
|
||||
and schema. Check restore misses, failed/partial producers, stale reports,
|
||||
removed output files, and consumers after producer failures. Shared writers
|
||||
need output ownership or locks; tests need isolated state, not an unnecessary
|
||||
global lock. Worker counts do not bound children spawned within each test.
|
||||
Preserve intentionally concurrent scenarios while bounding unrelated batches.
|
||||
|
||||
Check matrix expansion and all job pages before saying CI passed. Inspect
|
||||
required gates after failed, cancelled, and skipped dependencies. `always()`
|
||||
alone does not preserve failures; inspect the aggregation logic. Test/report
|
||||
uploads should retain available diagnostics after failure, while absent or
|
||||
malformed required results must fail closed. Do not demand an optional report
|
||||
from a lane intentionally excluded by the selector.
|
||||
|
||||
## Runner waits and console noise
|
||||
|
||||
Use per-job and per-step timestamps where available. Workflow creation to job
|
||||
start includes dependencies and other delays; it is not necessarily runner
|
||||
queue time. Identify when the job became eligible before attributing waits to
|
||||
runner capacity. If eligibility/queue timestamps are unavailable, report the
|
||||
combined delay and unknown split. Compare equivalent lanes and runner types
|
||||
across enough runs; a single busy runner is not a workflow regression.
|
||||
|
||||
For console reductions, retain checker exit codes, actionable errors, useful
|
||||
summary counts, and complete machine-readable artifacts. Verify authored code
|
||||
remains in the actual checker inventory when excluding generated outputs.
|
||||
Introduce an authored error as a control; it must still fail. An ignore pattern
|
||||
can hide real source as well as noise. Show that cached reports are restored,
|
||||
stale successful reports cannot mask current failures, and report parse errors
|
||||
remain failures. Measure readability or byte reduction separately from runtime.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
|
||||
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
|
||||
contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
For implementation or verification, consult [testing](../../../docs/testing.md)
|
||||
for the affected consumer and provider lanes, and
|
||||
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
|
||||
provider/workflow guides only as the task requires.
|
||||
@@ -1,285 +0,0 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "41"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
Use REA when a claim depends on a shipped binary or package, decompilation,
|
||||
passive application runtime evidence, or comparison with
|
||||
behavior not established by available source. For ordinary analysis of a
|
||||
complete source repository, use normal repository tools and do not run REA
|
||||
readiness or provider commands.
|
||||
|
||||
## Connect only when needed
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
investigation. Use the connected server's actual tool list and input schemas;
|
||||
a skill installed from repository main may describe capabilities absent from
|
||||
an older npm release. Keep complete inline Evidence when that server does not
|
||||
advertise retained references.
|
||||
|
||||
If tools are absent, a call is blocked by the client, or a required provider is
|
||||
unavailable, read [connection and recovery](references/connection-and-recovery.md).
|
||||
Diagnose the observed blocker, continue useful supported work, and resume the
|
||||
original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Binwalk or Unblob.
|
||||
|
||||
## Route the target first
|
||||
|
||||
Choose the first tool from the target the user supplied. Use `open_binary` for
|
||||
active-target native or archive workflows; target-free tools take their own
|
||||
explicit path or endpoint and do not need it.
|
||||
|
||||
- ASAR or extracted JavaScript/Electron tree:
|
||||
`analyze_javascript_application`.
|
||||
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX or DMG):
|
||||
`open_binary` with the supplied local path; use `inspect_artifact` when its
|
||||
graph and findings help answer the question.
|
||||
- Android APK code, classes, methods, or incoming references:
|
||||
`inspect_android_package`, then focused Android tools when advertised.
|
||||
Archive member inventory still uses the archive route above.
|
||||
- Managed PE/CLI assembly: `inspect_managed_artifact`.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
connect Everywhere's separate MCP server for live traffic.
|
||||
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
|
||||
original record ordinals when useful; inspect producer fields and byte/number
|
||||
sidecars without fetching recorded URLs or claiming live attribution. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
pages; select one trace, stream ID and direction for contribution/range pages.
|
||||
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
|
||||
and PCAP correlation are unsupported.
|
||||
- User-owned Electron runtime already open: `list_electron_targets`.
|
||||
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
This does not perform live capture.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
or selected decoder help, including channels and options. Query and page the
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
This reads existing files; it does not acquire hardware signals.
|
||||
- Explicit local WebAssembly artifact: `inspect_wasm_artifact`. Supply WABT 1.0.42
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. Preserve carrier/decoded digests and treat selectors,
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. For the complete inferred interface without hex or
|
||||
duplicate stdout, discover support and select `detail: "summary"`; complete
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
first (compact inventory by default), or select an exact ordinary table
|
||||
and `columns` when rows are needed. Defaults are selected-table schema and
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
- Native executable, library, or analysis database: `open_binary`, then
|
||||
use focused analysis tools directly; call `binary_overview` when metadata or
|
||||
inventory context is useful and available from the selected provider.
|
||||
|
||||
For an existing IDA MCP configuration, use `open_binary` with the original
|
||||
input binary and `provider_id: "ida"`; do not pass an `.idb` or `.i64` database.
|
||||
The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
facets remain unknown; live IDA results are not replayed from snapshots.
|
||||
|
||||
If the user says only "my app" without a name, path, or target already selected
|
||||
for this task, ask which app before searching for artifacts. Paths in unrelated
|
||||
saved reports and nearby test fixtures do not select the user's target. Resolve
|
||||
an actual human-readable app name to one clear installed artifact when possible;
|
||||
ask when matches are ambiguous. Never choose an example app on the user's behalf.
|
||||
|
||||
In a target-free session, use `open_binary` to bind any archive/package or
|
||||
native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
Start with the default result and use its inline Evidence and graph context.
|
||||
For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
Recheck session state when a lifecycle change makes that status useful.
|
||||
Make a focused follow-up only when the
|
||||
returned result leaves a specific question unanswered.
|
||||
|
||||
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
||||
`inspect_analysis_view` with its reported same-session Evidence reference for a
|
||||
summary, one section/module, or a stable page. Use `trace_application_feature`
|
||||
once a seed is known, or call `export_evidence_bundle` to write the complete
|
||||
session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
canonical JSON or repeat the producer. For JavaScript, read the
|
||||
[module-view and trace recipes](references/javascript-applications.md) before
|
||||
drilling into a large graph; a broad bidirectional trace can expand unrelated
|
||||
containment relationships even after summary analysis.
|
||||
|
||||
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
||||
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
||||
that static analysis observed execution. Runtime requests execute the declared
|
||||
target and lifecycle; do not broaden the target or action beyond those fields.
|
||||
|
||||
Within the user's requested investigation, call available analysis tools
|
||||
directly. REA does not require permission grants or per-call approval flags.
|
||||
The agent client can independently require approval. Follow the declared request
|
||||
scope and the host's actual access requirements; use the recovery guide for an
|
||||
observed client-policy rejection.
|
||||
|
||||
## Plan broader investigations
|
||||
|
||||
For requests that span multiple features or subsystems, use a staged workflow:
|
||||
|
||||
1. Turn the request into a checklist of questions and the evidence each answer
|
||||
needs. Resolve target identity and constraints from the conversation and
|
||||
workspace before asking for information again.
|
||||
2. Inspect the current REA session, artifact identity, saved analysis database,
|
||||
bookmarks, and prior evidence. Reuse matching state; do not open duplicate
|
||||
sessions or repeat identical analysis.
|
||||
3. Start with the smallest useful overview or inventory. Follow each question
|
||||
from its entry point through relevant data and state changes to its result.
|
||||
Batch related operations around a specific hypothesis, then expand only when
|
||||
the returned evidence leaves a concrete gap.
|
||||
4. Inspect relevant packaged resources and configuration alongside code when
|
||||
they affect the question. Use format-aware inventory and parsers; do not
|
||||
infer behavior from filenames, strings, or layout alone.
|
||||
5. Corroborate a conclusion with the evidence type it requires. Use runtime
|
||||
observation only when static evidence cannot answer the question and the
|
||||
required host runtime and OS access are available. For JavaScript behavior,
|
||||
run probes against the actual app through the available browser, Electron,
|
||||
or process capture workflow.
|
||||
6. Decompose work into independent questions. When parallel workers are
|
||||
available and the questions do not depend on one another, assign distinct
|
||||
scopes, point workers to existing evidence, and ask them to return sources,
|
||||
conclusions, and unresolved gaps. Otherwise, work sequentially.
|
||||
7. Keep a concise finding ledger linking each conclusion to Evidence IDs,
|
||||
confidence/evidence type, search boundary, and remaining unknowns. Update
|
||||
the shared index or investigation report so later passes can reuse results.
|
||||
|
||||
Before finishing, revisit the original checklist. Mark each question as
|
||||
answered, partially answered, or unresolved based on its evidence; keep
|
||||
bounded negative searches bounded, and do not describe a broad investigation
|
||||
as complete while required questions remain open.
|
||||
|
||||
## Read only the relevant guide
|
||||
|
||||
- Native binaries, managed assemblies, archives, and extraction:
|
||||
[references/native-and-artifacts.md](references/native-and-artifacts.md)
|
||||
- ASARs, extracted JavaScript, feature tracing, and version comparison:
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Passive browser/Electron observation and static/runtime reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -1,127 +0,0 @@
|
||||
# Connection and task recovery
|
||||
|
||||
Read this guide only for an observed connection, command, approval or provider
|
||||
blocker. Keep the user's target and question in view. Explain what prevented the
|
||||
operation and the next useful step; after repair, continue the investigation.
|
||||
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
CODEX_HOME is under `/tmp`; use an authorized profile outside the OS temporary
|
||||
directory. REA setup cannot repair the client's executor. Preserve the sandbox
|
||||
instead of silently disabling it to make an analysis command work.
|
||||
|
||||
Static JavaScript and ordinary ZIP inventory need neither Hopper nor Ghidra.
|
||||
HAR inspection is offline and does not require a live browser. Native
|
||||
decompilation, Android analysis and firmware extraction have their own
|
||||
bring-your-own prerequisites. Use supported inventory/static evidence when it
|
||||
answers part of the question, and name the unavailable evidence separately.
|
||||
Do not substitute another target. Label observations from ordinary shell tools
|
||||
as such; only actual REA output supplies REA Evidence.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
|
||||
```bash
|
||||
rea doctor --client codex --json
|
||||
rea setup --client codex --dry-run --json
|
||||
```
|
||||
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
then apply the same scope with `rea setup --client codex --yes`. Add
|
||||
`--install-hopper` only when separately needed and explicitly approved. Do not
|
||||
install or upgrade unrelated prerequisites. Restart/reconnect, verify tools
|
||||
actually appear, and resume the original task. If they remain absent, inspect
|
||||
the launch failure rather than looping through setup.
|
||||
|
||||
## Codex tool approval is separate from REA readiness
|
||||
|
||||
REA tools have no permission-grant handshake. Codex can still require approval
|
||||
because an operation creates owned resources, launches a helper or has cleanup
|
||||
effects. An advertised tool or successful doctor result does not override that
|
||||
client policy. `workspace-write` alone does not resolve an MCP approval
|
||||
rejection, and `approval_policy = "never"` does not mean every MCP tool is
|
||||
preapproved.
|
||||
|
||||
When interactive approval is available, use the client's approval flow for the
|
||||
requested operation. In an unattended run, continue through an authorized CLI
|
||||
workflow if the execution environment permits it, or report the blocked
|
||||
operation and the approval needed to proceed. Do not retry the same rejected
|
||||
call, silently broaden approvals, or rewrite MCP effects as read-only.
|
||||
|
||||
For an operator who explicitly wants to preapprove named operations in a
|
||||
disposable Codex profile, current Codex supports per-tool configuration. For
|
||||
example, opening and closing an analysis session can be scoped independently:
|
||||
|
||||
```toml
|
||||
[mcp_servers.rea.tools.open_binary]
|
||||
approval_mode = "approve"
|
||||
|
||||
[mcp_servers.rea.tools.close_binary]
|
||||
approval_mode = "approve"
|
||||
```
|
||||
|
||||
Use the actual server ID and only the operations whose effects were approved;
|
||||
this example does not approve inspection, extraction or runtime execution.
|
||||
Show the proposed config edit and preserve unrelated entries and backups before
|
||||
writing. Managed policy can constrain client settings. Consult the
|
||||
[Codex configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
```bash
|
||||
command -v rea
|
||||
command -v node
|
||||
node --version
|
||||
rea --version
|
||||
```
|
||||
|
||||
Use the public launcher at its known installation path with a supported existing
|
||||
Node on PATH when the child shell loses a prefix installation. Avoid internal
|
||||
`dist/cli.js` entrypoints or global toolchain changes. MCP may have its own valid
|
||||
launcher even when a shell cannot resolve `rea`.
|
||||
|
||||
For a user-supplied JavaScript tree or ASAR, select output files in the authorized
|
||||
workspace and save complete Evidence before reading a view:
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
These commands do not register MCP or install an analysis engine. CLI JavaScript
|
||||
JSON is complete and can be very large; redirect it rather than loading it into
|
||||
context. Use an existing JSON processor if `jq` is absent, without installing
|
||||
one just for this recipe. Inspect a module page or item using the same inline
|
||||
source and the [JavaScript recipes](javascript-applications.md). Preserve graph
|
||||
coverage, limitations and unknowns. Native CLI analysis still requires its
|
||||
selected engine, and all CLI operations retain the host's execution permissions.
|
||||
@@ -1,67 +0,0 @@
|
||||
# Evidence, comparison, and verification workflows
|
||||
|
||||
Evidence-producing tools return the complete Evidence record directly in MCP
|
||||
`structuredContent` and CLI JSON. Read `normalized_result` for operation data,
|
||||
and cite `evidence_id` when another tool or your explanation needs a stable
|
||||
reference. Preserve authority, limitations, coverage, and residual unknowns. If a result
|
||||
is incomplete or paginated, continue only when the remaining data matters to
|
||||
the task; do not fetch a bundle or resource merely to read a result already
|
||||
returned inline. When complete `inspect_binary_layout` or
|
||||
`analyze_javascript_application` Evidence is already retained, call
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
Other inventory collections are `nodes`, `edges` and
|
||||
`integrity_contradictions`. Keep `source_complete` separate from page exhaustion:
|
||||
a missing match in partial source Evidence does not establish absence. Use the
|
||||
returned parent Evidence ID for subsequent views and export before closing the
|
||||
session when the source must survive it.
|
||||
When `tools/list` advertises `inspect_artifact.detail`, request `"summary"` to
|
||||
retain its complete source and receive that view immediately. Without that
|
||||
advertised choice, keep the existing complete call. CLI inspection returns
|
||||
portable complete Evidence; save it before projecting a selected inline view.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
Use `update_unknown` with the current revision; reread after a stale revision
|
||||
instead of retrying blindly. Only qualifying observed evidence can verify a
|
||||
resolution.
|
||||
|
||||
Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
Process captures are opt-in behavioral evidence, not a security sandbox. V3
|
||||
captures cannot be upgraded to V4; rerun the original scenario. Distinguish root
|
||||
exit from descendant settlement and require freshness when the task needs it.
|
||||
A timed-out target that must write a final report needs `finalization_ms`: REA
|
||||
sends SIGTERM to the captured root, keeps observing, and sends SIGKILL once
|
||||
only if the target has not exited by the end of the interval.
|
||||
`exit.finalization.signals` lists each attempt with its delivery result; the
|
||||
observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
||||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
`inspect_web_network_capture` HAR Evidence to group observed request lines by
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
@@ -1,175 +0,0 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`recover_javascript_sources` with an absolute `path` and absent
|
||||
`output_directory`. Supply related runtime/shared/lazy files in
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
schema advertises `source_paths`, select exact inventoried paths such as
|
||||
`["package.json", "main.js", "preload.js"]`. Include supporting manifests or
|
||||
source maps explicitly; dependencies are not implicitly analyzed. REA retains
|
||||
the complete artifact inventory and identity but reads source text and analyzes
|
||||
only selected files. The result and summary record `source_selection`; files
|
||||
outside scope and their relationships remain unknown, so partial application
|
||||
coverage does not mean the selected source failed. Use repeated `--source-path`
|
||||
on either static application CLI route. Omit the selection when the question
|
||||
requires complete application analysis. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
packages use their own map. Wildcard, recursive, and external-package import
|
||||
targets stay unresolved with that limitation. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
Evidence record with a distinct ID and a `normalized_result.parent_evidence_id`
|
||||
for the complete retained application analysis. Source-map contents are part of
|
||||
that static analysis. Read findings, coverage, limitations and unknowns, then
|
||||
follow a specific unanswered question. Older servers without `detail` return
|
||||
the complete application Evidence inline when it fits.
|
||||
|
||||
When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
|
||||
these select the isolated source analyzer's old-space heap and per-source
|
||||
deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
|
||||
process heap. A Bun-hosted REA does not enforce the heap choice; on Windows it
|
||||
instead limits the analyzer's committed memory to that choice plus a 512 MiB
|
||||
runtime reserve.
|
||||
Resource-limited analyses preserve completed static facts and
|
||||
report partial semantic coverage with specific unknown scopes; this does not
|
||||
mean the source is malformed. Inspect those limits before retrying a source.
|
||||
When expanded relationships exceed the parent's available memory, inspect the
|
||||
file's modules item view for retained `static_analysis` and
|
||||
`semantic_module_analysis`. Their facts remain available even when graph links
|
||||
are unknown. Cumulative result exhaustion can stop analysis before every file
|
||||
is examined; the inventory and remaining unknown scopes distinguish that case.
|
||||
Timeout or cancellation after source analysis begins retains partial Evidence
|
||||
in the same session. A returned `error.details.partial_observation` reference
|
||||
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
|
||||
the Evidence bundle before inspecting the partial analysis.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
The following IDs are placeholders to replace with returned values. When the
|
||||
module's node ID is not already known, page identities before selecting an item.
|
||||
Pages return only node IDs, paths and roles; choose a useful page size and follow
|
||||
`coverage.next_offset` when another page matters:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": { "kind": "page", "collection": "modules", "offset": 0, "limit": 16 }
|
||||
}
|
||||
```
|
||||
|
||||
Select the relevant page entry by its exact `node_id` and recorded role:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "item",
|
||||
"collection": "modules",
|
||||
"selector": { "node_id": "jag_node_ACTUAL_NODE_ID" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Module pages include asset, bundled and source module roles, so one path can
|
||||
identify several entries. Use `selector.path` only when uniqueness is established;
|
||||
the returned node ID avoids needing a complete inventory to prove uniqueness. On ambiguity,
|
||||
select the matching node ID from the page or the error's candidates. Follow
|
||||
`next_offset` when another page matters. An item exposes that module node's
|
||||
recorded observations, identity, hashes, exports and source ranges; it does not
|
||||
include every child syntax observation, related edge or original source file.
|
||||
An absent item fact is not evidence of absence from the whole application.
|
||||
|
||||
## Trace only the remaining question
|
||||
|
||||
Use `trace_application_feature` on the parent application Evidence for one
|
||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||
Select `incoming` for relationships leading to the seed, `outgoing` for those
|
||||
leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
matches exactly.
|
||||
|
||||
## Recover an oversized display
|
||||
|
||||
An oversized response reports `resource_constraint` and
|
||||
`details.resource: "transport"`. Reuse
|
||||
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
||||
summary, one module, or a stable page of module identities.
|
||||
`export_evidence_bundle` writes the complete canonical session to a
|
||||
caller-selected file. These workflows keep the original Evidence and coverage;
|
||||
a broad follow-up can also exceed framing.
|
||||
|
||||
A client can truncate a successful result inside its text envelope without a
|
||||
REA `resource_constraint`. Treat a truncation notice or incomplete JSON as a
|
||||
display limitation; do not claim the full record was examined. Reuse the known
|
||||
parent ID for smaller views, or export complete Evidence to a selected file
|
||||
while the session remains open. If the parent ID was lost from the preview,
|
||||
use the prior analysis summary or explicit ledger entries to recover it; do
|
||||
not invent an ID or rerun analysis just to reproduce the same display.
|
||||
|
||||
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
||||
registrations, utility processes, and native binding requests are static syntax
|
||||
observations. Only a unique exact literal IPC channel match supports an inferred
|
||||
pairing. Dynamic or ambiguous channels remain unresolved. A requested `.node`
|
||||
member is not a verified native export. Never claim runtime reachability,
|
||||
registration, defaults, or policy enforcement from static analysis.
|
||||
|
||||
For version comparison, analyze each version once, then call
|
||||
`compare_application_versions`. Accept only its digest, source-map, structural
|
||||
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
||||
are not persistent identity. Report added or removed only with complete
|
||||
opposite-side coverage; otherwise report unknown.
|
||||
|
||||
When the question asks how one exact exported callable's returned object shape
|
||||
changed, analyze each version once and then call
|
||||
`compare_javascript_export_shapes` with explicit module paths and export names.
|
||||
Use the returned IDs on the same connection, or complete inline Evidence
|
||||
records, from both analysis calls. Accept variant
|
||||
pairing only through the tool's unique exact literal discriminant. Read
|
||||
`property_inventories` and each change's `presence` before treating `unknown` as
|
||||
"the name was not observed": inventories list observed property names even when
|
||||
values stay unknown, with a `source_range` for each paired or unpaired variant.
|
||||
Array holes and mutation-invalidated slots are not observed properties.
|
||||
Complete parent coverage reports presence-only
|
||||
`added`/`removed` independently of unresolved values. Cite the
|
||||
comparison Evidence and report JSON Pointer changes; dynamic values, ambiguous
|
||||
variants, and incomplete parent-property coverage stay unknown. This is static
|
||||
inference, not runtime behavior. When runtime semantics are needed, run
|
||||
behavioral probes against the relevant application versions and capture them
|
||||
through the available browser, Electron, or process workflows.
|
||||
|
||||
## Reusing application Evidence
|
||||
|
||||
When advertised by the connected server, application trace and compare tools
|
||||
accept complete inline Evidence or
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`
|
||||
for their application input (`application`, or `left`/`right`). `inspect_analysis_view`
|
||||
uses the same retained-reference form in `source`, or portable inline Evidence,
|
||||
to project a summary, module page, or one module with its recorded observations. This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
@@ -1,249 +0,0 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
inspect historical thread registers, signals, raw notes and original file
|
||||
ranges. It is independent of the active disassembler target. Source-bound
|
||||
register values and note bytes are observations; missing notes, signal-thread
|
||||
association and current executable/library identity remain unknown. PIDs are
|
||||
historical metadata and never authorize live attach or control.
|
||||
|
||||
The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
|
||||
Use `inspect_binary_layout` with an explicit local ELF path for file-backed
|
||||
sections, symbols, relocations, linkage names, and static mitigation inferences
|
||||
without opening a disassembler target. When that complete record is already
|
||||
retained, call `inspect_analysis_view` for a summary, mitigations or linkage
|
||||
facet, one section or symbol, or a stable page. Do not re-run the decoder to
|
||||
read a single object.
|
||||
|
||||
## Offline WASM
|
||||
|
||||
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
|
||||
For counts, validation and WAT identity, select `detail: "summary"` on the first
|
||||
MCP call. The complete original Evidence stays retained; use
|
||||
`normalized_result.parent_evidence_id` for exact exports or text pages. Omitted
|
||||
or complete detail keeps the existing complete result.
|
||||
For a completed result, use `inspect_analysis_view` with its reported retained
|
||||
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
|
||||
Facets also include sections, imports, candidates, glue, WAT, headers, details
|
||||
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
## Native targets
|
||||
|
||||
After `open_binary`, use focused search, procedure, or function tools directly.
|
||||
Use `binary_overview` when target metadata or inventory context helps answer the
|
||||
question. Prefer literal search, names, decompilation, callers, callees, and
|
||||
cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
Native facets also include procedure, pseudocode, assembly, basic blocks,
|
||||
comments, callers/callees, references, unresolved calls, referenced strings and
|
||||
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||
code units; use the returned `next_offset` for continuation. Provider limitations
|
||||
and unavailable facts remain explicit; the complete parent Evidence is retained.
|
||||
|
||||
`inspect_macho` on a large framework returns megabytes of imports and exports.
|
||||
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
|
||||
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
|
||||
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
|
||||
architectures, build_metadata, load_commands, dependencies, imports, exports,
|
||||
segments and provenance. `source_exhaustive: false` means unreturned symbols
|
||||
may exist.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## DOS MZ with Ghidra
|
||||
|
||||
For an admitted DOS MZ target, use `inspect_native_load_image` to check measured
|
||||
loaded bytes, file mappings, relocations and the entry against the immutable
|
||||
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
|
||||
reported static image, not runtime DOS or PC-98 hardware behavior.
|
||||
|
||||
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
|
||||
|
||||
Use `read_bytes` for initialized analysis memory and `address_to_file_offset` to
|
||||
anchor one observed address to original file bytes. Loader fixups can change a
|
||||
word; a source offset does not imply byte equality. Complete function body ranges
|
||||
use inclusive ends and can contain gaps. Do not treat the enclosing span as code.
|
||||
For packed targets, retain the original and separately derived artifact identities;
|
||||
decompiling an unpacking stub does not recover the unpacked program.
|
||||
|
||||
When reading Objective-C dispatch metadata, distinguish class implementation
|
||||
methods from protocol requirements: class methods report null requirement
|
||||
flags, while decoded protocol declarations retain required and optional facts.
|
||||
Property attributes remain raw name/value tokens; use the property's
|
||||
`atomicity` field for the decoded `N` marker or the default atomic behavior.
|
||||
Null class/meta flags, ivar counts, atomicity, or read-only values mean the
|
||||
selected provider did not establish that fact. Symbol names alone do not prove
|
||||
that a class is a root class or has no ivars.
|
||||
|
||||
## Managed PE/CLI
|
||||
|
||||
Start with `inspect_managed_artifact`. REA's canonical managed inspection is
|
||||
execution-free: do not claim it loaded, reflected, executed, or resolved the
|
||||
assembly. Keep managed/native boundaries and unavailable reconstruction facts
|
||||
explicit. A bring-your-own reconstruction oracle is separate from the canonical
|
||||
parser and must not become an implicit setup dependency.
|
||||
|
||||
For identity/classification questions, `inspect_managed_artifact` can explicitly
|
||||
select `detail: "summary"` on the first MCP call. It retains complete metadata
|
||||
and returns a linked overview with full classification/evidence, nullable
|
||||
module/MVID, frameworks, metadata coverage/issues/limitations and returned-row
|
||||
counts. Default complete requests remain unchanged. Read original rows with
|
||||
`inspect_analysis_view`, source `normalized_result.parent_evidence_id`, and
|
||||
`view: {kind: "managed", collection: "attributes", offset: 0, limit: 16}`.
|
||||
Collections are `references`, `resources` and `attributes`. Paging does not
|
||||
repeat parsing. Returned-row exhaustion
|
||||
does not prove complete metadata or absence under partial/unavailable coverage;
|
||||
preserve build-local tokens and their artifact/MVID scope.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
NativeAOT binaries contain native machine code, so ordinary CIL decompilers
|
||||
cannot recover the original method bodies. When managed metadata/CIL is absent,
|
||||
classify that as a native-analysis route rather than claiming source recovery.
|
||||
Open the native image with the selected deep provider and inspect functions,
|
||||
strings, calls, and references; describe decompiler output as pseudocode and
|
||||
inference.
|
||||
|
||||
NativeAOT images can be PE/COFF, ELF, or Mach-O executables and shared
|
||||
libraries. Use file bytes and loader metadata rather than suffixes to identify
|
||||
them. For packaged apps, inventory/extract with REA's artifact tools and retain
|
||||
the exact contained image identity. PDB, ELF debug, or dSYM sidecars are
|
||||
additional evidence only when the provider accepts and matches them.
|
||||
|
||||
REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR
|
||||
9.1 metadata. An optional headless Ghidra adapter adds database annotations and
|
||||
broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
no database mutation authority.
|
||||
|
||||
## Packages and extraction
|
||||
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
|
||||
`project_apple_application_graph`. It reports application roots and nested
|
||||
bundles by path convention: app extensions, XPC services, frameworks, login
|
||||
items, system and driver extensions, and plug-ins. It also lists privileged
|
||||
helpers, launchd plists, symlinks, and each bundle's `info_plist_path` and
|
||||
executable candidates. Roles are path conventions, not parsed plists; read the
|
||||
listed plists with `inspect_plist`.
|
||||
|
||||
After `open_binary` on a `.app` or Mach-O, use `trace_dylib_resolution` to see
|
||||
which file each `@rpath`, `@loader_path` and `@executable_path` load reaches for
|
||||
every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`@rpath` candidates that are absent. Narrow large bundles with `roots` or
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files are needed. It extracts the active
|
||||
target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
claim child inventory when only root identity is available. Extraction is a
|
||||
separate filesystem-writing operation.
|
||||
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, or `6502`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
that address. Hopper and IDA refuse this interpretation; Ghidra uses
|
||||
`BinaryLoader` with a little-endian generic language. Big-endian, Thumb-only,
|
||||
and Cortex-M variants are unsupported. `6502` selects NMOS 6502; the entry and
|
||||
complete file mapping must fit within `0x0000..0xffff`. 65C02, 65816, cartridge
|
||||
containers, bank switching, and mapper semantics are unsupported. Windows
|
||||
Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
Use `inspect_pe_resources` with an absolute `path`, or
|
||||
`rea inspect-pe-resources ./selected.exe --json`, for portable PE32/PE32+
|
||||
resource inspection without opening a native target. Preserve returned Evidence:
|
||||
type/name/language IDs and UTF-16 names, original directory/data/payload ranges,
|
||||
code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
@@ -1,8 +0,0 @@
|
||||
# Keep text checkouts consistent with the repository formatter on every platform.
|
||||
* text=auto eol=lf
|
||||
|
||||
# Windows command fixtures retain their native line endings.
|
||||
*.cmd text eol=crlf
|
||||
|
||||
# Preserve whitespace in original upstream licensing texts.
|
||||
licenses/npm-runtime-notices.txt whitespace=-blank-at-eol
|
||||
@@ -1,8 +1,5 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Contribution scope and integration requirements
|
||||
url: https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope
|
||||
about: Check supported capabilities, excluded workflows, and the fail-closed integration rule before proposing a feature.
|
||||
- name: REA Documentation
|
||||
url: https://github.com/morluto/rea#readme
|
||||
about: Read the README and AGENTS.md for usage guides and configuration details.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
name: Design discussion
|
||||
description: Discuss an REA architecture, provider, contract, or scope decision
|
||||
description: Resolve an REA architecture, provider, or contract question before implementation
|
||||
title: "[Design] "
|
||||
labels:
|
||||
- enhancement
|
||||
@@ -9,12 +9,7 @@ body:
|
||||
attributes:
|
||||
value: |
|
||||
Use this for a substantial provider, public/persisted contract, or
|
||||
authority-boundary decision. One question and a short explanation are
|
||||
enough to start. Choose "Not sure yet" where needed; maintainers can help
|
||||
with technical details, alternatives, and verification.
|
||||
|
||||
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Unresolved boundaries block integration, not opening a discussion.
|
||||
authority-boundary decision. Keep one decision per discussion.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
@@ -23,16 +18,14 @@ body:
|
||||
options:
|
||||
- label: I searched the open and closed issues and pull requests for a related discussion.
|
||||
required: true
|
||||
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
|
||||
required: true
|
||||
- label: This discussion does not include credentials, private samples, or sensitive target data.
|
||||
- label: I removed credentials, tokens, private paths, and sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: decision
|
||||
attributes:
|
||||
label: Decision needed
|
||||
description: What would you like maintainers to help decide? A short question is enough.
|
||||
description: State the exact question maintainers need to resolve.
|
||||
placeholder: Should REA ... or ...?
|
||||
validations:
|
||||
required: true
|
||||
@@ -41,57 +34,23 @@ body:
|
||||
id: context-evidence
|
||||
attributes:
|
||||
label: Context and evidence
|
||||
description: Explain the problem or motivation. If you have evidence, distinguish observed behavior from ideas or guesses.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: target-interaction
|
||||
attributes:
|
||||
label: Expected interaction with targets
|
||||
description: >-
|
||||
Select all known effects, including upstream tool behavior. Add "Not sure
|
||||
yet" if anything is unclear. Use "No target interaction" only on its own.
|
||||
These selections help discussion and do not establish scope approval.
|
||||
multiple: true
|
||||
options:
|
||||
- Read existing files or captured data
|
||||
- Launch or attach to a process
|
||||
- Inject instrumentation or run scripts/code
|
||||
- Change process memory or registers
|
||||
- Write or change files/configuration
|
||||
- Contact network services
|
||||
- Recover protected code or work around protections
|
||||
- No target interaction (documentation or usability only)
|
||||
- Not sure yet
|
||||
description: Explain why the decision is needed now. Separate observed behavior, derived evidence, and inference.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: boundary-invariants
|
||||
attributes:
|
||||
label: Boundary details (optional)
|
||||
description: >-
|
||||
If known, clarify target selection, selected effects, provider boundaries,
|
||||
non-goals, or evidence invariants. Leave blank if you need help identifying
|
||||
them. Do not include private samples or proof-of-authorization documents.
|
||||
label: Boundary and invariants
|
||||
description: Identify the owning layer, provider/bridge boundary, artifact or evidence invariants, and any process or authorization constraints.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: options
|
||||
attributes:
|
||||
label: Options and tradeoffs
|
||||
description: Optional approaches or tradeoffs. One idea is enough; maintainers can help compare alternatives.
|
||||
|
||||
- type: dropdown
|
||||
id: verification-readiness
|
||||
attributes:
|
||||
label: Verification available so far
|
||||
description: Pick the closest match. An idea without samples or test results is welcome.
|
||||
options:
|
||||
- Idea only; I need help defining checks
|
||||
- I can suggest a sample or manual check
|
||||
- I have a prototype or observed results
|
||||
- Not sure yet
|
||||
description: Compare a small number of viable approaches, including platform, compatibility, safety, evidence, and maintenance costs.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -99,10 +58,9 @@ body:
|
||||
id: success-criteria
|
||||
attributes:
|
||||
label: Success criteria and open questions
|
||||
description: >-
|
||||
Optional expected output, sample, possible check, or unresolved question.
|
||||
A complete test plan is not required to start; maintainers and implementers
|
||||
will resolve material verification gaps before integration.
|
||||
description: State what outcome would settle the decision and list only unresolved questions that could change the choice.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: references
|
||||
|
||||
@@ -8,13 +8,9 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Describe one problem and the outcome you would like. Short answers are
|
||||
welcome. Choose "Not sure yet" wherever needed; you do not need code, a
|
||||
finished design, or a complete test plan to propose an idea.
|
||||
|
||||
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Maintainers can help resolve scope and technical questions. Unresolved
|
||||
boundaries block integration, not opening a request for discussion.
|
||||
Keep one primary capability or workflow outcome per request. For a
|
||||
substantial provider, public contract, or architecture change, explain
|
||||
the decision maintainers need to make before implementation.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
@@ -23,9 +19,7 @@ body:
|
||||
options:
|
||||
- label: I searched the open and closed issues and pull requests for a related proposal.
|
||||
required: true
|
||||
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
|
||||
required: true
|
||||
- label: This request does not include credentials, private samples, or sensitive target data.
|
||||
- label: I removed credentials, tokens, private paths, and sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
@@ -42,7 +36,6 @@ body:
|
||||
- MCP or agent integration
|
||||
- Documentation
|
||||
- Other
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -50,7 +43,7 @@ body:
|
||||
id: current-limitation
|
||||
attributes:
|
||||
label: Current limitation
|
||||
description: What are you trying to understand or do, and what is difficult today? A short example is enough.
|
||||
description: What user or agent workflow is blocked, ambiguous, or unnecessarily difficult today?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -58,69 +51,23 @@ body:
|
||||
id: use-cases
|
||||
attributes:
|
||||
label: Use cases
|
||||
description: Optional additional examples of applications, targets, or callers. Leave blank if covered above.
|
||||
description: Give concrete applications, targets, callers, or investigation workflows. Explain why current tools are insufficient.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: desired-outcome
|
||||
attributes:
|
||||
label: Desired outcome and success criteria
|
||||
description: What would a useful result look like? Describe the output or behavior you want; implementation details are optional.
|
||||
placeholder: For a selected APK, show whether it is packed and include the detection evidence.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: target-interaction
|
||||
attributes:
|
||||
label: Expected interaction with targets
|
||||
description: >-
|
||||
Select all known effects, including upstream tool behavior. Add "Not sure
|
||||
yet" if anything is unclear. Use "No target interaction" only on its own.
|
||||
These selections help discussion and do not establish scope approval.
|
||||
multiple: true
|
||||
options:
|
||||
- Read existing files or captured data
|
||||
- Launch or attach to a process
|
||||
- Inject instrumentation or run scripts/code
|
||||
- Change process memory or registers
|
||||
- Write or change files/configuration
|
||||
- Contact network services
|
||||
- Recover protected code or work around protections
|
||||
- No target interaction (documentation or usability only)
|
||||
- Not sure yet
|
||||
description: Describe observable behavior, inputs, outputs, evidence, and compatibility criteria that would make this useful.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: boundaries
|
||||
attributes:
|
||||
label: Target and scope details (optional)
|
||||
description: >-
|
||||
If known, name the target/platform, clarify selected effects, or list a
|
||||
non-goal or scope question. You can leave this blank; maintainers can help.
|
||||
Do not include private samples or proof-of-authorization documents.
|
||||
|
||||
- type: dropdown
|
||||
id: verification-readiness
|
||||
attributes:
|
||||
label: Verification available so far
|
||||
description: Pick the closest match. An idea without samples or test results is welcome.
|
||||
options:
|
||||
- Idea only; I need help defining checks
|
||||
- I can suggest a sample or manual check
|
||||
- I have a prototype or observed results
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: verification
|
||||
attributes:
|
||||
label: Verification details (optional)
|
||||
description: >-
|
||||
Share an expected output, a possible check, or existing results if you have
|
||||
them. Self-owned or appropriately licensed samples can help, but are not
|
||||
required to open a request. Do not attach private or sensitive data.
|
||||
label: Authority boundaries and non-goals
|
||||
description: Call out process execution, target mutation, provider licensing, evidence provenance, supported platforms, and work that should remain out of scope.
|
||||
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
self-hosted-runner:
|
||||
labels:
|
||||
- hopper
|
||||
- ghidra-12-1-2
|
||||
# Hosted label introduced after the pinned actionlint release.
|
||||
- macos-15-intel
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Portable build cache
|
||||
description: Restore portable Turbo build outputs from the current run or cross-run cache.
|
||||
inputs:
|
||||
save:
|
||||
description: Save newly built outputs after the job succeeds.
|
||||
default: "false"
|
||||
build_cache_artifact:
|
||||
description: Reuse portable Turbo outputs uploaded by the current workflow run.
|
||||
default: ""
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Download portable build outputs from this workflow run
|
||||
if: inputs.build_cache_artifact != ''
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: ${{ inputs.build_cache_artifact }}
|
||||
path: .turbo/cache/
|
||||
- name: Resolve portable build task hashes
|
||||
id: plan
|
||||
if: inputs.build_cache_artifact == ''
|
||||
shell: bash
|
||||
env:
|
||||
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'package-lock.json') }}
|
||||
run: |
|
||||
npm run --silent test:prepare -- --dry=json |
|
||||
node --input-type=module -e '
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFileSync } from "node:fs";
|
||||
const { tasks } = JSON.parse(readFileSync(0, "utf8"));
|
||||
const hashes = tasks.map(({ taskId, hash }) => {
|
||||
if (typeof hash !== "string")
|
||||
throw new Error(`Build task ${taskId} must have an eager Turbo hash`);
|
||||
return `${taskId}:${hash}`;
|
||||
}).sort();
|
||||
const hash = createHash("sha256").update(hashes.join("\n")).digest("hex");
|
||||
console.log(`key=${process.env.BUILD_CACHE_PREFIX}-${hash}`);
|
||||
console.log(`prefix=${process.env.BUILD_CACHE_PREFIX}-`);
|
||||
' >> "$GITHUB_OUTPUT"
|
||||
- name: Restore portable build outputs
|
||||
if: inputs.build_cache_artifact == '' && inputs.save != 'true'
|
||||
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: ${{ steps.plan.outputs.key }}
|
||||
restore-keys: ${{ steps.plan.outputs.prefix }}
|
||||
- name: Restore and save portable build outputs
|
||||
if: inputs.build_cache_artifact == '' && inputs.save == 'true'
|
||||
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: ${{ steps.plan.outputs.key }}
|
||||
restore-keys: ${{ steps.plan.outputs.prefix }}
|
||||
@@ -19,38 +19,16 @@ describe the use case and observable outcome. -->
|
||||
<!-- Explain the chosen approach, why it fits REA's architecture, and what is
|
||||
intentionally not included. -->
|
||||
|
||||
Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Unresolved scope, authority, side effects, or required verification block
|
||||
integration. Link the required maintainer scope decision for new providers or
|
||||
capabilities involving target execution, injection, mutation, or protection
|
||||
recovery; ordinary fixes within existing boundaries may state "not applicable".
|
||||
Draft PRs may leave the scope decision pending and the final review checklist
|
||||
unchecked. Maintainers can help resolve open questions.
|
||||
|
||||
## Contract and boundary impact
|
||||
|
||||
<!-- Short answers are welcome. Complete applicable lines; mark unresolved facts as "unknown".
|
||||
Use "none" or "not applicable" only when established. -->
|
||||
<!-- Complete the applicable lines; use "none" or "not applicable" explicitly. -->
|
||||
|
||||
- Semantic owner and earliest changed stage:
|
||||
- Contribution scope fit and linked maintainer decision:
|
||||
- CLI and MCP/tool-catalog contract:
|
||||
- Provider, bridge, target-format, or platform compatibility:
|
||||
- Evidence, artifact, provenance, or reconstruction contract:
|
||||
- Target selection, authority, upstream behavior, and cleanup details (if applicable):
|
||||
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs):
|
||||
|
||||
Expected effects (check known effects; use documentation-only on its own).
|
||||
Unchecked items do not establish absence; leave uncertainties visible.
|
||||
|
||||
- [ ] Read existing files or captured data.
|
||||
- [ ] Launch or attach to a process.
|
||||
- [ ] Inject instrumentation or run scripts/code.
|
||||
- [ ] Change process memory, registers, files, or configuration.
|
||||
- [ ] Contact network services.
|
||||
- [ ] Recover protected code or work around protections.
|
||||
- [ ] Documentation or usability only; no target interaction.
|
||||
- [ ] Not sure yet; investigation or maintainer guidance needed.
|
||||
- Process execution, authorization, cleanup, or containment impact:
|
||||
- Generated metadata (`docs/product-catalog.json`), package, or installation impact (the regen-generated bot pushes tracked generated files; never hand-edit them):
|
||||
|
||||
## Evidence and regression coverage
|
||||
|
||||
@@ -81,22 +59,12 @@ relevant real-provider or platform workflow, or explain why it was unavailable.
|
||||
limits, security/privacy/containment implications, and meaningful proof gaps. -->
|
||||
|
||||
- Breaking changes or migration steps:
|
||||
<!-- For ! / BREAKING CHANGE: show a previously valid call/configuration,
|
||||
its new behavior and migration, and why compatibility/deprecation cannot
|
||||
preserve it. Every release increments minor, including breaking changes;
|
||||
keep breaking markers and migration notes so callers can assess compatibility.
|
||||
See docs/releasing.md. -->
|
||||
- Real Hopper/Ghidra, browser, or OS coverage:
|
||||
- Package or release metadata impact:
|
||||
- Security, privacy, process, or containment review:
|
||||
|
||||
## Before merge
|
||||
## Review checklist
|
||||
|
||||
<!-- Draft PRs may leave pending items unchecked. Resolve material scope and
|
||||
authority questions before integration. -->
|
||||
|
||||
- [ ] This PR addresses a concrete problem or an agreed enhancement.
|
||||
- [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
|
||||
- [ ] The PR has one focused outcome and the title follows `type(scope): outcome`.
|
||||
- [ ] Related issue is linked, or the reason for not linking one is stated above.
|
||||
- [ ] Tests cover changed observable behavior and meaningful failure paths.
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
name: Full PR qualification
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
types: [labeled]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
|
||||
jobs:
|
||||
qualification:
|
||||
name: Qualify labeled PR revision
|
||||
if: github.event.label.name == 'ci:full'
|
||||
uses: ./.github/workflows/ci.yml
|
||||
with:
|
||||
source_sha: ${{ github.sha }}
|
||||
@@ -1,46 +0,0 @@
|
||||
name: CI timings
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "17 * * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ci-timings-${{ github.repository }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
|
||||
jobs:
|
||||
report:
|
||||
name: Collect recent completed workflow attempts
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
# Consume only API data and execute batch code from the trusted default branch.
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Record queue, execution, and matched step timing
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
run: node scripts/ci/job-timings-batch.mjs
|
||||
- name: Upload CI timing artifacts
|
||||
if: always() && hashFiles('.cache/ci-job-timings/*') != ''
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: ci-job-timings-batch
|
||||
path: .cache/ci-job-timings/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 14
|
||||
+202
-555
@@ -5,179 +5,107 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
schedule:
|
||||
- cron: "23 0-2,4-23 * * *"
|
||||
- cron: "23 3 * * *"
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
source_sha:
|
||||
description: Exact qualification source; must match the caller revision.
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
# Complete qualification finishes even while main receives more pushes.
|
||||
# Main feedback also finishes, coalescing newer pushes to one pending run.
|
||||
# Scheduled/manual main runs coalesce to one active and the latest pending;
|
||||
# release and label-triggered qualification use exact-revision groups.
|
||||
group: ci-${{ (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full')) && 'qualification' || 'feedback' }}-${{ inputs.source_sha || github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' && inputs.source_sha == '' && !contains(github.event.pull_request.labels.*.name, 'ci:full') }}
|
||||
group: ci-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
pr-title:
|
||||
name: Conventional PR title
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Validate title for Release Please
|
||||
env:
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
run: >-
|
||||
node -e 'const title = process.env.PR_TITLE ?? "";
|
||||
const conventional = /^(?:feat|fix|docs|refactor|perf|test|build|ci|chore|revert)(?:\([a-z0-9._/-]+\))?!?: .+/u;
|
||||
if (!conventional.test(title)) {
|
||||
console.error("PR title must use Conventional Commits, for example: feat: add historical source import");
|
||||
process.exit(1);
|
||||
}'
|
||||
|
||||
changes:
|
||||
name: Classify changes
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
outputs:
|
||||
tier: ${{ steps.plan.outputs.tier }}
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
apple: ${{ steps.plan.outputs.apple }}
|
||||
fixtures: ${{ steps.plan.outputs.fixtures }}
|
||||
readiness: ${{ steps.plan.outputs.readiness }}
|
||||
docs: ${{ steps.plan.outputs.docs }}
|
||||
workflows: ${{ steps.plan.outputs.workflows }}
|
||||
browser: ${{ steps.plan.outputs.browser }}
|
||||
binary_layout: ${{ steps.plan.outputs.binary_layout }}
|
||||
evm: ${{ steps.plan.outputs.evm }}
|
||||
javascript_recovery: ${{ steps.plan.outputs.javascript_recovery }}
|
||||
recorded_crash: ${{ steps.plan.outputs.recorded_crash }}
|
||||
web_network: ${{ steps.plan.outputs.web_network }}
|
||||
web_runtime: ${{ steps.plan.outputs.web_runtime }}
|
||||
web_source_map: ${{ steps.plan.outputs.web_source_map }}
|
||||
nativeaot: ${{ steps.plan.outputs.nativeaot }}
|
||||
website: ${{ steps.plan.outputs.website }}
|
||||
matrix: ${{ steps.plan.outputs.matrix }}
|
||||
test_selection: ${{ steps.plan.outputs.test_selection }}
|
||||
code: ${{ steps.package.outputs.required }}
|
||||
package: ${{ steps.package.outputs.required }}
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.source_sha || github.sha }}
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Bind qualification to the caller revision
|
||||
if: inputs.source_sha != ''
|
||||
- name: Select package and Windows lanes
|
||||
id: package
|
||||
shell: bash
|
||||
env:
|
||||
SOURCE_SHA: ${{ inputs.source_sha }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
test "${SOURCE_SHA}" = "${GITHUB_SHA}"
|
||||
test "${SOURCE_SHA}" = "$(git rev-parse HEAD)"
|
||||
- name: Find a verified ancestor for push selection
|
||||
id: baseline
|
||||
if: github.event_name == 'push' && inputs.source_sha == ''
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
HEAD_SHA: ${{ github.sha }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/baseline.mjs
|
||||
- name: Admit scheduled full qualification
|
||||
id: qualification
|
||||
if: github.event_name == 'schedule' && inputs.source_sha == ''
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
HEAD_SHA: ${{ github.sha }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_REPOSITORY_ID: ${{ github.repository_id }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
CI_FORCE_FULL: ${{ github.event.schedule == '23 3 * * *' }}
|
||||
run: node scripts/ci/qualification.mjs
|
||||
- name: Select verification lanes
|
||||
id: plan
|
||||
env:
|
||||
BASE_SHA: ${{ steps.baseline.outputs.base || github.event.pull_request.base.sha }}
|
||||
BASELINE_REASON: ${{ steps.baseline.outputs.reason }}
|
||||
HEAD_SHA: ${{ inputs.source_sha || github.event.pull_request.head.sha || github.sha }}
|
||||
CI_FULL: ${{ inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full') }}
|
||||
CI_SKIP_FULL: ${{ steps.qualification.outputs.run_full == 'false' }}
|
||||
run: node scripts/ci/plan.mjs
|
||||
if [[ "${{ github.event_name }}" != "pull_request" ]]; then
|
||||
echo "required=true" >> "${GITHUB_OUTPUT}"
|
||||
elif git diff --quiet "${BASE_SHA}" "${HEAD_SHA}" -- \
|
||||
. \
|
||||
':(exclude)docs/**' \
|
||||
':(exclude)AGENTS.md' \
|
||||
':(exclude)CONTRIBUTING.md'; then
|
||||
echo "required=false" >> "${GITHUB_OUTPUT}"
|
||||
else
|
||||
echo "required=true" >> "${GITHUB_OUTPUT}"
|
||||
fi
|
||||
|
||||
static:
|
||||
name: Static checks
|
||||
needs: changes
|
||||
if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }}
|
||||
if: ${{ always() }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Require successful change classification
|
||||
env:
|
||||
CHANGES_RESULT: ${{ needs.changes.result }}
|
||||
run: test "${CHANGES_RESULT}" = "success"
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- name: Validate workflow syntax and expressions
|
||||
if: needs.changes.outputs.workflows == 'true'
|
||||
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
- if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
run: npm run check:ci
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
- if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
run: npm run verify:test-discovery
|
||||
- name: Check LLDB producer retention and launch contracts
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
run: npm run test:python
|
||||
- if: needs.changes.outputs.code != 'true'
|
||||
- if: github.event_name == 'pull_request' && needs.changes.outputs.code != 'true'
|
||||
run: npm run format:check
|
||||
- name: Upload complete static diagnostics
|
||||
if: always() && needs.changes.outputs.code == 'true'
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: static-check-reports
|
||||
path: .cache/static-checks/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
test-shard:
|
||||
name: Test shard (${{ matrix.shard }})
|
||||
needs: [changes, build]
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard: [1/6, 2/6, 3/6, 4/6, 5/6, 6/6]
|
||||
shard: [1/4, 2/4, 3/4, 4/4]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: .
|
||||
- name: Run complete portable suite with coverage
|
||||
if: needs.changes.outputs.tier == 'full'
|
||||
run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
|
||||
env:
|
||||
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
|
||||
- name: Run complete portable suite for fast feedback
|
||||
if: needs.changes.outputs.tier == 'fast'
|
||||
run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
|
||||
env:
|
||||
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
name: rea-dist
|
||||
path: dist
|
||||
- name: Generate source catalog from downloaded build
|
||||
run: node scripts/generate-mcp-tool-catalog.mjs
|
||||
- run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: vitest-report-${{ strategy.job-index }}
|
||||
@@ -190,114 +118,54 @@ jobs:
|
||||
build:
|
||||
name: Build package
|
||||
needs: changes
|
||||
# Reusable provider workflows consume this run's portable build artifact,
|
||||
# even when a workflow-only edit selects the lane without portable tests.
|
||||
if: >-
|
||||
needs.changes.outputs.code == 'true' ||
|
||||
needs.changes.outputs.browser == 'true' ||
|
||||
needs.changes.outputs.binary_layout == 'true' ||
|
||||
needs.changes.outputs.evm == 'true' ||
|
||||
needs.changes.outputs.javascript_recovery == 'true' ||
|
||||
needs.changes.outputs.recorded_crash == 'true' ||
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true'
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
- run: npm run build:cached
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
save: ${{ github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
|
||||
- run: npm run test:prepare
|
||||
- name: Find available trusted full-suite timing history
|
||||
id: history
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_REPOSITORY_ID: ${{ github.repository_id }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/test-history.mjs
|
||||
- name: Download previous timing report
|
||||
if: steps.history.outputs.run-id != ''
|
||||
continue-on-error: true
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: test-timings
|
||||
run-id: ${{ steps.history.outputs.run-id }}
|
||||
github-token: ${{ github.token }}
|
||||
path: .cache/ci-test-history
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: |
|
||||
dist/
|
||||
skills/
|
||||
.cache/mcp-tool-catalog.json
|
||||
.cache/ci-test-history/test-results.json
|
||||
docs/public/product-catalog.json
|
||||
docs/verification/managed-conformance-*.json
|
||||
include-hidden-files: true
|
||||
name: rea-dist
|
||||
path: dist
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
# test:prepare caches only portable JS, skill, and catalog outputs.
|
||||
# Native package jobs still run their build and prepack checks; Turbo
|
||||
# restores matching outputs or rebuilds when their inputs differ.
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: rea-build-cache
|
||||
path: .turbo/cache/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
sqlite-runtime:
|
||||
# Preserve all selected coverage, but admit expensive host/provider work
|
||||
# only after static validation. Portable tests still provide early feedback.
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.sqlite == 'true'
|
||||
uses: ./.github/workflows/sqlite-runtime.yml
|
||||
|
||||
test:
|
||||
name: Portable tests
|
||||
name: Test (coverage + timing)
|
||||
needs: [changes, test-shard]
|
||||
if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }}
|
||||
if: ${{ always() }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Require successful change classification
|
||||
- name: Require successful test shards
|
||||
env:
|
||||
CHANGES_RESULT: ${{ needs.changes.result }}
|
||||
run: test "${CHANGES_RESULT}" = "success"
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
|
||||
SHARD_RESULT: ${{ needs.test-shard.result }}
|
||||
run: |
|
||||
test "${CHANGES_RESULT}" = "success"
|
||||
test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
|
||||
- uses: actions/checkout@v4
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/setup-node@v4
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- if: needs.changes.outputs.code == 'true' && success()
|
||||
run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
- uses: actions/download-artifact@v4
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
with:
|
||||
path: .vitest/blob
|
||||
pattern: vitest-report-*
|
||||
merge-multiple: true
|
||||
- name: Merge coverage, JUnit, and timing reports
|
||||
id: merge
|
||||
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success()
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
run: >-
|
||||
npm run test:ci:merge --
|
||||
--coverage.reportsDirectory=coverage
|
||||
@@ -307,115 +175,66 @@ jobs:
|
||||
--reporter=default
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
--outputFile.junit=test-results.xml
|
||||
--outputFile.json=test-results.json
|
||||
- name: Merge fast-suite JUnit and timing reports
|
||||
id: merge-fast
|
||||
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast' && success()
|
||||
run: >-
|
||||
npm exec -- vitest --merge-reports
|
||||
--reporter=default
|
||||
--reporter=junit
|
||||
--reporter=json
|
||||
--reporter=./scripts/ci/test-timing-reporter.mjs
|
||||
--outputFile.junit=test-results.xml
|
||||
--outputFile.json=test-results.json
|
||||
- name: Summarize slowest test files
|
||||
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-timings.json') != ''
|
||||
env:
|
||||
SHARD_RESULT: ${{ needs.test-shard.result }}
|
||||
MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }}
|
||||
CI_VERIFY_TEST_INVENTORY: "true"
|
||||
CI_TEST_SELECTION: ${{ needs.changes.outputs.test_selection }}
|
||||
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: node scripts/ci/test-report.mjs
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != ''
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
shell: bash
|
||||
run: |
|
||||
node <<'NODE' >> "${GITHUB_STEP_SUMMARY}"
|
||||
const report = JSON.parse(require("node:fs").readFileSync("test-results.json", "utf8"));
|
||||
const files = (report.testResults ?? [])
|
||||
.map((result) => ({
|
||||
name: result.name,
|
||||
duration: Math.max(0, (result.endTime ?? result.startTime) - result.startTime),
|
||||
}))
|
||||
.sort((left, right) => right.duration - left.duration)
|
||||
.slice(0, 20);
|
||||
console.log("## Slowest deterministic test files\n");
|
||||
console.log("| File | Duration (ms) |");
|
||||
console.log("| --- | ---: |");
|
||||
for (const file of files) {
|
||||
console.log(`| \`${file.name}\` | ${file.duration} |`);
|
||||
}
|
||||
NODE
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: needs.changes.outputs.code == 'true' && always()
|
||||
with:
|
||||
name: test-results
|
||||
path: |
|
||||
test-results.xml
|
||||
test-results.json
|
||||
test-timings.json
|
||||
coverage/
|
||||
.cache/ci-test-selection/report.json
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
retention-days: 30
|
||||
- name: Publish successful timing history
|
||||
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() && needs.test-shard.result == 'success'
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: test-timings
|
||||
path: .cache/ci-test-history/test-results.json
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
retention-days: 30
|
||||
- name: Require successful test shards
|
||||
if: ${{ !cancelled() }}
|
||||
env:
|
||||
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
|
||||
SHARD_RESULT: ${{ needs.test-shard.result }}
|
||||
run: test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
|
||||
|
||||
conformance-fixtures:
|
||||
name: Source-built conformance fixtures
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.fixtures == 'true'
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- name: Verify fixtures from a clean checkout
|
||||
run: npm run verify:fixtures
|
||||
|
||||
windows-native-build:
|
||||
name: Build bundled Windows x64 native controls
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.runtime == 'true'
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Prepare cross compiler
|
||||
run: |
|
||||
# Slow Azure archive transfers can exhaust the job before compilation.
|
||||
# Keep the same signed packages and avoid slow HTTP mirror fallback.
|
||||
for source in /etc/apt/apt-mirrors.txt /etc/apt/sources.list /etc/apt/sources.list.d/ubuntu.sources; do
|
||||
if [ -f "$source" ]; then
|
||||
sudo sed -i 's|azure.archive.ubuntu.com|archive.ubuntu.com|g; s|http://archive.ubuntu.com|https://archive.ubuntu.com|g' "$source"
|
||||
fi
|
||||
done
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
run: sudo apt-get update && sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
- run: npm run build:windows-native
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- run: node scripts/build-windows-native.mjs --inspection-fixture
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
if-no-files-found: error
|
||||
# The process boundary observations are independent of the addon: they
|
||||
# report the caller's and child's own token, environment and breakaway
|
||||
# results from a separate executable. Cross-build it in the lane that
|
||||
# already has MinGW so the Windows job can observe them too.
|
||||
- name: Build Windows process boundary fixture
|
||||
run: |
|
||||
mkdir -p .cache/windows-process-boundary
|
||||
x86_64-w64-mingw32-g++ -municode -std=c++17 -O2 -static -Wall -Wextra \
|
||||
-o .cache/windows-process-boundary/process-boundary.exe \
|
||||
tests/fixtures/windows/processBoundary.cc -ladvapi32
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
if-no-files-found: error
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: windows-native-controls
|
||||
path: native/windows/build/
|
||||
@@ -423,23 +242,33 @@ jobs:
|
||||
|
||||
package-e2e:
|
||||
name: Native package baseline (${{ matrix.platform }} ${{ matrix.architecture }})
|
||||
needs: [changes, static, build, windows-native-build]
|
||||
if: needs.changes.outputs.runtime == 'true'
|
||||
needs: [changes, windows-native-build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
# The four platform jobs use independent hosted runners; keep each
|
||||
# runner's existing heap and threadpool limits while avoiding a queued
|
||||
# second platform wave.
|
||||
max-parallel: 4
|
||||
matrix: ${{ fromJSON(needs.changes.outputs.matrix) }}
|
||||
max-parallel: 2
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
platform: linux
|
||||
architecture: x64
|
||||
- os: ubuntu-24.04-arm
|
||||
platform: linux
|
||||
architecture: arm64
|
||||
- os: macos-15
|
||||
platform: darwin
|
||||
architecture: arm64
|
||||
- os: macos-15-intel
|
||||
platform: darwin
|
||||
architecture: x64
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 20
|
||||
env:
|
||||
NODE_OPTIONS: "--max-old-space-size=1024"
|
||||
UV_THREADPOOL_SIZE: "2"
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
@@ -453,97 +282,52 @@ jobs:
|
||||
assert.equal(process.arch, process.env.EXPECTED_CI_ARCHITECTURE);
|
||||
console.log(JSON.stringify({ platform: process.platform, architecture: process.arch, node: process.version }));"
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-build-cache
|
||||
path: .turbo/cache/
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: windows-native-controls
|
||||
path: native/windows/build/
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- name: Verify NativeAOT diagnostic applicability on this host
|
||||
run: npm run test:focused -- tests/evaluation/package/nativeAotDiagnostics.test.ts
|
||||
- name: Resolve npm cache before verifier HOME isolation
|
||||
id: npm-cache
|
||||
run: |
|
||||
cache_path="$(npm config get cache)"
|
||||
echo "path=${cache_path}" >> "${GITHUB_OUTPUT}"
|
||||
- run: npm run verify:package
|
||||
env:
|
||||
npm_config_cache: ${{ steps.npm-cache.outputs.path }}
|
||||
REA_VERIFIED_PACKAGE_DIRECTORY: ${{ needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64' && '.cache/verified-package' || '' }}
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
timeout-minutes: 10
|
||||
if: matrix.platform == 'darwin'
|
||||
run: npm run verify:inspector
|
||||
- name: Retain the exact qualified release package
|
||||
if: needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64'
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: verified-release-package
|
||||
path: .cache/verified-package/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 7
|
||||
|
||||
apple-artifacts:
|
||||
name: Real Apple artifact CLI/MCP E2E
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.apple == 'true'
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: macos-15
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-build-cache
|
||||
path: .turbo/cache/
|
||||
- name: Verify Apple artifact formats and metadata
|
||||
env:
|
||||
REA_ARTIFACT_NODE_COMPILE_CACHE: ${{ runner.temp }}/rea-artifact-node-cache
|
||||
run: |
|
||||
npm run verify:keyed-archive
|
||||
npm run verify:asset-catalog
|
||||
npm run verify:macos-bundle
|
||||
npm run verify:legacy-simulator
|
||||
npm run verify:apple-dispatch
|
||||
npm run verify:interface-builder
|
||||
- name: Run native Apple and process boundary regressions
|
||||
run: >-
|
||||
npm run test:focused --
|
||||
tests/boundary/filesystem/installScript.test.ts
|
||||
tests/boundary/filesystem/nibNativeHierarchy.test.ts
|
||||
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
|
||||
tests/boundary/cli/nativePlistCli.test.ts
|
||||
tests/boundary/cli/nativePlistPrototypeKeyCli.test.ts
|
||||
tests/boundary/cli/nativePlistSelectionFailureCli.test.ts
|
||||
tests/boundary/cli/nativeDemangleCli.test.ts
|
||||
tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
|
||||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/boundary/process/darwinProcessRunTokenReader.test.ts
|
||||
tests/boundary/process/directAnalysisSignals.test.ts
|
||||
tests/boundary/process/processCaptureLifecycle.test.ts
|
||||
tests/boundary/process/processCli.test.ts
|
||||
tests/boundary/mcp/sessionMcp.test.ts
|
||||
- run: npm run verify:keyed-archive
|
||||
- run: npm run verify:asset-catalog
|
||||
- run: npm run verify:macos-bundle
|
||||
- name: Native compiled NIB hierarchy regressions
|
||||
run: npm run test:focused -- tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
|
||||
- name: Real binary metadata and Interface Builder integration
|
||||
run: npm run verify:apple-dispatch && npm run verify:interface-builder
|
||||
- name: Native plist CLI selection regressions
|
||||
run: npm run test:focused -- tests/boundary/cli/nativePlistCli.test.ts tests/boundary/cli/nativePlistPrototypeKeyCli.test.ts tests/boundary/cli/nativePlistSelectionFailureCli.test.ts
|
||||
- name: Native Swift demangling CLI and MCP regressions
|
||||
run: npm run test:focused -- tests/boundary/cli/nativeDemangleCli.test.ts tests/boundary/mcp/nativeDemangleMcp.test.ts
|
||||
- name: Process filesystem snapshot descriptor regressions
|
||||
run: npm run test:focused -- src/process/capture/FilesystemSnapshot.test.ts
|
||||
|
||||
reconstruction-readiness:
|
||||
name: Reconstruction readiness
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.readiness == 'true'
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
@@ -554,7 +338,7 @@ jobs:
|
||||
run: |
|
||||
set -o pipefail
|
||||
npm run verify:readiness | tee readiness-summary.json
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: reconstruction-readiness
|
||||
@@ -564,25 +348,64 @@ jobs:
|
||||
if-no-files-found: error
|
||||
retention-days: 30
|
||||
|
||||
javascript-runtime-observation:
|
||||
name: JavaScript runtime observation (${{ matrix.os }})
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 2
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
platform: linux
|
||||
architecture: x64
|
||||
- os: ubuntu-24.04-arm
|
||||
platform: linux
|
||||
architecture: arm64
|
||||
- os: windows-latest
|
||||
platform: win32
|
||||
architecture: x64
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
NODE_OPTIONS: "--max-old-space-size=1024"
|
||||
UV_THREADPOOL_SIZE: "2"
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Verify native runtime identity
|
||||
env:
|
||||
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
|
||||
EXPECTED_CI_ARCHITECTURE: ${{ matrix.architecture }}
|
||||
run: >-
|
||||
node -e "const assert = require('node:assert/strict');
|
||||
assert.equal(process.platform, process.env.EXPECTED_CI_PLATFORM);
|
||||
assert.equal(process.arch, process.env.EXPECTED_CI_ARCHITECTURE);
|
||||
console.log(JSON.stringify({ platform: process.platform, architecture: process.arch, node: process.version }));"
|
||||
- run: npm ci
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
run: npm run verify:inspector
|
||||
|
||||
windows-curated:
|
||||
name: Windows capability and package lane
|
||||
needs: [changes, static, build, windows-native-build]
|
||||
if: needs.changes.outputs.package == 'true'
|
||||
needs: [changes, windows-native-build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 25
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-build-cache
|
||||
path: .turbo/cache/
|
||||
- run: npm run build:cached
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
- run: npm run typecheck
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: windows-native-controls
|
||||
path: native/windows/build/
|
||||
@@ -608,26 +431,17 @@ jobs:
|
||||
tests/boundary/filesystem/binaryTarget.test.ts
|
||||
tests/boundary/filesystem/safeOutputTree.test.ts
|
||||
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
|
||||
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
|
||||
tests/boundary/filesystem/setupUnavailableProvider.test.ts
|
||||
tests/boundary/filesystem/windowsNpxRegistration.test.ts
|
||||
tests/boundary/android/workspaceGuardianRetry.test.ts
|
||||
tests/acceptance/setup/homeDirectoryFallback.test.ts
|
||||
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
|
||||
tests/boundary/filesystem/webScriptManifestPaths.test.ts
|
||||
tests/boundary/process/ProcessEvidence.host.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
tests/acceptance/webSourceLocationCli.test.ts
|
||||
tests/boundary/cli/referenceSourceImportStatus.test.ts
|
||||
src/cliReferenceSourceImportStatus.test.ts
|
||||
tests/boundary/filesystem/asarInventoryFreshness.test.ts
|
||||
tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts
|
||||
tests/composition/javascript-applications/javascriptArtifactReconstruction.test.ts
|
||||
tests/conformance/providers/ghidra/windowsCapabilities.test.ts
|
||||
tests/conformance/providers/ghidra/ghidraWindowsFixture.test.ts
|
||||
src/ghidra/GhidraInstallation.test.ts
|
||||
tests/boundary/filesystem/ghidraJavaProbe.test.ts
|
||||
tests/boundary/providers/ghidra/ghidraTransport.test.ts
|
||||
tests/boundary/providers/ghidra/ghidraTargetSnapshot.test.ts
|
||||
tests/boundary/providers/ghidra/ghidraLauncher.test.ts
|
||||
tests/boundary/providers/hopper/hopperClient.cleanupRetry.test.ts
|
||||
src/windows/WindowsNativeLoader.test.ts
|
||||
src/windows/WindowsOwnedProcess.test.ts
|
||||
src/ghidra/GhidraLauncherEnvironment.test.ts
|
||||
@@ -638,46 +452,9 @@ jobs:
|
||||
src/application/Update.test.ts
|
||||
tests/process-global/updateMaintenance.test.ts
|
||||
tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-process-boundary-fixture
|
||||
path: .cache/windows-process-boundary/
|
||||
# Passing the fixture adds the breakaway, caller/child token, environment
|
||||
# semantics and ancestor reparse observations, which the lane skips
|
||||
# entirely when it is absent.
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: windows-native-inspection-fixture
|
||||
path: .cache/windows-native-inspection-fixture/
|
||||
- name: Verify native inspection failure ownership and recovery
|
||||
run: node scripts/verify-windows-native-inspection.mjs
|
||||
- name: Verify real Node Inspector discovery and observation
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
NODE_OPTIONS: "--max-old-space-size=1024"
|
||||
UV_THREADPOOL_SIZE: "2"
|
||||
run: npm run verify:inspector
|
||||
- run: npm run verify:windows-native
|
||||
- run: npm run verify:package:windows
|
||||
- name: Verify native directory identity on Node 24
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
tests/boundary/filesystem/directoryArtifactReaderOpen.test.ts
|
||||
tests/boundary/filesystem/javascriptPackagePrecedence.test.ts
|
||||
tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version: 22.19.0
|
||||
- name: Verify native path boundaries on Node 22
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
tests/boundary/filesystem/directoryArtifactReaderOpen.test.ts
|
||||
tests/boundary/filesystem/javascriptPackagePrecedence.test.ts
|
||||
tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts
|
||||
tests/boundary/process/sourceMapUncHosts.test.ts
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: windows-capabilities
|
||||
@@ -686,143 +463,13 @@ jobs:
|
||||
|
||||
docs:
|
||||
name: Docs Generation
|
||||
needs: changes
|
||||
if: needs.changes.outputs.docs == 'true'
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Generation leaves authored files unchanged
|
||||
run: git diff --exit-code
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: generated-docs
|
||||
path: |
|
||||
docs/public/product-catalog.json
|
||||
docs/verification/managed-conformance-*.json
|
||||
skills/reverse-engineer-anything
|
||||
if-no-files-found: error
|
||||
|
||||
browser:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.browser == 'true'
|
||||
uses: ./.github/workflows/real-browser.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
binary_layout:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.binary_layout == 'true'
|
||||
uses: ./.github/workflows/real-binary-layout.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
evm:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.evm == 'true'
|
||||
uses: ./.github/workflows/real-evm-interface.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
javascript_recovery:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.javascript_recovery == 'true'
|
||||
uses: ./.github/workflows/real-javascript-recovery.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
recorded_crash:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.recorded_crash == 'true'
|
||||
uses: ./.github/workflows/real-recorded-crash.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
web_network:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.web_network == 'true'
|
||||
uses: ./.github/workflows/real-web-network-captures.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
web_runtime:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.web_runtime == 'true'
|
||||
uses: ./.github/workflows/real-web-runtime.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
web_source_map:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.web_source_map == 'true'
|
||||
uses: ./.github/workflows/real-web-source-map.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
uses: ./.github/workflows/nativeaot-fixtures.yml
|
||||
|
||||
website:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.website == 'true'
|
||||
uses: ./.github/workflows/website-check.yml
|
||||
|
||||
required:
|
||||
name: CI required
|
||||
if: ${{ always() }}
|
||||
needs:
|
||||
[
|
||||
changes,
|
||||
static,
|
||||
build,
|
||||
test-shard,
|
||||
sqlite-runtime,
|
||||
test,
|
||||
conformance-fixtures,
|
||||
windows-native-build,
|
||||
package-e2e,
|
||||
apple-artifacts,
|
||||
reconstruction-readiness,
|
||||
windows-curated,
|
||||
docs,
|
||||
browser,
|
||||
binary_layout,
|
||||
evm,
|
||||
javascript_recovery,
|
||||
recorded_crash,
|
||||
web_network,
|
||||
web_runtime,
|
||||
web_source_map,
|
||||
nativeaot,
|
||||
website,
|
||||
]
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Require every selected lane to pass
|
||||
id: gate
|
||||
env:
|
||||
CI_JOBS: ${{ toJSON(needs) }}
|
||||
CI_QUALIFICATION_DIRECTORY: .cache/ci-qualification
|
||||
run: node scripts/ci/required.mjs
|
||||
- name: Upload full qualification receipt artifact
|
||||
if: needs.changes.outputs.tier == 'full' && success()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: ${{ steps.gate.outputs.receipt_name }}
|
||||
path: ${{ steps.gate.outputs.receipt_path }}
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 7
|
||||
|
||||
@@ -2,7 +2,12 @@ name: Source-built NativeAOT fixtures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
pull_request:
|
||||
paths:
|
||||
- "bridge/ghidra/extensions/nativeaot/**"
|
||||
- "scripts/generate-nativeaot-fixtures.mjs"
|
||||
- "tests/fixtures/nativeaot/**"
|
||||
- ".github/workflows/nativeaot-fixtures.yml"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -12,23 +17,22 @@ jobs:
|
||||
name: Build Windows x64 NativeAOT fixture
|
||||
# .NET 8.0.22's bundled linker discovery supports VS 2019/2022.
|
||||
runs-on: windows-2022
|
||||
# Two bounded restore attempts and one compile can each take five minutes.
|
||||
timeout-minutes: 20
|
||||
timeout-minutes: 15
|
||||
env:
|
||||
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
|
||||
DOTNET_PROCESSOR_COUNT: "1"
|
||||
COMPlus_GCHeapHardLimit: "30000000"
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1
|
||||
- uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: "8.0.416"
|
||||
- name: Build benign fixture and independent byte oracle
|
||||
run: node scripts/generate-nativeaot-fixtures.mjs
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: nativeaot-win-x64-${{ github.event.pull_request.head.sha || github.sha }}
|
||||
path: |
|
||||
|
||||
@@ -19,8 +19,8 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
name: PR title
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
types: [opened, synchronize, reopened, edited]
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: pr-title-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pr-title:
|
||||
name: Conventional PR title
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
steps:
|
||||
- name: Validate title for Release Please
|
||||
env:
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
run: >-
|
||||
node -e 'const title = process.env.PR_TITLE ?? "";
|
||||
const conventional = /^(?:feat|fix|docs|refactor|perf|test|build|ci|chore|revert)(?:\([a-z0-9._/-]+\))?!?: .+/u;
|
||||
if (!conventional.test(title)) {
|
||||
console.error("PR title must use Conventional Commits, for example: feat: add historical source import");
|
||||
process.exit(1);
|
||||
}'
|
||||
@@ -1,54 +0,0 @@
|
||||
name: Real offline binary layout
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
large_mcp:
|
||||
description: "Also verify the complete 65,281-section result over MCP (slow)"
|
||||
type: boolean
|
||||
default: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
REA_VERIFY_LARGE_ELF_MCP: ${{ inputs.large_mcp && '1' || '0' }}
|
||||
|
||||
jobs:
|
||||
real-binary-layout:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
layout_python_root="${RUNNER_TEMP}/rea-pwntools-layout"
|
||||
python3 -m venv "${layout_python_root}"
|
||||
"${layout_python_root}/bin/python" -m pip install --disable-pip-version-check \
|
||||
'pwntools==4.15.0' 'pyelftools==0.33' 'unicorn==2.1.2'
|
||||
command -v gcc
|
||||
command -v ld
|
||||
command -v strip
|
||||
command -v strace
|
||||
echo "REA_PWNTOOLS_PYTHON=${layout_python_root}/bin/python" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned fixtures through public CLI and MCP
|
||||
run: npm run verify:binary:layout
|
||||
@@ -1,15 +1,46 @@
|
||||
name: Real browser verification
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-browser.yml"
|
||||
- "scripts/verify-real-browser.mjs"
|
||||
- "scripts/verify-browser-network.mjs"
|
||||
- "scripts/verify-browser-scripts.mjs"
|
||||
- "scripts/verify-browser-modules.mjs"
|
||||
- "scripts/verify-browser-dom-destinations.mjs"
|
||||
- "scripts/lib/browser-*.mjs"
|
||||
- "scripts/fixtures/browser-network-site.mjs"
|
||||
- "scripts/fixtures/browser-script-site.mjs"
|
||||
- "scripts/fixtures/browser-module-site.mjs"
|
||||
- "scripts/fixtures/browser-verifier-site.mjs"
|
||||
- "scripts/fixtures/page-cdp-proxy.mjs"
|
||||
- "src/browser/**"
|
||||
- "src/domain/browser*.ts"
|
||||
- "src/domain/webScript*.ts"
|
||||
- "src/domain/webModule*.ts"
|
||||
- "src/application/WebModule*.ts"
|
||||
- "src/composition/webModules.ts"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "src/artifacts/ArtifactPaths.ts"
|
||||
- "src/application/WebScriptExportService.ts"
|
||||
- "src/contracts/webScriptToolContracts.ts"
|
||||
- "src/server/registerWebScriptTool.ts"
|
||||
- "src/server/registerWebModuleTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli/webModuleCommands.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli/webScriptCommands.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: real-browser-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
@@ -21,15 +52,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
node-version: 24
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
name: Real offline EVM interface
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
real-evm-interface:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire unchanged fixture compiler in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
evm_fixture_root="${RUNNER_TEMP}/rea-evm-fixture-compiler"
|
||||
mkdir -p "${evm_fixture_root}"
|
||||
npm install --prefix "${evm_fixture_root}" --ignore-scripts --no-audit --no-fund --save-exact solc@0.8.30
|
||||
command -v prlimit
|
||||
command -v strace
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_SOLC_MODULE=${evm_fixture_root}/node_modules/solc/index.js" >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned compiled bytecode through public CLI and MCP
|
||||
run: npm run verify:evm:interface
|
||||
- name: Verify content-hash-free and CBOR-free owned compiler artifacts
|
||||
run: npm run verify:solidity:metadata
|
||||
@@ -16,40 +16,19 @@ jobs:
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Prepare cross compiler
|
||||
run: |
|
||||
for source in /etc/apt/apt-mirrors.txt /etc/apt/sources.list /etc/apt/sources.list.d/ubuntu.sources; do
|
||||
if [ -f "$source" ]; then
|
||||
sudo sed -i 's|azure.archive.ubuntu.com|archive.ubuntu.com|g; s|http://archive.ubuntu.com|https://archive.ubuntu.com|g' "$source"
|
||||
fi
|
||||
done
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
run: sudo apt-get update && sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
- run: npm run build:windows-native
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: real-ghidra-windows-native
|
||||
path: native/windows/build/
|
||||
if-no-files-found: error
|
||||
# The boundary observations need a separate Windows executable, which the
|
||||
# native lane skips when it is absent. Cross-build it here, where MinGW is
|
||||
# already installed, so the proof run observes them too.
|
||||
- name: Build Windows process boundary fixture
|
||||
run: |
|
||||
mkdir -p .cache/windows-process-boundary
|
||||
x86_64-w64-mingw32-g++ -municode -std=c++17 -O2 -static -Wall -Wextra \
|
||||
-o .cache/windows-process-boundary/process-boundary.exe \
|
||||
tests/fixtures/windows/processBoundary.cc -ladvapi32
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: real-ghidra-windows-process-boundary
|
||||
path: .cache/windows-process-boundary/
|
||||
if-no-files-found: error
|
||||
|
||||
verify:
|
||||
name: Real Ghidra 12.1.4 Windows x64 PE proof
|
||||
@@ -59,66 +38,30 @@ jobs:
|
||||
runs-on: [self-hosted, Windows, x64, ghidra-12-1-2]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: real-ghidra-windows-native
|
||||
path: native/windows/build/
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: real-ghidra-windows-process-boundary
|
||||
path: .cache/windows-process-boundary/
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- run: npm run verify:windows-native
|
||||
- name: Verify all admitted Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows | Tee-Object -FilePath windows-ghidra-proof.log
|
||||
- name: Verify native x86 Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --x86 | Tee-Object -FilePath windows-ghidra-x86-proof.log
|
||||
- name: Verify native x64 DLL Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --dll | Tee-Object -FilePath windows-ghidra-dll-proof.log
|
||||
- name: Verify native x86 DLL Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-proof.log
|
||||
- name: Verify ordinary-user CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package | Tee-Object -FilePath windows-ghidra-package-proof.log
|
||||
- name: Verify ordinary-user x86 CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --x86 | Tee-Object -FilePath windows-ghidra-x86-package-proof.log
|
||||
- name: Verify ordinary-user x64 DLL CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --dll | Tee-Object -FilePath windows-ghidra-dll-package-proof.log
|
||||
- name: Verify ordinary-user x86 DLL CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-package-proof.log
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: windows-ghidra-proof
|
||||
path: |
|
||||
windows-ghidra-proof.log
|
||||
windows-ghidra-package-proof.log
|
||||
windows-ghidra-x86-proof.log
|
||||
windows-ghidra-x86-package-proof.log
|
||||
windows-ghidra-dll-proof.log
|
||||
windows-ghidra-x86-dll-proof.log
|
||||
windows-ghidra-dll-package-proof.log
|
||||
windows-ghidra-x86-dll-package-proof.log
|
||||
retention-days: 30
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
name: Real Go binary metadata
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-go-binary.yml"
|
||||
- "src/go/**"
|
||||
- "src/domain/go/**"
|
||||
- "src/domain/dosMz.ts"
|
||||
- "src/application/go/**"
|
||||
- "src/contracts/go/**"
|
||||
- "src/composition/go.ts"
|
||||
- "src/cli/goCommands.ts"
|
||||
- "src/server/registerGoTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/artifacts/ArtifactReader.ts"
|
||||
- "src/process/PrivateRuntimeRoot.ts"
|
||||
- "scripts/verify-go-binary.mjs"
|
||||
- "scripts/lib/mcp-verifier-results.mjs"
|
||||
- "scripts/rea.mjs"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
- ".nvmrc"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: real-go-binary-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
real-go-binary:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
|
||||
with:
|
||||
go-version: "1.26.1"
|
||||
cache: false
|
||||
- run: npm ci
|
||||
- name: Verify compiler-produced ELF, PE and Mach-O through public CLI and MCP
|
||||
run: npm run verify:go:binary
|
||||
@@ -15,8 +15,8 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 25
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
@@ -36,8 +36,8 @@ jobs:
|
||||
runs-on: [self-hosted, Linux, hopper]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
|
||||
@@ -15,8 +15,8 @@ jobs:
|
||||
runs-on: [self-hosted, macOS, hopper]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
|
||||
@@ -1,18 +1,41 @@
|
||||
name: Real JavaScript recovery
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-javascript-recovery.yml"
|
||||
- "src/javascript/recovery/**"
|
||||
- "src/domain/javascript/**"
|
||||
- "src/application/javascript/JavaScriptRecovery*.ts"
|
||||
- "src/composition/javascriptRecovery.ts"
|
||||
- "src/contracts/javascript/javascriptRecoveryToolContracts.ts"
|
||||
- "src/cli/javascriptRecoveryCommands.ts"
|
||||
- "src/cli/javascriptApplicationAnalysis.ts"
|
||||
- "src/server/registerJavaScriptRecoveryTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "scripts/rea.mjs"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/SafeOutputTree.ts"
|
||||
- "scripts/verify/javascript/recovery.mjs"
|
||||
- "scripts/fixtures/javascript-recovery/**"
|
||||
- "third_party/wakaru"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: javascript-recovery-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
NODE_OPTIONS: "--max-old-space-size=768"
|
||||
@@ -23,22 +46,19 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
node-version: 24
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Fetch pinned Wakaru release into the job workspace
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-wakaru"
|
||||
curl --fail --location --retry 3 https://github.com/pionxzh/wakaru/releases/download/v1.14.0/wakaru-linux-x64.tar.gz -o "$RUNNER_TEMP/rea-wakaru/release.tar.gz"
|
||||
curl --fail --location --retry 3 https://github.com/pionxzh/wakaru/releases/download/v1.13.0/wakaru-linux-x64.tar.gz -o "$RUNNER_TEMP/rea-wakaru/release.tar.gz"
|
||||
tar -xzf "$RUNNER_TEMP/rea-wakaru/release.tar.gz" -C "$RUNNER_TEMP/rea-wakaru" wakaru
|
||||
printf '%s %s\n' 664fa06782a61eda7ed15885134b52525fd565dd77607e642062ceb6cd927c38 "$RUNNER_TEMP/rea-wakaru/wakaru" | sha256sum --check
|
||||
printf '%s %s\n' e8c7ca052974604197389f6ace7464e4dd24dce64e2730971521bd6606ec9c9e "$RUNNER_TEMP/rea-wakaru/wakaru" | sha256sum --check
|
||||
- name: Install isolated fixture compilers
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-fixture-compilers" --ignore-scripts --no-audit --no-fund esbuild@0.25.10 webpack@5.101.3
|
||||
- run: npm run build:cached
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
- "package*.json"
|
||||
- ".nvmrc"
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
REA_CAPINFOS_COMMAND: /usr/bin/capinfos
|
||||
@@ -1,77 +0,0 @@
|
||||
name: Real recorded crash evidence
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
UV_CONCURRENT_BUILDS: "1"
|
||||
UV_CONCURRENT_INSTALLS: "1"
|
||||
UV_PYTHON_DOWNLOADS: never
|
||||
|
||||
jobs:
|
||||
real-recorded-crash:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install GDB on the disposable CI runner
|
||||
shell: bash
|
||||
run: |
|
||||
sudo apt-get update -qq
|
||||
sudo apt-get install --yes --no-install-recommends gdb
|
||||
- name: Acquire exact unchanged upstreams in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
command -v gcc
|
||||
command -v gdb
|
||||
command -v strace
|
||||
core_decoder_root="${RUNNER_TEMP}/rea-core-decoder"
|
||||
core_pwndbg_root="${RUNNER_TEMP}/rea-core-pwndbg"
|
||||
core_pwndbg_venv="${RUNNER_TEMP}/rea-core-pwndbg-venv"
|
||||
python3 -m venv "${core_decoder_root}"
|
||||
"${core_decoder_root}/bin/python" -m pip install --disable-pip-version-check \
|
||||
'pwntools==4.15.0' 'pyelftools==0.33' 'unicorn==2.1.2' 'uv==0.12.0'
|
||||
git init "${core_pwndbg_root}"
|
||||
git -C "${core_pwndbg_root}" remote add origin https://github.com/pwndbg/pwndbg.git
|
||||
git -C "${core_pwndbg_root}" fetch --depth=1 origin 55596aa676a2756efb8d67034f31675170eabe48
|
||||
git -C "${core_pwndbg_root}" checkout --detach FETCH_HEAD
|
||||
test "$(git -C "${core_pwndbg_root}" rev-parse HEAD)" = 55596aa676a2756efb8d67034f31675170eabe48
|
||||
UV_PROJECT_ENVIRONMENT="${core_pwndbg_venv}" \
|
||||
"${core_decoder_root}/bin/uv" sync --project "${core_pwndbg_root}" \
|
||||
--frozen --no-editable --no-default-groups --python /usr/bin/python3 --no-managed-python
|
||||
test -z "$(git -C "${core_pwndbg_root}" status --porcelain --untracked-files=no)"
|
||||
echo "REA_PWNTOOLS_PYTHON=${core_decoder_root}/bin/python" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_GDB=$(command -v gdb)" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_GDBINIT=${core_pwndbg_root}/gdbinit.py" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_VENV_PATH=${core_pwndbg_venv}" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
- name: Build and install the exact package in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
npm run build:cached
|
||||
npm pack --ignore-scripts --json --pack-destination "${RUNNER_TEMP}" > "${RUNNER_TEMP}/rea-core-pack.json"
|
||||
core_package_name=$(node -p 'JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8"))[0].filename' "${RUNNER_TEMP}/rea-core-pack.json")
|
||||
npm install --prefix "${RUNNER_TEMP}/rea-core-installed" --ignore-scripts --no-fund --no-audit "${RUNNER_TEMP}/${core_package_name}"
|
||||
- name: Verify installed CLI and MCP against the source-owned recording
|
||||
shell: bash
|
||||
run: node scripts/verify-recorded-crash.mjs "${RUNNER_TEMP}/rea-core-installed/node_modules/rea-agents/scripts/rea.mjs"
|
||||
@@ -1,50 +0,0 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
- "package*.json"
|
||||
- ".nvmrc"
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -1,57 +0,0 @@
|
||||
name: Real Termux browser verification
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: real-termux-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
real-termux:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Download the pinned Termux app and verify its release checksum
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-termux"
|
||||
cd "$RUNNER_TEMP/rea-termux"
|
||||
release=https://github.com/termux/termux-app/releases/download/v0.118.3
|
||||
apk=termux-app_v0.118.3+github-debug_x86_64.apk
|
||||
sums=termux-app_v0.118.3+github-debug_sha256sums
|
||||
curl --fail --location --retry 3 --output "$apk" "$release/$apk"
|
||||
curl --fail --location --retry 3 --output "$sums" "$release/$sums"
|
||||
awk -v apk="$apk" '$2 == apk || $2 == "*" apk' "$sums" > selected.sha256
|
||||
test "$(wc -l < selected.sha256)" -eq 1
|
||||
sha256sum --check selected.sha256
|
||||
- name: Enable emulator acceleration
|
||||
run: |
|
||||
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
|
||||
sudo udevadm control --reload-rules
|
||||
sudo udevadm trigger --name-match=kvm
|
||||
- name: Build and capture a page inside Termux
|
||||
uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0
|
||||
with:
|
||||
api-level: 30
|
||||
arch: x86_64
|
||||
target: google_apis
|
||||
ram-size: 4096M
|
||||
heap-size: 512M
|
||||
disk-size: 8192M
|
||||
disable-animations: true
|
||||
script: bash scripts/verify/termux/emulator.sh
|
||||
- name: Upload Termux and Android diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: termux-browser-diagnostics
|
||||
path: ${{ runner.temp }}/rea-termux/diagnostics
|
||||
if-no-files-found: warn
|
||||
@@ -1,49 +0,0 @@
|
||||
name: Real historical web network captures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
|
||||
jobs:
|
||||
real-web-network-captures:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Use pinned package manager
|
||||
run: npm install --global npm@11.16.0
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire fixed upstream native profile in runner temporary storage
|
||||
shell: bash
|
||||
run: |
|
||||
capture_tool_root="${RUNNER_TEMP}/rea-mitmproxy-12.2.3"
|
||||
mkdir -p "${capture_tool_root}"
|
||||
curl --fail --location --retry 3 --max-time 180 \
|
||||
https://downloads.mitmproxy.org/12.2.3/mitmproxy-12.2.3-linux-x86_64.tar.gz \
|
||||
--output "${capture_tool_root}/mitmproxy.tar.gz"
|
||||
echo '2e95286b618fa6fd33e5e62a78c2e5112571d85f42ec2bac29b97ee242bdb5c5 mitmproxy.tar.gz' \
|
||||
| (cd "${capture_tool_root}" && sha256sum --check --strict)
|
||||
tar -xzf "${capture_tool_root}/mitmproxy.tar.gz" -C "${capture_tool_root}"
|
||||
echo "REA_MITMDUMP_COMMAND=${capture_tool_root}/mitmdump" >> "${GITHUB_ENV}"
|
||||
- name: Verify upstream producer fixtures through public CLI and MCP
|
||||
run: npm run verify:web:network-captures
|
||||
@@ -1,15 +1,40 @@
|
||||
name: Real website runtime attribution
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-web-runtime.yml"
|
||||
- "scripts/verify-browser-runtime.mjs"
|
||||
- "scripts/lib/browser-runtime-*.mjs"
|
||||
- "src/process/ProviderProcess.ts"
|
||||
- "src/process/ProcessOwnership*.ts"
|
||||
- "scripts/fixtures/browser-runtime-site.mjs"
|
||||
- "src/browser/execution/**"
|
||||
- "src/browser/CdpConnection.ts"
|
||||
- "src/browser/CdpEndpoint.ts"
|
||||
- "src/browser/CdpTarget*.ts"
|
||||
- "src/browser/CdpCaptureValues.ts"
|
||||
- "src/browser/CdpAuthorized*.ts"
|
||||
- "src/domain/webRuntime.ts"
|
||||
- "src/domain/webExecution.ts"
|
||||
- "src/domain/webEventListeners.ts"
|
||||
- "src/domain/browserObservationError*.ts"
|
||||
- "src/application/WebRuntime*.ts"
|
||||
- "src/composition/webRuntime.ts"
|
||||
- "src/contracts/webRuntimeToolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/server/registerWebRuntimeTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli/webRuntimeCommands.ts"
|
||||
- "src/cli/commandCancellation.ts"
|
||||
- "src/cli.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-web-runtime-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
env:
|
||||
HUSKY: "0"
|
||||
jobs:
|
||||
@@ -17,15 +42,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
node-version: 24
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run verify:browser:runtime
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
|
||||
@@ -1,15 +1,33 @@
|
||||
name: Real website source-map verification
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-web-source-map.yml"
|
||||
- "scripts/verify-browser-source-maps.mjs"
|
||||
- "scripts/lib/browser-source-map-e2e.mjs"
|
||||
- "scripts/fixtures/browser-source-map-site.mjs"
|
||||
- "src/javascript/sourceMaps/**"
|
||||
- "src/browser/assets/**"
|
||||
- "src/domain/webSource*.ts"
|
||||
- "src/domain/webScript*.ts"
|
||||
- "src/application/WebSourceLocation*.ts"
|
||||
- "src/application/ValidateSelectedWebScript.ts"
|
||||
- "src/composition/webSourceLocations.ts"
|
||||
- "src/server/registerWebSourceLocationTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/contracts/webScriptToolContracts.ts"
|
||||
- "src/process/**"
|
||||
- "src/cli/webSourceLocationCommands.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-web-source-map-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
env:
|
||||
HUSKY: "0"
|
||||
jobs:
|
||||
@@ -17,15 +35,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
node-version: 24
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install the pinned compiler into an isolated fixture directory
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false esbuild@0.25.10
|
||||
- run: npm run verify:browser:source-maps
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
name: Regenerate generated metadata
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "src/**"
|
||||
- "skills/**/SKILL.md"
|
||||
- "scripts/generate-*.mjs"
|
||||
- "scripts/check-doc-facts.mjs"
|
||||
- "scripts/lib/*.mjs"
|
||||
- "scripts/verify/managed/**"
|
||||
- "scripts/fixtures/managed/**"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
|
||||
concurrency:
|
||||
group: regen-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
regen:
|
||||
name: Regenerate and push tracked generated files
|
||||
# Same-repo PRs only: the default token cannot push to forks.
|
||||
# The bot-actor guard stops looped runs triggered by our own push.
|
||||
if: >-
|
||||
github.actor != 'github-actions[bot]' &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.ref }}
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run docs:generate
|
||||
- name: Commit regenerated files
|
||||
shell: bash
|
||||
env:
|
||||
HEAD_REF: ${{ github.event.pull_request.head.ref }}
|
||||
run: |
|
||||
generated_paths=(
|
||||
docs/product-catalog.json
|
||||
docs/error-contract.schema.json
|
||||
docs/verification/managed-conformance-manifest.json
|
||||
docs/verification/managed-conformance-ledger.json
|
||||
skills/reverse-engineer-anything/SKILL.md
|
||||
src/generatedPackageMetadata.ts
|
||||
)
|
||||
if git diff --quiet -- "${generated_paths[@]}"; then
|
||||
echo "Generated metadata is already canonical."
|
||||
exit 0
|
||||
fi
|
||||
git diff --check
|
||||
# Refuse to sweep unrelated working-tree changes into the bot commit.
|
||||
unexpected="$(git diff --name-only | grep -vxF -f <(printf '%s\n' "${generated_paths[@]}") || true)"
|
||||
if [[ -n "${unexpected}" ]]; then
|
||||
echo "docs:generate left unexpected tracked changes:" >&2
|
||||
printf '%s\n' "${unexpected}" >&2
|
||||
exit 1
|
||||
fi
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add -- "${generated_paths[@]}"
|
||||
git commit -m "chore: regenerate generated metadata"
|
||||
git push origin "HEAD:${HEAD_REF}"
|
||||
+57
-198
@@ -1,11 +1,6 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
types: [closed]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
release_branch:
|
||||
@@ -23,113 +18,12 @@ env:
|
||||
HUSKY: "0"
|
||||
|
||||
concurrency:
|
||||
group: release-${{ inputs.release_branch || github.event.pull_request.merge_commit_sha || github.ref_name }}
|
||||
group: release-${{ inputs.release_branch }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
release-proposal:
|
||||
name: Update release proposal
|
||||
if: github.event_name == 'push'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Respect finalized release notes
|
||||
id: editorial
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
|
||||
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
|
||||
if [[ -n "${finalized}" ]]; then
|
||||
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "frozen=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: true
|
||||
|
||||
merged-release:
|
||||
name: Release reviewed main merge
|
||||
if: >-
|
||||
github.event_name == 'pull_request' && github.event.pull_request.merged == true &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository &&
|
||||
startsWith(github.event.pull_request.head.ref, 'release-please--branches--main--')
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
release_created: ${{ steps.release.outputs.release_created }}
|
||||
sha: ${{ steps.release.outputs.sha }}
|
||||
steps:
|
||||
- name: Check out reviewed release merge
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Validate reviewed merge before release creation
|
||||
env:
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
if [[ "${SOURCE_SHA}" != "${GITHUB_SHA}" ]]; then
|
||||
echo "The reviewed merge differs from the workflow source; refusing mismatched publication provenance." >&2
|
||||
exit 1
|
||||
fi
|
||||
node scripts/verify-release-checkpoint.mjs \
|
||||
--phase publish --release-branch main --source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate finalized notes before release creation
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
RELEASE_PR: ${{ github.event.pull_request.number }}
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA" --pr "$RELEASE_PR"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
id: release
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: false
|
||||
skip-github-pull-request: true
|
||||
|
||||
- name: Bind publication to the reviewed merge
|
||||
env:
|
||||
RELEASE_CREATED: ${{ steps.release.outputs.release_created }}
|
||||
RELEASE_SHA: ${{ steps.release.outputs.sha }}
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
if [[ "${RELEASE_CREATED}" != true ]]; then
|
||||
echo "Release Please did not create a release for the reviewed merge. Check its pending label and release metadata; for a partial publication, re-run failed jobs in the original run." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "${RELEASE_SHA}" != "${SOURCE_SHA}" ]]; then
|
||||
echo "The tagged release differs from the reviewed merge; refusing mismatched publication provenance. The GitHub release already exists; do not create another release as a retry." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
release-please:
|
||||
name: Release Please
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
@@ -190,43 +84,7 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Check out release controller and Git history
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for checkpoint validation
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install checkpoint validation dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Validate checkpoint version and ancestry before release creation
|
||||
env:
|
||||
RELEASE_PHASE: ${{ inputs.phase }}
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: >-
|
||||
node scripts/verify-release-checkpoint.mjs
|
||||
--phase "$RELEASE_PHASE" --release-branch "$RELEASE_BRANCH"
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate manual finalized notes before release creation
|
||||
if: inputs.phase == 'publish'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
- uses: googleapis/release-please-action@v4
|
||||
id: release
|
||||
with:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
@@ -255,15 +113,14 @@ jobs:
|
||||
|
||||
- name: Check out release pull request
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for generated documentation
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
@@ -272,54 +129,63 @@ jobs:
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Validate prepared release version and migration notes
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
env:
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: >-
|
||||
node scripts/verify-release-checkpoint.mjs
|
||||
--phase prepare --stage candidate --release-branch "$RELEASE_BRANCH"
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Regenerate release documentation
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:generate
|
||||
|
||||
- name: Validate generated release documentation
|
||||
- name: Commit canonical release catalog
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:check && git diff --exit-code
|
||||
|
||||
qualification:
|
||||
name: Qualify exact release revision
|
||||
needs: [merged-release, release-please]
|
||||
if: >-
|
||||
always() && !cancelled() &&
|
||||
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
|
||||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
uses: ./.github/workflows/ci.yml
|
||||
with:
|
||||
source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
shell: bash
|
||||
env:
|
||||
# Use the same identity for the follow-up commit so it also triggers CI.
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
RELEASE_PR_JSON: ${{ steps.release.outputs.pr }}
|
||||
run: |
|
||||
release_pr_branch="$(node --input-type=module -e '
|
||||
const value = JSON.parse(process.env.RELEASE_PR_JSON ?? "");
|
||||
if (
|
||||
typeof value !== "object" ||
|
||||
value === null ||
|
||||
typeof value.headBranchName !== "string" ||
|
||||
value.headBranchName.length === 0
|
||||
) {
|
||||
process.exit(1);
|
||||
}
|
||||
process.stdout.write(value.headBranchName);
|
||||
')"
|
||||
changed_paths="$(git diff --name-only)"
|
||||
if [[ -z "${changed_paths}" ]]; then
|
||||
echo "Release documentation is already canonical."
|
||||
exit 0
|
||||
fi
|
||||
if [[ "${changed_paths}" != "docs/product-catalog.json" ]]; then
|
||||
echo "docs:generate changed unexpected tracked paths:" >&2
|
||||
printf '%s\n' "${changed_paths}" >&2
|
||||
exit 1
|
||||
fi
|
||||
git diff --check
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add -- docs/product-catalog.json
|
||||
git commit -m "chore: normalize release product catalog"
|
||||
gh auth setup-git
|
||||
git push origin "HEAD:${release_pr_branch}"
|
||||
|
||||
publish:
|
||||
name: Publish to npm
|
||||
needs: [merged-release, release-please, qualification]
|
||||
if: >-
|
||||
always() && !cancelled() && needs.qualification.result == 'success'
|
||||
needs: release-please
|
||||
if: inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
ref: ${{ needs.release-please.outputs.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
@@ -327,28 +193,24 @@ jobs:
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Download the exact qualified package
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: verified-release-package
|
||||
path: .cache/verified-package/
|
||||
- name: Prepare Windows native build toolchain
|
||||
run: sudo apt-get update && sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
|
||||
- name: Verify qualified package provenance and digest
|
||||
id: package
|
||||
run: node scripts/ci/verified-package.mjs validate .cache/verified-package
|
||||
- name: Build and verify bundled Windows native controls
|
||||
run: npm run build:windows-native && npm run verify:windows-native:artifact
|
||||
|
||||
- name: Verify packaged release before publishing
|
||||
run: npm run verify:package
|
||||
|
||||
- name: Publish
|
||||
shell: bash
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
run: |
|
||||
version="$(node --input-type=module -e 'import packageJson from "./package.json" with { type: "json" }; process.stdout.write(packageJson.version)')"
|
||||
published_version="$(npm view "rea-agents@${version}" version --json 2>/dev/null || true)"
|
||||
if [ "${published_version}" = "\"${version}\"" ]; then
|
||||
echo "rea-agents@${version} is already published; verifying the existing package."
|
||||
else
|
||||
tag="$(node scripts/release-npm-tag.mjs "${version}")"
|
||||
npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}"
|
||||
npm publish --access public
|
||||
fi
|
||||
|
||||
- name: Verify published CLI from npm
|
||||
@@ -382,19 +244,16 @@ jobs:
|
||||
|
||||
publish-mcp:
|
||||
name: Publish to MCP Registry
|
||||
needs: [merged-release, release-please, publish]
|
||||
if: >-
|
||||
always() && !cancelled() && needs.publish.result == 'success' &&
|
||||
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
|
||||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
|
||||
needs: [release-please, publish]
|
||||
if: inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true' && needs.publish.result == 'success'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
ref: ${{ needs.release-please.outputs.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Verify package and registry metadata versions
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
name: SQLite runtime compatibility
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
sqlite:
|
||||
name: SQLite runtime compatibility (Node ${{ matrix.node }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Exercise unavailable-runtime diagnostics, the first supported release,
|
||||
# and new patches on both supported lines without hand-editing versions.
|
||||
# The repository-pinned Node 24 version also runs the complete test suite.
|
||||
node: ["22.x", "24.15.0", "24.x", "26.x"]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version: ${{ matrix.node }}
|
||||
check-latest: true
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: .
|
||||
- name: Verify SQLite discovery, inspection and cleanup
|
||||
run: npm run test:sqlite:run
|
||||
@@ -1,30 +1,31 @@
|
||||
name: Website checks
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
paths:
|
||||
- "website/**"
|
||||
- "scripts/verify-website.py"
|
||||
- "scripts/prepare-website.py"
|
||||
- ".github/workflows/**"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: website-check-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
check:
|
||||
name: Website checks
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Python for website assets
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y libcairo2 fonts-dejavu-core
|
||||
python3 -m pip install -r website/requirements.txt
|
||||
- name: Check shared HTML, links, assets and contributor workflows
|
||||
run: python3 scripts/website.py check
|
||||
uses: actions/checkout@v4
|
||||
- name: Prepare the example download
|
||||
run: python3 scripts/prepare-website.py
|
||||
- name: Check website links and publishing configuration
|
||||
run: python3 scripts/verify-website.py
|
||||
|
||||
@@ -2,15 +2,6 @@ name: Publish REA website
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
target:
|
||||
description: Publish both hosts, or Pages while configuring Cloudflare
|
||||
type: choice
|
||||
required: true
|
||||
default: both
|
||||
options:
|
||||
- both
|
||||
- pages
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -23,156 +14,35 @@ jobs:
|
||||
prepare:
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
pages: read
|
||||
steps:
|
||||
- name: Validate the publication target
|
||||
env:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
case "$PUBLISH_TARGET" in
|
||||
both|pages) ;;
|
||||
*) echo "::error::Unknown publication target: $PUBLISH_TARGET"; exit 1 ;;
|
||||
esac
|
||||
- name: Require Cloudflare publishing credentials
|
||||
if: inputs.target == 'both'
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
run: |
|
||||
if [[ -z "$CLOUDFLARE_API_TOKEN" || -z "$CLOUDFLARE_ACCOUNT_ID" ]]; then
|
||||
echo "::error::Set CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID in repository Actions secrets before publishing."
|
||||
exit 1
|
||||
fi
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Python for website assets
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y libcairo2 fonts-dejavu-core
|
||||
python3 -m pip install -r website/requirements.txt
|
||||
- name: Check shared HTML, links, assets and contributor workflows
|
||||
run: python3 scripts/website.py check
|
||||
- name: Record the published commit
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
Path("website/_site/deployment.json").write_text(
|
||||
json.dumps({"commit": os.environ["GITHUB_SHA"]}) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
PY
|
||||
uses: actions/checkout@v4
|
||||
- name: Prepare the example download
|
||||
run: python3 scripts/prepare-website.py
|
||||
- name: Check website links and publishing configuration
|
||||
run: python3 scripts/verify-website.py
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
uses: actions/configure-pages@v5
|
||||
- name: Upload the website
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
uses: actions/upload-pages-artifact@v4
|
||||
with:
|
||||
path: website/_site
|
||||
path: website/public
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
pages: write
|
||||
id-token: write
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Check out publishing configuration
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Node.js for Wrangler
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Download the verified website
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/website-archive
|
||||
- name: Extract the shared publishing artifact
|
||||
if: inputs.target == 'both'
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-website-public"
|
||||
tar -xf "$RUNNER_TEMP/website-archive/artifact.tar" -C "$RUNNER_TEMP/rea-website-public"
|
||||
- name: Publish the website to Cloudflare
|
||||
if: inputs.target == 'both'
|
||||
uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
|
||||
with:
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
wranglerVersion: "4.149.0"
|
||||
packageManager: npm
|
||||
command: deploy --cwd website --env production --assets "${{ runner.temp }}/rea-website-public"
|
||||
- name: Publish to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5
|
||||
- name: Verify the selected published revisions
|
||||
env:
|
||||
PAGES_URL: ${{ steps.deployment.outputs.page_url }}
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
sys.exit("GitHub Pages returned no published URL.")
|
||||
target = os.environ["PUBLISH_TARGET"]
|
||||
if target not in ("both", "pages"):
|
||||
sys.exit("Unknown publication target: " + target)
|
||||
hosts = {"GitHub Pages": pages_url}
|
||||
if target == "both":
|
||||
hosts["Cloudflare"] = "https://rea.tools/"
|
||||
pending = dict(hosts)
|
||||
for attempt in range(24):
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
del pending[name]
|
||||
else:
|
||||
print(f"Waiting for {name}: observed revision {actual}")
|
||||
except (URLError, TimeoutError, ValueError) as error:
|
||||
print(f"Waiting for {name}: {error}")
|
||||
if not pending:
|
||||
break
|
||||
if attempt < 23:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
PY
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
+2
-26
@@ -33,14 +33,11 @@ codedb.snapshot
|
||||
htmlcov/
|
||||
.coverage
|
||||
coverage/
|
||||
.vitest/
|
||||
.vitest-reports/
|
||||
test-results.xml
|
||||
test-timings.json
|
||||
.turbo/
|
||||
.cache/
|
||||
docs/.vitepress/cache/
|
||||
docs/.vitepress/dist/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
@@ -48,30 +45,9 @@ Thumbs.db
|
||||
|
||||
# Local reference sources and analysis artifacts
|
||||
/_reference/
|
||||
/experiments/analysis-efficiency/results/
|
||||
|
||||
# Generated website download
|
||||
/website/public/examples/notes-example.zip
|
||||
/website/public/deployment.json
|
||||
/website/public/sitemap.xml
|
||||
/website/public/assets/social-card.png
|
||||
/website/.venv/
|
||||
|
||||
# Local Cloudflare development state
|
||||
/website/.wrangler/
|
||||
/website/.dev.vars*
|
||||
/website/.env*
|
||||
!/website/.dev.vars.example
|
||||
!/website/.env*.example
|
||||
|
||||
# Derived catalogs, portable conformance projections, and packaged skill bundles
|
||||
# (regenerated by `npm run docs:generate`; never commit)
|
||||
/skills/
|
||||
/docs/public/product-catalog.json
|
||||
/docs/verification/managed-conformance-manifest.json
|
||||
/docs/verification/managed-conformance-ledger.json
|
||||
|
||||
# Complete static website output, including generated language editions.
|
||||
website/_site/
|
||||
# Temporary translation output prepared before replacing the complete site.
|
||||
website/.i18n-*/
|
||||
# Build-generated MCP tool catalog (regenerated by `npm run build`; never commit)
|
||||
src/generatedMcpToolCatalog.ts
|
||||
|
||||
+2
-1
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"threshold": 3.5,
|
||||
"_comment": "Test files are intentionally NOT excluded: the gate used to report zero clones because it skipped **/*.test.ts, hiding all test-tree duplication. Current actual is ~2.4% lines across src+ and tests/, down from 3.4% when tests were included. Lower this as duplication is removed.",
|
||||
"threshold": 2.5,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
|
||||
+1
-2
@@ -22,8 +22,7 @@
|
||||
],
|
||||
"max-params": ["warn", 4],
|
||||
"no-var": "warn",
|
||||
"no-var-requires": "error",
|
||||
"typescript/no-import-type-side-effects": "error"
|
||||
"no-var-requires": "error"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.4.0"
|
||||
".": "5.0.0"
|
||||
}
|
||||
|
||||
@@ -1,82 +1,106 @@
|
||||
# Repository Guidelines
|
||||
|
||||
## Product and Authority
|
||||
## Product Direction
|
||||
|
||||
REA is a local-only reverse-engineering tool with shared CLI/MCP workflows. Distinguish observations, derivations, inferences, and unknowns; preserve Evidence provenance and session semantics through both adapters. Provider support and setup live in [README.md](README.md#choosing-a-deep-analysis-provider) and its linked guides.
|
||||
REA exposes reverse-engineering tools through a CLI and MCP server. Hopper, the bring-your-own Ghidra adapter, and the bring-your-own IDA MCP adapter are operation-capable deep binary-analysis providers. Ghidra is supported on Linux x64 and macOS x64/arm64 with matching native decompiler tools, and has an experimental Windows x64 P0 boundary for approved native x86-64 PE applications; on Linux and macOS it supplies inventory, function analysis, and atomic function annotation edits in an ephemeral database, without modifying executable bytes or controlling a GUI. Windows P0 has no mutation authority. IDA adapts upstream legacy attached GUI and modern headless database-supervisor profiles for live read-only analysis; initial real verification covers Windows, and an attached GUI database is never saved or closed. Keep provider-specific code out of the domain and application layers.
|
||||
|
||||
Follow the [contribution scope and fail-closed integration rule](CONTRIBUTING.md#contribution-scope). Do not integrate capabilities while material scope, authority, side-effect, or required verification questions remain unresolved. For new providers or capabilities involving target execution, injection, mutation, or protection recovery, seek the required maintainer scope decision before substantial implementation for integration. Welcome early proposals, small exploratory prototypes, and draft PRs that help resolve questions; they do not authorize integration.
|
||||
Prioritize:
|
||||
|
||||
Ghidra uses ephemeral databases without modifying executable bytes or controlling a GUI; Windows Ghidra P0 has no mutation authority. IDA analysis is read-only; never save or close an attached GUI database.
|
||||
- tool results that distinguish observations, inferences, and unknowns;
|
||||
- equivalent behavior through the CLI and MCP;
|
||||
- additive, idempotent configuration with backups;
|
||||
- end-to-end tests for packaged artifacts and real Hopper/Ghidra claims.
|
||||
|
||||
Configuration changes must be additive and idempotent, with backups. Installers must not install or upgrade unrelated software, including Homebrew, Node.js, npm, Java, or Ghidra. Ghidra and IDA are bring-your-own. `rea setup` must show its planned changes and require approval before writing files or installing Hopper.
|
||||
Installers must not install or upgrade Homebrew, Node.js, npm, Java, Ghidra, or other unrelated software. Ghidra is bring-your-own. `rea setup` must print its planned changes and require approval before writing files or installing Hopper.
|
||||
|
||||
Preserve caller-selected inputs and local evidence, including output, URLs, paths, digests, and metadata. Redact transport authentication credentials and explicitly marked sensitive values; do not infer secrecy from names or text patterns. Do not persist the ambient environment merely because a child inherits it.
|
||||
REA is a local-only tool. Preserve caller-selected inputs, captured output, URLs, paths, digests, mismatch locations, and analysis metadata. Do not guess that local evidence is secret from environment-variable names, argument names, or text patterns. Redact transport authentication credentials and values the caller explicitly marks sensitive; do not persist the entire ambient environment merely because a child inherits it.
|
||||
|
||||
## Architecture and Cleanup
|
||||
## Project Structure & Module Organization
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
REA is a layered ESM TypeScript application. Dependencies flow inward from pure domain logic through contracts, providers, application workflows, and CLI/MCP adapters. See [docs/architecture.mermaid](docs/architecture.mermaid) for the component map.
|
||||
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
- `src/domain/` owns pure provider-neutral semantics; `src/contracts/` owns caller-visible schemas and the canonical tool inventory.
|
||||
- `src/hopper/`, `src/ghidra/`, `src/ida/`, `src/browser/`, `src/inspector/`, `src/native/`, `src/artifacts/`, and `src/dotnet/` own provider-specific boundaries. Keep provider protocols out of domain and application code.
|
||||
- `src/application/` composes shared CLI/MCP workflows; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
|
||||
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. `bridge/` contains provider-side adapters.
|
||||
- `tests/` contains unit, composition, boundary, acceptance, and conformance tests. `scripts/verify-*` and capability directories under `scripts/verify/` contain real-toolchain checks.
|
||||
- `docs/product-catalog.json` is generated. Update its source contracts and regenerate it; do not edit it directly.
|
||||
- `src/generatedMcpToolCatalog.ts` is build-generated and gitignored. Never commit it; resolve any trace of it in merges by deleting it and running `npm run build:cached`.
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
## Build, Test, and Development Commands
|
||||
|
||||
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
|
||||
- `npm ci`: install the locked dependencies.
|
||||
- `npm run build:cached`: build the CLI and MCP server.
|
||||
- `npm run test:local`: run changed source tests without building; pass exact source test paths to run them regardless of Git status.
|
||||
- `npm run test:focused -- PATH...`: run exact test files; build first for boundary, acceptance, or process-global tests.
|
||||
- `npm run check:changed`: run cached static checks and source tests affected since the branch merge base (default `origin/main`).
|
||||
- `npm run check:fast`: run cached typecheck and lint checks.
|
||||
- `npm run check:pr`: opt into the complete local deterministic gate and generated-document checks for broad changes; CI owns full coverage. Routine iterations need focused tests and relevant checks, not the whole gate each time.
|
||||
- `npm run docs:check`: check committed generated documents; `npm run docs:generate` regenerates them.
|
||||
- For provider-dependent changes, see [docs/testing.md](docs/testing.md) and run the matching real-provider verification.
|
||||
- Keep each verification lane's prerequisites limited to the claim it checks. Use host-native fixtures for host/provider acceptance; put optional cross-target formats and their external toolchains in a separate lane. Preflight required commands and report the missing dependency and lane clearly.
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
See [docs/testing.md](docs/testing.md) for test scopes and verification lanes and [CONTRIBUTING.md](CONTRIBUTING.md) for contribution checks. Pre-commit formats and lints staged source; pre-push runs `npm run check:fast`.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
## Configuration & Environment Variables
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
Configuration is parsed and validated by `src/config.ts` and `src/config/`. Keep user-facing setup and provider configuration in [README.md](README.md) and the relevant guide under `docs/`; do not maintain a second environment-variable catalogue here.
|
||||
|
||||
Validate advertised JSON Schemas after SDK conversion against their declared dialect and actual clients. Bind handlers to named contracts with exact types; catalog ordering must never select an operation or schema.
|
||||
## Coding Style & Naming Conventions
|
||||
|
||||
## Performance
|
||||
Use ESM TypeScript, two-space indentation, and the committed Oxfmt configuration. Keep compiler strictness intact (`strict`, `exactOptionalPropertyTypes`, `noUncheckedIndexedAccess`, `verbatimModuleSyntax`). Use `camelCase` for values/functions, `PascalCase` for classes/types, and `UPPER_SNAKE_CASE` for constants. Parse unknown values at every MCP, environment, and subprocess boundary. Avoid `any`, unchecked casts, non-null assertions, import-time I/O, and floating promises. Exported APIs require concise JSDoc. Model expected failures with the tagged error algebra and `Result`, not broad exception wrappers.
|
||||
|
||||
Design analysis workflows and agent interfaces for efficiency from the first
|
||||
implementation. Correctness, useful answers, Evidence provenance, caller choices,
|
||||
compatibility and safe resource bounds remain required.
|
||||
## Boundary Contracts
|
||||
|
||||
Optimize in this order:
|
||||
Treat a boundary as a contract between the producer's actual representation and the consumer's required meaning. When implementing or auditing a boundary, trace the value through parsing, normalization, authorization, serialization, and the CLI/MCP result. Establish affected callers from their code paths; similar tool names or workflows do not prove that they share a schema or failure mode.
|
||||
|
||||
1. Agent tokens and context. Return the smallest sufficient answer with
|
||||
necessary provenance, coverage and limitations. Use task selection,
|
||||
summaries, paging and retained Evidence where supported; honor requested
|
||||
detail and keep complete Evidence accessible. Count the whole task,
|
||||
including schemas, extra calls and retries.
|
||||
2. Analysis time. Reuse validated results and shared work. Cache keys must
|
||||
cover result-affecting inputs; define invalidation and lifetime. Validate
|
||||
freshness before reporting cached live state as current.
|
||||
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
|
||||
retained state and serialization growth.
|
||||
Keep portable evidence and scenario validation distinct from host-native execution checks. Absolute filesystem paths, file URLs, and HTTP paths have different semantics; do not substitute one platform's syntax for the domain concept. Interpret provider metadata according to its documented or observed producer behavior. When a transformation loses information, preserve the reported value and an explicit unknown rather than guessing a canonical identity.
|
||||
|
||||
For changes affecting these costs, measure representative completed tasks and
|
||||
compare equivalent workflows before and after where available. Report token
|
||||
usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
|
||||
do not establish fewer tokens or faster analysis.
|
||||
Preserve meaningful failure reasons through application and adapter layers. Malformed input is distinct from an unsupported target, unavailable provider, or host operating-system permission denial. Diagnostics should identify the failed constraint and the target or lifecycle request it applies to. Recovery advice must address that reason and point to an available workflow; generic catches must not erase actionable validation details.
|
||||
|
||||
## Development
|
||||
Leave meaningful target, action, capture, and output choices to the agent. A selected operation already expresses intent; do not require approval booleans or repeated permission declarations. Trace each setting to its consumer: remove ignored options and single-value confirmations, derive built-in lifecycle behavior, and supply defaults for omitted optional metadata. Report actual effects and limitations where they help interpret results rather than asking callers to restate them.
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
|
||||
Verify the representation consumed at the next boundary. Internal validator success does not establish JSON Schema validity or client compatibility: validate advertised input and output schemas against their declared dialect after SDK conversion, and keep generated contracts aligned. Use representative producer data and regressions that exercise the failed behavior. Platform and provider support claims require the corresponding real workflow; package startup, capability probes, and mock transport tests establish only their narrower claims. Report unverified coverage explicitly.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
- `npm run build:cached`: build the runtime and packaged skill.
|
||||
- `npm run test:local -- PATH...`: source tests without building.
|
||||
- `npm run test:focused -- PATH...`: exact tests with their required artifacts.
|
||||
- `npm run check:fast`: cached typecheck/lint and the pre-push check.
|
||||
- `npm run docs:check`: validate generated documentation.
|
||||
Bind tool handlers to named contracts, preserving their exact input and output types. Catalog ordering is presentation metadata; changing that order must not select a different schema or operation for an existing handler.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
## Designing MCP Tools
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
Start from the analyst question and desired result, not a provider API. Before adding a tool, inspect the existing contract and its nearest alternative.
|
||||
|
||||
## Tool and Test Changes
|
||||
Let agents compose experiments with ordinary commands, scripts, and local fixture servers. REA tools should perform the requested inspection or capture and return evidence. A custom orchestration language, replay engine, or separate prepare/execute plan needs an observed requirement that those primitives cannot satisfy. A plan-only tool without an executor does not establish runtime behavior; expose the useful inspection directly.
|
||||
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
- Prefer reusable, composable primitives: inspect one explicit object or
|
||||
relationship and return evidence that can support different analyst
|
||||
questions. Compose primitives into a workflow only for a recurring outcome
|
||||
that benefits from joining evidence sources; keep one-application or
|
||||
business-domain interpretations out of general tool contracts.
|
||||
- Use **inspect/search** tools for facts about a target or candidate set; use **trace** tools for relationships; use **compare** tools for explicitly paired inputs.
|
||||
- Add a **workflow** when observed agent use shows a repeated sequence that REA can compose without losing analyst control. Return useful results inline so callers can choose their next action.
|
||||
- Use **observe/capture** only when runtime activity is required, and declare authority and lifecycle effects in the contract.
|
||||
- Extend an existing tool when intent and result contract are unchanged. Add a tool for a distinct analyst outcome or materially different authority.
|
||||
- Keep caller-facing names and results provider-neutral. Put engine-specific behavior in provider adapters and report each provider's exact coverage.
|
||||
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Keep observed, derived, inferred, and unknown results distinct.
|
||||
- Include artifact identity, source locations, Evidence references, actionable errors, and relevant limitations when they affect conclusions. Return the evidence needed for the next analysis inline rather than requiring a resource or identifier lookup.
|
||||
- Implement shared application workflows behind CLI and MCP adapters. Update canonical contracts and generated catalog together.
|
||||
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
See [docs/tool-design.md](docs/tool-design.md) for the design checklist. When usability or tool selection changes, evaluate representative CLI/MCP tasks as well as schema and transport behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
## Testing Guidelines
|
||||
|
||||
Name tests `*.test.ts`. Use Vitest and production seams (`tests/fixtures/`) rather than module mocks. Domain tests assert pure behavior; adapter tests use fake launcher/socket seams; MCP tests connect with the client SDK version pinned in `package.json`. Preserve the canonical tool inventory defined by `TOOL_CONTRACTS` and verified through `CATALOG_IDENTITY` and generated product metadata. Cover malformed input, cancellation, lifecycle cleanup, and actual format, protocol, host-permission, and target-identity boundaries. Do not add tests that merely freeze arbitrary caps or prescribed call sequences. Real Hopper, Ghidra, browser, managed conformance, and any real managed-tool claims cannot be replaced by mocks; use the corresponding `verify:*` command.
|
||||
|
||||
Keep tool catalogs complete and self-describing; prefer capability- and session-scoped availability over schema truncation. Serialized bytes alone do not measure agent usability or model context cost.
|
||||
|
||||
## Cursor Cloud specific instructions
|
||||
|
||||
Development requires Node.js 24.18.0 and npm 11.16.0 (`.nvmrc` and `packageManager`). The Cloud Agent image places an older `node` on `PATH` ahead of a normal install. Environment setup installs the pinned toolchain under `/usr/local` and prepends `/usr/local/bin` for login shells. Confirm `node -v` is `v24.18.0` before installing dependencies.
|
||||
|
||||
`npm ci` installs locked dependencies. `npm run build:cached` produces the CLI and MCP server. `npm run check:fast` is the pre-push typecheck and lint. Hopper, Ghidra, and IDA are optional bring-your-own providers. JavaScript analysis and the deterministic Vitest suites do not need them. `rea doctor` reports those engines as missing until they are configured.
|
||||
|
||||
## Commit & Pull Request Guidelines
|
||||
|
||||
Use Conventional Commit subjects because Release Please derives versions and changelogs from them. Examples: `feat: add historical source import`, `fix(process): stop timers after exit`, and `docs: update architecture`. Use `!` or a `BREAKING CHANGE:` footer for breaking changes. Pull request titles must follow the same format because squash merges use the title as the release commit. Pull requests should describe contract or behavior changes, list verification commands, link issues, and include sanitized MCP examples when schemas change. State whether real Hopper/Ghidra verification was performed. Never commit binaries, Hopper or Ghidra project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts (e.g. `.codex/`).
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
|
||||
-797
@@ -1,802 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.4.0](https://github.com/morluto/rea/compare/rea-agents-6.3.0...rea-agents-6.4.0) (2026-10-11)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Android: added device inspection, split-APK acquisition, logs, screenshots, file transfers and explicit app lifecycle actions through ADB, plus manifest and localized-string decoding with Apktool. ([#1552](https://github.com/morluto/rea/pull/1552), [#1556](https://github.com/morluto/rea/pull/1556)) Thanks @3QL.
|
||||
- JEB: browse project units and decompile selected items through a caller-started local JEB MCP server. ([#1536](https://github.com/morluto/rea/pull/1536)) Thanks @3QL.
|
||||
- Large analyses: discover retained Evidence without loading every payload, and inspect JavaScript graphs, binary layouts and function dossiers through summaries and selected views. ([#1682](https://github.com/morluto/rea/pull/1682), [#1568](https://github.com/morluto/rea/pull/1568), [#1570](https://github.com/morluto/rea/pull/1570), [#1345](https://github.com/morluto/rea/pull/1345), [#1507](https://github.com/morluto/rea/pull/1507), [#1247](https://github.com/morluto/rea/issues/1247)) Thanks @jhuang-tw, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- JavaScript: isolate source analysis in an owned process with configurable heap and timeout limits; retain completed facts and explicit unknowns when analysis runs out of resources. ([#1651](https://github.com/morluto/rea/pull/1651), [#1563](https://github.com/morluto/rea/issues/1563), [#1565](https://github.com/morluto/rea/issues/1565)) Thanks @N0zoM1z0, @peter-research.
|
||||
- Ghidra: configure import language/compiler specifications and seed functions, code and labels before auto-analysis. ([#1667](https://github.com/morluto/rea/pull/1667)) Thanks @sqmagellan.
|
||||
- RPC discovery: recognize supported tRPC calls, HTTP-link URLs and named GraphQL operations in JavaScript applications and web bundles. ([#1670](https://github.com/morluto/rea/pull/1670)) Thanks @pompiliodonatoapp.
|
||||
|
||||
### Changes
|
||||
|
||||
- Added Ghidra support for Windows PE DLLs and bounded MIPS ELF analysis, including carved MIPS inputs with incomplete section tables. Windows P0 remains read-only. ([#1303](https://github.com/morluto/rea/pull/1303), [#1362](https://github.com/morluto/rea/pull/1362), [#1629](https://github.com/morluto/rea/pull/1629), [#1290](https://github.com/morluto/rea/issues/1290), [#718](https://github.com/morluto/rea/issues/718), [#1619](https://github.com/morluto/rea/issues/1619)) Thanks @bqw371, @jaelliot, @kvalls, @N0zoM1z0.
|
||||
- Enabled `set_address_name` and `set_addresses_names` for Ghidra sessions with mutation authority, including namespace-aware data labels and per-address batch results. ([#1635](https://github.com/morluto/rea/pull/1635), [#1634](https://github.com/morluto/rea/issues/1634)) Thanks @giri256.
|
||||
- Recovered additional signed AArch64 relative jump-table patterns. ([#1592](https://github.com/morluto/rea/pull/1592)) Thanks @RishTheFish21.
|
||||
- Added PE resource inspection and read-only NativeAOT metadata parsing for supported x64 PE32+/.NET 8 RTR 9.1 inputs. The NativeAOT parser’s Windows-host Ghidra workflow remains unverified on a real Windows host. ([#1482](https://github.com/morluto/rea/pull/1482), [76d8f68](https://github.com/morluto/rea/commit/76d8f68cb918dfb43869f2b649a0b6c0c1cf266d), [#1254](https://github.com/morluto/rea/issues/1254)) Thanks @N0zoM1z0, @thangldw.
|
||||
- Added Pi, Hermes and Qwen Code setup, including Hermes active-profile support. ([#1376](https://github.com/morluto/rea/pull/1376), [#1403](https://github.com/morluto/rea/pull/1403), [#1415](https://github.com/morluto/rea/pull/1415), [#1409](https://github.com/morluto/rea/issues/1409)) Thanks @rasatpetabit, @yiliang114, @zaydiscold.
|
||||
- Added `REA_MCP_INPUT_SCHEMA_PROFILE=compact` for size-capped MCP clients such as Kimi/Moonshot, while retaining canonical runtime validation. ([#1514](https://github.com/morluto/rea/pull/1514), [#1580](https://github.com/morluto/rea/pull/1580), [#1484](https://github.com/morluto/rea/issues/1484), [#1581](https://github.com/morluto/rea/issues/1581)) Thanks @haythemsellami, @N0zoM1z0.
|
||||
- Added live process-capture progress and optional `finalization_ms` to capture shutdown output before timeout escalation; cancellation remains immediate. ([#1483](https://github.com/morluto/rea/pull/1483), [#1583](https://github.com/morluto/rea/pull/1583), [#1252](https://github.com/morluto/rea/issues/1252), [#1253](https://github.com/morluto/rea/issues/1253), [modelcontextprotocol/typescript-sdk#2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997)) Thanks @N0zoM1z0, @nikolay-dementiev, @sathariels.
|
||||
- Improved Codex onboarding and recovery from observed client, PATH and provider blockers. ([#1613](https://github.com/morluto/rea/pull/1613)) Thanks @N0zoM1z0.
|
||||
- Reduced redirected JSON and terminal output size, and deferred heavy imports until their commands need them. ([#1413](https://github.com/morluto/rea/pull/1413), [#1450](https://github.com/morluto/rea/pull/1450), [#1439](https://github.com/morluto/rea/pull/1439), [21dbcf7](https://github.com/morluto/rea/commit/21dbcf7a1458219989ad291e44ce7d017378b727)) Thanks @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Electron analysis: follow renamed and default-imported APIs and optional calls, ignore overwritten identities and media preload attributes, and resolve packaged `app.asar` pages and unpacked companions. ([#1391](https://github.com/morluto/rea/pull/1391), [#1393](https://github.com/morluto/rea/pull/1393), [#1400](https://github.com/morluto/rea/pull/1400), [#1664](https://github.com/morluto/rea/pull/1664), [#1673](https://github.com/morluto/rea/pull/1673)) Thanks @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @suninrain086.
|
||||
- JavaScript data flow: preserve returned/yielded reference uncertainty, invalidate property-stored aliases correctly, and retain bundled-module runtime scope. ([#1437](https://github.com/morluto/rea/pull/1437), [#1445](https://github.com/morluto/rea/pull/1445), [#1471](https://github.com/morluto/rea/pull/1471), [#1577](https://github.com/morluto/rea/pull/1577), [#1428](https://github.com/morluto/rea/issues/1428), [#1442](https://github.com/morluto/rea/issues/1442), [#1449](https://github.com/morluto/rea/issues/1449), [#1576](https://github.com/morluto/rea/issues/1576)) Thanks @CassieFox23, @dakjdakd, @N0zoM1z0.
|
||||
- JavaScript packages: resolve exact exports subpaths, preserve trailing-slash directory requests and exact CommonJS keys, and respect shadowed `require` bindings. ([#1424](https://github.com/morluto/rea/pull/1424), [#1615](https://github.com/morluto/rea/pull/1615), [5485483](https://github.com/morluto/rea/commit/5485483f7da2f6cb66441bb784aaf2a1a5f8731a), [8b2f851](https://github.com/morluto/rea/commit/8b2f8514c50e469c3add8ce9ba7ebebef888b14f), [f998fcb](https://github.com/morluto/rea/commit/f998fcb1cca9994c3589fb70d6e0127fc215f76f), [1e51c73](https://github.com/morluto/rea/commit/1e51c732bbf08191688d6526bf58ab2650b601e3), [#1365](https://github.com/morluto/rea/issues/1365)) Thanks @akshitkhokhani, @AWUQAQ, @immanuwell, @rudycelekli.
|
||||
- Large JavaScript inputs: handle oversized inline source maps, deep returns and reassignment loops; avoid argument expansion limits and yield during large analysis passes. ([#1386](https://github.com/morluto/rea/pull/1386), [#1368](https://github.com/morluto/rea/pull/1368), [#1390](https://github.com/morluto/rea/pull/1390), [#1539](https://github.com/morluto/rea/pull/1539), [#1525](https://github.com/morluto/rea/pull/1525), [#1542](https://github.com/morluto/rea/pull/1542), [#1545](https://github.com/morluto/rea/pull/1545), [#1557](https://github.com/morluto/rea/pull/1557), [#1380](https://github.com/morluto/rea/issues/1380), [#1401](https://github.com/morluto/rea/issues/1401), [#1538](https://github.com/morluto/rea/issues/1538), [#1462](https://github.com/morluto/rea/issues/1462)) Thanks @cold-atom, @KadirJeager, @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @rudycelekli.
|
||||
- JavaScript receivers: expand escaped methods once, index returned-object targets, and avoid replaying earlier writes or duplicating retained graph allocations. ([#1517](https://github.com/morluto/rea/pull/1517), [#1537](https://github.com/morluto/rea/pull/1537), [#1444](https://github.com/morluto/rea/pull/1444), [#1601](https://github.com/morluto/rea/pull/1601), [#1495](https://github.com/morluto/rea/issues/1495)) Thanks @CassieFox23, @geekwolf, @kaoru0822-kitauji, @N0zoM1z0, @zeynelacikgoez.
|
||||
- Browser captures: retain WebSocket URLs, empty connections and frame order; compare only common known payload coverage and preserve incomplete WebMCP replay coverage. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988), [aa257ea](https://github.com/morluto/rea/commit/aa257ea4b16f2d2a35c12df63359d650b48b1154), [e52c0a7](https://github.com/morluto/rea/commit/e52c0a72c59c740315e89149881a27ce4d89a11c))
|
||||
- Browser scenarios: seed storage in the correct partition, normalize loopback attach identities, retain declared-secret redaction and require verified lineage before Windows Electron signals. ([#1604](https://github.com/morluto/rea/pull/1604), [#1590](https://github.com/morluto/rea/pull/1590), [#1588](https://github.com/morluto/rea/pull/1588), [8b4c3de](https://github.com/morluto/rea/commit/8b4c3de3b3a3b580adec266f94f29ccedf1da228), [#1497](https://github.com/morluto/rea/issues/1497), [#1591](https://github.com/morluto/rea/issues/1591), [#1589](https://github.com/morluto/rea/issues/1589)) Thanks @Adolanium, @akushonkamen, @dakjdakd.
|
||||
- Added Termux Android browser-scenario support. ([#1655](https://github.com/morluto/rea/pull/1655)) Thanks @tyloryang.
|
||||
- Web inputs: accept BOM-prefixed HAR, preserve the first duplicate DOM attribute and resolve SVG/XLink URLs in HTML and XHTML. ([#1509](https://github.com/morluto/rea/pull/1509), [#1516](https://github.com/morluto/rea/pull/1516), [#1602](https://github.com/morluto/rea/pull/1602), [#1683](https://github.com/morluto/rea/pull/1683), [#1496](https://github.com/morluto/rea/issues/1496), [#1586](https://github.com/morluto/rea/issues/1586)) Thanks @CassieFox23, @kaoru0822-kitauji, @sathariels, @taljeri.
|
||||
- Screenshot comparisons: validate PNG chunks, distinguish corruption from unsupported encodings and bound comparison memory. ([4e4b15b](https://github.com/morluto/rea/commit/4e4b15b8250b29e176d4191c9610a3931b693474), [b3e977e](https://github.com/morluto/rea/commit/b3e977e6d073ab1783cece8581279f87fc360411)) Thanks @rudycelekli.
|
||||
- Archives: preserve case-distinct names, report collisions at their diverging segment, detect nested formats from bytes and reject Windows alternate-stream extraction paths. ([#1480](https://github.com/morluto/rea/pull/1480), [#1505](https://github.com/morluto/rea/pull/1505), [#1553](https://github.com/morluto/rea/pull/1553), [573598c](https://github.com/morluto/rea/commit/573598c6324698cc48ceafd24ed3864670e08d00), [#1470](https://github.com/morluto/rea/issues/1470), [#1491](https://github.com/morluto/rea/issues/1491)) Thanks @CassieFox23, @kaoru0822-kitauji.
|
||||
- Cleanup: retain failed readers, ASAR handles, output trees and provider workers for retry without discarding completed output or partial observations. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [de0d341](https://github.com/morluto/rea/commit/de0d341524dc68250a45b41eb7e49fcecc1e32a9), [3f8c51e](https://github.com/morluto/rea/commit/3f8c51e9b506d88fc41aded7b1a22b9a09037c26), [d1235fa](https://github.com/morluto/rea/commit/d1235fadc901e1fb9ab3149d53063306c8545414), [026126e](https://github.com/morluto/rea/commit/026126eafe741e5a8b857433587fd615cdd44c19))
|
||||
- Providers: canonicalize IDA workspace roots before identity checks, respect IDA cancellation, serialize JEB acquisition and correct Ghidra no-return/imported-symbol recovery. ([#1481](https://github.com/morluto/rea/pull/1481), [#1419](https://github.com/morluto/rea/pull/1419), [5f30ea6](https://github.com/morluto/rea/commit/5f30ea6ff2dd89aab7af6da6e8dfa5e508ed459b), [f88c990](https://github.com/morluto/rea/commit/f88c99021b88ce67e513025901f57478d8107d0c), [49e6f00](https://github.com/morluto/rea/commit/49e6f00ed08b225abf9ddb7c762553607fdab115)) Thanks @AS9929, @JonahSimon, @Y0UW1SH.
|
||||
- Binary metadata: respect PE/.NET section extents, bound binary-plist expansion, scope keyed-archive traversal and decode app `Info.plist` files without `plutil`. ([#1678](https://github.com/morluto/rea/pull/1678), [#1515](https://github.com/morluto/rea/pull/1515), [#1378](https://github.com/morluto/rea/pull/1378), [#1476](https://github.com/morluto/rea/pull/1476), [975dff2](https://github.com/morluto/rea/commit/975dff2e25b14b634e14dd5e0db1537d0ca9815b), [76296c5](https://github.com/morluto/rea/commit/76296c5a3868ba3b54be6ed3896719283d6de5f8)) Thanks @jayhemnani9910, @kaoru0822-kitauji, @rudycelekli.
|
||||
- Windows runtimes: preserve drive-root paths when creating private provider directories. ([ee46323](https://github.com/morluto/rea/commit/ee46323b493994902b04824a7dc8aae2a8117903))
|
||||
- Compatibility: load Windows native controls under Node-API hosts with renamed executables, preserve dylib directory requirements, support Wakaru 1.x and reject directory-valued Hopper launchers. ([#1531](https://github.com/morluto/rea/pull/1531), [#1623](https://github.com/morluto/rea/pull/1623), [#1540](https://github.com/morluto/rea/pull/1540), [#1426](https://github.com/morluto/rea/pull/1426)) Thanks @evenafterallisaid, @Fazmin, @pionxzh, @xeropresence.
|
||||
- Reference imports: normalize internal symlink lookup consistently, preserve exact paths and exclusion provenance, report root access failures and reject FIFO replacements without blocking. ([#1535](https://github.com/morluto/rea/pull/1535), [2ca3ad2](https://github.com/morluto/rea/commit/2ca3ad238608adfca80a7ff1aaadea6e59a58225), [6636045](https://github.com/morluto/rea/commit/6636045c9c5b472bba51afee9d434394861e93a4), [3967cfa](https://github.com/morluto/rea/commit/3967cfa43e786c65a57e89c265343983d19ef9d0), [f1c58cc](https://github.com/morluto/rea/commit/f1c58cc53fbd83b968e11b3e2eccebca51299c54)) Thanks @anujsingh-cse.
|
||||
- Setup: honor active configuration and skill roots, preserve configured MCP environments, validate Hermes/OMP profiles before writing, recognize Omarchy and retain recovery backups. ([#1504](https://github.com/morluto/rea/pull/1504), [#1607](https://github.com/morluto/rea/pull/1607), [#1528](https://github.com/morluto/rea/pull/1528), [#1562](https://github.com/morluto/rea/pull/1562), [#1662](https://github.com/morluto/rea/pull/1662), [a011485](https://github.com/morluto/rea/commit/a01148553e2a37c72ae78405418ab648876a96a6), [#1606](https://github.com/morluto/rea/issues/1606), [#1521](https://github.com/morluto/rea/issues/1521), [#1561](https://github.com/morluto/rea/issues/1561)) Thanks @Adkid-Zephyr, @CassieFox23, @LuisTaffelli, @N0zoM1z0, @peter-research.
|
||||
- Process capture: retain descendant lineage after launcher exit, preserve identity uncertainty before signaling and release terminals after observation failures. ([80896a1](https://github.com/morluto/rea/commit/80896a1d64b216ef893907249cd4252f86215d49), [c916850](https://github.com/morluto/rea/commit/c9168508527302ac57bc86385ccf0d824697b5dc)) Thanks @Y0UW1SH.
|
||||
- CLI/MCP errors: accept JSON files for web comparisons, preserve native option help and union correction details, remove invalid aliases and validate doctor against its discovered output contract. ([#1397](https://github.com/morluto/rea/pull/1397), [#1395](https://github.com/morluto/rea/pull/1395), [#1425](https://github.com/morluto/rea/pull/1425), [#1461](https://github.com/morluto/rea/pull/1461), [8b4c96e](https://github.com/morluto/rea/commit/8b4c96e6c6b3ed6806e4cd72b556b5db6114304a), [#1421](https://github.com/morluto/rea/issues/1421), [#1454](https://github.com/morluto/rea/issues/1454)) Thanks @Fazmin, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- WebSocket results: use `network.websocket_connections` instead of `network.websocket_events`. Connections contain `request_id`, `url` and `events`; allow `payload_bytes: null` when size is unknown. No legacy alias remains. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988))
|
||||
- Direct resource APIs: pass a caller-owned `ArtifactResourceScope` to inventory/extraction, JavaScript analysis, reference imports and web exports; retain it through cleanup/shutdown. For `readJavaScriptArtifactFiles`, scope is argument three and abort signal is argument four. Keep each root inventory source’s single cleanup owner. Use `OwnedFileHandle` owners, injected ASAR `fd`/`read` handles instead of `createReadStream`, VCS `Result` values, and `WebScriptExportError`’s options object. CLI/MCP inputs for the scoped operations are unchanged. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [53e7fad](https://github.com/morluto/rea/commit/53e7fadbd078422ba3d55b5499a70d237c5f4908), [684199b](https://github.com/morluto/rea/commit/684199ba0be7822266d57ddb8a40e95f087fdc42), [be75ed5](https://github.com/morluto/rea/commit/be75ed5e4b4bfb146af497f8fa5d172a1ea19462), [be6272f](https://github.com/morluto/rea/commit/be6272f26115bd8e94f2ab0df4cf092004587a18), [e0e49e9](https://github.com/morluto/rea/commit/e0e49e9dd53d9d1f1d2330b23f7f880dc1c81774), [95a77b6](https://github.com/morluto/rea/commit/95a77b6e25420b3879bd0e32bdbb14d452b38776))
|
||||
- Saved reference graphs: exclusions require their matching `pattern` and distinguish `project-ignored` from `default-ignored`. Re-import graphs missing that provenance. ([6585293](https://github.com/morluto/rea/commit/6585293c562853b95c3a70df02c12ad8cc17afc4))
|
||||
- Terminal captures: keep initial and resized buffers within 1,000,000 cells using `columns × (rows + scrollback)`; oversized scenarios are rejected before launch. ([f9ea9a9](https://github.com/morluto/rea/commit/f9ea9a9268c854d31048da9f8a50cfa37703fe5a))
|
||||
- Optional agent-verification reports: accept schema version 4, nullable `expectedFirstTool` and `targetClarificationPassed`. Production CLI/MCP contracts are unchanged. ([#1613](https://github.com/morluto/rea/pull/1613))
|
||||
- Capture comparisons: recapture both sides with matching `literal-replacements-v1` semantics when custom normalization patterns are used. Legacy unversioned pattern captures remain readable but cannot be compared with new pattern captures; captures without patterns are unaffected. ([#1456](https://github.com/morluto/rea/pull/1456), [2b6f14f](https://github.com/morluto/rea/commit/2b6f14f528297cf12c776c960af0859f5cfbefbf)) Thanks @Madhavan006, @N0zoM1z0.
|
||||
- Artifact graph edges: read child artifact IDs and paths from the occurrence selected by `occurrence_id`, not edge fields `child_artifact_id`/`logical_path`. Earlier inventory Evidence no longer validates against the new commitment. ([#1493](https://github.com/morluto/rea/pull/1493)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic graph schema: nodes and relations no longer include `identifier_strategy`, and relation IDs no longer hash it. Earlier Evidence no longer validates against this schema. ([#1474](https://github.com/morluto/rea/pull/1474)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic identities: select IDs from the parent graph; read exact event names from `event_name`, endpoints from `endpoint` and literal values from `properties.value`. New analyses use digest-based IDs for long/reserved event and endpoint names and `value-sha256` literal role keys, with shorter labels; string literal labels now read `string literal`. Previously stored graphs retain their original IDs and remain valid workflow inputs. ([#1427](https://github.com/morluto/rea/pull/1427), [#1423](https://github.com/morluto/rea/pull/1423), [#1407](https://github.com/morluto/rea/pull/1407)) Thanks @N0zoM1z0.
|
||||
|
||||
### Contributors
|
||||
|
||||
- Thanks also to @BaaRai, @commit-regrets, @coygeek, @Franky100-pig, @inii-sayaa, @jopemachine, @JosephDemarest, @keemsisi, @Totopo27 for contributions included in this release.
|
||||
|
||||
Full contribution record: [336 commits and 163 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json).
|
||||
|
||||
## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09)
|
||||
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
|
||||
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
|
||||
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
|
||||
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
|
||||
* **javascript:** Semantic graph facts now store evidence context IDs and locations. Graph and semantic trace results require an inline evidence_contexts table.
|
||||
* **contracts:** source-map URLs now live in target observations selected by declared_url_sha256, and doctor skill identity no longer emits installed_catalog_digest. Use the documented observation join and identity.skill.state instead.
|
||||
* **javascript:** remove the never-incremented truncated_scopes statistic from JavaScript application analysis results and schemas.
|
||||
* **native:** Objective-C attribute tokens now contain only name and value. Properties report nullable atomicity and readonly facts; class flags, ivar counts, and method requirement flags are nullable when unobserved.
|
||||
|
||||
### Features
|
||||
|
||||
* **setup:** register OMP during setup ([#1301](https://github.com/morluto/rea/pull/1301)) ([8b4c359](https://github.com/morluto/rea/commit/8b4c3591358ee59297d8298ede1b63d08459d817))
|
||||
* **javascript:** record Electron integrity mismatches ([7915ee5](https://github.com/morluto/rea/commit/7915ee51781878ed67e094259924b6e6f3093b3f))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **inspector:** retain observations after capture failures ([dea3498](https://github.com/morluto/rea/commit/dea3498b2b8d919606eefc4fd62f6ebcbbe5ed30))
|
||||
* **artifacts:** bind ASAR interpretation to owned snapshots ([d79317c](https://github.com/morluto/rea/commit/d79317cda0bbe8240eb7e25b0a5de90dcffca5fa))
|
||||
* **artifacts:** infer legacy Mach-O simulator platforms ([5aa2002](https://github.com/morluto/rea/commit/5aa200285ab066e48b37b716495936e6e2a963fd))
|
||||
* **artifacts:** preserve archive facts and Unicode path identity ([5751f5c](https://github.com/morluto/rea/commit/5751f5caac86c770123ae739a4e4101a72b0989c))
|
||||
* **artifacts:** reject incomplete inventoried extraction ([5e8fb53](https://github.com/morluto/rea/commit/5e8fb531e255a4a04fc053c54096f66d5c84cde3))
|
||||
* **artifacts:** retain DMG ownership through uncertain attachment ([a27794c](https://github.com/morluto/rea/commit/a27794c246b3f11d7f92ae7f6d276718190ab731))
|
||||
* **artifacts:** validate only active extraction entries ([cfb12e4](https://github.com/morluto/rea/commit/cfb12e47dc241f057be93a00892f7d7361d6119d))
|
||||
* **browser:** follow only HTTP redirect statuses for source maps ([82bbe74](https://github.com/morluto/rea/commit/82bbe743336244d398e67512659ea6943d43a4a4))
|
||||
* **browser:** preserve payload shapes and WebMCP declarations ([4951a89](https://github.com/morluto/rea/commit/4951a897c53bb6882300c5d7a9660a3f43e374f5))
|
||||
* **browser:** release attached viewport overrides without resizing the external page ([f6cfbfc](https://github.com/morluto/rea/commit/f6cfbfcc554ff24173dd7ad5333e8e26ab9c962d))
|
||||
* **browser:** retain attached scenario emulation until cleanup ([a0523b3](https://github.com/morluto/rea/commit/a0523b375eb3147f1d2d1e7eb10a2f4546f5d0fc))
|
||||
* **browser:** retain malformed redirect diagnostics ([617cd84](https://github.com/morluto/rea/commit/617cd84d27d100e7cc3cd4b5721dea4a996c9775))
|
||||
* **browser:** serialize attached scenarios through cleanup settlement ([cfce91d](https://github.com/morluto/rea/commit/cfce91d418fbcde4a9ef283a61549279751854e7))
|
||||
* **ci:** scope verification and reuse native runners ([#1344](https://github.com/morluto/rea/issues/1344)) ([d5cb8ae](https://github.com/morluto/rea/commit/d5cb8aee063065a48233458e04771635167825c8))
|
||||
* **cli:** avoid duplicate primary cleanup diagnostics ([37fcea7](https://github.com/morluto/rea/commit/37fcea78cecbeeadd869e9a3d77dd133ed264a0a))
|
||||
* **cli:** classify JSON input permission denials ([08daf05](https://github.com/morluto/rea/commit/08daf056cc8e4d092f9592574d09d5182457b7bc))
|
||||
* **cli:** classify oversized JSON inputs ([f7cf4c8](https://github.com/morluto/rea/commit/f7cf4c890d4b9c70b1f56e6c627b44a59e58fd73))
|
||||
* **cli:** stream JSON inputs beyond the whole-document string limit ([#1360](https://github.com/morluto/rea/issues/1360)) ([6ce3288](https://github.com/morluto/rea/commit/6ce3288f3bd65a886e47905455ed5c9b661015e6)), closes [#1331](https://github.com/morluto/rea/issues/1331)
|
||||
* **cli:** stream JSON results at the shared command boundary ([810107a](https://github.com/morluto/rea/commit/810107a2ad15cc52afd6662aec24d231d8c77225))
|
||||
* **contracts:** advertise NUL with a portable hex escape ([3f59a84](https://github.com/morluto/rea/commit/3f59a84bd1039763bb349894542601ed78be2033)), closes [#1329](https://github.com/morluto/rea/issues/1329)
|
||||
* **contracts:** require observed provider output metadata ([a4ba8ae](https://github.com/morluto/rea/commit/a4ba8aecd507637b6261026afc17f79516840d50))
|
||||
* **electron:** preserve unrepresentable numeric option evidence ([41a228c](https://github.com/morluto/rea/commit/41a228c1b01a89127858b1706879ede08a81deaf))
|
||||
* **evidence:** order canonical projections independently of locale ([614fb76](https://github.com/morluto/rea/commit/614fb7681ee9e4eba7219c1ee9e90d6a85d78095))
|
||||
* **files:** reject nonregular capture and JSON inputs ([5f16124](https://github.com/morluto/rea/commit/5f16124a31fa4a82bd03037506809d6be959d768))
|
||||
* **filesystem:** reject unstable artifact targets ([7829b1a](https://github.com/morluto/rea/commit/7829b1aeef8f43f186be2d2adbd6c67541306a7a))
|
||||
* **ghidra:** reject a native decompiler built for another host ([#1280](https://github.com/morluto/rea/issues/1280)) ([3728c82](https://github.com/morluto/rea/commit/3728c8267b4fb6da78cbae42a2a6cbddff80ea84)), closes [#1192](https://github.com/morluto/rea/issues/1192)
|
||||
* **ghidra:** retain process ownership after failed launch rollback ([07675f7](https://github.com/morluto/rea/commit/07675f78c66ce9e8706e808d417265e18ba9f3a5))
|
||||
* **ghidra:** verify dense jump tables on an optimized fixture ([#1374](https://github.com/morluto/rea/issues/1374)) ([73a664b](https://github.com/morluto/rea/commit/73a664bc78ee618045a2733b8714821281f77403))
|
||||
* **hopper:** allow restart after verified owned-provider shutdown ([1c12349](https://github.com/morluto/rea/commit/1c123493a21cc579ba7532c40e7ef30df45db3e8))
|
||||
* **hopper:** probe interior navigation from multi-byte instructions ([#1285](https://github.com/morluto/rea/issues/1285)) ([bf934d5](https://github.com/morluto/rea/commit/bf934d5592e707ba5e7769043de3a430ed8d4923))
|
||||
* **ida:** compare admitted document paths consistently ([890bb66](https://github.com/morluto/rea/commit/890bb66b35d5f9f8ee54ff0049bf7dc7b82b2480))
|
||||
* **javascript:** avoid positive-zero evidence for signed zero ([9055bd6](https://github.com/morluto/rea/commit/9055bd695b75f72fb1b8a538d1485d30c0b0ee0b))
|
||||
* **javascript:** decline source-map UNC references with an unusable host ([#1372](https://github.com/morluto/rea/issues/1372)) ([f504c2a](https://github.com/morluto/rea/commit/f504c2a351027f17ac353fba2d7cfc3505ee17d6))
|
||||
* **javascript:** distinguish prefix-only builtins from packages ([2b4abbf](https://github.com/morluto/rea/commit/2b4abbf1ccb6dd6ad1248cf48e381df0842542c1))
|
||||
* **javascript:** honor module-sync package exports ([#1361](https://github.com/morluto/rea/issues/1361)) ([d0e6069](https://github.com/morluto/rea/commit/d0e606930acb701790c286340226b1e3e7060b4f))
|
||||
* **javascript:** parse declaration artifacts in ambient mode ([4896adc](https://github.com/morluto/rea/commit/4896adcd0f9316034e83129685ed3807ade6f3ea))
|
||||
* **javascript:** preserve agreeing module provenance ([db171be](https://github.com/morluto/rea/commit/db171be6e570fd1261a30d711e4da2d17de85a93))
|
||||
* **javascript:** preserve capture-time guards and ordering ([001851f](https://github.com/morluto/rea/commit/001851fde6f972e6ebd01fddccdff4c645a85f63))
|
||||
* **javascript:** preserve semantic evidence through satisfies ([e995b19](https://github.com/morluto/rea/commit/e995b19f834ba78faa294f5959fd3ad8a97290ca))
|
||||
* **javascript:** preserve uncertainty after references escape to calls ([#1292](https://github.com/morluto/rea/issues/1292)) ([355a292](https://github.com/morluto/rea/commit/355a292198bf58245c67ae315fdb4f77e62769cc))
|
||||
* **javascript:** reject invalid package exports configurations ([7a4f268](https://github.com/morluto/rea/commit/7a4f2685d41b134507331c322f0f5da8e1f6c3f4))
|
||||
* **javascript:** resolve exported callables through lexical bindings ([#1351](https://github.com/morluto/rea/issues/1351)) ([a9f5dad](https://github.com/morluto/rea/commit/a9f5dad5a8f196c88f6305d8d958b355cc0b52c2))
|
||||
* **javascript:** resolve query-only HTML references to their document ([55a46f7](https://github.com/morluto/rea/commit/55a46f7c7176075c9edda4fe61ed995f2ceadf6a))
|
||||
* **javascript:** retain array identity after length mutations ([4bcea1a](https://github.com/morluto/rea/commit/4bcea1aa76e3e7ca665197181739c5e27a62c40a))
|
||||
* **javascript:** retain conditional mutation ordering uncertainty ([b8a8190](https://github.com/morluto/rea/commit/b8a8190b2f30e377eee2d69d1ba31ea454fa70b9))
|
||||
* **javascript:** retain distinct module provenance path components ([2bd7de6](https://github.com/morluto/rea/commit/2bd7de6f8ca62fc1d48334cfa7fdced461b7b153))
|
||||
* **javascript:** retain unresolved value frontiers in provenance queries ([3e19875](https://github.com/morluto/rea/commit/3e19875c7fa9c61cff655bbe6046ddc4ab558533))
|
||||
* **javascript:** store source map URLs once ([ed8fcae](https://github.com/morluto/rea/commit/ed8fcae94b699b62b44162f880faa077a36c455d))
|
||||
* **json:** preserve runtime limits across JSON file readers ([#1350](https://github.com/morluto/rea/issues/1350)) ([9c4a68b](https://github.com/morluto/rea/commit/9c4a68bc19a0f7d39557e3cc039f5877f6550f63))
|
||||
* **managed:** bind graph facts to their source Evidence ([cc7b5af](https://github.com/morluto/rea/commit/cc7b5af0777fc68ebba10ec061244729ebf2357d))
|
||||
* **mcp:** admit composed analysis through result bookkeeping ([e11f4d3](https://github.com/morluto/rea/commit/e11f4d371abaa8cb65ebf7b94fa216329fd78d88))
|
||||
* **mcp:** advertise object roots for union input schemas ([#1373](https://github.com/morluto/rea/issues/1373)) ([39dabd0](https://github.com/morluto/rea/commit/39dabd0608ed9617d60f16ece05993089f24ff38))
|
||||
* **mcp:** advertise primitive semantic literal seeds ([f62b449](https://github.com/morluto/rea/commit/f62b44971deee0e99df6fcf32758c8e5592400ed))
|
||||
* **mcp:** preserve JavaScript workflow identity validation ([#1366](https://github.com/morluto/rea/issues/1366)) ([53ed942](https://github.com/morluto/rea/commit/53ed942a85f51ff9f0e4f01cbb53359df476071d)), closes [#1364](https://github.com/morluto/rea/issues/1364) [#1065](https://github.com/morluto/rea/issues/1065)
|
||||
* **mcp:** retry failed shutdown resources and retain failure reasons ([e64678e](https://github.com/morluto/rea/commit/e64678e8f9768dd819d8f5f12cc844a6660ff2c5))
|
||||
* **native:** distinguish symbol definitions from metadata ([70bad53](https://github.com/morluto/rea/commit/70bad53847b0e14489acee73d204ff645e1adfcd))
|
||||
* **native:** preserve unknown Objective-C metadata ([b112d44](https://github.com/morluto/rea/commit/b112d44a3961939881779c81a57e7a3f9d1372f7))
|
||||
* **native:** refuse to list architectures of non-Mach-O targets ([#1325](https://github.com/morluto/rea/issues/1325)) ([1b73b5a](https://github.com/morluto/rea/commit/1b73b5ab4bc75bd424e643de8e49f142ffa2b336))
|
||||
* **native:** retain LLDB cleanup ownership through client close ([7ad353f](https://github.com/morluto/rea/commit/7ad353fa8b3f07e14face80336678d23cc7c6507))
|
||||
* **process:** keep unobserved file contents unknown ([#1358](https://github.com/morluto/rea/issues/1358)) ([f91f279](https://github.com/morluto/rea/commit/f91f2798b4dc6adb61b3ffe28a24a5d55adbd6ff))
|
||||
* **process:** retain per-file digest open failures ([415145d](https://github.com/morluto/rea/commit/415145d04f09a15cbfb7dda8ea9dc363a77f9bf0))
|
||||
* **reference:** report malformed UTF-8 before import parsing ([f166a2a](https://github.com/morluto/rea/commit/f166a2ae15b072216173a51a8f5cad930a858366))
|
||||
* report malformed application evidence as input errors ([218c131](https://github.com/morluto/rea/commit/218c13160ec0ed7116d6633fc76559ec39dae4d7))
|
||||
* retain completed observations when later analysis steps fail ([35f121f](https://github.com/morluto/rea/commit/35f121f4acac821e848c485fb82a2c42f51f6ea1))
|
||||
* **setup:** install the skill for Claude Code ([b4a1919](https://github.com/morluto/rea/commit/b4a19193fe3baf9075f8ac419517021ea0c71ea3))
|
||||
* **setup:** recognize supported Nobara hosts ([f0dc233](https://github.com/morluto/rea/commit/f0dc233d64de4c215c453c346e516b7622713e95))
|
||||
* **skill:** pin packaged commands to the shipping package version ([2793672](https://github.com/morluto/rea/commit/2793672ee2a575f97a8011c7fec5288098cdd5d1))
|
||||
* **skill:** pin packaged commands to the shipping package version ([#1346](https://github.com/morluto/rea/issues/1346)) ([f39e71c](https://github.com/morluto/rea/commit/f39e71ce1897bb5c704b33a8b0049567f11e3a13))
|
||||
* **snapshot:** preserve complete investigation revision histories ([#1354](https://github.com/morluto/rea/issues/1354)) ([1c0cc8a](https://github.com/morluto/rea/commit/1c0cc8a6753b5ac781b28ebb1832f3a6a1771a68))
|
||||
* **targets:** cancel resolution before provider admission ([f3c4a33](https://github.com/morluto/rea/commit/f3c4a332f7bd198bd67f0187bc2a85c7b31bb960))
|
||||
* **update:** preserve disabled client registrations ([abf5e24](https://github.com/morluto/rea/commit/abf5e243c4d2cae03404b6e1fb1ccc77be917b53))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **javascript:** index version relationships once per comparison ([#1347](https://github.com/morluto/rea/issues/1347)) ([b1e85ad](https://github.com/morluto/rea/commit/b1e85ad711743a594c572dbd5b0d5cb664e2a502))
|
||||
* **javascript:** validate the version change graph once ([#1370](https://github.com/morluto/rea/issues/1370)) ([037709a](https://github.com/morluto/rea/commit/037709a52c0824f4d2bbee90d8e2642b2d88d364))
|
||||
* **json:** encode streamed JSON iteratively ([#1322](https://github.com/morluto/rea/issues/1322)) ([56598be](https://github.com/morluto/rea/commit/56598beb4e82968a313deb9efdf4b98db5bd1f4f))
|
||||
|
||||
|
||||
### Code Refactoring
|
||||
|
||||
* use canonical implementations and types directly ([765e809](https://github.com/morluto/rea/commit/765e809e366eff21b4b9c224f1a813ad64245ea3))
|
||||
* **artifacts:** remove unused root classifier ([5291bca](https://github.com/morluto/rea/commit/5291bca108af781214572fac709f9808bb8aa4db))
|
||||
* **artifacts:** test inventory changes at materialization boundary ([750fb65](https://github.com/morluto/rea/commit/750fb6565052e9eaad40b946b6a284d0730d14f8))
|
||||
* **browser:** open scenario sessions without a factory wrapper ([d1983a4](https://github.com/morluto/rea/commit/d1983a45767bfcdc020652f48109a78a3f9fa590))
|
||||
* **catalog:** import CLI metadata from its owner ([8c96baa](https://github.com/morluto/rea/commit/8c96baab80dc258d31ca1632d761fffb11dbaaae))
|
||||
* consolidate boundary semantics and prune redundant tests ([392f1f3](https://github.com/morluto/rea/commit/392f1f310733f2c365da7bc57a1154cf10ec4ba0))
|
||||
* **javascript:** normalize erased expression syntax once ([b628bba](https://github.com/morluto/rea/commit/b628bba1fa65fd3b51363bf4b745f0afc57e1dba))
|
||||
* **javascript:** parse runtime evidence into one verified owner ([006b295](https://github.com/morluto/rea/commit/006b295f1de19fd5351832cf52d98a202a5cd5cd))
|
||||
* **javascript:** remove obsolete scope truncation statistic ([449d570](https://github.com/morluto/rea/commit/449d57075eec71b2f064af8877c50cbbde30a283))
|
||||
* **javascript:** share Babel child traversal ([0f10499](https://github.com/morluto/rea/commit/0f10499245ec2fab8823f116caf34450b3a7385b))
|
||||
* **javascript:** share semantic evidence at fact construction ([e3bbb9a](https://github.com/morluto/rea/commit/e3bbb9ac453da8d15f3b32511180451613710616))
|
||||
* **process:** remove duplicate observation and trace entry points ([070339a](https://github.com/morluto/rea/commit/070339af50a50a0f23e82db859e761c53277ee79))
|
||||
* **providers:** use canonical artifact and managed identities ([87f0051](https://github.com/morluto/rea/commit/87f0051496aff1b05ab24d6b63f79884f4ca9b8f))
|
||||
* **reference:** simplify malformed UTF-8 handling ([377df71](https://github.com/morluto/rea/commit/377df715ea5b2523cd6c1084fdc286eb8a056182))
|
||||
* remove obsolete source exclusions and identity wrapper ([44c92f8](https://github.com/morluto/rea/commit/44c92f88c51841735741811a68cc89fde14efa80))
|
||||
* remove single-use forwarding helpers ([4144e45](https://github.com/morluto/rea/commit/4144e454f385c372d29e301e6bb959092c606a6e))
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* keep license sections text-only ([e10f1f0](https://github.com/morluto/rea/commit/e10f1f0901fa56fb4443837cf9528b31a12e140e))
|
||||
* align translated README artwork and layout ([ccfbe87](https://github.com/morluto/rea/commit/ccfbe87e984e456e12c66d0d2fe6d5a559d09518))
|
||||
* add README artwork and refine closing sections ([4b8e744](https://github.com/morluto/rea/commit/4b8e744d90579e353371649047215c4e4be7a5f6))
|
||||
* add repository-local REA tool design skill ([7738c1f](https://github.com/morluto/rea/commit/7738c1f6a9ccf1ec9868ef832f154abe9ba5eebd))
|
||||
* celebrate 40,000 GitHub stars 🎉 ([3edb217](https://github.com/morluto/rea/commit/3edb2172def0e0bd9076f23148f0c41cd2f341a3))
|
||||
* celebrate 40,000 GitHub stars 🎉 ([5ecc98a](https://github.com/morluto/rea/commit/5ecc98a5eac4c10d33e9c418b9ffae61c68e3ef2))
|
||||
* consolidate agent guidance around ownership and boundaries ([ba44222](https://github.com/morluto/rea/commit/ba4422296dc409d49836f25642555e00463bfe3b))
|
||||
* **contracts:** explain canonical URL and skill metadata ownership ([e36afa1](https://github.com/morluto/rea/commit/e36afa112fd7b7ab5a925ca452de93b68f12a6ab))
|
||||
* correct MCP error and capture import guidance ([077d6c4](https://github.com/morluto/rea/commit/077d6c407b3ea7443a7277ae0e5531b3927dd742))
|
||||
* correct README setup and runtime guidance ([5b3715e](https://github.com/morluto/rea/commit/5b3715ef825a30150b02d3f1bb7d40dd89453018))
|
||||
* explain CI scope and native runner reuse ([#1348](https://github.com/morluto/rea/issues/1348)) ([6c9161d](https://github.com/morluto/rea/commit/6c9161d61745e242643f224ce49e7468143a15f3))
|
||||
* prune repeated contributor and tool design instructions ([5d79387](https://github.com/morluto/rea/commit/5d79387505ebae1ec8d800ee3ffd901920a9d8be))
|
||||
* remove retired architecture and migration artifacts ([7a0c800](https://github.com/morluto/rea/commit/7a0c80085128e6329ca2e4f54f3539f3245c57d1))
|
||||
|
||||
|
||||
### Tests
|
||||
|
||||
* share the client registration command to restore the jscpd gate ([#1317](https://github.com/morluto/rea/pull/1317)) ([5d16be2](https://github.com/morluto/rea/commit/5d16be2a7185db8b70a5fcbc5f5e1be63297f2c8))
|
||||
* **mcp:** remove arbitrary schema inventory thresholds ([24673aa](https://github.com/morluto/rea/commit/24673aaae497ca4ef0902544c0cd69817f5424ed))
|
||||
* **android:** allow provider readiness under load ([961c3c3](https://github.com/morluto/rea/commit/961c3c3de79ec44d608d011eb3fa3d6bd645cf6a))
|
||||
* **artifacts:** verify extraction races with real filesystem changes ([1ac8389](https://github.com/morluto/rea/commit/1ac8389c5c1f75faf8d7f2f3faf2f24febad5fea))
|
||||
* **browser:** avoid duplicating source-map listener setup ([9367dc1](https://github.com/morluto/rea/commit/9367dc13bd8d2732d36d5376b5de170c9f4af424))
|
||||
* **browser:** report redirect status cases independently ([5a61562](https://github.com/morluto/rea/commit/5a61562dc200120c7b8020854085c17b8203e95d))
|
||||
* **cli:** remove duplicate malformed JSON assertion ([5d32e42](https://github.com/morluto/rea/commit/5d32e4252941e93065cadd83b9f02aa6427544d8))
|
||||
* exercise source test selection through Git and Vitest ([25711ad](https://github.com/morluto/rea/commit/25711ad2590af136dfb301839dddd6d30971ba76))
|
||||
* **filesystem:** bound FIFO regressions at consumers ([0bfb6e0](https://github.com/morluto/rea/commit/0bfb6e00abae1e67a96e548d64954bb21861f621))
|
||||
* **ghidra:** allow rollback tests to reach spawned process ([d761d73](https://github.com/morluto/rea/commit/d761d7358b45fe0724b182a206241fd605f338b0))
|
||||
* **javascript:** share primitive expansion producer fixtures ([c867bcd](https://github.com/morluto/rea/commit/c867bcd0aadd63473865f9a328459e0db3c89565))
|
||||
* **javascript:** share semantic graph test facts ([ac356c4](https://github.com/morluto/rea/commit/ac356c46ad6a17c8864bc5d350984786db62d2f5))
|
||||
* **javascript:** verify distinct module origins through reconstruction ([96b2b24](https://github.com/morluto/rea/commit/96b2b245fdefc9c78f5dcf6ce054232b51f7c117))
|
||||
* **json:** remove assertions covered by CLI output consumers ([a959a83](https://github.com/morluto/rea/commit/a959a8309db79fbee6477efd7ee541615e625627))
|
||||
* **managed:** include shared dossier fixture in artifact inputs ([66a20a2](https://github.com/morluto/rea/commit/66a20a2597b5daebc0a292361aa62a78b7df6e7d))
|
||||
* **native:** retain trace diagnostics when observed hits differ ([048b798](https://github.com/morluto/rea/commit/048b79878c70c594789909082499dc675472b63a))
|
||||
* **package:** share validated native dossier fixtures ([ca90109](https://github.com/morluto/rea/commit/ca9010936a934a49fbc6cf69eb1d7fc2242b6da9))
|
||||
* **process:** publish capture readiness atomically ([74be526](https://github.com/morluto/rea/commit/74be52611e66f680ef1919d521fc3c304bc2326e))
|
||||
* **process:** report cleanup details on cancellation failures ([79d5b9a](https://github.com/morluto/rea/commit/79d5b9a826b9204c1ef59b0a5c5ec83e7a2f2bec))
|
||||
* **process:** use owned CLI and workspace fixtures ([4940a86](https://github.com/morluto/rea/commit/4940a867e2e56cbb974bfa321008a8f35db47d52))
|
||||
* share snapshot replay and managed graph fixtures ([d83cbd2](https://github.com/morluto/rea/commit/d83cbd2abf1bbf15beef990f5353457fdbaaeb8e))
|
||||
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
* enforce architecture guards with portable checkout paths ([ecc67ca](https://github.com/morluto/rea/commit/ecc67cab2d85cacda245890ffe84b80ac395d79e))
|
||||
|
||||
## [6.2.0](https://github.com/morluto/rea/compare/rea-agents-6.1.0...rea-agents-6.2.0) (2026-10-09)
|
||||
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
|
||||
|
||||
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
|
||||
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
|
||||
* **Cleanup and evaluation:** A one-shot analysis reports cleanup failure while preserving the completed observation or primary diagnostic. Handle `cleanup_incomplete` and retained observations/resources. Configured agent-evaluation fixtures require Evidence-backed correctness, not routing or answer wording alone. ([#1211](https://github.com/morluto/rea/pull/1211), [27a9cea](https://github.com/morluto/rea/commit/27a9ceade7232c1c9a24dc7d454ae060948183a1))
|
||||
* **JavaScript:** Return-field projections require explicit presence; export-shape changes carry both sides' presence and property inventory coverage. Semantic object operations retain complete `property_path` segments. Regenerate old analysis and distinguish unknown values from unknown presence. ([475a0f8](https://github.com/morluto/rea/commit/475a0f84dac63d710e032bc7f731c1a3bda624ec))
|
||||
* **Function comparisons:** Remove the unreachable `truncated` dimension state and `summary.truncated` counter; update enum and display consumers while retaining explicit `unknown` facts. ([5510b27](https://github.com/morluto/rea/commit/5510b27d7f863a3df26212fba4a59f8df0da3712))
|
||||
* **Historical sources:** Unreadable/unknown symlink targets require `target: null`; external targets retain the observed absolute target. Reimport graphs containing display placeholders such as `<outside-root>` and handle nullable targets. ([b7b4333](https://github.com/morluto/rea/commit/b7b433370310c3ffe33d5027d8393151d5e862da))
|
||||
|
||||
### Features
|
||||
|
||||
* **javascript:** report observed export property presence ([#1160](https://github.com/morluto/rea/issues/1160)) ([45e3edc](https://github.com/morluto/rea/commit/45e3edc17f48954c89d8db86f7313c479bd39344))
|
||||
* **mcp:** select views of retained analysis results ([#1180](https://github.com/morluto/rea/issues/1180)) ([db675f1](https://github.com/morluto/rea/commit/db675f1d52acda62b6f57b5a8773532f49b259e7))
|
||||
* **website:** add an Aegis APK code-generation showcase ([0270dd5](https://github.com/morluto/rea/commit/0270dd5739480acced62e045b5890ec0ebde2631))
|
||||
* **website:** add an Android showcase and clearer project prompts ([6cc817e](https://github.com/morluto/rea/commit/6cc817e8bd377b064b3568ef1eacb27eaefcb4c1))
|
||||
* **website:** add beginner examples and synchronized publishing ([0fec65d](https://github.com/morluto/rea/commit/0fec65d06b2f5431e704c472e53015d4b3cc967c))
|
||||
* **website:** add beginner examples and synchronized publishing ([#1203](https://github.com/morluto/rea/issues/1203)) ([0fec65d](https://github.com/morluto/rea/commit/0fec65d06b2f5431e704c472e53015d4b3cc967c))
|
||||
* **website:** add Discord link beside GitHub badge ([cc39fc5](https://github.com/morluto/rea/commit/cc39fc53b829a349ae199823472990a53014c507))
|
||||
* **website:** add homepage navigation and faster reading paths ([863674b](https://github.com/morluto/rea/commit/863674b2e92dd7c699768db82d521ccc88b7ebe4))
|
||||
* **website:** add homepage navigation and faster reading paths ([c94f06d](https://github.com/morluto/rea/commit/c94f06d98f895eb8b6c5ffa5e999ef1d4bb5eea2))
|
||||
* **website:** add homepage navigation and faster reading paths ([#1213](https://github.com/morluto/rea/issues/1213)) ([863674b](https://github.com/morluto/rea/commit/863674b2e92dd7c699768db82d521ccc88b7ebe4))
|
||||
* **website:** add search metadata and automatic sitemap ([99f257b](https://github.com/morluto/rea/commit/99f257bbf03c27327e5e0a232fefd3c792acd5bc))
|
||||
* **website:** add search metadata and automatic sitemap ([b8851cb](https://github.com/morluto/rea/commit/b8851cb977cfdd608798d32bf5877f645b960831)), closes [#1283](https://github.com/morluto/rea/issues/1283)
|
||||
* **website:** clarify reverse engineering and showcase goals ([e750c26](https://github.com/morluto/rea/commit/e750c262fbf37100369ab25bf8659daa1a2c4cc7))
|
||||
* **website:** explain REA through a first investigation ([5160d9b](https://github.com/morluto/rea/commit/5160d9b717d0207a357c754bdf72e215e59d19d1))
|
||||
* **website:** explain REA with Calculator and a playable dinosaur lab ([11b9f03](https://github.com/morluto/rea/commit/11b9f0346cf5413045b41ff552a024e11ba46ccc))
|
||||
* **website:** lead with the playable dinosaur example ([19c52f5](https://github.com/morluto/rea/commit/19c52f5e3f42fb754fc154bbd16552c5111df0c5))
|
||||
* **website:** show manual analysis beside a one-prompt workflow ([291bc6f](https://github.com/morluto/rea/commit/291bc6fb70f47d28d23440f03a8a7fa49eb9a391))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **dev:** accept formatter-ignored staged documentation without failing the commit hook; malformed JSON remains rejected. ([bd2db2f](https://github.com/morluto/rea/commit/bd2db2f0fa495fb08dea443cd16d7683d4b86e6a))
|
||||
|
||||
* **release:** publish reviewed release-PR merges from their exact reviewed source. ([c12bad7](https://github.com/morluto/rea/commit/c12bad7f139431fc0db7a8b7c9db1eab95f96dc2))
|
||||
|
||||
* **android:** support owned JADX stdio on Windows ([#1276](https://github.com/morluto/rea/issues/1276)) ([636b245](https://github.com/morluto/rea/commit/636b245f1badf2a7ecfa63c6e2378ac2f7673cc3)), closes [#1262](https://github.com/morluto/rea/issues/1262)
|
||||
* **android:** validate engine archive prerequisites ([#1145](https://github.com/morluto/rea/issues/1145)) ([e9ebfb9](https://github.com/morluto/rea/commit/e9ebfb94eff0699b1ea84bba000c4467fa1cb998))
|
||||
* **apple:** report unusable dylib-resolution roots as invalid input ([6a1e558](https://github.com/morluto/rea/commit/6a1e558db3ec2b69561e91ea817f1c2065dfc676))
|
||||
* **apple:** report unusable dylib-resolution roots as invalid input ([24a30fd](https://github.com/morluto/rea/commit/24a30fd91c144b7f2533e5f5de8c91268773519c))
|
||||
* **artifacts:** keep cancellation and the caller path in extraction refusals ([37025cc](https://github.com/morluto/rea/commit/37025cc09f12d49f46e61eedc3564c710089146b))
|
||||
* **artifacts:** keep cancellation and the caller path in extraction refusals ([eacf846](https://github.com/morluto/rea/commit/eacf8461dad6bab11a3d5438e6db001a10f33a9b))
|
||||
* **artifacts:** match lipo arm64e variant names to their FAT slices ([de5bfec](https://github.com/morluto/rea/commit/de5bfec79fe862816350b2deeb42a12c0734d855))
|
||||
* **artifacts:** match lipo arm64e variant names to their FAT slices ([0b32577](https://github.com/morluto/rea/commit/0b32577a90aef97e36adf2a7af4f3eccd5d82357)), closes [#1168](https://github.com/morluto/rea/issues/1168)
|
||||
* **artifacts:** name only host-available workflows for non-archive plists ([f4c99ab](https://github.com/morluto/rea/commit/f4c99ab8a53806dd57dc24b3d5a600905bcb4686))
|
||||
* **artifacts:** name only host-available workflows for non-archive plists ([2295855](https://github.com/morluto/rea/commit/22958558eafdfccf71a7d5bbbcf360574a6b2e3c))
|
||||
* **artifacts:** recheck failed DMG detaches before reporting cleanup failure ([#1155](https://github.com/morluto/rea/issues/1155)) ([58a91f0](https://github.com/morluto/rea/commit/58a91f03d72162849991caf537ab058ccc918594))
|
||||
* **artifacts:** refuse unsupported extraction formats before scanning ([8e74c9c](https://github.com/morluto/rea/commit/8e74c9c62fb44556b90f20bb6c12771ac05a18f2))
|
||||
* **artifacts:** report encrypted archive entries as unsupported extraction ([de43da7](https://github.com/morluto/rea/commit/de43da71ae982dd0f3d7b5ae49c914729db3a6ec))
|
||||
* **artifacts:** report non-archive keyed-archive selections as unsupported targets ([fef38e4](https://github.com/morluto/rea/commit/fef38e4d22ee859ec43fb591d53e8b9973c69958))
|
||||
* **artifacts:** report non-archive keyed-archive selections as unsupported targets ([8c5088c](https://github.com/morluto/rea/commit/8c5088c8cdf32a9f96f6c479c6e70a71fa360168))
|
||||
* **artifacts:** report wrong target kinds as unsupported targets ([b34bdcd](https://github.com/morluto/rea/commit/b34bdcdb2877be394c59c9cbf70269dc1841ff78))
|
||||
* **artifacts:** report wrong target kinds as unsupported targets ([637bcb5](https://github.com/morluto/rea/commit/637bcb5e4182b6c60e922b18688642d8e1008e90))
|
||||
* **browser:** classify export-web-scripts capture selection failures by cause ([ddaa183](https://github.com/morluto/rea/commit/ddaa18301ab9c67e7a6f7ce66076075e8a4d5017))
|
||||
* **browser:** classify export-web-scripts capture selection failures by cause ([8c5e34c](https://github.com/morluto/rea/commit/8c5e34cd379392ca40f651ca5eeb6a947b44aa33))
|
||||
* **browser:** report Windows HAR heap exhaustion ([#1147](https://github.com/morluto/rea/issues/1147)) ([1393507](https://github.com/morluto/rea/commit/13935077ba039f6e342f9c11ab5eea935a335ae6))
|
||||
* **browser:** retain scenario observations across failures ([614e0c2](https://github.com/morluto/rea/commit/614e0c2e74819bb5e3ef3d2e16ff31ee6cab76ae))
|
||||
* **browser:** retain scenario resources until cleanup succeeds ([88640f6](https://github.com/morluto/rea/commit/88640f60fe566edbe2320688df62c7ef46c4c61c))
|
||||
* **ci:** pass selected environment to Windows Ghidra fixture ([#1281](https://github.com/morluto/rea/issues/1281)) ([f085589](https://github.com/morluto/rea/commit/f085589957922eaf421db814399e88e415712050))
|
||||
* **cli:** advertise defaulted inputs as optional in --schema and help ([#1175](https://github.com/morluto/rea/issues/1175)) ([b4927a0](https://github.com/morluto/rea/commit/b4927a0a4fa7e8935b101d214b396bfa78208281)), closes [#1169](https://github.com/morluto/rea/issues/1169)
|
||||
* **cli:** cancel routed JavaScript analysis in rea analyze ([#1156](https://github.com/morluto/rea/issues/1156)) ([089bbf5](https://github.com/morluto/rea/commit/089bbf58f3561e3f465c544bbcf8c307287b1256))
|
||||
* **cli:** complete cleanup after relayed interrupts ([#1202](https://github.com/morluto/rea/issues/1202)) ([911152a](https://github.com/morluto/rea/commit/911152a04d3aeb0d361141a8d54959009b269e49))
|
||||
* **cli:** list the read failure cause for an unreadable JSON input file ([db0afce](https://github.com/morluto/rea/commit/db0afce5f344268271c59acee04e96f986c5619a))
|
||||
* **cli:** list the read failure cause for an unreadable JSON input file ([6020df5](https://github.com/morluto/rea/commit/6020df53d9cd44845cf3b6b416db2d7802925578))
|
||||
* **cli:** resolve xrefs symbol selectors after address validation ([#1209](https://github.com/morluto/rea/issues/1209)) ([7e9c911](https://github.com/morluto/rea/commit/7e9c91199ecd133d26a51cb33b6e63e1f942df7e))
|
||||
* **config:** name the rejected setting and its constraint in configuration errors ([c3acc34](https://github.com/morluto/rea/commit/c3acc34475ca22387ea674839acfa1891b0b9f66))
|
||||
* **config:** name the rejected setting and its constraint in configuration errors ([0a0244d](https://github.com/morluto/rea/commit/0a0244db4212bc1f59dec4b925fa90b3b473136a)), closes [#1184](https://github.com/morluto/rea/issues/1184)
|
||||
* **doctor:** find the Homebrew Hopper cask without writing Homebrew caches ([98b07d0](https://github.com/morluto/rea/commit/98b07d07ddd20bea2c3bdf8d55ac2776a263bcf4))
|
||||
* **doctor:** find the Homebrew Hopper cask without writing Homebrew caches ([a503376](https://github.com/morluto/rea/commit/a503376b2f9d9ba14d1602e58e0e93059e9d9181)), closes [#1165](https://github.com/morluto/rea/issues/1165)
|
||||
* **domain:** report target read-permission denials as access_denied ([f36c10f](https://github.com/morluto/rea/commit/f36c10f9e8dc45a298d668e608f8e945bde487cf))
|
||||
* **domain:** report target read-permission denials as access_denied ([d0ff57a](https://github.com/morluto/rea/commit/d0ff57a45c8da8938e7ae16d00bfd5ec63310af3)), closes [#1188](https://github.com/morluto/rea/issues/1188)
|
||||
* **electron:** resolve effective preload after option overrides ([#1286](https://github.com/morluto/rea/issues/1286)) ([411498d](https://github.com/morluto/rea/commit/411498d45f860eba2fbc9d7051af20237b7a4cb1))
|
||||
* emit v-mode-compatible patterns for tool schemas ([#1220](https://github.com/morluto/rea/issues/1220)) ([f0da05f](https://github.com/morluto/rea/commit/f0da05f67fff1411cf2f7a48a752ee7b348c71ff))
|
||||
* **errors:** report caller-selected evidence path failures as invalid input ([d4a598e](https://github.com/morluto/rea/commit/d4a598e93649607dd293a9c81bbf79c211bc8dd4))
|
||||
* **errors:** report caller-selected evidence path failures as invalid input ([db177c9](https://github.com/morluto/rea/commit/db177c91b83adb0c3edc4f9f6f06e31ac7523d53))
|
||||
* **evidence:** cancel exports before atomic publication ([16624f1](https://github.com/morluto/rea/commit/16624f1324e156d14025b2c346596c9781c07488))
|
||||
* **ghidra:** classify modern Swift manglings in swift type analysis ([#1208](https://github.com/morluto/rea/issues/1208)) ([6935dc7](https://github.com/morluto/rea/commit/6935dc7fe1258b340bb9cdfb8cecae66ef7bc0d4))
|
||||
* **ghidra:** do not run Java from a relative JAVA_HOME in doctor and setup ([04a516b](https://github.com/morluto/rea/commit/04a516b1e369f7e57e08d0653cc08292ac497e07))
|
||||
* **ghidra:** do not run Java from a relative JAVA_HOME in doctor and setup ([c3f1698](https://github.com/morluto/rea/commit/c3f169823f690371ecfdd2d1b3b07a5749438cf1))
|
||||
* **ghidra:** keep and name a runtime directory that close could not remove ([5b769d3](https://github.com/morluto/rea/commit/5b769d3a781c645ed8885e4fabf01a00f0bbc89b))
|
||||
* **ghidra:** keep and name a runtime directory that close could not remove ([b79d379](https://github.com/morluto/rea/commit/b79d379c25bcc8ed61571c41a94b912b5704a067)), closes [#1190](https://github.com/morluto/rea/issues/1190)
|
||||
* **ghidra:** keep the NativeAOT target refusal's own recovery advice ([f198993](https://github.com/morluto/rea/commit/f198993f418e6aaa9755e7e6413abcb27c6cd684))
|
||||
* **ghidra:** preserve decoded return flow evidence ([b189c3c](https://github.com/morluto/rea/commit/b189c3ceb175dd9fa089eada9e9ca54e76307cb3))
|
||||
* **ghidra:** preserve decoded return flow evidence ([bcc8a77](https://github.com/morluto/rea/commit/bcc8a773e2a714e4899755b786a4fef1539bbace))
|
||||
* **ghidra:** reject relative installation and Java paths in doctor and setup ([8998438](https://github.com/morluto/rea/commit/8998438c23aa19e68f4ffa7574c5d341aa419165))
|
||||
* **ghidra:** reject relative installation and Java paths in doctor and setup ([5356856](https://github.com/morluto/rea/commit/53568568bf32b36274ba3a532f1e2cfb2daed271)), closes [#1183](https://github.com/morluto/rea/issues/1183)
|
||||
* **ghidra:** report NativeAOT host and target refusals as unsupported, not failures ([cfb49eb](https://github.com/morluto/rea/commit/cfb49eb8a88fc51f460907024c51e05a2f272502))
|
||||
* **ghidra:** report NativeAOT host and target refusals as unsupported, not failures ([42c276a](https://github.com/morluto/rea/commit/42c276af1ec02473a7b6700a31292aae9e53d53e)), closes [#1185](https://github.com/morluto/rea/issues/1185)
|
||||
* **ghidra:** resolve secondary symbols at function entries ([#1195](https://github.com/morluto/rea/issues/1195)) ([6fbdbd6](https://github.com/morluto/rea/commit/6fbdbd682038d012b04f0fe92c470cf1789fb101))
|
||||
* **ghidra:** stop dereferencing the removed error payload in address verification ([b80bd2d](https://github.com/morluto/rea/commit/b80bd2db966731fd3cfddfff7e92704053dbe955))
|
||||
* **ghidra:** stop dereferencing the removed error payload in address verification ([59fbaa2](https://github.com/morluto/rea/commit/59fbaa2d33359216534407f99f49705ac04a3426))
|
||||
* **ghidra:** use dot-free private session paths ([c3fc6c3](https://github.com/morluto/rea/commit/c3fc6c3aedc202e8ade015c4039a63e9f91cac6d))
|
||||
* **hopper:** keep the OS error when the FAT64 source cannot be reopened ([9f5dc52](https://github.com/morluto/rea/commit/9f5dc52be8420fb5e874595d63312bf80d07a3a2))
|
||||
* **hopper:** reserve Linux demo application before launch ([#1158](https://github.com/morluto/rea/issues/1158)) ([33d5ae0](https://github.com/morluto/rea/commit/33d5ae0b52b8684634ecf14d4e9469968c69d462))
|
||||
* **javascript:** allow cancellation during semantic graph commitment ([#1218](https://github.com/morluto/rea/issues/1218)) ([9405bd6](https://github.com/morluto/rea/commit/9405bd660def9946afaa5fcc1d99921a816911bf))
|
||||
* **javascript:** classify application input selection failures by cause ([fe5291e](https://github.com/morluto/rea/commit/fe5291e9166977a6ebbfa6ffcf7850af57192b34))
|
||||
* **javascript:** classify application input selection failures by cause ([0a14bbb](https://github.com/morluto/rea/commit/0a14bbbf05e0e61c98e06326fe7136e52fa4fb3c))
|
||||
* **javascript:** keep Wakaru version stderr out of the refusal message ([6489d58](https://github.com/morluto/rea/commit/6489d58851d055fd7ce621000d960bb42a0e4ce0))
|
||||
* **javascript:** match source-map originals that name parent directories ([#1215](https://github.com/morluto/rea/issues/1215)) ([889fb37](https://github.com/morluto/rea/commit/889fb374df0f8986f49e9d61b0fefd536327f781))
|
||||
* **javascript:** omit self-referential static references with disclosure ([#1157](https://github.com/morluto/rea/issues/1157)) ([b620e2a](https://github.com/morluto/rea/commit/b620e2a2bdcf3683b494d137bc409813735d5572))
|
||||
* **javascript:** preserve alias and container uncertainty ([73e98b2](https://github.com/morluto/rea/commit/73e98b2516dd792ea7b510aeed6b71f384bea21d))
|
||||
* **javascript:** preserve graphs with async chunk self-references ([#1152](https://github.com/morluto/rea/issues/1152)) ([7220166](https://github.com/morluto/rea/commit/72201668ed88c2cc822a07c08cf35a1e5ba88f35))
|
||||
* **javascript:** preserve property paths and uncertain slot presence ([475a0f8](https://github.com/morluto/rea/commit/475a0f84dac63d710e032bc7f731c1a3bda624ec))
|
||||
* **javascript:** reject encoded separators joined by HTML URL whitespace ([2ddd945](https://github.com/morluto/rea/commit/2ddd9456cd405f081784a4b3cb02cb77cd90fbd6))
|
||||
* **javascript:** reject encoded separators joined by HTML URL whitespace ([fb57c86](https://github.com/morluto/rea/commit/fb57c8643fb8b4733cbac9090ef58fd13a3b5347))
|
||||
* **javascript:** report located sources with differing digests as modified ([#1216](https://github.com/morluto/rea/issues/1216)) ([01174c4](https://github.com/morluto/rea/commit/01174c4a81fea109ae712ea1a746a83996f928ac))
|
||||
* **javascript:** resolve HTML script URLs after URL whitespace and percent decoding ([d601b80](https://github.com/morluto/rea/commit/d601b8016eaeb2955cea4e4c0aca10d17e7d12a7))
|
||||
* **javascript:** resolve HTML script URLs after URL whitespace and percent decoding ([502528c](https://github.com/morluto/rea/commit/502528c58df54c7b34eefdb5837f36e58f473405)), closes [#1236](https://github.com/morluto/rea/issues/1236)
|
||||
* **javascript:** resolve package exports targets only to the exact file ([#1181](https://github.com/morluto/rea/issues/1181)) ([2a620c0](https://github.com/morluto/rea/commit/2a620c06cdcaa32844c4a80aeb3df1793f1ce09e)), closes [#1174](https://github.com/morluto/rea/issues/1174)
|
||||
* keep the Ghidra session root free of dot-prefixed path elements ([a8b8288](https://github.com/morluto/rea/commit/a8b8288396cf2e053db768ef03f4db756aa8f71e))
|
||||
* **managed:** preserve literal library names during native matching ([81b70bc](https://github.com/morluto/rea/commit/81b70bc358031686c9d7afd049f21719c67a9525))
|
||||
* **managed:** separate metadata and signature completeness ([2f02eff](https://github.com/morluto/rea/commit/2f02eff9887fc715c33f5d8770a6e6191ba70ca9))
|
||||
* **mcp:** advertise input patterns without look-around ([1f099a6](https://github.com/morluto/rea/commit/1f099a6fc00b08fb0283dcabc5a238025d4c4c8b))
|
||||
* **mcp:** advertise input patterns without look-around ([a8701ea](https://github.com/morluto/rea/commit/a8701eaf489692449739d5b2490502b1a14951c5)), closes [#1282](https://github.com/morluto/rea/issues/1282)
|
||||
* **mcp:** deliver tool errors within advertised schema contracts ([da606ff](https://github.com/morluto/rea/commit/da606ff77718d9a3b10426fcf68c098138e6dcfa)), closes [#1261](https://github.com/morluto/rea/issues/1261)
|
||||
* **mcp:** deliver typed errors without invalid structured output ([#1277](https://github.com/morluto/rea/issues/1277)) ([7e96f25](https://github.com/morluto/rea/commit/7e96f25f8b1380a13a086a68f8d5a70380468151))
|
||||
* **mcp:** emit portable Unicode NUL patterns ([#1230](https://github.com/morluto/rea/issues/1230)) ([aa5750c](https://github.com/morluto/rea/commit/aa5750c263632d98d626c5ca3b985bb1519d2395))
|
||||
* **mcp:** measure tool errors against their text-only delivery ([fe34ec2](https://github.com/morluto/rea/commit/fe34ec2ef3e2968eeb06f64a1a0fdb8b825f56c4))
|
||||
* **mcp:** measure tool errors against their text-only delivery ([7b51121](https://github.com/morluto/rea/commit/7b51121f7cee21a5938f10ac3b5762e0bb41a58d))
|
||||
* **mcp:** only recommend exporting retained evidence ([5f749c2](https://github.com/morluto/rea/commit/5f749c2096af2617d4b6b387ff19c13001af8e54))
|
||||
* name host and tool requirements in optional adapter refusals ([68cb02f](https://github.com/morluto/rea/commit/68cb02fe8b123b9a0778c0065c13a15bcf0e577d))
|
||||
* name host and tool requirements in optional adapter refusals ([170d0bf](https://github.com/morluto/rea/commit/170d0bf9e2d45b2512ad556c02f31bdc81ea03ca))
|
||||
* **native:** distinguish batch selectors from resolved entries ([13c3b68](https://github.com/morluto/rea/commit/13c3b68730df80330fd867d921dea38770d5f880))
|
||||
* **native:** distinguish batch selectors from resolved entries ([bfaa5a6](https://github.com/morluto/rea/commit/bfaa5a6c2c0ddc9e6fcf16cd11ad06f24db174a2))
|
||||
* **native:** name the CLI flag that selects another plist ([9b69503](https://github.com/morluto/rea/commit/9b6950373cfaa1a2c0d28287bfadd4a421afb9e9))
|
||||
* **native:** name the plist selection that works when Info.plist is missing ([5612496](https://github.com/morluto/rea/commit/5612496f6d34a462dcd0fb69fa38337592e2f72d))
|
||||
* **native:** name the plist selection that works when Info.plist is missing ([a6fdf42](https://github.com/morluto/rea/commit/a6fdf426bd98c8cf7b7a7f589208d3193784062f))
|
||||
* **native:** refuse truncated executable headers at target admission ([d39a450](https://github.com/morluto/rea/commit/d39a450bdfac425962c3a6a6f47b45600057a230))
|
||||
* **native:** refuse truncated executable headers at target admission ([0b04d38](https://github.com/morluto/rea/commit/0b04d384cca4f22f271d60bcfe67f79cab89c8de)), closes [#1189](https://github.com/morluto/rea/issues/1189)
|
||||
* **process:** apply one timeout rule to both execFileOutput modes ([3d32819](https://github.com/morluto/rea/commit/3d328193d69a63a355644c51bfb70f524d99bca9))
|
||||
* **process:** await cleanup on CLI and MCP cancellation ([f6fe575](https://github.com/morluto/rea/commit/f6fe575094658e99ced78d45f385633fbfaaae32))
|
||||
* **process:** keep ports before a chunk-final period ([184ed97](https://github.com/morluto/rea/commit/184ed970b9021552a5454ced20d202d10cabcbb8))
|
||||
* **process:** localize capture truncation diagnostics ([#1284](https://github.com/morluto/rea/issues/1284)) ([33bf585](https://github.com/morluto/rea/commit/33bf585185ffcaea63d658e2d3145c900a680485))
|
||||
* **process:** normalize ports that end a sentence ([098d769](https://github.com/morluto/rea/commit/098d76900effdbc682b0f97bcde7f8711380ecea))
|
||||
* **process:** normalize ports that end a sentence ([db98a51](https://github.com/morluto/rea/commit/db98a51dd1750b09b3905e18efac526f7ebf90f7))
|
||||
* **process:** preserve numeric output and original PTY evidence ([c5c75c3](https://github.com/morluto/rea/commit/c5c75c38c3a4ff0153513b8f8ad600d8a5bacc09))
|
||||
* **process:** preserve numeric output and original PTY evidence ([00d8e44](https://github.com/morluto/rea/commit/00d8e44cbf82caa51709c2e0857fbaa69ea329b7))
|
||||
* **process:** preserve unknown filesystem absence in partial captures ([c8544b1](https://github.com/morluto/rea/commit/c8544b1574a32838da6d4ed40f2b72582edb60dd))
|
||||
* **process:** refuse malformed timeouts for stoppable commands ([136d4aa](https://github.com/morluto/rea/commit/136d4aab18d822f0f908096b8acfac556ef1873b))
|
||||
* **process:** refuse malformed timeouts for stoppable commands ([59c91c8](https://github.com/morluto/rea/commit/59c91c88fe88210b4b22b9f1365a16567513da4a))
|
||||
* **process:** require current captures and preserve cleanup failures ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
|
||||
* **process:** retain known events from incomplete captures ([5eb0049](https://github.com/morluto/rea/commit/5eb0049a9f502b404f30e48c3fe3430a942bb0b5))
|
||||
* **process:** stop cancelled Swift helper builds before removing their files ([cf41561](https://github.com/morluto/rea/commit/cf415619cadcfcb18cda59a54b26c9d164f750a4))
|
||||
* **process:** stop cancelled Swift helper builds before removing their files ([c717204](https://github.com/morluto/rea/commit/c7172047951e7dbb9df58b57ff1b43ac9bd47909)), closes [#1173](https://github.com/morluto/rea/issues/1173)
|
||||
* **process:** stop timed-out commands with the caller's stop signal ([4b38d18](https://github.com/morluto/rea/commit/4b38d1899566089d19b9fd137202f7d86b62fc6f))
|
||||
* **process:** stop timed-out commands with the caller's stop signal ([063de50](https://github.com/morluto/rea/commit/063de5014f19e85c38b6bec5984de4f38e71adaa))
|
||||
* **process:** treat non-ASCII letters as port continuations ([ad813e4](https://github.com/morluto/rea/commit/ad813e473b094ab0d3ab30798b7d252e2f4beb94))
|
||||
* **process:** validate terminal byte totals against retained frames ([69833fb](https://github.com/morluto/rea/commit/69833fb0d0fe9b320b5745a992822342f2c54a92))
|
||||
* **process:** validate terminal byte totals against retained frames ([39eb058](https://github.com/morluto/rea/commit/39eb0587c95ad028bb961c539ad1f326a1e8d81c))
|
||||
* **providers:** preserve observations and primary cleanup failures ([0696bde](https://github.com/morluto/rea/commit/0696bde2660bb5d6538d8c01da15aa023312aa9f))
|
||||
* **reference:** represent unreadable symlink targets explicitly ([b7b4333](https://github.com/morluto/rea/commit/b7b433370310c3ffe33d5027d8393151d5e862da))
|
||||
* restore caching, preserve failures, and verify fixture claims ([#1211](https://github.com/morluto/rea/issues/1211)) ([27a9cea](https://github.com/morluto/rea/commit/27a9ceade7232c1c9a24dc7d454ae060948183a1))
|
||||
* **server:** report 2025-era client identity and features in binary_session ([e6abaef](https://github.com/morluto/rea/commit/e6abaefbbc2bd81a6be34cbe71b87be801158c16))
|
||||
* **server:** report 2025-era client identity and features in binary_session ([b8312fa](https://github.com/morluto/rea/commit/b8312fa8d184ba9f422d1f5592fb41e49811544a)), closes [#1167](https://github.com/morluto/rea/issues/1167)
|
||||
* **session:** require an open target for session-bound native and artifact tools ([d8e314c](https://github.com/morluto/rea/commit/d8e314cb24e7a4a494347e45474234e0aad74ad1))
|
||||
* **session:** require an open target for session-bound native and artifact tools ([cf77d07](https://github.com/morluto/rea/commit/cf77d073fc06ab95ce4181c30d68b1346e990b64)), closes [#1166](https://github.com/morluto/rea/issues/1166)
|
||||
* **session:** retain failed provider cleanup for lifecycle retries ([#1260](https://github.com/morluto/rea/issues/1260)) ([24a3aa0](https://github.com/morluto/rea/commit/24a3aa08d1f69e25b441f086300ba3082d860a35))
|
||||
* **setup:** keep Codex TOML comments when adding or removing REA ([f5539cf](https://github.com/morluto/rea/commit/f5539cfc2f6c0daf6c275fc97f3351e2cf89b2c3))
|
||||
* **setup:** keep Codex TOML comments when adding or removing REA ([e9c48a6](https://github.com/morluto/rea/commit/e9c48a61e4c2c05c5b77ef4a815c5b2a65a9337e)), closes [#1163](https://github.com/morluto/rea/issues/1163)
|
||||
* **setup:** preserve initial backups and Codex comments ([#1204](https://github.com/morluto/rea/issues/1204)) ([6051cb2](https://github.com/morluto/rea/commit/6051cb2c0c723d88dc43ee5ef8a96971bf84d49e))
|
||||
* **setup:** stop remaining uninstall removals after a client fails ([fd975c7](https://github.com/morluto/rea/commit/fd975c75fa353840e6e887232ca68496cf8705ea))
|
||||
* **setup:** stop uninstall before removals when a client config is unsafe ([4fb2f3e](https://github.com/morluto/rea/commit/4fb2f3e6ed0233505eada3cddeeff0ea0bfc6fe5))
|
||||
* **setup:** stop uninstall before removals when a client config is unsafe ([5cc841d](https://github.com/morluto/rea/commit/5cc841d5c154f692bdc257e106cc4b212dc19dc0)), closes [#1164](https://github.com/morluto/rea/issues/1164)
|
||||
* **snapshot:** report query workflow and Evidence counts separately ([04aef6d](https://github.com/morluto/rea/commit/04aef6df331634238cfc61f320ba4a4e720750a4))
|
||||
* **snapshot:** report query workflow and Evidence counts separately ([0e77953](https://github.com/morluto/rea/commit/0e779533a9108a550b4d10dcb3920f91c7d48989))
|
||||
* **website:** clarify Android host and JDK requirements ([5691606](https://github.com/morluto/rea/commit/5691606aa948bddc641e3ecbccf5375a12b65924))
|
||||
* **website:** identify publication verifier requests ([e6905c1](https://github.com/morluto/rea/commit/e6905c18e09c41e04525fb1a84575483df789204))
|
||||
* **windows:** retain runtime authority after cleanup failure ([af7a439](https://github.com/morluto/rea/commit/af7a43963351c655eb86197bf52ae798a04e9c22))
|
||||
* **windows:** retain unsettled jobs for cleanup retry ([76b214e](https://github.com/morluto/rea/commit/76b214e45fa05fe723e216894327185b9b4e6f53))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **mcp:** share repeated output schema definitions. ([441813c](https://github.com/morluto/rea/commit/441813c0708dbee82ef554c91e973ae02964bf2e))
|
||||
|
||||
* **hopper:** resolve string search records once per query ([#1229](https://github.com/morluto/rea/issues/1229)) ([9426c3f](https://github.com/morluto/rea/commit/9426c3f1bbb94711d8b33097e604b9b8ed853b5e))
|
||||
* **mcp:** reuse input schemas without losing caller guidance ([#1162](https://github.com/morluto/rea/issues/1162)) ([e1afbc1](https://github.com/morluto/rea/commit/e1afbc1914cc7ff90265313e942f34ea4a58a4fc))
|
||||
* **snapshot:** avoid redundant result copies and serialization ([552634d](https://github.com/morluto/rea/commit/552634d634655a722b2204cfbc4b29f6d442c294))
|
||||
* **snapshot:** avoid redundant result copies and serialization ([569d2bc](https://github.com/morluto/rea/commit/569d2bc6b9f600114267a4c69f8400393ea1bc2f))
|
||||
|
||||
|
||||
### Code Refactoring
|
||||
|
||||
* **domain:** remove obsolete comparison representations ([5510b27](https://github.com/morluto/rea/commit/5510b27d7f863a3df26212fba4a59f8df0da3712))
|
||||
* enforce canonical evidence and provider boundaries ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
|
||||
* remove forwarding exports and redundant aliases ([ad59531](https://github.com/morluto/rea/commit/ad59531a960a0c51ef6857fa1391270dda0fa7a4))
|
||||
* **schemas:** share portable identifier and base64 patterns ([#1227](https://github.com/morluto/rea/issues/1227)) ([954badb](https://github.com/morluto/rea/commit/954badbaaadd24a210190ce3015e40c23d327567))
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* add French README translation ([84094af](https://github.com/morluto/rea/commit/84094af54a4e0a6247445e59abff27bc4eca361b))
|
||||
* add French README translation ([9664b2b](https://github.com/morluto/rea/commit/9664b2b101fe4ac334b2911f9210b50d58d90a89))
|
||||
* celebrate 30,000 GitHub stars 🎉 ([6987cb0](https://github.com/morluto/rea/commit/6987cb066a5aa5e51480bb06a93a758e6f67cd1e))
|
||||
* celebrate 30,000 GitHub stars 🎉 ([beca997](https://github.com/morluto/rea/commit/beca9977eb2b18fc7b9844cff0ca70004b6ff21a))
|
||||
* clarify REA has no affiliated cryptocurrency or token ([#1214](https://github.com/morluto/rea/issues/1214)) ([c2d8146](https://github.com/morluto/rea/commit/c2d8146f94c7533eb87681c2405e93bdfef21b59))
|
||||
* explain 6.2 contract and saved-data migration ([#1304](https://github.com/morluto/rea/issues/1304)) ([c267e99](https://github.com/morluto/rea/commit/c267e9922751c03df312655ebd623de1b480215c))
|
||||
* **ghidra:** explain supported heap and CPU controls ([#1278](https://github.com/morluto/rea/issues/1278)) ([0c6852b](https://github.com/morluto/rea/commit/0c6852b392b34319a6fd2c6dffcac10cee51ab94)), closes [#1255](https://github.com/morluto/rea/issues/1255)
|
||||
* keep the 30k milestone in Star History ([#1217](https://github.com/morluto/rea/issues/1217)) ([c399670](https://github.com/morluto/rea/commit/c399670653884673158f314ed38260aef74da593))
|
||||
* require focused brittleness cleanup and executable examples ([d6832c3](https://github.com/morluto/rea/commit/d6832c3632847cf3e62d4082b4b8b8d9af554633))
|
||||
* thank feature request contributors ([9db1fd0](https://github.com/morluto/rea/commit/9db1fd0bd7b17ed1dd21b9c72dabfa4ec6c23a3f))
|
||||
* **website:** continue the Calculator example with reconstruction ([4578647](https://github.com/morluto/rea/commit/4578647c59a7487bce597a65be6eab8be0412e84))
|
||||
* **website:** continue the Calculator example with reconstruction ([18a1e8c](https://github.com/morluto/rea/commit/18a1e8cd0efe14c06fd14aabe0c318d2ae390f78))
|
||||
* **website:** explain SEO maintenance and indexing follow-up ([bb8ff2c](https://github.com/morluto/rea/commit/bb8ff2cfba638ee3a77f951c01e98340c757f525))
|
||||
* **website:** introduce the worked homepage examples ([f6f5c16](https://github.com/morluto/rea/commit/f6f5c160979624cdcb33928f1a77cf000de403d4))
|
||||
* **website:** invite deeper projects and reader feedback ([3fb9ffa](https://github.com/morluto/rea/commit/3fb9ffaa4d67685412873cbd3101c64fabbeaff3))
|
||||
|
||||
|
||||
### Tests
|
||||
|
||||
* **apple:** verify compiled Mach-O root occurrence facts ([#1300](https://github.com/morluto/rea/issues/1300)) ([4bb4c6f](https://github.com/morluto/rea/commit/4bb4c6fe6d34525c7665a9d38b584b026dd5cdd9))
|
||||
* **artifacts:** assert asset-catalog target refusal only on macOS ([44d5b4b](https://github.com/morluto/rea/commit/44d5b4b25e0831769c1bd4e5426ef498d0bd3c74))
|
||||
* **artifacts:** pin per-occurrence roles for direct Mach-O roots and shared bytes ([701d90a](https://github.com/morluto/rea/commit/701d90aba48aaf4f23e1a73e46ebf9968fb85293))
|
||||
* **artifacts:** pin per-occurrence roles for direct Mach-O roots and shared bytes ([b9e29c5](https://github.com/morluto/rea/commit/b9e29c52d483704c64cd88f87eaa117066288602)), closes [#1234](https://github.com/morluto/rea/issues/1234) [#1235](https://github.com/morluto/rea/issues/1235)
|
||||
* **artifacts:** read /bin/ls slice layout from its FAT table ([#1223](https://github.com/morluto/rea/issues/1223)) ([c89b3a0](https://github.com/morluto/rea/commit/c89b3a00dce8df36ddb696b59c80e1adc6aceac1))
|
||||
* **artifacts:** verify compiled Mach-O roles and permissions ([4bb4c6f](https://github.com/morluto/rea/commit/4bb4c6fe6d34525c7665a9d38b584b026dd5cdd9))
|
||||
* consolidate redundant assertions into boundary workflows ([b71a591](https://github.com/morluto/rea/commit/b71a591a60e3fa5333889e3cf97e05f4ae03c686))
|
||||
* **ghidra:** preserve selected verification resource limits ([1c3766a](https://github.com/morluto/rea/commit/1c3766a8bcc8a2b6af679cdaa92a73fec42b3372))
|
||||
* **ghidra:** size oversized annotations for canonical delivery ([1e34197](https://github.com/morluto/rea/commit/1e341973cfad1dbf06755f3a086ffc23d42a723c))
|
||||
* **ghidra:** use host-native absolute session root fixtures ([1e83d27](https://github.com/morluto/rea/commit/1e83d270c2098b1bcb59591698a257a05d2296fc))
|
||||
* **javascript:** expect the invalid input_path code in the streamed CLI failure test ([d186ca7](https://github.com/morluto/rea/commit/d186ca7e41928fc2748de2631f5cb43115776571))
|
||||
* **mcp:** read bundle failures from public error content ([69c3b32](https://github.com/morluto/rea/commit/69c3b32a21c90ea96194fb695f7b58992e8b3714))
|
||||
* **mcp:** read typed errors in canonical Evidence regressions ([fd46cff](https://github.com/morluto/rea/commit/fd46cff017f0110aaab1216764cf01a71b341766))
|
||||
* **mcp:** validate public error diagnostics in acceptance workflows ([09058eb](https://github.com/morluto/rea/commit/09058ebc4a68c76d865a92dbddb65811ba8344f7))
|
||||
* **native:** assert batch identity in production registration ([933fcb3](https://github.com/morluto/rea/commit/933fcb33c1fa1eae109228b89c1200abb7745eb8))
|
||||
* **native:** build the .app fixture executable with a complete Mach-O header ([facaae8](https://github.com/morluto/rea/commit/facaae865af243656ab8f557aeb42f5909748f2b))
|
||||
* **process:** check malformed capture choices concurrently ([6020a0b](https://github.com/morluto/rea/commit/6020a0b3683ce16e1441963473605f870c247e29))
|
||||
* **process:** check malformed capture choices concurrently ([48de11c](https://github.com/morluto/rea/commit/48de11ce2e5af9810fc97d1c2f89df56bf3a3d4d))
|
||||
* remove duplicate workflow examples and self assertions ([d6d1cd2](https://github.com/morluto/rea/commit/d6d1cd248506015b59a2abdbbb84e3dbd02388fc))
|
||||
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
* **website:** allow explicitly requested Pages-only publication ([796b65e](https://github.com/morluto/rea/commit/796b65e1cd15f3cf819396716320db007655ee3f))
|
||||
* **website:** allow explicitly requested Pages-only publication ([864160d](https://github.com/morluto/rea/commit/864160d8858fad3c0afebff81c31b090a1ebf823))
|
||||
* **website:** allow explicitly requested Pages-only publication ([#1205](https://github.com/morluto/rea/issues/1205)) ([796b65e](https://github.com/morluto/rea/commit/796b65e1cd15f3cf819396716320db007655ee3f))
|
||||
* **website:** publish Cloudflare and Pages from one artifact ([761881c](https://github.com/morluto/rea/commit/761881cec33b18d7228bae5e2093e5d44b96ceb1))
|
||||
|
||||
## [6.1.0](https://github.com/morluto/rea/compare/rea-agents-6.0.0...rea-agents-6.1.0) (2026-10-09)
|
||||
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **hopper:** Hopper regex mode uses ECMAScript Unicode syntax instead of Python re. Matching runs in a cancellable worker with a five-second deadline; literal mode keeps Unicode casefold matching.
|
||||
* **hopper:** remove set_current_document. Use open_binary to switch targets; document arguments are scoped to the active target.
|
||||
|
||||
### Features
|
||||
|
||||
* **clients:** register Grok Build and Grok Bot during setup ([#1134](https://github.com/morluto/rea/issues/1134)) ([08561e0](https://github.com/morluto/rea/commit/08561e0eb7a4a01a256f3693a97061fbea7fb8fa))
|
||||
* **ghidra:** make the startup deadline configurable ([#974](https://github.com/morluto/rea/issues/974)) ([d84bc8f](https://github.com/morluto/rea/commit/d84bc8f11abd6cbf5612daf5c8f7e1805d23c16e))
|
||||
* **website:** add GitHub link with live star count ([2f59287](https://github.com/morluto/rea/commit/2f59287f1a1794a68102a45c61cc657659fa5de6))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **analysis:** bind overview document identity to the active target ([#1094](https://github.com/morluto/rea/issues/1094)) ([81b5104](https://github.com/morluto/rea/commit/81b51042767ad75310bad1a7a9faf954339841c0))
|
||||
* **analysis:** resolve procedure identities and preserve literal queries ([d830cb7](https://github.com/morluto/rea/commit/d830cb79b305e6d4d255f8080de69ca668cbe0af))
|
||||
* **android:** report actual provider readiness ([c715af8](https://github.com/morluto/rea/commit/c715af80587f808de816d3add6a6f989437ab66a))
|
||||
* **android:** report startup and selector failure causes ([6c13fc8](https://github.com/morluto/rea/commit/6c13fc8e0a318ebe7be6b038089b350d99f8e286))
|
||||
* **apple:** account for repeated archive projections ([e1df5e8](https://github.com/morluto/rea/commit/e1df5e85551125b27ef0debb7573de5507b149e1))
|
||||
* **apple:** budget Interface Builder decoding and projection ([dbcf8c3](https://github.com/morluto/rea/commit/dbcf8c33915d924fdbc881bf2cf9676587f3b96f))
|
||||
* **apple:** classify empty dyld settings by consumer semantics ([#1110](https://github.com/morluto/rea/issues/1110)) ([fe297e5](https://github.com/morluto/rea/commit/fe297e56e804130dd49cc0a95fcf2d115c183ae3))
|
||||
* **apple:** decode bundle plist bytes before executable lookup ([d575a73](https://github.com/morluto/rea/commit/d575a7340bb19b84733c79b14de9f4df7181c6b3))
|
||||
* **apple:** decode Interface Builder XML byte encodings ([#1095](https://github.com/morluto/rea/issues/1095)) ([f8e143a](https://github.com/morluto/rea/commit/f8e143a3bcd20ddcc0d76d90e73e238d456fe5a3))
|
||||
* **apple:** derive dispatch coverage from located decode facts ([488f448](https://github.com/morluto/rea/commit/488f448ae27fd62904967ec1e0ab072e834a9822))
|
||||
* **apple:** keep skipped catalog entries from reading as absence ([a079af2](https://github.com/morluto/rea/commit/a079af297bb454417dfd3d89b0037397554ad83e))
|
||||
* **apple:** report apps without asset catalogs as empty inventories ([e2f0378](https://github.com/morluto/rea/commit/e2f03786f496ed742572acd87f47270c958171ec))
|
||||
* **apple:** report apps without asset catalogs as empty inventories ([79e5a39](https://github.com/morluto/rea/commit/79e5a3923db896323c0491e31004214856529b74))
|
||||
* **apple:** report missing keyed archive hierarchy references ([#1104](https://github.com/morluto/rea/issues/1104)) ([3b2ec69](https://github.com/morluto/rea/commit/3b2ec69c54fc9cf65bf8441edbb96e9619c3a37e))
|
||||
* **apple:** validate dylib candidates and preserve search uncertainty ([#1100](https://github.com/morluto/rea/issues/1100)) ([7914e36](https://github.com/morluto/rea/commit/7914e36ab9adbbf591a6e1f3fafcb2e17f55d667))
|
||||
* **apple:** validate physical slices and dylib platform compatibility ([15f92f2](https://github.com/morluto/rea/commit/15f92f298891c8b16d7d13c78cb6ca66cb90cd85))
|
||||
* **application:** budget bridge candidate projection by bytes ([3d450a2](https://github.com/morluto/rea/commit/3d450a29fc91cef3e06ffce9a925ac70dcb24272))
|
||||
* **artifacts:** preserve keyed archive integer precision ([4034883](https://github.com/morluto/rea/commit/4034883e9860a97edee03260fab59f9229debfa6))
|
||||
* **artifacts:** report keyed-archive selection mistakes as invalid input ([5a55d5b](https://github.com/morluto/rea/commit/5a55d5b1c6aefb4a4e5df63ec43b7e2fc0a7d385))
|
||||
* **artifacts:** report keyed-archive selection mistakes as invalid input ([25fb6d3](https://github.com/morluto/rea/commit/25fb6d32a325d88bb0f4e05e16f5f82e4f91c5ca))
|
||||
* **artifacts:** retain failed DMG cleanup state and diagnostics ([#1103](https://github.com/morluto/rea/issues/1103)) ([4bb056b](https://github.com/morluto/rea/commit/4bb056bbb5b735e214ed8d725aa3de1c06fb0dc9))
|
||||
* **browser:** advertise zero-based units for runtime source coordinates ([#1085](https://github.com/morluto/rea/issues/1085)) ([6218d28](https://github.com/morluto/rea/commit/6218d289e241715d0868004caafe15b5c4758450))
|
||||
* **browser:** bind correlated replies to their selected session ([96d3caf](https://github.com/morluto/rea/commit/96d3cafb0725881b6cdab0e2185104957dc69f71))
|
||||
* **browser:** bound aggregate source-map decoding before expansion ([f983c71](https://github.com/morluto/rea/commit/f983c718c33789ee40aec43f84315f6b53e95bde))
|
||||
* **browser:** bound script metadata and preserve omissions ([faf209e](https://github.com/morluto/rea/commit/faf209e50d9faab03a185c1f044de3582552d11e))
|
||||
* **browser:** bound source-map fetches and retain predecessor requests ([eaf5734](https://github.com/morluto/rea/commit/eaf57340831a753faec58246290f9502d159b6b1))
|
||||
* **browser:** compose nested source-map column offsets ([#1109](https://github.com/morluto/rea/issues/1109)) ([8b08ec0](https://github.com/morluto/rea/commit/8b08ec0af936405fd20db25f941e2ae6318c5145))
|
||||
* **browser:** keep initiator callsites out of script coordinate guidance ([#1096](https://github.com/morluto/rea/issues/1096)) ([3c157b4](https://github.com/morluto/rea/commit/3c157b4ec6297e6737bd3c2ec3dc520c7403402f))
|
||||
* **browser:** normalize selected script manifest roots on Windows ([#1078](https://github.com/morluto/rea/issues/1078)) ([b49beb4](https://github.com/morluto/rea/commit/b49beb42d7a97f73d052a27fb99673b1bd0d3d39))
|
||||
* **browser:** preserve captures when response bodies are unavailable ([96af2fa](https://github.com/morluto/rea/commit/96af2fa7444a3777d51ff46202ef563146929dbd))
|
||||
* **browser:** preserve command failures and reject malformed replies ([6b92002](https://github.com/morluto/rea/commit/6b92002597d34555c32ebf24b9db6e4c17c0a08d))
|
||||
* **browser:** treat module-import initiator positions as script positions ([ae0630a](https://github.com/morluto/rea/commit/ae0630a619ee2c98a68e65ca44bf0b93424e2f8e))
|
||||
* **browser:** treat module-import initiator positions as script positions ([99d31ee](https://github.com/morluto/rea/commit/99d31ee24ad060babace9eca3325858418e0f55a))
|
||||
* **browser:** validate and bound source-map evidence ([0a0174b](https://github.com/morluto/rea/commit/0a0174b9ac8a73b61094e857a2950f4e5a06f544))
|
||||
* **build:** exclude the test-only generated catalog from packages ([#1116](https://github.com/morluto/rea/issues/1116)) ([d1c904b](https://github.com/morluto/rea/commit/d1c904bac7977d44b06bf2f823587baedcc88b9e))
|
||||
* **build:** make catalogs and packaged skills generated outputs ([#1086](https://github.com/morluto/rea/issues/1086)) ([e613405](https://github.com/morluto/rea/commit/e6134056cf519410160a196363f8a04f0729324e))
|
||||
* **capture:** retain Electron observations and HAR resource failures ([44fd183](https://github.com/morluto/rea/commit/44fd18305fa5a8893ebb5b952099fdf45dbd6e55))
|
||||
* **ci:** align diagnostic regressions and format merged changes ([67756da](https://github.com/morluto/rea/commit/67756dacc0d435a61d2d0c403412151769da028f))
|
||||
* **ci:** restore generated shard inputs and cleanup assertions ([5f93b4b](https://github.com/morluto/rea/commit/5f93b4b125e78f088a2349a178e60f8c0af3e1a2))
|
||||
* **cli:** accept exact native names through explicit selectors ([c313606](https://github.com/morluto/rea/commit/c3136064c7f8f72b2f6c6799ec41ea6424111bb7))
|
||||
* **cli:** drive mcp doctor help from a single option schema ([b7d803d](https://github.com/morluto/rea/commit/b7d803dd10c7ef8bb73f81f5e893168af8ea86e6))
|
||||
* **cli:** preserve blank JSON paths for input validation ([#1068](https://github.com/morluto/rea/issues/1068)) ([3dcb732](https://github.com/morluto/rea/commit/3dcb732da33f6ceef597506b14a4536f1c9aff96))
|
||||
* **cli:** preserve web cancellation exit status ([1d9b96a](https://github.com/morluto/rea/commit/1d9b96a9c497694592137cf95bea2af0a190abe7))
|
||||
* **cli:** support MCP doctor help ([a69b833](https://github.com/morluto/rea/commit/a69b8337ab38707aaad2ac2fedb652f05afeb71c))
|
||||
* **contracts:** preserve local inputs and comparison uncertainty ([70e8a8f](https://github.com/morluto/rea/commit/70e8a8f86060496452ccbc1a49b39822efef4113))
|
||||
* **docs:** validate translated links to canonical product facts ([34f7c4f](https://github.com/morluto/rea/commit/34f7c4fd661e07db66c560a4d54272b0163a52b6))
|
||||
* **domain:** order canonical paths by code point, not UTF-16 unit ([#1034](https://github.com/morluto/rea/issues/1034)) ([b5c2954](https://github.com/morluto/rea/commit/b5c2954ad67f0280732df043d5d9702f3d202819))
|
||||
* **domain:** preserve prototype-named JSON members through Evidence boundaries ([#1049](https://github.com/morluto/rea/issues/1049)) ([40e18b2](https://github.com/morluto/rea/commit/40e18b2e37f2be11f452606c2d875981b03ad03e))
|
||||
* **domain:** reject excessive JSON depth with actionable diagnostics ([#1029](https://github.com/morluto/rea/issues/1029)) ([bfbfc73](https://github.com/morluto/rea/commit/bfbfc7330aa7175018de04f1d099b8d4aeec3bd1))
|
||||
* **electron:** bound hook retention and report dropped coverage ([02d0702](https://github.com/morluto/rea/commit/02d07027617a5df014f92a351aa9418aade6f06f))
|
||||
* **evidence:** name the failed constraint when a bundle is rejected ([82a1a52](https://github.com/morluto/rea/commit/82a1a525d7aa725d939926ba953434fbf5db0ab0))
|
||||
* **evidence:** name the failed constraint when a bundle is rejected ([cf96ceb](https://github.com/morluto/rea/commit/cf96ceb2119f52f61a621d2a790b7c8804c65595))
|
||||
* **evidence:** report missing evidence files instead of permission advice ([e8b92a4](https://github.com/morluto/rea/commit/e8b92a40c003a5e6ce3cefe4ad18392d39759653))
|
||||
* **evidence:** report missing evidence files instead of permission advice ([89252f1](https://github.com/morluto/rea/commit/89252f1f35de91494bd49460b1645d8c09cc6d33))
|
||||
* **evidence:** require a complete record before the single-record hint ([38bbb65](https://github.com/morluto/rea/commit/38bbb656da2659088e30a1ca3e4434b2f65bab32))
|
||||
* **firmware:** reject malformed UTF-8 producer reports ([4a45c14](https://github.com/morluto/rea/commit/4a45c14742e6868f687cead2ddf736c3f7c49acd))
|
||||
* **ghidra:** admit Linux arm64 with matching native tools ([3e598b2](https://github.com/morluto/rea/commit/3e598b2d38a728cd9fd4a359741500a021ba9d3a))
|
||||
* **ghidra:** harden location resolution and provider lifecycle ([1f5be75](https://github.com/morluto/rea/commit/1f5be750bced759339183945488cea352b18f28b))
|
||||
* **ghidra:** make qualified function renames idempotent ([4458128](https://github.com/morluto/rea/commit/4458128a845c1448140b44d6e62d6ccfe611d37c))
|
||||
* **ghidra:** parse long procedure selectors without recursion ([480b16b](https://github.com/morluto/rea/commit/480b16b9c151e1d4bfbca8b49a0668d74ceaa400))
|
||||
* **ghidra:** preserve function references and snapshot boundaries ([84a17d5](https://github.com/morluto/rea/commit/84a17d55199b1b46493aea6bf4e41029a3b200fc))
|
||||
* **ghidra:** preserve POSIX runtime paths with spaces ([80c3d36](https://github.com/morluto/rea/commit/80c3d36dfa2597f34fdeb69e0ba75277ee50c85c))
|
||||
* **ghidra:** preserve source failures and snapshot ownership ([704fa76](https://github.com/morluto/rea/commit/704fa7699ca183d5d5e0518a6fcdbb953ec35a10))
|
||||
* **ghidra:** preserve Unicode boundaries without breaking sessions ([bef234d](https://github.com/morluto/rea/commit/bef234d6ce63cd296a654400443fcdbbcd44d7de))
|
||||
* **ghidra:** recover regex stack exhaustion without losing annotations ([bc4aa46](https://github.com/morluto/rea/commit/bc4aa46d7c3960a1f325523e48aed2089b176e84))
|
||||
* **ghidra:** reject address truncation before reads and edits ([dcae3c6](https://github.com/morluto/rea/commit/dcae3c6c148582d235f5650aace6f9744eeb153c))
|
||||
* **ghidra:** resolve exact external function entries ([549effa](https://github.com/morluto/rea/commit/549effa056729e5f9239d9ed7864537a9c49e421))
|
||||
* **hopper:** bind analysis to the active target and correct boundary contracts ([2f71375](https://github.com/morluto/rea/commit/2f71375efc517bb9e6f9800f8e8a6849766bc893))
|
||||
* **hopper:** bind file mappings to verified source bytes ([82626c4](https://github.com/morluto/rea/commit/82626c4499a53c227187575d40a845fadd5318e2))
|
||||
* **hopper:** diagnose MCP provider failures with stage, state, and request context ([#1044](https://github.com/morluto/rea/issues/1044)) ([5a67c7f](https://github.com/morluto/rea/commit/5a67c7fc540c494371de76dcdf353923656993d3))
|
||||
* **hopper:** enforce caller-supplied request deadlines ([#1119](https://github.com/morluto/rea/issues/1119)) ([e8939ca](https://github.com/morluto/rea/commit/e8939ca2208bc41454b73b90a32418714e6a2969))
|
||||
* **hopper:** isolate regex matching from the native API thread ([f201b04](https://github.com/morluto/rea/commit/f201b04fbbb29591b0d2bb3f3b24636526e4ec4e))
|
||||
* **hopper:** load verified FAT64 slices with native Mach-O semantics ([0a49d5b](https://github.com/morluto/rea/commit/0a49d5bfb931124d3d398dfbd64e9f6ae8a01405))
|
||||
* **hopper:** normalize native block endpoints and retain terminal evidence ([2847311](https://github.com/morluto/rea/commit/2847311a579ce2da4c8ded4d89fcdd01545e6da5))
|
||||
* **hopper:** preserve launcher evidence on readiness timeout ([458771c](https://github.com/morluto/rea/commit/458771cb583e38b848b3d8c58595d163143ff50a))
|
||||
* **hopper:** preserve native navigation and annotation semantics ([f2e22e1](https://github.com/morluto/rea/commit/f2e22e13a6956985bd65a2d75e4db7e3128cce81))
|
||||
* **hopper:** preserve owned basic-block endpoint instructions ([#1118](https://github.com/morluto/rea/issues/1118)) ([329cf20](https://github.com/morluto/rea/commit/329cf206d1111c051733c7115cabff4b46722deb))
|
||||
* **hopper:** preserve partial native call and string evidence ([2461702](https://github.com/morluto/rea/commit/246170238ebb8669a20b57b559b03cfc26ee1b8c))
|
||||
* **hopper:** preserve typed string values and verify real searches ([7d7b3f9](https://github.com/morluto/rea/commit/7d7b3f961af2dc9b7572e91aaec29e6c4eccf9d5))
|
||||
* **hopper:** report launcher failures before bridge timeout ([#1105](https://github.com/morluto/rea/issues/1105)) ([cd756b0](https://github.com/morluto/rea/commit/cd756b07d5b9c06e085d778e9f1c4fc9bd403219))
|
||||
* **hopper:** resolve FAT loaders and original file coordinates ([35f1ddd](https://github.com/morluto/rea/commit/35f1ddd6bc77588b9971961cd70968a815aae266))
|
||||
* **hopper:** validate annotation destinations and symbol extents ([b2d5bc1](https://github.com/morluto/rea/commit/b2d5bc110914f2ad42d17bc739d5f37c1b0f0a4d))
|
||||
* **hopper:** verify lease cleanup and repair the macOS test lane ([#1111](https://github.com/morluto/rea/issues/1111)) ([b743bd2](https://github.com/morluto/rea/commit/b743bd20a0cbc39728e2e822fa75c013d87e1730))
|
||||
* **inspector:** bound retained metadata and preserve cancellation ([e714fb0](https://github.com/morluto/rea/commit/e714fb027afdf11bfbb0838722523d3fa9ebf244))
|
||||
* **install:** guard empty prefix_args expansion on macOS Bash 3.2 under set -u ([#1061](https://github.com/morluto/rea/issues/1061)) ([68b9fa4](https://github.com/morluto/rea/commit/68b9fa489b0c07f580633785ec61c17fa20b5083))
|
||||
* **javascript:** bound primitive string growth and repeated evaluation ([4a1837d](https://github.com/morluto/rea/commit/4a1837d9d52b1a6454469e2dc63eeb8d716fb296))
|
||||
* **javascript:** bound semantic expansion and resolve lexical receivers ([16f4609](https://github.com/morluto/rea/commit/16f4609d689445c05a0befc744fce7e72f7047ec))
|
||||
* **javascript:** classify direct defaults and argv elements ([6495171](https://github.com/morluto/rea/commit/649517181268a0b729970b7cbbda15e04b199e32))
|
||||
* **javascript:** honor cancellation before evidence publication ([#1098](https://github.com/morluto/rea/issues/1098)) ([cc0ee2e](https://github.com/morluto/rea/commit/cc0ee2edc60dda42d7b7cfb2c08e39b0b736b5e9))
|
||||
* **javascript:** omit self-referential import edges with disclosure ([#1130](https://github.com/morluto/rea/issues/1130)) ([1d7115b](https://github.com/morluto/rea/commit/1d7115b7b470719a32e477ca3cbb681f0ad408b8))
|
||||
* **javascript:** parse plain TypeScript artifact dialects ([#1102](https://github.com/morluto/rea/issues/1102)) ([3db4681](https://github.com/morluto/rea/commit/3db46813f80f84504c5b87c73a4bece82a8fdc5e))
|
||||
* **javascript:** preserve HTML carriage return source ranges ([#1092](https://github.com/morluto/rea/issues/1092)) ([23862af](https://github.com/morluto/rea/commit/23862af1b57413e09abf394f6e4f8369f6355d5a))
|
||||
* **javascript:** preserve lexical open and exact template semantics ([d6cc9ca](https://github.com/morluto/rea/commit/d6cc9ca3a6e92604c837f61c193baf5abedaabc8))
|
||||
* **javascript:** preserve original BOM source coordinates ([#1093](https://github.com/morluto/rea/issues/1093)) ([83defc0](https://github.com/morluto/rea/commit/83defc054b69e94da888d70bb3b05d835e1bf74d))
|
||||
* **javascript:** retain NodeNext artifact source facts ([#1087](https://github.com/morluto/rea/issues/1087)) ([c1b370c](https://github.com/morluto/rea/commit/c1b370ce0dfcbd5725cf6aebe23b022dfc75ac84))
|
||||
* **javascript:** use Node legacy package entry fields ([1706005](https://github.com/morluto/rea/commit/170600541bf44f2a63731e8b3d42aae3cbb8d6b3))
|
||||
* **managed:** bind inspection evidence to artifact identity ([73b09f4](https://github.com/morluto/rea/commit/73b09f4c5c511ce80baa0c89717181bbe0ecba8f))
|
||||
* **managed:** match exact method bytes without decoded signatures ([17809ca](https://github.com/morluto/rea/commit/17809caacd0f44fcc7c1af42932ea655489d64ae))
|
||||
* **mcp:** flatten root union input schemas ([#1062](https://github.com/morluto/rea/issues/1062)) ([6a7650e](https://github.com/morluto/rea/commit/6a7650e93cba9169baad95d2c52b11351ae39d0e))
|
||||
* **mcp:** preserve retained evidence on oversized delivery ([0639a54](https://github.com/morluto/rea/commit/0639a541a3b712940119f59e8189bc962b267726))
|
||||
* **mcp:** retain complete application results across response limits ([a1ed423](https://github.com/morluto/rea/commit/a1ed4232de635d4acc0dd9eb293e309a29d03e80))
|
||||
* **mcp:** retain oversized errors before transport delivery ([b3f1b4e](https://github.com/morluto/rea/commit/b3f1b4e822aec5626d1952a88f50c7520dc265d3))
|
||||
* **metadata:** stop schema changes churning generated skill evidence ([#1075](https://github.com/morluto/rea/issues/1075)) ([d52f66d](https://github.com/morluto/rea/commit/d52f66d62a93ec09cccdf53e64e3ccd24d16e520))
|
||||
* **native:** bind signature observations to registered target versions ([#1099](https://github.com/morluto/rea/issues/1099)) ([c789af0](https://github.com/morluto/rea/commit/c789af04b76c0e76a522a5dece34f461288f5ecd))
|
||||
* **native:** bound retained observations and preserve failure evidence ([7c32387](https://github.com/morluto/rea/commit/7c323870c2dcab78654ee7a80128b06affdf2bbe))
|
||||
* **native:** preserve bridge failure types during cleanup ([1d0baa3](https://github.com/morluto/rea/commit/1d0baa352fc76d1760abbe29a3cc80fa17296916))
|
||||
* **native:** resample tool identity for each invocation ([14b19a0](https://github.com/morluto/rea/commit/14b19a0b8a5af401212c552173e761eac53bfe46))
|
||||
* **native:** share command supervision and retain failure output ([f2e9c49](https://github.com/morluto/rea/commit/f2e9c491ea00bf79d5665102d95a3339b63cfe4f))
|
||||
* **native:** supervise LLDB and preserve bounded capture uncertainty ([915855e](https://github.com/morluto/rea/commit/915855e75ca3fe3b1b54978ace6bbe10d1973f64))
|
||||
* **process:** avoid cold ownership helper deadline exhaustion ([#1139](https://github.com/morluto/rea/issues/1139)) ([65a809a](https://github.com/morluto/rea/commit/65a809a0da0555aa8eeafbc2ec2559517471be02))
|
||||
* **process:** derive executable names from the capture host ([#1108](https://github.com/morluto/rea/issues/1108)) ([34cc925](https://github.com/morluto/rea/commit/34cc9259c1076a0d9575a87679befca692ec86c2))
|
||||
* **process:** identify unknown live processes in cleanup diagnostics ([eb89a16](https://github.com/morluto/rea/commit/eb89a16d9bc51b65dcc75e05f8492de894320753))
|
||||
* **process:** name the reason two captures cannot be compared ([#1080](https://github.com/morluto/rea/issues/1080)) ([eb69763](https://github.com/morluto/rea/commit/eb69763d27e29fe3247c008925016e4c73cb6abe))
|
||||
* **process:** preserve observations after successful cleanup ([2af12a1](https://github.com/morluto/rea/commit/2af12a18a36bb80fb7d91ef4c8009212750daa0a))
|
||||
* **process:** retain ownership after incomplete cleanup ([e2fc249](https://github.com/morluto/rea/commit/e2fc2493b6a59f0663495d277c0cfec1aeb25a7a))
|
||||
* **process:** stabilize native capture and ownership prerequisites ([2b27515](https://github.com/morluto/rea/commit/2b2751547662f8cb142f96d91eff50388a1ecb31))
|
||||
* **process:** stop failing captures on tokens macOS cannot expose ([#1057](https://github.com/morluto/rea/issues/1057)) ([5547c1e](https://github.com/morluto/rea/commit/5547c1e697af8c2e1c771abe5344d10b7c1cd340))
|
||||
* **providers:** validate ELF versions and preserve probe cancellation ([8b01e0c](https://github.com/morluto/rea/commit/8b01e0cc8904b512b517b4471ef2150f40c98f0d))
|
||||
* **release:** always bump minor for releases ([be783ef](https://github.com/morluto/rea/commit/be783efaba62dc76c5d8d42229869a32afdb302a))
|
||||
* **release:** publish prereleases under the next npm tag ([bfb7776](https://github.com/morluto/rea/commit/bfb7776006303c6164dba2f22b8e81fdb7c20beb))
|
||||
* **release:** publish reviewed release PR merges ([cdddfdd](https://github.com/morluto/rea/commit/cdddfddf495ffd172f62a6a801f7697fe502847e))
|
||||
* **release:** recognize conventional titles in GitHub merge commits ([f5009ed](https://github.com/morluto/rea/commit/f5009ed55f081939409808870458162991f836ca))
|
||||
* **release:** restore automatic release proposals on main ([acb3e0f](https://github.com/morluto/rea/commit/acb3e0f2e5b38a9c97df7ed78c8e6e5f38bfa08f))
|
||||
* **release:** validate frozen checkpoint versions and breaking notes ([8a31508](https://github.com/morluto/rea/commit/8a31508c8958800364e255540ded1586298d7008))
|
||||
* **release:** validate frozen checkpoint versions and breaking notes ([60825fb](https://github.com/morluto/rea/commit/60825fb44751c9482bdf4cafdac3599da7c82d45)), closes [#1063](https://github.com/morluto/rea/issues/1063)
|
||||
* select Node legacy package entry fields for import and require ([#1035](https://github.com/morluto/rea/issues/1035)) ([1706005](https://github.com/morluto/rea/commit/170600541bf44f2a63731e8b3d42aae3cbb8d6b3))
|
||||
* **session:** serialize snapshot lifecycle with admitted requests ([e44cb45](https://github.com/morluto/rea/commit/e44cb45908af8d3902e0db044e0cf7b86cdd7ed0))
|
||||
* **setup:** launch Windows npx registrations through cmd.exe ([#1072](https://github.com/morluto/rea/issues/1072)) ([ef30343](https://github.com/morluto/rea/commit/ef30343154d64d78b620d41c37457eee5d99fc71))
|
||||
* **setup:** treat empty client config files as new documents ([#1117](https://github.com/morluto/rea/issues/1117)) ([9fd65ce](https://github.com/morluto/rea/commit/9fd65ce0711ce9257aee15e62a5ecf9832960035))
|
||||
* **snapshot:** retain MCP workflow replay bindings ([5e1afe4](https://github.com/morluto/rea/commit/5e1afe454f2f283eeeea4532d07a9bdbea8bd51e))
|
||||
* **target:** preserve app bundle filesystem failure reasons ([d4c2c3c](https://github.com/morluto/rea/commit/d4c2c3c18f691d04526fe38867347faa2530861d))
|
||||
* **update:** accept npm 12 release metadata ([#1138](https://github.com/morluto/rea/issues/1138)) ([567a82f](https://github.com/morluto/rea/commit/567a82f8d4feb629a7c10d2cc9180377eb1f6f66))
|
||||
* **verify:** close both sides of the browser proxy ([f7366a5](https://github.com/morluto/rea/commit/f7366a517ff0545f01df6202b1ed3231d9dd3b4a))
|
||||
* **verify:** isolate package archives and published canary caches ([3eb7dbd](https://github.com/morluto/rea/commit/3eb7dbdc1689737b44cc4ff044bcb1a311511969))
|
||||
* **verify:** remove stale mocked bridge package probe ([826583d](https://github.com/morluto/rea/commit/826583dd43fc918c70a87dfe148c2fdbc8612f3b))
|
||||
* **verify:** report final success only after resource cleanup ([ee043e2](https://github.com/morluto/rea/commit/ee043e2cbf30152c60b403fff050a0fc848a3286))
|
||||
* **workflows:** keep caller-input details in workflow input errors ([#1089](https://github.com/morluto/rea/issues/1089)) ([289aef6](https://github.com/morluto/rea/commit/289aef6838e424c14f331a9905020079e6b7539b))
|
||||
* **workflows:** retain residual unknowns through CLI and MCP ([0136ee8](https://github.com/morluto/rea/commit/0136ee8c22300ed351af5eab6329ab4d271b5bb6))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **mcp:** advertise each tool JSON Schema once per target ([#1059](https://github.com/morluto/rea/issues/1059)) ([5ccb924](https://github.com/morluto/rea/commit/5ccb9241be15bb6a19b99fbbfb27ff3356ba157b))
|
||||
* **build:** avoid duplicate documentation generation ([c701376](https://github.com/morluto/rea/commit/c701376777212ec9acfd10c97f45b1eaf0c885c6))
|
||||
* **javascript:** index configuration and module lookups ([67252b2](https://github.com/morluto/rea/commit/67252b23b9e35e9526fed7842dab47a2c7c9fe5f))
|
||||
* **javascript:** reuse validated immutable graphs ([#1112](https://github.com/morluto/rea/issues/1112)) ([ec6db0f](https://github.com/morluto/rea/commit/ec6db0f0630c4b0b35f0db84c0030784c1aa58a6))
|
||||
|
||||
|
||||
### Code Refactoring
|
||||
|
||||
* **artifacts:** centralize stable inventory identities ([135d102](https://github.com/morluto/rea/commit/135d102f9df844d70d352870afce80f67c0ae96f))
|
||||
* **browser:** charge source-map records before retention ([e762e32](https://github.com/morluto/rea/commit/e762e32afbba3fd32532fd42a3f4fdcd05252ad3))
|
||||
* **config:** prune unused format wrappers and launcher export ([81ae838](https://github.com/morluto/rea/commit/81ae8385679ef10b95d34d0f0b5ba11732ac86b5))
|
||||
* **input:** share schema issue error construction ([65a6288](https://github.com/morluto/rea/commit/65a6288257e4c3cab9adb53b0f9123ef567b61bd))
|
||||
* **setup:** validate formats through canonical configuration ([4b10665](https://github.com/morluto/rea/commit/4b1066501b379107b2375e270f386fc9855a85c1))
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* add Persian README translation with full language navigation ([7a7460f](https://github.com/morluto/rea/commit/7a7460f04f3cfd7e365d70ff891bdd0c474901b6))
|
||||
* add ten README translations and expand language navigation ([a61491a](https://github.com/morluto/rea/commit/a61491ad00b21e912e16f68bff829927aed6e16f))
|
||||
* align translated READMEs and website links with English ([d83f098](https://github.com/morluto/rea/commit/d83f0983d5aa18d29d95d24c4ccb3928be6f29f5))
|
||||
* celebrate 20,000 GitHub stars 🎉 ([#1079](https://github.com/morluto/rea/issues/1079)) ([9455e7c](https://github.com/morluto/rea/commit/9455e7c8bc9d5c4a9a1fc9077aa15007ad80997e))
|
||||
* check surrounding cases when changing boundaries ([46d195e](https://github.com/morluto/rea/commit/46d195e8a48b710fc355e84cb6b79461b9c39005))
|
||||
* clarify contribution scope and cosmetic suggestions ([1f79002](https://github.com/morluto/rea/commit/1f7900249783956070af49c10f86002a0dfdc42b))
|
||||
* clarify evidence ownership and verification guidance ([a9e00b8](https://github.com/morluto/rea/commit/a9e00b83f37651a8dd241375615b248062e04227))
|
||||
* move Portuguese toward the end of language navigation ([2a4979b](https://github.com/morluto/rea/commit/2a4979b7cdc1a517865573d6c36654c022347d0a))
|
||||
* move star milestone into history sections across readmes ([6e6ff7b](https://github.com/morluto/rea/commit/6e6ff7bd2eabb668602258ad5e31eb758b31f823))
|
||||
* require partial native facts at provider boundaries ([2b86ea9](https://github.com/morluto/rea/commit/2b86ea9c7cd3cfc2141a021f8b531dbc84395c81))
|
||||
* require surrounding boundary checks for review fixes ([9833c94](https://github.com/morluto/rea/commit/9833c94656e1d54e7711dc483bf8091ced14234f))
|
||||
* **testing:** prioritize real workflows and distinct regressions ([9588482](https://github.com/morluto/rea/commit/9588482bfeeea869d7274d27fb96bac70278dd42))
|
||||
* **website:** add a CTF showcase and FAQ ([#1077](https://github.com/morluto/rea/issues/1077)) ([a4d77f0](https://github.com/morluto/rea/commit/a4d77f0ff0242b7ab891030d2ffa09a4b4bc1c3c))
|
||||
|
||||
|
||||
### Tests
|
||||
|
||||
* **acceptance:** add application workflow MCP parity harness and scenarios ([5ac3dc9](https://github.com/morluto/rea/commit/5ac3dc96f064e691b357aeddc11fa76b7b4d66d1))
|
||||
* **artifacts:** retire simulated mounts after detach failures ([f591e37](https://github.com/morluto/rea/commit/f591e37a1e45d9b7179ed788c4e7ee77a708c6b0))
|
||||
* **artifacts:** verify native directory identity on supported Windows runtimes ([#1069](https://github.com/morluto/rea/issues/1069)) ([e94be95](https://github.com/morluto/rea/commit/e94be95c66515fa6fba2a46e4b29850d34141fef))
|
||||
* **browser:** pin DOM and source-map URL policy classifications ([#1120](https://github.com/morluto/rea/issues/1120)) ([c1d0002](https://github.com/morluto/rea/commit/c1d0002470dc2b6bde73e4ffe438e734dbb24d8d))
|
||||
* **browser:** reject unmodeled CDP commands in protocol fixtures ([017297c](https://github.com/morluto/rea/commit/017297c40c0c38b9d9c17f5194f0f4c51748f0c3))
|
||||
* classify application scenarios by their actual boundaries ([0e0b386](https://github.com/morluto/rea/commit/0e0b386dc35bdee1174df9a2d718e0f3e5793666))
|
||||
* **cli:** verify self-import parity and cleanup ([55c9ce2](https://github.com/morluto/rea/commit/55c9ce26d1b52f359a21512103088048f9aa7492))
|
||||
* consolidate failure guards and trim unused fixture APIs ([ffd9611](https://github.com/morluto/rea/commit/ffd9611308c91b894a7b0d27f331238f5117dacf))
|
||||
* **hopper:** cover real boundary failures and CLI parity ([64717ad](https://github.com/morluto/rea/commit/64717ad7daa3152eeb718b4eb5fdcc77f1d76542))
|
||||
* **hopper:** verify real boundaries and prune redundant happy paths ([7005166](https://github.com/morluto/rea/commit/7005166f5878737fb7b3c7ec4e3d2ee5f4dcce30))
|
||||
* **mcp:** serialize real process capture and report unavailable authority ([8f9da25](https://github.com/morluto/rea/commit/8f9da251647beb530f6851c9dad5040b19a50a35))
|
||||
* **process:** cover argv rewrites in native ownership inspection ([09d7b9b](https://github.com/morluto/rea/commit/09d7b9bab53ae968502fd6e19df694cca8cab817))
|
||||
* **process:** distinguish native startup from producer verification ([9a70fb4](https://github.com/morluto/rea/commit/9a70fb482c974658088167c9d279bd70159e5dc8))
|
||||
* **process:** expose unavailable capture and unexpected teardown failures ([90b06db](https://github.com/morluto/rea/commit/90b06db6c0b2c5863f292f63a226a5c5678a86ac))
|
||||
* **process:** report the cause of terminal capture failures ([8a4c4b6](https://github.com/morluto/rea/commit/8a4c4b657fe3b0c249dd2e2147f6960361684c59))
|
||||
* **process:** retain cancellation checks under unverified host cleanup ([9a5b077](https://github.com/morluto/rea/commit/9a5b0778993102460e333de6cad3ba85383c10be))
|
||||
* **process:** retire native caches at worker teardown ([a0784c0](https://github.com/morluto/rea/commit/a0784c0d2cb70c48ecfeb0be23c6d68de5ee5be1))
|
||||
* **process:** serialize real CLI capture boundaries ([82560eb](https://github.com/morluto/rea/commit/82560ebf593a7dfc704698ee312e4d8f1629be8f))
|
||||
* prune redundant cases and promote configuration lifecycle coverage ([5945915](https://github.com/morluto/rea/commit/5945915f91adf85de91e16b69534bc3ed1d1b69a))
|
||||
* prune resource-recovery wording assertions ([ae06dcf](https://github.com/morluto/rea/commit/ae06dcf83f65b06abb390d2e0b0ea33543278d6e))
|
||||
* remove duplicate result assertions before throwing guards ([a1b7d40](https://github.com/morluto/rea/commit/a1b7d40bb0212cdfdddd770a1bccbed097601e0a))
|
||||
* **session:** preserve live binding after cancelled switches ([1136ea8](https://github.com/morluto/rea/commit/1136ea881029eab59ffb0834149e86a6aff69e3b))
|
||||
* **setup:** scope removal regression to its fixture client ([fe48e77](https://github.com/morluto/rea/commit/fe48e779c264745d28b5a819e51c06e6b5bd2776))
|
||||
* **workflows:** verify residual questions in CLI snapshots ([770d2f6](https://github.com/morluto/rea/commit/770d2f60aa948cfdb61c71304be2f3e35f673840))
|
||||
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
* run signature target binding regressions on native macOS ([#1107](https://github.com/morluto/rea/issues/1107)) ([1e63144](https://github.com/morluto/rea/commit/1e63144390070f569998271fafd1459788ef3265))
|
||||
|
||||
## [6.0.0](https://github.com/morluto/rea/compare/rea-agents-5.0.0...rea-agents-6.0.0) (2026-10-08)
|
||||
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **contracts:** MCP filesystem inputs now require absolute host paths, including target/snapshot paths, evidence bundle paths, managed and firmware inputs, browser/Electron launch paths and expected server paths. Calls such as `open_binary({"path":"./app"})` must send an absolute path such as `/home/analyst/app` or `C:\analysis\app`. Optional paths remain optional. CLI operator-relative paths continue to work. See [#948](https://github.com/morluto/rea/pull/948) and [#1004](https://github.com/morluto/rea/pull/1004).
|
||||
|
||||
### Features
|
||||
|
||||
* **artifacts:** trace Mach-O dylib resolution in bundles ([78d594c](https://github.com/morluto/rea/commit/78d594cacd376725f833a99402069c103aae9aac))
|
||||
* **browser:** inspect historical web network captures ([#992](https://github.com/morluto/rea/issues/992)) ([2bf3663](https://github.com/morluto/rea/commit/2bf3663f654386c9fd8c1437c49659df97bd4697))
|
||||
* **evm:** inspect offline bytecode interfaces through EVMole ([#1051](https://github.com/morluto/rea/issues/1051)) ([7af7bc8](https://github.com/morluto/rea/commit/7af7bc88ce0e8f084eaa0db1e8f834d495440fd9))
|
||||
* **ghidra:** support native x86 PE applications on Windows ([#1030](https://github.com/morluto/rea/issues/1030)) ([c6dad93](https://github.com/morluto/rea/commit/c6dad932084032bcb32acc61e8066a7784497710))
|
||||
* **native:** inspect offline ELF layout through pwntools ([#1037](https://github.com/morluto/rea/issues/1037)) ([4fc6565](https://github.com/morluto/rea/commit/4fc6565b0ffb36d7105b3524e08d0c5d603ae07d))
|
||||
* **native:** inspect recorded crashes through pwntools and pwndbg ([#1054](https://github.com/morluto/rea/issues/1054)) ([163ee09](https://github.com/morluto/rea/commit/163ee09a7425479385e0ed0b5d74043e2587eab2))
|
||||
* **native:** observe function and Objective-C method calls under LLDB ([1d13cef](https://github.com/morluto/rea/commit/1d13cef5f46f5e07f594bde7b1a4725f189b9a5e))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **dotnet:** validate managed metadata and CIL boundaries ([#938](https://github.com/morluto/rea/pull/938)) ([bda4ada](https://github.com/morluto/rea/commit/bda4ada7fb6bd7dd70a444bb3c57e34f7f5cc00b))
|
||||
|
||||
* **build:** clarify generated-file check hint and drop redundant catalog step ([#998](https://github.com/morluto/rea/pull/998)) ([6f0ef16](https://github.com/morluto/rea/commit/6f0ef160501ea3b24dfa2939dfcd3eea0dc13ee7))
|
||||
|
||||
* **release:** extend npm propagation window and improve diagnostics ([#943](https://github.com/morluto/rea/pull/943)) ([06b047a](https://github.com/morluto/rea/commit/06b047a0929369d5b8635b3651da115c8fcd53a7))
|
||||
|
||||
* **contracts:** require absolute local paths for session filesystem inputs ([#948](https://github.com/morluto/rea/pull/948)) ([5b172b1](https://github.com/morluto/rea/commit/5b172b1042fc09a4e12d956018f15d86cf397ffe))
|
||||
|
||||
* **ci:** generate source catalog before shard tests ([#1002](https://github.com/morluto/rea/pull/1002)) ([eb8f649](https://github.com/morluto/rea/commit/eb8f6491a88bfa79d13f7923e4ef94d791f6ba4d))
|
||||
|
||||
* **contracts:** require absolute local paths for remaining caller-supplied file inputs ([#1004](https://github.com/morluto/rea/pull/1004)) ([cbc0dd1](https://github.com/morluto/rea/commit/cbc0dd1e7ca36fa3504a9ea8eae303a0876b176b))
|
||||
|
||||
* **cli:** classify ENOTDIR JSON input failures ([#1000](https://github.com/morluto/rea/pull/1000)) ([a6132be](https://github.com/morluto/rea/commit/a6132be0546d6a9e3fa83426c8b59b1280b3e5bd))
|
||||
|
||||
* **browser:** preserve authorized redirect hops ([#986](https://github.com/morluto/rea/pull/986)) ([ae0f725](https://github.com/morluto/rea/commit/ae0f725fad368730c6198d49716ccc410bca0536))
|
||||
|
||||
* **mcp:** reject unknown MCP tool inputs ([#975](https://github.com/morluto/rea/pull/975)) ([fe64fd6](https://github.com/morluto/rea/commit/fe64fd65d93eff202ad90145544dc9ff84651974))
|
||||
|
||||
* **release:** publish from explicit source checkpoints ([#1003](https://github.com/morluto/rea/pull/1003)) ([64c7188](https://github.com/morluto/rea/commit/64c71880ae4f7bc658ffce46eb729c4141dbdf19))
|
||||
|
||||
* **release:** publish from explicit source checkpoints ([#1006](https://github.com/morluto/rea/pull/1006)) ([b4fc0ec](https://github.com/morluto/rea/commit/b4fc0ecaa1f6b912dc27880e6b6127ace47abc2e))
|
||||
|
||||
* **javascript:** bound semantic node projection for resource safety ([#966](https://github.com/morluto/rea/pull/966)) ([96a46ad](https://github.com/morluto/rea/commit/96a46ada0a36f0349f2b0fc20b4c932401b4ad84))
|
||||
|
||||
* **ci:** avoid slow Azure archive for the native cross compiler ([#1016](https://github.com/morluto/rea/issues/1016)) ([51956c1](https://github.com/morluto/rea/commit/51956c1a49a7dc9ed70a6fba480a670a7e7b2b9b))
|
||||
* **ci:** decouple stdio smoke tests from Linux display probes ([#1014](https://github.com/morluto/rea/issues/1014)) ([93731e1](https://github.com/morluto/rea/commit/93731e17ddea76db86dfa84bb54536dcf8ebfa0b))
|
||||
* **cli:** stream large JavaScript JSON results ([#1024](https://github.com/morluto/rea/issues/1024)) ([0e25bef](https://github.com/morluto/rea/commit/0e25befe78f220691abe5d0291cc04a75c9cc877))
|
||||
* **javascript:** complete large-ASAR analysis with file-local projection ([#1055](https://github.com/morluto/rea/issues/1055)) ([b9fb438](https://github.com/morluto/rea/commit/b9fb43883570bef7cb29fd16f27c25d5a7e5bb54))
|
||||
* **javascript:** recognize digits in artifact URI schemes ([#1045](https://github.com/morluto/rea/issues/1045)) ([03424ae](https://github.com/morluto/rea/commit/03424ae91ef22efd57db1bdde9379a3ac450b8f7))
|
||||
* **javascript:** reconstruct HTML script loads accurately ([#1042](https://github.com/morluto/rea/issues/1042)) ([1154f2f](https://github.com/morluto/rea/commit/1154f2f03c0b0bb6cb6c5da09b8d8e1c837febc4))
|
||||
* **javascript:** report application analysis progress on the CLI ([#1008](https://github.com/morluto/rea/issues/1008)) ([7b18c58](https://github.com/morluto/rea/commit/7b18c581409848077de7a9cb6d7d1a0b91a02e12))
|
||||
* **javascript:** resolve module URLs and reject invalid exports targets ([#1031](https://github.com/morluto/rea/issues/1031)) ([8d5c88f](https://github.com/morluto/rea/commit/8d5c88f17cde4ab4fa54c5142f0a45e612619a53))
|
||||
* **javascript:** resolve root package exports arrays ([#1046](https://github.com/morluto/rea/issues/1046)) ([53409ab](https://github.com/morluto/rea/commit/53409ab1968d60c77f98feedefe237a08859a95c))
|
||||
* **mcp:** advertise JSON values without recursive schemas ([#995](https://github.com/morluto/rea/issues/995)) ([7425f70](https://github.com/morluto/rea/commit/7425f70fd0247a7ea7192deb959fd6f48ba83a21)), closes [#919](https://github.com/morluto/rea/issues/919)
|
||||
* **mcp:** bound capture comparison input schema depth ([#1005](https://github.com/morluto/rea/issues/1005)) ([6b55de0](https://github.com/morluto/rea/commit/6b55de017badc9066642e1ff2561eb1da0ff93f8))
|
||||
* **process:** preserve partial evidence and verify detached cleanup ([#1011](https://github.com/morluto/rea/issues/1011)) ([63cf28f](https://github.com/morluto/rea/commit/63cf28fd8f8acaf656e993300a2a24280b2fd692))
|
||||
* **release:** support built-in credentials and refresh release boundary ([6a278a6](https://github.com/morluto/rea/commit/6a278a68dd4fa69b9153697ecb7bba7de4c55c32))
|
||||
* **release:** support default credentials and record 5.0.0 boundary ([#1012](https://github.com/morluto/rea/issues/1012)) ([6a278a6](https://github.com/morluto/rea/commit/6a278a68dd4fa69b9153697ecb7bba7de4c55c32))
|
||||
* **release:** sync 5.0.0 checkpoint back to main ([966cca8](https://github.com/morluto/rea/commit/966cca8efe5ed08b3285cd06409688d871eb5e6b))
|
||||
* **snapshot:** replay validated workflow evidence ([#984](https://github.com/morluto/rea/issues/984)) ([1d62e54](https://github.com/morluto/rea/commit/1d62e54e1f976e64130e4e9b651c3180aa4fa727))
|
||||
* **windows:** allow Java to read retained target snapshots ([#1027](https://github.com/morluto/rea/issues/1027)) ([28d5c39](https://github.com/morluto/rea/commit/28d5c397c83f901e4086083868041711371ad32a))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **artifacts:** replace quadratic ZIP overlap checks ([#1013](https://github.com/morluto/rea/issues/1013)) ([76e7e51](https://github.com/morluto/rea/commit/76e7e510bb2d7d37728ca50749d524beb5890cf8))
|
||||
* **javascript:** cache canonical export comparison sort keys ([#1028](https://github.com/morluto/rea/issues/1028)) ([63ba742](https://github.com/morluto/rea/commit/63ba7424c0ab69b94397e8913f8e231de111583f))
|
||||
|
||||
|
||||
### Code Refactoring
|
||||
|
||||
* **build:** generate MCP tool catalog at build time instead of committing it ([#997](https://github.com/morluto/rea/pull/997)) ([35a4e18](https://github.com/morluto/rea/commit/35a4e18d3028014e24d2e209ec99a590fde63152))
|
||||
|
||||
* **process:** separate capture implementation from analyst workflows ([#988](https://github.com/morluto/rea/pull/988)) ([6aa9afb](https://github.com/morluto/rea/commit/6aa9afb5493a97801eb1126a0dcf11f9705165db))
|
||||
|
||||
* **artifacts:** move inventory and extraction into provider owners ([#1032](https://github.com/morluto/rea/issues/1032)) ([6e4535c](https://github.com/morluto/rea/commit/6e4535c1011f1982e9ab07b7ebcd600c19cd494d))
|
||||
* remove abandoned test-only analysis scaffolding ([c93422a](https://github.com/morluto/rea/commit/c93422a898fdbedb7d28709144b4337a2669440b))
|
||||
|
||||
|
||||
### Documentation
|
||||
|
||||
* improve Korean wording and particles ([#1047](https://github.com/morluto/rea/issues/1047)) ([c5d165b](https://github.com/morluto/rea/commit/c5d165bd03fed95a0b0105661d3ba7e0efbfa449))
|
||||
* simplify the English README and organize guides ([#1043](https://github.com/morluto/rea/issues/1043)) ([28f10cf](https://github.com/morluto/rea/commit/28f10cf2bc7b7c590700b50d47072015f3e3c66e))
|
||||
|
||||
|
||||
### Tests
|
||||
|
||||
* **process:** wait for observable snapshot ctime changes ([#999](https://github.com/morluto/rea/pull/999)) ([161ed91](https://github.com/morluto/rea/commit/161ed9157134d5c3b0fbae19ceb756ac58a26c04))
|
||||
|
||||
* **browser:** consolidate CDP workflows and colocate parser coverage ([2682904](https://github.com/morluto/rea/commit/26829045d0f0e744bb60d6babc2dc3312f4bfa53))
|
||||
* **ci:** stabilize browser source identity fixture delivery ([#1023](https://github.com/morluto/rea/issues/1023)) ([ed725da](https://github.com/morluto/rea/commit/ed725dace4a689e4e5c804dc6bf06c7fc663c7c9))
|
||||
* **cli:** reduce repeated startup and filesystem setup ([d88d6c1](https://github.com/morluto/rea/commit/d88d6c1897740ccbd6c57f4ba01861fc751fb3a8))
|
||||
* consolidate service and process fixtures ([f8a53a8](https://github.com/morluto/rea/commit/f8a53a8c065f1ccbdaa051a4e1ef267a2047d759))
|
||||
* **javascript:** consolidate source analysis and export regressions ([f517a5a](https://github.com/morluto/rea/commit/f517a5a1d7c0766ed50f8dfd5ee69e65525ac744))
|
||||
* **javascript:** cover plain long-line loader identities ([#1025](https://github.com/morluto/rea/issues/1025)) ([984fc27](https://github.com/morluto/rea/commit/984fc27ce19d57f708bdae688a258cf8663cdea6))
|
||||
* **mcp:** consolidate SDK schema checks and scenario ports ([9c3479e](https://github.com/morluto/rea/commit/9c3479ed95100d8ee5ed4d9158a4550e4ac2a752))
|
||||
* **providers:** verify framing through client socket boundaries ([1fd1989](https://github.com/morluto/rea/commit/1fd1989d342375a01b6d14b282ad58001b7d1800))
|
||||
* prune redundant conformance checks and document test priorities ([aafdf8a](https://github.com/morluto/rea/commit/aafdf8a01b42fb6fd927c0f2a66e758e1a510abc))
|
||||
|
||||
## [5.0.0](https://github.com/morluto/rea/compare/rea-agents-4.1.0...rea-agents-5.0.0) (2026-10-07)
|
||||
|
||||
|
||||
|
||||
+77
-228
@@ -1,130 +1,16 @@
|
||||
# Contributing to REA
|
||||
|
||||
REA welcomes focused bug fixes, documentation improvements, tests, and reverse-engineering workflows. For large contract or architecture changes, explain the intended scope and link the relevant issue or design discussion.
|
||||
REA welcomes focused bug fixes, documentation improvements, tests, and reverse-engineering workflow enhancements. Open an issue before a large contract or architecture change so its scope can be agreed before implementation.
|
||||
|
||||
## Choosing a contribution
|
||||
|
||||
Explain the concrete problem, keep related corrections together, and report
|
||||
verification and unavailable coverage. Contributions should justify their review
|
||||
and maintenance cost. Keep cosmetic polish with useful fixes; confusing or
|
||||
incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For capability organization and provider composition, use the
|
||||
[architecture map](docs/architecture.mermaid). Production factories in
|
||||
`src/composition/` construct fresh providers without starting engines or acquiring
|
||||
targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
|
||||
|
||||
For analysis workflows and agent interfaces, follow the
|
||||
[performance priorities](AGENTS.md#performance). Explain the selected output
|
||||
scope, reuse/cache decisions, and relevant before/after measurements when these
|
||||
costs change.
|
||||
For capability organization and provider composition, follow the incremental
|
||||
[migration guide](docs/capability-migration.md). Run `npm run verify:test-discovery`
|
||||
after adding or moving tests.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
|
||||
## Contribution scope
|
||||
|
||||
REA focuses on software structure, runtime behavior, and evidence through
|
||||
general-purpose reverse-engineering workflows. Evaluate a proposal by its actual
|
||||
capabilities, side effects, defaults, and complete workflow. A tool name or a
|
||||
reference to vulnerability research, malware analysis, or CTFs does not determine
|
||||
whether it belongs in REA.
|
||||
|
||||
Supported contributions include format inspection, ordinary archive and standard
|
||||
compression extraction, disassembly, decompilation, cross-references, general
|
||||
static deobfuscation, vulnerability analysis, and evidence reporting. Static
|
||||
protection detection is distinct from defeating that protection. Security fixes
|
||||
and isolated regression tests remain in scope.
|
||||
|
||||
Runtime investigation needs explicit boundaries. Launching or attaching to a
|
||||
process, injecting instrumentation, running scripts, and changing memory or
|
||||
registers require review of target selection, execution and mutation authority,
|
||||
provenance, resource limits, cancellation, and cleanup. General Frida or debugger
|
||||
support is not inherently read-only, and existing support does not authorize
|
||||
every workflow built on it. General-purpose interfaces are not classified as
|
||||
dedicated bypass workflows solely because caller-provided code could be misused;
|
||||
their execution and mutation authority must still be reviewed.
|
||||
|
||||
REA does not integrate or officially distribute:
|
||||
|
||||
- Purpose-built workflows to defeat commercial packing or code-virtualization
|
||||
protections and automatically recover protected code.
|
||||
- Anti-debugging or anti-instrumentation evasion, or DRM, license, anti-cheat, or
|
||||
access-control bypass workflows.
|
||||
- Exploit or attack-payload automation, credential theft, covert persistence,
|
||||
unauthorized access, or bulk attack workflows.
|
||||
|
||||
These boundaries apply to first-party providers, adapters, plugins, bundled
|
||||
scripts, defaults, examples, and documentation. Moving a capability into an
|
||||
optional dependency or plugin does not exempt it. Review what the integration
|
||||
exposes, including upstream defaults and raw-command or script execution paths.
|
||||
Ordinary extraction and general static deobfuscation are not excluded merely
|
||||
because the input is packed or obfuscated; assess added bypass or execution
|
||||
behavior separately.
|
||||
|
||||
This is a product and maintenance boundary, not a judgment about a contributor's
|
||||
intentions or a declaration that all excluded research is unlawful. Contributors
|
||||
must have the rights and authorization needed for their work. A local-only design
|
||||
or an "authorized use only" disclaimer does not establish containment or settle
|
||||
legal and scope questions.
|
||||
|
||||
### Fail-closed integration
|
||||
|
||||
If scope fit, target or authorization boundaries, side effects, or the verification
|
||||
needed to establish those boundaries remain materially unresolved, **do not merge,
|
||||
enable, or include the capability in official packages or distributions**. Keep
|
||||
the proposal pending, narrow it, or decline it. Record unknowns explicitly; do not
|
||||
turn them into "none" or assumed safety. Passing CI, completed implementation,
|
||||
upstream popularity, or silence from maintainers is not scope approval.
|
||||
|
||||
For a new provider or a capability involving target execution, injection,
|
||||
mutation, or protection recovery, seek an explicit maintainer scope decision in
|
||||
the linked issue or design discussion before substantial implementation intended
|
||||
for integration. Small exploratory prototypes and draft PRs may help inform that
|
||||
decision; they do not authorize integration. The decision must cover the final
|
||||
proposed behavior; a scope expansion needs a new decision. Routine fixes and
|
||||
documentation changes within existing boundaries do not need a separate advance
|
||||
decision, but must not silently expand those boundaries.
|
||||
|
||||
### Starting a proposal
|
||||
|
||||
To open a feature request or design discussion, describe the problem and desired
|
||||
outcome or decision. Short answers and the forms' "Not sure yet" options are
|
||||
welcome. You do not need code, a complete design or test plan, legal analysis, or
|
||||
private proof-of-authorization documents to start a discussion. Share what you
|
||||
know and leave optional details blank when you need help.
|
||||
|
||||
Form selections guide triage; they are not proof of authorization or scope
|
||||
approval. Unselected effects must not be treated as established absence.
|
||||
Maintainers and implementers work together to resolve the following before
|
||||
integration:
|
||||
|
||||
- The concrete analyst question, requested inputs and outputs, and why existing
|
||||
REA operations are insufficient.
|
||||
- Why the workflow fits this scope, its non-goals, and any protection recovery,
|
||||
bypass, or attack automation it would expose.
|
||||
- The selected targets and every execution, injection, mutation, network, or
|
||||
persistence effect, including behavior inherited from upstream tools.
|
||||
- How the boundaries and outcome will be verified with self-owned or appropriately
|
||||
licensed samples; identify real CLI/MCP and provider coverage, cleanup and
|
||||
resource checks, and remaining unknowns. Do not publish private binaries,
|
||||
credentials, or sensitive target data; follow [SECURITY.md](SECURITY.md) for
|
||||
private vulnerability reports.
|
||||
|
||||
Mixed proposals should separate acceptable analysis capabilities from excluded
|
||||
workflows so each can be decided independently. An issue may discuss unresolved
|
||||
questions; opening an issue or PR does not authorize integration. This is a
|
||||
maintainer integration gate, not a per-call confirmation requirement or a sandbox
|
||||
guarantee.
|
||||
|
||||
## Development setup
|
||||
|
||||
Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/installation.md#start-setup).
|
||||
REA development requires Node.js 24.18.x and npm 11.16.x (pinned toolchain via `nvm use`; the supported runtime range is Node.js ^22.19 || ^24.11 || >=26, as the README badge states). Real-Hopper verification additionally requires either macOS 12+ or an officially supported Linux host (Ubuntu 24.04+, Fedora 41+, 64-bit Arch, or CachyOS) and an installed Hopper application. Linux demo verification uses its own private Xvfb display and does not require a desktop session. Run `nvm use` before installing dependencies.
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
@@ -133,29 +19,12 @@ npm run check:fast
|
||||
|
||||
`npm ci` installs the exact dependencies and prepares the Husky hooks without
|
||||
building the project. Run `npm run build:cached` when you need the standalone
|
||||
CLI or MCP server; it compiles the runtime and bundles the authored skill.
|
||||
`npm run build` forces those tasks to rerun. Neither command generates test
|
||||
catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
CLI or MCP server; `npm run build` remains the uncached compiler leaf used by
|
||||
Turbo. Turbo caches deterministic builds and static checks across Git
|
||||
worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
### Windows source checkouts
|
||||
|
||||
The published npm package includes the Windows x64 native addon and its
|
||||
manifest. A source checkout does not: `native/windows/build/` is ignored by
|
||||
Git, and neither `npm ci` nor `npm run build:cached` (or `npm run build`)
|
||||
produces `rea-windows-x64.node` or `manifest.json` in that directory.
|
||||
Windows workflows and tests that require native controls need these artifacts;
|
||||
without them, those controls are unavailable even after a successful runtime build.
|
||||
|
||||
Follow the [Windows native build guide](native/windows/README.md#build) before
|
||||
running those workflows from source. The artifact build requires Linux,
|
||||
MinGW-w64, and `tar`; the build script does not run on Windows. See that guide
|
||||
for the build and artifact-verification commands and the separate real Windows
|
||||
verification prerequisites. Keep the generated addon and manifest uncommitted.
|
||||
Keep dependencies flowing inward through the existing domain, contracts, provider, application, server, and adapter layers. Parse unknown values at process and protocol boundaries, model expected failures with `Result`, and preserve the canonical tool inventory defined by `TOOL_CONTRACTS` unless a deliberate contract change updates every verifier, generated catalog artifact, and snapshot. Keep tool discovery complete and report capability- and session-scoped availability through `binary_session`.
|
||||
|
||||
## Documentation website
|
||||
|
||||
@@ -166,121 +35,101 @@ links, and `npm run docs:preview` to preview that build at `/rea/`.
|
||||
Site navigation lives in `docs/.vitepress/config.ts`. Keep links to guides
|
||||
relative so they work on GitHub and the website; link to repository files
|
||||
outside `docs/` using their full GitHub URLs. Generated reference documents
|
||||
use `npm run docs:generate`; `docs:build` generates them before building the website.
|
||||
still use `npm run docs:generate`; `docs:build` only builds the website.
|
||||
|
||||
Pull requests run `npm run docs:check`. `.github/workflows/pages.yml` is a
|
||||
manual VitePress build and does not publish. The public site at
|
||||
<https://rea.tools/> serves `website/public/` through Cloudflare Workers. See
|
||||
the [website README](website/README.md#cloudflare-workers) for previews and
|
||||
manual deployments.
|
||||
|
||||
The separate GitHub Pages host uses only the manual
|
||||
`.github/workflows/website-pages.yml` workflow on `main`. The repository's
|
||||
**Settings → Pages → Build and deployment → Source** must be set to
|
||||
**GitHub Actions** before deploying there.
|
||||
<https://morluto.github.io/rea/> is published only by the manual website
|
||||
workflow on `main`. The repository's **Settings → Pages → Build and
|
||||
deployment → Source** must be set to **GitHub Actions** before the first
|
||||
deployment.
|
||||
|
||||
## Development feedback and PR verification
|
||||
|
||||
Select checks for the behavior changed. For a source edit:
|
||||
For an ordinary edit, run the relevant regression and cached static checks:
|
||||
|
||||
```bash
|
||||
npm run test:focused -- src/config.test.ts
|
||||
npm run check:fast
|
||||
```
|
||||
|
||||
`test:local` runs source tests without building. `check:changed` combines cached
|
||||
static checks with source tests affected since the merge base with `origin/main`.
|
||||
Import-graph selection can miss runtime registration, bridges, and generated data;
|
||||
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
|
||||
for commands, test placement, pruning, and real-provider prerequisites.
|
||||
`test:local` selects dirty source tests without building; explicit source test
|
||||
paths run even on a clean tree. `check:changed` adds source tests affected since
|
||||
the branch merge base with `origin/main`. Use `npm run test:changed -- --base
|
||||
REVISION` to choose another base. Changed-test selection follows the import
|
||||
graph and is feedback, not complete correctness evidence. Inspect relevant
|
||||
boundary and provider behavior explicitly; see [docs/testing.md](docs/testing.md).
|
||||
|
||||
Before requesting review, run relevant tests and `npm run check`; add
|
||||
`npm run docs:check` when contracts or generated metadata change. Record the
|
||||
checks and unavailable coverage. CI owns the full deterministic suite and
|
||||
coverage. `npm run check:pr` is an optional full local gate for broad changes or
|
||||
CI diagnosis. Packaging/setup/distribution changes additionally require
|
||||
`npm run verify:package` and `npm pack --dry-run`; provider changes require the
|
||||
matching real-provider lane. State which host/provider workflows ran.
|
||||
Before handing off a PR, run the relevant tests, `npm run check`, and
|
||||
`npm run docs:check` when contracts or generated metadata change. Record which
|
||||
checks ran. CI owns complete deterministic tests and aggregate coverage. Use
|
||||
`npm run check:pr` for a deliberate full local gate on broad changes or when
|
||||
investigating CI failures; it is not required after each edit, rebase, or push.
|
||||
Packaging, setup, installation, or distribution changes also require
|
||||
`npm run verify:package` and `npm pack --dry-run`. Provider behavior changes
|
||||
require the matching real-provider `verify:*` lane.
|
||||
|
||||
For CI/test correctness, drift, or performance investigations, use the
|
||||
[`rea-ci-tests-audit` contributor skill](.agents/skills/rea-ci-tests-audit/SKILL.md).
|
||||
It guides reproduction and comparable measurements before changing checks.
|
||||
`check:fast` runs cached typecheck and lint, reporting diagnostics on failure.
|
||||
`check` adds formatting, dead-code, and package-metadata freshness checks.
|
||||
Formatting uses Oxfmt and the committed `.oxfmtrc.json`; generated sources use
|
||||
the same configuration. Pre-commit formats and lints staged files; pre-push runs
|
||||
`check:fast`.
|
||||
`docs:check` checks committed generated metadata. `docs:generate` regenerates
|
||||
those files, and the docs CI lane checks them.
|
||||
Real-provider execution remains uncached; deterministic builds use Turbo.
|
||||
|
||||
`check:pr` reuses build and static-check outputs while executing the test suite
|
||||
fresh, including the fast Python producer checks. `docs:check` also runs the
|
||||
onboarding example consistency suite for README-only changes.
|
||||
Local `npm test` runs every deterministic Vitest project without coverage or
|
||||
retries. CI runs four coverage shards and merges JUnit and timing reports.
|
||||
CI cancels superseded PR runs and skips package, Windows, and full test lanes
|
||||
for documentation-only PRs. Coverage thresholds remain in `vitest.config.ts`.
|
||||
|
||||
Pre-commit formats and lints staged files; pre-push runs `check:fast`.
|
||||
Tests that need a temporary directory must use
|
||||
`createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`. The
|
||||
helper binds exact-path, awaited cleanup to the current Vitest case, including
|
||||
failure and timeout completion. Run `npm run verify:test-temp-hygiene` to build
|
||||
REA, execute the complete suite under a fresh `TMPDIR`, and reject any remaining
|
||||
REA-owned temporary path. Never add a glob cleanup for shared `/tmp/rea-*`
|
||||
content.
|
||||
|
||||
### Generated files and command ownership
|
||||
Set `REA_LOG_LEVEL` to `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent` to control structured JSON diagnostics. MCP mode defaults to `info` and
|
||||
always writes logs to stderr so the stdio protocol remains intact. One-shot CLI
|
||||
logging is opt-in and writes to stdout when a level is configured, preserving
|
||||
machine-readable command output by default. Request arguments, bridge
|
||||
authentication tokens, and environment data are redacted.
|
||||
|
||||
`docs:generate` prepares documentation and portable managed evidence;
|
||||
`evidence:generate` prepares only the managed commitment and its runtime/skill
|
||||
dependencies. The ignored outputs are `docs/public/product-catalog.json`,
|
||||
`docs/verification/managed-conformance-*.json`, and `skills/`. Edit authored
|
||||
skill instructions in `.agents/skills/reverse-engineer-anything/`, where Skills
|
||||
can discover the public bundle directly from a clean checkout. `verify:package`
|
||||
installs that bundle with Skills 1.7.2 using default repository-root discovery
|
||||
and checks every installed file, alongside the packaged setup checks.
|
||||
CI validates generated outputs and retains them as artifacts; do not commit
|
||||
them. Reviewed source metadata, including
|
||||
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json`, remains
|
||||
tracked and checked for freshness.
|
||||
Changes that claim real Hopper behavior must also be tested against the
|
||||
source-owned, digest-bound conformance manifest. The verifier builds the
|
||||
platform-native fixtures before starting Hopper:
|
||||
|
||||
The packaged skill contains instructions and inventory metadata, not runtime
|
||||
schema hashes. Its conformance commitment covers that exact bundle and portable
|
||||
verification; it does not attest optional real-provider runs. Schema-only changes
|
||||
should not alter that commitment. Doctor and `binary_session` expose runtime
|
||||
schema identity.
|
||||
```bash
|
||||
npm run verify:hopper
|
||||
```
|
||||
|
||||
Build/generation commands hold a SQLite transaction around the entire Turbo
|
||||
graph, including cache restoration. `.cache/rea-command-locks/` databases identify
|
||||
locks, not stale files; never remove them while commands run. POSIX cancellation
|
||||
retains process-group ownership until settlement; Windows waits for completion.
|
||||
Real-provider execution is uncached.
|
||||
On a self-hosted Linux runner with the setup-installed Xvfb dependencies, use:
|
||||
|
||||
### Temporary files and diagnostics
|
||||
```bash
|
||||
npm run verify:hopper:linux
|
||||
```
|
||||
|
||||
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
|
||||
for exact-path, awaited cleanup, including failures and timeouts.
|
||||
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
|
||||
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
|
||||
Set `REA_HOPPER_CONFORMANCE_MANIFEST_PATH` only to verify another source-built
|
||||
manifest. The normal commands use `build/conformance/manifest.json`; generated
|
||||
fixtures and manifests remain ignored and must not be committed.
|
||||
|
||||
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
|
||||
is opt-in, so stdout carries only protocol messages and command results.
|
||||
Request arguments, bridge tokens, and environment data are redacted from
|
||||
diagnostic logs.
|
||||
The macOS and Linux real-Hopper workflows remain separate so a successful mock or package test cannot be reported as platform-runtime proof. Pull requests changing setup, launch, bridge, or Hopper behavior must state which real workflows ran and why either workflow was unavailable.
|
||||
|
||||
## Real Hopper changes
|
||||
|
||||
Run `npm run verify:hopper` on a supported macOS host or
|
||||
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
|
||||
native fixtures and verify the installed engine through CLI/MCP; package or
|
||||
simulated-provider checks do not establish engine behavior. See
|
||||
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
|
||||
for supported hosts and unattended startup.
|
||||
|
||||
Use the default `build/conformance/manifest.json` unless explicitly verifying
|
||||
another source-built manifest with `REA_HOPPER_CONFORMANCE_MANIFEST_PATH`.
|
||||
Generated fixtures and manifests remain ignored.
|
||||
Describe the behavior change and verification performed in the pull request. Never commit binaries, Hopper documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Maintainer release checklist
|
||||
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
|
||||
The Release workflow validates and tags that merge; both publishers build its
|
||||
exact SHA. Ordinary main pushes cannot publish packages. An optional manual
|
||||
`release/VERSION` checkpoint retains explicit preparation and publication.
|
||||
|
||||
For grouped release notes with verified contributor thanks, use the repository's
|
||||
`rea-changelog-update` skill in `.agents/skills/rea-changelog-update/`. Its inventory,
|
||||
credit checks, and finalization process are described in the release guide.
|
||||
Use the [checkpoint release guide](docs/releasing.md). Releases start from an
|
||||
explicit `release/VERSION` branch cut at a recorded commit. Manually prepare
|
||||
the bot PR, wait for its exact-head CI and review, merge into that frozen
|
||||
branch, and manually publish through the official Release workflow. Main
|
||||
pushes do not update release PRs or publish packages. Both publishers build
|
||||
the exact SHA tagged by Release Please.
|
||||
|
||||
Keep new implementation commits on main for the next release. The workflow
|
||||
owns packaged-artifact verification, npm publication, the published CLI/MCP
|
||||
canary, and MCP Registry publication. Sync manual checkpoint metadata back to
|
||||
main after publication; see the guide for recovery and verification.
|
||||
canary, and MCP Registry publication. Sync release metadata back to main after
|
||||
publication; see the guide for partial-publication recovery and verification.
|
||||
|
||||
-136
@@ -1,136 +0,0 @@
|
||||
# Disclaimer
|
||||
|
||||
## Responsible use
|
||||
|
||||
REA is built for lawful analysis: security research, malware and incident
|
||||
analysis, vulnerability research and coordinated disclosure, CTFs and education,
|
||||
interoperability, software preservation, and debugging or auditing software you
|
||||
own or are authorized to examine.
|
||||
|
||||
The REA project does not support, and will not help with:
|
||||
|
||||
- circumventing DRM, license checks, anti-cheat systems or other technical
|
||||
protection measures;
|
||||
- developing game cheats or tampering with online services;
|
||||
- bypassing authentication, bot protection or access controls on systems you are
|
||||
not authorized to test;
|
||||
- copying proprietary software or reproducing its protected code and assets;
|
||||
- evading the usage policies or safety measures of AI model providers;
|
||||
- reselling or sharing AI accounts or API access.
|
||||
|
||||
You are responsible for obtaining any required authorization and for complying
|
||||
with applicable laws, license agreements and terms of service. Reverse
|
||||
engineering may be restricted in your jurisdiction or by the license of the
|
||||
software you analyze.
|
||||
|
||||
Requests for help with the uses above will be closed without response in issues
|
||||
and discussions, and may result in removal from the REA community.
|
||||
|
||||
## User responsibility
|
||||
|
||||
Do not use REA for unauthorized access or exploitation, credential theft,
|
||||
malicious deployment of malware, unlawful collection or disclosure of data,
|
||||
unauthorized service disruption, or unlawful circumvention of access controls or
|
||||
licensing restrictions. Legitimate security research and malware analysis must
|
||||
remain within the rights and permissions applicable to the work.
|
||||
|
||||
You are responsible for your use of REA, including commands, configurations,
|
||||
scripts, and actions you direct an AI agent to perform. Establish the legal
|
||||
authority and scope required for the targets, accounts, devices, software, data,
|
||||
and operations involved. Comply with applicable laws, contracts,
|
||||
intellectual-property rights, privacy obligations, and tool licenses. Public
|
||||
availability of software, possession of a file, or access to a debugging
|
||||
endpoint does not by itself establish permission for every operation.
|
||||
|
||||
## Runtime, network, and device operations
|
||||
|
||||
Some REA workflows execute or observe programs, attach to runtime or browser
|
||||
endpoints, capture network traffic, interact with applications, or operate on
|
||||
connected devices. These actions can affect application state, files, accounts,
|
||||
devices, and services. Authorization to inspect an artifact does not necessarily
|
||||
authorize execution, interception, device changes, or interaction with another
|
||||
party's system.
|
||||
|
||||
Review the operations you request, including agent-generated plans and scripts,
|
||||
and use an appropriate environment for untrusted targets. You are responsible
|
||||
for protecting credentials, personal data, captured traffic, and investigation
|
||||
artifacts, and for obtaining any permission required to collect, retain,
|
||||
process, or share them. Local operation does not establish authorization or
|
||||
guarantee that executed targets and third-party tools have no external effects.
|
||||
|
||||
## Analysis results and generated output
|
||||
|
||||
Decompilation, reconstruction, heuristics, and AI-assisted analysis may be
|
||||
incomplete or incorrect. Results do not guarantee accuracy, completeness,
|
||||
security, compatibility, or legal compliance. Review the underlying evidence and
|
||||
verify consequential conclusions before relying on them.
|
||||
|
||||
Obtaining source-like output or an analysis report does not grant rights to the
|
||||
original program, its data, or the generated output. Check the target's rights
|
||||
and the producing tool's applicable terms before publishing, sharing, or using
|
||||
results in another product or service.
|
||||
|
||||
## Third-party tools and licenses
|
||||
|
||||
Third-party components and analysis tools retain their own licenses, copyright
|
||||
notices, and usage conditions. Consult
|
||||
[Third-party notices](THIRD_PARTY_NOTICES.md) for their roles, licensing
|
||||
records, and preserved upstream texts.
|
||||
|
||||
You must legally obtain IDA Pro and Hex-Rays decompilers, Hopper, JEB, and any
|
||||
other proprietary tools, together with the rights required for your use. REA
|
||||
provides no commercial-tool license, key, or activation entitlement. Permitted
|
||||
trials and demos remain subject to vendor terms. An open-source bridge or
|
||||
accessible API does not grant additional user, machine, feature, automation,
|
||||
hosting, or output-sharing rights. Cracked copies, stolen keys, unauthorized key
|
||||
sharing, and bypassing license restrictions are not authorized by REA.
|
||||
|
||||
## Disclaimer of warranty
|
||||
|
||||
REA is provided **AS IS**, without express or implied warranties, including
|
||||
merchantability, fitness for a particular purpose, and noninfringement, as set
|
||||
out in the [MIT License](LICENSE). You assume the risks of using the software
|
||||
and deciding whether it is suitable for your task. No project description,
|
||||
example, analysis result, or third-party integration guarantees a particular
|
||||
outcome or establishes that your intended use is lawful or safe.
|
||||
|
||||
## Limitation of liability
|
||||
|
||||
The warranty and liability provisions of the [MIT License](LICENSE) govern REA's
|
||||
authors and copyright holders, subject to applicable law. That license disclaims
|
||||
liability for claims, damages, and other liability arising from or in connection
|
||||
with the software or its use, whether in contract, tort, or otherwise. This
|
||||
document does not expand those provisions or override rights and obligations
|
||||
that cannot be excluded under applicable law.
|
||||
|
||||
## Disclaimer of endorsement
|
||||
|
||||
References to third-party projects, contributors, products, services, and
|
||||
trademarks identify their work and REA's integrations. Credits and thanks do not
|
||||
imply affiliation, sponsorship, approval, or endorsement by the named parties,
|
||||
including the National Security Agency and commercial tool vendors.
|
||||
Redistribution, bundling, or hosting by a third party does not by itself imply
|
||||
affiliation with or endorsement by the REA project.
|
||||
|
||||
## Relationship to software licenses
|
||||
|
||||
This document explains REA's intended use, user responsibilities, and community
|
||||
support boundaries. It does not amend the MIT license, impose additional
|
||||
copyright-license conditions, replace upstream licenses, or grant rights to
|
||||
third-party software or targets. The applicable software licenses and agreements
|
||||
retain their own terms.
|
||||
|
||||
## Versions, distribution channels, and platforms
|
||||
|
||||
The project's position on lawful use and non-endorsement of illegal or
|
||||
unauthorized activity covers all REA versions, distribution forms, and
|
||||
platforms. This includes historical releases and Git commits, npm packages,
|
||||
bundled or separately distributed REA skills, source checkouts, release
|
||||
archives, and redistributed copies, regardless of installation method, agent
|
||||
client, operating system, or platform. Using an older version or another
|
||||
distribution channel does not imply project endorsement or remove obligations
|
||||
under applicable law or to respect third-party rights.
|
||||
|
||||
Each copy remains subject to the licenses accompanying that copy. This statement
|
||||
does not retroactively amend existing license grants, add license conditions, or
|
||||
expand historical warranty or liability exclusions.
|
||||
+341
@@ -0,0 +1,341 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **العربية**
|
||||
|
||||
# REA: اعكس هندسة أي شيء
|
||||
|
||||
### وسيط MCP واحد للهندسة العكسية عبر الملفات التنفيذية والتطبيقات وسلوك وقت التشغيل.
|
||||
|
||||
**اعثر على ميزة تعجبك. افهم آلية عملها. ابنها بالطريقة التي تريدها.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#منصة-أدوات-التحقيق)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[الموقع (بالإنجليزية)](https://morluto.github.io/rea/) · [أدلة الاستخدام](https://morluto.github.io/rea/guides/) · [دراسة حالة DX-Ball](https://morluto.github.io/rea/showcase/dx-ball/)**
|
||||
|
||||
[البدء السريع](#البدء-السريع) · [حالة الدعم](#حالة-الدعم) · [من الملف التنفيذي إلى السلوك](#من-الملف-التنفيذي-إلى-السلوك) · [منصة أدوات التحقيق](#منصة-أدوات-التحقيق) · [خطة العمل](#خطة-العمل) · [كيف يعمل؟](#كيف-يعمل)
|
||||
|
||||
<table aria-label="REA community">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="docs/assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>انضم إلى مجتمع الهندسة العكسية</strong>
|
||||
</a><br />
|
||||
<sub>Discord · أسئلة وأجوبة · شارك أعمالك</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="REA يشغّل جسر التحليل داخل Hopper لفحص ملف تنفيذي أصلي" width="1200" />
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
هل وجدت في تطبيق ميزة تريدها في منتجك؟ اطلب من وكيلك استقصاءها باستخدام REA. يستطيع فحص التطبيق من دون شيفرته المصدرية، وشرح طريقة عمل الميزة وعرض الأدلة، ثم بناء نسخة تناسب مشروعك.
|
||||
|
||||
يوفر REA أدوات لتحليل الملفات التنفيذية الأصلية وتطبيقات JavaScript وElectron وتجميعات .NET والمواقع. يمكنك استخدامها من وكيلك أو من الطرفية. يعمل التحليل محليًا، وتتضمن النتائج الأدلة والقيود.
|
||||
|
||||
يضبط Setup إعدادات الوكيل ويربطه بتثبيت موجود من Hopper أو Ghidra، ويثبّت سير عمل REA الموجّه. ويمكنه تثبيت Hopper بعد موافقتك. ولا يحتاج التحليل الثابت لتطبيقات JavaScript إلى أي محرك.
|
||||
|
||||
## اطلب من وكيلك مباشرة
|
||||
|
||||
بعد [الإعداد](#البدء-السريع)، أعد تشغيل الوكيل ثم اطلب:
|
||||
|
||||
```text
|
||||
استقصِ طريقة عمل البحث في تطبيق Notes، واعرض الأدلة، ثم ابنِ ميزة مشابهة لمشروعي.
|
||||
```
|
||||
|
||||
استبدل Notes بالتطبيق الذي تريد فهمه، أو اطلب نظرة عامة أولًا.
|
||||
|
||||
## من الملف التنفيذي إلى السلوك
|
||||
|
||||
| فك الترجمة | الفهم | إعادة الإنشاء |
|
||||
| ---------------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| استخرج الشيفرة شبه المصدرية، والتعليمات، والسلاسل النصية، والرموز | تتبع تدفق التحكم والمراجع المتبادلة والاستدعاءات والأنواع | حوّل ما تعلمه الوكيل إلى ميزة تناسب تقنياتك وواجهتك ومتطلباتك |
|
||||
|
||||
يوضح REA كيف وصل إلى نتائجه. وهو لا يدّعي استعادة الشيفرة المصدرية الأصلية أو استنساخ التطبيق تلقائيًا.
|
||||
|
||||
## لماذا REA؟
|
||||
|
||||
| | |
|
||||
| ------------------------ | ---------------------------------------------------------------------------- |
|
||||
| **مصمم للوكلاء** | اسأل عما يفعله التطبيق ودع الوكيل يفحصه بدلًا من التخمين. |
|
||||
| **CLI وMCP** | استخدم قدرات الهندسة العكسية نفسها من الطرفية أو الوكيل. |
|
||||
| **إعداد موجّه** | اضبط وكيلك، واربط أداة تحليل موجودة، أو ثبّت Hopper بموافقتك. |
|
||||
| **من الفهم إلى الشيفرة** | افهم الميزة ثم ابنِ نسختك الخاصة في جلسة البرمجة نفسها. |
|
||||
| **محلي حسب التصميم** | يجري التحليل على مضيفك المحلي المدعوم، ولا يرفع REA التطبيق إلى خدمة مستضافة. |
|
||||
| **يحافظ على السياق** | استقصِ عدة تطبيقات دون بدء التحليل من جديد عند كل سؤال. |
|
||||
|
||||
## البدء السريع
|
||||
|
||||
### تشغيل الإعداد (موصى به)
|
||||
|
||||
اربط REA بوكيلك:
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
```
|
||||
|
||||
يعرض Setup أولاً قائمة متعددة الاختيار للوكلاء الذين تريد ربطهم. تكون تسجيلات REA الموجودة محددة افتراضياً، ولا يؤدي اكتشاف عميل جديد وحده إلى تحديده؛ ويمكنك اختيار عميل غير مهيأ. راجع المسارات والتغييرات المحددة قبل الموافقة. يثبت Setup سير عمل REA للوكلاء المحددين افتراضياً. Hopper خيار منفصل يتطلب موافقة مستقلة، ويمكن أيضاً حفظ مسارات تثبيت Ghidra الموجودة.
|
||||
|
||||
يعرض التغييرات قبل تطبيقها وينشئ نسخة احتياطية من الإعدادات الموجودة. راجع [دليل التثبيت والإعداد](docs/installation.md) للمتطلبات والخيارات الإضافية.
|
||||
|
||||
### الاستخدام مع وكيل
|
||||
|
||||
بعد الإعداد، أعد تشغيل الوكيل واشرح التطبيق أو الميزة التي تريد استقصاءها. يدعم REA Claude Code وClaude Desktop وCodex وCursor وGemini CLI وWindsurf وDevin وOpenCode وAntigravity وGitHub Copilot CLI وCommand Code وVS Code. تكون تسجيلات REA الموجودة محددة افتراضياً، ويجب اختيار العملاء المكتشفين الآخرين يدوياً. يمكن للوكلاء الآخرين استخدام إعداد MCP أدناه.
|
||||
|
||||
يمكن تشغيل Hopper في الوضع التجريبي. إذا ظهرت رسالة عند التشغيل الأول، فاختر التجربة أو أدخل ترخيصًا موجودًا.
|
||||
|
||||
### مهارة مساعد البرمجة (اختياري)
|
||||
|
||||
أضف المهارة إلى مساعد البرمجة للحصول على سياق أغنى:
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea --skill reverse-engineer-anything
|
||||
```
|
||||
|
||||
توفر المهارة سير عمل التحقيق في REA. نفّذ الإعداد أعلاه لربط REA بالوكيل وضبط أدوات التحليل. يثبّت الإعداد افتراضيًا مهارة مطابقة للإصدار؛ وهذا الأمر يثبّت نسخة المستودع.
|
||||
|
||||
لشجرة JavaScript/Electron المستخرجة أو ملف ASAR، لا يلزم إعداد MCP ولا محرك أصلي:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
استبدل المسار بهدفك (مثل `"D:/apps/example"` على Windows). يدعم سير العمل الثابت هذه المسارات، بينما يتطلب التحليل الأصلي محركه الخاص.
|
||||
|
||||
### تثبيت أمر rea
|
||||
|
||||
ثبت واجهة سطر الأوامر:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
```
|
||||
|
||||
يجب تثبيت Node.js وnpm مسبقًا. عند تشغيله من طرفية، يثبت البرنامج `rea` ويبدأ الإعداد.
|
||||
|
||||
يمكنك بدلًا من ذلك التثبيت باستخدام npm ثم تشغيل الإعداد:
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
```
|
||||
|
||||
### المتطلبات
|
||||
|
||||
- macOS 12 أو أحدث
|
||||
- Ubuntu 24.04+ أو Fedora 41+ أو Arch Linux بنواة 64 بت أو CachyOS
|
||||
- Node.js 22.x (>=22.19) أو 24.x (>=24.11) أو 26+ وnpm
|
||||
|
||||
يتطلب تحليل الملفات التنفيذية الأصلية Hopper أو Ghidra. Hopper برنامج منفصل، وللوضع التجريبي قيود يحددها المورّد، لكن الترخيص المدفوع ليس إلزاميًا.
|
||||
|
||||
يدعم Ghidra نظامي Linux x64 وmacOS x64/arm64. ثبت Ghidra 12.1.x وJDK الكامل بنواة 64 بت الذي يعلنه ذلك التثبيت (`application.java.min` حتى `application.java.max`) بصورة منفصلة، ثم اضبط REA لاستخدامهما. تتطلب إصدارات 12.1 الحالية JDK 21 أو أحدث ولا تحدد حدًا أعلى. جرى التحقق من الجسر مع Ghidra 12.1.4 وJDK 21. يحتاج macOS أيضًا إلى أداة فك الترجمة الأصلية المطابقة لمعمارية الجهاز.
|
||||
|
||||
يستطيع Setup التحقق من التثبيت وحفظ المسارات؛ ولا يثبت أو يحدّث Ghidra أو Java أو Node.js أو npm أو Homebrew.
|
||||
|
||||
يتضمن فرع main والحزمة npm 4.1.0 دعمًا تجريبيًا لـ Windows x64 Ghidra P0 لتطبيقات PE الأصلية x86-64 (غير المُدارة وغير DLL) على NTFS محلي، مع ضوابط Job Object وقوائم DACL الخاصة وقبول المسارات المجمّعة. تحقق من [حد الإصدار](docs/installation.md#released-package-and-main) قبل توقع ذلك من حزمة npm أقدم. راجع [دليل Windows Ghidra P0](docs/windows-ghidra-p0.md) للمتطلبات والنطاق الموثق.
|
||||
|
||||
### تشخيص المشكلات
|
||||
|
||||
يشخّص `npx -y rea-agents@latest doctor` الجهاز والاعتماديات وأدوات التحليل وإعدادات الوكيل من دون تغييرها. أضف `--json` للحصول على تشخيص منظم.
|
||||
|
||||
على Linux، يفضّل REA الملف التنفيذي `/opt/hopper/bin/Hopper`، وإذا لم يكن متاحًا يتحقق تلقائيًا من `~/.local/share/rea/hopper/bin/Hopper`. استخدم `HOPPER_LAUNCHER_PATH` للمسارات الأخرى. إذا كان الملف موجودًا لكن محرك التحليل لا يظهر، فشغّل `ldd /absolute/path/to/Hopper | grep 'not found'` على مسار Hopper الفعلي للتحقق من المكتبات الناقصة. راجع [دليل Hopper](docs/installation.md#hopper).
|
||||
|
||||
### التحديث والإزالة
|
||||
|
||||
- يحدّث `rea update` تثبيت REA الحالي.
|
||||
- يزيل `rea uninstall` تسجيلات الوكلاء وملفات سير العمل التي يديرها REA، ويترك Hopper مثبتًا.
|
||||
- يزيل `rea uninstall --purge-data` أيضًا ذاكرة REA المؤقتة وحالته. استخدمه فقط إذا أردت حذف هذه البيانات.
|
||||
|
||||
## حالة الدعم
|
||||
|
||||
تُوصف قدرات المستودع الحالية ومتطلبات الأنظمة في [دليل الدعم الإنجليزي](README.md#current-status). قد يسبق فرع main [إصدار npm](docs/installation.md#released-package-and-main).
|
||||
|
||||
- يوفر Ghidra 25 عملية للقراءة فقط على Linux x64 وmacOS x64/arm64 وحدود Windows x64 P0 التجريبية. يدعم Linux/macOS أيضًا تعليقات الدوال الذرية على مستوى الجلسة. نظام Windows P0 للقراءة فقط، ولا يوفر Ghidra أي تحكم في الواجهة.
|
||||
- يتطلب الفحص الثابت لنظام Android تجهيز JADX/Java بشكل منفصل؛ ويغطي التحقق بالموفر الحقيقي نظام Linux. راجع [تحليل Android](docs/android-analysis.md).
|
||||
- تحدد طلبات المتصفح وElectron والعمليات هدفها وإجراءاتها ودورة حياتها مباشرة، وتعتمد على وصول المضيف الفعلي دون منح أذونات منفصلة من REA. ما زالت كتابة إعدادات Setup وتثبيت Hopper تتطلب الموافقة على خطتها الدقيقة.
|
||||
- يصف `rea capabilities` عمليات جلسة الملف التنفيذي لا كل أدوات التطبيقات ووقت التشغيل. راجع قائمة أدوات MCP الكاملة والدليل ذا الصلة.
|
||||
|
||||
## استقصاء كامل بطلب واحد
|
||||
|
||||
بعد الإعداد، يمكنك أن تطلب من وكيلك:
|
||||
|
||||
> افتح التطبيق، واكتشف آلية عمل البحث من دون اتصال، واشرح تدفق التحكم، ثم ابنِ نسخة لمشروعي باستخدام TypeScript وSQLite.
|
||||
|
||||
يمكن للوكيل تحويل ذلك إلى سير عمل قابل للتحقق:
|
||||
|
||||
```text
|
||||
فتح الهدف
|
||||
→ انتظار اكتمال تحليل Hopper
|
||||
→ جرد المعماريات والرموز والسلاسل النصية
|
||||
→ تحديد نقاط الدخول والدوال المهمة
|
||||
→ تتبع الاستدعاءات والمراجع المتبادلة وتدفق التحكم
|
||||
→ فك ترجمة التنفيذ ذي الصلة
|
||||
→ تلخيص السلوك والقيود والحالات الطرفية
|
||||
→ بناء الميزة بما يلائم تقنيات المشروع ومتطلباته
|
||||
```
|
||||
|
||||
## ما الذي يمكنك فعله؟
|
||||
|
||||
### فهم تطبيق لا تملك مصدره
|
||||
|
||||
اكشف نقاط الدخول، وأسماء الفئات، والسلاسل النصية، والأطر المستخدمة، ومسارات الميزات من الملف التنفيذي وحده.
|
||||
|
||||
### بناء ميزة تعجبك بطريقتك
|
||||
|
||||
استقصِ الميزة، وافهم سلوكها، ثم ابنِ نسخة ملائمة لمنتجك وتقنياتك ومتطلباتك.
|
||||
|
||||
### توثيق بروتوكول أو تنسيق غير موثق
|
||||
|
||||
تتبع الموزعات، والثوابت، والمقارنات، والقراءة والكتابة لفهم الرسائل وتنسيقات الملفات وآلات الحالات.
|
||||
|
||||
### إعادة إنشاء سلوك متوافق
|
||||
|
||||
حوّل منطقًا مفكوك الترجمة إلى مواصفات واضحة، ثم نفذ بديلًا نظيفًا تحكمه اختبارات السلوك الملحوظ.
|
||||
|
||||
### تدقيق منطق أمني حساس
|
||||
|
||||
افحص التحقق من الإدخال، والتشفير، والصلاحيات، والتخزين، ومسارات الأخطاء من دون رفع الملف التنفيذي إلى خدمة بعيدة.
|
||||
|
||||
## منصة أدوات التحقيق
|
||||
|
||||
| عائلة الأدوات | العدد | الاستخدام |
|
||||
| -------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| الفحص الأصلي | 41 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والاستدعاءات والمراجع والتعليقات وقراءة البايتات ومواضع الملفات |
|
||||
| سير عمل التحقيق | 14 | نظرات عامة على التطبيقات وملفات الدوال وواجهات API الأصلية والتوزيع وفك الترجمة الدفعي وتتبع الميزات ومسارات الاستدعاء ومخططاته واكتشاف Swift وObjective-C |
|
||||
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift دون تشغيل Hopper |
|
||||
| مخطط المخرجات | 5 | جرد الدلائل والحزم وملفات Interface Builder المترجمة وكتالوجات موارد Apple والاستخراج |
|
||||
| PE/CLI المُدار | 7 | هوية تجميعات .NET وبياناتها الوصفية وتعليمات CIL والاعتماديات الأصلية واستيراد إعادة البناء ومقارنة الإصدارات |
|
||||
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة على Linux واستخراجها الصريح |
|
||||
| Android APK | 5 | بيانات الحزمة وmanifest والبحث عن الأصناف وقوائم الأعضاء وفك ترجمة الأساليب والمراجع الثابتة |
|
||||
| مراقبة المتصفح | 11 | بنية الصفحات وبيانات الشبكة والبرامج النصية وخرائط المصدر واكتشاف WebMCP والصور ومقارنة الالتقاط |
|
||||
| تحليل Electron | 5 | مراقبة العارضين وخريطة التطبيق الثابتة والمواءمة بين نتائج الثبات والتشغيل |
|
||||
| وقت تشغيل JavaScript | 2 | اكتشاف أهداف Node/Electron Inspector ومواضع البرامج وأحداث سياقات التنفيذ |
|
||||
| سير عمل التطبيقات | 13 | تصدير النصوص البرمجية الملتقطة، تتبع الميزات عبر الطبقات ومقارنة الإصدارات ومواءمة المصدر التاريخي ومقارنة أشكال القيم المعادة والتحقق من إعادة الإنشاء |
|
||||
| مساحة العمل والملاحظة | 21 | الجلسات وحزم الأدلة وسياق التنقل ومقارنة العمليات والمخرجات والدوال وتسجيل الأسئلة غير المحسومة |
|
||||
|
||||
## خطة العمل
|
||||
|
||||
يشمل العمل التالي توسيع التحقق من الأهداف الأصلية، ومقارنة إصدارات JavaScript و.NET، ومراقبة وقت التشغيل والتحقق من إعادة الإنشاء. يتيح Setup بالفعل اختيار تكامل الوكيل وتثبيت Hopper. تثبيت أدوات تحليل أخرى ما زال عملًا لاحقًا. راجع [خطة التثبيت](docs/roadmap.md) و[تقييم أدوات التحليل](docs/provider-evaluation.md).
|
||||
|
||||
## استخدام REA مع وكلاء برمجة آخرين
|
||||
|
||||
يدعم Setup Claude Code وClaude Desktop وCodex وCursor وGemini CLI وWindsurf وDevin وOpenCode وAntigravity وGitHub Copilot CLI وCommand Code وVS Code. تكون تسجيلات REA الموجودة محددة افتراضياً، ويجب اختيار العملاء المكتشفين الآخرين يدوياً. يمكن لأي وكيل يدعم خوادم MCP المحلية الاتصال باستخدام الإعداد التالي:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@5.0.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
## كيف يعمل؟
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Human[المستخدم] --> Agent[وكيل البرمجة]
|
||||
Agent --> REA[REA]
|
||||
Human -->|CLI| REA
|
||||
REA --> Hopper[عمليات تحليل Hopper]
|
||||
Hopper --> App[تطبيقك]
|
||||
REA --> Ghidra[تحليل Ghidra للقراءة فقط<br/>جرد + تحليل الدوال]
|
||||
Ghidra --> App
|
||||
```
|
||||
|
||||
يستخدم CLI وخادم MCP سير عمل التحليل نفسه. تحرر أوامر الطرفية جلسة الجسر الخاصة بها عند انتهائها، بينما يمكن لجلسة الوكيل الحفاظ على الاتصال أثناء الاستقصاء. إغلاق جلسة REA لا يغلق تطبيق Hopper الذي يستخدمه المستخدم.
|
||||
|
||||
## أوامر CLI
|
||||
|
||||
سير عمل الوكيل أعلاه هو أسهل طريقة لاستخدام REA. للحصول على نظرة سريعة على تطبيق من الطرفية:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze /Applications/Notes.app
|
||||
npx -y rea-agents@latest compare /absolute/path/to/left-evidence.json /absolute/path/to/right-evidence.json
|
||||
```
|
||||
|
||||
شغّل `npx -y rea-agents@latest --help` لمعرفة خيارات فك الترجمة المباشر والخيارات الأخرى.
|
||||
|
||||
يمكن لـ REA فتح مجلد `.app` على Mac مباشرة. إذا لم يعثر الوكيل على التطبيق، فأخبره بمكان تثبيته.
|
||||
|
||||
## سلوك Hopper
|
||||
|
||||
يشغّل REA برنامج Hopper عند الحاجة. على macOS قد تظهر نافذته في المقدمة أو يحتاج أحد الحوارات إلى استجابة منك. لا يمكن ضمان بقاء النافذة في الخلفية.
|
||||
|
||||
يحدد REA تنسيق الهدف ومعماريته لتجنب حوارات الاختيار المعتادة. عند إغلاق الجلسة، يوقف الجسر وينظف موارده الخاصة، لكنه لا يغلق تطبيق Hopper الذي تستخدمه.
|
||||
|
||||
## الأمان والخصوصية
|
||||
|
||||
- يبقى التحليل محليًا بين REA وموفر التحليل المختار عبر مقابس محلية خاصة، ومع IDA عبر تسجيل MCP المحلي المضبوط.
|
||||
- لا يرفع REA الأهداف إلى خدمة مستضافة.
|
||||
- تعمل أدوات التحليل والأهداف المطلقة بصلاحيات المستخدم نفسه؛ وما زال التقاط واجهة النظام على macOS يعتمد على الوصول إلى تسهيلات الاستخدام وتسجيل الشاشة. ولا ينفّذ التحليل الثابت للـ JavaScript الوحدات المستخرجة.
|
||||
- تستخدم جلسات Ghidra مشروعًا مؤقتًا معزولًا ولا تفتح مشاريع Ghidra التي يملكها المستخدم أو تعدّلها.
|
||||
- راجع الملفات التنفيذية غير الموثوقة واعزلها كما تفعل مع أي مدخل أصلي قد يكون ضارًا.
|
||||
- أبلغ عن الثغرات وفق [سياسة الأمان](SECURITY.md)، وليس عبر قضية عامة.
|
||||
|
||||
## الأسئلة الشائعة
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يتضمن REA محرك فك ترجمة خاصًا به؟</strong></summary>
|
||||
|
||||
لا. يستطيع Setup تثبيت Hopper، لكنه يظل برنامجًا منفصلًا بترخيص خاص به. يوفر REA واجهة CLI وخادم MCP وسير العمل المخصص للوكلاء.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يجب أن يكون Hopper مفتوحًا مسبقًا؟</strong></summary>
|
||||
|
||||
لا. يستطيع REA تشغيله. لكن Hopper قد يصبح مرئيًا أو يطلب إدخالًا عند الحاجة إلى قرار تفاعلي.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يعمل REA على Linux أو Windows؟</strong></summary>
|
||||
|
||||
يدعم REA نظام macOS 12+ وUbuntu 24.04+ وFedora 41+ وArch Linux بنواة 64 بت وCachyOS. يعمل تحليل Ghidra للقراءة فقط على Linux x64 وmacOS x64/arm64 مع Ghidra 12.1.x وJDK الكامل الذي يعلنه التثبيت (JDK 21 أو أحدث دون حد أعلى في إصدارات 12.1 الحالية)، إضافة إلى حد Windows x64 P0 التجريبي لتطبيقات PE الأصلية x86-64 على NTFS محلي؛ راجع [دليل Windows Ghidra P0](docs/windows-ghidra-p0.md) للنطاق الموثق.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يمكنني تحليل قواعد Hopper المحفوظة؟</strong></summary>
|
||||
|
||||
نعم. يدعم وقت التشغيل الملفات التنفيذية وقواعد `.hop`.
|
||||
|
||||
</details>
|
||||
|
||||
## التطوير
|
||||
|
||||
راجع [CONTRIBUTING.md](CONTRIBUTING.md) لإعداد بيئة التطوير وبنية المشروع والاختبارات وإرشادات الإصدار.
|
||||
|
||||
## المشروع
|
||||
|
||||
- [حزمة npm](https://www.npmjs.com/package/rea-agents)
|
||||
- [GitHub](https://github.com/morluto/rea)
|
||||
- [القضايا والطلبات](https://github.com/morluto/rea/issues)
|
||||
- [Hopper Disassembler](https://www.hopperapp.com/)
|
||||
- [سياسة الأمان](SECURITY.md)
|
||||
|
||||
## الترخيص
|
||||
|
||||
[MIT](LICENSE)
|
||||
+371
@@ -0,0 +1,371 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · **日本語** · [한국어](README_ko.md) · [العربية](README_ar.md)
|
||||
|
||||
# REA：あらゆるものをリバースエンジニアリング
|
||||
|
||||
### バイナリ、アプリ、実行時の動作をひとつの MCP でリバースエンジニアリング。
|
||||
|
||||
**気に入った機能を見つけたら、その仕組みをバイナリのレベルまで理解する。**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#調査ツールカタログ)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[ウェブサイト（英語）](https://morluto.github.io/rea/) · [ガイド](https://morluto.github.io/rea/guides/) · [DX-Ball の事例](https://morluto.github.io/rea/showcase/dx-ball/)**
|
||||
|
||||
[クイックスタート](#クイックスタート) · [現在の対応状況](#現在の対応状況) · [バイナリから動作へ](#バイナリから動作へ) · [調査ツールカタログ](#調査ツールカタログ) · [ロードマップ](#ロードマップ) · [仕組み](#仕組み)
|
||||
|
||||
<table aria-label="REA community">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="docs/assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>リバースエンジニアリングのコミュニティに参加</strong>
|
||||
</a><br />
|
||||
<sub>Discord · 質問と回答 · 成果の共有</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="Hopper 内で解析ブリッジを起動し、ネイティブバイナリを調べる REA" width="1200" />
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
自分のプロダクトにも欲しい機能を他のアプリで見つけたら、REA を使ってエージェントに調査を依頼できます。エージェントはソースコードがなくてもアプリを調べ、機能の仕組みを説明して根拠を示し、あなたのプロジェクト向けの実装まで進められます。
|
||||
|
||||
REA は、ネイティブバイナリ、JavaScript/Electron アプリ、.NET アセンブリ、Web サイトを調べるためのツールをエージェントに提供します。同じツールはターミナルからも使えます。解析はローカルで実行され、結果には各結論の根拠と制限が含まれます。
|
||||
|
||||
セットアップは REA をエージェントに登録し、対応するワークフロー手順もインストールします。ネイティブ解析には既存の Hopper または Ghidra を使えます。承認すれば、セットアップで Hopper をインストールすることもできます。静的な JavaScript 解析にはどちらのエンジンも不要です。
|
||||
|
||||
## エージェントに頼むだけ
|
||||
|
||||
[セットアップ](#クイックスタート)後にエージェントを再起動し、次のように依頼します：
|
||||
|
||||
```text
|
||||
メモアプリの検索機能を調べて根拠を示し、私のプロジェクト向けに同様の機能を実装してください。
|
||||
```
|
||||
|
||||
「メモアプリ」を調べたいアプリに置き換えてください。まずは概要の調査を依頼することもできます。
|
||||
|
||||
## バイナリから動作へ
|
||||
|
||||
| 逆コンパイル | 理解 | 再現 |
|
||||
| -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| ネイティブアプリや実行ファイルから、プロシージャ、疑似コード、アセンブリ、文字列、シンボル、セグメント、メタデータを復元します。 | 呼び出し元、呼び出し先、クロスリファレンス、コールグラフをたどり、機能やアルゴリズムの実際の動作を説明できる状態にします。 | エージェントが得た知見を、あなたの技術スタック、画面、要件に合うプロダクト機能へ変えます。 |
|
||||
|
||||
REA は結論に至った根拠を示します。元のソースコードを復元できる、あるいはアプリ全体を自動的に複製できるとは主張しません。
|
||||
|
||||
## REA を選ぶ理由
|
||||
|
||||
| | |
|
||||
| -------------------------- | -------------------------------------------------------------------------------------------------------- |
|
||||
| **エージェント向け** | アプリの動作を質問すると、エージェントが推測ではなく実際に調べて答えます。 |
|
||||
| **CLI と MCP** | 同じリバースエンジニアリング機能を、ターミナルからもエージェントからも使えます。 |
|
||||
| **ガイド付きセットアップ** | エージェントの設定、既存の解析ツールへの接続、承認したうえでの Hopper のインストールに対応します。 |
|
||||
| **洞察からコードへ** | 機能を理解したら、同じコーディングセッションで自分のプロダクト向けに実装できます。 |
|
||||
| **ローカルで解析** | 解析は対応するローカルホストで実行されます。REA がアプリをホスト型の解析サービスへ送ることはありません。 |
|
||||
| **コンテキストを維持** | 質問のたびに解析をやり直すことなく、複数のアプリを続けて調査できます。 |
|
||||
|
||||
## クイックスタート
|
||||
|
||||
### セットアップを実行（推奨）
|
||||
|
||||
REA をエージェントで使えるように設定します：
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
```
|
||||
|
||||
セットアップでは、まず REA を使うエージェントを選び（複数選択可）、具体的なパスと変更内容を確認してから承認します。既存の REA 登録は最初から選択されています。新しく検出されたエージェントは選択肢に表示されますが、検出されただけでは選択されません。選択したエージェントには、MCP 接続と REA のガイド付きワークフローが追加されます。Hopper のインストールは別の任意項目で、個別に承認が必要です。既存の Ghidra のインストール先を登録することもできます。
|
||||
|
||||
セットアップは変更を適用する前に内容を表示し、既存の設定をバックアップします。要件とセットアップのオプションは[インストールガイド](docs/installation.md)を参照してください。
|
||||
|
||||
### エージェントで使う（推奨）
|
||||
|
||||
セットアップ後にエージェントを再起動し、理解したいアプリや機能を[説明してください](#エージェントに頼むだけ)。Hopper はデモモードでも動作します。初回起動の画面が表示されたら、デモを選ぶか既存のライセンスを入力してください。
|
||||
|
||||
REA は Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf、Devin、OpenCode、Antigravity、GitHub Copilot CLI、Command Code、VS Code に対応しています。セットアップでは既存の REA 登録が最初から選択され、それ以外の検出済みエージェントは選択するまで対象になりません。その他のエージェントは[手動の MCP 設定](#他のコーディングエージェントで使う)を使えます。
|
||||
|
||||
### AI コーディングアシスタント向けスキル（任意）
|
||||
|
||||
AI コーディングアシスタントにスキルを追加すると、より豊富なコンテキストを利用できます：
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea --skill reverse-engineer-anything
|
||||
```
|
||||
|
||||
このスキルは REA の調査ワークフローを提供します。REA をエージェントに接続して解析ツールを設定するには、上記のセットアップを実行してください。セットアップは既定でバージョンの一致したスキルをインストールします。このコマンドはリポジトリ版のスキルをインストールします。
|
||||
|
||||
### ターミナルで最初の結果を得る
|
||||
|
||||
展開済みの JavaScript/Electron アプリのディレクトリまたは ASAR に対して、次を実行します：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
パスは調査対象に置き換えてください（Windows では `"D:/apps/example"` など）。MCP の設定、Hopper、Ghidra を必要とせず、アプリを実行することもなく、Evidence、復元したグラフ、制限、未解決事項をその場で返します。ネイティブアプリの場合は、先にエンジンを設定してから、そのアプリのパスを指定して `analyze` を使います。`doctor` は問題を診断したいときに使うもので、解析のたびに実行する必要はありません。
|
||||
|
||||
### 作業に合わせて準備状況を確認する
|
||||
|
||||
オプションなしの `rea doctor` は連携全体の監査です。検出したすべてのエージェント登録、インストール済みのスキル、すべての任意の解析エンジンを確認するため、今の作業には支障がなくても `healthy: false` を報告することがあります。作業に合わせて確認範囲を指定してください：
|
||||
|
||||
| 作業 | 準備状況の確認 |
|
||||
| ------------------------------------------ | ---------------------------------------------------------------------------------------------- |
|
||||
| 静的な JavaScript/Electron 解析 | 不要です。`analyze-javascript-application` を直接実行します。 |
|
||||
| 特定の解析エンジンのトラブルシューティング | `rea doctor --provider ghidra --json`（または `hopper`、`ida`） |
|
||||
| 特定のエージェントの MCP 登録を確認 | `rea doctor --client codex --json`（[クライアント ID](docs/installation.md#supported-agents)） |
|
||||
| インストール済みのスキルを確認 | `rea doctor --skill --json` |
|
||||
|
||||
範囲を指定したレポートでは、`healthy` と終了ステータスを決めるのは `scope_checks` だけです。それ以外の項目は `informational_checks` に表示され、今の作業のために修正する必要はありません。たとえば、使っていないエンジンが見つからなくても対処は不要です。
|
||||
|
||||
ネイティブ対象に対応するエンジンが複数インストールされている場合、REA は自動ではどれも選びません。対象を開くと `code: "capability_unavailable"` と `details.selection_reason: "ambiguous"` で失敗し、`details.candidate_ids` に候補が示されます。CLI の `--provider` か `open_binary` の `provider_id` で一度選ぶか、常用する設定として `REA_ANALYSIS_PROVIDER` を指定してください。詳しくは[英語版の解説](README.md#choosing-a-deep-analysis-provider)を参照してください。
|
||||
|
||||
### rea コマンドをインストール
|
||||
|
||||
コマンドラインツールをインストールします：
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
```
|
||||
|
||||
Node.js と npm を先にインストールしておいてください。ターミナルで実行すると、インストーラーは `rea` を追加してセットアップを開始します。
|
||||
|
||||
npm でインストールしてから、セットアップを実行することもできます：
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
```
|
||||
|
||||
### 要件
|
||||
|
||||
静的な JavaScript 解析に必要なのは Node.js と npm だけです。ホストと外部ツールの前提条件は使うワークフローによって異なり、ネイティブ解析プロバイダーの対応プラットフォームは各ガイドで説明しています。
|
||||
|
||||
- macOS 12 以降
|
||||
- Ubuntu 24.04+、Fedora 41+、64 ビット Arch Linux、または CachyOS
|
||||
- Node.js 22.x (>=22.19)、24.x (>=24.11)、または 26+
|
||||
- npm（特定のバージョンは不要で、REA が npm をインストールすることもありません）
|
||||
|
||||
ネイティブバイナリの詳細な解析には Hopper、Ghidra、IDA Pro のいずれかが必要です。Hopper は独自のライセンスを持つ別製品で、デモ版でもベンダーが定める制限の範囲で解析できます。Ghidra と IDA は、利用者が用意したものを使います。
|
||||
|
||||
Ghidra は Linux x64 と macOS x64/arm64 に対応しています。Ghidra 12.1.x と、そのインストールが宣言する 64 ビットの完全な JDK（`application.java.min` から `application.java.max` まで）を別途インストールし、REA で使うように設定してください。現行の 12.1 リリースは JDK 21 以降を必要とし、上限はありません。ブリッジは Ghidra 12.1.4 と JDK 21 で検証しています。macOS では、ホストのアーキテクチャに合うネイティブデコンパイラーも必要です。
|
||||
|
||||
```bash
|
||||
export GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra_12.1.4_PUBLIC
|
||||
export JAVA_HOME=/absolute/path/to/jdk-21 # java と javac が PATH から解決できる場合は省略可
|
||||
rea doctor --provider ghidra --json
|
||||
rea setup
|
||||
```
|
||||
|
||||
セットアップはこれらのインストールを確認し、承認後にパスを保存します。Ghidra、Java、Node.js、npm、Homebrew のインストールや更新は行いません。
|
||||
|
||||
IDA Pro は、すでに動作している [mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp) の MCP 登録を再利用します。開いている GUI の対象を読み取り専用で解析するか（`attached`、既定）、データベースのスーパーバイザーで指定したバイナリをヘッドレスで開いて解析します（`headless`）。REA が attached の GUI データベースを保存したり閉じたりすることはありません。最初の実環境での検証は、Windows の GUI と Windows x64 のヘッドレス IDA 9.3 が対象です。設定方法と対応範囲は [IDA プロバイダーガイド](docs/ida-provider.md)を参照してください。
|
||||
|
||||
```bash
|
||||
export REA_IDA_MCP_CONFIG=/absolute/path/to/ida-mcp.json
|
||||
rea function /absolute/path/to/program main --provider ida --json
|
||||
```
|
||||
|
||||
リポジトリの main と npm 4.1.0 には、ローカル NTFS 上のネイティブ x86-64 PE アプリケーション（マネージドコードと DLL を除く）を対象とする、実験的な Windows x64 Ghidra P0 対応が含まれています。Job Object、プライベート DACL、パスの受け入れ検査の各制御も同梱されています。古い npm パッケージでこの機能を使えるか確認するには[リリース境界](docs/installation.md#released-package-and-main)を、前提条件と検証済みの範囲は [Windows Ghidra P0](docs/windows-ghidra-p0.md) を参照してください。
|
||||
|
||||
### トラブルシューティング
|
||||
|
||||
`npx -y rea-agents@latest doctor` は、ホスト、依存関係、解析ツール、エージェント設定を変更せずに確認します。構造化された診断結果が必要な場合は `--json` を追加してください。
|
||||
|
||||
Linux では実行可能な `/opt/hopper/bin/Hopper` を優先し、利用できない場合は `~/.local/share/rea/hopper/bin/Hopper` を自動的に確認します。それ以外の場所にインストールした場合は `HOPPER_LAUNCHER_PATH` を設定してください。ファイルがあるのに解析エンジンが見つからないと報告される場合は、実際の Hopper のパスに対して `ldd /absolute/path/to/Hopper | grep 'not found'` を実行し、不足しているライブラリを確認してください。詳しくは [Hopper ガイド](docs/installation.md#hopper)を参照してください。
|
||||
|
||||
### 更新とアンインストール
|
||||
|
||||
- `rea update` は現在の REA インストールを更新します。
|
||||
- `rea uninstall` は REA が管理するエージェント登録とスキルだけを削除します。Hopper、Node.js、Evidence ファイル、キャプチャ、無関係なスキル、他の MCP サーバーは残ります。
|
||||
- `rea uninstall --purge-data` は `~/.rea/cache` と `~/.rea/state` も削除します。それらを消したい場合にだけ使ってください。
|
||||
|
||||
## 現在の対応状況
|
||||
|
||||
リポジトリの現在の機能とプラットフォーム要件は[英語の対応状況ガイド](README.md#current-status)で説明しています。main は [npm リリース](docs/installation.md#released-package-and-main)より先行する場合があります。
|
||||
|
||||
- Ghidra は Linux x64、macOS x64/arm64、実験的な Windows x64 P0 境界で 25 種類の読み取り専用操作を提供します。Linux と macOS では、セッション内での関数名とエントリーコメントのアトミックな編集にも対応します。Windows P0 には変更の権限がなく、Ghidra には REA から操作できる GUI 機能はありません。
|
||||
- IDA アダプターは、ドキュメントに記載された読み取り専用の関数・文字列操作を提供します。
|
||||
- 静的な Android APK の検査は Linux と macOS に対応し、別途用意したヘッドレス JADX と完全な JDK が必要です。現在のメタデータブリッジは macOS arm64 で検証しています。詳しくは [Android 解析](docs/android-analysis.md)を参照してください。
|
||||
- ブラウザー、Electron、プロセスの各リクエストは対象、操作、ライフサイクルを直接指定し、ホストで実際に許可されたアクセスの範囲で動作します。REA 側で別途権限を付与する必要はありません。セットアップによる設定の書き込みと Hopper のインストールには、引き続き具体的な計画の承認が必要です。
|
||||
- `rea capabilities` と `rea providers` はバイナリセッションのプロバイダーと補助操作を説明するもので、ブラウザー、Android、アプリのすべてのワークフローの一覧ではありません。MCP の機能全体は、接続中の MCP ツール一覧と `binary_session` が示すツールの利用可否で確認してください。
|
||||
|
||||
## ひとつのプロンプトで調査を完結
|
||||
|
||||
```text
|
||||
メモアプリをリバースエンジニアリングし、オフライン検索機能の仕組みを説明して、
|
||||
TypeScript と SQLite を使って私のプロジェクト向けに実装してください。
|
||||
```
|
||||
|
||||
| 手順 | エージェントの処理 | REA ツール |
|
||||
| ---: | ------------------------------ | ---------------------------------------------------------------- |
|
||||
| 1 | バイナリを開いて識別 | `open_binary`, `binary_overview` |
|
||||
| 2 | オフライン検索の手掛かりを探す | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 手掛かりを実行コードに結び付け | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 関連する制御フローを復元 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 関連する処理を逆コンパイル | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | プロジェクトに機能を実装 | 技術スタック、プロダクト、要件に合わせたコード |
|
||||
|
||||
手順 1〜5 のアプリ解析は REA が担当します。手順 6 は、アプリについて得た知見をもとに、エージェントが通常の編集ツールとテストツールで行います。
|
||||
|
||||
## エージェントにできること
|
||||
|
||||
- 気に入った機能を調査し、自分のプロダクトに合わせた形で実装する。
|
||||
- ソースコードが手に入らない機能の仕組みを説明する。
|
||||
- アプリの認証、保存、更新、ネットワーク処理の流れを復元する。
|
||||
- 文書化されていない形式やインターフェースについて、文書化に必要な構造を復元する。
|
||||
- 文字列やシンボルから、疑わしい動作を実装しているコードまでたどる。
|
||||
- 復元した動作を、プロダクト機能、テスト、移行メモ、移植版、相互運用できる代替実装に変える。
|
||||
- マングルされたシンボルをひとつずつ手作業で読み解かなくても、Swift と Objective-C のメタデータを解析する。
|
||||
- Hopper に名前、コメント、ブックマークを残し、人とエージェントの解析を互いに活かす。
|
||||
|
||||
## 調査ツールカタログ
|
||||
|
||||
| ツール分類 | 数 | 用途 |
|
||||
| -------------------- | --: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ネイティブ検査 | 41 | 関数、疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照、注釈、バイト読み取り、ファイルオフセット |
|
||||
| 調査ワークフロー | 14 | アプリ概要、関数ドシエ、ネイティブ API とディスパッチ、一括逆コンパイル、機能トレース、コールパス、コールグラフ、Swift と Objective-C の検出 |
|
||||
| macOS ネイティブ | 7 | Hopper を起動せずに扱える Mach-O メタデータ、コード署名、plist、アーキテクチャ、Swift シンボルのデマングル |
|
||||
| 成果物グラフ | 5 | ディレクトリとパッケージの目録、コンパイル済み Interface Builder ファイル、Apple アセットカタログ、抽出 |
|
||||
| マネージド PE/CLI | 7 | .NET の識別情報、メタデータ、CIL 命令、ネイティブ依存関係、再構築結果のインポート、ビルド比較 |
|
||||
| ファームウェア | 2 | Linux ファームウェア領域の検査と明示的な展開 |
|
||||
| Android APK | 5 | パッケージと manifest の宣言、クラス検索、メンバー一覧、メソッドの逆コンパイル、静的な被参照 |
|
||||
| ブラウザー観察 | 11 | ページ構成、ネットワークメタデータ、スクリプト、ソースマップ、WebMCP の検出、スクリーンショット、キャプチャ比較 |
|
||||
| Electron 解析 | 5 | レンダラーの観察、静的なアプリのマッピング、静的結果と実行時結果の照合 |
|
||||
| JavaScript 実行時 | 2 | Node/Electron Inspector の対象検出、スクリプトの位置、実行コンテキストのイベント |
|
||||
| アプリワークフロー | 13 | 取得した Web スクリプトのエクスポート、Android/Apple の目録の投影、層をまたぐ機能トレース、ビルド比較、過去のソースとの対応付け、静的な戻り値形状の比較、再構築の検証 |
|
||||
| ワークスペースと観察 | 21 | セッション、Evidence バンドル、ナビゲーションの文脈、プロセス・成果物・関数の比較、未解決事項の記録 |
|
||||
|
||||
## ロードマップ
|
||||
|
||||
[現在の対応状況](#現在の対応状況)は提供済みの機能です。今後の作業は次のとおりです。
|
||||
|
||||
- **現在：** ツール、プロバイダー、セットアップの選択肢、バージョンの変更に合わせてドキュメントを正確に保ち、Hopper と Ghidra で検証するネイティブバイナリのアーキテクチャと間接呼び出しの範囲を広げます。
|
||||
- **次：** 機能トレースに静的抽出と実行時の観察を追加し、難読化された .NET アセンブリの比較を改善し、プロセス、プロトコル、ファイルシステム、再接続、バージョン比較の範囲を広げます。
|
||||
- **その後：** ブラウザーと Electron のシナリオ操作を増やし、LLDB や Frida などによるネイティブアプリの実行時観察と、他の解析ツールや対象の評価を進めます。
|
||||
|
||||
セットアップでは、すでにエージェント連携と Hopper のインストールを選べます。その他の解析ツールのインストールへの対応は今後の作業です。[インストールのロードマップ](docs/roadmap.md)と[解析ツールの評価](docs/provider-evaluation.md)を参照してください。
|
||||
|
||||
## 他のコーディングエージェントで使う
|
||||
|
||||
セットアップは Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf、Devin、OpenCode、Antigravity、GitHub Copilot CLI、Command Code、VS Code に対応しています。既存の REA 登録は最初から選択され、それ以外の検出済みエージェントは選択するまで対象になりません。ローカル MCP サーバーに対応するエージェントであれば、次の設定でも接続できます。
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@5.0.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
常用する登録では、パッケージのバージョンを 1 つに固定してください。`rea setup` はその固定バージョンを維持し、同梱のスキルも同時に更新します。
|
||||
|
||||
## 仕組み
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["コーディングエージェント"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["ターミナル"] --> REA
|
||||
REA --> Hopper["Hopper プロバイダー"]
|
||||
REA --> Ghidra["Ghidra プロバイダー<br/>インベントリ + 関数解析 + 注釈"]
|
||||
REA --> Ida["IDA MCP プロバイダー<br/>attached GUI または headless データベース"]
|
||||
Hopper --> App["あなたのアプリ"]
|
||||
Ghidra --> App
|
||||
Ida --> App
|
||||
```
|
||||
|
||||
CLI と MCP サーバーは、同じアプリケーションワークフローと Evidence の契約を使います。ターミナルのコマンドは完了すると自身のブリッジセッションを解放し、エージェントのセッションは調査中の対象と Evidence を保持できます。REA のセッションを閉じても、利用中の Hopper アプリは終了しません。
|
||||
|
||||
## CLI
|
||||
|
||||
上のエージェントワークフローが、REA を使う最も簡単な方法です。ターミナルからアプリの概要を一度だけ調べる場合は、次を実行します。
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
直接の逆コンパイルやその他のオプションは、`npx -y rea-agents@latest --help` で確認できます。
|
||||
|
||||
グローバルな `rea` コマンドとしてもインストールできます。
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA は Mac の `.app` フォルダーを直接開けます。エージェントがアプリを名前で見つけられない場合は、インストール場所を伝えてください。
|
||||
|
||||
## Hopper アプリの動作
|
||||
|
||||
REA は操作に必要になった時点で Hopper を起動します。Hopper のランチャーは内部でアプリをアクティブにするため、REA がバックグラウンドでの起動を要求しても、macOS では Hopper のウィンドウやダイアログが前面に出る場合があります。デモやライセンスの画面では操作が必要になることがあります。
|
||||
|
||||
Hopper は解析リクエストを一度に 1 つずつ処理します。待機をキャンセルしても、Hopper 内ですでに実行中の処理は止まりません。処理中かどうかはセッションが報告します。セッションを閉じると REA のブリッジと一時ソケットディレクトリを削除しますが、利用中の Hopper アプリは終了しません。後片付けを確認できなかった場合、`close_binary` は `cleanup_incomplete` と対象のリソースを報告します。
|
||||
|
||||
## セキュリティモデル
|
||||
|
||||
解析はローカルで実行されます。REA は Hopper や Ghidra とは認証付きのプライベートなローカルソケットで、IDA とは設定済みのローカル MCP 登録を通じて通信します。Ghidra セッションは隔離された一時プロジェクトを使い、ユーザーの Ghidra プロジェクトを開いたり変更したりしません。
|
||||
|
||||
これはサンドボックスではありません。解析ツールと起動した対象は現在のユーザー権限で動作し、同じ OS ユーザー権限で動く悪意のあるプロセスは防げません。Windows Ghidra P0 では、ネイティブの Job Object、保護されたプライベート DACL、ハンドルベースの受け入れ検査を自動的に使います。脆弱性は [SECURITY.md](SECURITY.md) に記載された非公開の手順で報告してください。
|
||||
|
||||
## FAQ
|
||||
|
||||
<details><summary><strong>Hopper を先に起動しておく必要はありますか？</strong></summary>
|
||||
|
||||
いいえ。REA は必要になった時点で Hopper を起動します。すでに起動している Hopper でも使えますが、既存の GUI ドキュメントには接続せず、新しい解析ドキュメントを開きます。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA に Hopper は含まれていますか？</strong></summary>
|
||||
|
||||
含まれていません。セットアップで Hopper をインストールできますが、Hopper は独自のライセンスを持つ別製品です。REA は、エージェントが Hopper を使えるようにする CLI、MCP サーバー、ワークフローを提供します。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA は Ghidra や Java をインストールしたり同梱したりしますか？</strong></summary>
|
||||
|
||||
いいえ。REA は既存の Ghidra のインストールに接続します。[要件](#要件)にある Ghidra と Java のパスを設定してから、セットアップを実行してください。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>アプリはアップロードされますか？</strong></summary>
|
||||
|
||||
REA にホスト型の解析サービスはありません。現在のプロバイダーは、成果物の解析と動作のキャプチャをローカルで行います。エージェントやモデルの提供元には独自のデータポリシーがあるため、別途確認してください。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>元のソースコードを復元できますか？</strong></summary>
|
||||
|
||||
元のソースコードを確実に復元できるデコンパイラーはありません。REA はエージェントに疑似コード、アセンブリ、シンボル、文字列、メタデータ、それらの関係を提供し、観察された動作の説明や互換性のある再現に役立てられるようにします。
|
||||
|
||||
</details>
|
||||
|
||||
## 開発
|
||||
|
||||
開発環境、アーキテクチャ、テスト、リリース手順は [CONTRIBUTING.md](CONTRIBUTING.md) を参照してください。
|
||||
|
||||
## ライセンス
|
||||
|
||||
[MIT](LICENSE)
|
||||
+320
@@ -0,0 +1,320 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · [日本語](README_ja.md) · **한국어** · [العربية](README_ar.md)
|
||||
|
||||
# REA: 무엇이든 리버스 엔지니어링
|
||||
|
||||
### 에이전트로 앱의 동작부터 네이티브 바이너리까지 조사하세요.
|
||||
|
||||
**마음에 드는 기능을 찾고, 작동 방식을 이해하고, 원하는 방식으로 구현하세요.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#조사-도구-카탈로그)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[웹사이트(영문)](https://morluto.github.io/rea/) · [가이드](https://morluto.github.io/rea/guides/) · [DX-Ball 사례](https://morluto.github.io/rea/showcase/dx-ball/)**
|
||||
|
||||
[빠른 시작](#빠른-시작) · [현재 지원 범위](#현재-지원-범위) · [바이너리에서 동작까지](#바이너리에서-동작까지) · [조사 도구 카탈로그](#조사-도구-카탈로그) · [로드맵](#로드맵) · [작동 방식](#작동-방식)
|
||||
|
||||
<table aria-label="REA community">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="docs/assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>리버스 엔지니어링 커뮤니티에 참여하세요</strong>
|
||||
</a><br />
|
||||
<sub>Discord · 질문과 답변 · 작업 공유</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="Hopper에서 분석 브리지를 시작하고 네이티브 바이너리를 조사하는 REA" width="1200" />
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
앱에서 마음에 드는 기능을 찾았다면 에이전트에게 REA로 조사해 달라고 요청하세요. 소스 코드가 없어도 앱을 분석하고, 기능의 동작과 근거를 설명한 뒤, 프로젝트에 맞는 기능을 구현할 수 있습니다.
|
||||
|
||||
REA는 네이티브 바이너리, JavaScript/Electron 앱, .NET 어셈블리, 웹사이트 분석 도구를 제공합니다. 에이전트와 터미널에서 같은 도구를 사용할 수 있습니다. 분석은 로컬에서 실행되며 결과에는 근거와 제한 사항이 포함됩니다.
|
||||
|
||||
Setup은 에이전트를 설정하고 기존 Hopper 또는 Ghidra 설치에 연결합니다. 승인 후 Hopper를 설치할 수도 있습니다.
|
||||
|
||||
## 에이전트에게 바로 요청하기
|
||||
|
||||
[설정](#빠른-시작)을 마친 뒤 에이전트를 다시 시작하고 요청하세요.
|
||||
|
||||
```text
|
||||
메모 앱의 검색 기능이 어떻게 동작하는지 조사하고 근거를 보여 주세요.
|
||||
그다음 제 프로젝트에 비슷한 기능을 구현해 주세요.
|
||||
```
|
||||
|
||||
메모 앱 대신 조사할 앱을 지정하거나 먼저 개요를 요청할 수 있습니다.
|
||||
|
||||
## 바이너리에서 동작까지
|
||||
|
||||
| 디컴파일 | 이해 | 재현 |
|
||||
| ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| 네이티브 앱이나 실행 파일에서 프로시저, 의사 코드, 어셈블리, 문자열, 심볼, 세그먼트, 메타데이터를 복구합니다. | 호출자, 피호출자, 상호 참조, 호출 그래프를 따라 기능이나 알고리즘의 실제 동작을 설명합니다. | 에이전트가 배운 내용을 사용자의 기술 스택, 화면, 요구 사항에 맞는 제품 기능으로 만듭니다. |
|
||||
|
||||
REA는 분석을 바이너리 증거에 근거하게 합니다. 원본 소스 코드를 복원하거나 앱 전체를 자동으로 복제한다고 주장하지 않습니다.
|
||||
|
||||
## REA를 사용하는 이유
|
||||
|
||||
| | |
|
||||
| ------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| **에이전트용** | 앱의 동작에 대해 질문하고 추측 대신 에이전트가 조사하게 합니다. |
|
||||
| **CLI와 MCP** | 터미널과 에이전트에서 동일한 리버스 엔지니어링 기능을 사용합니다. |
|
||||
| **안내식 설정** | 에이전트 설정, 기존 분석 도구 연결, 승인 후 Hopper 설치를 지원합니다. |
|
||||
| **통찰에서 코드로** | 기능을 이해한 뒤 같은 코딩 세션에서 자신의 제품에 맞는 버전을 구현합니다. |
|
||||
| **로컬 분석** | 분석은 지원되는 로컬 호스트에서 실행되며 REA는 바이너리를 호스팅 분석 서비스에 업로드하지 않습니다. |
|
||||
| **컨텍스트 유지** | 질문마다 분석을 처음부터 시작하지 않고 여러 앱을 연속으로 조사합니다. |
|
||||
|
||||
## 빠른 시작
|
||||
|
||||
### 설정 실행하기(권장)
|
||||
|
||||
REA를 에이전트에 연결하세요.
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
```
|
||||
|
||||
Setup은 먼저 연결할 에이전트를 여러 개 선택하도록 안내합니다. 기존 REA 등록은 기본 선택되며, 감지만 된 클라이언트는 자동 선택되지 않습니다. 아직 설정되지 않은 클라이언트도 직접 선택할 수 있습니다. 정확한 경로와 변경 사항을 검토한 뒤 승인하세요. 선택한 에이전트에는 기본적으로 REA 워크플로가 설치됩니다. Hopper는 별도의 선택 사항이며 따로 승인해야 합니다. 기존 Ghidra 경로도 등록할 수 있습니다.
|
||||
|
||||
변경 전에 계획을 표시하고 기존 설정을 백업합니다. 요구 사항과 추가 옵션은 [설치 안내](docs/installation.md)를 참고하세요.
|
||||
|
||||
### 에이전트에서 사용하기
|
||||
|
||||
설정 후 에이전트를 다시 시작하고 조사할 앱이나 기능을 설명하세요. REA는 Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, Devin, OpenCode, Antigravity, GitHub Copilot CLI, Command Code, VS Code를 지원합니다. 기존 REA 등록은 기본 선택되고, 그 밖의 감지된 클라이언트는 직접 선택해야 합니다. 다른 에이전트는 아래 MCP 설정을 사용할 수 있습니다.
|
||||
|
||||
Hopper는 데모 모드로 사용할 수 있습니다. 첫 실행 안내가 나오면 데모를 선택하거나 기존 라이선스를 입력하세요.
|
||||
|
||||
### AI 코딩 어시스턴트용 스킬(선택)
|
||||
|
||||
AI 코딩 어시스턴트에 스킬을 추가하면 더 풍부한 컨텍스트를 얻을 수 있습니다.
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea --skill reverse-engineer-anything
|
||||
```
|
||||
|
||||
이 스킬은 REA의 조사 워크플로를 제공합니다. 위 setup을 실행해 REA를 에이전트에 연결하고 분석 도구를 설정하세요. Setup은 기본적으로 버전에 맞는 스킬을 설치합니다. 이 명령은 저장소 버전의 스킬을 설치합니다.
|
||||
|
||||
압축을 해제한 JavaScript/Electron 앱 트리나 ASAR라면 MCP 설정이나 네이티브 엔진 없이 바로 분석할 수 있습니다.
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
경로를 조사 대상으로 바꾸세요(Windows에서는 `"D:/apps/example"` 같은 경로). 이 정적 워크플로에서는 이런 경로도 지원됩니다. 네이티브 분석에는 전용 엔진 설정이 필요합니다.
|
||||
|
||||
### rea 명령 설치하기
|
||||
|
||||
명령줄 도구를 설치하세요.
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
```
|
||||
|
||||
Node.js와 npm이 먼저 설치되어 있어야 합니다. 터미널에서 실행하면 설치 프로그램이 `rea`를 설치하고 설정을 시작합니다.
|
||||
|
||||
npm으로 설치한 뒤 설정을 실행할 수도 있습니다.
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
```
|
||||
|
||||
### 요구 사항
|
||||
|
||||
- macOS 12 이상
|
||||
- Ubuntu 24.04+, Fedora 41+, 64비트 Arch Linux 또는 CachyOS
|
||||
- Node.js 22.x (>=22.19), 24.x (>=24.11) 또는 26+와 npm
|
||||
|
||||
네이티브 바이너리 분석에는 Hopper 또는 Ghidra가 필요합니다. Hopper는 별도 소프트웨어입니다. 데모에는 공급업체의 제한이 있지만 유료 라이선스가 필수는 아닙니다.
|
||||
|
||||
Ghidra는 Linux x64와 macOS x64/arm64를 지원합니다. Ghidra 12.1.x와 그 설치본이 선언한 완전한 64비트 JDK(`application.java.min`부터 `application.java.max`까지)를 별도로 설치한 뒤 REA가 사용하도록 설정하세요. 현재 12.1 릴리스는 JDK 21 이상을 요구하고 상한은 없습니다. 브리지는 Ghidra 12.1.4와 JDK 21에서 검증됩니다. macOS에서는 호스트 아키텍처에 맞는 네이티브 디컴파일러도 필요합니다.
|
||||
|
||||
Setup은 설치를 확인하고 경로를 저장할 수 있습니다. Ghidra, Java, Node.js, npm, Homebrew를 설치하거나 업데이트하지 않습니다.
|
||||
|
||||
저장소 main과 npm 4.1.0에는 로컬 NTFS상의 네이티브 x86-64 PE 애플리케이션(비관리, 비 DLL 대상)을 위한 실험적인 Windows x64 Ghidra P0 지원이 포함되어 있습니다. Job Object, 비공개 DACL, 경로 승인 제어가 함께 제공됩니다. 이전 npm 패키지에 이 기능이 있을 것으로 기대하기 전에 [릴리스 경계](docs/installation.md#released-package-and-main)를 확인하세요. 전제 조건과 검증 범위는 [Windows Ghidra P0](docs/windows-ghidra-p0.md)를 참고하세요.
|
||||
|
||||
### 문제 해결
|
||||
|
||||
`npx -y rea-agents@latest doctor`는 호스트, 의존성, 분석 도구, 에이전트 설정을 변경 없이 확인합니다. 구조화된 진단에는 `--json`을 추가하세요.
|
||||
|
||||
Linux에서는 실행 가능한 `/opt/hopper/bin/Hopper`를 우선 사용하고, 사용할 수 없으면 `~/.local/share/rea/hopper/bin/Hopper`를 자동으로 확인합니다. 다른 위치에는 `HOPPER_LAUNCHER_PATH`를 설정하세요. 파일이 있는데도 분석 엔진이 없다고 하면 실제 Hopper 경로에 대해 `ldd /absolute/path/to/Hopper | grep 'not found'`를 실행해 누락된 라이브러리를 확인하세요. 자세한 내용은 [Hopper 안내](docs/installation.md#hopper)를 참고하세요.
|
||||
|
||||
### 업데이트와 제거
|
||||
|
||||
- `rea update`는 현재 REA 설치를 업데이트합니다.
|
||||
- `rea uninstall`은 REA가 관리하는 에이전트 등록과 워크플로 파일을 제거합니다. Hopper는 유지합니다.
|
||||
- `rea uninstall --purge-data`는 REA 캐시와 상태도 삭제합니다. 해당 데이터를 지우려는 경우에만 사용하세요.
|
||||
|
||||
## 현재 지원 범위
|
||||
|
||||
저장소의 현재 기능과 플랫폼 요구 사항은 [영문 지원 가이드](README.md#current-status)에 설명되어 있습니다. main은 [npm 릴리스](docs/installation.md#released-package-and-main)보다 앞설 수 있습니다.
|
||||
|
||||
- Ghidra는 Linux x64, macOS x64/arm64, 실험적인 Windows x64 P0 경계에서 25개의 읽기 전용 작업을 제공합니다. Linux/macOS에서는 원자적 세션 함수 주석도 추가로 지원합니다. Windows P0는 읽기 전용이며 Ghidra에는 GUI 제어가 없습니다.
|
||||
- 정적 Android 검사는 별도로 준비한 JADX/Java가 필요합니다. 실제 제공자 검증은 Linux를 대상으로 합니다. [Android 분석](docs/android-analysis.md)을 참고하세요.
|
||||
- 브라우저, Electron, 프로세스 요청은 대상·동작·수명 주기를 직접 지정하며 호스트의 실제 접근에 의존합니다. 별도의 REA 권한 승인은 필요 없습니다. Setup의 설정 쓰기와 Hopper 설치는 여전히 정확한 계획에 대한 승인이 필요합니다.
|
||||
- `rea capabilities`는 모든 앱/런타임 도구의 목록이 아니라 바이너리 세션 작업을 설명합니다. 연결된 MCP 도구 목록과 관련 가이드를 참고하세요.
|
||||
|
||||
## 하나의 프롬프트로 전체 조사
|
||||
|
||||
```text
|
||||
메모 앱을 리버스 엔지니어링해 오프라인 검색 기능의 작동 방식을 설명한 다음,
|
||||
TypeScript와 SQLite를 사용해 제 프로젝트에 맞는 버전을 구현해 주세요.
|
||||
```
|
||||
|
||||
| 단계 | 에이전트 작업 | REA 도구 |
|
||||
| ---: | ----------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | 바이너리 열기 및 식별 | `open_binary`, `binary_overview` |
|
||||
| 2 | 오프라인 검색 단서 찾기 | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 단서를 실행 코드에 연결 | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 제어 흐름 재구성 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 관련 루틴 디컴파일 | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | 프로젝트에 기능 구현 | 기술 스택, 제품, 요구 사항에 맞는 코드 |
|
||||
|
||||
REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이전트가 일반 파일 편집 및 테스트 도구로 수행합니다.
|
||||
|
||||
## 에이전트가 할 수 있는 일
|
||||
|
||||
- 소스 코드가 없는 기능의 동작을 설명합니다.
|
||||
- 앱의 인증, 저장소, 업데이트 또는 네트워크 흐름을 재구성합니다.
|
||||
- 문서화되지 않은 형식이나 인터페이스의 구조를 복구합니다.
|
||||
- 문자열이나 심볼에서 의심스러운 동작을 구현한 코드까지 추적합니다.
|
||||
- 한 세션에서 두 앱 버전을 전환하며 구현 경로를 비교합니다.
|
||||
- 마음에 드는 기능을 조사하고 자신의 제품에 맞는 버전으로 구현합니다.
|
||||
- 복구한 동작을 제품 기능, 테스트, 마이그레이션 문서, 포팅 또는 상호 운용 가능한 대체품으로 변환합니다.
|
||||
- Swift 및 Objective-C 메타데이터를 분석합니다.
|
||||
- Hopper에 이름, 주석, 북마크를 남겨 사람과 에이전트의 분석을 연결합니다.
|
||||
|
||||
## 조사 도구 카탈로그
|
||||
|
||||
| 도구 분류 | 수 | 용도 |
|
||||
| ----------------- | --: | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 네이티브 검사 | 41 | 함수, 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조, 주석, 바이트 읽기, 파일 오프셋 |
|
||||
| 조사 워크플로 | 14 | 앱 개요, 함수 기록서, 네이티브 API와 디스패치, 일괄 디컴파일, 기능 추적, 호출 경로, 호출 그래프, Swift와 Objective-C 탐색 |
|
||||
| macOS 네이티브 | 7 | Hopper 실행 없이 처리하는 Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원 |
|
||||
| 아티팩트 그래프 | 5 | 디렉터리와 패키지 목록, 컴파일된 Interface Builder 파일, Apple 애셋 카탈로그, 추출 |
|
||||
| 관리 PE/CLI | 7 | .NET 식별, 메타데이터, CIL 명령어, 네이티브 종속성, 재구성 가져오기, 빌드 비교 |
|
||||
| 펌웨어 | 2 | Linux 펌웨어 영역 검사 및 명시적 추출 |
|
||||
| Android APK | 5 | 패키지와 manifest 선언, 클래스 검색, 멤버 목록, 메서드 디컴파일, 정적 참조 |
|
||||
| 브라우저 관찰 | 11 | 페이지 구조, 네트워크 메타데이터, 스크립트, 소스 맵, WebMCP 탐색, 스크린샷, 캡처 비교 |
|
||||
| Electron 분석 | 5 | 렌더러 관찰, 정적 앱 매핑, 정적·런타임 결과 연결 |
|
||||
| JavaScript 런타임 | 2 | Node/Electron Inspector 대상 탐색, 스크립트 위치, 실행 컨텍스트 이벤트 |
|
||||
| 앱 워크플로 | 13 | 캡처한 웹 스크립트 내보내기, 계층 간 기능 추적, 빌드 비교, 히스토리 소스 매핑, 정적 반환 구조 비교, 재구현 검증 |
|
||||
| 작업 공간과 관찰 | 21 | 세션, 근거 번들, 탐색 컨텍스트, 프로세스·아티팩트·함수 비교, 미해결 항목 기록 |
|
||||
|
||||
## 로드맵
|
||||
|
||||
다음 작업은 네이티브 대상 검증 확대, JavaScript와 .NET의 버전 비교, 런타임 관찰과 재구현 검증입니다. Setup은 에이전트 연결과 Hopper 설치 선택을 이미 지원합니다. 다른 분석 도구 설치는 향후 작업입니다. [설치 로드맵](docs/roadmap.md)과 [분석 도구 평가](docs/provider-evaluation.md)를 참고하세요.
|
||||
|
||||
## 다른 코딩 에이전트에서 사용하기
|
||||
|
||||
Setup은 Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, Devin, OpenCode, Antigravity, GitHub Copilot CLI, Command Code, VS Code를 지원합니다. 기존 REA 등록은 기본 선택되고, 그 밖의 감지된 클라이언트는 직접 선택해야 합니다. 로컬 MCP 서버를 지원하는 에이전트는 다음 설정으로 연결할 수 있습니다.
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@5.0.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
## 작동 방식
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["코딩 에이전트"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["터미널"] --> REA
|
||||
REA --> Hopper["Hopper 분석 작업"]
|
||||
Hopper --> App["사용자의 앱"]
|
||||
REA --> Ghidra["Ghidra 읽기 전용 분석<br/>인벤토리 + 함수 분석"]
|
||||
Ghidra --> App
|
||||
```
|
||||
|
||||
CLI와 MCP 서버는 같은 분석 워크플로를 사용합니다. 터미널 명령은 완료 후 자신의 브리지 세션을 해제하고, 에이전트 세션은 조사 중 연결을 유지할 수 있습니다. REA 세션을 닫아도 사용자가 이용 중인 Hopper 앱은 종료하지 않습니다.
|
||||
|
||||
## CLI
|
||||
|
||||
위의 에이전트 워크플로가 REA를 사용하는 가장 쉬운 방법입니다. 터미널에서 앱을 한 번 빠르게 살펴보려면 다음을 실행하세요.
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
직접 디컴파일하는 방법과 기타 옵션은 `npx -y rea-agents@latest --help`에서 확인할 수 있습니다.
|
||||
|
||||
전역 `rea` 명령으로 설치할 수도 있습니다.
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA는 Mac의 `.app` 폴더를 직접 열 수 있습니다. 에이전트가 앱을 찾지 못하면 설치 위치를 알려 주세요.
|
||||
|
||||
## Hopper 앱 동작
|
||||
|
||||
REA는 필요할 때 Hopper를 시작합니다. Hopper 런처는 내부적으로 앱을 활성화하므로 대상을 열 때 Hopper가 다른 창 앞으로 나타날 수 있습니다. REA는 백그라운드 시작을 요청하지만 항상 뒤에 머무르는 것을 보장할 수 없습니다.
|
||||
|
||||
명시적인 형식과 아키텍처 인자로 일반적인 FAT/ARM 선택 대화상자를 피하지만, 다른 Hopper 또는 macOS 대화상자에는 사람이 응답해야 할 수 있습니다. 세션 종료 시 브리지와 소켓 디렉터리를 제거하지만 사용자가 작업 중인 Hopper 앱은 종료하지 않습니다.
|
||||
|
||||
## 보안 모델
|
||||
|
||||
각 세션은 무작위 capability token과 현재 사용자 전용 Unix 소켓을 사용합니다. Ghidra 세션은 격리된 임시 프로젝트도 사용하며 사용자 소유 Ghidra 프로젝트를 열거나 수정하지 않습니다. 이는 샌드박스가 아니며 같은 운영 체제 사용자로 실행되는 악성 프로세스를 방어하지 않습니다. 신뢰할 수 없는 바이너리를 열면 선택한 로컬 공급자가 현재 사용자 권한으로 분석합니다. 취약점은 [SECURITY.md](SECURITY.md)의 비공개 절차로 신고하세요.
|
||||
|
||||
## FAQ
|
||||
|
||||
<details><summary><strong>Hopper를 미리 실행해야 하나요?</strong></summary>
|
||||
|
||||
아니요. REA가 필요할 때 시작합니다. Hopper가 실행 중이어도 사용할 수 있지만 기존 GUI 문서에 연결하는 대신 새 분석 문서를 엽니다.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA에 Hopper가 포함되나요?</strong></summary>
|
||||
|
||||
아니요. Setup이 Hopper를 설치할 수 있지만 Hopper는 자체 라이선스가 필요한 별도 소프트웨어입니다. REA는 CLI, MCP 서버, 에이전트용 워크플로를 제공합니다.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>바이너리가 업로드되나요?</strong></summary>
|
||||
|
||||
REA에는 호스팅 분석 서비스가 없습니다. 로컬 Unix 소켓을 통해 Hopper를 조작합니다. 에이전트나 모델 공급자의 데이터 정책은 별도로 확인하세요.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>원본 소스 코드를 복구할 수 있나요?</strong></summary>
|
||||
|
||||
보장할 수 없습니다. REA는 의사 코드, 어셈블리, 심볼, 문자열, 메타데이터 및 관계를 제공하여 에이전트가 관찰된 동작을 설명하거나 호환되게 재현하도록 돕습니다.
|
||||
|
||||
</details>
|
||||
|
||||
## 개발
|
||||
|
||||
개발 환경, 아키텍처, 테스트, 릴리스 지침은 [CONTRIBUTING.md](CONTRIBUTING.md)를 참고하세요.
|
||||
|
||||
## 라이선스
|
||||
|
||||
[MIT](LICENSE)
|
||||
+321
@@ -0,0 +1,321 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · **简体中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [العربية](README_ar.md)
|
||||
|
||||
# REA：逆向工程一切
|
||||
|
||||
### 使用智能体逆向工程任何程序，从应用行为到原生二进制文件。
|
||||
|
||||
**看到喜欢的功能。理解它的原理。按照你的方式实现。**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#调查工具目录)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[网站（英文）](https://morluto.github.io/rea/) · [使用指南](https://morluto.github.io/rea/guides/) · [DX-Ball 案例](https://morluto.github.io/rea/showcase/dx-ball/)**
|
||||
|
||||
[快速开始](#快速开始) · [当前状态](#当前状态) · [从二进制到行为](#从二进制到行为) · [调查工具目录](#调查工具目录) · [路线图](#路线图) · [工作原理](#工作原理)
|
||||
|
||||
<table aria-label="REA community">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="docs/assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>加入逆向工程社区</strong>
|
||||
</a><br />
|
||||
<sub>Discord · 问答 · 成果分享</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="REA 在 Hopper 中启动分析桥并检查原生二进制文件" width="1200" />
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
看到某个应用中想加入自己产品的功能？让智能体用 REA 调查它。即使没有源代码，智能体也可以检查应用、解释功能的工作方式、展示证据，并为你的项目实现类似功能。
|
||||
|
||||
REA 提供原生二进制、JavaScript 与 Electron 应用、.NET 程序集和网站的分析工具。你可以从智能体或终端使用它们。分析在本机运行，结果会说明证据和限制。
|
||||
|
||||
Setup 可以配置智能体、连接已安装的 Hopper 或 Ghidra，也可以在你批准后安装 Hopper。
|
||||
|
||||
## 直接询问智能体
|
||||
|
||||
完成[设置](#快速开始)后，重启智能体并提出问题：
|
||||
|
||||
```text
|
||||
解释“备忘录”应用的搜索功能，展示证据，然后为我的项目实现类似功能。
|
||||
```
|
||||
|
||||
把“备忘录”替换成你想了解的应用，也可以先请求概览。
|
||||
|
||||
## 从二进制到行为
|
||||
|
||||
| 反编译 | 理解 | 重建 |
|
||||
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| 打开原生应用或可执行文件，恢复过程、伪代码、汇编、字符串、符号、段和元数据。 | 沿调用者、被调用者、交叉引用和调用图追踪，直到智能体能够解释功能或算法的实际工作方式。 | 将智能体学到的内容变成适合你的技术栈、界面和需求的产品功能。 |
|
||||
|
||||
REA 让调查始终以二进制证据为依据。它不会声称能恢复原始源代码，也不会自动克隆整个应用。
|
||||
|
||||
## 为什么选择 REA
|
||||
|
||||
| | |
|
||||
| ---------------- | -------------------------------------------------------------------- |
|
||||
| **为智能体设计** | 直接询问应用的行为，让智能体去检查，而不是猜测。 |
|
||||
| **CLI 与 MCP** | 在终端或智能体中使用同一套逆向工程能力。 |
|
||||
| **引导式设置** | 配置智能体、连接已有分析工具，或在你批准后安装 Hopper。 |
|
||||
| **从洞察到代码** | 理解功能后，在同一次编程会话中构建属于你自己的版本。 |
|
||||
| **本地运行** | 分析在受支持的本机系统上运行；REA 不会把二进制上传到托管式分析服务。 |
|
||||
| **保留上下文** | 连续调查多个应用，无需为每个问题重新开始整个分析过程。 |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 运行设置（推荐）
|
||||
|
||||
配置 REA 与智能体的连接：
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
```
|
||||
|
||||
Setup 首先让你多选要连接的智能体。已有 REA 注册默认选中；仅被检测到的客户端不会自动选中，也可以手动选择尚未配置的客户端。查看具体路径和变更后再批准。所选智能体默认安装 REA 工作流；Hopper 是单独的可选操作，需要单独批准。Setup 也可以记录现有 Ghidra 的路径。
|
||||
|
||||
Setup 会先展示变更并备份已有配置。要求与更多选项见[安装与设置](docs/installation.md)。
|
||||
|
||||
### 使用智能体
|
||||
|
||||
设置完成后，重启智能体并描述你想了解的应用或功能。REA 支持 Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf、Devin、OpenCode、Antigravity、GitHub Copilot CLI、Command Code 和 VS Code。已有 REA 注册默认选中；其他检测到的客户端需手动选择。其他智能体可以使用下方的 MCP 配置。
|
||||
|
||||
Hopper 支持演示模式。如果首次启动时出现提示，选择演示模式或输入已有许可证。
|
||||
|
||||
### AI 编程助手技能（可选）
|
||||
|
||||
为你的 AI 编程助手添加 skill，以获得更丰富的上下文：
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea --skill reverse-engineer-anything
|
||||
```
|
||||
|
||||
该 skill 提供 REA 的调查工作流。先运行上文的 setup，把 REA 连接到你的智能体并配置分析工具。Setup 默认会安装版本匹配的 skill；这条命令安装的是仓库版本。
|
||||
|
||||
对于已解包的 JavaScript/Electron 应用目录或 ASAR，无需 MCP 配置，也无需原生引擎：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
把路径换成你的目标（例如 Windows 上的 `"D:/apps/example"`）。这条静态流程支持此类路径。原生分析仍需要先配置对应的分析引擎。
|
||||
|
||||
### 安装 rea 命令
|
||||
|
||||
安装命令行工具：
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
```
|
||||
|
||||
需要预先安装 Node.js 和 npm。在终端中运行时，安装程序会安装 `rea` 并启动设置。
|
||||
|
||||
也可以用 npm 安装，再运行设置：
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
```
|
||||
|
||||
### 系统要求
|
||||
|
||||
- macOS 12 或更高版本
|
||||
- Ubuntu 24.04+、Fedora 41+、64 位 Arch Linux 或 CachyOS
|
||||
- Node.js 22.x (>=22.19)、24.x (>=24.11) 或 26+，以及 npm
|
||||
|
||||
原生二进制分析需要 Hopper 或 Ghidra。Hopper 是独立软件；演示模式有厂商规定的限制，不要求购买许可证。
|
||||
|
||||
Ghidra 支持 Linux x64 和 macOS x64/arm64。单独安装 Ghidra 12.1.x，以及该安装在 `application.java.min` 到 `application.java.max` 中声明的完整 64 位 JDK，然后配置 REA 使用它们。当前 12.1 发行版要求 JDK 21 或更新版本，并且不设置上限。桥接在 Ghidra 12.1.4 和 JDK 21 上完成验证。macOS 还需要对应架构的原生反编译器。
|
||||
|
||||
Setup 可以验证安装并保存路径，不会安装或升级 Ghidra、Java、Node.js、npm 或 Homebrew。
|
||||
|
||||
仓库 main 分支与 npm 4.1.0 已包含实验性的 Windows x64 Ghidra P0 支持，适用于本地 NTFS 上的原生 x86-64 PE 应用（非托管、非 DLL），并附带 Job Object、私有 DACL 与路径准入控制。请先确认[发布边界](docs/installation.md#released-package-and-main)，再假定旧版 npm 包具备该能力。详见 [Windows Ghidra P0](docs/windows-ghidra-p0.md) 的前置条件与已验证范围。
|
||||
|
||||
### 故障排除
|
||||
|
||||
运行 `npx -y rea-agents@latest doctor`，检查主机、依赖、分析工具和智能体配置。该命令不会修改文件。添加 `--json` 可获取结构化诊断。
|
||||
|
||||
Linux 上 REA 会优先使用可执行的 `/opt/hopper/bin/Hopper`；如果不可用，则自动检查 `~/.local/share/rea/hopper/bin/Hopper`。其他路径可通过 `HOPPER_LAUNCHER_PATH` 指定。如果文件存在但仍报告缺少分析引擎，请对实际 Hopper 路径运行 `ldd /absolute/path/to/Hopper | grep 'not found'` 检查缺少的系统库。安装详情见 [Hopper 指南](docs/installation.md#hopper)。
|
||||
|
||||
### 更新与卸载
|
||||
|
||||
- `rea update` 更新当前 REA 安装。
|
||||
- `rea uninstall` 移除 REA 管理的智能体配置项和工作流文件，保留 Hopper。
|
||||
- `rea uninstall --purge-data` 还会删除 REA 的缓存和状态。仅在需要移除这些数据时使用。
|
||||
|
||||
## 当前状态
|
||||
|
||||
仓库当前能力与平台要求见[英文支持指南](README.md#current-status)。main 分支可能领先于 [npm 发布版](docs/installation.md#released-package-and-main)。
|
||||
|
||||
- Ghidra 在 Linux x64、macOS x64/arm64 以及实验性的 Windows x64 P0 边界上提供 25 个只读操作。Linux/macOS 额外支持原子化的会话级函数注释。Windows P0 为只读；Ghidra 不提供 GUI 控制。
|
||||
- 静态 Android 检查需要自备 JADX/Java，前置条件独立；真机验证覆盖 Linux。详见 [Android 分析](docs/android-analysis.md)。
|
||||
- 浏览器、Electron 与进程请求需直接声明目标、动作与生命周期，并依赖宿主的实际访问权限，无需 REA 另行授权。Setup 的配置写入与 Hopper 安装仍需逐项批准确切计划。
|
||||
- `rea capabilities` 描述的是二进制会话操作，而非全部应用/运行时工具。请以已连接 MCP 工具列表与相关指南为准。
|
||||
|
||||
## 一个提示词，完成一次完整调查
|
||||
|
||||
```text
|
||||
逆向工程“备忘录”应用，找到离线搜索功能的工作方式，解释其控制流，
|
||||
并使用 TypeScript 和 SQLite 为我的项目构建一个版本。
|
||||
```
|
||||
|
||||
| 步骤 | 智能体的操作 | REA 工具 |
|
||||
| ---: | ---------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | 打开并识别二进制文件 | `open_binary`, `binary_overview` |
|
||||
| 2 | 搜索可能的离线搜索线索 | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 将线索连接到可执行代码 | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 重建相关控制流 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 反编译相关程序 | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | 在你的项目中构建该功能 | 适合你的技术栈、产品和需求的代码 |
|
||||
|
||||
REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常规文件编辑与测试工具完成。
|
||||
|
||||
## 智能体可以完成什么
|
||||
|
||||
- 在没有源代码时解释某项功能的实现方式。
|
||||
- 重建应用的身份验证、存储、更新或网络流程。
|
||||
- 恢复足够的结构，以记录未公开的格式或接口。
|
||||
- 从字符串或符号追踪到实现可疑行为的代码。
|
||||
- 在一个会话中切换两个应用版本并比较实现路径。
|
||||
- 调查你喜欢的功能，并为自己的产品构建量身定制的版本。
|
||||
- 将恢复的行为转换为产品功能、测试、迁移说明、移植代码或互操作替代品。
|
||||
- 分析 Swift 和 Objective-C 元数据。
|
||||
- 在 Hopper 中留下名称、注释与书签，使人与智能体的分析互相增强。
|
||||
|
||||
## 调查工具目录
|
||||
|
||||
| 工具类别 | 数量 | 用途 |
|
||||
| ----------------- | ---: | ------------------------------------------------------------------------------------------------------ |
|
||||
| 原生检查 | 41 | 函数、伪代码、汇编、字符串、符号、调用、引用、注释、字节读取与文件偏移 |
|
||||
| 调查工作流 | 14 | 应用概览、函数档案、原生 API 与分发、批量反编译、功能追踪、调用路径与调用图、Swift 与 Objective-C 发现 |
|
||||
| macOS 原生工具 | 7 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原，无需启动 Hopper |
|
||||
| 产物图 | 5 | 目录与软件包清单、编译后的 Interface Builder 文件、Apple 资源目录与提取 |
|
||||
| 托管 PE/CLI | 7 | .NET 程序集身份、元数据、CIL 指令、原生依赖声明、重建导入与构建比较 |
|
||||
| 固件 | 2 | Linux 固件区域检查与显式提取 |
|
||||
| Android APK | 5 | 包与 manifest 声明、类搜索、成员清单、方法反编译与静态引用 |
|
||||
| 浏览器观察 | 11 | 页面结构、网络元数据、脚本、来源映射、WebMCP 发现、截图与捕获比较 |
|
||||
| Electron 分析 | 5 | 渲染进程观察、静态应用映射、静态/运行时结果关联 |
|
||||
| JavaScript 运行时 | 2 | Node/Electron Inspector 目标发现、脚本位置与执行上下文事件 |
|
||||
| 应用工作流 | 13 | 捕获的网站脚本导出、跨层功能追踪、构建比较、历史源码映射、静态返回结构比较与重建验证 |
|
||||
| 工作区与观察 | 21 | 会话、证据包、导航上下文、进程/产物/函数比较与待解决问题跟踪 |
|
||||
|
||||
## 路线图
|
||||
|
||||
接下来的工作包括更多原生目标验证、JavaScript 与 .NET 跨版本比较，以及运行时观察和重建验证。当前设置已支持选择智能体集成和 Hopper 安装；安装其他分析工具仍是后续工作。详见[安装路线图](docs/roadmap.md)和[分析工具评估](docs/provider-evaluation.md)。
|
||||
|
||||
## 与其他编程智能体一起使用
|
||||
|
||||
Setup 支持 Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf、Devin、OpenCode、Antigravity、GitHub Copilot CLI、Command Code 和 VS Code。已有 REA 注册默认选中；其他检测到的客户端需手动选择。任何支持本地 MCP 服务器的智能体都可以使用以下配置连接 REA。
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@5.0.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
## 工作原理
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["编程智能体"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["终端"] --> REA
|
||||
REA --> Hopper["Hopper 分析操作"]
|
||||
Hopper --> App["你的应用"]
|
||||
REA --> Ghidra["Ghidra 只读分析<br/>清单 + 函数分析"]
|
||||
Ghidra --> App
|
||||
```
|
||||
|
||||
CLI 与 MCP 服务器使用相同的分析流程。终端命令完成后释放自己创建的桥接会话；智能体会话可以在调查期间保持连接。关闭 REA 会话不会退出用户正在使用的 Hopper 应用。
|
||||
|
||||
## CLI
|
||||
|
||||
上面的智能体工作流是使用 REA 最简单的方式。如果只想在终端中快速了解一个应用：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
运行 `npx -y rea-agents@latest --help` 查看直接反编译和其他选项。
|
||||
|
||||
也可以全局安装 `rea` 命令：
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA 可以直接打开 Mac 的 `.app` 文件夹。如果智能体找不到应用，请告诉它应用安装在哪里。
|
||||
|
||||
## Hopper 应用行为
|
||||
|
||||
REA 会在需要时启动 Hopper，无需预先运行。Hopper 启动器内部会激活应用，因此打开目标时 Hopper 可能出现在其他窗口前。REA 会请求 macOS 在后台启动 Hopper，但无法保证窗口始终位于后台。
|
||||
|
||||
REA 会推导明确的格式和架构参数，以避免常见的 FAT 与 ARM 选择对话框。其他 Hopper 或 macOS 对话框仍可能需要人工响应。关闭 REA 会话会终止桥并删除私有套接字目录，但不会退出用户正在使用的 Hopper 应用。
|
||||
|
||||
## 安全模型
|
||||
|
||||
每个桥会话都使用随机能力令牌和仅限当前用户的 Unix 套接字。诊断保留本机路径、摘要和错误位置等排查信息，同时移除凭据和认证令牌。Ghidra 会话还使用隔离的临时项目，不会打开或修改用户自己的 Ghidra 项目。
|
||||
|
||||
这不是沙箱，也无法防御以同一操作系统用户身份运行的恶意进程。打开不可信二进制文件会让所选本地提供商以当前用户权限进行解析和分析。请按照 [SECURITY.md](SECURITY.md) 中的私密流程报告漏洞。
|
||||
|
||||
## 常见问题
|
||||
|
||||
<details><summary><strong>Hopper 是否需要提前运行？</strong></summary>
|
||||
|
||||
不需要。REA 会在操作需要时启动 Hopper。Hopper 已经运行时也可以使用，但 REA 会打开新的分析文档，不会连接并复用现有 GUI 文档。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 是否包含 Hopper？</strong></summary>
|
||||
|
||||
不包含。Setup 可以为你安装 Hopper，但 Hopper 仍是需要单独授权的软件。REA 提供 CLI、MCP 服务器和面向智能体的工作流。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 会上传我的二进制文件吗？</strong></summary>
|
||||
|
||||
REA 不提供托管分析服务，而是通过本地 Unix 套接字把操作交给 Hopper。你的智能体或模型服务商可能有自己的数据政策，请单独核查。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 能恢复原始源代码吗？</strong></summary>
|
||||
|
||||
不能保证。REA 提供伪代码、汇编、符号、字符串、元数据和关系，智能体可据此解释或兼容地重建观察到的行为。
|
||||
|
||||
</details>
|
||||
|
||||
## 开发
|
||||
|
||||
开发环境、架构、测试和发布说明请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。
|
||||
|
||||
## 许可证
|
||||
|
||||
[MIT](LICENSE)
|
||||
+3
-6
@@ -24,12 +24,9 @@ provider-dependent behavior to be reproduced safely.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
REA authenticates each local provider bridge session with a random capability
|
||||
token and a current-user Unix socket. Tokens are transferred through private
|
||||
session descriptors rather than process arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
running as the same operating-system user. Opening an untrusted binary
|
||||
delegates parsing and analysis to the selected local provider with that user's
|
||||
|
||||
@@ -1,134 +0,0 @@
|
||||
# Third-party notices
|
||||
|
||||
REA-authored code is licensed under [MIT](LICENSE). Third-party material retains
|
||||
its own copyright, license, attribution, and NOTICE requirements. REA's MIT
|
||||
license grants no rights to proprietary analysis tools. Project and product
|
||||
names identify integrations; they do not imply affiliation or endorsement.
|
||||
|
||||
This inventory covers the locked npm dependency graph, code or assets derived
|
||||
from upstream projects, and separately obtained engines used by REA. A listed
|
||||
integration is not a statement that its executable is bundled. Upstream
|
||||
components and caller-selected versions may have additional terms; retain the
|
||||
notices supplied with the software you actually obtain or redistribute.
|
||||
|
||||
## Distributed dependencies, derived code, and assets
|
||||
|
||||
| Material | Use and licensing records |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `evmole@0.9.3`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
| Dinosaur website example | Selected excerpts from [wayou/t-rex-runner](https://github.com/wayou/t-rex-runner/tree/5455bfa408ec6b707c7300ff194b7390733a766d), derived from Chromium's dinosaur game. The website retains the [original BSD-3-Clause notices](website/public/assets/licenses/dinosaur.txt), including Chromium authors and wayou. This asset is not part of the npm runtime. |
|
||||
|
||||
Two locked runtime releases, [`clean-git-ref@2.0.1`](https://github.com/TheSavior/clean-git-ref/tree/bf3c8afc8032bcdc6a53634c18b601dc6df478fb) (declared Apache-2.0) and
|
||||
[`minimisted@2.0.1`](https://github.com/kt3k/minimisted/tree/95fdfb8d6efb9546c1117854fe3a58105f6332ce) (declared MIT), omit a full license/copyright document in their
|
||||
published archives and inspected upstream release sources. Their runtime notice
|
||||
entries preserve the declared license and author metadata and include a clearly
|
||||
labeled standard license text. These are documented upstream notice gaps, not
|
||||
recovered copyright notices or a claim that attribution has been fully verified.
|
||||
|
||||
The optional `frida@17.23.2` npm archive and tagged Node binding source also
|
||||
omit a standalone license/copyright document. Its declared
|
||||
`LGPL-2.0 WITH WxWindows-exception-3.1` license is recorded with unchanged
|
||||
[Frida licensing text](licenses/frida-17.23.2-COPYING.txt) and the
|
||||
[GNU Library GPL v2](licenses/LGPL-2.0.txt). These are licensing supplements,
|
||||
not recovered Node binding or prebuild-component copyright notices.
|
||||
|
||||
## Open-source engine credits and licenses
|
||||
|
||||
We thank the maintainers and contributors of the projects below. Their
|
||||
decompilers, binary-analysis engines, debuggers, parsers, and capture tools
|
||||
provide capabilities used through REA's integrations. Each row identifies the
|
||||
upstream project, its own license, and its role in REA. In particular, JADX
|
||||
provides the Android decompiler, while jadx-headless-mcp provides the separate
|
||||
headless integration used by REA; both projects deserve credit.
|
||||
|
||||
These acknowledgments are voluntary thanks. Required upstream copyright and
|
||||
NOTICE texts remain in their original form and are not replaced by this
|
||||
summary. Credits do not imply endorsement or change any license.
|
||||
|
||||
REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra 12.1.4. Copies of its [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
copyright protection, while other portions retain their authors' rights. Its
|
||||
modules include additional third-party licenses, and standalone programs under
|
||||
`GPL/` have GPLv3 terms. The three retained top-level documents do not replace
|
||||
the module notices and license directories in a complete Ghidra distribution.
|
||||
REA is not affiliated with or endorsed by the NSA.
|
||||
|
||||
## Commercial tools and bridges
|
||||
|
||||
**You must legally obtain the tools and the rights needed for your use.** REA
|
||||
does not include an IDA Pro, Hex-Rays decompiler, Hopper, or JEB license, license
|
||||
key, or activation entitlement. A permitted trial/demo may be used only within
|
||||
its official terms. Cracked copies, stolen keys, unauthorized sharing, and
|
||||
bypassing activation or license limits are not authorized by REA.
|
||||
|
||||
| Project / product | Applicable terms and distinction |
|
||||
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| IDA Pro and Hex-Rays decompilers | Proprietary Hex-Rays software governed by the [EULA](https://hex-rays.com/eula), purchase agreement, and applicable license scheme. Obtain rights for the required product, user, machine, decompiler, and automation interfaces. Availability of an API or a bridge does not grant hosting, service, or additional-user rights. |
|
||||
| ida-pro-mcp | The bridge's [MIT license](https://github.com/mrexodia/ida-pro-mcp/blob/main/LICENSE) belongs to Duncan Ogilvie. It licenses the bridge code, not IDA or Hex-Rays products. Its [README](https://github.com/mrexodia/ida-pro-mcp#prerequisites) requires IDA Pro and explicitly states that IDA Free is unsupported. Follow the prerequisites for the version you actually install. The bridge is separately supplied, not bundled by REA. |
|
||||
| Hopper Disassembler | Proprietary Cryptic Apps software under its [license agreement](https://www.hopperapp.com/license_agreement.html). Personal and computer licenses have different permitted-user and installation rules; an official evaluation trial is available. REA setup may obtain the official installer after approval, but this grants no paid license or activation entitlement. |
|
||||
| JEB | Proprietary PNF Software product under its [EULA](https://www.pnfsoftware.com/jeb/LICENSE.TXT) and edition-specific agreement. Check user/seat/server rights and whether your edition permits automation, backend/service integration, and disclosure of generated output. The published EULA restricts public or third-party disclosure of decompiled code, reconstructed artifacts, and analysis reports unless expressly permitted by the license type. |
|
||||
|
||||
The vendor's applicable agreement controls. These summaries do not interpret
|
||||
your individual contract or expand the rights it grants. Verify permissions
|
||||
before publishing analysis output, sharing tool access, or operating a service.
|
||||
Vendor terms above were checked on 2026-10-12.
|
||||
|
||||
## Runtimes, development tooling, and fixtures
|
||||
|
||||
REA uses a caller-supplied Node.js runtime ([MIT and component notices](https://github.com/nodejs/node/blob/main/LICENSE)).
|
||||
Optional workflows may also use Python ([PSF and component licenses](https://docs.python.org/3/license.html)),
|
||||
OpenJDK ([GPLv2 with Classpath Exception and component notices](https://openjdk.org/legal/)),
|
||||
Electron ([MIT and component notices](https://github.com/electron/electron/blob/main/LICENSE)),
|
||||
or a Chromium-based browser ([BSD-3-Clause and third-party licenses](https://chromium.googlesource.com/chromium/src/+/main/LICENSE)).
|
||||
These external installations are not licensed or bundled by REA's MIT license.
|
||||
|
||||
Development, test, and website npm packages are included in the locked inventory.
|
||||
GitHub Actions, operating-system utilities, downloaded verification fixtures,
|
||||
and their runtimes retain the terms supplied by their publishers. Fixture
|
||||
identities and provenance are recorded under `scripts/fixtures/` and the
|
||||
existing `third_party/` records; downloaded executable fixtures are not tracked
|
||||
or included in the npm package. This notice does not claim to inventory every
|
||||
dependency of a caller's operating system, agent, or separately installed tool.
|
||||
|
||||
## Maintaining and redistributing notices
|
||||
|
||||
Refresh the dependency inventory and runtime notice collection when the lockfile
|
||||
changes. Review the actual versions and distributed contents, retain the
|
||||
upstream license and copyright texts, and account for optional platform
|
||||
packages. See [licenses/README.md](licenses/README.md) for source provenance,
|
||||
collection scope, and known upstream notice gaps.
|
||||
|
||||
The npm package includes this document, the `licenses/` directory, and the
|
||||
existing redistributed license records under `third_party/`. If you additionally
|
||||
bundle an external engine or modify upstream code, review that actual
|
||||
distribution and retain its full applicable licenses, copyright notices,
|
||||
NOTICE files, modification notices, and source obligations. REA's inventory is
|
||||
not a substitute for those obligations.
|
||||
@@ -3,14 +3,8 @@ import com.google.gson.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.server.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.types.*;
|
||||
import jadx.api.*;
|
||||
import jadx.api.plugins.input.data.ICodeReader;
|
||||
import jadx.api.plugins.input.data.IMethodRef;
|
||||
import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import java.io.PrintStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.stream.Collectors;
|
||||
@@ -31,11 +25,6 @@ class ReaJadxBridge {
|
||||
private record MethodSnapshot(String name, String shortId, Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
private static CallToolResult text(String value, boolean failed) {
|
||||
@@ -85,17 +74,11 @@ class ReaJadxBridge {
|
||||
ordered.sort(Comparator.comparing(MethodNode::getAlias));
|
||||
List<MethodSnapshot> methods = ordered.stream().map(method -> {
|
||||
String name = method.getAlias();
|
||||
String shortId = method.getMethodInfo().getShortId();
|
||||
String rawName = method.getMethodInfo().getName();
|
||||
if (!shortId.startsWith(rawName + "("))
|
||||
throw new IllegalStateException("JADX method ID has no raw-name prefix: " + shortId);
|
||||
String descriptor = shortId.substring(rawName.length());
|
||||
String signature = name + "(" + method.getArgTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + method.getReturnType().toString();
|
||||
return new MethodSnapshot(name, shortId, Map.of(
|
||||
return new MethodSnapshot(name, method.getMethodInfo().getShortId(), Map.of(
|
||||
"name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor,
|
||||
"is_constructor", method.isConstructor(), "def_pos", 0));
|
||||
}).toList();
|
||||
List<Map<String, Object>> fields = node.getFields().stream()
|
||||
@@ -117,9 +100,8 @@ class ReaJadxBridge {
|
||||
}
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
+ "; use search_android_classes to discover fully qualified names.");
|
||||
if (candidates.size() != 1) throw new InputFailure("class_name", "Ambiguous metadata class identity: " + name
|
||||
if (candidates == null) throw new IllegalArgumentException("Exact class not found: " + name);
|
||||
if (candidates.size() != 1) throw new IllegalArgumentException("Ambiguous metadata class identity: " + name
|
||||
+ "; raw identities: " + candidates.stream().map(candidate -> candidate.node().getRawName()).toList());
|
||||
return candidates.get(0);
|
||||
}
|
||||
@@ -127,8 +109,7 @@ class ReaJadxBridge {
|
||||
private List<MethodSnapshot> methods(ClassSnapshot cls, JsonObject args) {
|
||||
String name = string(args, "method_name");
|
||||
List<MethodSnapshot> result = cls.methods().stream().filter(method -> method.name().equals(name)).toList();
|
||||
if (result.isEmpty()) throw new InputFailure("method_name", "Method not found: " + name
|
||||
+ "; use inspect_android_class to discover available methods.");
|
||||
if (result.isEmpty()) throw new IllegalArgumentException("Method not found: " + name);
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -138,71 +119,6 @@ class ReaJadxBridge {
|
||||
return result;
|
||||
}
|
||||
|
||||
private JadxSession.SmartCode methodBody(JadxSession current, ClassSnapshot cls,
|
||||
MethodSnapshot selected, JavaMethod method) {
|
||||
if (!selected.shortId().equals("<clinit>()V"))
|
||||
return current.getMethodBodySmart(method, current.getMaxSourceBytes(), true);
|
||||
// Java code generation can fold all or part of <clinit> into field declarations.
|
||||
// Its method text cannot establish coverage, even when a static block remains.
|
||||
JadxSession.MethodSmali smali = current.getMethodSmali(cls.node(), method.getName());
|
||||
if (!smali.getFound() || smali.getBlocks().size() != 1)
|
||||
throw new IllegalArgumentException("Static initializer disassembly requires one exact method: " + selected.shortId());
|
||||
return new JadxSession.SmartCode(
|
||||
TextUtilKt.truncateToBytes(smali.getBlocks().get(0), current.getMaxSourceBytes()),
|
||||
"smali", true, List.of("static_initializer_disassembly"));
|
||||
}
|
||||
|
||||
// The local transport admits 8 MiB frames. Reserve 1 KiB for the fixed
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
for (Object value : site.values())
|
||||
if (value instanceof String text) bound += 12L * text.length();
|
||||
if (bound > remaining) return false;
|
||||
long bytes = escapedJsonBytes(site) - 2L + 1L;
|
||||
if (bytes > remaining) return false;
|
||||
remaining -= bytes;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
});
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, JsonObject args) {
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
"engine_reported_version", BuildInfo.VERSION,
|
||||
@@ -225,9 +141,9 @@ class ReaJadxBridge {
|
||||
if (name.equals("get_method_body")) {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= overloads.size()) throw new InputFailure("overload_index", "overload_index out of range: " + index);
|
||||
if (index >= overloads.size()) throw new IllegalArgumentException("overload_index out of range: " + index);
|
||||
JavaMethod method = method(cls, overloads.get(index));
|
||||
JadxSession.SmartCode code = methodBody(current, cls, overloads.get(index), method);
|
||||
JadxSession.SmartCode code = current.getMethodBodySmart(method, current.getMaxSourceBytes(), true);
|
||||
return json(Map.of("class_name", cls.summary().get("full_name"),
|
||||
"method_name", overloads.get(index).name(), "full_name", method.getFullName(),
|
||||
"overload_index", index, "overload_count", overloads.size(),
|
||||
@@ -236,52 +152,17 @@ class ReaJadxBridge {
|
||||
}
|
||||
List<JavaNode> uses;
|
||||
String target = string(args, "class_name");
|
||||
MethodSnapshot selectedMethod = null;
|
||||
int overloadIndex = 0;
|
||||
if (name.equals("get_xrefs_to_class")) uses = cls.node().getUseIn();
|
||||
else {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
if (overloads.size() != 1 && !args.containsKey("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + overloads.size() + " method overloads");
|
||||
overloadIndex = integer(args, "overload_index", 0);
|
||||
if (overloadIndex >= overloads.size())
|
||||
throw new InputFailure("overload_index", "overload_index out of range: " + overloadIndex);
|
||||
selectedMethod = overloads.get(overloadIndex);
|
||||
target += "." + selectedMethod.name();
|
||||
uses = method(cls, selectedMethod).getUseIn();
|
||||
if (overloads.size() != 1) throw new IllegalArgumentException("Method references require a unique overload");
|
||||
target += "." + overloads.get(0).name();
|
||||
uses = method(cls, overloads.get(0)).getUseIn();
|
||||
}
|
||||
// REA consumes all references without materializing source line information.
|
||||
MethodInfo targetInfo = selectedMethod == null ? null : method(cls, selectedMethod).getMethodNode().getMethodInfo();
|
||||
List<Map<String, Object>> items = uses.stream().map(node -> {
|
||||
Map<String, Object> item = new LinkedHashMap<>(current.describeUsage(node, false));
|
||||
if (targetInfo != null && node instanceof JavaMethod) {
|
||||
item.put("direct_invokes", List.of());
|
||||
item.put("direct_invokes_exhaustive", false);
|
||||
}
|
||||
return item;
|
||||
}).toList();
|
||||
Map<String, Object> result = new LinkedHashMap<>(Map.of("target", target,
|
||||
"total", uses.size(), "count", items.size(), "resolve_line", false, "items", items));
|
||||
if (selectedMethod != null) {
|
||||
result.put("overload_index", overloadIndex);
|
||||
result.put("reported_signature", selectedMethod.summary().get("signature"));
|
||||
}
|
||||
if (targetInfo != null) {
|
||||
InvokeBudget budget = new InvokeBudget(MAX_XREF_TEXT_BYTES - escapedJsonBytes(result));
|
||||
for (int index = 0; index < uses.size(); index++) {
|
||||
if (!(uses.get(index) instanceof JavaMethod source)) continue;
|
||||
DirectInvokes invokes = directInvokes(source, targetInfo, budget);
|
||||
Map<String, Object> item = items.get(index);
|
||||
if (invokes == null) {
|
||||
item.remove("direct_invokes");
|
||||
item.remove("direct_invokes_exhaustive");
|
||||
} else {
|
||||
item.put("direct_invokes", invokes.sites());
|
||||
item.put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
}
|
||||
}
|
||||
}
|
||||
return json(result);
|
||||
List<Map<String, Object>> items = uses.stream().map(node -> current.describeUsage(node, false)).toList();
|
||||
return json(Map.of("target", target, "total", uses.size(), "count", items.size(),
|
||||
"resolve_line", false, "items", items));
|
||||
}
|
||||
|
||||
public static void main(String[] args) throws Exception {
|
||||
@@ -315,7 +196,7 @@ class ReaJadxBridge {
|
||||
List<String> fields = switch (name) {
|
||||
case "list_classes" -> List.of("offset", "limit");
|
||||
case "get_method_body" -> List.of("class_name", "method_name", "overload_index");
|
||||
case "get_xrefs_to_method" -> List.of("class_name", "method_name", "overload_index");
|
||||
case "get_xrefs_to_method" -> List.of("class_name", "method_name");
|
||||
default -> List.of("class_name");
|
||||
};
|
||||
Map<String, kotlinx.serialization.json.JsonElement> properties = new LinkedHashMap<>();
|
||||
@@ -323,15 +204,13 @@ class ReaJadxBridge {
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Indexed pages of all top-level class names, with offset and limit.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation.";
|
||||
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
|
||||
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
|
||||
case "get_method_body" -> "Source for an exact class and stable same-name overload index, with smali fallback.";
|
||||
default -> "Complete static references to an exact class or unique method; source lines are unresolved.";
|
||||
};
|
||||
Tool narrowed = new Tool(name, new ToolSchema(null, new JsonObject(properties),
|
||||
tool.getInputSchema().getRequired(), null), description, null, null, null, null, null, null);
|
||||
server.addTool(narrowed, (connection, request, continuation) -> {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
import java.util.Map;
|
||||
|
||||
import ghidra.app.plugin.core.analysis.AutoAnalysisManager;
|
||||
import ghidra.app.util.headless.HeadlessScript;
|
||||
|
||||
/** Disable the named auto-analyzers before default auto-analysis. */
|
||||
public final class ReaGhidraAnalysisOptions extends HeadlessScript {
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 1 || arguments[0].isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"ReaGhidraAnalysisOptions requires one comma-separated analyzer-name argument");
|
||||
}
|
||||
String[] names = arguments[0].split(",");
|
||||
AutoAnalysisManager analysis = AutoAnalysisManager.getAnalysisManager(currentProgram);
|
||||
Map<String, String> options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (analysis.getAnalyzer(name) == null || !options.containsKey(name)) {
|
||||
throw new IllegalArgumentException("Unknown Ghidra analyzer for this program: " + name);
|
||||
}
|
||||
}
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
setAnalysisOption(currentProgram, requestedName.trim(), "false");
|
||||
}
|
||||
options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (!"false".equals(options.get(name))) {
|
||||
throw new IllegalStateException("Ghidra analyzer remained enabled: " + name);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,151 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.address.AddressSpace;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.symbol.SourceType;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Apply caller-declared analysis seeds before default auto-analysis.
|
||||
*
|
||||
* The single argument is REA's canonical seed snapshot: one
|
||||
* {@code 0x<hex>\t(function|code|label)[\tname]} line per seed. Code seeds are
|
||||
* applied after functions (highest address first), then labels. The outcome is stored as a
|
||||
* program property that the bridge reports during its handshake.
|
||||
*/
|
||||
public final class ReaGhidraApplySeeds extends GhidraScript {
|
||||
static final String REPORT_OPTION = "REA analysis seeds";
|
||||
|
||||
private record Seed(Address address, String kind, String name) {}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] args = getScriptArgs();
|
||||
if (args.length != 1) {
|
||||
throw new IllegalArgumentException("REA seed script requires exactly one seed file");
|
||||
}
|
||||
Path path = Path.of(args[0]);
|
||||
if (!Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)) {
|
||||
throw new IllegalArgumentException("REA seed file is not a regular file");
|
||||
}
|
||||
byte[] bytes = Files.readAllBytes(path);
|
||||
String sha256 = HexFormat.of().formatHex(
|
||||
MessageDigest.getInstance("SHA-256").digest(bytes)
|
||||
);
|
||||
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
|
||||
List<Seed> seeds = new ArrayList<>();
|
||||
for (String line : new String(bytes, StandardCharsets.UTF_8).split("\n")) {
|
||||
if (line.isEmpty()) continue;
|
||||
String[] fields = line.split("\t", -1);
|
||||
if (fields.length < 2 || fields.length > 3 || !fields[0].startsWith("0x")) {
|
||||
throw new IllegalArgumentException("REA seed snapshot is malformed");
|
||||
}
|
||||
Address address;
|
||||
try {
|
||||
address = space.getAddress(Long.parseUnsignedLong(fields[0].substring(2), 16));
|
||||
} catch (ghidra.program.model.address.AddressOutOfBoundsException e) {
|
||||
address = null;
|
||||
}
|
||||
seeds.add(new Seed(address, fields[1], fields.length == 3 ? fields[2] : null));
|
||||
}
|
||||
|
||||
// Highest address first: a function body computed before analysis stops at
|
||||
// an existing function entry, so later creations cannot swallow earlier ones.
|
||||
seeds.sort((left, right) -> {
|
||||
if (left.address() == null) return right.address() == null ? 0 : 1;
|
||||
if (right.address() == null) return -1;
|
||||
return right.address().compareTo(left.address());
|
||||
});
|
||||
int unmapped = 0;
|
||||
int logged = 0;
|
||||
int codeDecoded = 0, codeFailed = 0;
|
||||
int functionCreated = 0, functionExisting = 0, functionFailed = 0;
|
||||
int labelApplied = 0, labelFailed = 0;
|
||||
for (String pass : new String[] { "function", "code", "label" }) {
|
||||
for (Seed seed : seeds) {
|
||||
if (!seed.kind().equals(pass)) continue;
|
||||
monitor.checkCancelled();
|
||||
MemoryBlock block = seed.address() == null
|
||||
? null
|
||||
: currentProgram.getMemory().getBlock(seed.address());
|
||||
if (block == null || (!pass.equals("label") && !block.isInitialized())) {
|
||||
unmapped++;
|
||||
continue;
|
||||
}
|
||||
switch (pass) {
|
||||
case "code" -> {
|
||||
if (decoded(seed.address())) codeDecoded++;
|
||||
else {
|
||||
codeFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: code " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
case "function" -> {
|
||||
Function existing = getFunctionAt(seed.address());
|
||||
if (existing != null) {
|
||||
functionExisting++;
|
||||
if (seed.name() != null) {
|
||||
existing.setName(seed.name(), SourceType.USER_DEFINED);
|
||||
}
|
||||
} else if (decoded(seed.address()) &&
|
||||
createFunction(seed.address(), seed.name()) != null) {
|
||||
functionCreated++;
|
||||
} else {
|
||||
functionFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: function " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
default -> {
|
||||
try {
|
||||
createLabel(seed.address(), seed.name(), true, SourceType.USER_DEFINED);
|
||||
labelApplied++;
|
||||
} catch (Exception e) {
|
||||
labelFailed++;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
String report = "{\"format\":\"rea-ghidra-seeds-v1\",\"sha256\":\"" + sha256 + "\"" +
|
||||
",\"entries\":" + seeds.size() +
|
||||
",\"function_created\":" + functionCreated +
|
||||
",\"function_existing\":" + functionExisting +
|
||||
",\"function_failed\":" + functionFailed +
|
||||
",\"code_decoded\":" + codeDecoded +
|
||||
",\"code_failed\":" + codeFailed +
|
||||
",\"label_applied\":" + labelApplied +
|
||||
",\"label_failed\":" + labelFailed +
|
||||
",\"unmapped\":" + unmapped + "}";
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(REPORT_OPTION, report);
|
||||
println("REA seeds applied: " + report);
|
||||
}
|
||||
|
||||
private String why(Address address) {
|
||||
if (getDataAt(address) != null) return "defined data " + getDataAt(address).getDataType().getName();
|
||||
var containing = getInstructionContaining(address);
|
||||
if (containing != null && !containing.getAddress().equals(address))
|
||||
return "inside instruction at " + containing.getAddress();
|
||||
Function owner = getFunctionContaining(address);
|
||||
if (owner != null) return "inside function " + owner.getEntryPoint();
|
||||
return getInstructionAt(address) == null ? "undecodable" : "function creation refused";
|
||||
}
|
||||
|
||||
/** Decode one instruction flow at an address, keeping an existing decode. */
|
||||
private boolean decoded(Address address) {
|
||||
if (getInstructionAt(address) != null) return true;
|
||||
if (getDataAt(address) != null || getInstructionContaining(address) != null) {
|
||||
return false;
|
||||
}
|
||||
disassemble(address);
|
||||
return getInstructionAt(address) != null;
|
||||
}
|
||||
}
|
||||
+260
-1362
File diff suppressed because it is too large
Load Diff
@@ -1,14 +0,0 @@
|
||||
import org.osgi.framework.BundleActivator;
|
||||
import org.osgi.framework.BundleContext;
|
||||
|
||||
// Compile the no-op activator with REA's scripts. Otherwise Ghidra generates
|
||||
// one afterward and opens a second Java compiler and file manager.
|
||||
public final class ReaGhidraBundleActivator implements BundleActivator {
|
||||
@Override
|
||||
public void start(BundleContext context) {
|
||||
}
|
||||
|
||||
@Override
|
||||
public void stop(BundleContext context) {
|
||||
}
|
||||
}
|
||||
@@ -1,176 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.AbstractFloatDataType;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.ParameterDefinition;
|
||||
import ghidra.program.model.data.TypeDef;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Soft-float o32 passes and returns float/double values exactly where same-size integers go.
|
||||
* Ghidra's MIPS32 compiler specifications model only hard-float o32, so every typed float
|
||||
* prototype (functions, externals, thunks via their targets, and function-pointer definitions)
|
||||
* would otherwise read $f12/$f14/$f0. Derive a program-specific prototype model from the
|
||||
* program's own default o32 model with its floating-point entries and rules removed, install it
|
||||
* as a specification extension, and assign it to every callable whose formal types use floats.
|
||||
*/
|
||||
public final class ReaGhidraMipsSoftFloat extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA MIPS soft-float storage";
|
||||
static final String POLICY = "o32-soft-float-prototype-v1";
|
||||
static final String PROTOTYPE = "__rea_soft_float";
|
||||
private static final Pattern FPU_REGISTER = Pattern.compile("f\\d+(_\\d+)?");
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String processor = currentProgram.getLanguage().getProcessor().toString();
|
||||
if (!processor.equals("MIPS") || currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA soft-float storage requires a 32-bit MIPS program, not " + processor);
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
PrototypeModel base = spec.getDefaultCallingConvention();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(softFloatPrototype(base, spec), monitor);
|
||||
PrototypeModel soft = currentProgram.getCompilerSpec().getCallingConvention(PROTOTYPE);
|
||||
if (soft == null)
|
||||
throw new IllegalStateException("REA soft-float prototype was not installed");
|
||||
requireIntegerStorage(
|
||||
soft.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
FloatDataType.dataType, IntegerDataType.dataType, DoubleDataType.dataType,
|
||||
FloatDataType.dataType, DoubleDataType.dataType
|
||||
},
|
||||
false),
|
||||
"soft-float prototype self-check");
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk() || !usesFloat(function)) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(PROTOTYPE);
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters()) storage.add(parameter.getVariableStorage());
|
||||
requireIntegerStorage(storage.toArray(VariableStorage[]::new), function.getName(true));
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition) || !usesFloat(definition))
|
||||
continue;
|
||||
definition.setCallingConvention(PROTOTYPE);
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + POLICY + "\",\"functions\":" + functions +
|
||||
",\"definitions\":" + definitions + "}");
|
||||
println("REA MIPS soft-float prototype: functions=" + functions + " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default o32 model without floating-point registers or float-consuming rules. */
|
||||
private static String softFloatPrototype(PrototypeModel base, CompilerSpec spec) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", PROTOTYPE);
|
||||
List<Node> removed = new ArrayList<>();
|
||||
collect(root.getElementsByTagName("pentry"), "metatype", "float", null, removed);
|
||||
collect(root.getElementsByTagName("consume"), "storage", "float", "rule", removed);
|
||||
if (removed.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** Select matching elements, or their nearest `ancestor` element when one is named. */
|
||||
private static void collect(
|
||||
NodeList nodes, String attribute, String value, String ancestor, List<Node> selected) {
|
||||
for (int index = 0; index < nodes.getLength(); index += 1) {
|
||||
Element element = (Element) nodes.item(index);
|
||||
if (!value.equals(element.getAttribute(attribute))) continue;
|
||||
Node target = element;
|
||||
while (ancestor != null && target != null && !ancestor.equals(target.getNodeName()))
|
||||
target = target.getParentNode();
|
||||
if (target != null && !selected.contains(target)) selected.add(target);
|
||||
}
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(VariableStorage[] storage, String owner) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || item.isUnassignedStorage())
|
||||
throw new IllegalStateException("REA soft-float storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FPU_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA soft-float storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean usesFloat(Function function) {
|
||||
if (isFloat(function.getReturn().getFormalDataType())) return true;
|
||||
for (Parameter parameter : function.getParameters()) {
|
||||
if (isFloat(parameter.getFormalDataType())) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean usesFloat(FunctionDefinition definition) {
|
||||
if (isFloat(definition.getReturnType())) return true;
|
||||
for (ParameterDefinition argument : definition.getArguments()) {
|
||||
if (isFloat(argument.getDataType())) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloat(DataType type) {
|
||||
DataType current = type;
|
||||
while (current instanceof TypeDef typeDef) current = typeDef.getBaseDataType();
|
||||
return current instanceof AbstractFloatDataType;
|
||||
}
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
import ghidra.app.cmd.function.CreateFunctionCmd;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.symbol.FlowType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.pcode.PcodeOp;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.Deque;
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
|
||||
/**
|
||||
* Undo wrong no-return flags from auto-analysis. A function flagged no-return that contains a
|
||||
* real RET is an ordinary function (stream readers, cxa_guard_acquire, ...). Known returning
|
||||
* libc/TLS imports have no body to inspect, so clear their flags by exact external ABI name.
|
||||
* Callers treated the call as a terminator and the decompiler truncated them. Clear the flag, then re-disassemble only
|
||||
* the call sites of the functions that were cleared and refit their owning functions in place.
|
||||
*/
|
||||
public final class ReaGhidraNoReturnFix extends GhidraScript {
|
||||
private static final Set<String> RETURNING_IMPORTS = Set.of(
|
||||
"malloc", "__tls_get_addr", "memcpy", "memset", "strlen", "__errno_location");
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
// Windows P0 does not admit analysis-database mutation.
|
||||
if (System.getProperty("os.name", "").startsWith("Windows")) return;
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
Listing li = currentProgram.getListing();
|
||||
Set<Address> cleared = new HashSet<>();
|
||||
Deque<Address> pending = new ArrayDeque<>();
|
||||
for (Function f : fm.getExternalFunctions()) {
|
||||
monitor.checkCancelled();
|
||||
// Do not apply import-name assumptions to local functions or to arbitrary externals.
|
||||
if (!f.hasNoReturn() || !RETURNING_IMPORTS.contains(externalAbiName(f))) continue;
|
||||
f.setNoReturn(false);
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
for (Function f : fm.getFunctions(true)) {
|
||||
monitor.checkCancelled();
|
||||
if (clearDecodedReturn(f, li)) {
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
}
|
||||
int redone = 0;
|
||||
TreeSet<Address> owners = new TreeSet<>();
|
||||
while (!pending.isEmpty()) {
|
||||
Address target = pending.removeFirst();
|
||||
TreeSet<Address> sites = new TreeSet<>();
|
||||
// References include computed import calls after Ghidra resolves their destination.
|
||||
// Collect before redisassembly mutates the reference table.
|
||||
for (Reference reference : currentProgram.getReferenceManager().getReferencesTo(target)) {
|
||||
monitor.checkCancelled();
|
||||
sites.add(reference.getFromAddress());
|
||||
}
|
||||
for (Address s : sites) {
|
||||
monitor.checkCancelled();
|
||||
Instruction ins = li.getInstructionAt(s);
|
||||
if (ins == null) continue;
|
||||
FlowType ft = ins.getFlowType();
|
||||
if (!(ft.isCall() && ft.isTerminal())) continue;
|
||||
Address[] flows = ins.getFlows();
|
||||
boolean callsTarget = false;
|
||||
for (Address flow : flows) {
|
||||
if (flow.equals(target)) callsTarget = true;
|
||||
}
|
||||
if (!callsTarget) continue;
|
||||
// Do not redisassemble into an independently identified function.
|
||||
Address next = ins.getMaxAddress().add(1);
|
||||
if (fm.getFunctionAt(next) != null || !currentProgram.getMemory().contains(next)) continue;
|
||||
Function owner = fm.getFunctionContaining(s);
|
||||
if (owner != null && !owner.isThunk()) owners.add(owner.getEntryPoint());
|
||||
clearListing(ins.getMinAddress(), ins.getMaxAddress());
|
||||
disassemble(s);
|
||||
redone++;
|
||||
// Refit the body in place so the name, signature and calling convention survive.
|
||||
if (owner != null && !owner.isThunk()) {
|
||||
CreateFunctionCmd.fixupFunctionBody(currentProgram, owner, monitor);
|
||||
// Auto-analysis may have propagated the false flag to this truncated caller.
|
||||
// Its RET becomes visible only now; repair its callers in the same pass.
|
||||
if (clearDecodedReturn(owner, li)) enqueueReturning(owner, cleared, pending);
|
||||
}
|
||||
}
|
||||
}
|
||||
// No analyzeChanges here: on libpl2.so it adds about 70 s, which pushes startup past the 330 s deadline.
|
||||
println(
|
||||
"REA no-return fix: cleared=" + cleared.size() + " sites=" + redone + " refit=" + owners.size());
|
||||
}
|
||||
|
||||
private void enqueueReturning(Function function, Set<Address> cleared, Deque<Address> pending) {
|
||||
if (cleared.add(function.getEntryPoint())) pending.addLast(function.getEntryPoint());
|
||||
// A thunk inherits the flag of the function it forwards to, including through other thunks.
|
||||
Address[] thunks = function.getFunctionThunkAddresses(true);
|
||||
if (thunks == null) return;
|
||||
for (Address thunk : thunks) {
|
||||
if (cleared.add(thunk)) pending.addLast(thunk);
|
||||
}
|
||||
}
|
||||
|
||||
private String externalAbiName(Function function) {
|
||||
String name = function.getName();
|
||||
// Mach-O stores C-linkage symbols with one object-format underscore prefix.
|
||||
if (currentProgram.getExecutableFormat().contains("Mach-O") && name.startsWith("_"))
|
||||
return name.substring(1);
|
||||
return name;
|
||||
}
|
||||
|
||||
private boolean clearDecodedReturn(Function function, Listing listing) throws Exception {
|
||||
if (!function.hasNoReturn() || function.isThunk()) return false;
|
||||
for (Instruction instruction : listing.getInstructions(function.getBody(), true)) {
|
||||
monitor.checkCancelled();
|
||||
FlowType flow = instruction.getFlowType();
|
||||
if (flow.isJump() || flow.isCall()) continue;
|
||||
for (PcodeOp operation : instruction.getPcode()) {
|
||||
if (operation.getOpcode() != PcodeOp.RETURN) continue;
|
||||
function.setNoReturn(false);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.mem.MemoryBlockSourceInfo;
|
||||
|
||||
/** Seed the caller-supplied raw-binary entry before default auto-analysis. */
|
||||
public final class ReaGhidraPrepareRaw extends GhidraScript {
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 2) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary preparation requires the committed language and entry address"
|
||||
);
|
||||
}
|
||||
String expectedLanguage = arguments[0];
|
||||
String entryHex = arguments[1];
|
||||
if (!currentProgram.getLanguageID().getIdAsString().equals(expectedLanguage) ||
|
||||
!currentProgram.getExecutableFormat().equals("Raw Binary")) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary preparation requires the committed language and BinaryLoader import"
|
||||
);
|
||||
}
|
||||
Address entry = currentProgram.getAddressFactory().getDefaultAddressSpace().getAddress(entryHex);
|
||||
MemoryBlock block = entry == null ? null : currentProgram.getMemory().getBlock(entry);
|
||||
boolean fileStart = false;
|
||||
if (block != null) {
|
||||
// Processor-defined zero page/stack blocks can precede the imported file.
|
||||
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
|
||||
if (source.getFileBytes().isPresent() && source.getFileBytesOffset() == 0 &&
|
||||
entry.equals(source.getMinAddress())) {
|
||||
fileStart = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!fileStart) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary import must map file offset 0 at the caller-supplied entry"
|
||||
);
|
||||
}
|
||||
addEntryPoint(entry);
|
||||
if (!disassemble(entry) || createFunction(entry, "entry") == null) {
|
||||
throw new IllegalStateException("Raw binary entry cannot be decoded as a function");
|
||||
}
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
|
||||
}
|
||||
}
|
||||
+195
-407
@@ -8,12 +8,12 @@ API access on this thread: moving dispatch to a worker can deadlock Hopper.
|
||||
import json
|
||||
import hmac
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
from typing import Any, Optional, Protocol, Sequence
|
||||
|
||||
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
|
||||
# The native label setter truncates NSString names at this UTF-16 extent.
|
||||
HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
_selected_document = None
|
||||
_rea_session_document = None
|
||||
_search_inventory_cache = {}
|
||||
_pseudocode_cache = {}
|
||||
@@ -89,11 +89,9 @@ def _session_document():
|
||||
global _rea_session_document
|
||||
documents = _api().documents()
|
||||
if _rea_session_document is not None:
|
||||
# Hopper recreates Python wrappers; equality compares native handles.
|
||||
# Retain our bound wrapper so cache identity stays stable across calls.
|
||||
return (
|
||||
_rea_session_document
|
||||
if any(document == _rea_session_document for document in documents)
|
||||
if any(document is _rea_session_document for document in documents)
|
||||
else None
|
||||
)
|
||||
return _bind_session_document(documents)
|
||||
@@ -176,89 +174,46 @@ def _json_safe(value):
|
||||
|
||||
|
||||
def _document(name=None):
|
||||
"""Keep observations and mutations bound to the active REA target."""
|
||||
documents = _api().documents()
|
||||
session_document = _session_document()
|
||||
if session_document is None:
|
||||
raise CapabilityUnavailableError("The active REA target's Hopper document is no longer open; reopen the target with open_binary")
|
||||
"""Resolve an explicit or session-selected document without changing Hopper UI."""
|
||||
global _selected_document
|
||||
api = _api()
|
||||
documents = api.documents()
|
||||
if name is not None:
|
||||
# A document name is display metadata; scope it to the bound native
|
||||
# handle even when another open target happens to have the same name.
|
||||
if session_document.getDocumentName() == name:
|
||||
return session_document
|
||||
if any(candidate.getDocumentName() == name for candidate in documents):
|
||||
raise InvalidRequestError("Hopper document belongs to another target; use open_binary to select that target")
|
||||
matches = [candidate for candidate in documents if candidate.getDocumentName() == name]
|
||||
if len(matches) == 1:
|
||||
return matches[0]
|
||||
if len(matches) > 1:
|
||||
current = api.current_document()
|
||||
if current is not None and current.getDocumentName() == name:
|
||||
return current
|
||||
raise InvalidRequestError("Hopper document name is ambiguous")
|
||||
raise InvalidRequestError("Unknown Hopper document")
|
||||
return session_document
|
||||
|
||||
|
||||
def _api_text(value, field):
|
||||
"""Validate text before passing it to Hopper's native UTF-8 string API."""
|
||||
if not isinstance(value, str):
|
||||
raise InvalidRequestError("%s must be a string" % field)
|
||||
if "\0" in value:
|
||||
raise InvalidRequestError("%s contains a NUL character, which Hopper's Python API cannot represent" % field)
|
||||
try:
|
||||
value.encode("utf-8")
|
||||
except UnicodeEncodeError as error:
|
||||
raise InvalidRequestError("%s contains an unpaired Unicode surrogate, which Hopper's Python API cannot represent" % field) from error
|
||||
return value
|
||||
|
||||
|
||||
def _validate_name_targets(document, names):
|
||||
"""Keep Hopper's unique-label moves within the explicitly selected addresses."""
|
||||
targets = {address for _, address, _ in names}
|
||||
if len(targets) != len(names):
|
||||
raise InvalidRequestError("Batch rename selects the same address more than once; use one canonical address per target")
|
||||
labels = set()
|
||||
for _, address, name in names:
|
||||
_segment(document, address)
|
||||
if not name:
|
||||
continue
|
||||
if name in labels:
|
||||
raise InvalidRequestError("Batch rename assigns the same name more than once: %s" % name)
|
||||
labels.add(name)
|
||||
owner = document.getAddressForName(name)
|
||||
if owner not in BAD_ADDRESSES and owner != address and owner not in targets:
|
||||
raise InvalidRequestError("Name %s is already assigned to %s; include that address in the batch with a replacement name" % (name, _hex(owner)))
|
||||
|
||||
|
||||
def _symbol_name(value, field):
|
||||
"""Reject native label truncation without altering the caller's name."""
|
||||
value = _api_text(value, field)
|
||||
if len(value.encode("utf-16-le")) // 2 > HOPPER_SYMBOL_NAME_UTF16_UNITS:
|
||||
raise InvalidRequestError("%s exceeds Hopper's %s UTF-16 code-unit symbol-name limit" % (field, HOPPER_SYMBOL_NAME_UTF16_UNITS))
|
||||
return value
|
||||
|
||||
|
||||
def _name_matches(document, address, name):
|
||||
observed = _segment(document, address).getNameAtAddress(address)
|
||||
return observed == name or (not name and observed is None)
|
||||
if _selected_document is not None:
|
||||
for candidate in documents:
|
||||
if candidate.getDocumentName() == _selected_document:
|
||||
return candidate
|
||||
session_document = _session_document()
|
||||
if session_document is not None:
|
||||
return session_document
|
||||
current = api.current_document()
|
||||
if current is None:
|
||||
raise CapabilityUnavailableError("No Hopper document is loaded")
|
||||
return current
|
||||
|
||||
|
||||
def _address(document, value=None):
|
||||
"""Keep explicit hexadecimal coordinates distinct from symbol names."""
|
||||
"""Resolve hexadecimal addresses first, then fall back to Hopper symbol names."""
|
||||
if value is None:
|
||||
return document.getCurrentAddress()
|
||||
_api_text(value, "Address")
|
||||
if value.lower().startswith("0x"):
|
||||
try:
|
||||
address = int(value, 16)
|
||||
except ValueError as error:
|
||||
raise InvalidRequestError("Invalid hexadecimal Hopper address: %s" % value) from error
|
||||
else:
|
||||
# Names such as `add` and `face` are valid hexadecimal strings too.
|
||||
# Prefer an observed symbol, retaining bare-hex compatibility only
|
||||
# when Hopper has no symbol with the supplied name.
|
||||
address = document.getAddressForName(value)
|
||||
if address in BAD_ADDRESSES:
|
||||
try:
|
||||
address = int(value, 16)
|
||||
except ValueError as error:
|
||||
raise InvalidRequestError("Unknown Hopper address or name: %s" % value) from error
|
||||
if not 0 <= address < 0xFFFFFFFFFFFFFFFF:
|
||||
raise InvalidRequestError("Address is outside Hopper's usable unsigned 64-bit range: %s" % value)
|
||||
return address
|
||||
if not isinstance(value, str):
|
||||
raise InvalidRequestError("Address must be a string")
|
||||
try:
|
||||
return int(value, 16)
|
||||
except ValueError:
|
||||
result = document.getAddressForName(value)
|
||||
if result in BAD_ADDRESSES:
|
||||
raise InvalidRequestError("Unknown Hopper address or name")
|
||||
return result
|
||||
|
||||
|
||||
def _segment(document, address):
|
||||
@@ -294,18 +249,14 @@ def _procedure_identity(procedure):
|
||||
|
||||
|
||||
def _procedure_locals(procedure):
|
||||
"""Preserve public local-variable names and observed stack displacements."""
|
||||
result = []
|
||||
for local in procedure.getLocalVariableList():
|
||||
name = local.name()
|
||||
displacement = local.displacement()
|
||||
result.append({
|
||||
"description": "%s (stack displacement %s)" % (name if name is not None else "Unnamed local", displacement),
|
||||
"name": name,
|
||||
"stack_displacement": str(displacement) if displacement is not None else None,
|
||||
"""Project opaque Hopper local-variable objects into an exact public shape."""
|
||||
return [
|
||||
{
|
||||
"description": str(local),
|
||||
"provenance": "hopper-public-python-api",
|
||||
})
|
||||
return result
|
||||
}
|
||||
for local in procedure.getLocalVariableList()
|
||||
]
|
||||
|
||||
|
||||
def _containing_procedure(document, address):
|
||||
@@ -316,75 +267,26 @@ def _containing_procedure(document, address):
|
||||
return (procedure, None) if procedure is not None else (None, "not_in_procedure")
|
||||
|
||||
|
||||
def _basic_block_end(procedure, block):
|
||||
"""Normalize native block endpoints to an exclusive byte address."""
|
||||
owner_at = getattr(procedure, "getBasicBlockAtAddress", None)
|
||||
if not callable(owner_at):
|
||||
raise CapabilityUnavailableError(
|
||||
"Instruction enumeration requires Hopper basic-block ownership"
|
||||
)
|
||||
end = block.getEndingAddress()
|
||||
# Hopper 6.1 returns the final instruction address, despite the Python
|
||||
# documentation describing an exclusive end. Membership distinguishes
|
||||
# that representation from an actual exclusive endpoint in other builds.
|
||||
if owner_at(end) == block:
|
||||
instruction = procedure.getSegment().getInstructionAtAddress(end)
|
||||
if instruction is not None and instruction.getInstructionLength() > 0:
|
||||
return end + instruction.getInstructionLength()
|
||||
return end
|
||||
|
||||
|
||||
def _block_instruction_addresses(procedure, block):
|
||||
"""Walk either endpoint convention using Hopper's block ownership API.
|
||||
|
||||
Some Hopper builds return the final instruction start, despite the public
|
||||
API documenting an exclusive end. Equality compares fresh native wrappers;
|
||||
probing ownership also keeps an exclusive endpoint in the next block out.
|
||||
"""
|
||||
segment = procedure.getSegment()
|
||||
address = block.getStartingAddress()
|
||||
end = _basic_block_end(procedure, block)
|
||||
while address < end:
|
||||
if procedure.getBasicBlockAtAddress(address) != block:
|
||||
break
|
||||
instruction = segment.getInstructionAtAddress(address)
|
||||
if instruction is None:
|
||||
break
|
||||
length = instruction.getInstructionLength()
|
||||
if length <= 0 or address + length > end:
|
||||
break
|
||||
yield address
|
||||
address += length
|
||||
|
||||
|
||||
def _instruction_addresses(procedure):
|
||||
result = []
|
||||
seen = set()
|
||||
segment = procedure.getSegment()
|
||||
for block in procedure.basicBlockIterator():
|
||||
for address in _block_instruction_addresses(procedure, block):
|
||||
if address in seen:
|
||||
continue
|
||||
address = block.getStartingAddress()
|
||||
end = block.getEndingAddress()
|
||||
while address < end and address not in seen:
|
||||
seen.add(address)
|
||||
instruction = segment.getInstructionAtAddress(address)
|
||||
if instruction is None:
|
||||
break
|
||||
result.append(address)
|
||||
length = instruction.getInstructionLength()
|
||||
if length <= 0:
|
||||
break
|
||||
address += length
|
||||
return result
|
||||
|
||||
|
||||
def _native_calls(procedure, direction):
|
||||
references = procedure.getAllCallees() if direction == "outgoing" else procedure.getAllCallers()
|
||||
kinds = {0: "none", 1: "unknown", 2: "direct", 3: "objective_c"}
|
||||
calls = {}
|
||||
unresolved = []
|
||||
for reference in references:
|
||||
source, target, kind = reference.fromAddress(), reference.toAddress(), reference.type()
|
||||
if source in BAD_ADDRESSES:
|
||||
continue
|
||||
if target in BAD_ADDRESSES:
|
||||
unresolved.append({"address": _hex(source), "reason": "Native CallReference has no resolved target (type %s)" % kind})
|
||||
continue
|
||||
calls[(source, target)] = {"classification": kinds.get(kind, "unknown"), "provider_type": kind, "provenance": "hopper-public-python-api:CallReference"}
|
||||
return calls, sorted(unresolved, key=lambda item: int(item["address"], 16))
|
||||
|
||||
|
||||
def _procedure_references(document, params):
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
direction = params.get("direction", "outgoing")
|
||||
@@ -397,8 +299,6 @@ def _procedure_references(document, params):
|
||||
references = segment.getReferencesFromAddress(address) if direction == "outgoing" else segment.getReferencesOfAddress(address)
|
||||
for reference in references:
|
||||
edges.add((address, reference) if direction == "outgoing" else (reference, address))
|
||||
calls, unresolved = _native_calls(procedure, direction)
|
||||
edges.update(calls)
|
||||
ordered = sorted(edges)
|
||||
items = []
|
||||
for source, target in ordered:
|
||||
@@ -409,15 +309,16 @@ def _procedure_references(document, params):
|
||||
"target_address": _hex(target),
|
||||
"source_procedure": _procedure_identity(source_procedure) if source_procedure is not None else None,
|
||||
"target_procedure": _procedure_identity(target_procedure) if target_procedure is not None else None,
|
||||
"kind": _unavailable("Hopper exposes partial CallReference classification, but not detailed reference flags"),
|
||||
**({"call": calls[(source, target)]} if (source, target) in calls else {}),
|
||||
"kind": _unavailable("Hopper's public Python API does not classify reference kinds"),
|
||||
})
|
||||
return {
|
||||
"procedure": _procedure_identity(procedure), "direction": direction,
|
||||
"reference_kinds_available": False,
|
||||
# These are native CallReference objects, not proof that every unresolved
|
||||
# dispatch site has been enumerated by Hopper.
|
||||
"unresolved_calls": unresolved,
|
||||
# The public Hopper API returns observed references but does not expose
|
||||
# enough flow metadata to enumerate calls with no resolved target.
|
||||
# Keep the required list field explicit; the provider records this as
|
||||
# unknown coverage rather than claiming the list is exhaustive.
|
||||
"unresolved_calls": [],
|
||||
"references": items,
|
||||
}
|
||||
|
||||
@@ -432,61 +333,11 @@ def _procedure_map(document):
|
||||
|
||||
|
||||
def _strings(document):
|
||||
return {address: record["value"] for address, record in _string_inventory(document).items()}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
end = segment.getStartingAddress() + segment.getLength()
|
||||
if isinstance(length, bool) or not isinstance(length, int) or length <= 0 or length > end - address:
|
||||
raise CapabilityUnavailableError("Hopper reported an invalid string object extent at %s" % _hex(address))
|
||||
raw = segment.readBytes(address, length)
|
||||
if not isinstance(raw, (bytes, bytearray)) or len(raw) != length:
|
||||
raise CapabilityUnavailableError("Hopper could not read the complete string object at %s" % _hex(address))
|
||||
kind = segment.getTypeAtAddress(address)
|
||||
encodings = ("utf-8", "latin-1") if kind == segment.TYPE_ASCII else ("utf-16-le", "utf-16-be") if kind == segment.TYPE_UNICODE else ()
|
||||
# Native display escaping is not invertible (a literal backslash-n and a
|
||||
# newline can render alike). Decode byte strings directly; for UTF-16,
|
||||
# compare rendered candidates to establish an unambiguous byte order.
|
||||
candidates = []
|
||||
for encoding in encodings:
|
||||
terminator = b"\x00\x00" if encoding.startswith("utf-16") else b"\x00"
|
||||
terminated = raw.endswith(terminator)
|
||||
payload = raw[:-len(terminator)] if terminated else raw
|
||||
try:
|
||||
value = payload.decode(encoding)
|
||||
except UnicodeDecodeError:
|
||||
continue
|
||||
record = {"value": value, "provider_value": display, "string": {"encoding": encoding, "encoding_status": "inferred", "termination": "present_or_not_required" if terminated else "missing", "byte_length": length}}
|
||||
if kind == segment.TYPE_ASCII:
|
||||
return record
|
||||
rendered = value.replace('"', '\\"').replace("\n", "\\n").replace("\r", "\\r").replace("\t", "\\t")
|
||||
if rendered == display or (display.endswith("\u2026") and display[:-1] and rendered.startswith(display[:-1])):
|
||||
candidates.append(record)
|
||||
if candidates and len({candidate["value"] for candidate in candidates}) == 1:
|
||||
return candidates[0]
|
||||
raise CapabilityUnavailableError("Hopper's string display cannot be reconciled with its typed bytes at %s; inspect the bytes and encoding explicitly" % _hex(address))
|
||||
|
||||
|
||||
def _string_inventory(document, addresses=None):
|
||||
key = (id(document), "string_objects")
|
||||
objects = _search_inventory_cache.get(key)
|
||||
if objects is None:
|
||||
objects = {address: (segment, display) for segment in document.getSegmentsList() for display, address in segment.getStringsList()}
|
||||
_search_inventory_cache[key] = objects
|
||||
key = (id(document), "string_records")
|
||||
records = _search_inventory_cache.setdefault(key, {})
|
||||
selected = sorted(objects if addresses is None else objects.keys() & addresses)
|
||||
for address in selected:
|
||||
name = _hex(address)
|
||||
if name not in records:
|
||||
segment, display = objects[address]
|
||||
try:
|
||||
records[name] = _string_record(segment, address, display)
|
||||
except CapabilityUnavailableError as error:
|
||||
records[name] = {"value": display, "provider_value": display, "decoding": _unavailable(str(error))}
|
||||
return {_hex(address): records[_hex(address)] for address in selected}
|
||||
result = {}
|
||||
for segment in document.getSegmentsList():
|
||||
for value, address in segment.getStringsList():
|
||||
result[_hex(address)] = value
|
||||
return result
|
||||
|
||||
|
||||
def _invalidate_search_inventory(document):
|
||||
@@ -546,18 +397,17 @@ def _search_results(document, kind, params):
|
||||
needle = pattern if case_sensitive else pattern.casefold()
|
||||
matches = lambda value: needle in (value if case_sensitive else value.casefold())
|
||||
else:
|
||||
raise CapabilityUnavailableError("Regex matching requires REA's supervised worker; the native bridge only executes literal searches")
|
||||
try:
|
||||
expression = re.compile(pattern, 0 if case_sensitive else re.IGNORECASE)
|
||||
except (re.error, OverflowError) as error:
|
||||
raise InvalidRequestError("Invalid regex pattern") from error
|
||||
matches = lambda value: expression.search(value) is not None
|
||||
|
||||
selected = []
|
||||
for item in _search_inventory(document, kind):
|
||||
if not matches(item[1]):
|
||||
continue
|
||||
selected.append(item)
|
||||
if not selected:
|
||||
return []
|
||||
if kind == "string":
|
||||
records = _string_inventory(document)
|
||||
return [{"address": address, **records[address]} for address, _ in selected]
|
||||
return [
|
||||
{
|
||||
"address": address,
|
||||
@@ -600,11 +450,25 @@ def _render_instruction(segment, address):
|
||||
|
||||
def _assembly(procedure):
|
||||
"""Render assembly while guarding against malformed instruction cycles."""
|
||||
lines = []
|
||||
segment = procedure.getSegment()
|
||||
return "\n".join(
|
||||
line for address in _instruction_addresses(procedure)
|
||||
if (line := _render_instruction(segment, address)) is not None
|
||||
)
|
||||
seen = set()
|
||||
for block in procedure.basicBlockIterator():
|
||||
address = block.getStartingAddress()
|
||||
end = block.getEndingAddress()
|
||||
while address < end and address not in seen:
|
||||
seen.add(address)
|
||||
instruction = segment.getInstructionAtAddress(address)
|
||||
if instruction is None:
|
||||
break
|
||||
line = _render_instruction(segment, address)
|
||||
if line is not None:
|
||||
lines.append(line)
|
||||
length = instruction.getInstructionLength()
|
||||
if length <= 0:
|
||||
break
|
||||
address += length
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _read_function_instructions(document, params):
|
||||
@@ -641,7 +505,7 @@ def _analyze_function(document, params):
|
||||
successors.append(_hex(successor))
|
||||
blocks.append({
|
||||
"start": _hex(block.getStartingAddress()),
|
||||
"end": _hex(_basic_block_end(procedure, block)),
|
||||
"end": _hex(block.getEndingAddress()),
|
||||
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
|
||||
})
|
||||
pseudo = _pseudocode(document, procedure) or ""
|
||||
@@ -665,10 +529,6 @@ def _analyze_function(document, params):
|
||||
incoming = []
|
||||
outgoing = []
|
||||
procedure_addresses = set(addresses)
|
||||
outgoing_calls, unresolved_calls = _native_calls(procedure, "outgoing")
|
||||
incoming_calls, _ = _native_calls(procedure, "incoming")
|
||||
calls = {**incoming_calls, **outgoing_calls}
|
||||
edges.update(calls)
|
||||
for source, target in sorted(edges):
|
||||
source_procedure, _ = _containing_procedure(document, source)
|
||||
target_procedure, _ = _containing_procedure(document, target)
|
||||
@@ -677,14 +537,16 @@ def _analyze_function(document, params):
|
||||
"target_address": _hex(target),
|
||||
"source_procedure": _procedure_identity(source_procedure) if source_procedure is not None else None,
|
||||
"target_procedure": _procedure_identity(target_procedure) if target_procedure is not None else None,
|
||||
"kind": _unavailable("Hopper exposes partial CallReference classification, but not detailed reference flags"),
|
||||
**({"call": calls[(source, target)]} if (source, target) in calls else {}),
|
||||
"kind": _unavailable("Hopper's public Python API does not classify reference kinds"),
|
||||
}
|
||||
if target in procedure_addresses and source not in procedure_addresses:
|
||||
incoming.append(item)
|
||||
if source in procedure_addresses:
|
||||
outgoing.append(item)
|
||||
string_map = {int(address, 16): record for address, record in _string_inventory(document, {int(edge["target_address"], 16) for edge in outgoing}).items()}
|
||||
string_map = {
|
||||
int(address, 16): value
|
||||
for address, value in _search_inventory(document, "string")
|
||||
}
|
||||
name_map = {
|
||||
int(address, 16): value
|
||||
for address, value in _search_inventory(document, "name")
|
||||
@@ -694,7 +556,7 @@ def _analyze_function(document, params):
|
||||
for edge in outgoing:
|
||||
target = int(edge["target_address"], 16)
|
||||
if target in string_map:
|
||||
referenced_strings.append({"address": edge["target_address"], **string_map[target], "source_address": edge["source_address"]})
|
||||
referenced_strings.append({"address": edge["target_address"], "value": string_map[target], "source_address": edge["source_address"]})
|
||||
if target in name_map:
|
||||
referenced_names.append({"address": edge["target_address"], "value": name_map[target], "source_address": edge["source_address"]})
|
||||
comments.sort(key=lambda item: (int(item["address"], 16), item["kind"]))
|
||||
@@ -712,16 +574,13 @@ def _analyze_function(document, params):
|
||||
"callers": callers, "callees": callees,
|
||||
"incoming_references": incoming,
|
||||
"outgoing_references": outgoing,
|
||||
"unresolved_calls": unresolved_calls,
|
||||
"referenced_strings": referenced_strings,
|
||||
"referenced_names": referenced_names,
|
||||
"basic_blocks": blocks,
|
||||
"native_api": None,
|
||||
"native_value_flow": None,
|
||||
"limitations": [
|
||||
"Native CallReference classifications are observed where available; detailed reference flags remain unknown.",
|
||||
"Hopper's public Python API does not classify reference kinds.",
|
||||
"Hopper's public Python API does not expose equivalent external or thunk classification in this dossier.",
|
||||
"Native calls without resolved target addresses are not represented as edges; enumeration beyond reported CallReference objects is unknown.",
|
||||
"Unresolved indirect calls without reported target addresses are not represented as call edges.",
|
||||
"Pseudocode and assembly are provider-specific representations, not original source.",
|
||||
],
|
||||
}
|
||||
@@ -729,6 +588,7 @@ def _analyze_function(document, params):
|
||||
|
||||
def _dispatch(method, params):
|
||||
"""Dispatch only the closed operation set implemented by REA's public tools."""
|
||||
global _selected_document
|
||||
if method == "health":
|
||||
return {"name": "REA Hopper bridge", "version": "1.0.0", "run_id": REA_RUN_ID}
|
||||
if method in ("shutdown", "shutdown_document"):
|
||||
@@ -762,9 +622,14 @@ def _dispatch(method, params):
|
||||
"document_closed": document_closed,
|
||||
}
|
||||
if method == "list_documents":
|
||||
return [document.getDocumentName() for document in _api().documents()]
|
||||
return [document.getDocumentName() for document in Document.getAllDocuments()]
|
||||
if method == "current_document":
|
||||
return _document().getDocumentName()
|
||||
if method == "set_current_document":
|
||||
document = _document(params.get("document"))
|
||||
_selected_document = document.getDocumentName()
|
||||
return _selected_document
|
||||
|
||||
document = _document(params.get("document"))
|
||||
if method in EXHAUSTIVE_ANALYSIS_METHODS:
|
||||
_api().require_analysis_complete(document, method)
|
||||
@@ -785,17 +650,10 @@ def _dispatch(method, params):
|
||||
"Hopper's public Python API does not expose readBytes"
|
||||
)
|
||||
address = _address(document, params.get("address"))
|
||||
segment = _segment(document, address)
|
||||
# Document.readBytes delegates to one Segment and returns False when
|
||||
# the requested range crosses its end. Preserve the readable prefix.
|
||||
available = segment.getStartingAddress() + segment.getLength() - address
|
||||
value = reader(address, min(length, available))
|
||||
if value is False or value is None:
|
||||
raise InvalidRequestError("Hopper could not read the requested byte range at %s" % _hex(address))
|
||||
value = reader(address, length)
|
||||
if not isinstance(value, (bytes, bytearray)):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper readBytes returned %s for the requested %s-byte range at %s"
|
||||
% (type(value).__name__, length, _hex(address))
|
||||
"Hopper's public Python API returned an unsupported byte representation"
|
||||
)
|
||||
data = bytes(value)
|
||||
return {
|
||||
@@ -820,26 +678,7 @@ def _dispatch(method, params):
|
||||
or offset < 0
|
||||
):
|
||||
raise InvalidRequestError("Address has no authoritative file-offset mapping")
|
||||
inverse = getattr(document, "getAddressFromFileOffset", None)
|
||||
if not callable(inverse):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API cannot verify the file-offset mapping by reverse lookup"
|
||||
)
|
||||
# Synthetic memory can report offset zero, which belongs to the actual
|
||||
# file header. An integer alone does not prove source-file provenance.
|
||||
if inverse(offset) != address:
|
||||
raise InvalidRequestError("Address has no authoritative file-offset mapping: reverse lookup disagrees at %s" % _hex(address))
|
||||
if offset > 9007199254740991:
|
||||
raise CapabilityUnavailableError("Provider file offset exceeds the exact JSON integer range")
|
||||
header_address = inverse(0)
|
||||
reader = getattr(document, "readBytes", None)
|
||||
header = reader(header_address, 12) if header_address not in BAD_ADDRESSES and document.getSegmentAtAddress(header_address) is not None and callable(reader) else None
|
||||
return {
|
||||
"address": _hex(address),
|
||||
"provider_file_offset": offset,
|
||||
"provider_source_path": document.getExecutableFilePath(),
|
||||
"provider_image_header_hex": bytes(header).hex() if isinstance(header, (bytes, bytearray)) and len(header) == 12 else None,
|
||||
}
|
||||
return {"address": _hex(address), "file_offset": offset}
|
||||
if method == "resolve_containing_procedure":
|
||||
address = _address(document, params.get("address"))
|
||||
procedure, reason = _containing_procedure(document, address)
|
||||
@@ -855,13 +694,8 @@ def _dispatch(method, params):
|
||||
return _procedure_name(_procedure(document))
|
||||
if method == "goto_address":
|
||||
address = _address(document, params.get("address"))
|
||||
segment = _segment(document, address)
|
||||
expected = segment.getInstructionStart(address)
|
||||
document.moveCursorAtAddress(address)
|
||||
observed = document.getCurrentAddress()
|
||||
if observed != (address if expected in BAD_ADDRESSES else expected):
|
||||
raise InvalidRequestError("Hopper cursor did not reach the requested object at %s; observed %s" % (_hex(address), _hex(observed)))
|
||||
return _hex(observed)
|
||||
return _hex(address)
|
||||
if method in ("address_name", "comment", "inline_comment", "xrefs"):
|
||||
target = _address(document, params.get("address"))
|
||||
segment = _segment(document, target)
|
||||
@@ -874,22 +708,12 @@ def _dispatch(method, params):
|
||||
return [_hex(value) for value in segment.getReferencesOfAddress(target)]
|
||||
if method in ("next_address", "prev_address"):
|
||||
target = _address(document, params.get("address"))
|
||||
segment = _segment(document, target)
|
||||
if method == "next_address":
|
||||
start = segment.getInstructionStart(target)
|
||||
if start in BAD_ADDRESSES:
|
||||
raise InvalidRequestError("No analyzed object exists at the requested address")
|
||||
length = segment.getObjectLength(start)
|
||||
if length in BAD_ADDRESSES or length <= 0:
|
||||
raise InvalidRequestError("No next analyzed object exists at the requested address")
|
||||
result = start + length
|
||||
result = target + max(1, document.getObjectLength(target))
|
||||
else:
|
||||
previous_segment = document.getSegmentAtAddress(target - 1) if target > 0 else None
|
||||
if previous_segment is None:
|
||||
raise InvalidRequestError("No previous address exists in mapped memory")
|
||||
result = previous_segment.getInstructionStart(target - 1)
|
||||
if result in BAD_ADDRESSES or document.getSegmentAtAddress(result) is None:
|
||||
raise InvalidRequestError("No adjacent analyzed object exists in mapped memory")
|
||||
result = document.getInstructionStart(max(0, target - 1))
|
||||
if result in BAD_ADDRESSES:
|
||||
raise InvalidRequestError("No adjacent address")
|
||||
return _hex(result)
|
||||
if method == "list_segments":
|
||||
result = []
|
||||
@@ -923,9 +747,12 @@ def _dispatch(method, params):
|
||||
if method == "list_procedures":
|
||||
return [{"address": address, "value": value} for address, value in _search_inventory(document, "procedure")]
|
||||
if method == "list_strings":
|
||||
values = dict(_search_inventory(document, "string"))
|
||||
requested = params.get("address")
|
||||
values = _string_inventory(document, None if requested is None else {_address(document, requested)})
|
||||
return [{"address": address, **record} for address, record in values.items()]
|
||||
if requested is not None:
|
||||
key = _hex(_address(document, requested))
|
||||
values = {key: values[key]} if key in values else {}
|
||||
return [{"address": address, "value": value} for address, value in values.items()]
|
||||
if method == "list_names":
|
||||
result = dict(_search_inventory(document, "name"))
|
||||
requested = params.get("address")
|
||||
@@ -950,7 +777,7 @@ def _dispatch(method, params):
|
||||
return sorted((_hex(item.getEntryPoint()) for item in procedure.getAllCalleeProcedures()), key=lambda value: int(value, 16))
|
||||
if method == "procedure_info":
|
||||
blocks = list(procedure.basicBlockIterator())
|
||||
length = sum(max(0, _basic_block_end(procedure, block) - block.getStartingAddress()) for block in blocks)
|
||||
length = sum(max(0, block.getEndingAddress() - block.getStartingAddress()) for block in blocks)
|
||||
return {
|
||||
"name": _procedure_name(procedure),
|
||||
"entrypoint": _hex(procedure.getEntryPoint()),
|
||||
@@ -963,51 +790,35 @@ def _dispatch(method, params):
|
||||
}
|
||||
if method == "set_address_name":
|
||||
address = _address(document, params.get("address"))
|
||||
name = _symbol_name(params["name"], "Name")
|
||||
_validate_name_targets(document, [(params.get("address"), address, name)])
|
||||
try:
|
||||
result = document.setNameAtAddress(address, name)
|
||||
result = document.setNameAtAddress(address, params["name"])
|
||||
finally:
|
||||
_invalidate_search_inventory(document)
|
||||
_invalidate_pseudocode(document)
|
||||
return bool(result) and _name_matches(document, address, name)
|
||||
return result
|
||||
if method == "set_addresses_names":
|
||||
# Validate all destinations before applying any annotation. A malformed
|
||||
# later address must not discard the result of an earlier mutation.
|
||||
names = [(key, _address(document, key), _symbol_name(value, "Name for %s" % key)) for key, value in params["names"].items()]
|
||||
_validate_name_targets(document, names)
|
||||
try:
|
||||
result = {key: document.setNameAtAddress(address, value) for key, address, value in names}
|
||||
result = {key: document.setNameAtAddress(_address(document, key), value) for key, value in params["names"].items()}
|
||||
finally:
|
||||
_invalidate_search_inventory(document)
|
||||
_invalidate_pseudocode(document)
|
||||
return {key: bool(result[key]) and _name_matches(document, address, value) for key, address, value in names}
|
||||
return result
|
||||
if method in ("set_comment", "set_inline_comment"):
|
||||
address = _address(document, params.get("address"))
|
||||
comment = _api_text(params["comment"], "Comment")
|
||||
segment = _segment(document, address)
|
||||
setter = segment.setCommentAtAddress if method == "set_comment" else segment.setInlineCommentAtAddress
|
||||
getter = segment.getCommentAtAddress if method == "set_comment" else segment.getInlineCommentAtAddress
|
||||
setter(address, comment)
|
||||
setter(address, params["comment"])
|
||||
_invalidate_pseudocode(document)
|
||||
observed = getter(address)
|
||||
return observed == comment or (comment == "" and observed is None)
|
||||
return getter(address) == params["comment"]
|
||||
if method == "list_bookmarks":
|
||||
return [{"address": _hex(item), "name": document.getBookmarkName(item)} for item in document.getBookmarks()]
|
||||
if method == "set_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
_segment(document, address)
|
||||
name = params.get("name")
|
||||
if name is not None:
|
||||
_api_text(name, "Bookmark name")
|
||||
document.setBookmarkAtAddress(address, name)
|
||||
document.setBookmarkAtAddress(address, params.get("name"))
|
||||
return document.hasBookmarkAtAddress(address)
|
||||
if method == "unset_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
# Permit removing a legacy orphan bookmark, but do not report success
|
||||
# for an unmapped address with no bookmark to remove.
|
||||
if not document.hasBookmarkAtAddress(address):
|
||||
_segment(document, address)
|
||||
document.removeBookmarkAtAddress(address)
|
||||
return not document.hasBookmarkAtAddress(address)
|
||||
raise InvalidRequestError("Unknown bridge method")
|
||||
@@ -1016,87 +827,73 @@ def _dispatch(method, params):
|
||||
def _serve_connection(connection):
|
||||
"""Serve one capability-authenticated NDJSON connection."""
|
||||
file = connection.makefile("rwb")
|
||||
should_stop = False
|
||||
try:
|
||||
while True:
|
||||
line = file.readline()
|
||||
if not line:
|
||||
break
|
||||
request_id = None
|
||||
authenticated = False
|
||||
should_stop = False
|
||||
try:
|
||||
try:
|
||||
request = json.loads(line.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise InvalidRequestError("Invalid bridge request JSON") from error
|
||||
if not isinstance(request, dict) or set(request) != {"id", "token", "method", "params"}:
|
||||
raise InvalidRequestError("Invalid bridge request shape")
|
||||
request_id = request["id"]
|
||||
if isinstance(request_id, bool) or not isinstance(request_id, int) or request_id < 0:
|
||||
raise InvalidRequestError("Invalid bridge request id")
|
||||
if (
|
||||
not isinstance(request["method"], str) or not request["method"]
|
||||
or not isinstance(request["params"], dict)
|
||||
):
|
||||
raise InvalidRequestError("Invalid bridge method or parameters")
|
||||
if (
|
||||
not isinstance(request["token"], str) or not request["token"].isascii()
|
||||
or not hmac.compare_digest(request["token"], REA_TOKEN)
|
||||
):
|
||||
raise PermissionError("Invalid bridge capability")
|
||||
authenticated = True
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "progress",
|
||||
"phase": "hopper_bridge",
|
||||
"completed": 0,
|
||||
"total": 1,
|
||||
"message": "Hopper bridge started request",
|
||||
}})
|
||||
result = _dispatch(request["method"], request["params"])
|
||||
should_stop = request["method"] in ("shutdown", "shutdown_document") and (
|
||||
result.get("shutdown") is True and (
|
||||
(result.get("document_closed") is True and result.get("analysis_stopped") is True)
|
||||
or (result.get("document_closed") is False and result.get("document_retained") is True)
|
||||
)
|
||||
)
|
||||
safe_result = _json_safe(result)
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "progress",
|
||||
"phase": "hopper_bridge",
|
||||
"completed": 1,
|
||||
"total": 1,
|
||||
"message": "Hopper bridge completed request",
|
||||
"terminal": True,
|
||||
}})
|
||||
response = {"id": request_id, "result": safe_result}
|
||||
except Exception as error:
|
||||
diagnostic = _safe_diagnostic(error)
|
||||
if authenticated:
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "diagnostic",
|
||||
"error": diagnostic,
|
||||
}})
|
||||
response_id = request_id if (
|
||||
isinstance(request_id, int)
|
||||
and not isinstance(request_id, bool)
|
||||
and request_id >= 0
|
||||
) else 0
|
||||
response = {"id": response_id, "error": diagnostic}
|
||||
_write_message(file, response)
|
||||
if should_stop:
|
||||
break
|
||||
except (ConnectionError, OSError):
|
||||
# A lost client cannot confirm document closure or revoke this run.
|
||||
pass
|
||||
finally:
|
||||
while True:
|
||||
line = file.readline()
|
||||
if not line:
|
||||
break
|
||||
request_id = None
|
||||
authenticated = False
|
||||
should_stop = False
|
||||
try:
|
||||
file.close()
|
||||
except OSError:
|
||||
# Closing a failed writer can repeat the same disconnected-pipe error.
|
||||
pass
|
||||
connection.close()
|
||||
return should_stop
|
||||
try:
|
||||
request = json.loads(line.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise InvalidRequestError("Invalid bridge request JSON") from error
|
||||
if not isinstance(request, dict) or set(request) != {"id", "token", "method", "params"}:
|
||||
raise InvalidRequestError("Invalid bridge request shape")
|
||||
request_id = request["id"]
|
||||
if isinstance(request_id, bool) or not isinstance(request_id, int) or request_id < 0:
|
||||
raise InvalidRequestError("Invalid bridge request id")
|
||||
if (
|
||||
not isinstance(request["method"], str) or not request["method"]
|
||||
or not isinstance(request["params"], dict)
|
||||
):
|
||||
raise InvalidRequestError("Invalid bridge method or parameters")
|
||||
if (
|
||||
not isinstance(request["token"], str) or not request["token"].isascii()
|
||||
or not hmac.compare_digest(request["token"], REA_TOKEN)
|
||||
):
|
||||
raise PermissionError("Invalid bridge capability")
|
||||
authenticated = True
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "progress",
|
||||
"phase": "hopper_bridge",
|
||||
"completed": 0,
|
||||
"total": 1,
|
||||
"message": "Hopper bridge started request",
|
||||
}})
|
||||
result = _dispatch(request["method"], request["params"])
|
||||
should_stop = request["method"] == "shutdown_document" or (
|
||||
request["method"] == "shutdown" and not result.get("cleanup_required", False)
|
||||
)
|
||||
safe_result = _json_safe(result)
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "progress",
|
||||
"phase": "hopper_bridge",
|
||||
"completed": 1,
|
||||
"total": 1,
|
||||
"message": "Hopper bridge completed request",
|
||||
"terminal": True,
|
||||
}})
|
||||
response = {"id": request_id, "result": safe_result}
|
||||
except Exception as error:
|
||||
diagnostic = _safe_diagnostic(error)
|
||||
if authenticated:
|
||||
_write_message(file, {"id": request_id, "event": {
|
||||
"type": "diagnostic",
|
||||
"error": diagnostic,
|
||||
}})
|
||||
response_id = request_id if (
|
||||
isinstance(request_id, int)
|
||||
and not isinstance(request_id, bool)
|
||||
and request_id >= 0
|
||||
) else 0
|
||||
response = {"id": response_id, "error": diagnostic}
|
||||
_write_message(file, response)
|
||||
if should_stop:
|
||||
break
|
||||
file.close()
|
||||
connection.close()
|
||||
|
||||
|
||||
def _write_message(file, message):
|
||||
@@ -1116,14 +913,14 @@ def _diagnostic_type(error):
|
||||
|
||||
|
||||
def _safe_diagnostic(error):
|
||||
"""Preserve bridge validation reasons while excluding transport credentials."""
|
||||
"""Project one exception without retaining provider or credential text."""
|
||||
diagnostic_type = _diagnostic_type(error)
|
||||
if diagnostic_type == "capability_unavailable":
|
||||
message = str(error)
|
||||
elif diagnostic_type == "authorization":
|
||||
message = "Invalid bridge capability"
|
||||
elif diagnostic_type == "invalid_request":
|
||||
message = str(error)
|
||||
message = "Invalid Hopper bridge request"
|
||||
else:
|
||||
message = "%s: Hopper bridge operation failed" % type(error).__name__
|
||||
return {"code": -32000, "message": message, "type": diagnostic_type}
|
||||
@@ -1138,17 +935,8 @@ def _run():
|
||||
os.chmod(REA_SOCKET, 0o600)
|
||||
server.listen(1)
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
connection, _ = server.accept()
|
||||
except socket.timeout:
|
||||
break
|
||||
connection.settimeout(None)
|
||||
if _serve_connection(connection):
|
||||
break
|
||||
# Match REA's cleanup handshake budget without pinning Hopper's
|
||||
# Python thread indefinitely after a client disappears.
|
||||
server.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
|
||||
connection, _ = server.accept()
|
||||
_serve_connection(connection)
|
||||
finally:
|
||||
server.close()
|
||||
if os.path.exists(REA_SOCKET):
|
||||
|
||||
@@ -1,208 +0,0 @@
|
||||
"""REA offline adapter using mitmproxy's unchanged native tnetstring decoder.
|
||||
|
||||
No FlowReader migrations, HAR conversion, proxy listener, or target requests.
|
||||
The containing mitmdump process and its configuration directory are REA-owned.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import resource
|
||||
from pathlib import Path
|
||||
|
||||
from mitmproxy import ctx, version
|
||||
from mitmproxy.io import tnetstring
|
||||
|
||||
PROFILE = "12.2.3"
|
||||
OUTPUT_BYTES = 96 * 1024 * 1024
|
||||
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
|
||||
|
||||
|
||||
class CaptureFailure(Exception):
|
||||
def __init__(self, reason, message, pointer=""):
|
||||
self.reason, self.pointer = reason, pointer
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
def pointer(parent, key):
|
||||
return parent + "/" + str(key).replace("~", "~0").replace("/", "~1")
|
||||
|
||||
|
||||
class BoundedReader:
|
||||
"""Refuse malicious declared lengths before the upstream parser allocates."""
|
||||
def __init__(self, handle, size):
|
||||
self.handle, self.size = handle, size
|
||||
|
||||
def read(self, count):
|
||||
if count < 0 or count > self.size - self.handle.tell():
|
||||
raise CaptureFailure("format", "Native record declares bytes beyond the retained capture.")
|
||||
return self.handle.read(count)
|
||||
|
||||
|
||||
def project_record(state, secrets):
|
||||
binaries, numbers, redactions = [], [], []
|
||||
byte_secrets = []
|
||||
for secret in secrets:
|
||||
try:
|
||||
byte_secrets.append(secret.encode("utf-8", errors="strict"))
|
||||
except UnicodeEncodeError:
|
||||
# Non-scalar JSON strings have no UTF-8 byte representation.
|
||||
# Keep them in secrets for the ordinary string projection.
|
||||
pass
|
||||
|
||||
def redact(text, path, original):
|
||||
if any(secret in text or secret in original for secret in secrets):
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
return None
|
||||
return text
|
||||
|
||||
def transport_value(value, path, transport_url):
|
||||
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
|
||||
return value, False
|
||||
binary = isinstance(value, bytes)
|
||||
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
|
||||
replacement = rb"\1" if binary else r"\1"
|
||||
separator = b"@" if binary else "@"
|
||||
safe_url = re.sub(pattern, replacement, value, flags=re.I)
|
||||
if path.endswith("/authority") and separator in safe_url:
|
||||
safe_url = safe_url.rsplit(separator, 1)[1]
|
||||
changed = safe_url != value
|
||||
if changed:
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
return safe_url, changed
|
||||
|
||||
def validate_omitted(value, path, depth):
|
||||
before = len(binaries), len(numbers), len(redactions)
|
||||
visit(value, path, depth)
|
||||
del binaries[before[0]:]
|
||||
del numbers[before[1]:]
|
||||
del redactions[before[2]:]
|
||||
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False):
|
||||
if depth > 64:
|
||||
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
|
||||
if credential:
|
||||
validate_omitted(value, path, depth)
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
if isinstance(value, bytes):
|
||||
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
|
||||
return None
|
||||
if isinstance(value, bytes):
|
||||
safe_bytes, credential_url = transport_value(value, path, transport_url)
|
||||
content_base64 = base64.b64encode(value).decode("ascii")
|
||||
digest = hashlib.sha256(value).hexdigest()
|
||||
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
|
||||
safe_text = None
|
||||
try:
|
||||
safe_text = safe_bytes.decode("utf-8", errors="strict")
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
if hidden:
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
exclude = hidden or credential_url
|
||||
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted" if exclude else "retained", "content_base64": None if exclude else content_base64, "bytes": None if exclude else len(value), "sha256": None if exclude else digest})
|
||||
if hidden:
|
||||
return None
|
||||
return safe_text
|
||||
if isinstance(value, str):
|
||||
safe_text, _ = transport_value(value, path, transport_url)
|
||||
return redact(safe_text, path, value)
|
||||
if isinstance(value, bool) or value is None:
|
||||
return value
|
||||
if isinstance(value, (int, float)):
|
||||
literal = repr(value)
|
||||
if any(secret in literal for secret in secrets):
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
return None
|
||||
numbers.append({"pointer": path, "producer_type": "integer" if isinstance(value, int) else "float", "literal": literal})
|
||||
return value if (abs(value) <= 9007199254740991 if isinstance(value, int) else math.isfinite(value)) else None
|
||||
if isinstance(value, (list, tuple)):
|
||||
fields = re.fullmatch(r"/(?:backup/)*(?:request|response)/(?:headers|trailers)", path)
|
||||
if fields:
|
||||
result = []
|
||||
for index, field in enumerate(value):
|
||||
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
|
||||
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
|
||||
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
|
||||
return result
|
||||
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
|
||||
if isinstance(value, dict):
|
||||
result = {}
|
||||
for key, item in value.items():
|
||||
if not isinstance(key, str):
|
||||
raise CaptureFailure("format", "Native record has a non-string dictionary key.", path)
|
||||
if any(secret in key for secret in secrets):
|
||||
# Validate the omitted subtree too, then discard all of its
|
||||
# coordinates rather than inventing a replacement identity.
|
||||
validate_omitted(item, pointer(path, key), depth + 1)
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
|
||||
continue
|
||||
result[key] = visit(item, pointer(path, key), depth + 1)
|
||||
return result
|
||||
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
|
||||
|
||||
return {"reported": visit(state, ""), "binary_fields": binaries, "numeric_literals": numbers, "redactions": redactions}
|
||||
|
||||
|
||||
def decode(request):
|
||||
if version.VERSION != PROFILE:
|
||||
raise CaptureFailure("unsupported", "Expected mitmproxy " + PROFILE + "; observed " + version.VERSION + ".")
|
||||
records = []
|
||||
capture = Path(request["snapshot_path"])
|
||||
size = capture.stat().st_size
|
||||
with capture.open("rb") as handle:
|
||||
reader = BoundedReader(handle, size)
|
||||
while handle.tell() < size:
|
||||
start = handle.tell()
|
||||
try:
|
||||
state = tnetstring.load(reader)
|
||||
except CaptureFailure:
|
||||
raise
|
||||
except RecursionError:
|
||||
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
|
||||
except (ValueError, IndexError, UnicodeError, EOFError):
|
||||
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
|
||||
if not isinstance(state, dict):
|
||||
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
|
||||
|
||||
|
||||
def failure_reply(error):
|
||||
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
|
||||
if isinstance(error, CaptureFailure):
|
||||
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
|
||||
if isinstance(error, MemoryError):
|
||||
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
|
||||
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
|
||||
|
||||
|
||||
class OfflineCapture:
|
||||
def load(self, loader):
|
||||
loader.add_option("rea_request_path", str, "", "REA private offline capture request")
|
||||
|
||||
def running(self):
|
||||
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
|
||||
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
|
||||
try:
|
||||
response = {"ok": True, "value": decode(request)}
|
||||
except Exception as error:
|
||||
response = failure_reply(error)
|
||||
try:
|
||||
try:
|
||||
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
|
||||
except Exception as error:
|
||||
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
|
||||
if len(output) > OUTPUT_BYTES:
|
||||
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
|
||||
Path(request["reply_path"]).write_bytes(output)
|
||||
finally:
|
||||
ctx.master.shutdown()
|
||||
|
||||
|
||||
addons = [OfflineCapture()]
|
||||
@@ -1,839 +0,0 @@
|
||||
"""Bounded native call observation executed inside LLDB's embedded Python.
|
||||
|
||||
REA imports this module into `lldb --batch` and runs `rea_trace <config.json>`.
|
||||
It launches one owned process, stops at the requested functions and
|
||||
Objective-C methods, records each entry from registers and memory reads only,
|
||||
continues, and kills the process once it exits, the event or aggregate resource
|
||||
limit is reached, or the duration elapses. Target output is drained through
|
||||
bounded FIFOs. No expression is evaluated and no target code runs on the
|
||||
tracer's behalf. The result is written as JSON to the configured path.
|
||||
"""
|
||||
|
||||
import errno
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import select
|
||||
import threading
|
||||
import time
|
||||
|
||||
import lldb
|
||||
|
||||
ARGUMENT_REGISTERS = {
|
||||
"arm64": ["x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7"],
|
||||
"arm64e": ["x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7"],
|
||||
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
}
|
||||
MAX_SELECTOR_BYTES = 1024
|
||||
MAX_REPORTED_LOCATIONS = 64
|
||||
MAX_OTHER_STOPS = 256
|
||||
MAX_TRACE_BYTES = 8 * 1024 * 1024
|
||||
MAX_TRACE_FRAMES = 65_536
|
||||
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
|
||||
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
|
||||
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
|
||||
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
|
||||
OUTPUT_EOF_WAIT_SECONDS = 1.0
|
||||
OUTPUT_READER_JOIN_SECONDS = 0.5
|
||||
LISTENER_NAME = "rea-native-call-observation"
|
||||
|
||||
|
||||
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
|
||||
"""Whether a complete event fits both aggregate trace budgets."""
|
||||
return (
|
||||
current_bytes + event_bytes <= MAX_TRACE_BYTES
|
||||
and current_frames + event_frames <= MAX_TRACE_FRAMES
|
||||
)
|
||||
|
||||
|
||||
def _estimated_json_size(value):
|
||||
"""Estimate JSON size without encoding potentially oversized values."""
|
||||
if isinstance(value, str):
|
||||
# json.dumps defaults to ensure_ascii=True; supplementary characters
|
||||
# can become two six-byte Unicode escapes.
|
||||
return len(value) * 12 + 2
|
||||
if value is None or isinstance(value, (bool, int, float)):
|
||||
return 32
|
||||
if isinstance(value, dict):
|
||||
return 2 + sum(
|
||||
len(key) * 6 + 4 + _estimated_json_size(item)
|
||||
for key, item in value.items()
|
||||
)
|
||||
if isinstance(value, (list, tuple)):
|
||||
return 2 + sum(_estimated_json_size(item) + 1 for item in value)
|
||||
return 64
|
||||
|
||||
|
||||
def _fits_trace_budget(value, remaining_bytes):
|
||||
return _estimated_json_size(value) <= remaining_bytes
|
||||
|
||||
|
||||
class _ObservationJournal:
|
||||
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
|
||||
|
||||
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
|
||||
self.handle = handle
|
||||
self.max_bytes = max_bytes
|
||||
self.bytes = 0
|
||||
|
||||
def append(self, record):
|
||||
encoded = (json.dumps(record) + "\n").encode("utf-8")
|
||||
if self.bytes + len(encoded) > self.max_bytes:
|
||||
return False
|
||||
self.handle.write(encoded)
|
||||
self.handle.flush()
|
||||
self.bytes += len(encoded)
|
||||
return True
|
||||
|
||||
|
||||
def _set_launch_flags(launch):
|
||||
launch.SetLaunchFlags(
|
||||
launch.GetLaunchFlags()
|
||||
| lldb.eLaunchFlagDebug
|
||||
| lldb.eLaunchFlagStopAtEntry
|
||||
)
|
||||
|
||||
|
||||
def _hash_file(path):
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as target_file:
|
||||
while True:
|
||||
chunk = target_file.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _file_identity(path):
|
||||
info = os.stat(path)
|
||||
return str(info.st_dev), str(info.st_ino)
|
||||
|
||||
|
||||
def _module_identity(target):
|
||||
module = target.FindModule(target.GetExecutable())
|
||||
if not module.IsValid():
|
||||
return None, None, None
|
||||
spec = module.GetFileSpec()
|
||||
path = str(spec) if spec.IsValid() else None
|
||||
identity = None
|
||||
if path:
|
||||
try:
|
||||
identity = _file_identity(path)
|
||||
except OSError:
|
||||
pass
|
||||
return path, identity, module.GetUUIDString() or None
|
||||
|
||||
|
||||
def _wait_for_identity_ack(config):
|
||||
"""Hold the target at entry while the host adapter captures its start identity."""
|
||||
path = config.get("identity_ack_path")
|
||||
if not path:
|
||||
return
|
||||
while True:
|
||||
try:
|
||||
with open(path, "r") as handle:
|
||||
return handle.read(32).strip()
|
||||
except FileNotFoundError:
|
||||
time.sleep(0.025)
|
||||
|
||||
|
||||
class _OutputPipe:
|
||||
"""Drain a target FIFO while retaining only its bounded output prefix."""
|
||||
|
||||
def __init__(self, fifo_path, capture_path, max_bytes):
|
||||
os.mkfifo(fifo_path, 0o600)
|
||||
self.path = fifo_path
|
||||
try:
|
||||
self.fd = os.open(fifo_path, os.O_RDONLY | os.O_NONBLOCK)
|
||||
except Exception:
|
||||
os.unlink(fifo_path)
|
||||
raise
|
||||
try:
|
||||
self.capture = open(capture_path, "wb")
|
||||
except Exception:
|
||||
os.close(self.fd)
|
||||
os.unlink(fifo_path)
|
||||
raise
|
||||
self.max_bytes = max_bytes
|
||||
self.bytes = 0
|
||||
self.truncated = False
|
||||
self.closed = False
|
||||
self.stop_reader = threading.Event()
|
||||
self.reader_thread = None
|
||||
self.reader_error = None
|
||||
self.launched = False
|
||||
self.eof = threading.Event()
|
||||
self.complete = False
|
||||
|
||||
def start(self):
|
||||
if self.reader_thread is None:
|
||||
self.reader_thread = threading.Thread(target=self._reader_loop, daemon=True)
|
||||
self.reader_thread.start()
|
||||
|
||||
def _reader_loop(self):
|
||||
try:
|
||||
while not self.stop_reader.is_set():
|
||||
readable, _, _ = select.select([self.fd], [], [], 0.05)
|
||||
if not readable:
|
||||
continue
|
||||
if self.drain(OUTPUT_DRAIN_BYTES_PER_PASS) == 0:
|
||||
# A FIFO read end is readable at EOF before the target opens it.
|
||||
time.sleep(0.01)
|
||||
except Exception as error:
|
||||
self.reader_error = error
|
||||
self.stop_reader.set()
|
||||
|
||||
def mark_launched(self):
|
||||
self.launched = True
|
||||
|
||||
def drain(self, max_bytes=OUTPUT_DRAIN_BYTES_PER_PASS):
|
||||
consumed = 0
|
||||
while consumed < max_bytes:
|
||||
try:
|
||||
chunk = os.read(self.fd, min(65_536, max_bytes - consumed))
|
||||
except OSError as error:
|
||||
if error.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
return consumed
|
||||
raise
|
||||
if not chunk:
|
||||
if self.launched:
|
||||
self.eof.set()
|
||||
return consumed
|
||||
consumed += len(chunk)
|
||||
self.bytes += len(chunk)
|
||||
remaining = max(0, self.max_bytes - self.capture.tell())
|
||||
if remaining:
|
||||
self.capture.write(chunk[:remaining])
|
||||
if len(chunk) > remaining:
|
||||
self.truncated = True
|
||||
|
||||
def close(self, expected_exit=False):
|
||||
if self.closed:
|
||||
return
|
||||
if expected_exit and self.launched:
|
||||
self.eof.wait(OUTPUT_EOF_WAIT_SECONDS)
|
||||
self.stop_reader.set()
|
||||
if self.reader_thread is not None:
|
||||
self.reader_thread.join(OUTPUT_READER_JOIN_SECONDS)
|
||||
stopped = self.reader_thread is None or not self.reader_thread.is_alive()
|
||||
if stopped:
|
||||
try:
|
||||
self.drain(OUTPUT_DRAIN_BYTES_PER_PASS)
|
||||
finally:
|
||||
try:
|
||||
self.capture.close()
|
||||
finally:
|
||||
os.close(self.fd)
|
||||
self.closed = True
|
||||
else:
|
||||
# A daemon reader may still be unwinding a blocked filesystem write.
|
||||
# Leave its handles alone rather than racing that thread at shutdown.
|
||||
self.closed = True
|
||||
self.complete = bool(
|
||||
stopped
|
||||
and (not self.launched or (expected_exit and self.eof.is_set()))
|
||||
and self.reader_error is None
|
||||
)
|
||||
if not self.complete:
|
||||
self.truncated = True
|
||||
|
||||
|
||||
def _hex(value):
|
||||
return "0x%x" % value
|
||||
|
||||
|
||||
def _location(address, target):
|
||||
"""Load and file address, module and symbol of one code address."""
|
||||
module = address.GetModule()
|
||||
symbol = address.GetSymbol()
|
||||
file_address = address.GetFileAddress()
|
||||
location = {
|
||||
"load_address": _hex(address.GetLoadAddress(target)),
|
||||
"file_address": None
|
||||
if file_address == lldb.LLDB_INVALID_ADDRESS
|
||||
else _hex(file_address),
|
||||
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
|
||||
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
|
||||
"symbol": symbol.GetName() if symbol.IsValid() else None,
|
||||
}
|
||||
return location
|
||||
|
||||
|
||||
def _no_files():
|
||||
return lldb.SBFileSpecList()
|
||||
|
||||
|
||||
def _create_breakpoints(target, spec):
|
||||
"""LLDB breakpoints for one request; a class-qualified method may need two."""
|
||||
if spec["kind"] == "function":
|
||||
modules = lldb.SBFileSpecList()
|
||||
if spec.get("module"):
|
||||
modules.Append(lldb.SBFileSpec(spec["module"], False))
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
spec["name"], lldb.eFunctionNameTypeAuto, modules, _no_files()
|
||||
)
|
||||
]
|
||||
class_name = spec.get("class_name")
|
||||
method_type = spec.get("method_type", "any")
|
||||
if class_name is None:
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
spec["selector"], lldb.eFunctionNameTypeSelector, _no_files(), _no_files()
|
||||
)
|
||||
]
|
||||
prefixes = {"instance": ["-"], "class": ["+"], "any": ["-", "+"]}[method_type]
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
"%s[%s %s]" % (prefix, class_name, spec["selector"]),
|
||||
lldb.eFunctionNameTypeFull,
|
||||
_no_files(),
|
||||
_no_files(),
|
||||
)
|
||||
for prefix in prefixes
|
||||
]
|
||||
|
||||
|
||||
def _dynamic_class(target, pointer):
|
||||
"""Class of an Objective-C receiver from LLDB's runtime reader, without running code."""
|
||||
if pointer == 0:
|
||||
return None
|
||||
data = lldb.SBData.CreateDataFromUInt64Array(
|
||||
target.GetByteOrder(), target.GetAddressByteSize(), [pointer]
|
||||
)
|
||||
value = target.CreateValueFromData(
|
||||
"receiver", data, target.GetBasicType(lldb.eBasicTypeObjCID)
|
||||
)
|
||||
dynamic = value.GetDynamicValue(lldb.eDynamicDontRunTarget)
|
||||
name = dynamic.GetTypeName() if dynamic.IsValid() else None
|
||||
if not name or name in ("id", "objc_object *"):
|
||||
return None
|
||||
return name[:-2] if name.endswith(" *") else name
|
||||
|
||||
|
||||
def _wanted(spec, symbol):
|
||||
"""Selector-only requests may admit only instance or only class methods."""
|
||||
if spec["kind"] != "objc-method" or spec.get("class_name") is not None:
|
||||
return True
|
||||
method_type = spec.get("method_type", "any")
|
||||
if method_type == "any":
|
||||
return True
|
||||
return symbol.startswith("-[" if method_type == "instance" else "+[")
|
||||
|
||||
|
||||
def _read_register(frame, name):
|
||||
error = lldb.SBError()
|
||||
value = frame.FindRegister(name)
|
||||
if not value.IsValid():
|
||||
raise RuntimeError("LLDB breakpoint register %s is unavailable" % name)
|
||||
number = value.GetValueAsUnsigned(error)
|
||||
if error.Fail():
|
||||
raise RuntimeError(
|
||||
"LLDB breakpoint register %s could not be read: %s" % (name, error)
|
||||
)
|
||||
return number
|
||||
|
||||
|
||||
def _record(context, thread, frame, index, sequence, location, remaining_bytes):
|
||||
target, process, config = context["target"], context["process"], context["config"]
|
||||
symbol = location["symbol"] or ""
|
||||
if not _wanted(config["breakpoints"][index], symbol):
|
||||
return None
|
||||
names = ARGUMENT_REGISTERS.get(config["architecture"], [])
|
||||
objc_method = symbol.startswith("-[") or symbol.startswith("+[")
|
||||
values = {
|
||||
name: _read_register(frame, name)
|
||||
for name in names[: max(config["argument_registers"], 2 if objc_method else 0)]
|
||||
}
|
||||
registers = [
|
||||
{"name": name, "value": _hex(values[name])}
|
||||
for name in names[: config["argument_registers"]]
|
||||
]
|
||||
receiver_class = None
|
||||
selector = None
|
||||
if objc_method:
|
||||
error = lldb.SBError()
|
||||
text = process.ReadCStringFromMemory(
|
||||
values[names[1]],
|
||||
MAX_SELECTOR_BYTES,
|
||||
error,
|
||||
)
|
||||
selector = text if error.Success() and text else None
|
||||
if symbol.startswith("-["):
|
||||
receiver_class = _dynamic_class(target, values[names[0]])
|
||||
estimated_base = (
|
||||
_estimated_json_size(location)
|
||||
+ _estimated_json_size(receiver_class)
|
||||
+ _estimated_json_size(selector)
|
||||
+ 2048
|
||||
)
|
||||
if estimated_base > remaining_bytes:
|
||||
raise ValueError("trace event exceeds aggregate byte budget")
|
||||
record = {
|
||||
"sequence": sequence,
|
||||
"elapsed_ms": round((time.monotonic() - context["started"]) * 1000, 3),
|
||||
"thread_id": thread.GetThreadID(),
|
||||
"breakpoint_index": index,
|
||||
"receiver_class": receiver_class,
|
||||
"selector": selector,
|
||||
"registers": registers,
|
||||
"backtrace": [],
|
||||
}
|
||||
record.update(location)
|
||||
return record
|
||||
|
||||
|
||||
# One trace runs in each owned LLDB subprocess. Its callbacks execute at the
|
||||
# breakpoint before LLDB resumes, unlike snapshots of later public stop events.
|
||||
_TRACE_CONTEXT = None
|
||||
|
||||
|
||||
def _handle_breakpoint(context, frame, index):
|
||||
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
|
||||
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
|
||||
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
|
||||
thread = frame.GetThread()
|
||||
events = context["events"]
|
||||
location = _location(frame.GetPCAddress(), context["target"])
|
||||
symbol = location["symbol"] or ""
|
||||
if not _wanted(context["config"]["breakpoints"][index], symbol):
|
||||
return None
|
||||
frame_count = max(
|
||||
0,
|
||||
min(thread.GetNumFrames() - 1, context["config"]["backtrace_frames"]),
|
||||
)
|
||||
event_frame_count = 1 + frame_count
|
||||
if not _can_retain_trace(
|
||||
context["retained_trace_bytes"],
|
||||
context["retained_frames"],
|
||||
0,
|
||||
event_frame_count,
|
||||
):
|
||||
return "resource-limit"
|
||||
try:
|
||||
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
record = _record(
|
||||
context, thread, frame, index, len(events), location, remaining_bytes
|
||||
)
|
||||
except ValueError:
|
||||
return "resource-limit"
|
||||
if record is not None:
|
||||
if not _fits_trace_budget(record, remaining_bytes):
|
||||
return "resource-limit"
|
||||
record_bytes = len(json.dumps(record).encode("utf-8"))
|
||||
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
|
||||
return "resource-limit"
|
||||
for depth in range(1, frame_count + 1):
|
||||
try:
|
||||
location = _location(
|
||||
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
|
||||
)
|
||||
except ValueError:
|
||||
return "resource-limit"
|
||||
if (
|
||||
_estimated_json_size(location) + record_bytes + 1
|
||||
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
|
||||
):
|
||||
return "resource-limit"
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if record["backtrace"] else 0)
|
||||
if (
|
||||
context["retained_trace_bytes"]
|
||||
+ record_bytes
|
||||
+ addition
|
||||
> MAX_TRACE_BYTES
|
||||
):
|
||||
return "resource-limit"
|
||||
record["backtrace"].append(location)
|
||||
record_bytes += addition
|
||||
journal = context.get("observation_journal")
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "event", "event": record}
|
||||
):
|
||||
return "resource-limit"
|
||||
events.append(record)
|
||||
context["retained_frames"] += event_frame_count
|
||||
context["retained_trace_bytes"] += record_bytes
|
||||
if len(events) >= context["config"]["max_events"]:
|
||||
return "event-limit"
|
||||
return None
|
||||
|
||||
|
||||
def breakpoint_hit(frame, location, internal_dict):
|
||||
"""LLDB callback: capture before resuming, and stop on a limit or failure."""
|
||||
context = _TRACE_CONTEXT
|
||||
if context is None or context["outcome"] is not None or context["failure"] is not None:
|
||||
return True
|
||||
try:
|
||||
index = context["breakpoint_index"][location.GetBreakpoint().GetID()]
|
||||
context["outcome"] = _handle_breakpoint(context, frame, index)
|
||||
except Exception as error:
|
||||
context["failure"] = "%s: %s" % (type(error).__name__, error)
|
||||
return True
|
||||
return context["outcome"] is not None
|
||||
|
||||
|
||||
def _handle_stop(context, other_stops):
|
||||
"""Retain non-breakpoint stop observations; callbacks already capture hits."""
|
||||
for thread in context["process"]:
|
||||
if thread.GetStopReason() in (lldb.eStopReasonSignal, lldb.eStopReasonException):
|
||||
stop_description = thread.GetStopDescription(256)
|
||||
journal = context.get("observation_journal")
|
||||
if journal is not None and not journal.append(
|
||||
{"kind": "other-stop", "reason": stop_description}
|
||||
):
|
||||
return "resource-limit"
|
||||
other_stops.append(stop_description)
|
||||
if len(other_stops) >= MAX_OTHER_STOPS:
|
||||
return "stop-limit"
|
||||
return None
|
||||
|
||||
|
||||
def _breakpoint_report(target, created, config):
|
||||
report = []
|
||||
retained_location_bytes = 0
|
||||
locations_truncated = False
|
||||
for index, breakpoints in enumerate(created):
|
||||
locations = []
|
||||
count = 0
|
||||
for breakpoint in breakpoints:
|
||||
for position in range(breakpoint.GetNumLocations()):
|
||||
address = breakpoint.GetLocationAtIndex(position).GetAddress()
|
||||
location = _location(address, target)
|
||||
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
|
||||
continue
|
||||
count += 1
|
||||
if len(locations) < MAX_REPORTED_LOCATIONS:
|
||||
remaining_bytes = (
|
||||
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
|
||||
)
|
||||
if not _fits_trace_budget(location, remaining_bytes):
|
||||
locations_truncated = True
|
||||
continue
|
||||
location_bytes = len(json.dumps(location).encode("utf-8"))
|
||||
addition = location_bytes + (1 if locations else 0)
|
||||
if addition > remaining_bytes:
|
||||
locations_truncated = True
|
||||
continue
|
||||
locations.append(location)
|
||||
retained_location_bytes += addition
|
||||
report.append({"index": index, "location_count": count, "locations": locations})
|
||||
return report, locations_truncated
|
||||
|
||||
|
||||
def _launch_info(target, config, listener):
|
||||
launch = lldb.SBLaunchInfo(config["arguments"])
|
||||
launch.SetListener(listener)
|
||||
_set_launch_flags(launch)
|
||||
environment = target.GetEnvironment()
|
||||
for key, value in config["environment"].items():
|
||||
environment.Set(key, value, True)
|
||||
launch.SetEnvironment(environment, False)
|
||||
if config.get("working_directory"):
|
||||
launch.SetWorkingDirectory(config["working_directory"])
|
||||
launch.AddOpenFileAction(0, "/dev/null", True, False)
|
||||
launch.AddOpenFileAction(1, config["stdout_path"], False, True)
|
||||
launch.AddOpenFileAction(2, config["stderr_path"], False, True)
|
||||
return launch
|
||||
|
||||
|
||||
def _wait_for_exit(listener, process, seconds):
|
||||
event = lldb.SBEvent()
|
||||
deadline = time.monotonic() + seconds
|
||||
while process.GetState() != lldb.eStateExited and time.monotonic() < deadline:
|
||||
listener.WaitForEvent(1, event)
|
||||
|
||||
|
||||
def _failed_launch_description(process):
|
||||
if process.GetState() != lldb.eStateExited or process.GetExitStatus() != -1:
|
||||
return None
|
||||
description = process.GetExitDescription()
|
||||
if isinstance(description, str) and description.lower().startswith(("attach failed", "launch failed")):
|
||||
return description
|
||||
return None
|
||||
|
||||
|
||||
def trace(debugger, config):
|
||||
global _TRACE_CONTEXT
|
||||
debugger.SetAsync(True)
|
||||
# Stop at the symbol itself, where argument registers still hold the ABI
|
||||
# arguments, even when line tables would let LLDB skip the prologue.
|
||||
lldb.SBDebugger.SetInternalVariable(
|
||||
"target.skip-prologue", "false", debugger.GetInstanceName()
|
||||
)
|
||||
try:
|
||||
before_digest = _hash_file(config["executable"])
|
||||
before_identity = _file_identity(config["executable"])
|
||||
except OSError as error:
|
||||
return {"status": "target-integrity-error", "error": str(error)}
|
||||
if before_digest != config["expected_sha256"]:
|
||||
return {
|
||||
"status": "target-integrity-error",
|
||||
"error": "selected executable digest changed before LLDB target creation",
|
||||
}
|
||||
error = lldb.SBError()
|
||||
target = debugger.CreateTarget(
|
||||
config["executable"], config["architecture"], None, False, error
|
||||
)
|
||||
if not target.IsValid():
|
||||
return {"status": "target-error", "error": error.GetCString()}
|
||||
module_path_before, module_file_identity_before, module_uuid_before = (
|
||||
_module_identity(target)
|
||||
)
|
||||
if (
|
||||
module_file_identity_before is not None
|
||||
and module_file_identity_before != before_identity
|
||||
):
|
||||
return {
|
||||
"status": "target-integrity-error",
|
||||
"error": "LLDB executable module resolved to a different pre-launch file identity",
|
||||
}
|
||||
created = [_create_breakpoints(target, spec) for spec in config["breakpoints"]]
|
||||
breakpoint_index = {}
|
||||
for index, breakpoints in enumerate(created):
|
||||
for breakpoint in breakpoints:
|
||||
breakpoint_index[breakpoint.GetID()] = index
|
||||
listener = lldb.SBListener(LISTENER_NAME)
|
||||
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
|
||||
stdout = _OutputPipe(
|
||||
config["stdout_path"], config["stdout_capture_path"], output_budget
|
||||
)
|
||||
try:
|
||||
stderr = _OutputPipe(
|
||||
config["stderr_path"], config["stderr_capture_path"], output_budget
|
||||
)
|
||||
except Exception:
|
||||
stdout.close()
|
||||
raise
|
||||
stdout.start()
|
||||
stderr.start()
|
||||
observation_journal = None
|
||||
started = time.monotonic()
|
||||
process = None
|
||||
try:
|
||||
observation_path = config.get("observation_path")
|
||||
if observation_path:
|
||||
observation_journal = _ObservationJournal(
|
||||
open(observation_path, "wb")
|
||||
)
|
||||
process = target.Launch(_launch_info(target, config, listener), error)
|
||||
if not error.Success() or not process.IsValid():
|
||||
stdout.close()
|
||||
stderr.close()
|
||||
return {"status": "launch-error", "error": error.GetCString()}
|
||||
launch_failure = _failed_launch_description(process)
|
||||
if launch_failure is not None:
|
||||
stdout.close(expected_exit=True)
|
||||
stderr.close(expected_exit=True)
|
||||
return {"status": "launch-error", "error": launch_failure}
|
||||
stdout.mark_launched()
|
||||
stderr.mark_launched()
|
||||
# Publish the PID before any post-launch file or module inspection. The
|
||||
# process is stopped at entry and remains there until identity_ack.
|
||||
with open(config["pid_path"], "w") as handle:
|
||||
handle.write(str(process.GetProcessID()))
|
||||
identity_ack = _wait_for_identity_ack(config)
|
||||
if identity_ack != "readable":
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
expected_exit = process.GetState() == lldb.eStateExited
|
||||
stdout.close(expected_exit=expected_exit)
|
||||
stderr.close(expected_exit=expected_exit)
|
||||
return {
|
||||
"status": "target-identity-unavailable",
|
||||
"error": "host could not establish the launched process identity; target was stopped",
|
||||
}
|
||||
try:
|
||||
after_digest = _hash_file(config["executable"])
|
||||
after_identity = _file_identity(config["executable"])
|
||||
except OSError as identity_error:
|
||||
process.Kill()
|
||||
stdout.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
stderr.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
return {"status": "target-integrity-error", "error": str(identity_error)}
|
||||
if after_digest != config["expected_sha256"] or after_identity != before_identity:
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
stdout.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
stderr.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
return {
|
||||
"status": "target-integrity-error",
|
||||
"error": "selected executable digest or file identity changed during LLDB launch",
|
||||
}
|
||||
module_path_after, module_file_identity_after, module_uuid_after = (
|
||||
_module_identity(target)
|
||||
)
|
||||
if (
|
||||
module_file_identity_after is not None
|
||||
and module_file_identity_after != before_identity
|
||||
):
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
stdout.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
stderr.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
return {
|
||||
"status": "target-integrity-error",
|
||||
"error": "LLDB executable module resolved to a different file identity",
|
||||
}
|
||||
if (
|
||||
module_uuid_before is not None
|
||||
and module_uuid_after is not None
|
||||
and module_uuid_before != module_uuid_after
|
||||
):
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
stdout.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
stderr.close(expected_exit=process.GetState() == lldb.eStateExited)
|
||||
return {
|
||||
"status": "target-integrity-error",
|
||||
"error": "LLDB executable module identity changed during launch",
|
||||
}
|
||||
context = {
|
||||
"target": target,
|
||||
"process": process,
|
||||
"config": config,
|
||||
"started": started,
|
||||
"breakpoint_index": breakpoint_index,
|
||||
"retained_frames": 0,
|
||||
"retained_trace_bytes": 0,
|
||||
"observation_journal": observation_journal,
|
||||
}
|
||||
events = []
|
||||
context.update({"events": events, "outcome": None, "failure": None})
|
||||
_TRACE_CONTEXT = context
|
||||
for breakpoints in created:
|
||||
for breakpoint in breakpoints:
|
||||
breakpoint.SetScriptCallbackFunction("rea_lldb_tracer.breakpoint_hit")
|
||||
other_stops = []
|
||||
outcome = None
|
||||
deadline = started + config["duration_ms"] / 1000
|
||||
event = lldb.SBEvent()
|
||||
while outcome is None:
|
||||
if stdout.reader_error is not None:
|
||||
raise RuntimeError("stdout capture failed: %s" % stdout.reader_error)
|
||||
if stderr.reader_error is not None:
|
||||
raise RuntimeError("stderr capture failed: %s" % stderr.reader_error)
|
||||
if time.monotonic() >= deadline:
|
||||
outcome = "duration-elapsed"
|
||||
break
|
||||
if not listener.WaitForEvent(1, event):
|
||||
continue
|
||||
if not lldb.SBProcess.EventIsProcessEvent(event):
|
||||
continue
|
||||
state = lldb.SBProcess.GetStateFromEvent(event)
|
||||
if state == lldb.eStateExited:
|
||||
outcome = "exited"
|
||||
elif state == lldb.eStateStopped and not lldb.SBProcess.GetRestartedFromEvent(event):
|
||||
if context["failure"] is not None:
|
||||
raise RuntimeError(context["failure"])
|
||||
outcome = context["outcome"] or _handle_stop(context, other_stops)
|
||||
if outcome is None:
|
||||
process.Continue()
|
||||
pid = process.GetProcessID()
|
||||
exit_status = None
|
||||
exit_description = None
|
||||
if outcome == "exited":
|
||||
exit_status = process.GetExitStatus()
|
||||
exit_description = process.GetExitDescription()
|
||||
else:
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
process_exited = process.GetState() == lldb.eStateExited
|
||||
stdout.close(expected_exit=process_exited)
|
||||
stderr.close(expected_exit=process_exited)
|
||||
# SBTarget.Launch may succeed before debugserver reports an asynchronous
|
||||
# attach failure as an exited SBProcess. Its -1 status is not a target
|
||||
# exit code; preserve the debugger's concrete launch denial.
|
||||
launch_failure = _failed_launch_description(process)
|
||||
if launch_failure is not None:
|
||||
return {"status": "launch-error", "error": launch_failure}
|
||||
launch_identity_stable = (
|
||||
module_path_before is not None
|
||||
and module_path_after == module_path_before
|
||||
and module_file_identity_before == before_identity
|
||||
and module_file_identity_after == module_file_identity_before
|
||||
and module_uuid_before is not None
|
||||
and module_uuid_after == module_uuid_before
|
||||
)
|
||||
breakpoint_report, breakpoint_locations_truncated = _breakpoint_report(
|
||||
target, created, config
|
||||
)
|
||||
return {
|
||||
"status": "traced",
|
||||
"version": lldb.SBDebugger.GetVersionString().splitlines()[0],
|
||||
"pid": pid,
|
||||
"outcome": outcome,
|
||||
"exit_status": exit_status,
|
||||
"exit_description": exit_description,
|
||||
"killed": outcome != "exited",
|
||||
"terminated": process.GetState() == lldb.eStateExited,
|
||||
"elapsed_ms": round((time.monotonic() - started) * 1000, 3),
|
||||
"breakpoints": breakpoint_report,
|
||||
"breakpoint_locations_truncated": breakpoint_locations_truncated,
|
||||
"events": events,
|
||||
"resource_limit_reached": outcome == "resource-limit",
|
||||
"target_output": {
|
||||
"stdout_bytes": stdout.bytes,
|
||||
"stderr_bytes": stderr.bytes,
|
||||
"stdout_truncated": stdout.truncated,
|
||||
"stderr_truncated": stderr.truncated,
|
||||
"stdout_complete": stdout.complete,
|
||||
"stderr_complete": stderr.complete,
|
||||
},
|
||||
"target_identity": {
|
||||
"selected_file_sha256": after_digest,
|
||||
"loaded_image_sha256": None,
|
||||
"file_device": after_identity[0],
|
||||
"file_inode": after_identity[1],
|
||||
"module_path": module_path_after,
|
||||
"module_uuid": module_uuid_after,
|
||||
# This records matching path/module identity observations. LLDB
|
||||
# does not expose the exact mapped bytes, so it never proves a digest.
|
||||
"stable": launch_identity_stable,
|
||||
},
|
||||
"other_stops": other_stops,
|
||||
}
|
||||
|
||||
finally:
|
||||
_TRACE_CONTEXT = None
|
||||
if observation_journal is not None:
|
||||
try:
|
||||
observation_journal.handle.close()
|
||||
except Exception:
|
||||
pass
|
||||
if process is not None:
|
||||
try:
|
||||
if process.IsValid() and process.GetState() != lldb.eStateExited:
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
except Exception:
|
||||
pass
|
||||
for output_pipe in (stdout, stderr):
|
||||
try:
|
||||
output_pipe.close(
|
||||
expected_exit=(
|
||||
process is not None
|
||||
and process.IsValid()
|
||||
and process.GetState() == lldb.eStateExited
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def run(debugger, command, result, internal_dict):
|
||||
"""LLDB command entry point: `rea_trace <config.json>`."""
|
||||
with open(command.strip()) as handle:
|
||||
config = json.load(handle)
|
||||
try:
|
||||
output = trace(debugger, config)
|
||||
except Exception as error: # Reported as a tracer failure, never re-raised into LLDB.
|
||||
output = {"status": "tracer-error", "error": "%s: %s" % (type(error).__name__, error)}
|
||||
with open(config["result_path"], "w") as handle:
|
||||
json.dump(output, handle)
|
||||
|
||||
|
||||
def __lldb_init_module(debugger, internal_dict):
|
||||
debugger.HandleCommand("command script add -f rea_lldb_tracer.run rea_trace")
|
||||
@@ -1,273 +0,0 @@
|
||||
"""Source-owned tests for the LLDB bridge's bounded retention helpers."""
|
||||
|
||||
from io import BytesIO
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
|
||||
|
||||
fake_lldb = sys.modules.setdefault("lldb", types.ModuleType("lldb"))
|
||||
fake_lldb.LLDB_INVALID_ADDRESS = -1
|
||||
fake_lldb.eLaunchFlagDebug = 2
|
||||
fake_lldb.eLaunchFlagStopAtEntry = 4
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"rea_lldb_tracer", os.path.join(os.path.dirname(__file__), "rea_lldb_tracer.py")
|
||||
)
|
||||
tracer = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(tracer)
|
||||
|
||||
|
||||
class NativeTraceBudgetTests(unittest.TestCase):
|
||||
def test_journal_checks_exact_byte_budget_before_writing(self):
|
||||
record = {"kind": "other-stop", "reason": "signal"}
|
||||
encoded_size = len((tracer.json.dumps(record) + "\n").encode("utf-8"))
|
||||
stream = BytesIO()
|
||||
journal = tracer._ObservationJournal(stream, max_bytes=encoded_size - 1)
|
||||
self.assertFalse(journal.append(record))
|
||||
self.assertEqual(stream.getvalue(), b"")
|
||||
self.assertEqual(journal.bytes, 0)
|
||||
|
||||
def test_journal_appends_each_row_once_and_flushes_it(self):
|
||||
class CountingStream(BytesIO):
|
||||
writes = 0
|
||||
flushes = 0
|
||||
|
||||
def write(self, value):
|
||||
self.writes += 1
|
||||
return super().write(value)
|
||||
|
||||
def flush(self):
|
||||
self.flushes += 1
|
||||
return super().flush()
|
||||
|
||||
stream = CountingStream()
|
||||
journal = tracer._ObservationJournal(stream)
|
||||
journal.append({"kind": "event", "event": {"sequence": 0}})
|
||||
journal.append({"kind": "other-stop", "reason": "signal"})
|
||||
self.assertEqual((stream.writes, stream.flushes), (2, 2))
|
||||
self.assertEqual(journal.bytes, len(stream.getvalue()))
|
||||
|
||||
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
|
||||
self.assertTrue(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
11,
|
||||
2,
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
tracer._can_retain_trace(
|
||||
tracer.MAX_TRACE_BYTES - 10,
|
||||
tracer.MAX_TRACE_FRAMES - 2,
|
||||
10,
|
||||
3,
|
||||
)
|
||||
)
|
||||
|
||||
def test_oversized_producer_string_is_rejected_before_serialization(self):
|
||||
self.assertTrue(tracer._fits_trace_budget("method", 1024))
|
||||
self.assertFalse(tracer._fits_trace_budget("m" * 2048, 1024))
|
||||
self.assertGreater(
|
||||
tracer._estimated_json_size("🙂"), len('"🙂"'.encode("utf-8"))
|
||||
)
|
||||
|
||||
def test_launch_flags_preserve_default_debug_and_stop_at_entry(self):
|
||||
class Launch:
|
||||
flags = 10
|
||||
|
||||
def GetLaunchFlags(self):
|
||||
return self.flags
|
||||
|
||||
def SetLaunchFlags(self, flags):
|
||||
self.flags = flags
|
||||
|
||||
launch = Launch()
|
||||
tracer._set_launch_flags(launch)
|
||||
self.assertEqual(launch.flags, 14)
|
||||
|
||||
def test_output_pipe_counts_all_bytes_and_keeps_only_the_prefix(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
fifo = os.path.join(directory, "stdout.fifo")
|
||||
capture = os.path.join(directory, "stdout.capture")
|
||||
sink = tracer._OutputPipe(fifo, capture, 5)
|
||||
sink.start()
|
||||
sink.mark_launched()
|
||||
writer = os.open(fifo, os.O_WRONLY)
|
||||
try:
|
||||
for index in range(100):
|
||||
chunk = (b"hello world" if index == 0 else b"x" * 4096)
|
||||
os.write(writer, chunk)
|
||||
finally:
|
||||
os.close(writer)
|
||||
sink.drain()
|
||||
sink.close(expected_exit=True)
|
||||
with open(capture, "rb") as retained:
|
||||
self.assertEqual(retained.read(), b"hello")
|
||||
self.assertEqual(sink.bytes, 11 + 99 * 4096)
|
||||
self.assertTrue(sink.truncated)
|
||||
self.assertTrue(sink.complete)
|
||||
|
||||
def test_output_pipe_reports_incomplete_capture_after_unexpected_close(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
fifo = os.path.join(directory, "stdout.fifo")
|
||||
capture = os.path.join(directory, "stdout.capture")
|
||||
sink = tracer._OutputPipe(fifo, capture, 5)
|
||||
sink.start()
|
||||
sink.mark_launched()
|
||||
sink.close(expected_exit=False)
|
||||
self.assertFalse(sink.complete)
|
||||
self.assertTrue(sink.truncated)
|
||||
|
||||
def test_selector_location_report_uses_the_requested_method_type(self):
|
||||
class Symbol:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def GetName(self):
|
||||
return "+[Greeter shared]"
|
||||
|
||||
class FileSpec:
|
||||
def GetFilename(self):
|
||||
return "Fixture"
|
||||
|
||||
def __str__(self):
|
||||
return "/tmp/Fixture"
|
||||
|
||||
class Module:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def GetFileSpec(self):
|
||||
return FileSpec()
|
||||
|
||||
class Address:
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
def GetSymbol(self):
|
||||
return Symbol()
|
||||
|
||||
def GetFileAddress(self):
|
||||
return 1
|
||||
|
||||
def GetLoadAddress(self, target):
|
||||
return 2
|
||||
|
||||
class Location:
|
||||
def GetAddress(self):
|
||||
return Address()
|
||||
|
||||
class Breakpoint:
|
||||
def GetNumLocations(self):
|
||||
return 1
|
||||
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location()
|
||||
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(),
|
||||
[[Breakpoint()]],
|
||||
{
|
||||
"breakpoints": [
|
||||
{
|
||||
"kind": "objc-method",
|
||||
"selector": "shared",
|
||||
"method_type": "instance",
|
||||
}
|
||||
]
|
||||
},
|
||||
)
|
||||
self.assertEqual(report[0]["location_count"], 0)
|
||||
self.assertEqual(report[0]["locations"], [])
|
||||
self.assertFalse(locations_truncated)
|
||||
|
||||
def test_breakpoint_report_bounds_aggregate_location_json(self):
|
||||
tracer.lldb.LLDB_INVALID_ADDRESS = -1
|
||||
symbol_name = (
|
||||
"-[VeryLongButPlausibleObjectiveCController "
|
||||
"performTransactionWithRequestIdentifier:accountContext:"
|
||||
"securityPolicy:completionHandler:]"
|
||||
)
|
||||
module_path = (
|
||||
"/Applications/Enterprise Product.app/Contents/Frameworks/"
|
||||
"EnterpriseSubsystem.framework/Versions/A/EnterpriseSubsystem"
|
||||
)
|
||||
|
||||
class Symbol:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def GetName(self):
|
||||
return symbol_name
|
||||
|
||||
class FileSpec:
|
||||
def GetFilename(self):
|
||||
return "EnterpriseSubsystem"
|
||||
|
||||
def __str__(self):
|
||||
return module_path
|
||||
|
||||
class Module:
|
||||
def IsValid(self):
|
||||
return True
|
||||
|
||||
def GetFileSpec(self):
|
||||
return FileSpec()
|
||||
|
||||
class Address:
|
||||
def GetModule(self):
|
||||
return Module()
|
||||
|
||||
def GetSymbol(self):
|
||||
return Symbol()
|
||||
|
||||
def GetFileAddress(self):
|
||||
return 0x123456789
|
||||
|
||||
def GetLoadAddress(self, target):
|
||||
return 0x7123456789
|
||||
|
||||
class Location:
|
||||
def GetAddress(self):
|
||||
return Address()
|
||||
|
||||
class Breakpoint:
|
||||
def GetNumLocations(self):
|
||||
return 64
|
||||
|
||||
def GetLocationAtIndex(self, index):
|
||||
return Location()
|
||||
|
||||
breakpoint_count = 1000
|
||||
created = [[Breakpoint()] for _ in range(breakpoint_count)]
|
||||
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
|
||||
report, locations_truncated = tracer._breakpoint_report(
|
||||
object(), created, config
|
||||
)
|
||||
retained_bytes = sum(
|
||||
len(json.dumps(location).encode("utf-8"))
|
||||
+ (1 if index > 0 else 0)
|
||||
for row in report
|
||||
for index, location in enumerate(row["locations"])
|
||||
)
|
||||
self.assertTrue(locations_truncated)
|
||||
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
|
||||
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
|
||||
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,339 +0,0 @@
|
||||
import Darwin
|
||||
import Foundation
|
||||
|
||||
struct RunTokenObservation: Encodable {
|
||||
let pid: Int32
|
||||
let state: String
|
||||
let run_id: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
struct ProcessIdentityObservation: Encodable {
|
||||
let pid: Int32
|
||||
let state: String
|
||||
let identity: String?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
private struct ProcessMetadata {
|
||||
let identity: String
|
||||
let pointerSize: Int
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
fileprivate let metadata: [Int32: ProcessMetadata]
|
||||
let failure: String?
|
||||
}
|
||||
|
||||
typealias ProcessTableSysctlCall = (
|
||||
_ mib: inout [Int32],
|
||||
_ output: UnsafeMutableRawPointer?,
|
||||
_ size: inout Int
|
||||
) -> Int32
|
||||
|
||||
private func readProcessSnapshot() -> ProcessSnapshot {
|
||||
readProcessSnapshot { mib, output, size in
|
||||
sysctl(&mib, u_int(mib.count), output, &size, nil, 0)
|
||||
}
|
||||
}
|
||||
|
||||
func readProcessSnapshot(
|
||||
sysctlCall: ProcessTableSysctlCall
|
||||
) -> ProcessSnapshot {
|
||||
for _ in 0..<3 {
|
||||
var mib = [CTL_KERN, KERN_PROC, KERN_PROC_ALL, 0]
|
||||
var size = 0
|
||||
guard sysctlCall(&mib, nil, &size) == 0 else {
|
||||
let failure = errno
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(failure)")
|
||||
}
|
||||
guard size >= MemoryLayout<kinfo_proc>.size,
|
||||
size % MemoryLayout<kinfo_proc>.stride == 0 else {
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(errno)")
|
||||
}
|
||||
var processes = [kinfo_proc](
|
||||
repeating: kinfo_proc(),
|
||||
count: size / MemoryLayout<kinfo_proc>.stride
|
||||
)
|
||||
let result = processes.withUnsafeMutableBufferPointer { buffer in
|
||||
let output = buffer.baseAddress.map { UnsafeMutableRawPointer($0) }
|
||||
return sysctlCall(&mib, output, &size)
|
||||
}
|
||||
guard result == 0 else {
|
||||
let failure = errno
|
||||
if failure == ENOMEM { continue }
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(failure)")
|
||||
}
|
||||
guard size % MemoryLayout<kinfo_proc>.stride == 0 else {
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(errno)")
|
||||
}
|
||||
let processCount = size / MemoryLayout<kinfo_proc>.stride
|
||||
let metadata = Dictionary(
|
||||
processes.prefix(processCount).map { process in
|
||||
let start = process.kp_proc.p_starttime
|
||||
return (
|
||||
process.kp_proc.p_pid,
|
||||
ProcessMetadata(
|
||||
identity: "\(start.tv_sec):\(start.tv_usec)",
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
},
|
||||
uniquingKeysWith: { first, _ in first }
|
||||
)
|
||||
return ProcessSnapshot(metadata: metadata, failure: nil)
|
||||
}
|
||||
return ProcessSnapshot(metadata: [:], failure: "process_table_failed_\(ENOMEM)")
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
let snapshot = readProcessSnapshot()
|
||||
return pids.map { pid in
|
||||
guard let metadata = snapshot.metadata[pid] else {
|
||||
return unavailableIdentity(
|
||||
pid: pid,
|
||||
reason: snapshot.failure ?? "process_not_in_snapshot"
|
||||
)
|
||||
}
|
||||
return ProcessIdentityObservation(pid: pid, state: "readable", identity: metadata.identity, reason: nil)
|
||||
}
|
||||
}
|
||||
|
||||
private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityObservation {
|
||||
ProcessIdentityObservation(pid: pid, state: "unavailable", identity: nil, reason: reason)
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
case invalidBuffer
|
||||
case duplicateToken
|
||||
case invalidTokenEncoding
|
||||
case environmentUnavailable
|
||||
case appleVectorUnavailable
|
||||
case ambiguousEnvironmentBoundary
|
||||
}
|
||||
|
||||
struct ParsedRunToken {
|
||||
let runId: String?
|
||||
let environmentEntryCount: Int
|
||||
}
|
||||
|
||||
@_silgen_name("csops")
|
||||
private func csops(
|
||||
_ pid: Int32,
|
||||
_ ops: UInt32,
|
||||
_ addr: UnsafeMutableRawPointer?,
|
||||
_ size: Int
|
||||
) -> Int32
|
||||
|
||||
private let csOpsStatus: UInt32 = 0
|
||||
private let csPlatformBinary: UInt32 = 0x0400_0000
|
||||
private let csRestrict: UInt32 = 0x0000_0800
|
||||
|
||||
/** Read the target's current code-signing flags when csops permits inspection. */
|
||||
private func codeSigningFlags(pid: Int32) -> UInt32? {
|
||||
var flags: UInt32 = 0
|
||||
let result = withUnsafeMutablePointer(to: &flags) { pointer in
|
||||
csops(pid, csOpsStatus, UnsafeMutableRawPointer(pointer), MemoryLayout<UInt32>.size)
|
||||
}
|
||||
return result == 0 ? flags : nil
|
||||
}
|
||||
|
||||
/** Derive one process-token observation from parser facts and native flags. */
|
||||
func processRunTokenObservation(
|
||||
pid: Int32,
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: runId, reason: nil)
|
||||
}
|
||||
// A non-empty environment without REA's token proves this candidate is
|
||||
// unrelated. CS_RESTRICT cannot turn that readable fact into uncertainty.
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
|
||||
var mib = [CTL_KERN, KERN_PROCARGS2, Int32(pid)]
|
||||
var size = 0
|
||||
guard sysctl(&mib, u_int(mib.count), nil, &size, nil, 0) == 0, size >= MemoryLayout<Int32>.size else {
|
||||
return unavailable(pid: pid, reason: "sysctl_failed_\(errno)")
|
||||
}
|
||||
var bytes = [UInt8](repeating: 0, count: size)
|
||||
let result = bytes.withUnsafeMutableBytes { buffer in
|
||||
sysctl(&mib, u_int(mib.count), buffer.baseAddress, &size, nil, 0)
|
||||
}
|
||||
guard result == 0 else {
|
||||
return unavailable(pid: pid, reason: "sysctl_failed_\(errno)")
|
||||
}
|
||||
bytes = Array(bytes.prefix(size))
|
||||
let parse: Result<ParsedRunToken, RunTokenReadError>
|
||||
do {
|
||||
parse = .success(try parseProcessEnvironment(in: bytes, pointerSize: pointerSize))
|
||||
} catch let failure as RunTokenReadError {
|
||||
parse = .failure(failure)
|
||||
} catch {
|
||||
parse = .failure(.invalidBuffer)
|
||||
}
|
||||
return processRunTokenObservation(
|
||||
pid: pid,
|
||||
parse: parse,
|
||||
codeSigningFlags: { codeSigningFlags(pid: pid) }
|
||||
)
|
||||
}
|
||||
|
||||
func readRunTokens(pids: [Int32]) -> [RunTokenObservation] {
|
||||
let before = readProcessSnapshot()
|
||||
let observations = pids.map { pid -> RunTokenObservation in
|
||||
guard let metadata = before.metadata[pid] else {
|
||||
return unavailable(pid: pid, reason: before.failure ?? "process_not_in_snapshot")
|
||||
}
|
||||
return readRunToken(pid: pid, pointerSize: metadata.pointerSize)
|
||||
}
|
||||
let after = readProcessSnapshot()
|
||||
return zip(pids, observations).map { pid, observation in
|
||||
guard let initial = before.metadata[pid], let current = after.metadata[pid] else {
|
||||
return unavailable(pid: pid, reason: after.failure ?? "process_not_in_snapshot")
|
||||
}
|
||||
guard initial.identity == current.identity,
|
||||
initial.pointerSize == current.pointerSize else {
|
||||
return unavailable(pid: pid, reason: "process_identity_changed_during_token_read")
|
||||
}
|
||||
return observation
|
||||
}
|
||||
}
|
||||
|
||||
func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
var offset = MemoryLayout<Int32>.size
|
||||
guard let executablePath = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
guard pointerSize == 4 || pointerSize == 8 else { throw RunTokenReadError.invalidBuffer }
|
||||
let pathRecordSize = Array("executable_path=".utf8).count + executablePath.count + 1
|
||||
let padding = (pointerSize - pathRecordSize % pointerSize) % pointerSize
|
||||
guard offset + padding <= bytes.count,
|
||||
bytes[offset..<(offset + padding)].allSatisfy({ $0 == 0 }) else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
offset += padding
|
||||
for _ in 0..<argc {
|
||||
guard readString(in: bytes, offset: &offset, allowEmpty: true) != nil else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
}
|
||||
guard bytes.last == 0 else { throw RunTokenReadError.invalidBuffer }
|
||||
if offset == bytes.count {
|
||||
throw RunTokenReadError.environmentUnavailable
|
||||
}
|
||||
var trailingEntries: [ArraySlice<UInt8>] = []
|
||||
var zeroedAppleVectorStart: Int?
|
||||
while offset < bytes.count {
|
||||
if bytes[offset] == 0 {
|
||||
let start = offset
|
||||
while offset < bytes.count, bytes[offset] == 0 {
|
||||
offset += 1
|
||||
}
|
||||
if offset - start > 1 {
|
||||
zeroedAppleVectorStart = zeroedAppleVectorStart ?? trailingEntries.count
|
||||
}
|
||||
continue
|
||||
}
|
||||
guard let entry = readString(in: bytes, offset: &offset) else {
|
||||
throw RunTokenReadError.invalidBuffer
|
||||
}
|
||||
trailingEntries.append(entry)
|
||||
}
|
||||
// XNU appends its Apple vector after the environment. libc clears the
|
||||
// initial `pfz=` record after startup, leaving a multi-NUL run in its place.
|
||||
// A caller can also set `pfz`, so prefer the final intact sentinel when it
|
||||
// remains available and otherwise stop at the cleared record.
|
||||
let intactAppleVectorStart = trailingEntries.lastIndex(where: {
|
||||
$0.starts(with: Array("pfz=".utf8))
|
||||
})
|
||||
let preferZeroedBoundary = zeroedAppleVectorStart.flatMap { zeroedStart in
|
||||
intactAppleVectorStart.map { intactStart in intactStart < zeroedStart } ?? true
|
||||
} ?? false
|
||||
let appleVectorStart = preferZeroedBoundary
|
||||
? zeroedAppleVectorStart
|
||||
: intactAppleVectorStart
|
||||
guard let appleVectorStart else {
|
||||
throw RunTokenReadError.appleVectorUnavailable
|
||||
}
|
||||
if trailingEntries[appleVectorStart...].contains(where: {
|
||||
$0.starts(with: Array("REA_PROCESS_RUN_ID=".utf8))
|
||||
}) {
|
||||
throw RunTokenReadError.ambiguousEnvironmentBoundary
|
||||
}
|
||||
let environmentEntries = trailingEntries[..<appleVectorStart]
|
||||
var found: String?
|
||||
for entry in environmentEntries {
|
||||
if entry.starts(with: Array("REA_PROCESS_RUN_ID=".utf8)) {
|
||||
guard found == nil else { throw RunTokenReadError.duplicateToken }
|
||||
let value = entry.dropFirst("REA_PROCESS_RUN_ID=".utf8.count)
|
||||
guard let decoded = String(bytes: value, encoding: .utf8) else {
|
||||
throw RunTokenReadError.invalidTokenEncoding
|
||||
}
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
private func readString(
|
||||
in bytes: [UInt8],
|
||||
offset: inout Int,
|
||||
allowEmpty: Bool = false
|
||||
) -> ArraySlice<UInt8>? {
|
||||
guard offset < bytes.count, let end = bytes[offset...].firstIndex(of: 0),
|
||||
allowEmpty || end > offset else { return nil }
|
||||
let string = bytes[offset..<end]
|
||||
offset = end + 1
|
||||
return string
|
||||
}
|
||||
|
||||
private func unavailable(pid: Int32, reason: String) -> RunTokenObservation {
|
||||
RunTokenObservation(pid: pid, state: "unavailable", run_id: nil, reason: reason)
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
struct RunTokenResponse: Encodable {
|
||||
let results: [RunTokenObservation]
|
||||
}
|
||||
|
||||
struct ProcessIdentityResponse: Encodable {
|
||||
let results: [ProcessIdentityObservation]
|
||||
}
|
||||
|
||||
@main
|
||||
struct ProcessRunTokenReaderMain {
|
||||
static func main() {
|
||||
let arguments = Array(CommandLine.arguments.dropFirst())
|
||||
let identityMode = arguments.first == "--identities"
|
||||
let pidArguments = identityMode ? Array(arguments.dropFirst()) : arguments
|
||||
let pids = pidArguments.compactMap(Int32.init)
|
||||
guard pids.count == pidArguments.count else {
|
||||
fputs("invalid process ID input\n", stderr)
|
||||
exit(64)
|
||||
}
|
||||
do {
|
||||
let output: Data
|
||||
if identityMode {
|
||||
output = try JSONEncoder().encode(ProcessIdentityResponse(results: readProcessIdentities(pids: pids)))
|
||||
} else {
|
||||
output = try JSONEncoder().encode(RunTokenResponse(results: readRunTokens(pids: pids)))
|
||||
}
|
||||
FileHandle.standardOutput.write(output)
|
||||
} catch {
|
||||
fputs("could not encode process token observations\n", stderr)
|
||||
exit(70)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,101 +0,0 @@
|
||||
"""Fixed core-only GDB/pwndbg operation; no caller commands or historical PID lookup."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import resource
|
||||
import gdb
|
||||
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
|
||||
class ContextFailure(Exception):
|
||||
def __init__(self, reason, message):
|
||||
super().__init__(message)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def inspect(request):
|
||||
if gdb.selected_inferior().connection is not None:
|
||||
raise ContextFailure("decoder", "Owned GDB unexpectedly has an inferior connection before core loading.")
|
||||
root = Path(os.environ["REA_CORE_CONTEXT_REQUEST"]).parent
|
||||
empty = root / "no-libraries"
|
||||
empty.mkdir(mode=0o700)
|
||||
for command in ["set auto-load off", "set debuginfod enabled off", "set auto-solib-add off", "set history save off", "set pagination off",
|
||||
"set sysroot ./no-libraries", "set solib-search-path ./no-libraries"]:
|
||||
gdb.execute(command, to_string=True)
|
||||
entry = request["gdbinit"]
|
||||
try:
|
||||
exec(compile(Path(entry).read_bytes(), entry, "exec"), {"__name__": "__main__", "__file__": entry})
|
||||
import pwndbg.aglib.vmmap
|
||||
import pwndbg.lib.version
|
||||
except (ImportError, OSError) as error:
|
||||
raise ContextFailure("unavailable", "Selected pwndbg could not initialize: " + str(error)) from error
|
||||
version = pwndbg.lib.version.__version__
|
||||
if version != "2026.09.15" and not version.startswith("2026.09.15 build: "):
|
||||
raise ContextFailure("unsupported", "Expected unchanged pwndbg2026.09.15; selected plugin reports " + version + ".")
|
||||
gdb.execute('set context-sections ""', to_string=True)
|
||||
# core-file takes a raw filename and retains quote characters. Use a fixed
|
||||
# relative filename in the owned cwd so no selected path becomes GDB syntax.
|
||||
if Path(request["snapshot_path"]) != root.parent / "object.snapshot":
|
||||
raise ContextFailure("decoder", "Core context request did not select the shared owned snapshot.")
|
||||
gdb.execute("core-file ../object.snapshot", to_string=True)
|
||||
inferior = gdb.selected_inferior()
|
||||
if inferior.connection is None or inferior.connection.type != "core" or gdb.current_progspace().filename is not None:
|
||||
raise ContextFailure("decoder", "Requested core-only boundary was not established; no mapping query performed.")
|
||||
maps = []
|
||||
for page in pwndbg.aglib.vmmap.get():
|
||||
flags = page.flags
|
||||
maps.append({"start": hex(page.start), "end": hex(page.end), "offset": hex(page.offset), "reported_flags": flags,
|
||||
"permissions": {"read": bool(flags & 4), "write": bool(flags & 2), "execute": bool(flags & 1)} if 1 <= flags <= 7 else None,
|
||||
"pathname_display": page.objfile, "current_file_identity": "unknown"})
|
||||
return {"status": "available", "gdb_version": gdb.VERSION, "pwndbg_version": version, "connection": "core", "executable": None,
|
||||
"historical_pid": inferior.pid, "reported_limits": {"address_space_bytes": resource.getrlimit(resource.RLIMIT_AS)[0], "cpu_seconds": resource.getrlimit(resource.RLIMIT_CPU)[0], "file_size_bytes": resource.getrlimit(resource.RLIMIT_FSIZE)[0]}, "maps": maps, "confidence": "derived", "completeness": "unknown",
|
||||
"diagnostics": {"stdout": "", "stderr": "", "truncated": False}}
|
||||
|
||||
|
||||
def failure_exit(request_path, status, marker):
|
||||
try:
|
||||
descriptor = os.open(Path(request_path).parent / "resource.failure", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
os.write(descriptor, marker)
|
||||
os.close(descriptor)
|
||||
except (OSError, MemoryError):
|
||||
pass
|
||||
os._exit(status)
|
||||
|
||||
|
||||
def main():
|
||||
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
reply = {"ok": True, "value": inspect(request)}
|
||||
except ContextFailure as error:
|
||||
reply = {"ok": False, "reason": error.reason, "message": str(error)}
|
||||
except MemoryError:
|
||||
reserve = None
|
||||
failure_exit(request_path, 75, b"M")
|
||||
except Exception as error:
|
||||
if isinstance(error, OSError) and error.errno == 12:
|
||||
reserve = None
|
||||
failure_exit(request_path, 75, b"M")
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
try:
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds64 MiB reply budget."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
except MemoryError:
|
||||
reserve = None
|
||||
failure_exit(request_path, 75, b"M")
|
||||
except OSError as error:
|
||||
if error.errno == 27:
|
||||
failure_exit(request_path, 76, b"F")
|
||||
if error.errno == 12:
|
||||
reserve = None
|
||||
failure_exit(request_path, 75, b"M")
|
||||
raise
|
||||
|
||||
|
||||
main()
|
||||
@@ -1,39 +0,0 @@
|
||||
"""Catch typed resource failures while loading or running the owned debugger launcher.
|
||||
|
||||
Only this package's fixed sibling is executed; caller input stays in request.json.
|
||||
Interpreter startup failures before this boundary remain unclassified.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def resource_exit(status, marker):
|
||||
try:
|
||||
if len(sys.argv) > 2:
|
||||
descriptor = os.open(sys.argv[1], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
os.write(descriptor, marker)
|
||||
os.close(descriptor)
|
||||
except (OSError, MemoryError):
|
||||
# Without a complete marker the parent keeps this failure unclassified.
|
||||
pass
|
||||
os._exit(status)
|
||||
|
||||
try:
|
||||
implementation = os.path.join(os.path.dirname(__file__), "launch_impl.py")
|
||||
with open(implementation, "rb") as handle:
|
||||
source = handle.read()
|
||||
exec(compile(source, implementation, "exec"), {
|
||||
"__file__": implementation,
|
||||
"__name__": "__main__",
|
||||
})
|
||||
except MemoryError:
|
||||
# Reserved status: an observed allocation failure, including imports/compile.
|
||||
resource_exit(75, b"M")
|
||||
except OSError as error:
|
||||
# Linux ENOMEM is an observed allocation failure, independent of error text.
|
||||
if error.errno == 12:
|
||||
resource_exit(75, b"M")
|
||||
# Linux EFBIG is observed directly, including a partial limits/reply write.
|
||||
if error.errno == 27:
|
||||
resource_exit(76, b"F")
|
||||
raise
|
||||
@@ -1,34 +0,0 @@
|
||||
"""Lower inherited limits before replacing this owned process with GDB."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
runtime_path = Path(__file__).parent.parent / "pwntools" / "decoder_runtime.py"
|
||||
runtime = {"__file__": str(runtime_path), "__name__": "rea_decoder_runtime"}
|
||||
exec(compile(runtime_path.read_bytes(), str(runtime_path), "exec"), runtime)
|
||||
|
||||
|
||||
class LaunchFailure(Exception):
|
||||
def __init__(self, reason, message):
|
||||
super().__init__(message)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def launch(snapshot, cache):
|
||||
request_path = sys.argv[-1]
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
bridge = str(Path(__file__).with_name("core_context.py"))
|
||||
environment = dict(os.environ, REA_CORE_CONTEXT_REQUEST=request_path)
|
||||
# Only fixed package code appears in the GDB Python command. Selected paths stay JSON data.
|
||||
command = "python exec(compile(open(" + repr(bridge) + ", 'rb').read(), " + repr(bridge) + ", 'exec'), {'__name__':'__main__','__file__':" + repr(bridge) + "})"
|
||||
try:
|
||||
os.execve(request["gdb"], [request["gdb"], "-nx", "-nh", "--batch", "--quiet",
|
||||
"-iex", "set auto-load off", "-iex", "set debuginfod enabled off",
|
||||
"-iex", "set auto-solib-add off", "-ex", command], environment)
|
||||
except OSError as error:
|
||||
raise LaunchFailure("unavailable", "Selected GDB could not start (" + str(error.errno) + "): " + request["gdb"] + "; " + str(error)) from error
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
runtime["main"](sys.argv[-1], launch, LaunchFailure, "pwndbg-core-launcher")
|
||||
@@ -1,59 +0,0 @@
|
||||
"""Shared bounded reply/limit protocol, independent of the supplied artifact format."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import resource
|
||||
import sys
|
||||
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
def lower_resource_limits():
|
||||
limits = {}
|
||||
for name, kind, maximum in (
|
||||
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
|
||||
("cpu_seconds", resource.RLIMIT_CPU, 30),
|
||||
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
|
||||
):
|
||||
soft, hard = resource.getrlimit(kind)
|
||||
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
|
||||
effective = min(finite)
|
||||
# Preserve the inherited hard boundary and never raise a caller's soft limit.
|
||||
resource.setrlimit(kind, (effective, hard))
|
||||
limits[name] = effective
|
||||
return limits
|
||||
|
||||
|
||||
def main(request_path, inspect, failure_type, profile):
|
||||
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
|
||||
os.environ["PWNLIB_NOTERM"] = "1"
|
||||
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
|
||||
limits = None
|
||||
# Release bounded emergency headroom before constructing a MemoryError reply.
|
||||
# The outer exit-status contract also covers allocation failure during serialization.
|
||||
memory_reserve = bytearray(1024 * 1024)
|
||||
try:
|
||||
limits = lower_resource_limits()
|
||||
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
|
||||
json.dump(limits, handle)
|
||||
value = inspect(Path(request["snapshot_path"]), Path(request_path).parent / "cache")
|
||||
value["limitations"].append("Effective owned Python resource soft limits: " + json.dumps(limits, sort_keys=True) + ". Inherited tighter limits are retained.")
|
||||
reply = {"ok": True, "profile": profile, "value": value}
|
||||
except failure_type as error:
|
||||
reply = {"ok": False, "reason": error.reason, "message": str(error)}
|
||||
except MemoryError:
|
||||
memory_reserve = None
|
||||
reply = {"ok": False, "reason": "resource-limit", "message": "pwntools memory allocation failed under the effective resource limits; the exact allocation cause is unknown.", "reported_limits": limits}
|
||||
except Exception as error:
|
||||
if isinstance(error, OSError) and error.errno == 12:
|
||||
memory_reserve = None
|
||||
reply = {"ok": False, "reason": "resource-limit", "message": "pwntools reported OSError(ENOMEM); the exact allocation cause is unknown.", "reported_limits": limits}
|
||||
else:
|
||||
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -1,39 +0,0 @@
|
||||
"""Catch typed resource failures while loading or running the owned ELF adapter.
|
||||
|
||||
Only this package's fixed sibling is executed; caller input stays in request.json.
|
||||
Interpreter startup failures before this boundary remain unclassified.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def resource_exit(status, marker):
|
||||
try:
|
||||
if len(sys.argv) > 2:
|
||||
descriptor = os.open(sys.argv[1], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
os.write(descriptor, marker)
|
||||
os.close(descriptor)
|
||||
except (OSError, MemoryError):
|
||||
# Without a complete marker the parent keeps this failure unclassified.
|
||||
pass
|
||||
os._exit(status)
|
||||
|
||||
try:
|
||||
implementation = os.path.join(os.path.dirname(__file__), "layout_impl.py")
|
||||
with open(implementation, "rb") as handle:
|
||||
source = handle.read()
|
||||
exec(compile(source, implementation, "exec"), {
|
||||
"__file__": implementation,
|
||||
"__name__": "__main__",
|
||||
})
|
||||
except MemoryError:
|
||||
# Reserved status: an observed allocation failure, including imports/compile.
|
||||
resource_exit(75, b"M")
|
||||
except OSError as error:
|
||||
# Linux ENOMEM is an observed allocation failure, independent of error text.
|
||||
if error.errno == 12:
|
||||
resource_exit(75, b"M")
|
||||
# Linux EFBIG is observed directly, including a partial limits/reply write.
|
||||
if error.errno == 27:
|
||||
resource_exit(76, b"F")
|
||||
raise
|
||||
@@ -1,314 +0,0 @@
|
||||
"""Offline adapter to unchanged pwntools 4.15.0; never use libs/maps/libc.
|
||||
|
||||
Owned request/snapshot/reply paths are supplied by REA. No host target-process launch,
|
||||
user configuration initialization, debugger startup, or dependency installation.
|
||||
"""
|
||||
import base64
|
||||
import importlib.metadata
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import resource
|
||||
import sys
|
||||
|
||||
PROFILE = "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
def __init__(self, reason, message):
|
||||
super().__init__(message)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def address(value):
|
||||
return hex(int(value))
|
||||
|
||||
|
||||
def file_range(offset, size, length):
|
||||
return 0 <= offset <= length and 0 <= size <= length - offset
|
||||
|
||||
|
||||
def location(offset, size, length):
|
||||
if not file_range(offset, size, length):
|
||||
raise LayoutFailure("format", "ELF contains a file-backed range outside the selected snapshot.")
|
||||
return {"offset": address(offset), "bytes": address(size)}
|
||||
|
||||
|
||||
def name_reference(display, section, offset, content):
|
||||
header = getattr(section, "header", None)
|
||||
if header is None or header.sh_type != "SHT_STRTAB":
|
||||
return {"display": display, "bytes_base64": None, "location": None,
|
||||
"unknown_reason": "Referenced string table could not be resolved."}
|
||||
return name_in_table(display, header.sh_offset, header.sh_size, offset, content)
|
||||
|
||||
|
||||
def name_in_table(display, table_start, table_size, offset, content):
|
||||
location(table_start, table_size, len(content))
|
||||
if not 0 <= offset < table_size:
|
||||
raise LayoutFailure("format", f"ELF name offset {offset} lies outside its declared string table of {table_size} bytes at file offset {table_start}.")
|
||||
start = table_start + offset
|
||||
limit = table_start + table_size
|
||||
end = content.find(b"\0", start, limit)
|
||||
if end < 0:
|
||||
raise LayoutFailure("format", "ELF name lacks a terminator within its reported string table.")
|
||||
raw = content[start:end]
|
||||
return {"display": raw.decode("utf-8", "replace") if display is None else display, "bytes_base64": base64.b64encode(raw).decode("ascii"),
|
||||
"location": location(start, len(raw) + 1, len(content)), "unknown_reason": None}
|
||||
|
||||
|
||||
def dynamic_name_reference(tags, offset, image, content):
|
||||
starts = [tag.d_val for tag in tags if tag.d_tag == "DT_STRTAB"]
|
||||
sizes = [tag.d_val for tag in tags if tag.d_tag == "DT_STRSZ"]
|
||||
if not starts or not sizes:
|
||||
raise LayoutFailure("format", "ELF dependency requires DT_STRTAB and DT_STRSZ.")
|
||||
if len(starts) != 1 or len(sizes) != 1:
|
||||
raise LayoutFailure("unsupported-target", "ELF dependency string-table tags are ambiguous: " + repr({"DT_STRTAB": starts, "DT_STRSZ": sizes}))
|
||||
if not 0 <= offset < sizes[0]:
|
||||
raise LayoutFailure("format", f"ELF dependency name offset {offset} lies outside declared DT_STRSZ {sizes[0]}.")
|
||||
offsets = set()
|
||||
if len(starts) == 1 and len(sizes) == 1:
|
||||
for segment in image.iter_segments_by_type("PT_LOAD"):
|
||||
h = segment.header
|
||||
if starts[0] >= h.p_vaddr and starts[0] + sizes[0] <= h.p_vaddr + h.p_filesz:
|
||||
offsets.add(h.p_offset + starts[0] - h.p_vaddr)
|
||||
if len(offsets) != 1:
|
||||
raise LayoutFailure("unsupported-target", "Dynamic string table lacks a unique file-backed mapping: " + repr({"DT_STRTAB": starts, "DT_STRSZ": sizes, "candidate_file_offsets": sorted(offsets)}))
|
||||
return name_in_table(None, offsets.pop(), sizes[0], offset, content)
|
||||
|
||||
|
||||
def symbol_value_meaning(raw, image_type, sections):
|
||||
index = raw.st_shndx
|
||||
if raw.st_info.type == "STT_FILE": return "no-address"
|
||||
if index == "SHN_UNDEF": return "undefined"
|
||||
if index == "SHN_COMMON": return "alignment"
|
||||
if index == "SHN_ABS": return "absolute-value"
|
||||
if not isinstance(index, int) or not 0 < index < len(sections) or index >= 0xff00:
|
||||
return "unknown-section-index"
|
||||
if sections[index].header.sh_type == "SHT_NULL":
|
||||
return "unknown-section-index"
|
||||
if image_type == "ET_REL": return "section-offset"
|
||||
if raw.st_info.type == "STT_TLS": return "tls-offset"
|
||||
return "linked-virtual-address"
|
||||
|
||||
|
||||
def relocation_target(image_type, section_index, offset, sections):
|
||||
if image_type != "ET_REL":
|
||||
return {"kind": "linked-virtual-address", "address": address(offset)}
|
||||
if section_index == 0 or sections[section_index].header.sh_type == "SHT_NULL":
|
||||
return {"kind": "unknown-section", "reported_section_index": section_index,
|
||||
"offset": address(offset), "unknown_reason": "undefined-section-reference" if section_index == 0 else "inactive-section-header"}
|
||||
return {"kind": "section-offset", "section_index": section_index, "offset": address(offset)}
|
||||
|
||||
|
||||
def inspect_elf(path, cache):
|
||||
for package, expected in PACKAGES.items():
|
||||
try:
|
||||
version = importlib.metadata.version(package)
|
||||
except importlib.metadata.PackageNotFoundError as error:
|
||||
raise LayoutFailure("unavailable", "Selected Python lacks " + package + " " + expected + ".") from error
|
||||
if version != expected:
|
||||
raise LayoutFailure("unsupported", "Expected " + package + " " + expected + "; selected Python reports " + version + ".")
|
||||
content = path.read_bytes()
|
||||
if len(content) < 4 or content[:4] != b"\x7fELF":
|
||||
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
|
||||
if len(content) < 16:
|
||||
raise LayoutFailure("format", "Selected ELF identification is truncated.")
|
||||
if content[4] != 2 or content[5] != 1:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
|
||||
if len(content) < 64:
|
||||
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
|
||||
if int.from_bytes(content[18:20], "little") != 62:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
|
||||
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
from pwnlib.context import context
|
||||
with context.local(cache_dir=str(cache), log_level="warning"):
|
||||
from elftools.common.exceptions import ELFError
|
||||
from elftools.elf.elffile import ELFFile
|
||||
from pwnlib.elf import ELF
|
||||
try:
|
||||
headers = ELFFile(io.BytesIO(content))
|
||||
section_count = headers.num_sections()
|
||||
if headers.header.e_shstrndx == 0xffff and section_count == 0:
|
||||
raise LayoutFailure("format", "ELF extended section-name index requires a declared section-zero header.")
|
||||
raw_name_index = headers.header.e_shstrndx
|
||||
if 0xff00 <= raw_name_index < 0xffff:
|
||||
raise LayoutFailure("format", f"ELF section-name index {raw_name_index} is reserved and cannot directly reference a section.")
|
||||
section_name_index = headers.get_shstrndx()
|
||||
if raw_name_index == 0xffff and section_name_index < 0xff00:
|
||||
raise LayoutFailure("format", f"ELF SHN_XINDEX resolves to ordinary index {section_name_index}; the escape requires an index at least SHN_LORESERVE (0xff00).")
|
||||
if section_name_index != 0:
|
||||
if section_name_index >= section_count:
|
||||
raise LayoutFailure("format", f"ELF section-name table index {section_name_index} lies outside the declared section count {section_count}.")
|
||||
names_header = headers._get_section_header(section_name_index)
|
||||
if names_header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
|
||||
location(names_header.sh_offset, names_header.sh_size, len(content))
|
||||
except ELFError as error:
|
||||
raise LayoutFailure("format", str(error)) from error
|
||||
class LayoutELF(ELF):
|
||||
def _get_section_name(self, header):
|
||||
# SHN_UNDEF declares no names; upstream would otherwise read
|
||||
# strings at the inactive section-zero offset.
|
||||
return "" if section_name_index == 0 else super()._get_section_name(header)
|
||||
|
||||
# pyelftools 0.33 calls these documented filters internally;
|
||||
# pwntools 4.15.0's cached overrides do not accept the argument.
|
||||
# Forward the filter on this adapter instance, without modifying
|
||||
# either upstream package or process-global methods.
|
||||
def iter_segments(self, type=None):
|
||||
return (segment for segment in super().iter_segments()
|
||||
if type is None or segment.header.p_type == type)
|
||||
|
||||
def iter_sections(self, type=None):
|
||||
return (section for section in super().iter_sections()
|
||||
if type is None or section.header.sh_type == type)
|
||||
try:
|
||||
image = LayoutELF(str(path), checksec=False)
|
||||
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
|
||||
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
length = len(content)
|
||||
all_sections = list(image.iter_sections())
|
||||
shstrings = image.get_section(section_name_index) if all_sections and section_name_index != 0 else None
|
||||
sections, segments, symbols, relocations, packed_relatives = [], [], [], [], []
|
||||
needed, interpreters = [], []
|
||||
for index, section in enumerate(all_sections):
|
||||
h = section.header
|
||||
backing = h.sh_type not in ("SHT_NOBITS", "SHT_NULL")
|
||||
if backing: location(h.sh_offset, h.sh_size, length)
|
||||
sections.append({
|
||||
"index": index, "name": name_reference(section.name, shstrings, h.sh_name, content) if shstrings is not None else {"display": "", "bytes_base64": None, "location": None, "unknown_reason": "ELF declares no section-name table (SHN_UNDEF index 0)."}, "name_offset": address(h.sh_name), "type": h.sh_type,
|
||||
"header_location": location(image.header.e_shoff + index * image.header.e_shentsize, image.header.e_shentsize, length),
|
||||
"address": address(h.sh_addr), "offset": address(h.sh_offset),
|
||||
"size": address(h.sh_size), "alignment": address(h.sh_addralign),
|
||||
"flags": address(h.sh_flags), "link": h.sh_link, "info": h.sh_info,
|
||||
"entry_size": address(h.sh_entsize), "file_backing": "file" if backing else "none",
|
||||
})
|
||||
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
|
||||
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
|
||||
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
|
||||
strings = all_sections[h.sh_link]
|
||||
for entry, symbol in enumerate(section.iter_symbols()):
|
||||
raw = symbol.entry
|
||||
symbols.append({
|
||||
"table_index": index, "entry_index": entry,
|
||||
"name": name_reference(symbol.name, strings, raw.st_name, content), "name_offset": address(raw.st_name),
|
||||
"location": location(h.sh_offset + entry * h.sh_entsize, h.sh_entsize, length),
|
||||
"value": address(raw.st_value), "value_meaning": symbol_value_meaning(raw, image.header.e_type, all_sections),
|
||||
"size": address(raw.st_size), "binding": raw.st_info.bind, "type": raw.st_info.type,
|
||||
"visibility": raw.st_other.visibility, "section_index": raw.st_shndx,
|
||||
})
|
||||
if h.sh_type in ("SHT_REL", "SHT_RELA"):
|
||||
if h.sh_size % h.sh_entsize != 0:
|
||||
raise LayoutFailure(
|
||||
"format",
|
||||
f"ELF relocation section {index} size is not a multiple of its {h.sh_entsize}-byte entries.",
|
||||
)
|
||||
if image.header.e_type == "ET_REL" and h.sh_info >= len(all_sections):
|
||||
raise LayoutFailure("format", f"ELF relocation section {index} references missing target section {h.sh_info}.")
|
||||
symbol_table = None
|
||||
if h.sh_link != 0:
|
||||
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type not in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
raise LayoutFailure("format", f"ELF relocation section {index} links to invalid symbol table {h.sh_link}.")
|
||||
symbol_table = all_sections[h.sh_link]
|
||||
for entry, relocation in enumerate(section.iter_relocations()):
|
||||
raw = relocation.entry
|
||||
if raw.r_info_sym != 0 and (symbol_table is None or raw.r_info_sym >= symbol_table.num_symbols()):
|
||||
raise LayoutFailure("format", f"ELF relocation section {index} entry {entry} references missing symbol {raw.r_info_sym} in table {h.sh_link}.")
|
||||
relocations.append({
|
||||
"section_index": index, "entry_index": entry, "reported_offset": address(raw.r_offset),
|
||||
"target": relocation_target(image.header.e_type, h.sh_info, raw.r_offset, all_sections),
|
||||
"location": location(h.sh_offset + entry * h.sh_entsize, h.sh_entsize, length),
|
||||
"type": raw.r_info_type, "symbol_table_index": h.sh_link, "symbol_index": raw.r_info_sym,
|
||||
"symbol_reference_meaning": "zero-symbol-value" if raw.r_info_sym == 0 else "symbol-table-entry",
|
||||
"addend": None if "r_addend" not in raw else str(raw.r_addend),
|
||||
})
|
||||
if h.sh_type == "SHT_RELR":
|
||||
packed_relatives.append({
|
||||
"section_index": index,
|
||||
"location": location(h.sh_offset, h.sh_size, length),
|
||||
"encoded_bytes_base64": base64.b64encode(content[h.sh_offset:h.sh_offset + h.sh_size]).decode("ascii"),
|
||||
"entries": [{"decoded_index": decoded, "reported_offset": address(relocation.entry.r_offset)} for decoded, relocation in enumerate(section.iter_relocations())],
|
||||
"offset_meaning": "unknown" if image.header.e_type == "ET_REL" else "linked-virtual-address",
|
||||
"evidence_kind": "derived", "entry_source_locations": None, "addends": None,
|
||||
})
|
||||
for index, segment in enumerate(image.iter_segments()):
|
||||
h = segment.header
|
||||
backing = h.p_type != "PT_NULL" and h.p_filesz != 0
|
||||
if backing: location(h.p_offset, h.p_filesz, length)
|
||||
if h.p_type == "PT_LOAD" and h.p_memsz < h.p_filesz:
|
||||
raise LayoutFailure("format", "ELF load segment memory size is smaller than its file size.")
|
||||
segments.append({
|
||||
"index": index, "type": h.p_type, "offset": address(h.p_offset),
|
||||
"header_location": location(image.header.e_phoff + index * image.header.e_phentsize, image.header.e_phentsize, length),
|
||||
"file_size": address(h.p_filesz), "memory_size": address(h.p_memsz),
|
||||
"virtual_address": address(h.p_vaddr), "physical_address": address(h.p_paddr),
|
||||
"alignment": address(h.p_align), "flags": address(h.p_flags),
|
||||
"file_backing": "file" if backing else "none",
|
||||
"permissions": None if h.p_type == "PT_NULL" else {"read": bool(h.p_flags & 4), "write": bool(h.p_flags & 2), "execute": bool(h.p_flags & 1)},
|
||||
})
|
||||
if h.p_type == "PT_DYNAMIC":
|
||||
tag_size = image.structs.Elf_Dyn.sizeof()
|
||||
tags = []
|
||||
for tag_index in range(h.p_filesz // tag_size):
|
||||
tag = segment._get_tag(tag_index)
|
||||
tags.append(tag)
|
||||
if tag.d_tag == "DT_NULL":
|
||||
break
|
||||
else:
|
||||
raise LayoutFailure("format", f"ELF dynamic segment {index} has no complete DT_NULL terminator within its reported file range.")
|
||||
for tag in tags:
|
||||
if tag.d_tag == "DT_NEEDED":
|
||||
needed.append(dynamic_name_reference(tags, tag.d_val, image, content))
|
||||
if h.p_type == "PT_INTERP":
|
||||
raw = content[h.p_offset:h.p_offset + h.p_filesz]
|
||||
end = raw.find(b"\0")
|
||||
if end < 0:
|
||||
raise LayoutFailure("format", "ELF interpreter lacks a reported terminator.")
|
||||
interpreters.append({"display": raw[:end].decode("utf-8", "replace"), "bytes_base64": base64.b64encode(raw[:end]).decode("ascii"), "location": location(h.p_offset, end + 1, length), "unknown_reason": None})
|
||||
return {
|
||||
"format": "elf", "architecture": {"machine": image.header.e_machine, "bits": image.bits, "byte_order": image.endian},
|
||||
"image_type": image.header.e_type,
|
||||
"entry_point": {"reported_value": address(image.header.e_entry), "meaning": "not-applicable" if image.header.e_type == "ET_REL" else "absent" if image.header.e_entry == 0 else "linked-virtual-address", "execution_status": "unknown"},
|
||||
"runtime_load_base": None, "sections": sections, "segments": segments, "symbols": symbols, "relocations": relocations,
|
||||
"packed_relative_relocations": packed_relatives, "relocation_inventory_completeness": "unknown",
|
||||
"linkage": {"needed_libraries": needed, "interpreters": interpreters,
|
||||
"got": [{"display_name": name, "address": address(value)} for name, value in image.got.items()],
|
||||
"plt": [{"display_name": name, "address": address(value)} for name, value in image.plt.items()],
|
||||
"convenience_maps_completeness": "unknown", "runtime_library_paths": None},
|
||||
"mitigations": {"evidence_kind": "inferred", "position_independent": image.pie, "nx_indicator": image.nx, "executable_stack_indicator": image.execstack, "stack_canary_indicator": image.canary, "relro": image.relro},
|
||||
"limitations": [
|
||||
"PT_NULL payload fields are unused and retain their reported numbers without file-byte or permission claims. Zero-file-size segments provide no file bytes.",
|
||||
"SHN_XINDEX symbol values remain unresolved in this upstream representation; an external index table does not establish a resolved index in this report.",
|
||||
"Reported values preserve linked addresses, section offsets, TLS offsets, alignment and absolute values separately. Runtime load base and library paths are unknown.",
|
||||
"Names are display strings plus raw bytes and file ranges where resolvable; section/table/entry indices preserve identity and duplicates. Name ranges include their terminating NUL; raw name bytes exclude it.",
|
||||
"Ordinary relocation rows cover REL/RELA. RELR tables retain complete encoded bytes and upstream-decoded offsets as derived evidence; per-offset packed-word locations and implicit addends remain unknown. Relocation inventory completeness is unknown, including other encodings and missing section tables.",
|
||||
"Relocation symbol index zero uses a zero symbol value without a table lookup. Positive indices reference validated original symbol-table entries; raw table and symbol indices remain reported.",
|
||||
"Relocatable relocation targets with SHN_UNDEF or inactive SHT_NULL headers retain their reported section index/offset as unknown; they do not establish an associated target section.",
|
||||
"GOT/PLT are derived upstream convenience maps; PLT inference can emulate selected instructions in Unicorn. They do not establish target runtime behavior; aliases may collapse and warnings may indicate incomplete coverage. Their completeness is unknown.",
|
||||
"Mitigations are upstream static heuristics, not runtime protection. ET_DYN does not prove an executable; absent canary symbols do not prove every function unprotected.",
|
||||
"This profile inspects ELF EXEC/DYN/REL layout only. It does not analyze recorded cores, resolve loaded libraries, launch the target as a host process or start a debugger.",
|
||||
"Symbol/relocation inventories reflect original section tables. A sectionless image can still report dynamic dependency names; missing tables do not prove absence of dynamic symbols or relocations.",
|
||||
*(["Absent section-name table: section-dependent RELRO/canary heuristics may be incomplete; protection coverage is unknown."] if all_sections and section_name_index == 0 else []),
|
||||
*(["Sectionless images lack the .dynamic section and symbol tables used by upstream heuristics. RELRO/canary indicators may be incomplete; their values remain reported static candidates, with protection coverage unknown."] if not all_sections else []),
|
||||
],
|
||||
}
|
||||
except LayoutFailure:
|
||||
raise
|
||||
except (ELFError, ValueError, AssertionError, IndexError) as error:
|
||||
raise LayoutFailure("format", "Selected ELF failed unchanged upstream structural parsing: " + str(error)) from error
|
||||
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
runtime_path = Path(__file__).with_name("decoder_runtime.py")
|
||||
runtime = {"__file__": str(runtime_path), "__name__": "rea_decoder_runtime"}
|
||||
exec(compile(runtime_path.read_bytes(), str(runtime_path), "exec"), runtime)
|
||||
runtime["main"](sys.argv[-1], inspect_elf, LayoutFailure, PROFILE)
|
||||
@@ -1,39 +0,0 @@
|
||||
"""Catch typed resource failures while loading or running the owned recorded-core adapter.
|
||||
|
||||
Only this package's fixed sibling is executed; caller input stays in request.json.
|
||||
Interpreter startup failures before this boundary remain unclassified.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def resource_exit(status, marker):
|
||||
try:
|
||||
if len(sys.argv) > 2:
|
||||
descriptor = os.open(sys.argv[1], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
os.write(descriptor, marker)
|
||||
os.close(descriptor)
|
||||
except (OSError, MemoryError):
|
||||
# Without a complete marker the parent keeps this failure unclassified.
|
||||
pass
|
||||
os._exit(status)
|
||||
|
||||
try:
|
||||
implementation = os.path.join(os.path.dirname(__file__), "recorded_crash_impl.py")
|
||||
with open(implementation, "rb") as handle:
|
||||
source = handle.read()
|
||||
exec(compile(source, implementation, "exec"), {
|
||||
"__file__": implementation,
|
||||
"__name__": "__main__",
|
||||
})
|
||||
except MemoryError:
|
||||
# Reserved status: an observed allocation failure, including imports/compile.
|
||||
resource_exit(75, b"M")
|
||||
except OSError as error:
|
||||
# Linux ENOMEM is an observed allocation failure, independent of error text.
|
||||
if error.errno == 12:
|
||||
resource_exit(75, b"M")
|
||||
# Linux EFBIG is observed directly, including a partial limits/reply write.
|
||||
if error.errno == 27:
|
||||
resource_exit(76, b"F")
|
||||
raise
|
||||
@@ -1,147 +0,0 @@
|
||||
"""Bounded Linux core evidence via unchanged ELFFile/structs and pwntools ctypes.
|
||||
|
||||
No Corefile-selected thread, executable guessing, process lookup or target launch.
|
||||
"""
|
||||
import base64
|
||||
import ctypes
|
||||
import importlib.metadata
|
||||
import io
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
PROFILE = "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2"
|
||||
|
||||
|
||||
class CoreFailure(Exception):
|
||||
def __init__(self, reason, message):
|
||||
super().__init__(message)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
def encoded(value):
|
||||
return base64.b64encode(value).decode("ascii")
|
||||
|
||||
|
||||
def location(offset, size, limit):
|
||||
if not 0 <= offset <= limit or not 0 <= size <= limit - offset:
|
||||
raise CoreFailure("format", "Recorded source range exceeds its declared container.")
|
||||
return {"offset": hex(offset), "bytes": hex(size)}
|
||||
|
||||
|
||||
def inspect_core(path, cache):
|
||||
for package, required in {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}.items():
|
||||
try:
|
||||
actual = importlib.metadata.version(package)
|
||||
except importlib.metadata.PackageNotFoundError as error:
|
||||
raise CoreFailure("unavailable", "Selected Python lacks " + package + " " + required + ".") from error
|
||||
if actual != required:
|
||||
raise CoreFailure("unsupported", "Expected " + package + " " + required + "; selected Python reports " + actual + ".")
|
||||
content = path.read_bytes()
|
||||
if len(content) < 16 or content[:4] != b"\x7fELF":
|
||||
raise CoreFailure("format", "Selected recording lacks a complete ELF identification.")
|
||||
if content[4:6] != b"\x02\x01":
|
||||
raise CoreFailure("unsupported-target", "Initial recorded-core profile requires ELF64 little-endian.")
|
||||
if len(content) < 64:
|
||||
raise CoreFailure("format", "Selected ELF64 core header is truncated.")
|
||||
if content[6] != 1:
|
||||
raise CoreFailure("format", "ELF identification version is not EV_CURRENT.")
|
||||
if int.from_bytes(content[20:24], "little") != 1:
|
||||
raise CoreFailure("format", "ELF header version is not EV_CURRENT.")
|
||||
# Linux commonly emits ELFOSABI_NONE (0), also accepted as GNU/Linux (3).
|
||||
# Other declared OSABIs must never select Linux note ctypes.
|
||||
if content[7] not in (0, 3) or content[8] != 0:
|
||||
raise CoreFailure("unsupported-target", "Initial recorded-core profile requires System V or GNU/Linux OSABI with ABI version zero; selected OSABI=" + str(content[7]) + ", ABI version=" + str(content[8]) + ".")
|
||||
if int.from_bytes(content[16:18], "little") != 4 or int.from_bytes(content[18:20], "little") != 62:
|
||||
raise CoreFailure("unsupported-target", "Initial recorded-core profile requires x86-64 ET_CORE.")
|
||||
from elftools.common.exceptions import ELFError
|
||||
from elftools.common.utils import struct_parse
|
||||
from elftools.elf.elffile import ELFFile
|
||||
from pwnlib.elf.datatypes import elf_prstatus_amd64, elf_siginfo_64, user_regs_struct_amd64
|
||||
try:
|
||||
image = ELFFile(io.BytesIO(content))
|
||||
header = image.header
|
||||
count = image.num_segments()
|
||||
if count and header.e_phentsize < image.structs.Elf_Phdr.sizeof():
|
||||
raise CoreFailure("format", "Program-header entry is shorter than the required ELF64 structure.")
|
||||
location(header.e_phoff, count * header.e_phentsize, len(content))
|
||||
segments, notes, padding, threads, signals = [], [], [], [], []
|
||||
for segment_index, segment in enumerate(image.iter_segments()):
|
||||
h = segment.header
|
||||
backing = h.p_type != "PT_NULL" and h.p_filesz > 0
|
||||
if backing:
|
||||
location(h.p_offset, h.p_filesz, len(content))
|
||||
if h.p_type == "PT_LOAD" and h.p_filesz > h.p_memsz:
|
||||
raise CoreFailure("format", "Recorded PT_LOAD file bytes exceed memory size.")
|
||||
segments.append({"index": segment_index, "type": h.p_type,
|
||||
"header_location": location(header.e_phoff + segment_index * header.e_phentsize, header.e_phentsize, len(content)),
|
||||
"offset": hex(h.p_offset), "file_size": hex(h.p_filesz), "memory_size": hex(h.p_memsz),
|
||||
"virtual_address": hex(h.p_vaddr), "physical_address": hex(h.p_paddr), "alignment": hex(h.p_align), "flags": hex(h.p_flags), "file_backing": "file" if backing else "none"})
|
||||
if h.p_type != "PT_NOTE" or h.p_filesz == 0:
|
||||
continue
|
||||
position, end = h.p_offset, h.p_offset + h.p_filesz
|
||||
header_size = image.structs.Elf_Nhdr.sizeof()
|
||||
while position < end:
|
||||
if end - position < header_size:
|
||||
raw = content[position:end]
|
||||
if any(raw):
|
||||
raise CoreFailure("format", "Nonzero trailing note bytes lack a complete note header.")
|
||||
padding.append({"segment_index": segment_index, "location": location(position, len(raw), end), "bytes_base64": encoded(raw)})
|
||||
break
|
||||
note = struct_parse(image.structs.Elf_Nhdr, image.stream, stream_pos=position)
|
||||
owner_start = position + header_size
|
||||
descriptor_start = owner_start + ((note.n_namesz + 3) // 4) * 4
|
||||
note_end = descriptor_start + ((note.n_descsz + 3) // 4) * 4
|
||||
location(position, note_end - position, end)
|
||||
owner = content[owner_start:owner_start + note.n_namesz]
|
||||
if owner and b"\0" not in owner:
|
||||
raise CoreFailure("format", "Declared note owner has no NUL terminator.")
|
||||
display = owner.split(b"\0", 1)[0].decode("utf-8", "replace") if owner else None
|
||||
descriptor = content[descriptor_start:descriptor_start + note.n_descsz]
|
||||
note_index = len(notes)
|
||||
notes.append({"index": note_index, "segment_index": segment_index, "type": note.n_type,
|
||||
"location": location(position, note_end - position, end),
|
||||
"owner_location": location(owner_start, len(owner), end), "owner_bytes_base64": encoded(owner), "owner_display": display,
|
||||
"descriptor_location": location(descriptor_start, len(descriptor), end), "descriptor_bytes_base64": encoded(descriptor)})
|
||||
position = note_end
|
||||
# Only the initial Linux CORE ABI is interpreted. Other owners retain exact bytes.
|
||||
if owner != b"CORE\0":
|
||||
continue
|
||||
if note.n_type == "NT_PRSTATUS":
|
||||
if len(descriptor) < ctypes.sizeof(elf_prstatus_amd64):
|
||||
raise CoreFailure("format", "Linux amd64 NT_PRSTATUS descriptor is truncated.")
|
||||
status = elf_prstatus_amd64.from_buffer_copy(descriptor)
|
||||
registers = []
|
||||
for name, kind in user_regs_struct_amd64._fields_:
|
||||
relative = elf_prstatus_amd64.pr_reg.offset + getattr(user_regs_struct_amd64, name).offset
|
||||
registers.append({"name": name, "value": hex(getattr(status.pr_reg, name)), "location": location(descriptor_start + relative, ctypes.sizeof(kind), end)})
|
||||
threads.append({"note_index": note_index, "historical_pid": ctypes.c_int32(status.pr_pid).value, "recorded_current_signal": status.pr_cursig, "registers": registers})
|
||||
elif note.n_type == "NT_SIGINFO":
|
||||
if len(descriptor) < ctypes.sizeof(elf_siginfo_64):
|
||||
raise CoreFailure("format", "Linux amd64 NT_SIGINFO descriptor is truncated.")
|
||||
signal = elf_siginfo_64.from_buffer_copy(descriptor)
|
||||
number = ctypes.c_int32(signal.si_signo).value
|
||||
code = ctypes.c_int32(signal.si_code).value
|
||||
known = number == 11 and code in (1, 2)
|
||||
signals.append({"note_index": note_index, "number": number, "code": code, "errno": ctypes.c_int32(signal.si_errno).value,
|
||||
"fault_address": hex(signal.sigfault_addr) if known else None,
|
||||
"fault_address_meaning": "recorded-sigsegv-address" if known else "unknown", "thread_association": "unknown"})
|
||||
return {"format": "elf-core", "architecture": "x86_64-little-endian", "target_execution": "not-performed", "live_process_identity": "unknown",
|
||||
"segments": segments, "notes": notes, "note_padding": padding, "threads": threads, "signals": signals,
|
||||
"note_interpretation_completeness": "unknown",
|
||||
"limitations": ["The initial Linux note profile accepts System V/unspecified or GNU/Linux OSABI with ABI version zero; an unspecified OSABI alone does not prove operating-system origin.",
|
||||
"Historical PID and signal values are recorded metadata, not current process identity or attach authority.",
|
||||
"Only exact Linux CORE NT_PRSTATUS/NT_SIGINFO owners are interpreted; every descriptor and owner retains original bytes. Other note semantics remain unknown.",
|
||||
"NT_SIGINFO thread association is unknown. Only SIGSEGV SEGV_MAPERR/SEGV_ACCERR initially establish a fault-address union meaning.",
|
||||
"PT_LOAD file bytes are recorded bytes. Undumped memory and current executable/library identity remain unknown; no current host file is resolved from a recorded pathname.",
|
||||
"Missing notes/registers/signals do not establish their absence at the original crash. Full unwinding, executable pairing and live control are not provided."]}
|
||||
except CoreFailure:
|
||||
raise
|
||||
except (ELFError, ValueError, AssertionError, IndexError) as error:
|
||||
raise CoreFailure("format", "Recorded core failed unchanged upstream structural parsing: " + str(error)) from error
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
runtime_path = Path(__file__).with_name("decoder_runtime.py")
|
||||
runtime = {"__file__": str(runtime_path), "__name__": "rea_decoder_runtime"}
|
||||
exec(compile(runtime_path.read_bytes(), str(runtime_path), "exec"), runtime)
|
||||
runtime["main"](sys.argv[-1], inspect_core, CoreFailure, PROFILE)
|
||||
@@ -6,7 +6,6 @@ export default defineConfig({
|
||||
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
|
||||
lang: "en-US",
|
||||
base: "/rea/",
|
||||
srcExclude: ["readme/**"],
|
||||
themeConfig: {
|
||||
nav: [
|
||||
{ text: "Get started", link: "/installation" },
|
||||
@@ -18,8 +17,6 @@ export default defineConfig({
|
||||
text: "Getting started",
|
||||
items: [
|
||||
{ text: "Installation and setup", link: "/installation" },
|
||||
{ text: "Client configuration", link: "/client-configuration" },
|
||||
{ text: "CLI and Evidence", link: "/cli" },
|
||||
{ text: "Agent prompts", link: "/mcp-prompts" },
|
||||
{ text: "Roadmap", link: "/roadmap" },
|
||||
],
|
||||
@@ -28,7 +25,6 @@ export default defineConfig({
|
||||
text: "Investigation guides",
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
@@ -38,19 +34,16 @@ export default defineConfig({
|
||||
link: "/javascript-application-workflows",
|
||||
},
|
||||
{ text: "Browser observation", link: "/browser-observation" },
|
||||
{
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
{
|
||||
text: "Reconstruction readiness",
|
||||
link: "/reconstruction-readiness",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
@@ -60,9 +53,8 @@ export default defineConfig({
|
||||
{ text: "IDA", link: "/ida-provider" },
|
||||
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
|
||||
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
{ text: "Provider evaluation", link: "/provider-evaluation" },
|
||||
],
|
||||
},
|
||||
{
|
||||
@@ -72,12 +64,8 @@ export default defineConfig({
|
||||
{ text: "MCP contracts", link: "/mcp-contracts" },
|
||||
{ text: "Tool design", link: "/tool-design" },
|
||||
{ text: "Testing", link: "/testing" },
|
||||
{ text: "Client verification", link: "/client-verification" },
|
||||
{
|
||||
text: "Reconstruction readiness",
|
||||
link: "/reconstruction-readiness",
|
||||
},
|
||||
{ text: "Releasing", link: "/releasing" },
|
||||
{ text: "Architecture decisions", link: "/adr/README" },
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
# ADB device inspection and acquisition
|
||||
|
||||
REA inspects connected Android devices, reads device state, and transfers
|
||||
files and packages through the caller's `adb` binary. Emulators, USB
|
||||
devices, and network devices participate alike: REA neither starts nor
|
||||
manages them and never installs platform tools. Every operation composes a
|
||||
fixed adb argument vector — caller values occupy positional arguments only
|
||||
and are quoted when adb forwards them to the device shell.
|
||||
|
||||
The adb client may start its local server on first device query; that is adb
|
||||
client behavior, recorded in the tool result when observed.
|
||||
`inspect_adb_client` probes only the binary and never contacts the server.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use any Android platform-tools `adb` (verified with 34.0.5). Select it
|
||||
through the environment:
|
||||
|
||||
```sh
|
||||
# Optional: absolute path to the adb binary; without it, adb resolves from PATH.
|
||||
export REA_ADB_PATH=/absolute/path/to/adb
|
||||
```
|
||||
|
||||
REA never installs the Android SDK, an emulator, or a system image.
|
||||
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
|
||||
resolved binary, its version, and where it was selected from. A missing
|
||||
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
|
||||
rather than attempting an installation.
|
||||
|
||||
## Answer an analyst question
|
||||
|
||||
```sh
|
||||
# Device identity
|
||||
rea inspect-adb-client
|
||||
rea list-adb-devices
|
||||
rea inspect-adb-device emulator-5554
|
||||
rea inspect-adb-display emulator-5554
|
||||
rea list-adb-features emulator-5554
|
||||
rea list-adb-services emulator-5554
|
||||
|
||||
# Find and acquire an app (the easy-get flow)
|
||||
rea resolve-adb-packages emulator-5554 whatsapp
|
||||
rea list-adb-packages emulator-5554 --scope third_party
|
||||
rea inspect-adb-package emulator-5554 com.whatsapp
|
||||
rea pull-adb-package emulator-5554 com.whatsapp --output-directory /tmp/acq
|
||||
rea inspect-android-package /tmp/acq/com.whatsapp/base.apk
|
||||
|
||||
# Device state and files
|
||||
rea list-adb-processes emulator-5554
|
||||
rea inspect-adb-window emulator-5554
|
||||
rea read-adb-setting emulator-5554 global adb_enabled
|
||||
rea read-adb-logcat emulator-5554 --count 200
|
||||
rea list-adb-directory emulator-5554 /data/data/com.whatsapp
|
||||
rea pull-adb-file emulator-5554 /data/local/tmp/notes.txt --output-directory /tmp/acq
|
||||
rea push-adb-file emulator-5554 /tools/frida-server /data/local/tmp/frida-server
|
||||
rea capture-adb-screen emulator-5554 --output-directory /tmp/shots
|
||||
rea collect-adb-bugreport emulator-5554 --output-directory /tmp/bugs
|
||||
|
||||
# Package lifecycle and dynamic analysis (device-mutating)
|
||||
rea install-adb-package emulator-5554 /tmp/app.apk --replace
|
||||
rea start-adb-app emulator-5554 com.example.app
|
||||
rea read-adb-logcat emulator-5554 --pid 12345
|
||||
rea start-adb-activity emulator-5554 android.intent.action.VIEW --data-uri https://example.com
|
||||
rea stop-adb-app emulator-5554 com.example.app
|
||||
rea uninstall-adb-package emulator-5554 com.example.app
|
||||
```
|
||||
|
||||
Equivalent MCP methods use the snake_case names (`list_adb_devices`,
|
||||
`pull_adb_package`, `resolve_adb_packages`, …). Device operations take
|
||||
`serial` exactly as `adb devices` reports it. The pulled APK set feeds the
|
||||
existing analysis families directly, for example
|
||||
`rea inspect-android-package /tmp/acq/<package>/base.apk`.
|
||||
|
||||
## Operations
|
||||
|
||||
| Tool | Kind | What it does |
|
||||
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
|
||||
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
|
||||
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
|
||||
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
|
||||
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
|
||||
| `list_adb_services` | read | `service list` binder services and interfaces |
|
||||
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
|
||||
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
|
||||
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
|
||||
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
|
||||
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
|
||||
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
|
||||
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
|
||||
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
|
||||
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
|
||||
| `pull_adb_file` | read | One absolute device path pulled and digested |
|
||||
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
|
||||
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
|
||||
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); refuses existing targets without explicit overwrite; verifies via the device's own sha256sum |
|
||||
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
|
||||
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **Directory names retain whitespace and newlines.** Symlink targets do too,
|
||||
including trailing newlines. Records containing invalid UTF-8 remain as
|
||||
labelled hex in `unparsed_lines` and make directory coverage partial.
|
||||
- **Inferences carry their basis.** Device `kind` reports
|
||||
`kind_basis` (serial prefix or USB transport); APK `role` reports
|
||||
`role_basis: file_name`; `emulator_observed` is the build's own
|
||||
`ro.kernel.qemu` value, `null` when absent.
|
||||
- **Device paths are not host paths.** `base_apk_device_path` values live on
|
||||
the device filesystem. Only pulls and captures produce host-local paths,
|
||||
each paired with byte size and SHA-256 digest. A pulled file's host name is
|
||||
the device path's final POSIX segment. REA preserves the original device
|
||||
path and filename; a missing final segment or `.`/`..` is refused before
|
||||
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
|
||||
while POSIX hosts retain valid POSIX filenames such as names with spaces or
|
||||
colons. A package pull checks reported names against the destination
|
||||
filesystem before pulling, and refuses names that would share a local path.
|
||||
- **Splits are part of the answer.** `pm path` enumerates the base APK and
|
||||
every installed split; pulls cover the whole set. System packages whose
|
||||
APKs keep other names report the `unknown` role rather than guessing.
|
||||
- **Partial results keep facts.** Failed individual pulls return successful
|
||||
artifacts with `coverage: "partial"` and per-file `failures`. Listings
|
||||
that contain unrecognized lines report them as `unparsed_lines` and mark
|
||||
partial coverage instead of dropping rows.
|
||||
- **Mutations are explicit.** `push_adb_file`, `install_adb_package`, and
|
||||
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
|
||||
pushes refuse existing device files unless `overwrite` is set and verify
|
||||
the transfer with the device's own `sha256sum`; installs preserve the
|
||||
device's `Failure [...]` reason verbatim.
|
||||
- **Producer identity stays attached.** Every result embeds the producing
|
||||
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
|
||||
|
||||
## Resource and lifecycle limits
|
||||
|
||||
- Caller values reach adb only as positional arguments in fixed vectors;
|
||||
no tool accepts an arbitrary shell command line, `input`, or `monkey`.
|
||||
The app/activity lifecycle tools use `am start -W` and `am force-stop`.
|
||||
- Per-invocation budgets: client/devices 10-20 s, shell reads 20-60 s with
|
||||
64 KiB-8 MiB capture caps (logcat 4 MiB, screen 32 MiB), transfers and
|
||||
bugreports 600 s. Exceeded budgets report `resource_constraint`; stopped
|
||||
commands report `provider_timeout`.
|
||||
- Pull and capture outputs are additive: the destination must not already
|
||||
exist. Re-running a completed pull refuses the existing directory instead
|
||||
of overwriting evidence.
|
||||
- Cancellation during a package pull preserves caller-selected output. The
|
||||
error includes partial observations identifying completed artifacts and the
|
||||
interrupted local path; remaining APKs were not attempted.
|
||||
- Bugreports collect into isolated staging and publish with exclusive creation;
|
||||
filename collisions select the next numeric suffix. If publication fails, the
|
||||
completed archive remains in staging. The error's partial observation gives
|
||||
its recoverable `local_path`, the original `adb_reported_path`, and an unknown
|
||||
digest status. Later operations and shutdown preserve this retained evidence.
|
||||
- Failed package acquisition rollback retains its directory identity and retries
|
||||
cleanup before later work and during shutdown. Cleanup refuses a replacement
|
||||
directory. Terminal shutdown stops admission and drains active operations.
|
||||
- The provider holds no long-lived processes; every invocation is one
|
||||
bounded adb run. The adb server belongs to the caller.
|
||||
|
||||
## Related capability and follow-up lanes
|
||||
|
||||
UI interaction beyond intent starts (tapping, typing, monkey fuzzing, screen
|
||||
recording) remains a separate lane: candidates such as
|
||||
[mobile-device-mcp](https://github.com/srmorete/mobile-device-mcp) install
|
||||
driver software on the device and pair with the Frida instrumentation lane
|
||||
(#1359) behind explicit mutation-authority contracts. Network transports
|
||||
beyond already-connected devices (`adb connect`) are not exposed yet.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:adb` exercises every operation against a live device. It
|
||||
requires `adb` (or `REA_ADB_PATH`) and any attached emulator or phone;
|
||||
`--serial` selects one when several are attached, `--pull PACKAGE` pulls a
|
||||
real package and verifies the returned digests against the files on disk,
|
||||
and `--install-apk PATH` drives a full install → resolve → uninstall
|
||||
lifecycle with a lane-owned probe.
|
||||
|
||||
Record: verified against adb 34.0.5 on Linux against an Android 14
|
||||
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
|
||||
287 services, 92 features, a real two-APK split set pulled with byte-exact
|
||||
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
|
||||
complete package lifecycle; see `docs/testing.md` for the lane summary.
|
||||
@@ -0,0 +1,578 @@
|
||||
# ADR-0001: Provider selection and analysis profiles
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-07-15
|
||||
- Implementation status: The provider registry, deterministic selection,
|
||||
target binding, analysis-profile commitment, and snapshot/Evidence migration
|
||||
are implemented. Ghidra discovery, target/profile resolution, doctor checks,
|
||||
the private headless-session foundation, thirteen read-only inventory/load-image
|
||||
capabilities, and twelve function-analysis capabilities are implemented with
|
||||
real cross-format conformance. Linux/macOS also supports atomic session
|
||||
function annotations; experimental Windows x64 P0 uses bundled native controls
|
||||
and admits only the 25 read-only operations. See the
|
||||
[release boundary](../installation.md#released-package-and-main).
|
||||
|
||||
## Context
|
||||
|
||||
Before this decision, REA composed the artifact, native macOS, and Hopper
|
||||
providers with `CompositeProvider`. That class intentionally requires every
|
||||
operation to have one route. Hopper and Ghidra will both implement the deep
|
||||
static-analysis operation family, so putting both into that composite would
|
||||
either fail at construction or make array order an undeclared selection policy.
|
||||
|
||||
The active `BinarySession` also owned one composite client. `open_binary` had no
|
||||
provider selector, and the domain `BinaryTarget` contained Hopper loader
|
||||
arguments. Analysis snapshot v1 committed those loader arguments and each
|
||||
cached entry's provider identity, but it did not commit the selected analysis
|
||||
engine, engine version, or complete analysis configuration. Two analyses of the
|
||||
same bytes could therefore disagree because of language, compiler, loader,
|
||||
analyzer, or provider-version differences while appearing target-compatible.
|
||||
|
||||
The decision addressed the earlier representations of
|
||||
[`CompositeProvider`](https://github.com/morluto/rea/blob/main/src/application/binary/CompositeProvider.ts),
|
||||
[runtime composition](https://github.com/morluto/rea/blob/main/src/composition/binary.ts),
|
||||
[`BinarySession`](https://github.com/morluto/rea/blob/main/src/application/binary/BinarySession.ts),
|
||||
[`BinaryTarget`](https://github.com/morluto/rea/blob/main/src/domain/binaryTarget.ts),
|
||||
[analysis snapshot](https://github.com/morluto/rea/blob/main/src/domain/analysisSnapshot.ts), and
|
||||
[session lifecycle inputs](https://github.com/morluto/rea/blob/main/src/contracts/sessionLifecycleInputs.ts).
|
||||
|
||||
The next deep provider must not weaken the existing product invariants:
|
||||
|
||||
- stable provider-neutral CLI and MCP tool names;
|
||||
- exact provenance and explicit unavailable or degraded capabilities;
|
||||
- CLI and MCP parity;
|
||||
- no silent provider fallback;
|
||||
- provider-specific code outside the domain and application models;
|
||||
- local, actionable diagnostics with only genuine secrets redacted; and
|
||||
- snapshot reuse only when semantic analysis inputs are identical.
|
||||
|
||||
## Decision drivers
|
||||
|
||||
1. Hopper and Ghidra need to declare overlapping operations without ambiguous
|
||||
dispatch.
|
||||
2. A target must have one explainable deep-analysis producer for its lifetime.
|
||||
3. Target-free discovery must remain useful and must not start an analysis
|
||||
process.
|
||||
4. Existing Hopper-only configurations must keep working when no selector is
|
||||
supplied.
|
||||
5. Adding a second installed provider must not silently change results.
|
||||
6. Snapshot and Evidence compatibility must include every setting that can
|
||||
materially change analysis.
|
||||
7. Provider-specific open settings must not leak into `BinaryTarget` or become
|
||||
generic application fields.
|
||||
|
||||
## Terminology
|
||||
|
||||
- **Deep provider**: an engine such as Hopper or Ghidra that implements the
|
||||
overlapping static-analysis operation family.
|
||||
- **Auxiliary provider**: a provider with a disjoint operation family, such as
|
||||
artifact inventory or native macOS inspection.
|
||||
- **Candidate**: a registered deep provider before target selection.
|
||||
- **Binding**: the selected deep provider and analysis profile attached to one
|
||||
active target.
|
||||
- **Analysis profile**: a canonical commitment to the selected provider,
|
||||
provider version, and normalized semantic analysis settings.
|
||||
- **Available**: the adapter and its required local runtime can be used on this
|
||||
host.
|
||||
- **Supported**: the candidate accepts the parsed target and supplied open
|
||||
options. Availability and target support are separate observations.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Separate disjoint composition from overlapping selection
|
||||
|
||||
`CompositeProvider` remains the mechanism for operation families whose routes
|
||||
are disjoint. It must continue rejecting duplicate operation declarations. It
|
||||
must not acquire priority, racing, fallback, or provider-selection behavior.
|
||||
|
||||
A new `AnalysisProviderRegistry` owns the configured deep-provider candidates.
|
||||
Provider IDs are unique; duplicate IDs are a configuration error. The registry
|
||||
may hold multiple candidates that declare the same operation. It performs
|
||||
candidate discovery, target-support evaluation, option parsing, profile
|
||||
resolution, and selection, but it does not choose a different provider for each
|
||||
request.
|
||||
|
||||
After selection, the session composes exactly one bound deep provider with the
|
||||
auxiliary disjoint families:
|
||||
|
||||
```text
|
||||
target-free discovery
|
||||
|
|
||||
AnalysisProviderRegistry
|
||||
/ \
|
||||
Hopper Ghidra
|
||||
\ /
|
||||
one selected binding
|
||||
|
|
||||
+-------------------------+-------------------------+
|
||||
| | |
|
||||
selected deep family artifact family native family
|
||||
+-------------------------+-------------------------+
|
||||
|
|
||||
BinarySession
|
||||
```
|
||||
|
||||
An auxiliary operation continues to report its actual auxiliary provider as
|
||||
provenance. A deep operation always reports the provider in the active binding.
|
||||
The synthetic composite identity is never evidence that two engines jointly
|
||||
produced one observation.
|
||||
|
||||
Operation ownership is fixed by family:
|
||||
|
||||
| Operation family | Executor |
|
||||
| ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
|
||||
| Deep direct operations and provider-native function analysis | The one bound Hopper or Ghidra provider |
|
||||
| Application-enhanced deep workflows | Application composition over the bound provider's declared operations |
|
||||
| Native macOS inspection | The disjoint native provider |
|
||||
| Artifact inventory and extraction | The disjoint artifact provider |
|
||||
| Browser and Electron observation | Their independently authorized CDP providers; they do not participate in deep-provider selection |
|
||||
| Session lifecycle, Evidence, comparisons, and reconstruction | The application/session layer using the exact provider provenance recorded by their source operations |
|
||||
|
||||
Selecting a deep provider selects the whole deep family. If that provider marks
|
||||
an operation unavailable, REA returns the unavailability; it does not borrow
|
||||
that one operation from another deep provider. Application-enhanced workflows
|
||||
run only when the bound provider supplies their declared dependencies.
|
||||
|
||||
### 2. Discover candidates without launching analysis
|
||||
|
||||
Each registered candidate exposes three bounded adapter-owned observations:
|
||||
|
||||
1. identity and declared capability descriptors;
|
||||
2. host availability, including an actionable rejection when unavailable; and
|
||||
3. target support plus normalized profile resolution when a target is known.
|
||||
|
||||
Target-free discovery may inspect configuration, manifests, executable
|
||||
metadata, and documented version files. It must not import a target, create a
|
||||
project, open a GUI, or start the long-lived analysis process. Any short-lived
|
||||
version probe added later must be explicitly bounded and truthfully declared as
|
||||
a process effect.
|
||||
|
||||
Candidate output is sorted by provider ID for determinism. Sort order is not
|
||||
selection priority. Availability and support use stable codes plus a human
|
||||
diagnostic:
|
||||
|
||||
| Code | Meaning |
|
||||
| ---------------------------- | --------------------------------------------------------------- |
|
||||
| `not_configured` | Required provider location or configuration is absent |
|
||||
| `executable_missing` | The configured provider entry point is missing or not runnable |
|
||||
| `runtime_missing` | A required provider runtime, such as Java, is unavailable |
|
||||
| `unsupported_host` | The host platform or architecture is unsupported |
|
||||
| `unsupported_version` | The discovered provider or runtime version is unsupported |
|
||||
| `target_kind_unsupported` | The provider does not accept this target kind |
|
||||
| `target_format_unsupported` | The provider does not accept this target format |
|
||||
| `architecture_unsupported` | The selected target architecture is unsupported |
|
||||
| `target_role_unsupported` | The provider rejects this executable image role |
|
||||
| `managed_target_unsupported` | The provider does not admit managed code through this deep path |
|
||||
| `open_options_invalid` | The selected adapter rejected its open options |
|
||||
| `version_unresolved` | A concrete analysis-engine version could not be committed |
|
||||
|
||||
Diagnostics retain useful local paths, observed versions, architectures, and
|
||||
mismatch details. Credentials, authorization headers, license secrets, and
|
||||
other genuine secrets remain redacted.
|
||||
|
||||
### 3. Use one deterministic selection algorithm
|
||||
|
||||
The caller-visible selectors are:
|
||||
|
||||
- MCP `open_binary.provider_id`;
|
||||
- `rea ... --provider <provider-id|auto>` on every one-shot command that needs
|
||||
deep analysis; and
|
||||
- `REA_ANALYSIS_PROVIDER=<provider-id|auto>`, defaulting to `auto`.
|
||||
|
||||
The literal value `auto` is reserved and cannot be a provider ID. Precedence is
|
||||
request or CLI flag, then environment configuration, then the default `auto`.
|
||||
An explicit request for `auto` overrides a concrete environment preference and
|
||||
re-enters automatic selection.
|
||||
|
||||
Selection occurs after provider-neutral target parsing and before a deep client
|
||||
is created:
|
||||
|
||||
1. A concrete requested provider ID must exist, be available, support the
|
||||
target, accept its options, and resolve a concrete version and profile. Any
|
||||
failure returns a typed selection error; another provider is never tried.
|
||||
2. `auto` filters candidates by availability and target support.
|
||||
3. Exactly one usable candidate is selected with source
|
||||
`auto-single-candidate`.
|
||||
4. More than one usable candidate is an `ambiguous` selection error. The
|
||||
diagnostic lists the candidates and asks for `provider_id`, `--provider`, or
|
||||
`REA_ANALYSIS_PROVIDER`. REA does not prefer Hopper, Ghidra, registration
|
||||
order, the fastest startup, or the first health response.
|
||||
5. When no deep candidate is usable, an automatic open may still establish an
|
||||
unbound target so disjoint artifact or native operations remain available.
|
||||
Deep operations are then explicitly unavailable with the candidate
|
||||
rejections. A concrete request, or a concrete environment preference for a
|
||||
deep-eligible target, remains a hard requirement and fails instead of
|
||||
becoming unbound.
|
||||
6. A target that no deep adapter declares support for, such as an artifact-only
|
||||
target, is `not_applicable` rather than a failed deep analysis. Supplying a
|
||||
concrete provider in that open request is still an actionable unsupported-
|
||||
target error. A global environment preference does not make an artifact-only
|
||||
target require a deep provider.
|
||||
|
||||
This preserves target opening for non-Hopper capabilities on hosts without a
|
||||
deep engine while ensuring that installing a second usable deep engine never
|
||||
changes analysis silently.
|
||||
|
||||
### 4. Bind once and never fail over mid-session
|
||||
|
||||
An active deep binding contains:
|
||||
|
||||
```text
|
||||
canonical target identity
|
||||
selected provider identity and concrete version
|
||||
selection source: request | environment | auto-single-candidate
|
||||
analysis profile commitment
|
||||
owned provider client, once started
|
||||
```
|
||||
|
||||
The binding is immutable until `close_binary` or an explicit open/switch
|
||||
transaction. Provider client creation may remain lazy, but laziness cannot
|
||||
change which provider was selected.
|
||||
|
||||
- Opening the same canonical target without a new selector keeps the existing
|
||||
binding.
|
||||
- Opening the same target with the same provider and profile is idempotent.
|
||||
- A different explicit provider or a profile-changing option is a real switch:
|
||||
drain active calls, close owned resources, resolve the new binding, and start
|
||||
it as one serialized transaction.
|
||||
- If a switch fails, restore the prior target and binding on a best-effort
|
||||
transactional path and return the original switch error.
|
||||
- Provider process exit, timeout, cancellation, capability failure, or adapter
|
||||
error never triggers fallback. The binding stays visible and unavailable
|
||||
until the caller explicitly switches or closes it.
|
||||
- A snapshot never chooses or changes the provider. Selection happens first;
|
||||
snapshot compatibility is checked against the resulting binding.
|
||||
|
||||
Calls already admitted to one binding complete or cancel against that binding.
|
||||
No request can cross a concurrent target or provider transition.
|
||||
|
||||
### 5. Keep provider open options adapter-owned
|
||||
|
||||
`BinaryTarget` retains only provider-neutral source identity and classification:
|
||||
canonical path, source digest, generic kind and format, selected architecture,
|
||||
and available architectures. Hopper loader arguments leave the domain model.
|
||||
Provider-owned databases use a generic analysis-database classification; their
|
||||
adapter determines support. A legacy `.hop` target is therefore accepted only
|
||||
by Hopper without teaching the domain how to open it. Ghidra v1 continues to use
|
||||
private temporary projects and does not open user projects.
|
||||
|
||||
The generic transport may carry one bounded JSON `provider_options` object.
|
||||
Only the selected adapter parses its keys and reports `open_options_invalid` at
|
||||
that boundary. If this object is exposed over MCP, the CLI parity surface is
|
||||
`--provider-options-json` with the same JSON schema and limits. Unknown keys,
|
||||
wrong-provider options, excessive depth or size, and non-JSON values fail
|
||||
closed.
|
||||
|
||||
Environment settings that are inherently adapter-specific, such as the Hopper
|
||||
launcher or future Ghidra installation path, remain in adapter configuration.
|
||||
They do not become `BinaryTarget` fields. The application layer transports an
|
||||
opaque, already-bounded JSON value and the generic commitment described below;
|
||||
it does not interpret Hopper or Ghidra option names.
|
||||
|
||||
### 6. Commit a canonical analysis profile
|
||||
|
||||
Before creating a deep client, the selected adapter normalizes every semantic
|
||||
input that can affect analysis. Examples include Hopper loader selection and
|
||||
architecture flags, or Ghidra language ID, compiler specification, import mode,
|
||||
analyzer set, and analyzer options. Defaults are made explicit, unordered sets
|
||||
are sorted, and implicit host-dependent choices are resolved.
|
||||
|
||||
The adapter returns this provider-neutral envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": { "id": "ghidra", "version": "<concrete version>" },
|
||||
"parameters": {},
|
||||
"digest": "<sha256 of RFC 8785 canonical JSON excluding digest>"
|
||||
}
|
||||
```
|
||||
|
||||
`parameters` is bounded JSON interpreted only by that provider adapter. It
|
||||
contains analysis semantics, not credentials, authorization data, temporary
|
||||
paths, installation paths, process IDs, timestamps, or other incidental host
|
||||
state. The generic domain validates the envelope and digest but treats the
|
||||
parameters as opaque data.
|
||||
|
||||
A selected deep binding requires a concrete provider version. If the adapter
|
||||
cannot determine it, the candidate is unavailable with `version_unresolved`;
|
||||
REA does not persist a cache that merely commits `null` or "whatever is
|
||||
installed". Provider identity, normalized parameters, or provider version
|
||||
changes produce a different digest.
|
||||
|
||||
The same target, configuration, and provider must resolve the same profile over
|
||||
CLI and MCP. Profile resolution must be side-effect-free and complete before
|
||||
the provider imports the target.
|
||||
|
||||
### 7. Make snapshots provider- and profile-exact
|
||||
|
||||
Analysis snapshots replace Hopper loader arguments with an explicit binding
|
||||
commitment:
|
||||
|
||||
```text
|
||||
target: digest + generic format/kind + selected architecture
|
||||
binding: provider identity + analysis profile commitment
|
||||
entries: exact immutable queries
|
||||
evidence_bundle: retained Evidence records
|
||||
```
|
||||
|
||||
The snapshot target identity, canonical snapshot content digest, and every
|
||||
query ID commit the provider ID, concrete provider version, and profile digest.
|
||||
A cached entry is reusable only when all of the following match exactly:
|
||||
|
||||
- target digest, generic format and kind, and selected architecture;
|
||||
- provider ID and provider version;
|
||||
- profile digest;
|
||||
- operation and canonical parameters; and
|
||||
- the existing cacheability and immutability rules.
|
||||
|
||||
Provider or profile switches select a separate cache partition and cannot reuse
|
||||
entries from the previous binding. Auxiliary operations are cacheable only if
|
||||
their provider defines an equivalent explicit profile commitment; otherwise
|
||||
they remain Evidence but not snapshot entries.
|
||||
|
||||
Snapshot v1 cannot be parsed as the current snapshot format. Its `loader_args`
|
||||
cannot prove which engine version, defaults, or analyzer settings produced the
|
||||
entries, so snapshots must be recaptured before they can be used as a cache.
|
||||
An embedded Evidence bundle can still be imported separately through the
|
||||
Evidence import flow; it is never promoted into an analysis cache.
|
||||
|
||||
### 8. Add profile commitments to new deep Evidence while preserving legacy records
|
||||
|
||||
The optional `analysis_profile` field uses the canonical envelope above. Every
|
||||
newly created deep-analysis observation must include it, and its provider ID
|
||||
and version must match the record's provider. The field participates in the
|
||||
Evidence semantic ID.
|
||||
|
||||
Existing Evidence records without `analysis_profile` remain valid and keep
|
||||
their existing IDs. Absence on a legacy deep record means the analysis profile
|
||||
is unknown; REA does not infer or enrich it during import. Such evidence can be
|
||||
displayed and cited but cannot establish profile compatibility, seed snapshot
|
||||
replay, or support an "unchanged" conclusion that depends on identical analysis
|
||||
semantics.
|
||||
|
||||
Different providers or profile digests are never snapshot-compatible. A
|
||||
derived comparison may compare cross-provider normalized facts only when its
|
||||
own contract explicitly defines that semantic comparison. Exact pseudocode,
|
||||
provider text, or provider-specific metadata is not treated as equivalent. A
|
||||
missing or incompatible profile yields unknown or incompatible, never equal.
|
||||
|
||||
### 9. Expose selection truth in status and diagnostics
|
||||
|
||||
`binary_session`, `rea providers`, and `rea capabilities` expose the same
|
||||
candidate and binding model. The target-free form does not start providers and
|
||||
uses `unknown` for target support:
|
||||
|
||||
```json
|
||||
{
|
||||
"analysis_provider_binding": null,
|
||||
"analysis_provider_candidates": [
|
||||
{
|
||||
"provider": {
|
||||
"id": "hopper",
|
||||
"name": "Hopper Disassembler",
|
||||
"version": "<detected version>"
|
||||
},
|
||||
"availability": {
|
||||
"status": "available",
|
||||
"code": null,
|
||||
"reason": null,
|
||||
"diagnostics": {}
|
||||
},
|
||||
"target_support": {
|
||||
"status": "unknown",
|
||||
"code": null,
|
||||
"reason": "No target is open."
|
||||
},
|
||||
"selected": false,
|
||||
"capabilities": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
With a binding, `analysis_provider_binding` includes the selected identity,
|
||||
selection source, and complete profile commitment. Every candidate reports
|
||||
`supported`, `unsupported`, or `unknown` target support and whether it was
|
||||
selected. Candidate capability descriptors preserve supported, unsupported,
|
||||
and degraded truth independently.
|
||||
|
||||
An open target also exposes `analysis_run`. Its run ID is allocated by the
|
||||
session before client construction and propagated into selected Hopper or
|
||||
Ghidra bridge handshake and process environment. Dynamic clients retain
|
||||
provider-attributed lineage snapshots only after revalidating launcher
|
||||
parent/command and every observed launcher or descendant run token. `not_observed`
|
||||
means no dynamic provider has supplied a snapshot. Once one or more start,
|
||||
`snapshots` preserves every provider observation; each remains explicitly
|
||||
`unavailable` or `verified` and records its `observed_at` time. Retained
|
||||
snapshots are not live inventories, and a point-in-time observation never proves
|
||||
historical absence.
|
||||
|
||||
The existing flat `provider`, `providers`, and `capabilities` status fields stay
|
||||
during the 1.x compatibility window. Their effective deep-operation entries
|
||||
come only from the active binding; target-free ambiguity is unavailable rather
|
||||
than arbitrarily collapsed. The new candidate fields are authoritative for
|
||||
selection. A future major release may remove the synthetic composite provider
|
||||
field after callers have migrated.
|
||||
|
||||
Selection failures extend the tagged `ProviderSelectionError` with a stable
|
||||
reason:
|
||||
|
||||
- `unknown_provider`;
|
||||
- `provider_unavailable`;
|
||||
- `target_unsupported`;
|
||||
- `ambiguous`;
|
||||
- `invalid_options`.
|
||||
|
||||
The error includes the requested ID or `auto`, candidate IDs, rejection codes,
|
||||
and actionable local diagnostics. `AnalysisCapabilityUnavailableError` remains
|
||||
the error for an operation the selected provider cannot supply.
|
||||
`ProviderAdapterError` and more specific provider errors remain failures of the
|
||||
already selected provider. None authorizes fallback.
|
||||
|
||||
A snapshot whose provider or profile differs from the selected binding returns
|
||||
`EvidenceIntegrityError` with stable `profile_mismatch` details. It is an
|
||||
integrity incompatibility after selection, not permission to select the
|
||||
snapshot's provider.
|
||||
|
||||
### 10. Preserve CLI and MCP parity
|
||||
|
||||
- `open_binary.provider_id` and one-shot `--provider` use the same parser and
|
||||
selection service.
|
||||
- `provider_id: "auto"`, `--provider auto`, and
|
||||
`REA_ANALYSIS_PROVIDER=auto` have identical auto semantics.
|
||||
- Startup targets opened from environment configuration use the same service.
|
||||
- `rea providers`, `rea capabilities`, and target-free `binary_session` use the
|
||||
same candidate snapshot and reason codes.
|
||||
- Guided prompt `provider_id` context is a preference to forward into the open
|
||||
operation; it does not create a hidden routing channel.
|
||||
- Tool names and normalized output contracts remain provider-neutral. REA does
|
||||
not add `ghidra_decompile` or `hopper_decompile` tools.
|
||||
|
||||
## Compatibility and migration
|
||||
|
||||
| Existing behavior or data | Decision |
|
||||
| ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
||||
| No provider selector, only Hopper usable | `auto` selects the sole candidate; behavior remains equivalent |
|
||||
| No deep provider usable | Automatic open may remain unbound for auxiliary operations; deep capabilities explain why they are unavailable |
|
||||
| Hopper and Ghidra both usable, no preference | Open fails with `ambiguous`; installation order never becomes policy |
|
||||
| Existing Hopper environment settings | Remain supported as Hopper adapter configuration |
|
||||
| `open_binary` and CLI structured `loaderArgs` output | Retained as a deprecated Hopper-only compatibility projection for 1.x; the analysis profile is authoritative |
|
||||
| `BinaryTarget.loaderArgs` | Removed; Hopper derives them inside its adapter and commits normalized semantics in the profile |
|
||||
| Legacy `.hop` input | Classified generically as an analysis database and accepted only by Hopper |
|
||||
| Snapshot v1 | Not accepted as the current snapshot format; recapture before cache replay |
|
||||
| Existing Evidence without a profile | Accepted unchanged; profile compatibility is unknown |
|
||||
| New deep-analysis Evidence | Must include `analysis_profile`; its semantic ID commits the field |
|
||||
| Existing status fields | Retained for 1.x with additive binding and candidate fields |
|
||||
| Provider failure after selection | Returned from the selected provider; no transparent retry through another engine |
|
||||
|
||||
The deprecated `loaderArgs` response must never be read to select a provider or
|
||||
validate a snapshot. It is empty for non-Hopper bindings and may be removed only
|
||||
with a separately documented major contract change.
|
||||
|
||||
## Required behavior scenarios
|
||||
|
||||
| Scenario | Result |
|
||||
| ---------------------------------------------------------- | --------------------------------------------------------------------------- |
|
||||
| Hopper is the only usable candidate, selector omitted | Hopper is bound by `auto-single-candidate` |
|
||||
| Ghidra is the only usable candidate, selector omitted | Ghidra is bound by `auto-single-candidate` |
|
||||
| Both are usable, selector omitted | Typed `ambiguous` error; neither provider starts |
|
||||
| Environment selects Ghidra, request omits a selector | Ghidra is required and bound or the open fails with its rejection |
|
||||
| Environment selects Ghidra, request selects Hopper | Request wins; Hopper is required |
|
||||
| Environment selects Ghidra, request selects `auto` | Auto policy runs and is ambiguous when both are usable |
|
||||
| Unknown concrete provider ID | Typed `unknown_provider`; no target transition |
|
||||
| Selected provider rejects the target or options | Typed rejection from that candidate only |
|
||||
| Selected provider times out after binding | Binding remains selected and unavailable; no fallback |
|
||||
| Same target is reopened without a selector | Existing binding remains unchanged |
|
||||
| Same bytes are opened with a different profile | Real provider switch and a separate snapshot partition |
|
||||
| Snapshot provider or profile differs from the binding | Typed `EvidenceIntegrityError` with `profile_mismatch`; nothing is imported |
|
||||
| Artifact-only target has no deep candidate | Target opens unbound; disjoint artifact capabilities remain usable |
|
||||
| Environment prefers Ghidra for an artifact-only target | Deep selection is `not_applicable`; auxiliary operations remain usable |
|
||||
| Concrete deep provider is requested for artifact-only data | Typed `target_unsupported`; the request is not silently ignored |
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive consequences:
|
||||
|
||||
- Hopper and Ghidra can coexist without ambiguous operation routes.
|
||||
- Provider choice is reproducible, visible, and stable for a target.
|
||||
- Installation of another provider cannot silently change analysis.
|
||||
- Snapshot and Evidence identities become analysis-configuration aware.
|
||||
- Provider adapters retain ownership of their option schemas and diagnostics.
|
||||
- Auxiliary capabilities remain usable when no deep engine is installed.
|
||||
|
||||
Costs and tradeoffs:
|
||||
|
||||
- Multi-provider hosts must choose when more than one candidate is usable.
|
||||
- Snapshots must be recaptured when their provider or profile binding changes.
|
||||
- Providers must expose deterministic availability, support, version, and
|
||||
profile resolution before they can be admitted.
|
||||
- Status and error contracts gain additive structured metadata.
|
||||
- Hopper needs an adapter-owned version/profile implementation before profile-
|
||||
exact snapshot persistence is complete.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
### Let `CompositeProvider` choose the first duplicate route
|
||||
|
||||
Array order is not a product policy, becomes fragile during wiring changes, and
|
||||
hides ambiguity from callers.
|
||||
|
||||
### Race providers and keep the first healthy response
|
||||
|
||||
Machine load would determine provenance and results. It also starts resources
|
||||
the operator did not select and makes cancellation and cleanup harder.
|
||||
|
||||
### Fall back after a provider error
|
||||
|
||||
Mid-session fallback can combine incompatible engine state and evidence under
|
||||
one apparent target. Explicit close or switch is required instead.
|
||||
|
||||
### Add provider-prefixed tools
|
||||
|
||||
Parallel Hopper and Ghidra tool families would leak adapters into the product
|
||||
contract and prevent shared workflows from using capability metadata.
|
||||
|
||||
### Store raw provider options in `BinaryTarget`
|
||||
|
||||
That repeats the current Hopper coupling and makes every generic consumer learn
|
||||
provider schemas. Only canonical profile commitments cross the adapter boundary.
|
||||
|
||||
### Treat matching target bytes and provider ID as sufficient for snapshots
|
||||
|
||||
Provider versions, analyzer defaults, loaders, languages, and compiler models
|
||||
can change results without changing either value.
|
||||
|
||||
### Silently upgrade snapshot v1
|
||||
|
||||
The missing engine version and analysis settings cannot be reconstructed from
|
||||
Hopper loader arguments. A fabricated profile would turn unknown provenance
|
||||
into false certainty.
|
||||
|
||||
## Implementation progress
|
||||
|
||||
The first six implementation stages are shipped:
|
||||
|
||||
1. Provider-specific target state was replaced by generic profile and snapshot
|
||||
v2 commitments while preserving Hopper execution behavior.
|
||||
2. The registry, selectors, candidate status, binding lifecycle, and
|
||||
overlapping-provider production seams were added.
|
||||
3. Proven process-lifecycle primitives were extracted for shared use.
|
||||
4. Ghidra availability, target/profile resolution, doctor checks, and a private
|
||||
read-only headless session were admitted without declaring operations.
|
||||
5. Program identity, procedures, strings, symbols, memory blocks,
|
||||
address/name resolution, containing-procedure resolution, and search
|
||||
were admitted with exact wire schemas and real debug/stripped ELF
|
||||
conformance.
|
||||
6. Function metadata, persistent decompilation, assembly, resolved
|
||||
callers/callees, typed references, xrefs, CFG, and complete function dossiers
|
||||
were admitted. A serial per-Program queue keeps API access ordered without a
|
||||
fixed queue length or per-operation deadline. Real conformance covers
|
||||
x86-64 and AArch64 ELF, PE, Mach-O, stripped symbols, targetless indirect
|
||||
calls, cancellation, and cleanup.
|
||||
|
||||
Future stages may deepen format and indirect-flow coverage, but must continue
|
||||
to compare normalized semantics rather than provider-specific pseudocode or
|
||||
assembly text. Linux and macOS now admit atomic function-name and entry-comment
|
||||
edits in the ephemeral database, with readback, rollback, and cache invalidation.
|
||||
Executable bytes remain unchanged. Windows P0 retains its read-only boundary;
|
||||
GUI and other unimplemented mutations remain unavailable.
|
||||
@@ -0,0 +1,551 @@
|
||||
# ADR-0002: Controlled JavaScript replay authority and sandbox policy
|
||||
|
||||
- Status: Superseded; the controlled JavaScript replay tool was removed
|
||||
- Date: 2026-07-16
|
||||
- Implementation status: The implementation was removed. The design below is
|
||||
retained as historical context and does not describe a supported REA tool.
|
||||
|
||||
> Historical design note: The tool, sandbox backend, and plan/execute workflow
|
||||
> described here are no longer part of REA. For runtime behavior, run probes
|
||||
> directly against the application and use the available browser, Electron, or
|
||||
> process capture workflows.
|
||||
|
||||
## Context
|
||||
|
||||
REA can reconstruct JavaScript and Electron artifacts, observe an existing
|
||||
browser or Electron page through passive CDP, capture an explicitly approved
|
||||
process scenario, and correlate those observations without executing bundle
|
||||
code in the REA server. The next research workflow needs to execute selected,
|
||||
extracted JavaScript modules with controlled inputs and deterministic stubs so
|
||||
that parsers, sanitizers, serializers, and similar pure logic can be compared
|
||||
across versions.
|
||||
|
||||
That workflow crosses a qualitatively different boundary. Source recovered
|
||||
from an application is untrusted code. It can intentionally or accidentally
|
||||
read files, contact services, spawn processes, exhaust resources, corrupt the
|
||||
analysis process, forge protocol output, or interact with another same-user
|
||||
process. A JavaScript realm or Node.js `vm` context is useful for constructing
|
||||
an API surface, but it is not a security boundary. Node.js also documents its
|
||||
permission model as protection against accidental access by trusted code, not
|
||||
as containment for malicious code.
|
||||
|
||||
Existing authorities cannot be widened to cover this behavior:
|
||||
|
||||
- `browser_observe` and `electron_observe` authorize passive attachment to an
|
||||
already-running, operator-owned target. They do not authorize evaluation,
|
||||
navigation, input, or target lifecycle changes.
|
||||
- `process_capture` authorizes one declared host process scenario and explicitly
|
||||
states that it is not a sandbox. It cannot silently become permission to run
|
||||
recovered code.
|
||||
- static artifact reads authorize inspection of bytes, not execution of those
|
||||
bytes.
|
||||
- the Evidence authority `controlled-replay` already describes observations
|
||||
from an approved process experiment. Reusing that epistemic authority is
|
||||
correct, but reusing the process permission capability is not.
|
||||
|
||||
The implementation following this ADR must fail closed on hosts where REA
|
||||
cannot establish the promised operating-system boundary. It must not copy the
|
||||
research `vm.runInContext` scripts into the production server and call that
|
||||
isolation.
|
||||
|
||||
## Decision drivers
|
||||
|
||||
1. Passive observation must never imply authority to execute application code.
|
||||
2. The exact code, inputs, stubs, runtime, sandbox policy, and limits approved
|
||||
by the operator must be the ones executed.
|
||||
3. Recovered code must not run in the MCP server, CLI adapter, or any provider
|
||||
process.
|
||||
4. A sandbox failure must make the capability unavailable, not select a weaker
|
||||
execution mode.
|
||||
5. The default policy must provide no host network and no writable host path.
|
||||
6. Time, memory, process count, temporary storage, protocol, and output must be
|
||||
bounded independently.
|
||||
7. Results must distinguish an observed return, an observed exception or
|
||||
termination, a derived comparison, and an unresolved application-level
|
||||
claim.
|
||||
8. A reproducer must commit enough provenance to detect every relevant change
|
||||
without committing proprietary application source to the repository.
|
||||
9. CLI and MCP must expose the same plan, approval, execution, cancellation,
|
||||
failure, and Evidence semantics.
|
||||
|
||||
## Terminology
|
||||
|
||||
- **Passive observation**: bounded reads from an existing browser or Electron
|
||||
target without evaluating code, driving input, navigating, or controlling
|
||||
its lifecycle.
|
||||
- **Controlled replay**: execution of operator-selected code in a declared,
|
||||
isolated environment with committed inputs and limits. This is an Evidence
|
||||
authority, not a claim that the real application behaved identically.
|
||||
- **Extracted-module replay**: the only admitted JavaScript replay mode in v1.
|
||||
REA evaluates a finite manifest of module bytes and explicit stubs without
|
||||
launching the application.
|
||||
- **Application interaction**: launching or driving the original application,
|
||||
browser, renderer, or Electron main process. This is not extracted-module
|
||||
replay and is not authorized by this ADR.
|
||||
- **Replay plan**: the canonical, content-addressed description of one proposed
|
||||
run before execution.
|
||||
- **Replay worker**: the disposable Node.js process inside the OS sandbox. It
|
||||
is never the REA server process.
|
||||
- **Sandbox backend**: the OS mechanisms and adapter that establish namespaces,
|
||||
resource controls, filesystem visibility, and owned cleanup.
|
||||
|
||||
## Threat model
|
||||
|
||||
### Protected assets
|
||||
|
||||
The boundary protects:
|
||||
|
||||
- the REA MCP/CLI process, its memory, credentials, sockets, and Evidence
|
||||
ledger;
|
||||
- user files outside the approved source reads and all host files from writes;
|
||||
- other same-user processes and their control interfaces;
|
||||
- host network interfaces, loopback services, Unix sockets, and external
|
||||
services;
|
||||
- process, memory, CPU, temporary-storage, output, and protocol capacity;
|
||||
- the integrity of the replay plan, result, Evidence provenance, and exported
|
||||
reproducer; and
|
||||
- proprietary module bytes, which remain local and are never repository
|
||||
fixtures.
|
||||
|
||||
### Adversaries and failure sources
|
||||
|
||||
REA treats all of the following as untrusted:
|
||||
|
||||
- recovered module source, including transitive modules and source maps;
|
||||
- caller-provided JavaScript stubs;
|
||||
- structured test inputs and generated fuzz cases;
|
||||
- returned values, exceptions, console data, and worker protocol messages;
|
||||
- source paths, symbolic links, and files that can change between planning and
|
||||
execution; and
|
||||
- a worker that crashes, hangs, forks, allocates aggressively, or exits while
|
||||
descendants remain alive.
|
||||
|
||||
An operator-controlled sandbox executable, kernel, Node.js runtime, and REA
|
||||
installation are trusted computing base. A hostile host administrator, kernel
|
||||
or sandbox vulnerability, hardware side channel, and denial of service by an
|
||||
already-compromised same-user process are out of scope. These exclusions do not
|
||||
permit REA to omit the per-run resource controls in this decision.
|
||||
|
||||
### Required mitigations
|
||||
|
||||
The implementation must address:
|
||||
|
||||
| Threat | Required control |
|
||||
| -------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
||||
| Server-process compromise | Separate worker process inside the OS sandbox; no in-process fallback |
|
||||
| Host file reads | Empty sandbox root; only a minimal read-only runtime closure; module bytes enter over a private bounded channel |
|
||||
| Host file writes | No writable bind mounts; size-limited private tmpfs only |
|
||||
| Network access | Separate network namespace with no host or external connectivity; no allowlist mode in v1 |
|
||||
| Host process control | PID/user/IPC/UTS isolation, no inherited control sockets, new session, dropped capabilities |
|
||||
| Kernel escape/inspection surface | Architecture-checked seccomp denial policy in addition to namespaces, capabilities, and cgroups |
|
||||
| Fork, memory, or CPU exhaustion | Per-run cgroup v2 task, memory, swap, and CPU limits plus an outer wall-clock deadline |
|
||||
| Output/protocol exhaustion | Independent byte/message/depth limits enforced by the parent before parsing or retaining output |
|
||||
| Symlink and file-change races | Canonical authorization, descriptor-backed reads, digesting, staging, and exact plan revalidation |
|
||||
| Undeclared dependencies | Closed module manifest; exact imports only; missing/dynamic imports fail rather than use host resolution |
|
||||
| Nondeterministic globals | Versioned clock, PRNG, locale, timezone, platform, and API stubs; undeclared globals are unavailable |
|
||||
| Orphan descendants | Owned PID namespace and cgroup kill followed by an observable empty-cgroup cleanup check |
|
||||
| Forged result claims | Framed worker protocol, strict schemas, parent-computed commitments, and bounded stderr kept separate |
|
||||
| Secret leakage | Empty environment, no credential inheritance, bounded local results, and existing genuine-secret redaction |
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Add a distinct `javascript_replay` permission capability
|
||||
|
||||
JavaScript replay receives a new permission capability named
|
||||
`javascript_replay`. It does not inherit, imply, or satisfy
|
||||
`process_capture`, `browser_observe`, `electron_observe`, artifact-read, or
|
||||
workspace permissions. Grants for any of those capabilities do not satisfy a
|
||||
JavaScript replay request, and a JavaScript replay grant does not authorize
|
||||
them.
|
||||
|
||||
The administrator ceiling is disabled by default. Its eventual configuration
|
||||
must commit:
|
||||
|
||||
- canonical roots from which module and optional stub files may be read;
|
||||
- the exact Node.js and sandbox-backend executables;
|
||||
- filesystem mount authority needed only to construct the private sandbox;
|
||||
- network mode `none`; and
|
||||
- no inherited environment names.
|
||||
|
||||
Configuration establishes only the maximum authority. Every execution also
|
||||
requires an active permission grant and literal per-call approval bound to the
|
||||
exact replay plan. Project, session, and administrator grants may satisfy the
|
||||
permission-policy layer, but none replace that per-run approval.
|
||||
|
||||
Reproducer export is a separate host write after the sandbox has stopped. This
|
||||
ADR's root-scoped `evidence_write` authority was removed; current exports use
|
||||
the path supplied by the caller, while replay still waits for complete sandbox
|
||||
cleanup before writing.
|
||||
|
||||
### 2. Admit extracted modules only
|
||||
|
||||
The v1 mode executes a finite, closed manifest of JavaScript modules. The
|
||||
manifest identifies one entry module and every admitted dependency by a stable
|
||||
sandbox alias and SHA-256 digest. Imports resolve only to another manifest
|
||||
entry or to an explicitly declared, versioned stub. Bare package lookup,
|
||||
ambient `node_modules`, import maps from the host, network imports, native
|
||||
addons, WASI, FFI, child processes, workers, inspector activation, and implicit
|
||||
dynamic import are unavailable.
|
||||
|
||||
Modules and stubs are read and hashed by the authorized parent, then copied as
|
||||
bytes through a bounded private channel. Original host paths are not mounted
|
||||
into the sandbox. The worker cannot reopen the operator's source path.
|
||||
|
||||
Source maps may supply locations, but source-map source code is not executed
|
||||
unless its bytes are separately present in the approved manifest. A source-map
|
||||
name or Webpack module ordinal is never execution authority.
|
||||
|
||||
REA does not launch, attach to, navigate, click, type into, or terminate the
|
||||
user's application under this capability. A full application process remains
|
||||
an independently approved Process Capture experiment; browser and Electron
|
||||
targets remain passive. Any future mode that drives an application or permits
|
||||
host/allowlisted network requires a new ADR and a distinct permission scope.
|
||||
|
||||
### 3. Use a two-phase, content-bound approval flow
|
||||
|
||||
One public replay surface supports a plan phase and an execute phase with the
|
||||
same schema through CLI and MCP.
|
||||
|
||||
The plan phase:
|
||||
|
||||
1. validates all caller input and hard limits before file access;
|
||||
2. obtains authority to read every selected module or stub path;
|
||||
3. canonicalizes and opens each file without following an unapproved identity;
|
||||
4. reads bounded bytes, computes digests, and closes the source descriptors;
|
||||
5. resolves and probes the exact runtime and sandbox backend without running
|
||||
application code;
|
||||
6. normalizes stubs, inputs, generator settings, and limits; and
|
||||
7. returns the complete proposed effects and a canonical `plan_digest`.
|
||||
|
||||
The plan must disclose at least:
|
||||
|
||||
- module aliases, canonical local paths, byte counts, and SHA-256 digests;
|
||||
- input and stub identities, provenance, and digests;
|
||||
- runtime and sandbox executable paths, versions, and executable digests;
|
||||
- sandbox policy version and digest;
|
||||
- network `none`, visible read-only runtime paths, and private writable tmpfs
|
||||
size;
|
||||
- time, CPU, memory, swap, task, temporary-storage, input, output, protocol,
|
||||
case-count, and serialization limits;
|
||||
- deterministic clock, PRNG seed/algorithm, locale, timezone, and platform
|
||||
projection; and
|
||||
- expected Evidence and cleanup behavior.
|
||||
|
||||
Execution requires the exact `plan_digest`. REA
|
||||
rebuilds the plan immediately before admission. Any changed file, runtime,
|
||||
sandbox executable, policy, stub, input, limit, or environment commitment
|
||||
returns `plan_stale` before application code starts. Approval is not transferable
|
||||
to a different plan and cannot be represented by a bare Boolean alone.
|
||||
|
||||
Interactive CLI may print the plan and ask for confirmation. Non-interactive
|
||||
CLI and MCP must perform the same explicit plan/execute exchange; they may not
|
||||
auto-approve because the caller requested JSON output or because permission
|
||||
elicitation is supported.
|
||||
|
||||
### 4. Require an OS sandbox; fail closed elsewhere
|
||||
|
||||
The first production backend is Linux-only. It must combine:
|
||||
|
||||
- an empty mount namespace rooted in private tmpfs;
|
||||
- a new user namespace with all capabilities dropped and further nested user
|
||||
namespaces disabled;
|
||||
- new PID, network, IPC, UTS, and cgroup namespaces;
|
||||
- a new terminal session and parent-death behavior;
|
||||
- no host D-Bus, agent, browser, Docker, Wayland/X11, SSH, GPG, audio, device,
|
||||
or application sockets;
|
||||
- a minimal, read-only runtime closure rather than `/`, the user's home,
|
||||
working directory, or host `/tmp`;
|
||||
- a fresh `/proc` for the sandbox PID namespace and a minimal private `/dev`;
|
||||
- a parent-generated, architecture-checked seccomp policy that denies at least
|
||||
namespace or mount changes, `ptrace` and cross-process memory inspection,
|
||||
BPF/perf access, kernel keyring and module control, reboot/swap controls, and
|
||||
the `TIOCSTI` terminal injection path;
|
||||
- cgroup v2 `memory.max`, `memory.swap.max=0`, `pids.max`, and bounded CPU
|
||||
controls owned for the one run; and
|
||||
- an outer REA supervisor that can kill the complete cgroup and verify it is
|
||||
empty.
|
||||
|
||||
The seccomp program is passed by descriptor, has a version and digest committed
|
||||
to the replay plan, validates the syscall architecture before the syscall
|
||||
number, and never permits `ptrace`-based mediation. Seccomp reduces exposed
|
||||
kernel surface; it is not described as a complete sandbox by itself.
|
||||
|
||||
The reference backend for the implementation PR is Bubblewrap for namespace,
|
||||
mount, and seccomp construction plus a delegated cgroup v2 manager for resource
|
||||
enforcement. REA must feature-probe the actual required operations; version
|
||||
text alone is insufficient. A setuid Bubblewrap installation, a backend that
|
||||
silently skips a requested namespace or filter, unavailable user namespaces,
|
||||
missing cgroup delegation, or inability to prove cleanup makes the capability
|
||||
unavailable. REA setup must not install or upgrade Bubblewrap, systemd, Node.js,
|
||||
or kernel components.
|
||||
|
||||
macOS, Windows, Linux hosts without every required feature, containers that
|
||||
block the required namespaces, and alternative backends are unavailable in
|
||||
v1. They must return precise diagnostics and remediation, not a process-only or
|
||||
`vm`-only degraded mode.
|
||||
|
||||
Node.js runs with its permission model enabled and without grants for network,
|
||||
filesystem, child processes, workers, native addons, WASI, FFI, or inspector.
|
||||
That layer is defense in depth. It is never cited as the OS security boundary,
|
||||
because Node.js explicitly does not promise containment of malicious code.
|
||||
|
||||
### 5. Make the worker disposable and deterministic by construction
|
||||
|
||||
Each approved run receives a new sandbox, runtime root, worker process, module
|
||||
loader, and deterministic state. No module cache, realm, timer, file, global,
|
||||
or child survives into another run. Within a multi-case run, every case gets a
|
||||
fresh realm and fresh module instances; case order and generator state are
|
||||
committed.
|
||||
|
||||
The worker exposes a small versioned set of language primitives and explicitly
|
||||
selected stubs. It does not expose `process`, `require`, host `console`, timers,
|
||||
`fetch`, sockets, filesystem APIs, `Buffer`, worker APIs, native bindings, or
|
||||
ambient environment variables. Date/time, randomness, locale, timezone, and
|
||||
platform values come only from committed deterministic providers. An
|
||||
unsupported global or import produces an observed exception; REA does not
|
||||
substitute a convenient host implementation.
|
||||
|
||||
Caller-provided stubs are executable untrusted code and receive the same
|
||||
sandbox treatment as recovered modules. Built-in stubs have stable IDs,
|
||||
versions, canonical source digests, and documented semantics. No undeclared
|
||||
fallback stub is allowed.
|
||||
|
||||
Inputs are bounded canonical JSON. Results are projected without invoking
|
||||
getters, proxies, `toJSON`, or arbitrary coercion: only admitted primitives,
|
||||
arrays, and plain data properties are serialized. Cycles, accessors, functions,
|
||||
symbols, unsupported prototypes, excessive depth, excessive nodes, and
|
||||
oversized values become explicit serialization outcomes. Worker stdout is the
|
||||
framed protocol only; bounded diagnostic stderr and bounded, opt-in console
|
||||
observations remain separate.
|
||||
|
||||
### 6. Enforce independent resource ceilings
|
||||
|
||||
Every limit has a conservative product default. The v1 plan and result commit:
|
||||
|
||||
- an outer wall-clock deadline for the complete disposable run;
|
||||
- cgroup memory, swap, task, and CPU ceilings;
|
||||
- Node/V8 heap ceiling as defense in depth;
|
||||
- private tmpfs bytes;
|
||||
- serialized worker-protocol bytes, aggregate module bytes, and case-input
|
||||
bytes; preset-generated cases also have a fixed preflight count ceiling;
|
||||
- stdout, stderr, exception, stack, and aggregate retained-output bytes; and
|
||||
- structured-result depth and node count.
|
||||
|
||||
The parent enforces protocol and output limits while bytes arrive, before JSON
|
||||
parse or Evidence retention. The worker cannot request wider limits. A limit
|
||||
hit terminates the complete run cgroup and is an observed termination, not a
|
||||
partial successful return.
|
||||
|
||||
### 7. Preserve exact provenance and the right Evidence authority
|
||||
|
||||
A run that reaches worker admission produces Evidence with:
|
||||
|
||||
- provider `rea-javascript-replay` and a concrete implementation version;
|
||||
- operation `run_controlled_replay`;
|
||||
- authority `controlled-replay`;
|
||||
- confidence `observed` for the exact returned value, thrown exception,
|
||||
sandbox denial, limit termination, signal, exit, or crash that REA directly
|
||||
observed;
|
||||
- environment isolation `container`, the host platform/architecture, runtime
|
||||
identity, sandbox-backend identity, and sandbox-policy digest;
|
||||
- the replay plan digest, module and stub digests, canonical input or generated
|
||||
case digests, deterministic providers, and every effective limit;
|
||||
- bounded raw input, output, and exception data when retained, plus their
|
||||
canonical digests and truncation state;
|
||||
- start/admission/termination/cleanup states; and
|
||||
- explicit limitations for incomplete output, nondeterminism not checked,
|
||||
sandbox diagnostics, or cleanup that could not be proven.
|
||||
|
||||
The host module path remains an actionable local diagnostic and Evidence
|
||||
subject location, while the worker sees only a stable alias. Module source is
|
||||
not embedded in ordinary Evidence. A separately approved reproducer may copy
|
||||
module/stub bytes or may retain path-plus-digest references; its manifest says
|
||||
which mode was used. Repository tests use source-owned fixtures, and real-app
|
||||
verification stores only digests and expected structural facts.
|
||||
|
||||
Derived differential results use confidence `derived` and link every source
|
||||
run Evidence ID. A repeat with the same commitment can establish
|
||||
`repeat-confirmed` for those runs; a single deterministic plan is only
|
||||
`committed-not-confirmed`. Neither state proves behavior in the original
|
||||
application.
|
||||
|
||||
When replay observations are added to the JavaScript Application Graph, their
|
||||
graph-fact authority is `controlled-replay`. A claim that the real renderer,
|
||||
preload, main process, or remote service behaves the same remains inferred or
|
||||
unknown until compatible runtime authority exists. Static inference, passive
|
||||
CDP observation, controlled replay, and real application interaction are never
|
||||
promoted into one another.
|
||||
|
||||
### 8. Treat post-admission failures as observations
|
||||
|
||||
Pre-admission failures return a typed error and create no claim that module code
|
||||
ran. Once the worker is admitted, REA retains a bounded execution record even
|
||||
when the requested function does not return normally.
|
||||
|
||||
| Condition | Caller result and Evidence semantics |
|
||||
| ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Disabled capability or missing grant | `permission_required`/`capability_unavailable`; no worker admission and no replay observation |
|
||||
| Sandbox/runtime probe failure | `capability_unavailable` with exact failed feature; no module execution |
|
||||
| Plan digest mismatch or changed source | `plan_stale`; no module execution |
|
||||
| Sandbox setup fails before worker admission | `execution_failure`; no controlled-replay claim about the module |
|
||||
| Module returns admitted data | Observed return value with complete or explicitly limited coverage |
|
||||
| Module throws or result serialization rejects | Successful replay observation with outcome `exception` or `serialization_error`; application claim unknown |
|
||||
| Node permission or sandbox denies an operation | Observed denial/termination when attributable; no claim that all attempted side effects were enumerated |
|
||||
| Timeout, memory, CPU, task, tmpfs, or output hit | Observed limit termination; requested functional result unknown |
|
||||
| Signal, runtime crash, or malformed protocol | Observed termination plus bounded diagnostics; functional result unknown |
|
||||
| Caller cancellation after admission | Kill full cgroup, record cancellation/cleanup if possible, and project the public cancellation contract |
|
||||
| Cleanup proves an empty cgroup and removed root | `cleanup: complete` |
|
||||
| Cleanup cannot prove emptiness or removal | `cleanup_incomplete`, name every residual resource, and do not report success |
|
||||
|
||||
An exception is data, not an adapter failure. A sandbox setup error is not
|
||||
evidence that the module crashed. A timeout is not evidence that the function
|
||||
would never return. An output limit is not evidence that the truncated prefix
|
||||
equals the full result. These distinctions are part of the public contract.
|
||||
|
||||
Cancellation and cleanup must be idempotent. REA first stops admission and
|
||||
input, kills the owned cgroup, waits for an empty-cgroup observation, removes
|
||||
the private runtime root, closes protocol descriptors, and releases timers.
|
||||
Failure at one cleanup step does not skip the remaining steps.
|
||||
|
||||
### 9. Keep replay records reproducible without overstating determinism
|
||||
|
||||
The canonical replay manifest commits:
|
||||
|
||||
```text
|
||||
plan structure and policy identity
|
||||
REA/provider/runtime/sandbox executable identities and digests
|
||||
module aliases, dependency graph, bytes digests, and source references
|
||||
stub IDs, versions, source digests, and caller-provided stub digests
|
||||
canonical inputs or generator algorithm, seed, case order, and case digests
|
||||
clock, PRNG, timezone, locale, platform, and exposed-global commitments
|
||||
all resource and serialization limits
|
||||
network and filesystem projections
|
||||
expected entry export and invocation convention
|
||||
```
|
||||
|
||||
The result manifest adds canonical output/exception/termination digests,
|
||||
coverage, dropped-data counts, cleanup state, and source Evidence links.
|
||||
Canonical digests are computed by the trusted parent; worker-provided digests
|
||||
are ignored.
|
||||
|
||||
Reproduction requires exact manifest compatibility. A runtime, sandbox policy,
|
||||
built-in stub, module byte, generated case, limit, architecture, or platform
|
||||
change is a different experiment. Cross-environment comparison may still be a
|
||||
derived investigation, but it is never an exact replay.
|
||||
|
||||
### 10. Expose availability and policy truth before execution
|
||||
|
||||
`rea doctor`, capability inventory, and the planning response must report the
|
||||
JavaScript replay capability without executing application code. Diagnostics
|
||||
name the exact missing executable, unsupported host, failed namespace feature,
|
||||
cgroup delegation failure, setuid rejection, runtime mismatch, or cleanup-probe
|
||||
failure. Genuine secrets remain redacted; local executable paths, versions,
|
||||
digests, and kernel feature failures remain visible.
|
||||
|
||||
The capability must be absent from executable tool routing until implemented.
|
||||
Accepting this ADR does not increase the current tool count and does not add a
|
||||
hidden experimental flag.
|
||||
|
||||
## Required implementation and verification gate
|
||||
|
||||
The implementation PR cannot merge until all of the following are true:
|
||||
|
||||
1. `javascript_replay` exists in configuration, permission policy, project
|
||||
grants, policy explanation, CLI, MCP, docs, and capability inventory with
|
||||
fail-closed defaults.
|
||||
2. Plan and execute phases have schema-identical CLI/MCP behavior and an exact
|
||||
plan-digest stale check.
|
||||
3. No replay-disabled static, browser, Electron, session, setup, doctor, or
|
||||
application workflow can reach module execution.
|
||||
4. The worker never imports into the REA server process and no adapter offers a
|
||||
`vm`-only fallback.
|
||||
5. The Linux backend feature-probes namespaces, seccomp installation and
|
||||
denial, delegated cgroup v2 limits, and the exact admitted architecture.
|
||||
Every plan content-addresses the worker, seccomp filter, Node executable,
|
||||
ELF loader, shared-library closure, and sandbox executables.
|
||||
6. Source-owned fixtures cover parser, sanitizer, clipboard, exception,
|
||||
ambient side-effect attempts, memory pressure, and busy loops. The worker
|
||||
exposes no filesystem, environment, network, process, worker, native-addon,
|
||||
timer, inspector, or ambient package-resolution API.
|
||||
7. Tests and the real verifier prove bounded protocol/diagnostics, timeout and
|
||||
OOM cgroup termination, and observed cgroup cleanup.
|
||||
8. Evidence tests cover observed return/exception/termination, derived
|
||||
comparison with source Evidence links, stale admission, cancellation
|
||||
projection, and explicit limitations on application-level claims.
|
||||
9. Package verification exercises the target-free public contract. The
|
||||
dedicated real-Linux workflow exercises the sandbox backend; environments
|
||||
without delegated user namespaces or cgroups report truthful unavailability
|
||||
rather than selecting a weaker backend.
|
||||
10. A real operator-local Notion-like parser or sanitizer benchmark can run
|
||||
without committing its source, and exports only approved local reproducer
|
||||
material.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
### Reuse `browser_observe`, `electron_observe`, or `process_capture`
|
||||
|
||||
Rejected because those grants authorize different effects. Reuse would turn a
|
||||
passive or unsandboxed process permission into recovered-code execution.
|
||||
|
||||
### Execute in the MCP server with `vm.runInContext`
|
||||
|
||||
Rejected because a VM context is not a security boundary and a crash, resource
|
||||
attack, or escape would compromise the long-lived session and its credentials.
|
||||
|
||||
### Use only the Node.js permission model
|
||||
|
||||
Rejected because Node.js documents it as a seat belt for trusted code and
|
||||
explicitly does not guarantee containment of malicious code. It remains a
|
||||
defense-in-depth layer inside the OS sandbox.
|
||||
|
||||
### Ship a process-only fallback on unsupported hosts
|
||||
|
||||
Rejected because it changes the security claim while preserving the same tool
|
||||
name. Unsupported hosts return unavailable.
|
||||
|
||||
### Permit network allowlists in v1
|
||||
|
||||
Rejected because DNS, redirects, proxies, credentials, service identity, and
|
||||
response nondeterminism would substantially expand the threat and Evidence
|
||||
model. The v1 namespace has no host or external network.
|
||||
|
||||
### Mount the approved project root read-only
|
||||
|
||||
Rejected because one approved module does not imply authority to disclose
|
||||
every file in its project. The parent stages only exact, digested bytes.
|
||||
|
||||
### Treat exceptions, crashes, or limit hits as ordinary tool errors only
|
||||
|
||||
Rejected because that discards the principal behavioral observation and makes
|
||||
negative results impossible to reproduce. Post-admission termination is
|
||||
retained with explicit functional uncertainty.
|
||||
|
||||
## Consequences
|
||||
|
||||
- JavaScript replay is Linux-only at first and may be unavailable on containers
|
||||
or distributions that disable user namespaces or cgroup delegation.
|
||||
- The implementation requires more host probing and lifecycle code than a
|
||||
simple Node worker, but the public security claim remains stable.
|
||||
- Plans require two calls in non-interactive environments and become stale when
|
||||
any committed input changes.
|
||||
- Network-dependent modules and full application interactions are excluded;
|
||||
their absence is explicit rather than emulated implicitly.
|
||||
- Results are reproducible for the committed isolated experiment, not proof of
|
||||
production behavior.
|
||||
- Adding another sandbox backend, network mode, or application-interaction mode
|
||||
requires a separate reviewed decision and conformance suite.
|
||||
|
||||
## References
|
||||
|
||||
- [Node.js permissions](https://nodejs.org/api/permissions.html) documents that
|
||||
the permission model does not protect against malicious code and describes
|
||||
its known filesystem and cross-process limitations.
|
||||
- [Bubblewrap README](https://github.com/containers/bubblewrap/blob/main/README.md)
|
||||
describes its namespace construction and makes clear that the caller's
|
||||
arguments, not Bubblewrap alone, define the sandbox security model.
|
||||
- [Linux seccomp filter documentation](https://www.kernel.org/doc/html/latest/userspace-api/seccomp_filter.html)
|
||||
explains that syscall filtering reduces kernel surface but is not a complete
|
||||
sandbox by itself.
|
||||
- [ADR-0001](0001-provider-selection-and-analysis-profiles.md) establishes the
|
||||
existing rule that provider selection, Evidence provenance, and capability
|
||||
unavailability remain explicit and never silently fall back.
|
||||
@@ -0,0 +1,452 @@
|
||||
# ADR-0003: Managed-code evidence and provider boundary
|
||||
|
||||
> Historical design record: The non-executing runtime-correlation planning
|
||||
> tool described in earlier revisions was removed. This ADR's current
|
||||
> implementation status below covers only the static managed tools that ship.
|
||||
|
||||
- Status: Superseded for runtime planning; static evidence boundary remains in force
|
||||
- Date: 2026-07-16
|
||||
- Implementation status: Read-only PE/CLI triage and exact identity are shipped
|
||||
through `inspect_managed_artifact` / `rea inspect-managed-artifact`. Bounded
|
||||
metadata, signatures, method bodies, raw CIL hashes, limited
|
||||
decoded-instruction-tuple v1 hashes, exception regions, call edges, and
|
||||
field-access anchors are shipped through `inspect_managed_members`; `rea
|
||||
inspect-managed-members` provides CLI parity. Static member comparison and build-local
|
||||
token remapping are shipped through `compare_managed_members` /
|
||||
`rea compare-managed-members`. Declaration-only managed/native boundary
|
||||
inventory for ModuleRef, ImplMap/PInvoke, ReadyToRun indicators, and non-IL
|
||||
method flags is shipped through `inspect_managed_native_boundaries` /
|
||||
`rea inspect-managed-native-boundaries`. Managed/native export or function
|
||||
Evidence matching is shipped through `verify_managed_native_boundaries` /
|
||||
`rea verify-managed-native-boundaries`; it verifies only supplied P/Invoke
|
||||
export/function observations and does not map managed tokens to native
|
||||
addresses. Decompiler reconstruction import is
|
||||
shipped through `import_managed_reconstruction` /
|
||||
`rea import-managed-reconstruction`; it records user-supplied C#/IL/pseudocode
|
||||
as analyst inference only after exact static member locks match. Managed
|
||||
inspection does not execute assemblies or observe CLR internals. Managed static graph
|
||||
projection is shipped through
|
||||
`project_managed_application_graph` /
|
||||
`rea project-managed-application-graph`; it projects authenticated managed
|
||||
Evidence into the application graph without changing managed tokens into
|
||||
native addresses. Native-body bridge mapping and an actual runtime executor
|
||||
remain future contracts.
|
||||
|
||||
## Context
|
||||
|
||||
Professional C# reverse engineering is not one decompiler operation. A file
|
||||
described as “.NET” can be ordinary Common Intermediate Language (CIL),
|
||||
ReadyToRun code containing both CIL and native implementations, a single-file
|
||||
deployment, C++/CLI mixed mode, Unity Mono, Unity IL2CPP, or NativeAOT. Those
|
||||
forms have different sources of truth and different native-analysis needs.
|
||||
Choosing a decompiler before classifying the artifact can therefore produce a
|
||||
plausible but false model.
|
||||
|
||||
The durable static facts in an ordinary managed PE are its PE/CLI headers,
|
||||
metadata tables, signatures, method bodies, exception regions, and CIL. A C#
|
||||
decompiler reconstructs source-like control flow and language constructs from
|
||||
those facts. That reconstruction is valuable, but it is not the original
|
||||
source and cannot replace the underlying metadata and CIL evidence. Obfuscation
|
||||
makes names especially weak while tokens remain coordinates in only one build.
|
||||
|
||||
REA already separates one selected deep native provider from disjoint auxiliary
|
||||
providers under [ADR-0001](0001-provider-selection-and-analysis-profiles.md).
|
||||
It also separates static evidence, passive observation, process capture, and
|
||||
interactive observation authorities. Managed analysis must fit those boundaries
|
||||
without requiring a CLR to inspect an artifact, loading target code into the
|
||||
REA process, or treating a managed token as a native address.
|
||||
|
||||
The design is informed by exact-build, reflection-free metadata extraction,
|
||||
hash-locked method manifests, independent behavior models, and managed/native
|
||||
boundary analysis performed against a local C# application. The application,
|
||||
its decompiler output, and its runtime observations are research inputs only;
|
||||
they are not distributable REA fixtures.
|
||||
|
||||
## Decision drivers
|
||||
|
||||
1. Classify the deployment before selecting an analysis route.
|
||||
2. Make metadata and CIL independently inspectable without a CLR or target
|
||||
execution.
|
||||
3. Preserve exact-build provenance while enabling structural comparison across
|
||||
builds whose MVIDs, tokens, names, and layout differ.
|
||||
4. Compose managed and native analysis without introducing another ambiguous
|
||||
deep-provider candidate.
|
||||
5. Keep decompiled C# useful but epistemically subordinate to canonical bytes.
|
||||
6. Analyze Linux, macOS, and Windows managed artifacts through the same local
|
||||
CLI/MCP contracts on REA's supported hosts.
|
||||
7. Keep setup additive and prevent managed analysis from installing .NET,
|
||||
ILSpy, an SDK, or another unrelated toolchain.
|
||||
8. Prove semantics with source-built conformance artifacts and compact,
|
||||
operator-local real-application checks.
|
||||
|
||||
## Terminology
|
||||
|
||||
- **Managed artifact**: a PE, bundle component, or related metadata artifact
|
||||
that carries a CLI header, CLI metadata, CIL, or another managed-runtime
|
||||
commitment.
|
||||
- **Classification vector**: independently observed container, runtime family,
|
||||
implementation form, and architecture. It is not one guessed label.
|
||||
- **Canonical static observation**: a bounded value decoded directly from the
|
||||
target bytes under a versioned parser contract.
|
||||
- **Reconstruction**: source-like output, including decompiled C#, derived from
|
||||
canonical observations.
|
||||
- **Structural inference**: a relationship or cross-version match derived from
|
||||
signatures, APIs, constants, control/data-flow shape, or other observations.
|
||||
- **Validation**: an independently checked proposition, for example a
|
||||
source-built oracle assertion or a behavior-model corpus result.
|
||||
- **Runtime observation**: a separately authorized fact obtained by attaching,
|
||||
loading, debugging, instrumenting, or otherwise executing code.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Classify before analyzing
|
||||
|
||||
Managed triage returns a classification vector and the observations supporting
|
||||
each dimension. Absence of a marker produces `unknown` or `not_applicable`, not
|
||||
an invented runtime family.
|
||||
|
||||
| Dimension | Initial values |
|
||||
| ------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| Container | PE file, single-file bundle, bundle component, Unity metadata pair, native image, malformed, unknown |
|
||||
| Runtime family | .NET Framework, modern .NET, Unity Mono, Unity IL2CPP, NativeAOT, mixed CLR/native, unknown |
|
||||
| Implementation form | CIL, CIL plus ReadyToRun, native only, C++/CLI mixed, metadata only, unsupported, unknown |
|
||||
| Architecture | PE machine plus CLI flags; component-specific architecture for bundles; conflicting or unknown when required |
|
||||
|
||||
Target-framework attributes, assembly references, metadata stream versions,
|
||||
bundle manifests, ReadyToRun directories, and native headers are observations.
|
||||
They may support a runtime-family inference, but no single string is treated as
|
||||
authoritative when the other bytes conflict.
|
||||
|
||||
Classification selects these routes:
|
||||
|
||||
| Observed form | Managed route | Native route and limitation |
|
||||
| --------------------------------- | ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Ordinary Framework or modern CIL | Full metadata, signatures, method bodies, and CIL | Optional only for declared interop boundaries |
|
||||
| Obfuscated CIL | Same canonical route; names receive no special trust | Optional for declared interop boundaries |
|
||||
| Unity Mono assembly | Same canonical route with Unity context retained | Optional for engine/native boundaries |
|
||||
| ReadyToRun | Metadata/CIL where present; implementation availability is per method | Selected Hopper/Ghidra analyzes native sections; CIL absence is reported per method |
|
||||
| C++/CLI mixed mode | Metadata/CIL for managed members and explicit mixed-mode boundary observations | Selected Hopper/Ghidra analyzes native bodies; REA never converts tokens directly to native addresses |
|
||||
| Single-file deployment | Inventory the bundle, then analyze each authenticated managed component | Selected Hopper/Ghidra analyzes the host/native components; unsupported compression remains explicit |
|
||||
| Unity IL2CPP | Pair and authenticate available Unity metadata; do not claim canonical CIL | Selected Hopper/Ghidra analyzes the generated native binary; semantic recovery is degraded until supported |
|
||||
| NativeAOT | Report any independently present managed metadata, otherwise no CIL claim | Treat as a native target with NativeAOT provenance; managed source reconstruction is unavailable |
|
||||
| Malformed or unsupported artifact | Return bounded partial observations and typed unavailable/unknown scopes | Do not silently retry by loading or executing the file |
|
||||
|
||||
Classification is byte-driven and bounded. File extensions, product names,
|
||||
process names, and an operator's description are hints only.
|
||||
|
||||
### 2. Implement managed static analysis as a disjoint auxiliary provider
|
||||
|
||||
The initial producer is a REA-owned auxiliary provider with ID
|
||||
`rea-dotnet-static`. It owns a disjoint managed-artifact operation family. It is
|
||||
not registered as a Hopper/Ghidra deep-provider candidate because it does not
|
||||
compete for the same operation names and must be able to coexist with the one
|
||||
selected native provider.
|
||||
|
||||
```text
|
||||
one opened artifact/session
|
||||
|
|
||||
SessionProviderRouter
|
||||
/ \
|
||||
rea-dotnet-static auxiliary selected deep binding
|
||||
PE/CLI metadata and CIL Hopper or Ghidra
|
||||
\ /
|
||||
explicit managed/native links
|
||||
|
|
||||
Evidence and workflows
|
||||
```
|
||||
|
||||
An ordinary managed artifact can therefore remain unbound to a deep native
|
||||
provider while managed operations work. ReadyToRun, C++/CLI, IL2CPP, NativeAOT,
|
||||
and interop workflows may additionally require a caller-selected native
|
||||
binding. Native-provider failure never causes managed evidence to be relabeled,
|
||||
and managed-parser failure never causes an implicit native fallback.
|
||||
|
||||
Provider mechanics live under a provider directory such as `src/dotnet/`.
|
||||
Provider-neutral identities, evidence states, and bounded values may live in
|
||||
the domain/contracts layers. Application code composes results but does not
|
||||
parse PE headers, metadata tables, signatures, CIL, ReadyToRun headers, or
|
||||
provider-specific output.
|
||||
|
||||
### 3. Make the production static path byte-only and execution-free
|
||||
|
||||
`rea-dotnet-static` reads approved local bytes and implements the required
|
||||
PE/CLI metadata and CIL subset in TypeScript. It does not call any reflection
|
||||
API and does not require the .NET runtime. In particular, static inspection
|
||||
must never:
|
||||
|
||||
- call `Assembly.Load`, `Assembly.LoadFrom`, `ReflectionOnlyLoad`, or an
|
||||
equivalent runtime loader;
|
||||
- resolve target dependencies by executing target-controlled resolver code;
|
||||
- run module initializers, constructors, entry points, build tasks, plugins, or
|
||||
generated helpers;
|
||||
- start the target, attach to a process, contact a service, or inherit target
|
||||
configuration and credentials; or
|
||||
- interpret a decompiler, CLR, or native-provider diagnostic as an observation
|
||||
from the artifact bytes.
|
||||
|
||||
The parser uses checked offsets and additions, explicit endianness, bounded
|
||||
table/heap/body counts, cycle-safe traversal, and typed partial results. A bad
|
||||
row, signature, body, resource, or stream does not authorize an unbounded retry
|
||||
or a CLR load. Complete-coverage claims require every applicable bounded region
|
||||
to have been admitted; otherwise coverage is partial, unknown, or unavailable
|
||||
with exact offsets and reasons.
|
||||
|
||||
The caller selects the local input path directly. Local artifact
|
||||
paths, digests, MVIDs, tokens, malformed offsets, and analysis metadata are
|
||||
actionable diagnostics and are not secrets. Credentials, authorization data,
|
||||
and genuine secrets remain redacted.
|
||||
|
||||
### 4. Bind every finding to exact artifact and producer identity
|
||||
|
||||
Every managed result carries a commitment with at least:
|
||||
|
||||
```text
|
||||
artifact: canonical path + byte length + SHA-256
|
||||
component: container identity + component path/name + component SHA-256, when applicable
|
||||
classification: container + runtime family + implementation form + architecture
|
||||
assembly/module: simple name + version + culture + public-key identity + module name + MVID
|
||||
producer: provider ID + exact version + analysis-affecting parameters + profile digest
|
||||
```
|
||||
|
||||
An assembly-qualified name does not replace the artifact digest or MVID.
|
||||
Neither an MVID nor an Authenticode signature proves who authored the behavior.
|
||||
Bundle components retain both their container and component commitments so that
|
||||
bytes extracted from another host cannot be silently substituted.
|
||||
|
||||
Method observations add:
|
||||
|
||||
```text
|
||||
declaring type + normalized CLI signature
|
||||
build-local metadata token
|
||||
RVA/file extent when present
|
||||
raw CIL byte length + SHA-256 when the body is admitted
|
||||
decoded-instruction-tuple SHA-256 when CIL is valid
|
||||
separate max-stack, locals-token, and exception-region observations
|
||||
```
|
||||
|
||||
The shipped `il_sha256` commits only the raw CIL byte stream identified by the
|
||||
method header's `il_size`. It does not commit the method header, local-signature
|
||||
blob, alignment padding, or extra sections such as exception clauses. Those
|
||||
facts remain separate observations and the enclosing artifact SHA-256 binds
|
||||
them to the exact file. A digest over the complete method body is not shipped.
|
||||
|
||||
The caller-visible field remains named `normalized_il_sha256`, but its shipped
|
||||
v1 meaning is deliberately narrower than a canonical semantic CIL identity.
|
||||
For a completely decoded method, REA serializes the instruction-order array of
|
||||
`[opcode, operand_kind, operand]` tuples with JavaScript `JSON.stringify`,
|
||||
encodes that text as UTF-8, and reports its lowercase SHA-256. The tuple
|
||||
projection is defined here; the member result carries only the described
|
||||
normalized digest.
|
||||
|
||||
Operands retain the current decoder projection: metadata and user-string
|
||||
operands are build-local lowercase hexadecimal tokens, scalar branch operands
|
||||
are decimal target CIL byte offsets, and `switch` retains only its decimal case
|
||||
count. Constants and variable indexes are decimal strings; an operand-free
|
||||
instruction uses kind `none` and value `null`. Opcode names preserve distinct
|
||||
short forms and prefix instructions, but instruction offsets, branch target
|
||||
instruction identity, switch targets, resolved token identities, method
|
||||
headers, locals, and exception regions are not part of this digest. A partial
|
||||
or malformed decode has no `normalized_il_sha256`.
|
||||
|
||||
Consequently v1 is a deterministic decoded-tuple fingerprint, not proof of
|
||||
semantic equivalence. In particular, methods with the same opcodes and switch
|
||||
case counts can share it despite different switch targets, and token movement
|
||||
can change it without a semantic change. Cross-build comparison may use it as
|
||||
one unique-only structural tier while retaining artifact/MVID commitments and
|
||||
ambiguity; it must not describe the digest itself as token-remapped canonical
|
||||
CIL. Canonical operand resolution, complete control-flow operands,
|
||||
exception/local commitments, and an explicit algorithm version require a new
|
||||
normalized-CIL schema.
|
||||
|
||||
Metadata tokens are reported because they are precise coordinates inside one
|
||||
module. Cross-version identity never consists of a token alone. Matching tries
|
||||
exact CIL/signature identity, then an exact declared-type/name/signature tuple,
|
||||
then structural shape; a name by itself never selects a pair. Partial or
|
||||
malformed bodies leave body facets unknown after a signature tuple pairs the
|
||||
methods. Matches cite both build commitments, both local tokens, and all
|
||||
competing candidates. V1 cannot cite a separate matching-algorithm version
|
||||
because none is exposed.
|
||||
|
||||
### 5. Separate five evidence layers
|
||||
|
||||
Managed workflows retain these layers independently:
|
||||
|
||||
| Layer | Examples | Permitted claim |
|
||||
| ---------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------- |
|
||||
| Static observation | Metadata row, signature, CIL opcode, literal, exception region | “These bounded bytes decode to this value.” |
|
||||
| Reconstruction | Decompiled C#, async/iterator source shape, expression simplification | “This is one source-like representation of cited observations.” |
|
||||
| Structural inference | Likely caller, field flow, behavior slice, cross-build method match | “These cited structures support this relationship.” |
|
||||
| Independent validation | Synthetic oracle, corpus result, second parser agreement | “This declared proposition passed the named independent check.” |
|
||||
| Runtime observation | Exact-build attach/load/instrumentation event | “This separately authorized experiment observed this event.” |
|
||||
|
||||
Each layer has its own producer, authority, state, confidence, coverage, and
|
||||
limitations. A readable C# method is never emitted as original source or exact
|
||||
semantic proof. A validator does not upgrade unrelated claims. Runtime
|
||||
agreement does not retroactively turn a static inference into an observation.
|
||||
|
||||
Negative results are scope-qualified. For example, “no declared P/Invoke rows
|
||||
were found in the admitted metadata tables” does not prove the application has
|
||||
no native, dynamically resolved, generated, server-side, or protected path.
|
||||
|
||||
### 6. Admit ILSpy only as an optional reconstruction and conformance tool
|
||||
|
||||
REA's canonical static operations do not depend on ILSpy, a .NET SDK/runtime,
|
||||
dnlib, or Mono.Cecil. The following review was current on the decision date:
|
||||
|
||||
| Tool | Reviewed coordinate | License | Runtime/host observation | Decision |
|
||||
| ---------------------------- | ------------------- | ------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `System.Reflection.Metadata` | 10.0.10 | MIT | Library targets .NET Framework 4.6.2, .NET 8/9/10, and .NET Standard 2.0 | Pinned independent metadata oracle for conformance; not shipped or invoked by production parsing |
|
||||
| `ICSharpCode.Decompiler` | 10.1.1.8388 | MIT | Library targets .NET Standard 2.0; output is reconstruction | Pinned reconstruction/differential oracle candidate |
|
||||
| `ilspycmd` | 9.1.0.7988 | MIT | Cross-platform .NET 8 tool; newer 10.1.1.8388 requires .NET 10 | Initial optional BYO reconstruction coordinate because it matches the source research baseline |
|
||||
| dnlib | 4.5.0 | MIT | Cross-platform library; also exposes assembly-writing APIs | Differential test candidate only; its writable model is outside the production trust boundary |
|
||||
| Mono.Cecil | 0.11.6 | MIT | Cross-platform .NET Framework/.NET Standard library with read/write support | Differential test candidate only; not a production dependency |
|
||||
|
||||
Coordinates and package hashes used by real verification are checked into a
|
||||
source-owned lock manifest when that verifier is implemented. “Latest” is
|
||||
never a reproducibility coordinate. A version change requires conformance and
|
||||
an explicit profile/version change.
|
||||
|
||||
Artifact platform and REA host support are separate. The byte-only parser must
|
||||
inspect PE/CLI artifacts produced for Windows on a supported Linux or macOS REA
|
||||
host; that is not a claim that REA itself currently runs on Windows. Managed
|
||||
library/tool candidates above are cross-platform at their reviewed targets, but
|
||||
native composition remains limited by the selected Hopper/Ghidra host and
|
||||
format support. Real verification runs the canonical path on Linux and macOS
|
||||
and may run pinned oracle checks on Windows without turning that development
|
||||
check into a public Windows-host claim.
|
||||
|
||||
If a caller-visible C# reconstruction operation is admitted, it is a distinct
|
||||
optional capability. It accepts only an exact supported BYO `ilspycmd`, records
|
||||
the tool and decompiler version, runs in an owned bounded process without
|
||||
loading target code into REA, and links output to the canonical method
|
||||
observation. Setup and installers do not download or install ILSpy, .NET, an
|
||||
SDK, or a runtime. Doctor may inspect an explicitly configured executable and
|
||||
report exact actionable incompatibility. Missing reconstruction never removes
|
||||
canonical metadata/CIL capabilities.
|
||||
|
||||
This decision does not make a decompiler process a security sandbox. The
|
||||
initial public path can defer C# reconstruction while canonical contracts and
|
||||
process isolation are proven.
|
||||
|
||||
### 7. Use source-built conformance and operator-local real targets
|
||||
|
||||
The repository stores source, build descriptions, expected semantic facts,
|
||||
and tool lock manifests, but not compiled managed binaries. An isolated build
|
||||
step creates fixtures outside source directories and records their SHA-256 and
|
||||
MVID before verification. At minimum, the corpus covers:
|
||||
|
||||
- Framework-compatible and modern .NET metadata;
|
||||
- AnyCPU, x86, and x64 PE/CLI flags;
|
||||
- generics, nested types, overloaded signatures, properties, events, resources,
|
||||
attributes, and constants;
|
||||
- exceptions, filters, prefixes, switch branches, P/Invoke, async and iterator
|
||||
state machines;
|
||||
- obfuscated and Unicode names without assigning semantics from names;
|
||||
- malformed headers, tables, heaps, signatures, bodies, and exception regions;
|
||||
and
|
||||
- two semantically related builds with different MVIDs, tokens, layout, and
|
||||
selected names.
|
||||
|
||||
Expected facts are checked directly and against pinned
|
||||
`System.Reflection.Metadata`; ILSpy, dnlib, or Mono.Cecil may add differential
|
||||
coverage but do not vote by majority. Disagreement is a failing or explicitly
|
||||
unsupported case, not silently normalized output.
|
||||
|
||||
The real-application benchmark is an operator-local, opt-in osu! check derived
|
||||
from the research workflow. It accepts an explicit path plus a compact
|
||||
expectation manifest; verifies target SHA-256, MVID, architecture, and expected
|
||||
assembly identity before analysis; and emits only bounded identities, tokens,
|
||||
signatures, IL lengths/hashes, match states, and assertion results. It does not
|
||||
emit full IL, decompiled source, runtime logs, user data, credentials, or
|
||||
service/account material. No proprietary binary, hash-locked application
|
||||
manifest, or derived dump is committed to REA.
|
||||
|
||||
### 8. Keep runtime behavior separate from static managed analysis
|
||||
|
||||
No managed runtime planning or execution tool ships. Managed inspection does
|
||||
not execute assemblies or establish which methods ran. Where supported, direct
|
||||
process capture can record target-level inputs and outputs, but it does not
|
||||
prove CLR method execution or resolution. Internal managed behavior remains
|
||||
unknown until independently observed with suitable runtime evidence.
|
||||
|
||||
## Public contract consequences
|
||||
|
||||
- Managed static operations will have CLI and MCP parity and produce the same
|
||||
Evidence commitments.
|
||||
- Adding managed operations changes the caller-visible tool inventory and
|
||||
generated product catalog only in the implementation PR that admits them.
|
||||
- Partial parsing includes admitted/dropped counts, exact limitations, and
|
||||
coverage; it never returns a partial list labeled complete.
|
||||
- Managed addresses distinguish file offsets, RVAs, native virtual addresses,
|
||||
metadata tokens, and CIL offsets. They are not interchangeable.
|
||||
- Cache compatibility is exact on artifact/component commitment, member-result
|
||||
fields, parser profile, and producer version. For shipped v1,
|
||||
`normalized_il_sha256` has no independent algorithm-version field.
|
||||
- Provider-neutral workflows may compose canonical managed findings with the
|
||||
one native binding, but every source operation retains its actual producer.
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
### Make ILSpy the only managed provider
|
||||
|
||||
This would make canonical metadata/IL inventory depend on a separately
|
||||
installed CLR and conflate source reconstruction with byte observations. It
|
||||
would also make tool-version drift part of ordinary inventory. ILSpy remains a
|
||||
valuable optional reconstruction and conformance tool instead.
|
||||
|
||||
### Use `Assembly.Load` or reflection-only loading
|
||||
|
||||
Both delegate parsing and dependency behavior to a CLR and make static support
|
||||
runtime-dependent. Ordinary loading can execute target code; reflection-only
|
||||
loading still establishes a CLR load boundary and is unavailable across modern
|
||||
runtimes in a uniform way. Neither is required to decode ECMA-335 bytes.
|
||||
|
||||
### Register the managed analyzer as another deep candidate
|
||||
|
||||
That would force callers to choose managed analysis instead of Hopper/Ghidra
|
||||
for ReadyToRun and mixed-mode artifacts even though both are needed. Disjoint
|
||||
auxiliary composition represents the real ownership boundary.
|
||||
|
||||
### Treat tokens, names, or decompiler output as durable identity
|
||||
|
||||
Tokens and obfuscated names change across builds, while decompiler output
|
||||
changes across tools and versions. They remain useful build-local coordinates
|
||||
or reconstruction labels but are insufficient for exact or structural
|
||||
cross-version identity.
|
||||
|
||||
### Package a .NET runtime or mutable managed library in REA
|
||||
|
||||
This would expand installer effects, package size, patching obligations, and
|
||||
the target-parsing attack surface. The initial byte parser keeps canonical
|
||||
support within REA's existing Node runtime; external managed tools remain BYO
|
||||
or development-only oracles.
|
||||
|
||||
## Rollout gates
|
||||
|
||||
1. Add read-only triage and identity with malformed-input bounds. Shipped.
|
||||
2. Add metadata, signatures, method bodies, deterministic decoded-CIL tuple
|
||||
fingerprints, and exact artifact Evidence. Shipped with the v1 limitations
|
||||
above; full normalized-CIL semantics remain planned.
|
||||
3. Add obfuscation-resistant slices and cross-build match states. Shipped.
|
||||
4. Compose explicit managed/native boundaries with Hopper/Ghidra. Declaration
|
||||
inventory and native export/function Evidence matching shipped; native-body
|
||||
bridge mapping remains planned.
|
||||
5. Add source-built, pinned real-tool, package, CLI, and MCP conformance.
|
||||
Source-owned conformance is shipped; pinned external real-tool checks remain
|
||||
planned.
|
||||
6. Project authenticated managed static Evidence into the application graph.
|
||||
Shipped.
|
||||
|
||||
Every gate must keep unsupported ReadyToRun, NativeAOT, IL2CPP, mixed-mode, or
|
||||
bundle cases explicit rather than broadening claims to make a fixture pass.
|
||||
|
||||
## References
|
||||
|
||||
- [ECMA-335: Common Language Infrastructure](https://ecma-international.org/publications-and-standards/standards/ecma-335/)
|
||||
- [.NET ReadyToRun deployment overview](https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run)
|
||||
- [.NET Native AOT deployment overview](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/)
|
||||
- [.NET single-file deployment overview](https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview)
|
||||
- [`System.Reflection.Metadata` package](https://www.nuget.org/packages/System.Reflection.Metadata/10.0.10)
|
||||
- [ILSpy repository and releases](https://github.com/icsharpcode/ILSpy)
|
||||
- [dnlib repository](https://github.com/0xd4d/dnlib)
|
||||
- [Mono.Cecil repository](https://github.com/jbevain/cecil)
|
||||
@@ -0,0 +1,11 @@
|
||||
# Architecture decision records
|
||||
|
||||
Accepted decisions describe the architecture that implementation PRs must
|
||||
follow. Acceptance does not by itself mean the behavior is shipped; each record
|
||||
states its implementation status separately.
|
||||
|
||||
| ADR | Status | Implementation |
|
||||
| ------------------------------------------------------------------------------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [0001: Provider selection and analysis profiles](0001-provider-selection-and-analysis-profiles.md) | Accepted | Provider selection and 25 read-only Ghidra operations on Linux/macOS and experimental Windows x64 P0; Linux/macOS session annotations |
|
||||
| [0002: Controlled JavaScript replay authority and sandbox policy](0002-controlled-replay-authority-and-sandbox.md) | Superseded | Historical design; the controlled JavaScript replay tool was removed |
|
||||
| [0003: Managed-code evidence and provider boundary](0003-managed-code-evidence-and-provider-boundary.md) | Partially superseded | Seven static managed tools remain; runtime planning removed |
|
||||
+12
-147
@@ -9,27 +9,6 @@ This family is present on repository main and in npm 4.1.0. Check the
|
||||
[released package boundary](installation.md#released-package-and-main) and the
|
||||
connected server's tool list before selecting it.
|
||||
|
||||
## Optional package delivery
|
||||
|
||||
The MCP `inspect_android_package` request keeps complete output when `detail` is
|
||||
omitted or `"complete"`. Select `detail: "summary"` explicitly to receive all
|
||||
package, SDK and permission declarations, engine identity and class/resource
|
||||
counts without decoded manifest text or duplicate raw provider replies. A
|
||||
retaining session keeps the complete original Evidence and returns a distinct
|
||||
linked `inspect_analysis_view` summary. This changes delivery, not JADX analysis.
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view`, source kind `retained-evidence`, and
|
||||
`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read the
|
||||
exact decoded manifest without reopening the APK. Offsets and limits count UTF-16
|
||||
code units; follow `coverage.next_offset`. The page preserves `source_status` and
|
||||
`reported_total_bytes`: exhausting retained text does not make a partial decoded
|
||||
manifest complete. Complete source Evidence remains exportable under the existing
|
||||
session lifetime. Servers without retention reject summary before starting JADX.
|
||||
One-shot CLI output remains complete. Older servers require the existing complete
|
||||
request after schema discovery. APK authenticity and runtime behavior remain
|
||||
unverified.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use a **full JDK 17 or newer** (including its compiler module) and
|
||||
@@ -44,37 +23,10 @@ export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
|
||||
export JAVA_HOME=/absolute/path/existing-jdk
|
||||
```
|
||||
|
||||
On Windows x64, use a REA package with its matching bundled Windows native
|
||||
controls **including owned protocol stdin support**. This Windows boundary is
|
||||
new on repository main; an older released package can still reject this host.
|
||||
PowerShell accepts paths containing spaces without additional escaping
|
||||
inside these environment values:
|
||||
|
||||
```powershell
|
||||
$env:REA_JADX_MCP_JAR = 'D:\tools\jadx-headless-mcp-0.7.1-all.jar'
|
||||
$env:JAVA_HOME = 'D:\tools\jdk-21'
|
||||
rea inspect-android-package 'D:\targets\Example.apk'
|
||||
```
|
||||
|
||||
REA selects `JAVA_HOME\bin\java.exe`, or resolves `java.exe` on the selected
|
||||
Windows `PATH`. A missing or mismatched native artifact reports the package
|
||||
prerequisite rather than attempting an unowned Java launch. Windows arm64 is
|
||||
outside this provider's current native boundary.
|
||||
|
||||
Readiness checks the selected JAR's ZIP directory for the classes consumed by
|
||||
REA's metadata bridge, as well as Java's compiler and version modules. It does
|
||||
not load an APK or execute engine code. Invalid archives and missing bridge
|
||||
classes report JAR-specific recovery; classpath inventory alone does not prove
|
||||
that every class or engine operation will load successfully. Discovery limits
|
||||
each ZIP metadata read to 8 MiB before allocation; a larger directory reports
|
||||
the exhausted read budget and the selected JAR path.
|
||||
|
||||
The current metadata bridge is verified on macOS arm64 with OpenJDK 21 and the
|
||||
public Appium ApiDemos fixture. The POSIX adapter also supports Linux; the new
|
||||
bridge has not yet undergone real verification on Linux/JDK 25.
|
||||
Windows x64 uses an owned Job Object with bidirectional stdio, atomic job
|
||||
assignment and kill-on-owner-close. Real Windows verification uses JDK 21 and
|
||||
the pinned JADX 0.7.1 JAR; see the real verification lane below.
|
||||
Windows is unsupported for this provider's owned stdio process boundary.
|
||||
|
||||
## Answer an analyst question
|
||||
|
||||
@@ -86,7 +38,6 @@ rea inspect-android-method /path/Example.apk example.MainActivity onCreate
|
||||
rea inspect-android-method /path/Example.apk example.MainActivity select --overload-index 1
|
||||
rea trace-android-references /path/Example.apk example.MainActivity
|
||||
rea trace-android-references /path/Example.apk example.MainActivity --method-name onCreate
|
||||
rea trace-android-references /path/Example.apk example.MainActivity --method-name select --overload-index 1
|
||||
```
|
||||
|
||||
Equivalent MCP methods are `inspect_android_package`, `search_android_classes`,
|
||||
@@ -107,22 +58,15 @@ Every request includes `path`. Class/method operations use `class_name`,
|
||||
```
|
||||
|
||||
Use `inspect_android_class` to choose an overload. REA rejects an ambiguous
|
||||
method request rather than silently selecting the first match. Method reference
|
||||
queries accept the same `overload_index`; the bridge resolves the selected
|
||||
method node and the result reports its index and displayed signature. These
|
||||
indices are tied to the current artifact and engine, not stable across builds.
|
||||
method request rather than silently selecting the first match. The engine's
|
||||
method reference operation cannot select overloads, so REA reports that boundary
|
||||
as unsupported when multiple same-name methods exist.
|
||||
|
||||
The engine's smali fallback also joins same-name overloads; REA rejects that
|
||||
fallback for an overloaded method instead of attributing all bodies to one
|
||||
selected overload. Native/abstract methods with no decompiled body report
|
||||
`body_status: "not_available"` without inferring which implementation flag caused it.
|
||||
|
||||
Static initializers (`<clinit>`) return smali with the
|
||||
`static_initializer_disassembly` marker. Java decompilation can move their
|
||||
instructions into field declarations, leaving an empty or incomplete method
|
||||
body. The selected method's disassembly preserves those instructions and retains
|
||||
the same source byte limit and explicit truncation reporting.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- Package inspection retains decoded manifest XML, summary declarations,
|
||||
@@ -130,60 +74,27 @@ the same source byte limit and explicit truncation reporting.
|
||||
upstream engine and should be checked against the retained XML when needed.
|
||||
Signature verification is explicitly `not_performed`.
|
||||
- Class search consumes all pages and checks their counts. An empty `query`
|
||||
inventories all class-name rows; class names are decompiler representations.
|
||||
Repeated names remain repeated in `matches` so the inventory row count and
|
||||
producer evidence stay visible. `duplicate_names` identifies matching names
|
||||
with multiple rows and their occurrence counts; a name alone does not
|
||||
distinguish which provider class node a later lookup selects.
|
||||
inventories all class names; class names are decompiler representations.
|
||||
- Class inventories read parsed metadata without generating source and return
|
||||
full reported display types, fields and inner class names. Synthetic members
|
||||
that code generation hides can appear in the inventory; it stays consistent
|
||||
after method decompilation within the retained session.
|
||||
Exact DEX prototypes are reported from frozen JADX method metadata as
|
||||
`dex_descriptor`, for example `(I)V`. Display names can be JADX aliases;
|
||||
the descriptor alone is not a complete raw DEX method ID. Overload indices
|
||||
are tied to this artifact and engine, not stable identities across builds.
|
||||
Exact DEX descriptors are unknown. Overload indices are tied to this artifact
|
||||
and engine, not stable identities across different builds.
|
||||
- Method inspection reports Java or smali, fallback markers and complete/partial
|
||||
returned text. Decompiled source is a derived representation. A complete text
|
||||
result does not establish complete semantics or a byte-identical reconstruction.
|
||||
- Reference results are incoming static relationships between provider nodes.
|
||||
`reference_unit=using_node` means `total` counts containing JADX nodes, not
|
||||
individual invoke instructions; two calls from one method can appear as one
|
||||
entry. `coverage=complete` covers this node inventory only. For a method
|
||||
reference whose using node has raw code, `direct_invokes` lists matching DEX
|
||||
invoke instructions with their DEX entry, raw source method ID, code-unit
|
||||
offset, method index and opcode. An empty array means no direct invoke was
|
||||
matched in that node; `null` means raw code was unavailable or the using node
|
||||
was not a method. This list does not establish complete call-site coverage
|
||||
outside JADX's using-node inventory or through reflection and dynamic loading.
|
||||
A virtual or interface invoke also does not establish its runtime implementation.
|
||||
Source lines and the containing node's own `dex_offset` remain unknown.
|
||||
Method-reference results also carry the selected `dex_descriptor`;
|
||||
class-reference results report `null`.
|
||||
These relationships are not observed runtime calls.
|
||||
Source lines and DEX instruction offsets remain unknown. They are not observed
|
||||
runtime calls.
|
||||
- Original requests, APK SHA-256, actual engine JAR SHA-256 and raw producer
|
||||
responses remain in Evidence. Source revision is populated only for bytes
|
||||
matching the audited release JAR.
|
||||
|
||||
If a later step fails, the error's `details.partial_observation` retains the
|
||||
selected and resolved APK paths, digest, observed provider identity and versions,
|
||||
and each upstream request and reply collected before the failure. Failed replies
|
||||
remain available even when their payload cannot be interpreted. Cancellation
|
||||
also stops target hashing before provider admission.
|
||||
|
||||
These operations do not execute the APK or provide split APK/AAB handling,
|
||||
signature validation, full resource-table semantics, native-library analysis or
|
||||
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
|
||||
Android tools and the application graph do not treat it as a standalone APK.
|
||||
Existing artifact inventory/extraction tools can supply archive evidence.
|
||||
Inventory records an APK root only for ZIP bytes whose name ends in `.apk`.
|
||||
The projection adds a limitation when no root `AndroidManifest.xml` was
|
||||
observed, because such a ZIP may not be an APK. Nested members take a ZIP or
|
||||
ASAR family from their bytes; an unread member keeps an unknown format.
|
||||
Inventory keeps ZIP names that differ only in case and records that a
|
||||
case-insensitive destination cannot store both spellings. Extraction fails
|
||||
only when that destination directory cannot keep both names. Project that
|
||||
Evidence with the execution-free inventory tool:
|
||||
Android runtime capture. Existing artifact inventory/extraction tools can supply
|
||||
archive evidence. Project that Evidence with the execution-free inventory tool:
|
||||
|
||||
```sh
|
||||
rea project-android-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
|
||||
@@ -200,19 +111,7 @@ rea project-android-application-graph '{"inventory_evidence":[<inventory_artifac
|
||||
|
||||
The projection reports exact component paths and hashes, runtime-family hints,
|
||||
and path-based bridge hypotheses. It does not decode DEX or claim observed
|
||||
runtime calls. Framework hints match complete native library names, such as
|
||||
`libreactnativejni.so`, `libreactnative.so`, `libhermes.so`,
|
||||
`libhermes_executor.so`, `libflutter.so`, and `libunity.so`; unrelated names
|
||||
such as `libcommunity.so` and `libreactivity.so` do not identify those families.
|
||||
The bridge basis is inferred from the native path and repeated for each managed
|
||||
component, so it does not identify a specific JNI binding. Bridge hypotheses form a DEX/JVM-class by native-library
|
||||
Cartesian product. The projection computes the product and exact serialized
|
||||
candidate-array size before creating candidate objects, then retains a
|
||||
deterministic prefix within a 2 MiB UTF-8 JSON budget. If candidates are omitted,
|
||||
`bridge_candidate_coverage` reports total, emitted, and omitted counts; overall
|
||||
coverage is `partial` while component arrays still include every component
|
||||
from the supplied inventory pages. A path pair does not establish that a
|
||||
managed declaration names, loads, or calls the native library.
|
||||
runtime calls.
|
||||
|
||||
## Resource and lifecycle limits
|
||||
|
||||
@@ -224,21 +123,6 @@ target/configuration changes, failure, timeout, cancellation, MCP disconnect or
|
||||
server shutdown. One-shot CLI commands always join cleanup before returning.
|
||||
REA never writes to the original APK or launches target code.
|
||||
|
||||
On Windows, stdin writes preserve stream backpressure and run outside the Node
|
||||
event loop. Cancellation terminates the retained job, including descendants,
|
||||
before workspace cleanup. On Windows, a detached REA helper owns the private
|
||||
workspace's native directory handle. An abrupt CLI/server owner termination
|
||||
closes the Java job and the helper's input pipe; the helper then removes the
|
||||
workspace by handle. The real Windows lane checks both effects. If the helper
|
||||
also dies or the host loses power before it finishes, private files can remain.
|
||||
|
||||
If Java cannot start the bridge, REA reports the selected executable and retained
|
||||
startup diagnostics. Select a full JDK through `JAVA_HOME` or `PATH`, and use
|
||||
`java --list-modules` to confirm `jdk.compiler` is present. A runnable JRE alone
|
||||
cannot compile the bridge. Missing or ambiguous class and method selections are
|
||||
input failures; discover names with `search_android_classes` and
|
||||
`inspect_android_class` before correcting the selected argument.
|
||||
|
||||
If process cleanup cannot be confirmed, REA retains the workspace and reports its
|
||||
location. That provider instance blocks subsequent engine launches until the
|
||||
caller resolves the reported resources and starts a fresh instance.
|
||||
@@ -286,27 +170,8 @@ export REA_ANDROID_TEST_APK="$PWD/_reference/apk-integration/ApiDemos-debug.apk"
|
||||
npm run verify:android
|
||||
```
|
||||
|
||||
The lane also uses the selected full JDK's `jlink` to create a disposable JRE
|
||||
without the compiler module. It checks real Java startup failures, missing
|
||||
selectors, recovery after failures, and CLI cancellation with owned cleanup.
|
||||
The original APK and host Java configuration remain unchanged.
|
||||
|
||||
The same lane runs on Windows x64 with the bundled native controls and a full
|
||||
JDK. It checks CLI/MCP results, real MCP cancellation and disconnect, and Java
|
||||
exit and workspace removal after abrupt CLI owner termination. Windows forced
|
||||
termination is checked separately from the POSIX CLI SIGTERM handler. A
|
||||
synthetic two-owner test confirms that one owner's exit does not remove the
|
||||
other owner's workspace. The lane does not claim a structured cancellation
|
||||
result or power-loss recovery.
|
||||
|
||||
The download script verifies fixed SHA-256 values, reuses matching files and
|
||||
refuses to overwrite a different existing file. APK/JAR files live under ignored
|
||||
`_reference/` and are excluded from package contents. The lane exercises actual
|
||||
REA CLI and MCP results with the real engine, separately from synthetic protocol
|
||||
regressions. It needs no Gradle build, emulator, Android SDK or Ghidra.
|
||||
|
||||
Direct invoke sites retain exact DEX code-unit offsets separately from each
|
||||
containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false`
|
||||
means instruction rows were bounded by the local bridge frame budget; `null`
|
||||
means the producer did not establish instruction coverage. Using-node totals
|
||||
and coverage do not imply complete instruction evidence.
|
||||
|
||||
@@ -1,94 +0,0 @@
|
||||
# Apktool resource decoding
|
||||
|
||||
REA decodes Android resource tables and the binary manifest through the
|
||||
caller's Apktool launcher. This completes the static triad for Android
|
||||
targets: Java/Kotlin sources through the JADX family, engine-grade
|
||||
decompilation through JEB, and resources through this family. REA never
|
||||
installs Apktool or a Java runtime.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use Apktool 2.7.0 or newer (verified with 2.7.0) and a Java runtime the
|
||||
launcher can find. Select the launcher through the environment:
|
||||
|
||||
```sh
|
||||
# Optional: absolute path to the apktool launcher (upstream wrapper or a
|
||||
# java -jar script); without it, apktool resolves from PATH.
|
||||
export REA_APKTOOL_COMMAND=/absolute/path/apktool
|
||||
```
|
||||
|
||||
Readiness is `rea inspect-apktool-client`: it runs `apktool --version` and
|
||||
reports the resolved launcher, its selection source, and its version. A
|
||||
missing launcher reports `capability_unavailable` with the
|
||||
`REA_APKTOOL_COMMAND` remediation rather than attempting an installation.
|
||||
|
||||
On Windows, a `.bat` or `.cmd` launcher (such as Chocolatey's `apktool.bat`)
|
||||
runs through an explicit `cmd.exe /d /s /c` call. That call has no AutoRun,
|
||||
no delayed expansion, and quotes every argument. A launcher path that
|
||||
contains `"`, `%`, `^`, `!`, `&`, `|`, `<` or `>` is reported as
|
||||
`not_configured` instead of being passed to `cmd.exe`. If REA's temporary
|
||||
decode paths contain those characters, decoding through a batch launcher
|
||||
reports `capability_unavailable`.
|
||||
|
||||
## Answer an analyst question
|
||||
|
||||
```sh
|
||||
rea inspect-apktool-client
|
||||
rea decode-android-resources /targets/Example.apk
|
||||
rea decode-android-resources /targets/Example.apk --locale de
|
||||
rea decode-android-resources /targets/Example.apk --no-strings
|
||||
```
|
||||
|
||||
Equivalent MCP methods are `inspect_apktool_client` and
|
||||
`decode_android_resources`. The decode request takes `path`, an optional
|
||||
`locale` (project `res/values-<locale>/strings.xml` instead of the default
|
||||
table), and `include_strings` (default true). Regional locales such as `pt-BR`
|
||||
select Android's `values-pt-rBR`; Android qualifiers such as `pt-rBR` and
|
||||
`b+zh+Hans+CN` may also be supplied directly.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **An owned APK snapshot is digested and decoded.** `target.sha256` and
|
||||
`target.bytes` bind the observation to the exact APK that was decoded.
|
||||
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
|
||||
keep their own spelling (`'7'` versus `7`); `package_name` is read from
|
||||
the decoded manifest, not inferred.
|
||||
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
|
||||
truncated when larger.
|
||||
- **Strings are projected verbatim.** Escaped entities stay escaped, and
|
||||
entries containing nested markup (`<b>`, `<xliff:g>`) are counted as
|
||||
unparsed rather than flattened — the limitation names the count. The
|
||||
default table is bounded to 2000 entries with partial coverage when more
|
||||
exist; `locales` lists every `values-<locale>` that has a strings table.
|
||||
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
|
||||
recovery belongs to the JADX family. Resource smali edits and rebuild are
|
||||
a follow-up lane with mutation authority.
|
||||
- **Workspaces are owned and short-lived.** Decoding happens in a
|
||||
provider-owned temporary directory that is removed before the result is
|
||||
returned. The APK snapshot and framework cache both live inside this directory.
|
||||
If removal fails, the result names the residual path and the provider retains
|
||||
ownership for a cleanup retry on close, including failed decodes.
|
||||
|
||||
## Resource and lifecycle limits
|
||||
|
||||
- The decode runs with a 600 s deadline and a 4 MiB output capture budget;
|
||||
a stopped decode reports `provider_timeout` and exceeded budgets report
|
||||
`resource_constraint`.
|
||||
- Manifest reads are bounded to 512 KiB before decoding UTF-8. Strings XML
|
||||
reads are bounded to 8 MiB; larger documents are skipped with partial
|
||||
coverage and an explicit limit, rather than reported as missing.
|
||||
- `decoded_file_count` is capped at 100,000 files; the cap is reported in
|
||||
the limitations when hit.
|
||||
- Apktool's own decode failures (`AndrolibException`, directory traversal
|
||||
refusals) surface as `unsupported_target` with apktool's reason preserved
|
||||
verbatim.
|
||||
|
||||
## Real verification
|
||||
|
||||
`npm run verify:apktool -- --apk PATH` exercises both operations against a
|
||||
real launcher. Record: apktool 2.7.0-dirty (Debian packaging) on Linux with
|
||||
OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
|
||||
locales — launcher version, on-disk digest agreement, apktool.yml metadata
|
||||
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
|
||||
tables projected. Parser goldens in the test suite come from the same
|
||||
decode. See `docs/testing.md` for the lane summary.
|
||||
@@ -7,16 +7,6 @@ hashes, detected formats, bundle anatomy, runtime-family hints, and path-based
|
||||
bridge hypotheses. It does not parse plist or CMS semantics or claim observed
|
||||
runtime calls.
|
||||
|
||||
Framework hints match complete path segments such as `React.framework`,
|
||||
`ReactNative.framework`, `hermes.framework`, `Flutter.framework`, and
|
||||
`UnityFramework.framework`. Substrings in `Preact.framework`,
|
||||
`Community.framework`, or `MyApp.framework` do not identify those families.
|
||||
A generic `App.framework` also does not identify Flutter by itself. Runtime
|
||||
families remain inventory-derived hints, not observed loading. Generic framework
|
||||
and dylib paths do not establish Swift or Objective-C; this projection does not
|
||||
decode language metadata. A bridge basis is inferred from the native path and
|
||||
repeated for each paired script, rather than proving a particular binding.
|
||||
|
||||
```sh
|
||||
rea project-apple-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
|
||||
```
|
||||
@@ -30,35 +20,16 @@ rea project-apple-application-graph '{"inventory_evidence":[<inventory_artifact
|
||||
}
|
||||
```
|
||||
|
||||
The result includes every component in each inventory category. Bridge
|
||||
hypotheses are the path-based JavaScript/native pairs within each application
|
||||
root. To keep their Cartesian expansion bounded, the projection limits the
|
||||
serialized candidate array to 2 MiB. It computes the pair count and
|
||||
exact UTF-8 JSON size before creating candidate objects, then retains a
|
||||
deterministic prefix when the byte budget is exceeded. The result's
|
||||
`bridge_candidate_coverage` reports total, emitted, and omitted pair counts;
|
||||
overall coverage becomes `partial` and a limitation explains the omission.
|
||||
Duplicate inventory pages are merged by artifact identity and occurrence path;
|
||||
their Evidence IDs remain attached as source Evidence.
|
||||
|
||||
Content nodes identify bytes by SHA-256 and report only content-derived formats.
|
||||
Each occurrence reports its own `artifact_kind`, `artifact_format`, and executable
|
||||
permission. Identical bytes can therefore appear as a script, resource, or native
|
||||
addon at different paths without overwriting another occurrence's role. Package
|
||||
formats such as APK and IPA belong to the occurrence; their ZIP bytes share a
|
||||
content format. Graph and directory identities use locale-independent Unicode
|
||||
ordering.
|
||||
|
||||
For IPA inventories, archive components outside every application root remain
|
||||
in the component lists. Their JavaScript and native candidates are grouped with
|
||||
other unrooted components, preserving the inventory projection's path pairing
|
||||
semantics without attributing them to an application bundle.
|
||||
The result includes every component in each inventory category and every
|
||||
JavaScript-to-native bridge candidate pair. It has no caller-selected component
|
||||
budget, prefix truncation, or omitted-count fields. Duplicate inventory pages
|
||||
are merged by artifact identity and occurrence path; their Evidence IDs remain
|
||||
attached as source Evidence.
|
||||
|
||||
Coverage is `complete-within-inventory` when the supplied pages reconstruct the
|
||||
complete authenticated inventory, and `partial` otherwise. A partial result
|
||||
states that absence is unknown. Component arrays remain complete when candidate
|
||||
hypotheses are omitted; component arrays still include every component from
|
||||
the supplied inventory pages, and overall coverage is `partial` in that case.
|
||||
states that absence is unknown. Component and bridge-candidate arrays are still
|
||||
the complete projection of the inventory that was supplied.
|
||||
|
||||
## Application roots
|
||||
|
||||
@@ -139,90 +110,3 @@ The projection does not read plists. Use `inspect_plist` on a bundle's
|
||||
`info_plist_path` or a launchd plist to read `CFBundleExecutable`, identifiers,
|
||||
`SMPrivilegedExecutables`, `BundleProgram`, URL schemes, and usage descriptions.
|
||||
Use `inspect_signature` on an executable candidate for code-signing claims.
|
||||
|
||||
## Dylib load resolution
|
||||
|
||||
`trace_dylib_resolution` answers which file each Mach-O dylib load reaches. It
|
||||
runs on the active Mach-O, or on every executable in the active `.app`. Each
|
||||
executable in a bundle (the main app, XPC services, app extensions, login items,
|
||||
privileged helpers, and helper tools) is a separate process root with its own
|
||||
`@executable_path`.
|
||||
|
||||
```sh
|
||||
rea trace-dylib-resolution MyApp.app --json
|
||||
rea trace-dylib-resolution MyApp.app --root Contents/MacOS/MyApp --architecture arm64 --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "trace_dylib_resolution",
|
||||
"arguments": { "roots": ["Contents/MacOS/MyApp"], "architecture": "arm64" }
|
||||
}
|
||||
```
|
||||
|
||||
Load commands are parsed in TypeScript from Mach-O and FAT headers, including
|
||||
the `dylib_use_command` encoding newer linkers emit for flags such as delayed
|
||||
initialization. Nothing is executed and no Apple tool is required, so the tool
|
||||
also works off macOS.
|
||||
|
||||
Resolution follows dyld:
|
||||
|
||||
- `@executable_path` is the directory of the process root's executable. It is
|
||||
undetermined when the root is a library.
|
||||
- `@loader_path` is the directory of the image that holds the load command, or
|
||||
of the image that owns the `LC_RPATH` entry being expanded.
|
||||
- `@rpath` tries each `LC_RPATH` of the loading image first, then those of the
|
||||
image that loaded it, up to the process root.
|
||||
- Dependencies load dependents-first in load-command order, and each image
|
||||
loads once per process. A later @rpath request whose install name matches an
|
||||
already loaded image reuses it.
|
||||
- Symlinks such as `Versions/Current` are followed inside the analyzed root.
|
||||
An escape through `..` or an absolute link is `escapes-target`.
|
||||
|
||||
Each edge lists every candidate path tried, with its outcome: `resolved`,
|
||||
`absent`, `not-mach-o`, `malformed`, `unsupported` (recognized Mach-O outside
|
||||
parser coverage, such as big-endian images), `architecture-missing`,
|
||||
`outside-target`, `escapes-target`, or `undetermined`. Slices follow dyld's
|
||||
graded architectures, so an `x86_64h` process also loads `x86_64`. The edge's `resolution` is one of:
|
||||
|
||||
| Status | Meaning |
|
||||
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `resolved` | An image inside the analyzed root loads, and no earlier candidate was undeterminable. |
|
||||
| `conditional` | An image inside the root resolves, but an earlier candidate outside the root (for example `/usr/lib/swift`) would win if it exists on the running system. |
|
||||
| `unresolved` | Every candidate is definitively absent or unusable inside the root. |
|
||||
| `undetermined` | No candidate resolves inside the root, and some candidate lies outside it or depends on the environment. |
|
||||
|
||||
A `conditional` image is still traced, because it is the image the analyzed
|
||||
root supplies. Every edge below it has `loader_conditional: true`, and a
|
||||
request that reuses it by install name is `conditional` too. Findings on those
|
||||
edges say that they apply only if the conditional image loads.
|
||||
|
||||
Absolute install names such as `/usr/lib/libSystem.B.dylib` are
|
||||
`outside-target`. On macOS 11 and later most of them live in the dyld shared
|
||||
cache rather than on disk, so REA does not check them against the host.
|
||||
|
||||
`findings` are derived, not observed:
|
||||
|
||||
- `required-load-unresolved`, `weak-load-unresolved`, and `lazy-load-unresolved`.
|
||||
`LC_LAZY_LOAD_DYLIB` dependencies are resolved but not traversed, because dyld
|
||||
loads them on first use; an unresolved one is not a launch failure.
|
||||
- `earlier-rpath-candidate-absent`: a compatible Mach-O placed at an earlier
|
||||
modeled search path could take precedence, subject to unmodeled search inputs
|
||||
and code-signing library validation. These findings do not establish which
|
||||
image dyld will load; use `inspect_signature` for signing metadata.
|
||||
- `dyld-environment-present`: preserves observed `LC_DYLD_ENVIRONMENT` entries.
|
||||
Only an executable root supplies process settings; entries on library and
|
||||
other non-executable roots do not alter resolution or coverage. Applicable
|
||||
executable image-selection overrides are not modeled.
|
||||
|
||||
Empty dyld settings have different consumers. Modern dyld appends a slash and
|
||||
image name to ordinary and fallback directory entries, so an empty entry can
|
||||
search `/child.dylib` or `/Foo.framework/Foo`. REA retains uncertainty for those
|
||||
unchecked paths; an empty fallback does not preempt an ordinary-search hit.
|
||||
Empty versioned-directory lists do not scan a directory, and empty suffixes
|
||||
only repeat the original candidate, so they do not add search uncertainty.
|
||||
Reported command values are retained in all cases.
|
||||
|
||||
`verify:macos-bundle` checks the parser against `otool -l` for every traced
|
||||
image. It also compares the predicted load order of two process roots with the
|
||||
images dyld actually loads (`DYLD_PRINT_LIBRARIES`).
|
||||
|
||||
@@ -6,7 +6,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
@@ -26,14 +26,11 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Provider Layers"
|
||||
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
|
||||
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
|
||||
EVM["src/evm/<br/>unchanged EVMole/WASM in bounded owned worker<br/>explicit carrier parsing and inferred interface adapter"]
|
||||
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
|
||||
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory and extraction<br/>format readers and native capabilities"]
|
||||
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
@@ -59,7 +56,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Configuration"
|
||||
CONFIG["src/config/<br/>Zod-validated env parsing<br/>provider settings and origins"]
|
||||
CONFIG["src/config.ts + src/config/<br/>Zod-validated env parsing<br/>provider settings and origins"]
|
||||
end
|
||||
|
||||
subgraph "Tests"
|
||||
@@ -95,12 +92,6 @@ graph TD
|
||||
APP -->|record callbacks| RECORDS
|
||||
SERVER -->|narrow record ports| RECORDS
|
||||
RECORDS --> DOMAIN
|
||||
COMPOSITION --> WASM
|
||||
WASM --> PROCESS
|
||||
WASM --> DOMAIN
|
||||
COMPOSITION --> PWNTOOLS
|
||||
PWNTOOLS --> PROCESS
|
||||
PWNTOOLS --> DOMAIN
|
||||
COMPOSITION --> HOPPER
|
||||
COMPOSITION --> GHIDRA
|
||||
COMPOSITION --> IDA
|
||||
@@ -149,8 +140,3 @@ graph TD
|
||||
TESTS -.-> BROWSER
|
||||
TESTS -.-> EVALUATION
|
||||
TESTS -.-> ERRORS
|
||||
|
||||
COMPOSITION --> EVM
|
||||
APP -->|EVM interface port| EVM
|
||||
EVM --> PROCESS
|
||||
EVM --> DOMAIN
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 48 KiB |
@@ -1,194 +0,0 @@
|
||||
# Offline binary diagnostics
|
||||
|
||||
`inspect_binary_layout` / `inspect-binary-layout` answers what an explicit
|
||||
file contains: section/segment layout, original symbols/relocations, static
|
||||
linkage names and mitigation indicators. It works without an active Hopper,
|
||||
Ghidra or IDA target. CTF is one possible use; crash triage, compatibility and
|
||||
ordinary binary inspection use the same modular capability.
|
||||
|
||||
## Bring your own engine
|
||||
|
||||
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
|
||||
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
|
||||
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
```sh
|
||||
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
|
||||
rea inspect-binary-layout ./selected.elf --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_binary_layout",
|
||||
"arguments": { "path": "/artifacts/selected.elf" }
|
||||
}
|
||||
```
|
||||
|
||||
REA never installs Python, packages or GDB, changes a user init file, launches
|
||||
the selected object as a host process, or requests runtime library resolution. The Python process
|
||||
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
|
||||
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/pwntools/README.md).
|
||||
|
||||
## Interpreting results
|
||||
|
||||
- Artifact path, SHA-256 and size identify original selected bytes. All parsing
|
||||
reads an owned stable snapshot; the original is unchanged.
|
||||
- Addresses, offsets, lengths and flags are hexadecimal strings. A linked
|
||||
address is not a runtime address. Runtime load base remains null. A zero
|
||||
EXEC/DYN entry means absent; a relocatable entry is not applicable.
|
||||
- Each symbol retains its table and entry index. Its reported value can mean
|
||||
undefined, alignment, absolute value, no address, unknown section index,
|
||||
section offset, TLS offset or linked virtual address. Duplicate names remain
|
||||
separate entries. SHN_XINDEX remains unresolved in the selected upstream
|
||||
representation; its external index table is not silently treated as resolved.
|
||||
- Relocatable objects have section-relative relocations and no executable entry
|
||||
claim. Signed REL/RELA relocation addends are decimal strings. Packed RELR
|
||||
tables retain full original bytes and upstream-decoded offsets as derived
|
||||
evidence, with per-offset packed-word locations and implicit addends unknown.
|
||||
Overall relocation inventory completeness remains unknown.
|
||||
A relocatable SHN_UNDEF target or inactive SHT_NULL header remains an explicit
|
||||
unknown with the original reported section index and offset.
|
||||
- REL/RELA symbol references retain their original table and entry indices.
|
||||
Positive indices resolve through a validated symbol table; symbol index zero
|
||||
means a zero symbol value without a table lookup, including when no table is
|
||||
linked. Malformed references fail with the affected section and symbol index.
|
||||
- Section-name and symbol tables require declared SHT_STRTAB links.
|
||||
SHN_UNDEF explicitly means no section names: display is empty, raw name and
|
||||
location unknown, with the original name offsets retained. Extended section-name
|
||||
indices resolve through section zero and undergo the same validation.
|
||||
- Symbol tables require a declared SHT_STRTAB link. Name offsets and terminators
|
||||
must stay inside the declared string table; malformed references fail before
|
||||
unrelated bytes can become names. Dynamic dependencies use DT_STRTAB/DT_STRSZ
|
||||
with a unique file-backed mapping; ambiguous or unbacked mappings are
|
||||
explicitly unsupported.
|
||||
- Name display strings use UTF-8 replacement for opaque bytes, including
|
||||
sectionless dependency names and interpreter paths. Raw name
|
||||
bytes and string-table ranges retain observed identity where resolvable;
|
||||
ranges include the terminating NUL and base64 bytes exclude it.
|
||||
- NOBITS and NULL sections provide no file bytes. Reported file-backed ranges
|
||||
are checked against snapshot size again at the public boundary. PT_NULL payload
|
||||
fields are unused: reported numbers remain, file backing is none and interpreted
|
||||
permissions are null. Segments with zero file size also have no file bytes.
|
||||
- Sectionless dynamic images still report dependency names with raw bytes and
|
||||
file locations where uniquely mapped. Missing symbol/relocation section tables
|
||||
do not establish that those runtime tables are absent.
|
||||
Upstream RELRO/canary heuristics rely on sections and can be incomplete in
|
||||
this case; their reported candidates retain an explicit coverage limitation.
|
||||
- DT_NEEDED/PT_INTERP are reported names, not resolved runtime paths. GOT/PLT
|
||||
maps are derived convenience views and may collapse aliases. PLT inference
|
||||
can emulate selected instructions in Unicorn; its outputs are static candidates.
|
||||
Exec syscall tracing observes host process launches. Map completeness
|
||||
remains unknown and original upstream warnings are returned inline.
|
||||
- Canary, PIE, NX, executable-stack and RELRO are static inferences. NX and
|
||||
executable-stack are separate upstream indicators: on x86-64 an executable
|
||||
stack can accompany an unknown (`null`) NX indicator. ET_DYN can be a shared
|
||||
library, and an absent canary symbol does not prove every function unprotected.
|
||||
|
||||
When complete layout Evidence is already retained, `inspect_analysis_view` /
|
||||
`inspect-analysis-view` projects a summary, the mitigations or linkage facet,
|
||||
one section or symbol, or a stable page without re-running the decoder. CLI
|
||||
one-shot invocations pass portable inline Evidence JSON; an MCP session uses
|
||||
the retained `evidence_id`. Page `limit` is required and bounded by the
|
||||
measured MCP stdio budget documented on the tool contract.
|
||||
|
||||
Over MCP, `inspect_binary_layout` with `"detail": "summary"` retains the
|
||||
complete layout Evidence in the session and returns only its summary view.
|
||||
Its `normalized_result.parent_evidence_id` names the retained layout for later
|
||||
section, symbol, mitigation or linkage views. The default `complete` detail and
|
||||
CLI output are unchanged.
|
||||
|
||||
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
|
||||
and 30-second owned command deadline. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the
|
||||
input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The
|
||||
reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
|
||||
copies and encoding overhead. The
|
||||
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
|
||||
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
|
||||
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
|
||||
These settings affect delivery, not decoder coverage or limits. The large
|
||||
extended-index verification uses 256 MiB and an explicit five-minute request
|
||||
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
|
||||
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
|
||||
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
|
||||
Unicorn needs a large virtual map. Limits fail with no partial success, and
|
||||
owned process/root cleanup completes independently of cancellation.
|
||||
|
||||
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
|
||||
inspection are separate increments tracked by #969. This increment makes no
|
||||
real support claim for those features or other operating systems/architectures.
|
||||
|
||||
Typed decoder failures retain bounded `captured_output` (stdout, stderr and an
|
||||
explicit truncation flag) alongside the original failure category and reason.
|
||||
Successful `diagnostics` also exposes the supervisor's `truncated` flag. Output
|
||||
beyond the complete diagnostic budget is rejected, including when the decoder
|
||||
writes a valid reply; cleanup and late cancellation reuse the observed flag.
|
||||
Memory allocation failures use `resource_constraint`, with effective resource
|
||||
limits (or an explicit unknown) and memory-specific recovery guidance. A
|
||||
MemoryError alone does not establish the exact failed allocation or that a
|
||||
particular budget was exhausted. Invalid undersized symbol entries are rejected
|
||||
before REA reports source ranges for them.
|
||||
The bridge reserves 1 MiB to report allocation failures. If even error reporting
|
||||
or serialization fails with MemoryError, its reserved exit status preserves the
|
||||
resource classification with unknown effective limits.
|
||||
Reserved exits require a matching private marker written by the bridge's actual
|
||||
failure branch. Bare launcher exits or missing/unwritable markers retain process
|
||||
diagnostics with the resource cause unverified.
|
||||
Signal termination alone is never classified as observed memory exhaustion.
|
||||
The small bootstrap also covers catchable MemoryError during adapter imports,
|
||||
compilation and initialization. Python interpreter startup failures before the
|
||||
bootstrap runs retain their observed process diagnostics without guessed causes.
|
||||
SIGXCPU retains a CPU resource diagnostic and CPU-specific recovery advice.
|
||||
Observed EFBIG writes or SIGXFSZ termination retain a file-size resource
|
||||
diagnostic and file-size-specific recovery advice. If a tight file limit also
|
||||
prevents writing the limit record, effective values remain unknown.
|
||||
The bridge records actual limits in owned storage before analysis; missing or
|
||||
malformed limit reports remain unknown with their read failure preserved. A
|
||||
received signal alone does not establish its exact cause. Dynamic tags require
|
||||
a complete DT_NULL inside PT_DYNAMIC; interpreter names and ranges end at the
|
||||
first NUL, with any padding still represented by the original segment range.
|
||||
|
||||
## Portable PE resources
|
||||
|
||||
`inspect_pe_resources` / `inspect-pe-resources` inspects an explicit PE32 or
|
||||
PE32+ file without an active target, native engine, Python, or Windows SDK:
|
||||
|
||||
```sh
|
||||
rea inspect-pe-resources ./selected.exe --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_pe_resources",
|
||||
"arguments": { "path": "/artifacts/selected.exe" }
|
||||
}
|
||||
```
|
||||
|
||||
The complete Evidence identifies the stable artifact snapshot by path, byte
|
||||
count and SHA-256. Resources retain separate type, name and language identities:
|
||||
numeric IDs are distinct from UTF-16 names, whose original bytes are also
|
||||
reported. Directory headers, entries, data entries and payloads retain original
|
||||
file offsets and lengths. Each opaque payload has its own SHA-256; code page and
|
||||
reserved data-entry fields are preserved.
|
||||
|
||||
RT_GROUP_ICON records expose original image descriptors and RT_ICON candidates.
|
||||
A reference resolves only when an icon with the same numeric ID and language
|
||||
exists. Missing references, other-language candidates and declared-size mismatch
|
||||
remain explicit. Encoded dimensions are preserved (zero denotes 256 pixels).
|
||||
This is static reference inspection; no image decoding, language fallback,
|
||||
resource loading or selected-file execution occurs.
|
||||
|
||||
An absent resource data directory is a complete empty inventory. Malformed
|
||||
headers, truncated tables, cycles, duplicate sibling identities, unbacked or
|
||||
ambiguous RVA mappings, nonidentical overlapping ranges and unsupported tree
|
||||
shapes fail explicitly, rather than returning partial inventories. Exact shared
|
||||
payload ranges are supported. The supported tree has three levels: type, name,
|
||||
and language. Other resource payload formats remain opaque.
|
||||
|
||||
`max_file_bytes` defaults to 64 MiB (maximum 512 MiB); `max_entries` defaults to
|
||||
16384, the largest stock Windows resource image measured with headroom
|
||||
(maximum 65536). CLI options are `--max-file-bytes` and `--max-entries`.
|
||||
Metadata is bounded to 8 MiB; exceeding any budget fails without truncation.
|
||||
Large Evidence envelopes may require a caller-configured MCP transport receive
|
||||
buffer because envelope copies add overhead. Requests honor cancellation during
|
||||
snapshot reading, directory traversal and payload hashing. The operation reads
|
||||
regular files only and rejects symbolic links.
|
||||
@@ -8,49 +8,21 @@ exported script's native imports under an explicit URL/import-map context.
|
||||
|
||||
REA can attach to a user-owned Chrome-family browser through the Chrome DevTools Protocol (CDP) and produce bounded Evidence about an existing page. This is a passive reverse-engineering capability, not a general browser automation or remote-control surface.
|
||||
|
||||
For request/response bodies and traffic debugging, see
|
||||
[Fiddler Everywhere captures and MCP](web-network-captures.md#reading-fiddler-captures-saz).
|
||||
Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate MCP server.
|
||||
|
||||
## Shipped surfaces
|
||||
|
||||
- `list_browser_targets` / `rea list-browser-targets` discovers all eligible page targets at the endpoint by default, with an optional exact-origin filter.
|
||||
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
|
||||
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
|
||||
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete structural input-schema summary inline. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata.
|
||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
||||
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
||||
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
|
||||
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
|
||||
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
|
||||
|
||||
Target discovery returns the complete in-scope target array in one result.
|
||||
|
||||
## Screenshot comparison memory budget
|
||||
|
||||
`compare_web_screenshots` admits both PNGs against a 256 MiB estimated
|
||||
working-memory budget before inflating either image. This is a processing
|
||||
buffer budget for this operation, not a PNG dimension limit, CDP transport
|
||||
limit, or process RSS guarantee. Header-valid dimensions that exceed the
|
||||
budget return `resource_constraint` with `details.resource: "memory"` and the
|
||||
estimated and maximum working-set bytes. Invalid PNG structure remains
|
||||
malformed input; encodings outside supported pixel-comparison coverage remain
|
||||
unsupported-target errors.
|
||||
|
||||
The conservative estimate includes the two decoder-owned decoded input
|
||||
buffers, the current image's concatenated IDAT buffer, Node zlib's default
|
||||
16 KiB output chunk backing rounded up for retained chunks, the concatenated
|
||||
inflate result while its output chunks are retained, one possible extra output
|
||||
chunk before zlib checks `maxOutputLength`, unfiltered scanline storage,
|
||||
RGB-to-RGBA expansion, and the first retained RGBA image while the second is
|
||||
decoded. RGBA output aliases its unfiltered buffer and is counted once.
|
||||
Admission uses checked integer arithmetic before image-sized allocation.
|
||||
Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
@@ -80,40 +52,8 @@ google-chrome \
|
||||
|
||||
REA does not launch, own, or terminate this browser. Use a dedicated profile and stop it yourself when the investigation is complete. Do not expose the debugging port on a non-loopback interface.
|
||||
|
||||
Other Chromium-family browsers, such as Chromium or Brave, work the same way
|
||||
with the same two flags. On macOS, run the executable inside the application
|
||||
bundle, for example
|
||||
`"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"` or
|
||||
`"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"`. Fedora installs
|
||||
Chromium as `chromium-browser`.
|
||||
|
||||
REA needs the browser's HTTP discovery endpoint on a loopback port. A browser
|
||||
driven over `--remote-debugging-pipe` or a private channel, for example by an
|
||||
automation SDK, exposes no such endpoint, so `list_browser_targets` has nothing
|
||||
to attach to. Connection failures report `endpoint_unreachable`; a reachable
|
||||
HTTP service that returns a non-200 or non-JSON discovery response reports
|
||||
`invalid_endpoint_response`.
|
||||
Reproduce that flow in a Chromium-family browser started with
|
||||
`--remote-debugging-port`.
|
||||
|
||||
In a minimal Linux container, headed Chromium under Xvfb can keep running while
|
||||
its DevTools port never opens. Install the GLES, EGL, and Mesa DRI libraries
|
||||
(on Fedora: `libglvnd-gles`, `mesa-libEGL`, and `mesa-dri-drivers`), then
|
||||
restart the browser.
|
||||
|
||||
On Linux, a reachable but locked desktop keyring can stall Chrome before its
|
||||
first navigation. For a dedicated investigation profile, add
|
||||
`--password-store=basic` to bypass that keyring. Empty or placeholder frame URLs
|
||||
then describe an unloaded page; REA continues to require an allowed, committed
|
||||
URL. See [the startup investigation](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686).
|
||||
|
||||
## Request boundary
|
||||
|
||||
CDP connections enforce the shared 64 MiB limit per WebSocket message. A larger
|
||||
reply, including a valid reply from Chrome, fails with `payload_limit`. The limit
|
||||
counts message bytes before JSON parsing; it does not bound the memory used by
|
||||
decoded strings or JSON objects.
|
||||
|
||||
Discovery supplies a literal loopback CDP endpoint and returns all eligible
|
||||
page targets unless an exact HTTP(S) origin filter narrows the result. Follow-up
|
||||
inspection requests select one target; their default scope is its current origin. `localhost`, private-LAN
|
||||
@@ -194,11 +134,11 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Accessibility structure and roles are retained by default, while names and descriptions require `include_accessibility_text: true`.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free property paths/types, and discarded. `Network.getResponseBody` is never sent unless JSON body shape capture is selected.
|
||||
- Console observations with a stack source retain call type, argument types, timestamp, and source location. When console text capture is selected, already-delivered primitive values are retained verbatim; objects, getters, and remote properties are never expanded.
|
||||
- WebSocket observations retain direction, opcode, and payload byte length. When shape capture is selected, text frames are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw frames are never retained.
|
||||
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache bodies above 64 KiB and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
|
||||
- Script metadata is included only when CDP supplies a URL on an allowed origin. Stable keys exclude transient CDP script IDs, and exact transient raw URLs are used only during script/resource reconciliation. URL-less scripts are excluded because their origin cannot be established. When CDP supplies an execution-context association, the accepted script retains its authorized frame ID for later attribution. Script metadata admission bounds retained identity and source-map strings to 8 MiB and 50,000 records. When a source-map URL cannot fit, the script identity remains and source-map coverage reports `resource_budget_exhausted`; rejected script observations are counted without retaining their IDs. Source content is omitted unless explicitly requested and is returned inline as a self-verifying artifact with its SHA-256 digest.
|
||||
- Script metadata is included only when CDP supplies a URL on an allowed origin. Stable keys exclude transient CDP script IDs, and exact transient raw URLs are used only during script/resource reconciliation. URL-less scripts are excluded because their origin cannot be established. When CDP supplies an execution-context association, the accepted script retains its authorized frame ID for later attribution. Source content is omitted unless explicitly requested and is returned inline as a self-verifying artifact with its SHA-256 digest.
|
||||
|
||||
Frames, resources, scripts, events, and workers outside the selected origin scope are excluded. An explicit `allowed_origins` list can include additional origins; excluded target details are counted without being exposed.
|
||||
Retained workers include validated opener-target and parent-frame IDs when CDP
|
||||
@@ -208,40 +148,18 @@ prove static module ownership.
|
||||
## Completeness and limits
|
||||
|
||||
The default inspection window is 500 ms; callers choose its duration. The
|
||||
capture returns in-scope frames, DOM and accessibility nodes, admitted scripts,
|
||||
capture returns all in-scope frames, DOM and accessibility nodes, scripts,
|
||||
resources, workers, storage names, and observed network, console, and WebSocket
|
||||
events. Full selected text, script sources, and JSON shapes are returned inline.
|
||||
The result distinguishes
|
||||
`complete_within_window`, `policy_filtered`, `attach_limited`, and `truncated`
|
||||
coverage. It reports excluded items and any values omitted or truncated by
|
||||
validation and resource bounds. Omitted script metadata makes its affected
|
||||
section explicitly truncated.
|
||||
validation; it does not drop records to meet an internal collection quota.
|
||||
Disallowed-origin entries are filtered before retention.
|
||||
|
||||
CDP discovery, connection, and commands wait for the peer or caller cancellation;
|
||||
a stalled peer can leave an operation waiting. WebSocket messages remain subject
|
||||
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
|
||||
pending-command quota. On cancellation, it closes only its own WebSocket,
|
||||
waiting up to one second before terminating it. Malformed unsolicited events
|
||||
and malformed replies to pending commands poison the connection so later
|
||||
commands fail closed. A correlated reply's session ID must match the command's
|
||||
selected session. An optional method is unavailable only when the browser
|
||||
rejects it as unknown; other command rejections retain their reported failure.
|
||||
CDP discovery, WebSocket connection, and commands continue until the peer responds or the caller cancels the operation; REA has no internal byte, message-count, or pending-command quota for these exchanges. Without caller cancellation, a stalled peer can leave an operation waiting. On cancellation, REA closes only its own WebSocket, waiting up to one second before terminating it. Malformed unsolicited protocol events poison the connection so later commands fail closed.
|
||||
|
||||
Network and console coverage starts only after REA attaches and enables the relevant CDP domains. `prior_activity_available` is always `false`; absence from these arrays is not evidence that an event never occurred. Scripts are different: scripts the page parsed before REA attached are still reported under `scripts`. A page that loaded before attach can therefore list its scripts while `network.requests` lacks the requests that fetched them. If CDP reuses a request ID without supplying redirect metadata, REA retains the predecessor observation and marks network coverage incomplete.
|
||||
|
||||
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation has a 30-second deadline and a 64 MiB aggregate streamed-response budget across redirects. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
|
||||
|
||||
source-map items retain the script's `declared_url` separately from the observed
|
||||
`effective_url`. successful maps use the latter as their source-resolution base.
|
||||
failed fetches retain the last response URL observed within the approved origins;
|
||||
`effective_url` is `null` when no response was observed. query parameters are
|
||||
preserved, and both URLs count toward the inline evidence budget.
|
||||
Current fetches emit a string or null; historical observations may omit
|
||||
`effective_url` without changing their stored content or Evidence identity.
|
||||
An observed URL must fit before its response body is read. If it cannot fit,
|
||||
remaining map results are explicitly unknown; failed cancellation retains its
|
||||
cleanup context and resource constraint.
|
||||
Network and console coverage starts only after REA attaches and enables the relevant CDP domains. `prior_activity_available` is always `false`; absence from these arrays is not evidence that an event never occurred. Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Every requested map, mapping, original source, AST node, and finding from parsed content is returned inline; failed maps remain explicitly unavailable and make bundle completeness partial. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
|
||||
|
||||
## Non-goals and threat model
|
||||
|
||||
|
||||
@@ -156,18 +156,6 @@ Storage cookies are scoped to the page URL at each capture. Unsupported action
|
||||
tags, unknown fields, duplicate step IDs, and undeclared secret references fail
|
||||
validation.
|
||||
|
||||
Cookies are installed before the initial navigation. Local storage entries are
|
||||
seeded once per browser storage partition; session storage entries are seeded
|
||||
once per origin in each top-level browsing context. Seeding occurs before the
|
||||
first document scripts for that area. An origin first reached by a later action
|
||||
receives its seed on that first visit. Application updates, deletions, and
|
||||
empty values then persist across later navigations in the same area; a new
|
||||
scenario starts with its own seeds. Partitioned frames can receive separate
|
||||
local storage seeds even when they share an origin. The pre-script mechanism
|
||||
uses CDP debugger instrumentation. The observer resumes only pauses whose top
|
||||
frame belongs to its private seed script; page breakpoints and `debugger`
|
||||
statements remain under the attached debugger client's control.
|
||||
|
||||
The result is Evidence with an initial state followed by one record per
|
||||
declared action. Each step reports action status, elapsed time, sanitized URLs,
|
||||
event bounds, and independently typed capture states for screenshot, DOM,
|
||||
@@ -248,13 +236,12 @@ network metadata observation; listing `network` in `events` is then optional.
|
||||
Each observed request object receives a capture-scoped `transaction_id`.
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Request events require both transaction fields;
|
||||
an absent ID is malformed capture data, while a null redirect predecessor is
|
||||
an explicit unknown.
|
||||
that no earlier hop existed. Older captures without IDs remain readable and
|
||||
their transaction association is unknown.
|
||||
|
||||
`scenario.network_content` records the resolved content selection. Comparing
|
||||
captures with different selections reports network evidence as unknown with a
|
||||
recovery instruction.
|
||||
captures with different selection, or legacy captures without that coverage
|
||||
information, reports network evidence as unknown with a recovery instruction.
|
||||
Capturing more data alone does not establish a website change.
|
||||
|
||||
`network-content` events reference the exact request or response event through
|
||||
@@ -262,13 +249,8 @@ Capturing more data alone does not establish a website change.
|
||||
states. Captured bodies retain base64 bytes, byte count, media type, and SHA-256
|
||||
of the retained bytes. Request bytes are those exposed by Playwright; a
|
||||
`not_exposed` request body does not prove absence or complete multipart file
|
||||
coverage. Response bytes are the result of Playwright's `Response.body()` API.
|
||||
Playwright may satisfy that API with a credentialed follow-up load of the
|
||||
request URL when CDP has no response body. Event and transaction references
|
||||
identify the associated Playwright response object, but do not verify that the
|
||||
retained bytes came from the original network transaction. Successful selected
|
||||
response bodies include this limitation in the capture. These bytes are not
|
||||
the compressed wire representation.
|
||||
coverage. Response bytes are decoded by the browser, not the compressed wire
|
||||
representation. No later fetch is substituted for the original response.
|
||||
Headers preserve producer order and duplicates. Authorization,
|
||||
Proxy-Authorization, Cookie, and Set-Cookie values are null with
|
||||
`redacted: true`; ordinary values remain intact. Declared UTF-8 secret values
|
||||
@@ -312,9 +294,3 @@ artifacts carry their normalized before/after digests and capture states.
|
||||
Every changed or unknown artifact record is returned inline for each aligned
|
||||
step. Missing, truncated, or mismatched capture coverage remains unknown
|
||||
instead of being treated as equality.
|
||||
|
||||
`scenario.event_families` records the sorted, de-duplicated `capture.events`
|
||||
selection. Captures with different selections report event evidence as unknown
|
||||
with a recovery instruction, even when no selected family emitted an event.
|
||||
Captures written before this field existed omit it, so their event evidence is
|
||||
also unknown.
|
||||
|
||||
@@ -0,0 +1,453 @@
|
||||
# Incremental capability migration
|
||||
|
||||
This working guide tracks the migration in #740. Update it as concrete domains
|
||||
migrate. Existing public contracts and domain-specific ports remain authoritative;
|
||||
there is no new universal provider interface or fixed module manifest.
|
||||
|
||||
## Ownership to preserve
|
||||
|
||||
| Responsibility | Current owner | Migration direction |
|
||||
| --------------------------------------------- | ----------------------------------- | ----------------------------------------------------- |
|
||||
| Analyst semantics and Evidence representation | domain | Keep pure and provider-neutral |
|
||||
| Named public input/output contracts | contracts | Keep exact CLI/MCP meaning and complete discovery |
|
||||
| Producer parsing and external tool protocols | provider adapters | Keep behind the relevant typed port |
|
||||
| Shared analyst workflows | application | Share between CLI and MCP |
|
||||
| Concrete provider construction | composition, MCP startup | Keep fresh typed factories at the production boundary |
|
||||
| Investigation Evidence and Unknowns | InvestigationRecords/EvidenceLedger | Consume narrow read/write/atomic record ports |
|
||||
| Target/profile snapshots and invalidation | BinarySessionRecords/BinarySession | Keep binary-owned |
|
||||
| Subprocess ownership and host primitives | process/windows | Reuse; preserve cancellation and cleanup |
|
||||
| Public translation | CLI/MCP adapters | Delegate to shared workflows and named contracts |
|
||||
|
||||
Retain exact Evidence IDs, provenance, detached reads, atomic Evidence/Unknown
|
||||
mutations and optimistic revisions. Preserve current close and failed-cleanup
|
||||
record clearing while extracting ownership; #721 owns any retention-policy change.
|
||||
Each CLI invocation and MCP connection keeps its own state. Provider registration
|
||||
must not launch engines or install prerequisites.
|
||||
|
||||
## Migrate before generalizing
|
||||
|
||||
1. Record the actual callers, state, environment, protocol and verification owner
|
||||
of the capability being migrated.
|
||||
2. Move concrete construction behind its existing typed port. Preserve injected
|
||||
ports, selected environment, queues, failure state and cleanup behavior.
|
||||
3. Test both caller paths and the applicable package/real-provider boundary.
|
||||
4. Use independently migrated capabilities to identify repeated metadata and
|
||||
composition needs. Derive a small source declaration only when those needs
|
||||
are established; keep unlike target/lifecycle semantics separate.
|
||||
5. Move a coherent capability's files, fixtures and verification lanes in a
|
||||
mechanical PR. Update all path-sensitive consumers in that same PR.
|
||||
|
||||
Pure helpers do not need an interface each. A capture-format adapter or typed
|
||||
factory may be sufficient. Avoid generic JSON execution, external MCP catalog
|
||||
passthrough, eager provider acquisition, new permission fields or a multi-package
|
||||
rewrite. Optional loading should preserve successful peers and report the failed
|
||||
capability without hiding tools from the catalog.
|
||||
|
||||
## Checks before and after a move
|
||||
|
||||
`npm run verify:test-discovery` compares actual Vitest file discovery with every
|
||||
repository-owned `*.test.ts` under src and tests, including newly added files. The
|
||||
check requires Git, includes untracked non-ignored tests, and excludes removed
|
||||
paths during a move. It
|
||||
rejects missing files and overlapping project ownership. It only lists files;
|
||||
it does not replace behavior tests or real-provider verification.
|
||||
|
||||
Check imports, project isolation and coverage, development-test selection,
|
||||
lint exceptions, catalog/generator imports, npm and CI verification entrypoints,
|
||||
fixture ownership, and installed-package bridge/native path resolution.
|
||||
Domain and contracts must not import application workflows, caller adapters or
|
||||
provider implementations. Production composition is a deliberate outer boundary.
|
||||
Other existing adapter imports will migrate incrementally; do not widen a lint
|
||||
exception to conceal new protocol code inside application workflows.
|
||||
|
||||
`npm run verify:module-boundaries` parses repository-owned source imports,
|
||||
reexports, type imports and literal dynamic imports, then checks their resolved
|
||||
source layer. Nested names such as `domain/android` are domain code; `src/android`
|
||||
is the provider adapter. The check runs with lint, including cached static gates.
|
||||
It covers tracked and untracked non-ignored TypeScript source and skips cached
|
||||
paths removed during a move. Pass exact source paths for a focused check.
|
||||
|
||||
Pure source and its tests keep inward dependencies. Application/server provider
|
||||
construction guards cover the migrated deep, Android, firmware, JavaScript recovery and observation
|
||||
implementations. Application workflows cannot import composition, including the
|
||||
former binary runtime entrypoint; shared browser capture/export helpers remain admitted while their
|
||||
ownership is reviewed. Test lane restrictions still use Oxlint. Producer-dependent
|
||||
browser and managed comparison tests live in boundary lanes with their original
|
||||
fixtures and assertions. This is a development guard, not runtime authorization.
|
||||
|
||||
Select verification by the claim changed: schema/composition, package, real
|
||||
engine or host-native workflow. Record host/target prerequisites and source-owned
|
||||
fixtures. Run heavy lanes serially with bounded workers/heaps and reuse existing
|
||||
engines. Recording ports do not prove real provider or platform support.
|
||||
|
||||
## Extension examples to validate during migration
|
||||
|
||||
An implementation of an existing port should require a provider adapter, typed
|
||||
production factory selection, exact producer parsing/provenance and matching
|
||||
verification. Existing workflow meaning and public contracts should stay stable.
|
||||
|
||||
A new analyst capability deliberately adds semantics, named contracts, a typed
|
||||
port/workflow when needed, both caller adapters, availability, documentation and
|
||||
verification. Its integration is broader than swapping a provider. The migration
|
||||
should eliminate incidental duplicate wiring, not erase those design decisions.
|
||||
|
||||
Update these examples with real migrated entrypoints before closing #740.
|
||||
|
||||
## First production factory migration
|
||||
|
||||
`src/composition/android.ts` and `src/composition/firmware.ts` construct providers
|
||||
behind the existing AndroidAnalysisPort and FirmwareAnalysisPort. CLI registrars
|
||||
pass their selected environment; MCP uses the process environment unless a caller
|
||||
injects a port. Each factory call returns a fresh instance and acquires no process,
|
||||
workspace or toolchain. Queues, cleanup-failure state and per-request cleanup stay
|
||||
inside the provider. The existing launcher seams remain available to boundary
|
||||
fixtures, which now exercise the production factories.
|
||||
|
||||
Application workflows and MCP registration must not import these concrete
|
||||
providers. Composition may import their implementations. Binary composition also
|
||||
uses this outer boundary after its direct-analysis callers were migrated.
|
||||
This concrete pilot does not introduce a shared provider interface or module
|
||||
manifest.
|
||||
|
||||
## Independent optional observation adapters
|
||||
|
||||
Browser observation, browser scenarios, Electron observation, Electron scenarios
|
||||
and V8 Inspector observation now have separate factories in `src/composition/`.
|
||||
CLI commands reuse them; MCP startup dynamically imports each factory independently.
|
||||
`OptionalObservationProviders` describes these five existing typed ports only. It
|
||||
is provisional startup wiring, not an extensible plugin manifest or a universal
|
||||
provider interface.
|
||||
|
||||
A failed import or constructor retains the successful peers. The complete tool
|
||||
catalog remains advertised. Session availability reports the failed adapter and
|
||||
its actual reason, and affected handlers return the existing capability-unavailable
|
||||
error. Explicit loading failures take precedence over configured availability.
|
||||
Passive Electron and active Electron failures are independent; static JavaScript
|
||||
analysis and runtime reconciliation remain usable without either adapter. CLI
|
||||
Electron commands also load only their selected runtime factory at execution.
|
||||
|
||||
Source identities are available without importing the optional implementations.
|
||||
Registration acquires no engines, probes no endpoints, changes no permissions and
|
||||
installs nothing. Core transport failures and shutdown still use their existing
|
||||
lifecycle. Fault-injection coverage uses the production loading/startup path and
|
||||
actual MCP SDK calls; recording ports establish composition behavior, while the
|
||||
real Inspector lane separately establishes runtime behavior.
|
||||
|
||||
## Investigation record ownership pilot
|
||||
|
||||
`src/application/investigation/InvestigationRecords.ts` owns one existing
|
||||
EvidenceLedger per runtime. The production session composition constructs that
|
||||
owner explicitly; the direct BinarySession constructor retains a fresh default
|
||||
for compatibility. There is no second ledger, global store, persistence or new
|
||||
retention policy.
|
||||
|
||||
The ledger, Unknown Evidence helper and the ledger's two original behavior tests
|
||||
now live beside the owner in `src/application/investigation/`. Bundle validation
|
||||
and record-owner consumers follow the new paths; retained-reference access still
|
||||
uses the narrow record ports. The move preserves implementation bodies and test
|
||||
assertions. Binary snapshot/cache files remain with their existing owner.
|
||||
|
||||
EvidenceReader, EvidenceWriter and EvidenceUnknownWriter describe existing
|
||||
read/write/atomic callback needs. UnknownRegistryPort preserves optimistic
|
||||
revisions and consistency verification. Non-binary MCP registrars use these
|
||||
record types without depending on the binary execution/lifecycle surface.
|
||||
BinarySessionPort retains its compatible record methods through composition.
|
||||
|
||||
BinarySessionRecords remains the snapshot owner and compatibility facade. It
|
||||
passes the actual active target explicitly for recordUnknown/updateUnknown;
|
||||
recordEvidenceWithUnknown keeps its target-free mutation subject. Snapshot import
|
||||
merges records without notifying midway, then emits after the cache commits.
|
||||
Bundle import retains the full change delta so Unknown-only changes notify even
|
||||
when recordsAdded is zero. Clearing resets records, cache and invalidation before
|
||||
notification; observer failures remain best effort after a committed mutation.
|
||||
|
||||
Close and failed-cleanup clearing, ordinary target-switch retention, detached
|
||||
reads, Evidence identities and deterministic bundle transfer keep their existing
|
||||
behavior. Snapshots still require an active target and concrete analysis profile;
|
||||
extracting investigation ownership does not make them a target-free record store.
|
||||
#721 remains responsible for any future lifetime-policy change.
|
||||
|
||||
## Inspector adapter ownership
|
||||
|
||||
`src/inspector/` owns V8 Inspector discovery, passive capture, script/target
|
||||
location interpretation and its lightweight provider identity. The composition
|
||||
factory and optional loader use this owner. Browser CDP connection, endpoint/value
|
||||
and Electron file-location helpers remain deliberate shared adapter utilities at
|
||||
their existing paths; no provider wire protocol moves into domain or process.
|
||||
|
||||
Loopback fixtures and producer boundaries now live under
|
||||
`tests/fixtures/inspector/` and `tests/boundary/inspector/`. The socket-backed
|
||||
unresolved-target reconciliation test moves from composition into the forked
|
||||
boundary lane with its original assertions. Adapter tests retain fork isolation.
|
||||
`npm run verify:inspector` keeps its public development entrypoint and delegates
|
||||
to `scripts/verify/inspector/runtime-observation.mjs`, using the real Node fixture
|
||||
under `tests/conformance/inspector/`. Catalog imports, source guards and test
|
||||
discovery follow the new paths. Observation authority, lifecycle, producer
|
||||
interpretation and caller-visible contracts stay unchanged.
|
||||
|
||||
## Auxiliary source declaration pilot
|
||||
|
||||
Artifact, macOS native and managed static metadata now belongs to each adapter's
|
||||
`*ProviderMetadata.ts` module. The implementation and
|
||||
`src/composition/auxiliaryAnalysisProviders.ts` reuse the same identity and exact
|
||||
capability descriptors. The declarations reference existing AnalysisProvider
|
||||
loaders; they cover these three disjoint auxiliary implementations only. Deep
|
||||
candidate selection and the five typed observation ports remain separate.
|
||||
|
||||
The binary runtime reads these source declarations while auxiliary construction
|
||||
stays lazy. The separate managed-only entrypoint retains direct construction for
|
||||
its execution-free workflow. The MCP catalog generator reads declarations without
|
||||
constructing implementations, using its canonical macOS platform; actual runtime
|
||||
host restrictions remain explicit.
|
||||
Discovery and binary operation-kind routing now consume canonical source contracts
|
||||
rather than their generated JSON artifact. No declaration imports its generated
|
||||
output. Generated catalog facts and public schemas stay unchanged.
|
||||
|
||||
All observation registrars resolve contracts by name. An isolated compiled-process
|
||||
test reverses presentation arrays and performs actual MCP SDK calls to verify
|
||||
advertised schemas and handler meaning. Reordering metadata must not select a
|
||||
different operation, schema, provider or deep-provider priority.
|
||||
|
||||
## Android layer and verification ownership
|
||||
|
||||
Android workflows and their existing typed port live in `src/application/android/`,
|
||||
pure APK/inventory semantics in `src/domain/android/`, and the named inspection
|
||||
contracts in `src/contracts/android/`. The headless JADX producer remains in
|
||||
`src/android/`; `src/composition/android.ts` supplies its fresh factory to both
|
||||
caller adapters. Public names, result meaning and JVM/queue/cleanup limits are
|
||||
unchanged. Inventory-only application projection remains execution-free.
|
||||
|
||||
The real lane and signal-cleanup helper belong to `scripts/verify/android/`; its
|
||||
explicit downloader and fixed manifest belong to `scripts/fixtures/android/`.
|
||||
`verify:android` and `fixtures:android` keep their command names and repository-root
|
||||
`_reference/apk-integration/` fixture default. Synthetic producer fixtures stay in
|
||||
`tests/fixtures/android/`, with matching boundary and composition ownership. Real
|
||||
JADX CLI/MCP parity requires the fixed APK and existing audited JAR; synthetic
|
||||
protocol/cancellation success does not establish that engine or an unverified host.
|
||||
|
||||
## Deep-provider source declarations
|
||||
|
||||
Hopper, Ghidra and IDA declare their implemented analyst operations in their
|
||||
own `*ProviderCapabilities.ts` modules. Hopper's inventory is typed against the
|
||||
existing direct-operation contracts; Ghidra reuses its inventory/function adapter
|
||||
operations; IDA retains its read-only operation family. Frozen operation arrays
|
||||
feed the existing capability builders without importing generated catalog data.
|
||||
|
||||
The catalog generator reads these source declarations and joins operation names
|
||||
to canonical `TOOL_CONTRACTS`. A missing contract identifies the declaring provider
|
||||
and operation. Public schemas and descriptions remain owned by contracts; the
|
||||
declarations describe implementation coverage. Provider adapters must not consume
|
||||
the generated MCP catalog that is derived from them. The resolved module guard
|
||||
checks this edge while admitting canonical source contracts, generated package
|
||||
metadata and the existing application snapshot-cache consumer.
|
||||
|
||||
Exact provider effects and lifecycle differences remain in their adapters:
|
||||
Hopper's GUI and mutation effects, Ghidra's experimental Windows authority and
|
||||
mutation restrictions, and IDA's attached/headless effects and live observations.
|
||||
This source increment introduces no common provider lifecycle or new interface.
|
||||
|
||||
## JavaScript application ownership
|
||||
|
||||
`src/application/javascript/` owns JavaScript/Electron artifact analysis and
|
||||
graph projection, feature/version/export/source comparisons, runtime Evidence
|
||||
and reconciliation, observation ports, and source recovery workflows. Pure
|
||||
analysis and public schemas keep their existing domain/contract ownership;
|
||||
shared artifact inventory, investigation records and execution types remain
|
||||
shared. CLI, MCP and production factories import the owning workflow directly.
|
||||
|
||||
`src/domain/javascript/` owns static parsing and semantic IR, application and
|
||||
semantic graphs, Electron facts, runtime reconciliation, recovery result
|
||||
semantics, and source-to-bundle comparisons. Shared Evidence, digests, ordering,
|
||||
comparison semantics and historical-source graphs stay at their existing owners.
|
||||
Native handoffs link exact-subject Evidence without selecting or executing a
|
||||
provider.
|
||||
|
||||
`src/contracts/javascript/` owns the JavaScript/Electron observation and recovery
|
||||
contracts, JavaScript-specific workflow requests, and their examples. The shared
|
||||
`applicationToolContracts.ts` aggregate stays at the contract root because it also
|
||||
owns reconstruction and Android/Apple projection operations. Canonical tool
|
||||
aggregation, named handler binding and SDK schema presentation remain unchanged.
|
||||
Catalog source paths, compiled verifier/SDK fixtures and recovery CI filters
|
||||
follow the domain/contract owners.
|
||||
|
||||
`scripts/verify/javascript/` groups application-workflow, runtime-reconciliation,
|
||||
large-digest and real-recovery entrypoints. Their existing npm commands retain
|
||||
their invocation semantics. Recovery fixture compilation lives beside its
|
||||
unchanged sources under `scripts/fixtures/javascript-recovery/`. Its default
|
||||
REA entrypoint still resolves to `scripts/rea.mjs`. Managed conformance imports
|
||||
the relocated compiled workflow; owning CI source filters follow the new paths.
|
||||
|
||||
`src/artifacts/javascript/` owns directory/ASAR reader construction and relevant
|
||||
file acquisition, including root/nested byte verification, UTF-8 diagnostics and
|
||||
nested reader cleanup. Its narrow factory returns the existing `ArtifactReader`;
|
||||
the application retains input admission, inventory, graph construction and its
|
||||
original root-reader `finally` close. Pure file facts live in
|
||||
`src/domain/javascript/javascriptArtifactFiles.ts`, and the shared immutable
|
||||
inventory snapshot lives in `src/domain/artifactInventorySnapshot.ts`. Inventory
|
||||
options and integrity policy remain application-owned. Shared stream hashing
|
||||
lives in `src/artifacts/ArtifactHash.ts`, preserving complete byte counts and the
|
||||
bounded classification prefix. Acquisition/hash guards reject imports from
|
||||
application, composition or caller adapters; JavaScript workflows use the factory
|
||||
instead of importing concrete ASAR/directory reader classes.
|
||||
|
||||
ZIP, native-DMG and Mach-O inventory reader selection keep their existing
|
||||
lifecycles. Runtime ports retain their distinct Inspector, passive Electron and
|
||||
active Electron effects; shipped Electron hooks remain at their existing
|
||||
provider boundary.
|
||||
|
||||
## Managed capability ownership
|
||||
|
||||
`src/domain/managed/` owns managed artifact and member semantics, reconstruction
|
||||
import, native-boundary verification and application graph projection.
|
||||
`src/application/managed/` owns their shared Evidence workflows, and
|
||||
`src/contracts/managed/` owns their named contracts and examples. Producer
|
||||
metadata/IL parsing stays in `src/dotnet/`; target resolution, Evidence, ordering
|
||||
and investigation records retain their shared owners. Path-based member
|
||||
comparison preserves its actual target/byte-digest admission and existing parser
|
||||
helper.
|
||||
|
||||
`scripts/verify/managed/` groups the portable conformance entrypoint and its
|
||||
support, manifest, oracle and completion-report helpers. Its PE byte fixture
|
||||
builder and declaration live in `scripts/fixtures/managed/`, shared with the
|
||||
installed-package lane and existing managed conformance tests. `verify:managed`
|
||||
keeps its command name and optional caller-cwd manifest resolution. Compiled
|
||||
catalog and generator imports follow the owners; document-check cache inputs
|
||||
include the relocated runtime verifier and fixture files.
|
||||
|
||||
Portable byte fixtures, optional operator-local manifests, actual ILSpy oracles
|
||||
and real Ghidra NativeAOT checks retain their distinct proof levels and
|
||||
prerequisites. The existing NativeAOT fixture/engine lane keeps its owner. This
|
||||
move preserves implementations, Evidence identities and generated commitments.
|
||||
|
||||
## Binary application ownership
|
||||
|
||||
`src/application/binary/` owns the active session, deep-provider registry and
|
||||
evaluation, operation routing, disjoint provider composition, lazy auxiliary
|
||||
clients, cancellation and client cleanup. Its records facade and snapshot
|
||||
cache/files remain bound to the exact target and analysis profile. Investigation
|
||||
records still have their separate owner in `src/application/investigation/`.
|
||||
|
||||
`AnalysisProvider.ts` stays shared because its identity/execution types also serve
|
||||
nonbinary ports. `BinaryTargetResolver.ts` stays shared because Android, firmware,
|
||||
managed and artifact workflows use its target parsing. Production wiring lives
|
||||
in `src/composition/binary.ts`; application workflows gain no outward composition
|
||||
dependency.
|
||||
|
||||
The three colocated application tests move with their owner. Composition cases
|
||||
remain in `tests/composition/analysis-sessions/`; filesystem and SDK cases retain
|
||||
their boundary lanes. The shared `tests/fixtures/binarySession.ts` factory supports
|
||||
several capability families. Existing recursive application globs discover the
|
||||
moved tests; bridge assets and real-verifier entrypoints remain at their owners.
|
||||
|
||||
## Firmware layer and verification ownership
|
||||
|
||||
Firmware workflows and their existing typed port live in
|
||||
`src/application/firmware/`, pure request/result semantics in
|
||||
`src/domain/firmware/`, and named contracts in `src/contracts/firmware/`.
|
||||
`src/firmware/` keeps producer commands, report parsing, source copies, publication
|
||||
and cleanup; `src/composition/firmware.ts` supplies a fresh typed factory.
|
||||
|
||||
The real lane belongs to `scripts/verify/firmware/analysis.mjs`; the fixture runner
|
||||
and its unchanged Python producer belong together in `scripts/fixtures/firmware/`.
|
||||
The npm command names and selected `REA_FIRMWARE_FIXTURE_ROOT` are unchanged. The
|
||||
runner retains its working-directory-relative default; the verifier retains its
|
||||
repository-root default. Both point to `_reference/firmware-integration/generated`
|
||||
when invoked through the documented npm commands. Optional ext4/Ghidra lanes keep
|
||||
their own prerequisites. Source-fixture generation and synthetic producer tests
|
||||
do not establish real Binwalk/Unblob analysis or another host's execution support.
|
||||
|
||||
## Apple artifact capability ownership
|
||||
|
||||
`src/domain/apple/` owns Apple application anatomy/projection, asset catalog
|
||||
facts, Interface Builder graphs, keyed archive semantics and plist values.
|
||||
`src/application/apple/AppleApplicationService.ts` retains the shared workflow
|
||||
that projects authenticated IPA/macOS inventory Evidence.
|
||||
|
||||
`src/artifacts/apple/` owns the actual producer boundary: directory acquisition,
|
||||
assetutil invocation, plist/NIB decoding, encoded view-parent interpretation and
|
||||
keyed archive inspection. These helpers are used by ArtifactProvider, retain
|
||||
its existing typed analysis boundary, injected assetutil seam and original reader
|
||||
cleanup, and do not require a new port for each helper. Producer code no longer
|
||||
lives in the application layer. Guards reject application/composition/caller
|
||||
imports from this owner while admitting pure facts and shared artifact readers.
|
||||
Adapter-source tests retain their assertions in the existing forked adapter lane.
|
||||
|
||||
`scripts/verify/apple/` groups archive, asset catalog, Interface Builder,
|
||||
Objective-C/Swift dispatch and macOS bundle verification. The macOS bundle fixture
|
||||
builder lives in `scripts/fixtures/apple/`; NIB byte fixtures stay beside their
|
||||
artifact decoder. Existing format-specific conformance sources and goldens keep
|
||||
their locations, and relocated verifiers resolve them against their own file
|
||||
URLs. Public npm entrypoints and CLI/MCP contracts stay stable; generated-document
|
||||
cache inputs include the relocated runtime scripts and fixture builder.
|
||||
|
||||
Portable decoder and injected producer tests establish their format/boundary
|
||||
claims. Actual Swift/Xcode, assetutil, native NIB, bundle signing and DMG lifecycle
|
||||
claims require the real Apple lane. Native binary API/value-flow semantics and
|
||||
shared cross-domain contract aggregates keep their separate owners.
|
||||
|
||||
## Binary production composition closure
|
||||
|
||||
`src/composition/binary.ts` is the shared production owner for deep candidate
|
||||
construction, lazy auxiliary composition and the managed-only session factory.
|
||||
It preserves the existing constructor bodies, selection policy and fresh session
|
||||
per invocation. Factory registration acquires no engine or target.
|
||||
|
||||
`DirectAnalysis` and `DirectAnalysisStatus` receive these two existing factories
|
||||
through `DirectAnalysisDependencies`. They retain configuration parsing, selected
|
||||
status environment, cancellation, snapshot replay/binding, Evidence projection
|
||||
and finally-close. The dependency uses the existing BinarySession type, including
|
||||
its replay policy; it adds no universal backend or lifecycle contract.
|
||||
|
||||
`src/composition/directAnalysis.ts` binds the production factories for CLI and
|
||||
one-shot managed MCP callers. Binding creates no session and stores no retained
|
||||
analysis state. MCP startup constructs its connection session through the same
|
||||
binary factory. Source and compiled fixtures use these entrypoints. The former
|
||||
application runtime path is removed, and the source guard no longer admits its
|
||||
outward provider/composition imports.
|
||||
|
||||
## Native analyst capability ownership
|
||||
|
||||
`src/domain/native/` owns portable native inspection, instruction/data types,
|
||||
load-image facts, API boundaries, annotations, metadata recovery, UI observation
|
||||
semantics and value/investigation graphs. `src/application/native/` owns shared
|
||||
API projection, call routes, dispatch inspection and UI/value tracing. Its tests
|
||||
keep the existing service lane. `src/contracts/native/` owns the named native
|
||||
adapter contracts; shared official/enhanced aggregates remain at their owners.
|
||||
|
||||
Producer interpretation, external commands and host UI lifecycle remain in
|
||||
`src/native/`, Hopper, Ghidra and IDA. The shared function dossier still lives in
|
||||
`domain/hopperValues.ts`: its historical name does not make its meaning exclusive
|
||||
to Hopper. This move retains existing typed ports, exact result schemas,
|
||||
observed/derived/unknown distinctions, profiles and Evidence links.
|
||||
|
||||
Source and compiled verifier imports follow the new owners. Host UI and real
|
||||
native-value verifier locations are retained until their own proof lanes migrate;
|
||||
interactive macOS UI success and real dependency tracing require their respective
|
||||
host/engine workflows. Portable source fixtures and injected call-route tests
|
||||
establish their narrower boundaries, without expanding provider/platform support.
|
||||
|
||||
## Process capture and analyst workflow ownership
|
||||
|
||||
`src/domain/process/` owns scenario/capture validation, portable process trees,
|
||||
trace specifications, observations, comparisons and the canonical process Evidence
|
||||
identity. The standalone contract example lives in `src/contracts/process/`;
|
||||
shared session contracts retain their broader investigation owner.
|
||||
|
||||
`src/process/capture/` owns the actual PTY boundary, terminal rendering, sampling,
|
||||
selected child environment, filesystem snapshots/effects, runtime path admission,
|
||||
event journal, settlement and resource cleanup. It reuses `src/process/` ownership
|
||||
primitives and the Windows host substrate. Its identity import follows the
|
||||
existing pure declaration directly. A source guard rejects outward application,
|
||||
composition and caller imports; no new provider or lifecycle interface is added.
|
||||
|
||||
`src/application/process/` retains Evidence projection and file-backed CLI
|
||||
capture/comparison workflows. Capture helper tests use the forked adapter lane;
|
||||
Evidence/host integration tests use the serial process-boundary lane. Their
|
||||
fixtures and assertions remain unchanged. Installed capability-probe URLs follow
|
||||
the new compiled adapter path, including the optional-dependency failure probe.
|
||||
|
||||
Windows PTY capture retains its existing unavailable outcome because descendant
|
||||
cleanup is not yet verified. Native Job Objects and a PTY binary do not establish
|
||||
that capture workflow. Real POSIX capture/cleanup and installed terminal/PTY module
|
||||
resolution require their actual host and package lanes; portable comparison
|
||||
fixtures establish their separate evidence semantics.
|
||||
-281
@@ -1,281 +0,0 @@
|
||||
# CLI and Evidence
|
||||
|
||||
Use the CLI for a direct inspection or a script. It uses the same application
|
||||
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
|
||||
separate process; save results when you want to use them in a later command.
|
||||
|
||||
## Run a command
|
||||
|
||||
After [installing REA](installation.md), use `rea`. You can also run a command
|
||||
without a global installation:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
Use an extracted application directory or an ASAR as the target. Static
|
||||
JavaScript analysis returns the application graph and Evidence directly.
|
||||
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
|
||||
files when neither `--provider` nor `--snapshot` is supplied. See
|
||||
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
||||
for results, integrity checks and coverage.
|
||||
|
||||
The complete application Evidence can be hundreds of megabytes for a real
|
||||
application. When an agent or a person reads the result, save it to a file and
|
||||
project a summary, a module page or one module from the saved Evidence:
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
module page. See
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
## Managed assemblies (.NET)
|
||||
|
||||
For a .NET PE/CLI assembly, inspect its identity and deployment form before
|
||||
inspecting members. These commands read metadata and Common Intermediate
|
||||
Language (CIL) without loading or executing the assembly and require no native
|
||||
analysis engine:
|
||||
|
||||
```bash
|
||||
rea inspect-managed-artifact /absolute/path/Assembly.dll --json
|
||||
rea inspect-managed-members /absolute/path/Assembly.dll --json
|
||||
rea inspect-managed-members /absolute/path/Assembly.dll --type Example.Parser --json
|
||||
```
|
||||
|
||||
Replace `Example.Parser` with an exact `full_name` from the member inspection,
|
||||
or a declaring type name you already know. Omit `--type` to inspect all members,
|
||||
or repeat it to select several types.
|
||||
Selection limits decoding to the selected types; member references remain an
|
||||
assembly-wide inventory, so the output can still be large.
|
||||
|
||||
Choose the workflow from the artifact's contents rather than its `.exe` or
|
||||
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
|
||||
unpack .NET single-file hosts or decode IL2CPP metadata. NativeAOT method bodies
|
||||
require native analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
|
||||
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
|
||||
table. This command does not require a native analysis provider. See
|
||||
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
|
||||
source-file handling and row-value semantics.
|
||||
|
||||
## Native analysis
|
||||
|
||||
Inspect embedded Go build metadata without installing Go or configuring a
|
||||
disassembler:
|
||||
|
||||
```bash
|
||||
rea inspect-go-binary /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The shared CLI/MCP workflow reports compiler version, modules, replacements,
|
||||
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
|
||||
for supported containers and absent or malformed metadata.
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), or the
|
||||
[IDA adapter](ida-provider.md), before analyzing a native target. Substitute
|
||||
your target, search text and function name or address in these examples:
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/program --provider ghidra --json
|
||||
rea search /absolute/path/to/program "search" --provider ghidra --json
|
||||
rea function /absolute/path/to/program main --provider ghidra --json
|
||||
rea decompile /absolute/path/to/program 0x1000 --provider ghidra --json
|
||||
rea xrefs /absolute/path/to/program 0x1000 --provider ghidra --json
|
||||
rea trace /absolute/path/to/program "search" --provider ghidra --json
|
||||
```
|
||||
|
||||
`analyze` and `inspect` share the native overview workflow. `function` returns
|
||||
a function dossier; `decompile` returns pseudocode. For assembly instructions
|
||||
alone, use `rea instructions`. A macOS `.app` bundle can be supplied directly.
|
||||
|
||||
Run `rea --help` or a command's `--help` for arguments and output options.
|
||||
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
|
||||
tools. Provider-specific target support is described in the
|
||||
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
|
||||
[raw binary guide](raw-binary.md), and
|
||||
[Windows Ghidra guide](windows-ghidra-p0.md).
|
||||
|
||||
## Choose a provider
|
||||
|
||||
List the available providers and their supported operations:
|
||||
|
||||
```bash
|
||||
rea providers --json
|
||||
rea capabilities --json
|
||||
```
|
||||
|
||||
These commands describe binary-session providers and auxiliary capabilities.
|
||||
For the complete MCP surface, use the connected server's tool list and the
|
||||
session's advertised availability; individual guides describe prerequisites.
|
||||
|
||||
With automatic selection, REA uses the single available provider that supports
|
||||
the target. When several providers support it, select one explicitly:
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/program --provider hopper
|
||||
```
|
||||
|
||||
Set `REA_ANALYSIS_PROVIDER` for a standing preference. An explicit `--provider`
|
||||
overrides it. In MCP, pass `provider_id` to `open_binary`:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/absolute/path/to/program",
|
||||
"provider_id": "hopper"
|
||||
}
|
||||
```
|
||||
|
||||
The selected provider remains bound to the session until an explicit switch
|
||||
or close. Provider failures are returned with their original reason. For an
|
||||
`ambiguous` selection error, choose from `details.candidate_ids`; for
|
||||
`provider_unavailable`, run `rea doctor --provider ID --json` to diagnose the
|
||||
selected engine. See [task readiness](installation.md#check-readiness-for-your-task)
|
||||
and [provider selection](mcp-contracts.md#identity-and-discovery).
|
||||
|
||||
The session reports work still in progress through `analysis_activity`.
|
||||
A client timeout can end its wait while the provider continues analyzing.
|
||||
`cleanup_incomplete` identifies resources whose shutdown or removal could not
|
||||
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
|
||||
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for import, client timeouts and recovery.
|
||||
|
||||
## Save and reuse analysis snapshots
|
||||
|
||||
A snapshot retains successful analysis results for later queries. REA reuses
|
||||
an exact result when the target bytes, operation, parameters, provider and
|
||||
settings match. Mutations and cursor-dependent calls are excluded from the
|
||||
cache. Snapshot files are local and use owner-only permissions.
|
||||
|
||||
Dispatching a metadata write invalidates the active session's snapshot cache,
|
||||
even if its reply fails: the provider may already have applied the write.
|
||||
Close and reopen the target before importing or saving another snapshot.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
related questions must still belong to the saved target. If those dependencies
|
||||
cannot be retained, the entire history is excluded instead of reverting the
|
||||
question to an earlier revision or disposition.
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/path/to/analysis/program.json
|
||||
# Repeat the same query to reuse its saved result.
|
||||
rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/path/to/analysis/program.json
|
||||
```
|
||||
|
||||
An exact CLI cache hit is normally read before starting a provider process.
|
||||
On Windows, Ghidra first admits the selected source through its native snapshot
|
||||
owner, so replay can start that provider.
|
||||
In MCP, `open_binary` accepts `snapshot_path` to import a snapshot atomically
|
||||
for its matching target; an MCP provider may still start before returning a
|
||||
cached result. `close_binary` accepts `snapshot_path` and optional
|
||||
`overwrite: true` to save before releasing provider resources. A failed save
|
||||
leaves the session open so the caller can resolve the output failure.
|
||||
|
||||
The MCP save receipt reports `primitive_entries`, `workflow_entries`, and
|
||||
`evidence_records` separately. Zero primitive bindings can still accompany retained workflow
|
||||
results and Evidence. These are cached observations, not a saved provider
|
||||
database: only eligible exact queries can reuse a result, and new or live
|
||||
queries can still require provider startup. The CLI uses the same snapshot
|
||||
format and preserves these records when loading and updating it.
|
||||
|
||||
## Import, export and compare Evidence
|
||||
|
||||
Evidence records retain artifact identity, source locations, observations,
|
||||
inferences and unresolved findings. Validate an existing bundle, export its
|
||||
canonical form, or compare two supplied bundles:
|
||||
|
||||
```bash
|
||||
rea evidence-import /absolute/path/to/evidence/bundle.json
|
||||
rea evidence-export /absolute/path/to/evidence/bundle.json /absolute/path/to/evidence/canonical.json
|
||||
rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/right.json
|
||||
```
|
||||
|
||||
Exports preserve an existing destination unless `--overwrite` is explicit.
|
||||
For application traces and version comparisons, see
|
||||
[JavaScript application workflows](javascript-application-workflows.md).
|
||||
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
|
||||
belong to one connection. Separate CLI calls consume full saved records.
|
||||
|
||||
## Import historical source
|
||||
|
||||
Import an older source tree to compare with the current artifact:
|
||||
|
||||
```bash
|
||||
rea import-reference-source /absolute/path/to/source
|
||||
```
|
||||
|
||||
The import records hashes and metadata for the supplied files, separately
|
||||
from observations of the current app. For a directory inside a Git checkout,
|
||||
it records the HEAD of the nearest enclosing repository; whether the working
|
||||
tree matches that commit remains unknown. It honors the source tree's `.gitignore`
|
||||
and applies default exclusions for common generated and dependency paths such
|
||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
||||
matched pattern and whether it came from project, default, caller, or sensitive
|
||||
path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||
patterns when you want to mark selected paths as sensitive; those patterns are
|
||||
reported separately as configured-secret exclusions.
|
||||
|
||||
Re-import saved graphs whose pattern-based exclusions lack `pattern` before
|
||||
using them in source-to-bundle comparisons.
|
||||
|
||||
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
||||
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
||||
does not infer module targets from replacement characters.
|
||||
|
||||
Historical-source import requires safe no-follow file opens on Linux or
|
||||
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
|
||||
or another supported host. See
|
||||
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
|
||||
for mapping identities, inferences and unknowns.
|
||||
|
||||
## Capture runtime behavior
|
||||
|
||||
Choose the guide for the target and intended interaction:
|
||||
|
||||
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
|
||||
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
|
||||
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
|
||||
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
|
||||
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
|
||||
|
||||
Runtime requests name the target and actions. Launched targets run with your
|
||||
user permissions; consult the chosen guide for host requirements and effects.
|
||||
|
||||
## Output and exit status
|
||||
|
||||
The default terminal format is TOON. Use `--json` when saving results for a
|
||||
JSON consumer. `--json` is indented on a terminal and compact when piped or
|
||||
redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
| Status | Meaning |
|
||||
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `0` | The operation completed. Its result may include partial evidence, warnings or unresolved questions. |
|
||||
| `1` | The operation could not complete. Structured output identifies invalid input, permissions, cancellation, timeouts or another failure when available. |
|
||||
| `128 + N` | Signal `N` ended the process, where the shell or runtime preserves the conventional signal-derived status. |
|
||||
|
||||
`setup --dry-run` returns `planned` and exits `0`; a cancelled setup also exits
|
||||
`0`. Setup returns `1` for `needs_confirmation` or `needs_human`.
|
||||
`doctor` returns `1` when required checks in its selected readiness scope fail;
|
||||
unavailable optional providers remain informational for unrelated tasks.
|
||||
|
||||
Enable `pipefail` in a supporting shell so a downstream formatter preserves
|
||||
REA's failure status:
|
||||
|
||||
```bash
|
||||
set -o pipefail
|
||||
rea inspect-artifact ./app.asar --json | jq . > inspection.json
|
||||
```
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user