Compare commits

...
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>morluto
a56f7bb668 chore(main): release rea-agents 6.4.0 (#1399)
* chore(main): release rea-agents 6.4.0

* docs(release): write contributor-aware 6.4.0 changelog

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-11 11:57:29 +08:00
morluto ee46323b49 fix(windows): preserve drive-root private runtime paths
Join the admitted parent and generated child name without duplicating a drive-root separator. Reuse that name for handle-relative creation and path provenance.

Add native snapshot, readback, and cleanup coverage through the Ghidra dot-prefixed temp fallback, plus a session-root contract regression.

Fixes #1690
2026-10-11 11:51:39 +08:00
morluto a99857df23 test(release): keep credits attached to their contribution 2026-10-11 11:51:17 +08:00
morluto 6b043dc3de fix(release): avoid duplicated contributor thanks 2026-10-11 11:49:49 +08:00
morluto 4003151ec1 fix(ci): allow browser runtime action transport latency 2026-10-11 11:45:46 +08:00
92dac97102 docs(website): add beginner help to the native analysis guide (#1689)
Add a short glossary, an analysis-tool chooser linked to native setup,
a note that REA does not include DX-Ball, and help for copying a file's
full path.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:43:06 -07:00
YsopwnableandGitHub 89ea2cad19 perf(domain): reuse canonical JSON key encodings
Reuse exact JSON key encodings within the existing 8,192-code-unit digest-buffer quantum and avoid ordinary member value frames. Preserve canonical bytes and cooperative yielding; resume objects when a pending property is omitted.

Flameox paired experiments establish a local digest-workload improvement, with complete-result/EvidenceID oracles unchanged. Full CLI acceleration and billed token reductions are not claimed.

Refs #1065.
2026-10-11 11:40:47 +08:00
ea06264a56 docs(website): list the setup guide first on the Guides page (#1688)
Move the "Set up REA" entry ahead of the native, JavaScript and browser
guides so new readers see setup before the worked examples.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:33:30 -07:00
morluto 59b4a3c319 test(windows): verify native addon in a renamed Node host 2026-10-11 11:32:07 +08:00
xeropresenceandGitHub ea1c164e5e fix(windows): load native controls in Node-API hosts not named node.exe (#1531)
fix(windows): resolve Node-API imports from the host executable

Delay-load the Node-API import library and bind "node.exe" to the running
process, matching node-gyp's win_delay_load_hook. The addon now loads under
any Node-API host (Node.js, Bun) regardless of its executable file name.
2026-10-11 11:30:41 +08:00
dc676b4d85 docs(website): add beginner help to the Get started page (#1684)
Add collapsible help sections for installing Node.js, choosing a coding
agent, opening a terminal, restarting a client, copying a folder's full
path and finding an Electron app's .asar file.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 11:27:12 +08:00
ad3e41d46e fix(browser): resolve XLink and SVG prefixes in XHTML dom_urls (#1683)
* fix(browser): resolve XLink and SVG prefixes in XHTML dom_urls

DOMSnapshot has no namespace URI. HTML documents still use the parser's
local names. In XHTML and other XML documents, resolve in-scope xmlns
declarations so an element is SVG only in the SVG namespace, and report
any XLink href as href when plain href is absent.

Refs #1586

* fix(browser): bound XML namespace resolution and reject cyclic ancestry

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-11 11:26:26 +08:00
YsopwnableandGitHub 17f704050f feat(evidence): discover retained records without full payloads (#1682)
* feat(evidence): discover retained records without full payloads

* fix(evidence): guide oversized results to retained discovery

* style(tests): format retained discovery recovery assertion
2026-10-11 11:18:26 +08:00
c8820e03db fix(apple): decode binary app Info.plist files without plutil (#1476)
* fix(apple): decode binary app Info.plist files without plutil

* test(apple): use Windows-legal names in the binary plist bundle test

* fix(apple): ignore __proto__ entries in binary app Info.plist files

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-11 11:14:15 +08:00
morluto ab80ec640a fix(ci): report package verifier stage progress 2026-10-11 11:11:38 +08:00
morluto 9ce6ef895e test: reuse stdio MCP server in acceptance file 2026-10-11 11:11:38 +08:00
morluto b3e977e6d0 fix(browser): bound screenshot comparison memory 2026-10-11 11:11:38 +08:00
morlutoandYou Wish a99b2fd3b5 test: retain focused CommonJS export regressions
Keep uncovered assignment and binding regressions in the shared CLI and MCP runtime oracle, with a POSIX escaped-path fixture.

Co-authored-by: You Wish <29498139+Y0UW1SH@users.noreply.github.com>
2026-10-11 11:07:15 +08:00
Rudy Celekliandmorluto 4e4b15b825 fix(browser): validate PNG chunks and preserve unsupported encodings
Validate chunk checksums, type bytes, ordering and end boundaries before pixel comparison. Distinguish malformed PNGs and corrupt DEFLATE streams from recognized unsupported encodings through CLI and MCP error projection.

Adapted from #1375 with additional decoder and public boundary regressions. Eighteen decoder tests and six compiled CLI/MCP cases passed.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-11 11:07:15 +08:00
morluto 21dbcf7a14 perf(cli): defer binary provider composition 2026-10-11 11:07:15 +08:00
Kaoru0822andmorluto 76296c5a38 fix(native): bind NativeAOT metadata to declared PE section extents
Section headers distinguish the declared virtual extent from the on-disk raw extent, which may include alignment padding. Resource RVA ownership and overlap use VirtualSize, falling back to SizeOfRawData when VirtualSize is zero, and payload reads still require file-backed bytes.

The NativeAOT snapshot reader applies the same range rule to RVA reads, metadata scans, executable targets, and derived overlay bounds. Page-rounded loader mappings may expose tail bytes, but those bytes lie outside the declared section range used by these parsers.
2026-10-11 11:07:15 +08:00
morlutoandYou Wish e38550462e test(native): stabilize resource-limited binary layout controls
Keep the configured launcher alive while its limited Python child runs so ownership checks retain the admitted executable identity. Trigger memory exhaustion with one oversized ELF zero-fill tail rather than racing section traversal against a timeout.

Adapted from #1357. JavaScript syntax, strict C syntax compilation, formatting, and static checks passed. The Linux pwntools verification lane was not run on this macOS host.

Co-authored-by: You Wish <29498139+Y0UW1SH@users.noreply.github.com>
2026-10-11 11:06:10 +08:00
morlutoandYou Wish 80896a1d64 fix(process): settle launcher exit without dropping descendant lineage
Refresh a mismatched launcher snapshot only when a new live process table proves its PID absent. Preserve uncertainty for detached descendants and continue checking ownership of surviving group members before signaling. Index retained lineage membership instead of rescanning it for each refreshed process.

Adapted from #1357. Validation: 71 source tests passed, one Linux-only identity case skipped; 32 real process-boundary tests passed on macOS.

Co-authored-by: You Wish <29498139+Y0UW1SH@users.noreply.github.com>
2026-10-11 11:06:10 +08:00
71 changed files with 13320 additions and 456 deletions
+18 -11
View File
@@ -25,6 +25,7 @@ node scripts/release-notes.mjs inventory \
--manifest /tmp/rea-VERSION.contributions.json --manifest /tmp/rea-VERSION.contributions.json
node scripts/release-notes.mjs record \ node scripts/release-notes.mjs record \
--manifest /tmp/rea-VERSION.contributions.json \ --manifest /tmp/rea-VERSION.contributions.json \
--summary \
--output /tmp/rea-VERSION.record.md --output /tmp/rea-VERSION.record.md
``` ```
@@ -56,25 +57,31 @@ version/date/link heading. Use single-line Markdown list items under:
PR/commit references required by the existing checkpoint validator. PR/commit references required by the existing checkpoint validator.
Group related work by analyst workflow or user impact. Explain the resulting Group related work by analyst workflow or user impact. Explain the resulting
behavior; internal cleanup, tests, CI, and routine documentation generally behavior in short, concrete entries. Highlights summarize the release; avoid
belong only in the contribution record. Read ambiguous PRs/diffs before making repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
claims. Distinguish simulated-provider checks from real-provider verification. claims. Distinguish simulated-provider checks from real-provider verification.
Each bullet covering a PR carries its link and `Thanks @...` for its verified Credit each represented contribution once in the public notes with its link
human author, known co-authors, and confirmed closing-issue reporters. Grouping and `Thanks @...` for its verified human author, known co-authors, and confirmed
must preserve **all** those credits in the same bullet. Include the issue closing-issue reporters. Grouping preserves all those credits in one line.
links. Direct-commit bullets carry the commit link and known human credit. When Highlights, detailed entries or migrations reference the same work,
put its thanks in one of those entries, not all of them. Include issue links.
Direct-commit bullets carry the commit link and known human credit.
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
type `User`; the credit policy excludes them while retaining their metadata. type `User`; the credit policy excludes them while retaining their metadata.
Do not copy OpenClaw's maintainer-specific exclusions. At the REA maintainer's request, omit `@morluto` self-thanks while preserving
authorship in the inventory. Other human contributors remain eligible.
Use explicit GitHub links for cross-repository references to avoid ambiguous Use explicit GitHub links for cross-repository references to avoid ambiguous
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit. `#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
Append the generated `### Complete contribution record` verbatim as the last Append the generated `record --summary` provenance link verbatim at the end
section of this release. It accounts for every discovered in-range PR and of this release. The linked inventory accounts for every discovered in-range
direct contribution, including internal work omitted from the prose. Do not PR and direct contribution, including internal work omitted from the prose.
hand-edit generated credits to make validation pass. Refresh the inventory Do not paste the full `record` output into public notes: it duplicates thanks
and obscures the user-facing changes. Full records remain available for review.
Do not hand-edit generated credits to make validation pass. Refresh the inventory
when product history changes; a prose-only correction can reuse the snapshot. when product history changes; a prose-only correction can reuse the snapshot.
## Finalize with Release Please ## Finalize with Release Please
@@ -135,6 +135,16 @@ session to a selected path. The analysis remains complete in the ledger; avoid
repeating it merely to request the same oversized response. Complete CLI JSON repeating it merely to request the same oversized response. Complete CLI JSON
output also streams without a single MCP frame. output also streams without a single MCP frame.
When the retained parent ID is unknown, use `get_evidence_bundle` with
`detail: "summary"` and exact filters for operation, target SHA-256,
analysis-profile digest or canonical procedure address. Select its returned
Evidence ID for the next view instead of requesting the complete bundle.
`retention: "complete-record"` means the whole record is stored; native
value-flow truncation and unknown metadata remain separate. For an exported
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
`detail: "summary"` and the same `filters`. Reading that file still requires
validating its full contents within the local memory budget.
A client can also truncate a successful tool result without a REA transport A client can also truncate a successful tool result without a REA transport
error. Treat that preview as incomplete; use the retained parent ID for smaller error. Treat that preview as incomplete; use the retained parent ID for smaller
views or export when the full record is needed. Do not parse the preview as views or export when the full record is needed. Do not parse the preview as
+1 -1
View File
@@ -1,3 +1,3 @@
{ {
".": "6.3.0" ".": "6.4.0"
} }
+64
View File
@@ -1,5 +1,69 @@
# Changelog # Changelog
## [6.4.0](https://github.com/morluto/rea/compare/rea-agents-6.3.0...rea-agents-6.4.0) (2026-10-11)
### Highlights
- Android: added device inspection, split-APK acquisition, logs, screenshots, file transfers and explicit app lifecycle actions through ADB, plus manifest and localized-string decoding with Apktool. ([#1552](https://github.com/morluto/rea/pull/1552), [#1556](https://github.com/morluto/rea/pull/1556)) Thanks @3QL.
- JEB: browse project units and decompile selected items through a caller-started local JEB MCP server. ([#1536](https://github.com/morluto/rea/pull/1536)) Thanks @3QL.
- Large analyses: discover retained Evidence without loading every payload, and inspect JavaScript graphs, binary layouts and function dossiers through summaries and selected views. ([#1682](https://github.com/morluto/rea/pull/1682), [#1568](https://github.com/morluto/rea/pull/1568), [#1570](https://github.com/morluto/rea/pull/1570), [#1345](https://github.com/morluto/rea/pull/1345), [#1507](https://github.com/morluto/rea/pull/1507), [#1247](https://github.com/morluto/rea/issues/1247)) Thanks @jhuang-tw, @kaoru0822-kitauji, @N0zoM1z0.
- JavaScript: isolate source analysis in an owned process with configurable heap and timeout limits; retain completed facts and explicit unknowns when analysis runs out of resources. ([#1651](https://github.com/morluto/rea/pull/1651), [#1563](https://github.com/morluto/rea/issues/1563), [#1565](https://github.com/morluto/rea/issues/1565)) Thanks @N0zoM1z0, @peter-research.
- Ghidra: configure import language/compiler specifications and seed functions, code and labels before auto-analysis. ([#1667](https://github.com/morluto/rea/pull/1667)) Thanks @sqmagellan.
- RPC discovery: recognize supported tRPC calls, HTTP-link URLs and named GraphQL operations in JavaScript applications and web bundles. ([#1670](https://github.com/morluto/rea/pull/1670)) Thanks @pompiliodonatoapp.
### Changes
- Added Ghidra support for Windows PE DLLs and bounded MIPS ELF analysis, including carved MIPS inputs with incomplete section tables. Windows P0 remains read-only. ([#1303](https://github.com/morluto/rea/pull/1303), [#1362](https://github.com/morluto/rea/pull/1362), [#1629](https://github.com/morluto/rea/pull/1629), [#1290](https://github.com/morluto/rea/issues/1290), [#718](https://github.com/morluto/rea/issues/718), [#1619](https://github.com/morluto/rea/issues/1619)) Thanks @bqw371, @jaelliot, @kvalls, @N0zoM1z0.
- Enabled `set_address_name` and `set_addresses_names` for Ghidra sessions with mutation authority, including namespace-aware data labels and per-address batch results. ([#1635](https://github.com/morluto/rea/pull/1635), [#1634](https://github.com/morluto/rea/issues/1634)) Thanks @giri256.
- Recovered additional signed AArch64 relative jump-table patterns. ([#1592](https://github.com/morluto/rea/pull/1592)) Thanks @RishTheFish21.
- Added PE resource inspection and read-only NativeAOT metadata parsing for supported x64 PE32+/.NET 8 RTR 9.1 inputs. The NativeAOT parser’s Windows-host Ghidra workflow remains unverified on a real Windows host. ([#1482](https://github.com/morluto/rea/pull/1482), [76d8f68](https://github.com/morluto/rea/commit/76d8f68cb918dfb43869f2b649a0b6c0c1cf266d), [#1254](https://github.com/morluto/rea/issues/1254)) Thanks @N0zoM1z0, @thangldw.
- Added Pi, Hermes and Qwen Code setup, including Hermes active-profile support. ([#1376](https://github.com/morluto/rea/pull/1376), [#1403](https://github.com/morluto/rea/pull/1403), [#1415](https://github.com/morluto/rea/pull/1415), [#1409](https://github.com/morluto/rea/issues/1409)) Thanks @rasatpetabit, @yiliang114, @zaydiscold.
- Added `REA_MCP_INPUT_SCHEMA_PROFILE=compact` for size-capped MCP clients such as Kimi/Moonshot, while retaining canonical runtime validation. ([#1514](https://github.com/morluto/rea/pull/1514), [#1580](https://github.com/morluto/rea/pull/1580), [#1484](https://github.com/morluto/rea/issues/1484), [#1581](https://github.com/morluto/rea/issues/1581)) Thanks @haythemsellami, @N0zoM1z0.
- Added live process-capture progress and optional `finalization_ms` to capture shutdown output before timeout escalation; cancellation remains immediate. ([#1483](https://github.com/morluto/rea/pull/1483), [#1583](https://github.com/morluto/rea/pull/1583), [#1252](https://github.com/morluto/rea/issues/1252), [#1253](https://github.com/morluto/rea/issues/1253), [modelcontextprotocol/typescript-sdk#2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997)) Thanks @N0zoM1z0, @nikolay-dementiev, @sathariels.
- Improved Codex onboarding and recovery from observed client, PATH and provider blockers. ([#1613](https://github.com/morluto/rea/pull/1613)) Thanks @N0zoM1z0.
- Reduced redirected JSON and terminal output size, and deferred heavy imports until their commands need them. ([#1413](https://github.com/morluto/rea/pull/1413), [#1450](https://github.com/morluto/rea/pull/1450), [#1439](https://github.com/morluto/rea/pull/1439), [21dbcf7](https://github.com/morluto/rea/commit/21dbcf7a1458219989ad291e44ce7d017378b727)) Thanks @kaoru0822-kitauji.
### Fixes
- Electron analysis: follow renamed and default-imported APIs and optional calls, ignore overwritten identities and media preload attributes, and resolve packaged `app.asar` pages and unpacked companions. ([#1391](https://github.com/morluto/rea/pull/1391), [#1393](https://github.com/morluto/rea/pull/1393), [#1400](https://github.com/morluto/rea/pull/1400), [#1664](https://github.com/morluto/rea/pull/1664), [#1673](https://github.com/morluto/rea/pull/1673)) Thanks @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @suninrain086.
- JavaScript data flow: preserve returned/yielded reference uncertainty, invalidate property-stored aliases correctly, and retain bundled-module runtime scope. ([#1437](https://github.com/morluto/rea/pull/1437), [#1445](https://github.com/morluto/rea/pull/1445), [#1471](https://github.com/morluto/rea/pull/1471), [#1577](https://github.com/morluto/rea/pull/1577), [#1428](https://github.com/morluto/rea/issues/1428), [#1442](https://github.com/morluto/rea/issues/1442), [#1449](https://github.com/morluto/rea/issues/1449), [#1576](https://github.com/morluto/rea/issues/1576)) Thanks @CassieFox23, @dakjdakd, @N0zoM1z0.
- JavaScript packages: resolve exact exports subpaths, preserve trailing-slash directory requests and exact CommonJS keys, and respect shadowed `require` bindings. ([#1424](https://github.com/morluto/rea/pull/1424), [#1615](https://github.com/morluto/rea/pull/1615), [5485483](https://github.com/morluto/rea/commit/5485483f7da2f6cb66441bb784aaf2a1a5f8731a), [8b2f851](https://github.com/morluto/rea/commit/8b2f8514c50e469c3add8ce9ba7ebebef888b14f), [f998fcb](https://github.com/morluto/rea/commit/f998fcb1cca9994c3589fb70d6e0127fc215f76f), [1e51c73](https://github.com/morluto/rea/commit/1e51c732bbf08191688d6526bf58ab2650b601e3), [#1365](https://github.com/morluto/rea/issues/1365)) Thanks @akshitkhokhani, @AWUQAQ, @immanuwell, @rudycelekli.
- Large JavaScript inputs: handle oversized inline source maps, deep returns and reassignment loops; avoid argument expansion limits and yield during large analysis passes. ([#1386](https://github.com/morluto/rea/pull/1386), [#1368](https://github.com/morluto/rea/pull/1368), [#1390](https://github.com/morluto/rea/pull/1390), [#1539](https://github.com/morluto/rea/pull/1539), [#1525](https://github.com/morluto/rea/pull/1525), [#1542](https://github.com/morluto/rea/pull/1542), [#1545](https://github.com/morluto/rea/pull/1545), [#1557](https://github.com/morluto/rea/pull/1557), [#1380](https://github.com/morluto/rea/issues/1380), [#1401](https://github.com/morluto/rea/issues/1401), [#1538](https://github.com/morluto/rea/issues/1538), [#1462](https://github.com/morluto/rea/issues/1462)) Thanks @cold-atom, @KadirJeager, @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @rudycelekli.
- JavaScript receivers: expand escaped methods once, index returned-object targets, and avoid replaying earlier writes or duplicating retained graph allocations. ([#1517](https://github.com/morluto/rea/pull/1517), [#1537](https://github.com/morluto/rea/pull/1537), [#1444](https://github.com/morluto/rea/pull/1444), [#1601](https://github.com/morluto/rea/pull/1601), [#1495](https://github.com/morluto/rea/issues/1495)) Thanks @CassieFox23, @geekwolf, @kaoru0822-kitauji, @N0zoM1z0, @zeynelacikgoez.
- Browser captures: retain WebSocket URLs, empty connections and frame order; compare only common known payload coverage and preserve incomplete WebMCP replay coverage. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988), [aa257ea](https://github.com/morluto/rea/commit/aa257ea4b16f2d2a35c12df63359d650b48b1154), [e52c0a7](https://github.com/morluto/rea/commit/e52c0a72c59c740315e89149881a27ce4d89a11c))
- Browser scenarios: seed storage in the correct partition, normalize loopback attach identities, retain declared-secret redaction and require verified lineage before Windows Electron signals. ([#1604](https://github.com/morluto/rea/pull/1604), [#1590](https://github.com/morluto/rea/pull/1590), [#1588](https://github.com/morluto/rea/pull/1588), [8b4c3de](https://github.com/morluto/rea/commit/8b4c3de3b3a3b580adec266f94f29ccedf1da228), [#1497](https://github.com/morluto/rea/issues/1497), [#1591](https://github.com/morluto/rea/issues/1591), [#1589](https://github.com/morluto/rea/issues/1589)) Thanks @Adolanium, @akushonkamen, @dakjdakd.
- Added Termux Android browser-scenario support. ([#1655](https://github.com/morluto/rea/pull/1655)) Thanks @tyloryang.
- Web inputs: accept BOM-prefixed HAR, preserve the first duplicate DOM attribute and resolve SVG/XLink URLs in HTML and XHTML. ([#1509](https://github.com/morluto/rea/pull/1509), [#1516](https://github.com/morluto/rea/pull/1516), [#1602](https://github.com/morluto/rea/pull/1602), [#1683](https://github.com/morluto/rea/pull/1683), [#1496](https://github.com/morluto/rea/issues/1496), [#1586](https://github.com/morluto/rea/issues/1586)) Thanks @CassieFox23, @kaoru0822-kitauji, @sathariels, @taljeri.
- Screenshot comparisons: validate PNG chunks, distinguish corruption from unsupported encodings and bound comparison memory. ([4e4b15b](https://github.com/morluto/rea/commit/4e4b15b8250b29e176d4191c9610a3931b693474), [b3e977e](https://github.com/morluto/rea/commit/b3e977e6d073ab1783cece8581279f87fc360411)) Thanks @rudycelekli.
- Archives: preserve case-distinct names, report collisions at their diverging segment, detect nested formats from bytes and reject Windows alternate-stream extraction paths. ([#1480](https://github.com/morluto/rea/pull/1480), [#1505](https://github.com/morluto/rea/pull/1505), [#1553](https://github.com/morluto/rea/pull/1553), [573598c](https://github.com/morluto/rea/commit/573598c6324698cc48ceafd24ed3864670e08d00), [#1470](https://github.com/morluto/rea/issues/1470), [#1491](https://github.com/morluto/rea/issues/1491)) Thanks @CassieFox23, @kaoru0822-kitauji.
- Cleanup: retain failed readers, ASAR handles, output trees and provider workers for retry without discarding completed output or partial observations. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [de0d341](https://github.com/morluto/rea/commit/de0d341524dc68250a45b41eb7e49fcecc1e32a9), [3f8c51e](https://github.com/morluto/rea/commit/3f8c51e9b506d88fc41aded7b1a22b9a09037c26), [d1235fa](https://github.com/morluto/rea/commit/d1235fadc901e1fb9ab3149d53063306c8545414), [026126e](https://github.com/morluto/rea/commit/026126eafe741e5a8b857433587fd615cdd44c19))
- Providers: canonicalize IDA workspace roots before identity checks, respect IDA cancellation, serialize JEB acquisition and correct Ghidra no-return/imported-symbol recovery. ([#1481](https://github.com/morluto/rea/pull/1481), [#1419](https://github.com/morluto/rea/pull/1419), [5f30ea6](https://github.com/morluto/rea/commit/5f30ea6ff2dd89aab7af6da6e8dfa5e508ed459b), [f88c990](https://github.com/morluto/rea/commit/f88c99021b88ce67e513025901f57478d8107d0c), [49e6f00](https://github.com/morluto/rea/commit/49e6f00ed08b225abf9ddb7c762553607fdab115)) Thanks @AS9929, @JonahSimon, @Y0UW1SH.
- Binary metadata: respect PE/.NET section extents, bound binary-plist expansion, scope keyed-archive traversal and decode app `Info.plist` files without `plutil`. ([#1678](https://github.com/morluto/rea/pull/1678), [#1515](https://github.com/morluto/rea/pull/1515), [#1378](https://github.com/morluto/rea/pull/1378), [#1476](https://github.com/morluto/rea/pull/1476), [975dff2](https://github.com/morluto/rea/commit/975dff2e25b14b634e14dd5e0db1537d0ca9815b), [76296c5](https://github.com/morluto/rea/commit/76296c5a3868ba3b54be6ed3896719283d6de5f8)) Thanks @jayhemnani9910, @kaoru0822-kitauji, @rudycelekli.
- Windows runtimes: preserve drive-root paths when creating private provider directories. ([ee46323](https://github.com/morluto/rea/commit/ee46323b493994902b04824a7dc8aae2a8117903))
- Compatibility: load Windows native controls under Node-API hosts with renamed executables, preserve dylib directory requirements, support Wakaru 1.x and reject directory-valued Hopper launchers. ([#1531](https://github.com/morluto/rea/pull/1531), [#1623](https://github.com/morluto/rea/pull/1623), [#1540](https://github.com/morluto/rea/pull/1540), [#1426](https://github.com/morluto/rea/pull/1426)) Thanks @evenafterallisaid, @Fazmin, @pionxzh, @xeropresence.
- Reference imports: normalize internal symlink lookup consistently, preserve exact paths and exclusion provenance, report root access failures and reject FIFO replacements without blocking. ([#1535](https://github.com/morluto/rea/pull/1535), [2ca3ad2](https://github.com/morluto/rea/commit/2ca3ad238608adfca80a7ff1aaadea6e59a58225), [6636045](https://github.com/morluto/rea/commit/6636045c9c5b472bba51afee9d434394861e93a4), [3967cfa](https://github.com/morluto/rea/commit/3967cfa43e786c65a57e89c265343983d19ef9d0), [f1c58cc](https://github.com/morluto/rea/commit/f1c58cc53fbd83b968e11b3e2eccebca51299c54)) Thanks @anujsingh-cse.
- Setup: honor active configuration and skill roots, preserve configured MCP environments, validate Hermes/OMP profiles before writing, recognize Omarchy and retain recovery backups. ([#1504](https://github.com/morluto/rea/pull/1504), [#1607](https://github.com/morluto/rea/pull/1607), [#1528](https://github.com/morluto/rea/pull/1528), [#1562](https://github.com/morluto/rea/pull/1562), [#1662](https://github.com/morluto/rea/pull/1662), [a011485](https://github.com/morluto/rea/commit/a01148553e2a37c72ae78405418ab648876a96a6), [#1606](https://github.com/morluto/rea/issues/1606), [#1521](https://github.com/morluto/rea/issues/1521), [#1561](https://github.com/morluto/rea/issues/1561)) Thanks @Adkid-Zephyr, @CassieFox23, @LuisTaffelli, @N0zoM1z0, @peter-research.
- Process capture: retain descendant lineage after launcher exit, preserve identity uncertainty before signaling and release terminals after observation failures. ([80896a1](https://github.com/morluto/rea/commit/80896a1d64b216ef893907249cd4252f86215d49), [c916850](https://github.com/morluto/rea/commit/c9168508527302ac57bc86385ccf0d824697b5dc)) Thanks @Y0UW1SH.
- CLI/MCP errors: accept JSON files for web comparisons, preserve native option help and union correction details, remove invalid aliases and validate doctor against its discovered output contract. ([#1397](https://github.com/morluto/rea/pull/1397), [#1395](https://github.com/morluto/rea/pull/1395), [#1425](https://github.com/morluto/rea/pull/1425), [#1461](https://github.com/morluto/rea/pull/1461), [8b4c96e](https://github.com/morluto/rea/commit/8b4c96e6c6b3ed6806e4cd72b556b5db6114304a), [#1421](https://github.com/morluto/rea/issues/1421), [#1454](https://github.com/morluto/rea/issues/1454)) Thanks @Fazmin, @kaoru0822-kitauji, @N0zoM1z0.
### ⚠ BREAKING CHANGES
- WebSocket results: use `network.websocket_connections` instead of `network.websocket_events`. Connections contain `request_id`, `url` and `events`; allow `payload_bytes: null` when size is unknown. No legacy alias remains. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988))
- Direct resource APIs: pass a caller-owned `ArtifactResourceScope` to inventory/extraction, JavaScript analysis, reference imports and web exports; retain it through cleanup/shutdown. For `readJavaScriptArtifactFiles`, scope is argument three and abort signal is argument four. Keep each root inventory source’s single cleanup owner. Use `OwnedFileHandle` owners, injected ASAR `fd`/`read` handles instead of `createReadStream`, VCS `Result` values, and `WebScriptExportError`’s options object. CLI/MCP inputs for the scoped operations are unchanged. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [53e7fad](https://github.com/morluto/rea/commit/53e7fadbd078422ba3d55b5499a70d237c5f4908), [684199b](https://github.com/morluto/rea/commit/684199ba0be7822266d57ddb8a40e95f087fdc42), [be75ed5](https://github.com/morluto/rea/commit/be75ed5e4b4bfb146af497f8fa5d172a1ea19462), [be6272f](https://github.com/morluto/rea/commit/be6272f26115bd8e94f2ab0df4cf092004587a18), [e0e49e9](https://github.com/morluto/rea/commit/e0e49e9dd53d9d1f1d2330b23f7f880dc1c81774), [95a77b6](https://github.com/morluto/rea/commit/95a77b6e25420b3879bd0e32bdbb14d452b38776))
- Saved reference graphs: exclusions require their matching `pattern` and distinguish `project-ignored` from `default-ignored`. Re-import graphs missing that provenance. ([6585293](https://github.com/morluto/rea/commit/6585293c562853b95c3a70df02c12ad8cc17afc4))
- Terminal captures: keep initial and resized buffers within 1,000,000 cells using `columns × (rows + scrollback)`; oversized scenarios are rejected before launch. ([f9ea9a9](https://github.com/morluto/rea/commit/f9ea9a9268c854d31048da9f8a50cfa37703fe5a))
- Optional agent-verification reports: accept schema version 4, nullable `expectedFirstTool` and `targetClarificationPassed`. Production CLI/MCP contracts are unchanged. ([#1613](https://github.com/morluto/rea/pull/1613))
- Capture comparisons: recapture both sides with matching `literal-replacements-v1` semantics when custom normalization patterns are used. Legacy unversioned pattern captures remain readable but cannot be compared with new pattern captures; captures without patterns are unaffected. ([#1456](https://github.com/morluto/rea/pull/1456), [2b6f14f](https://github.com/morluto/rea/commit/2b6f14f528297cf12c776c960af0859f5cfbefbf)) Thanks @Madhavan006, @N0zoM1z0.
- Artifact graph edges: read child artifact IDs and paths from the occurrence selected by `occurrence_id`, not edge fields `child_artifact_id`/`logical_path`. Earlier inventory Evidence no longer validates against the new commitment. ([#1493](https://github.com/morluto/rea/pull/1493)) Thanks @kaoru0822-kitauji.
- Semantic graph schema: nodes and relations no longer include `identifier_strategy`, and relation IDs no longer hash it. Earlier Evidence no longer validates against this schema. ([#1474](https://github.com/morluto/rea/pull/1474)) Thanks @kaoru0822-kitauji.
- Semantic identities: select IDs from the parent graph; read exact event names from `event_name`, endpoints from `endpoint` and literal values from `properties.value`. New analyses use digest-based IDs for long/reserved event and endpoint names and `value-sha256` literal role keys, with shorter labels; string literal labels now read `string literal`. Previously stored graphs retain their original IDs and remain valid workflow inputs. ([#1427](https://github.com/morluto/rea/pull/1427), [#1423](https://github.com/morluto/rea/pull/1423), [#1407](https://github.com/morluto/rea/pull/1407)) Thanks @N0zoM1z0.
### Contributors
- Thanks also to @BaaRai, @commit-regrets, @coygeek, @Franky100-pig, @inii-sayaa, @jopemachine, @JosephDemarest, @keemsisi, @Totopo27 for contributions included in this release.
Full contribution record: [336 commits and 163 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json).
## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09) ## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09)
+23
View File
@@ -24,6 +24,29 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
Target discovery returns the complete in-scope target array in one result. Target discovery returns the complete in-scope target array in one result.
## Screenshot comparison memory budget
`compare_web_screenshots` admits both PNGs against a 256 MiB estimated
working-memory budget before inflating either image. This is a processing
buffer budget for this operation, not a PNG dimension limit, CDP transport
limit, or process RSS guarantee. Header-valid dimensions that exceed the
budget return `resource_constraint` with `details.resource: "memory"` and the
estimated and maximum working-set bytes. Invalid PNG structure remains
malformed input; encodings outside supported pixel-comparison coverage remain
unsupported-target errors.
The conservative estimate includes the two decoder-owned decoded input
buffers, the current image's concatenated IDAT buffer, Node zlib's default
16 KiB output chunk backing rounded up for retained chunks, the concatenated
inflate result while its output chunks are retained, one possible extra output
chunk before zlib checks `maxOutputLength`, unfiltered scanline storage,
RGB-to-RGBA expansion, and the first retained RGBA image while the second is
decoded. RGBA output aliases its unfiltered buffer and is counted once.
Admission uses checked integer arithmetic before image-sized allocation.
Caller-owned JSON/base64 strings, input parsing and validation, transport
buffers, garbage-collector behavior, and other runtime memory are outside this
estimate; the budget does not promise a bound on total process memory.
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md). Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
## Provider authorities ## Provider authorities
+1 -1
View File
@@ -880,7 +880,7 @@ For a client that requires manual configuration, use:
"mcpServers": { "mcpServers": {
"rea": { "rea": {
"command": "npx", "command": "npx",
"args": ["-y", "rea-agents@6.3.0", "mcp"] "args": ["-y", "rea-agents@6.4.0", "mcp"]
} }
} }
} }
+17
View File
@@ -308,6 +308,23 @@ that reference for a summary, one section/module, or a stable page, or call
bundle exports stream canonical JSON into an atomically published file. A broad bundle exports stream canonical JSON into an atomically published file. A broad
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
preserves the complete session without sending it through a single MCP frame. preserves the complete session without sending it through a single MCP frame.
To discover existing records without their large payloads, call
`get_evidence_bundle` with `detail: "summary"`. Its `result.records` contain
Evidence IDs, artifact/provider identities, analysis-profile digests and native
procedure metadata. Optional `filters` match exact `evidence_id`, `operation`,
`target_sha256`, `analysis_profile_digest` or `procedure_address`. Use a discovered
ID with `inspect_analysis_view`; neither discovery nor retained selection starts
a provider. `retention: "complete-record"` describes stored Evidence, while
`native_dossier.value_flow` independently reports truncation, unavailable or
unknown metadata. Missing omission counts remain null. Summary output carries
every matching record; actual transport admission still applies. Omitted
`detail` preserves complete canonical bundle delivery; filters require summary.
For a portable exported bundle, the CLI shares this projection:
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
The CLI validates and materializes that local bundle under its existing memory
budget; summary delivery does not make arbitrary large files cheap to import.
Cancelling an export stops further serialization and removes its staging file. Cancelling an export stops further serialization and removes its staging file.
The destination changes only when a complete export is atomically published. The destination changes only when a complete export is atomically published.
File diff suppressed because it is too large Load Diff
+5 -1
View File
@@ -21,7 +21,11 @@ Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.a
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full `Thanks @...` credit. `npm run release:notes -- inventory` collects the full
selected Git range and paginated GitHub PR/author/closing-issue metadata; selected Git range and paginated GitHub PR/author/closing-issue metadata;
`record` produces the complete contribution record, `check` validates credits `record --summary` produces a compact provenance link for public notes, while
`record` produces the full record for review. Credit a contribution once even
when it appears in highlights, details and migrations; omit maintainer
`@morluto` self-thanks. Keep complete authorship metadata in the inventory
instead of duplicating a credit ledger in the release body. `check` validates credits
and optional PR-body equality, and `render` writes the verified release section. and optional PR-body equality, and `render` writes the verified release section.
Run a subcommand without its required options to see its usage. GitHub reads Run a subcommand without its required options to see its usage. GitHub reads
require an authenticated `gh`; record/check/render work offline from the saved require an authenticated `gh`; record/check/render work offline from the saved
+12
View File
@@ -1,5 +1,7 @@
#include "authority.hpp" #include "authority.hpp"
#include <delayimp.h>
namespace rea { namespace rea {
static const napi_type_tag RESOURCE_TAG = {0x45fcbf0b49e84fe3ULL, 0x9e3e5074130de843ULL}; static const napi_type_tag RESOURCE_TAG = {0x45fcbf0b49e84fe3ULL, 0x9e3e5074130de843ULL};
@@ -121,3 +123,13 @@ static napi_value initialize(napi_env env, napi_value exports) {
} // namespace rea } // namespace rea
NAPI_MODULE(NODE_GYP_MODULE_NAME, rea::initialize) NAPI_MODULE(NODE_GYP_MODULE_NAME, rea::initialize)
// Node-API symbols are delay-imported from "node.exe". Resolve them against the
// host executable so Node.js, Bun, and other Node-API hosts load the addon
// regardless of their executable file name.
static FARPROC WINAPI reaHostHook(unsigned event, PDelayLoadInfo info) {
if (event != dliNotePreLoadLibrary || _stricmp(info->szDll, "node.exe") != 0) return nullptr;
return reinterpret_cast<FARPROC>(GetModuleHandleW(nullptr));
}
decltype(__pfnDliNotifyHook2) __pfnDliNotifyHook2 = reaHostHook;
+3 -2
View File
@@ -219,9 +219,10 @@ std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std
require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0, require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0,
"Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE); "Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE);
auto result = std::make_unique<Runtime>(); auto result = std::make_unique<Runtime>();
result->path = parent->path + L"\\" + prefix + hex(random.data(), random.size()); const auto name = prefix + hex(random.data(), random.size());
result->path = parent->path + (parent->path.back() == L'\\' ? L"" : L"\\") + name;
Security security; Security security;
result->directory = createRelative(parent->get(), prefix + hex(random.data(), random.size()), result->directory = createRelative(parent->get(), name,
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL, DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL,
true, security, result->path); true, security, result->path);
try { try {
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "rea-agents", "name": "rea-agents",
"version": "6.3.0", "version": "6.4.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "rea-agents", "name": "rea-agents",
"version": "6.3.0", "version": "6.4.0",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@babel/parser": "8.0.4", "@babel/parser": "8.0.4",
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "rea-agents", "name": "rea-agents",
"mcpName": "io.github.morluto/rea", "mcpName": "io.github.morluto/rea",
"version": "6.3.0", "version": "6.4.0",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.", "description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"license": "MIT", "license": "MIT",
"repository": { "repository": {
+3 -2
View File
@@ -84,7 +84,8 @@ if (!(await exists(join(headers, "node_api.h")))) {
} }
const artifact = "rea-windows-x64.node"; const artifact = "rea-windows-x64.node";
// MSVC short-import libraries do not produce a correct Node import table with // MSVC short-import libraries do not produce a correct Node import table with
// GNU ld. Generate a GNU import library for the exact stable Node-API symbols. // GNU ld. Generate a GNU delay-import library for the exact stable Node-API
// symbols; addon.cc binds it to the host executable, whatever its file name.
const sources = ["addon.cc", "filesystem.cc", "process.cc"]; const sources = ["addon.cc", "filesystem.cc", "process.cc"];
const sourceText = ( const sourceText = (
await Promise.all( await Promise.all(
@@ -106,7 +107,7 @@ await writeFile(
definition, definition,
`LIBRARY node.exe\nEXPORTS\n${symbols.join("\n")}\n`, `LIBRARY node.exe\nEXPORTS\n${symbols.join("\n")}\n`,
); );
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-l", library]); run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-y", library]);
run(compiler, [ run(compiler, [
"-std=c++17", "-std=c++17",
"-O2", "-O2",
@@ -0,0 +1,61 @@
/* Keep the configured launcher identity alive while its resource-limited
* Python child runs. Both remain in REA's owned process group. */
#define _POSIX_C_SOURCE 200809L
#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
int main(int argc, char **argv) {
char *python = getenv("REA_VERIFY_LIMITED_PYTHON");
char *limit = getenv("REA_VERIFY_LIMIT_OPTION");
if (python == NULL || python[0] != '/' || limit == NULL) {
fputs("limited Python fixture requires an absolute Python and limit\n", stderr);
return 127;
}
struct rlimit core = {0, 0};
if (setrlimit(RLIMIT_CORE, &core) != 0) {
perror("setrlimit");
return 127;
}
char **arguments = calloc((size_t)argc + 4, sizeof(char *));
if (arguments == NULL) return 127;
arguments[0] = "/usr/bin/prlimit";
arguments[1] = limit;
arguments[2] = "--";
arguments[3] = python;
for (int index = 1; index < argc; index++) arguments[index + 3] = argv[index];
pid_t child = fork();
if (child == 0) {
execv(arguments[0], arguments);
perror("execv");
_exit(127);
}
free(arguments);
if (child < 0) {
perror("fork");
return 127;
}
int status;
while (waitpid(child, &status, 0) < 0) {
if (errno == EINTR) continue;
perror("waitpid");
return 127;
}
if (WIFEXITED(status)) return WEXITSTATUS(status);
if (WIFSIGNALED(status)) {
int signum = WTERMSIG(status);
signal(signum, SIG_DFL);
sigset_t signals;
sigemptyset(&signals);
sigaddset(&signals, signum);
sigprocmask(SIG_UNBLOCK, &signals, NULL);
raise(signum);
return 128 + signum;
}
return 127;
}
+15 -3
View File
@@ -28,6 +28,11 @@ import {
isolatedFixtureUrl, isolatedFixtureUrl,
} from "./browser-runtime-world-e2e.mjs"; } from "./browser-runtime-world-e2e.mjs";
// Arming, progress delivery, isolated-world proof and the page action cross
// separate transports. Leave time for their round trips on shared CI runners;
// coverage can include a late action even after request collection has ended.
const executionObservationMs = 2_000;
/** Exercise native browser runtime attribution through public CLI and stdio MCP, including arming. */ /** Exercise native browser runtime attribution through public CLI and stdio MCP, including arming. */
export async function verifyBrowserRuntime( export async function verifyBrowserRuntime(
executable, executable,
@@ -210,7 +215,7 @@ async function executionMcp(
const response = await client.callTool( const response = await client.callTool(
{ {
name: "observe_web_execution", name: "observe_web_execution",
arguments: { ...input, observation_ms: 300 }, arguments: { ...input, observation_ms: executionObservationMs },
}, },
{ {
timeout: 40_000, timeout: 40_000,
@@ -259,7 +264,7 @@ async function executionCli(entrypoint, input, env, action, isolatedContext) {
input.cdp_endpoint, input.cdp_endpoint,
input.target_id, input.target_id,
"--observation-ms", "--observation-ms",
"300", String(executionObservationMs),
"--json", "--json",
], ],
{ env, stdio: ["ignore", "pipe", "pipe"] }, { env, stdio: ["ignore", "pipe", "pipe"] },
@@ -382,7 +387,14 @@ function assertExecution(evidence, site, requireBlock = true) {
const request = result.requests.find((item) => const request = result.requests.find((item) =>
item.url.endsWith("/evidence?marker=chosen"), item.url.endsWith("/evidence?marker=chosen"),
); );
assert.ok(request, "The selected request initiator must be retained"); assert.ok(
request,
`The selected request initiator must be retained: ${JSON.stringify({
window: result.window,
requests: result.requests.map((item) => item.url),
excluded_requests: result.excluded_requests,
})}`,
);
assert.ok( assert.ok(
request.callsites.some( request.callsites.some(
(site) => (site) =>
+33 -6
View File
@@ -101,10 +101,13 @@ export async function readReleaseHistory(run, base, target) {
// REA history maps Anthropic and Cursor's agent trailers to ordinary User actors. // REA history maps Anthropic and Cursor's agent trailers to ordinary User actors.
// Keep those source identities in the inventory while excluding automation credit. // Keep those source identities in the inventory while excluding automation credit.
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]); const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]);
// REA's maintainer requested external-contributor thanks without self-credit.
const MAINTAINER_ACCOUNTS = new Set(["morluto"]);
const human = (value) => const human = (value) =>
value?.__typename === "User" && value?.__typename === "User" &&
!value.login.endsWith("[bot]") && !value.login.endsWith("[bot]") &&
!AGENT_ACCOUNTS.has(value.login.toLowerCase()) !AGENT_ACCOUNTS.has(value.login.toLowerCase()) &&
!MAINTAINER_ACCOUNTS.has(value.login.toLowerCase())
? value.login ? value.login
: undefined; : undefined;
const unique = (values) => const unique = (values) =>
@@ -163,6 +166,12 @@ export function contributionRecord(raw) {
return `${lines.join("\n")}\n`; return `${lines.join("\n")}\n`;
} }
/** Keep the full authorship snapshot available without duplicating public thanks. */
export function contributionSummary(raw) {
const inventory = releaseInventorySchema.parse(raw);
return `Full contribution record: [${inventory.commits.length} commits and ${inventory.pullRequests.length} merged PR${inventory.pullRequests.length === 1 ? "" : "s"}](https://github.com/${inventory.repository}/blob/rea-agents-${inventory.version}/docs/releases/${inventory.version}.contributions.json).\n`;
}
export function selectedReleaseNotes(text, version) { export function selectedReleaseNotes(text, version) {
const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)]; const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)];
if (headings[0]?.[1] !== version) if (headings[0]?.[1] !== version)
@@ -236,7 +245,8 @@ function requiredCredits(inventory, bullet, references) {
if (matches[0].parents.length > 1) continue; if (matches[0].parents.length > 1) continue;
for (const identity of matches[0].authors) { for (const identity of matches[0].authors) {
const handle = human(identity.user); const handle = human(identity.user);
if (handle) required.push({ handle, context: `commit ${match[2]}` }); if (handle)
required.push({ handle, context: `commit ${matches[0].sha}` });
} }
} }
return required; return required;
@@ -264,6 +274,8 @@ function verifyEditorialCredits(inventory, prose) {
]).map((handle) => handle.toLowerCase()), ]).map((handle) => handle.toLowerCase()),
); );
const errors = []; const errors = [];
const requiredByContext = new Map();
const thankedByContext = new Map();
for (const bullet of proseBullets(prose)) { for (const bullet of proseBullets(prose)) {
const references = bulletPrs(bullet, inventory.repository); const references = bulletPrs(bullet, inventory.repository);
const start = bullet.indexOf("Thanks "); const start = bullet.indexOf("Thanks ");
@@ -282,13 +294,21 @@ function verifyEditorialCredits(inventory, prose) {
bullet, bullet,
references, references,
)) { )) {
if (!thanked.has(handle.toLowerCase())) const required = requiredByContext.get(context) ?? new Set();
errors.push(`Missing Thanks @${handle} in bullet for ${context}`); required.add(handle);
requiredByContext.set(context, required);
const credited = thankedByContext.get(context) ?? new Set();
for (const login of thanked) credited.add(login);
thankedByContext.set(context, credited);
} }
for (const handle of thanked) for (const handle of thanked)
if (!eligible.has(handle)) if (!eligible.has(handle))
errors.push(`Unverified human credit @${handle}`); errors.push(`Unverified human credit @${handle}`);
} }
for (const [context, required] of requiredByContext)
for (const handle of required)
if (!thankedByContext.get(context)?.has(handle.toLowerCase()))
errors.push(`Missing Thanks @${handle} for ${context}`);
return errors; return errors;
} }
@@ -305,8 +325,15 @@ export function verifyReleaseNotes(raw, text, body) {
"Release notes exceed GitHub's body budget; shorten prose without dropping credit", "Release notes exceed GitHub's body budget; shorten prose without dropping credit",
); );
const record = contributionRecord(inventory).trim(); const record = contributionRecord(inventory).trim();
const start = notes.indexOf("### Complete contribution record"); const summary = contributionSummary(inventory).trim();
if (start < 0 || notes.slice(start).trim() !== record) const start = notes.endsWith(summary)
? notes.length - summary.length
: notes.indexOf("### Complete contribution record");
if (
start < 0 ||
(notes.slice(start).trim() !== record &&
notes.slice(start).trim() !== summary)
)
throw new Error( throw new Error(
"Contribution record differs from inventory; regenerate it with release-notes record", "Contribution record differs from inventory; regenerate it with release-notes record",
); );
+8 -3
View File
@@ -6,6 +6,7 @@ import { collectGitHubCredits } from "./lib/release-notes-github.mjs";
import { verifyNotesPublication } from "./lib/release-notes-publication.mjs"; import { verifyNotesPublication } from "./lib/release-notes-publication.mjs";
import { import {
contributionRecord, contributionRecord,
contributionSummary,
readReleaseHistory, readReleaseHistory,
releaseInventorySchema, releaseInventorySchema,
selectedReleaseNotes, selectedReleaseNotes,
@@ -73,6 +74,7 @@ async function main() {
"pr-body": { type: "string" }, "pr-body": { type: "string" },
source: { type: "string" }, source: { type: "string" },
output: { type: "string" }, output: { type: "string" },
summary: { type: "boolean", default: false },
}, },
}); });
const [command] = positionals; const [command] = positionals;
@@ -92,7 +94,7 @@ async function main() {
!values.manifest !values.manifest
) )
throw new Error( throw new Error(
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE]", "Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE] [--summary for record]",
); );
if (command === "inventory") { if (command === "inventory") {
await createInventory(values); await createInventory(values);
@@ -103,11 +105,14 @@ async function main() {
); );
await verifyInventoryHistory(run, inventory, values.source); await verifyInventoryHistory(run, inventory, values.source);
if (command === "record") { if (command === "record") {
const record = values.summary
? contributionSummary(inventory)
: contributionRecord(inventory);
if (values.output) if (values.output)
await writeFile(values.output, contributionRecord(inventory), { await writeFile(values.output, record, {
flag: "wx", flag: "wx",
}); });
else process.stdout.write(contributionRecord(inventory)); else process.stdout.write(record);
return; return;
} }
const notes = await readFile(values.notes, "utf8"); const notes = await readFile(values.notes, "utf8");
+84 -40
View File
@@ -15,6 +15,10 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client"; import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js"; import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../dist/process/OwnedCommand.js";
import { parseEvidence } from "../dist/domain/evidence.js"; import { parseEvidence } from "../dist/domain/evidence.js";
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs"; import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs"; import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
@@ -56,6 +60,13 @@ const root = await PrivateRuntimeRoot.create({
const environment = Object.fromEntries( const environment = Object.fromEntries(
Object.entries(process.env).filter(([, value]) => typeof value === "string"), Object.entries(process.env).filter(([, value]) => typeof value === "string"),
); );
const limitedPython = join(root.path, "limited-python");
const limitedEnvironment = (option) => ({
...environment,
REA_PWNTOOLS_PYTHON: limitedPython,
REA_VERIFY_LIMITED_PYTHON: python,
REA_VERIFY_LIMIT_OPTION: option,
});
const client = new Client({ name: "binary-layout-verifier", version: "1" }); const client = new Client({ name: "binary-layout-verifier", version: "1" });
const transport = new StdioClientTransport({ const transport = new StdioClientTransport({
command: process.execPath, command: process.execPath,
@@ -73,6 +84,46 @@ let cases = 0;
let bootstrapCases = 0; let bootstrapCases = 0;
const failures = []; const failures = [];
try { try {
await execute(
"gcc",
[
"-std=c11",
"-Wall",
"-Wextra",
"-Werror",
fileURLToPath(
new URL("./fixtures/binary-layout-limited-python.c", import.meta.url),
),
"-o",
limitedPython,
],
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
);
// A live limited child must retain its configured launcher's identity so
// timeout cleanup can still verify and terminate the entire owned group.
await assert.rejects(
runOwnedCommand(
{
command: limitedPython,
arguments: [
"-I",
"-c",
"import time; print('limited child ready', flush=True); time.sleep(60)",
],
cwd: root.path,
runId: `${run.run_id}-limited-python-timeout`,
hostEnvironment: limitedEnvironment("--as=2147483648"),
},
{ timeoutMs: 2000, diagnosticBytes: 1024 },
),
(error) => {
assert.ok(error instanceof OwnedCommandFailure);
assert.equal(error.reason, "timeout");
assert.equal(error.cleanupFailure, null);
assert.ok(error.snapshot.stdout.text.includes("limited child ready"));
return true;
},
);
const bootstrapRoot = join(root.path, "bootstrap-boundary"); const bootstrapRoot = join(root.path, "bootstrap-boundary");
await mkdir(bootstrapRoot); await mkdir(bootstrapRoot);
const bootstrapPath = join(bootstrapRoot, "layout.py"); const bootstrapPath = join(bootstrapRoot, "layout.py");
@@ -796,19 +847,12 @@ try {
assert.deepEqual(await readFile(extendedPath), extendedObject); assert.deepEqual(await readFile(extendedPath), extendedObject);
cases++; cases++;
} }
for (const [name, option, expected] of [ for (const [option, expected] of [
["address-space", "--as=2147483648", '"address_space_bytes": 2147483648'], ["--as=2147483648", '"address_space_bytes": 2147483648'],
["cpu", "--cpu=20", '"cpu_seconds": 20'], ["--cpu=20", '"cpu_seconds": 20'],
["file-size", "--fsize=33554432", '"file_size_bytes": 33554432'], ["--fsize=33554432", '"file_size_bytes": 33554432'],
]) { ]) {
const wrapper = join(root.path, `python-limit-${name}`); const selectedEnvironment = limitedEnvironment(option);
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
await writeFile(
wrapper,
`#!/bin/sh\nexec /usr/bin/prlimit ${option} -- ${quotedPython} "$@"\n`,
{ mode: 0o700 },
);
const limitedEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
const limitedClient = new Client({ const limitedClient = new Client({
name: "limited-layout-verifier", name: "limited-layout-verifier",
version: "1", version: "1",
@@ -816,7 +860,7 @@ try {
const limitedTransport = new StdioClientTransport({ const limitedTransport = new StdioClientTransport({
command: process.execPath, command: process.execPath,
args: [entrypoint, "mcp"], args: [entrypoint, "mcp"],
env: limitedEnvironment, env: selectedEnvironment,
stderr: "pipe", stderr: "pipe",
}); });
try { try {
@@ -826,7 +870,7 @@ try {
mode, mode,
join(root.path, "protected"), join(root.path, "protected"),
undefined, undefined,
limitedEnvironment, selectedEnvironment,
limitedClient, limitedClient,
); );
assert.ok(value.limitations.some((item) => item.includes(expected))); assert.ok(value.limitations.some((item) => item.includes(expected)));
@@ -862,17 +906,29 @@ try {
sectionHeavy.writeUInt16LE(sectionCount, 60); sectionHeavy.writeUInt16LE(sectionCount, 60);
const sectionHeavyPath = join(root.path, "section-heavy.o"); const sectionHeavyPath = join(root.path, "section-heavy.o");
await writeFile(sectionHeavyPath, sectionHeavy); await writeFile(sectionHeavyPath, sectionHeavy);
const memoryWrapper = join(root.path, "python-memory-constraint"); // pwntools materializes each PT_LOAD zero-filled tail during inspection.
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'"; // One allocation larger than the whole address-space budget fails promptly;
await writeFile( // a large section count instead races CPU work against the command deadline.
memoryWrapper, const memoryHeavy = Buffer.from(await readFile(join(root.path, "protected")));
`#!/bin/sh\nexec /usr/bin/prlimit --as=100663296 -- ${quotedPython} "$@"\n`, const memoryPhOffset = Number(memoryHeavy.readBigUInt64LE(32));
{ mode: 0o700 }, const memoryPhSize = memoryHeavy.readUInt16LE(54);
const memoryPhCount = memoryHeavy.readUInt16LE(56);
const lastLoadIndex = Array.from(
{ length: memoryPhCount },
(_, index) => index,
).findLast(
(index) =>
memoryHeavy.readUInt32LE(memoryPhOffset + index * memoryPhSize) === 1,
); );
const memoryEnvironment = { assert.notEqual(lastLoadIndex, undefined);
...environment, const memorySizeOffset = memoryPhOffset + lastLoadIndex * memoryPhSize + 40;
REA_PWNTOOLS_PYTHON: memoryWrapper, memoryHeavy.writeBigUInt64LE(
}; memoryHeavy.readBigUInt64LE(memorySizeOffset) + 128n * 1024n * 1024n,
memorySizeOffset,
);
const memoryHeavyPath = join(root.path, "memory-zero-tail");
await writeFile(memoryHeavyPath, memoryHeavy);
const memoryEnvironment = limitedEnvironment("--as=100663296");
const memoryClient = new Client({ const memoryClient = new Client({
name: "memory-layout-verifier", name: "memory-layout-verifier",
version: "1", version: "1",
@@ -888,7 +944,7 @@ try {
for (const mode of ["cli", "mcp"]) { for (const mode of ["cli", "mcp"]) {
const error = await inspect( const error = await inspect(
mode, mode,
sectionHeavyPath, memoryHeavyPath,
"resource_constraint", "resource_constraint",
memoryEnvironment, memoryEnvironment,
memoryClient, memoryClient,
@@ -903,7 +959,7 @@ try {
assert.ok(error.remediation.action.includes("memory")); assert.ok(error.remediation.action.includes("memory"));
cases++; cases++;
} }
assert.deepEqual(await readFile(sectionHeavyPath), sectionHeavy); assert.deepEqual(await readFile(memoryHeavyPath), memoryHeavy);
} finally { } finally {
try { try {
await memoryClient.close(); await memoryClient.close();
@@ -924,13 +980,7 @@ try {
cpuHeavy.writeBigUInt64LE(BigInt(cpuSectionCount), sectionOffset + 32); cpuHeavy.writeBigUInt64LE(BigInt(cpuSectionCount), sectionOffset + 32);
const cpuHeavyPath = join(root.path, "cpu-section-heavy.o"); const cpuHeavyPath = join(root.path, "cpu-section-heavy.o");
await writeFile(cpuHeavyPath, cpuHeavy); await writeFile(cpuHeavyPath, cpuHeavy);
const cpuWrapper = join(root.path, "python-cpu-constraint"); const cpuEnvironment = limitedEnvironment("--cpu=1:");
await writeFile(
cpuWrapper,
`#!/bin/sh\nexec /usr/bin/prlimit --cpu=1: --core=0: -- ${quotedPython} "$@"\n`,
{ mode: 0o700 },
);
const cpuEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: cpuWrapper };
const cpuClient = new Client({ name: "cpu-layout-verifier", version: "1" }); const cpuClient = new Client({ name: "cpu-layout-verifier", version: "1" });
const cpuTransport = new StdioClientTransport({ const cpuTransport = new StdioClientTransport({
command: process.execPath, command: process.execPath,
@@ -963,13 +1013,7 @@ try {
} }
} }
for (const limit of [1024, 1]) { for (const limit of [1024, 1]) {
const wrapper = join(root.path, `python-file-size-${limit}`); const fileEnvironment = limitedEnvironment(`--fsize=${limit}:`);
await writeFile(
wrapper,
`#!/bin/sh\nexec /usr/bin/prlimit --fsize=${limit}: --core=0: -- ${quotedPython} "$@"\n`,
{ mode: 0o700 },
);
const fileEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
const fileClient = new Client({ const fileClient = new Client({
name: "file-size-layout-verifier", name: "file-size-layout-verifier",
version: "1", version: "1",
+145 -97
View File
@@ -38,104 +38,147 @@ let update;
let repositorySkill; let repositorySkill;
let report; let report;
const runStage = async (name, action) => {
const started = performance.now();
process.stderr.write(`[verify:package] ${name}: start\n`);
const result = await action();
process.stderr.write(
`[verify:package] ${name}: complete (${Math.round(performance.now() - started)} ms)\n`,
);
return result;
};
try { try {
repositorySkill = await verifyRepositorySkill({ root, workspace }); repositorySkill = await runStage("repository skill", () =>
({ tarball } = await verifyPackagePack({ root, workspace })); verifyRepositorySkill({ root, workspace }),
const environmentData = await verifyPackageEnvironment({ );
root, ({ tarball } = await runStage("package archive", () =>
workspace, verifyPackagePack({ root, workspace }),
evidenceRoot, ));
referenceRoot, const environmentData = await runStage("package environment", () =>
}); verifyPackageEnvironment({
const { cli, packageRunnerCli } = await verifyPackageInstall({ root,
tarball, workspace,
prefix: environmentData.prefix, evidenceRoot,
workspace, referenceRoot,
environment: environmentData.environment, }),
}); );
update = await verifyPackageUpdate({ const { cli, packageRunnerCli } = await runStage("package install", () =>
prefix: environmentData.prefix, verifyPackageInstall({
packageRoot: join( tarball,
environmentData.prefix, prefix: environmentData.prefix,
"lib", workspace,
"node_modules", environment: environmentData.environment,
PRODUCT_IDENTITY.packageName, }),
), );
tarball, update = await runStage("package update", () =>
workspace, verifyPackageUpdate({
environment: environmentData.environment, prefix: environmentData.prefix,
}); packageRoot: join(
mcpStartup = await measureMcpStartup({ environmentData.prefix,
command: cli, "lib",
args: ["mcp"], "node_modules",
environment: { PRODUCT_IDENTITY.packageName,
...environmentData.environment, ),
}, tarball,
policy: MCP_STARTUP_POLICY, workspace,
}); environment: environmentData.environment,
mcpModuleLoading = await profileMcpModuleLoading({ }),
command: cli, );
args: ["mcp"], mcpStartup = await runStage("MCP startup", () =>
environment: { measureMcpStartup({
...environmentData.environment, command: cli,
}, args: ["mcp"],
policy: MCP_STARTUP_POLICY, environment: {
packageName: PRODUCT_IDENTITY.packageName, ...environmentData.environment,
}); },
const { supportedSetupHost, hopperSetupSupported } = policy: MCP_STARTUP_POLICY,
await verifyPackageDiscovery({ }),
);
mcpModuleLoading = await runStage("MCP module loading", () =>
profileMcpModuleLoading({
command: cli,
args: ["mcp"],
environment: {
...environmentData.environment,
},
policy: MCP_STARTUP_POLICY,
packageName: PRODUCT_IDENTITY.packageName,
}),
);
const { supportedSetupHost, hopperSetupSupported } = await runStage(
"package discovery",
() =>
verifyPackageDiscovery({
cli,
environment: environmentData.environment,
}),
);
const { artifactArchive } = await runStage("artifact and Electron CLI", () =>
verifyPackageArtifactAndElectron({
cli,
workspace,
environment: environmentData.environment,
}),
);
await runStage("managed analysis CLI", () =>
verifyManaged({
cli,
workspace,
environment: environmentData.environment,
}),
);
await runStage("unknown provider CLI", () =>
verifyUnknownProvider({
cli, cli,
environment: environmentData.environment, environment: environmentData.environment,
}); }),
const { artifactArchive } = await verifyPackageArtifactAndElectron({ );
cli, await runStage("platform CLI", () =>
workspace, verifyPackagePlatform({
environment: environmentData.environment, cli,
}); environment: environmentData.environment,
await verifyManaged({ }),
cli, );
workspace, await runStage("capabilities and search CLI", () =>
environment: environmentData.environment, verifyPackageCapabilitiesAndSearch({
}); cli,
await verifyUnknownProvider({ environment: environmentData.environment,
cli, }),
environment: environmentData.environment, );
}); await runStage("evidence CLI", () =>
await verifyPackagePlatform({ verifyPackageEvidence({
cli, cli,
environment: environmentData.environment, evidenceRoot,
}); referenceRoot,
await verifyPackageCapabilitiesAndSearch({ environment: environmentData.environment,
cli, }),
environment: environmentData.environment, );
}); await runStage("setup and uninstall CLI", () =>
await verifyPackageEvidence({ verifyPackageSetup({
cli, cli,
evidenceRoot, packageRunnerCli,
referenceRoot, environment: environmentData.environment,
environment: environmentData.environment, home: environmentData.home,
}); npxLog: environmentData.npxLog,
await verifyPackageSetup({ claudeConfig: environmentData.claudeConfig,
cli, codexConfig: environmentData.codexConfig,
packageRunnerCli, cursorConfig: environmentData.cursorConfig,
environment: environmentData.environment, codexTarget: environmentData.codexTarget,
home: environmentData.home, cursorTarget: environmentData.cursorTarget,
npxLog: environmentData.npxLog, supportedSetupHost,
claudeConfig: environmentData.claudeConfig, hopperSetupSupported,
codexConfig: environmentData.codexConfig, root,
cursorConfig: environmentData.cursorConfig, }),
codexTarget: environmentData.codexTarget, );
cursorTarget: environmentData.cursorTarget, await runStage("MCP tools and evidence", () =>
supportedSetupHost, verifyPackageMcp({
hopperSetupSupported, cli,
root, environment: environmentData.environment,
}); evidenceRoot,
await verifyPackageMcp({ artifactArchive,
cli, }),
environment: environmentData.environment, );
evidenceRoot,
artifactArchive,
});
report = { report = {
cli: true, cli: true,
analysisCli: true, analysisCli: true,
@@ -173,8 +216,13 @@ try {
mcpModuleLoading, mcpModuleLoading,
}; };
} finally { } finally {
await rm(workspace, { recursive: true, force: true }); await runStage("workspace cleanup", () =>
rm(workspace, { recursive: true, force: true }),
);
} }
const verifierLineage = await runStage("verifier lineage", () =>
completeVerifierRun(verifierRun),
);
process.stdout.write( process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`, `${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`,
); );
+71 -1
View File
@@ -6,6 +6,7 @@ import { createHash } from "node:crypto";
import { once } from "node:events"; import { once } from "node:events";
import { import {
access, access,
copyFile,
mkdir, mkdir,
mkdtemp, mkdtemp,
readFile, readFile,
@@ -16,7 +17,7 @@ import {
open, open,
} from "node:fs/promises"; } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join, resolve } from "node:path"; import { dirname, join, parse, resolve } from "node:path";
import { pathToFileURL } from "node:url"; import { pathToFileURL } from "node:url";
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs"; import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
@@ -32,6 +33,9 @@ const native = requireWindowsNativeAuthority();
const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import( const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import(
pathToFileURL(join(packageRoot, "dist/windows/WindowsPrivateRuntime.js")) pathToFileURL(join(packageRoot, "dist/windows/WindowsPrivateRuntime.js"))
); );
const { ghidraSessionRoot } = await import(
pathToFileURL(join(packageRoot, "dist/ghidra/GhidraSessionRoot.js"))
);
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-")); const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
const environment = Object.entries(process.env) const environment = Object.entries(process.env)
.filter(([, value]) => value !== undefined) .filter(([, value]) => value !== undefined)
@@ -77,6 +81,31 @@ const wait = async (child) => {
}; };
const report = { ok: false, artifact: "verified", controls: {}, processes: {} }; const report = { ok: false, artifact: "verified", controls: {}, processes: {} };
try { try {
// Exercise the loader in a process whose image is not named node.exe. The
// ordinary Windows lane otherwise cannot detect eager Node-API imports.
const renamedHost = join(workspace, "rea-renamed-node.exe");
const renamedHostFixture = join(workspace, "renamed-host.mjs");
const loaderUrl = pathToFileURL(
join(packageRoot, "dist/windows/WindowsNativeLoader.js"),
).href;
await copyFile(process.execPath, renamedHost);
await writeFile(
renamedHostFixture,
`import {basename} from 'node:path';
import {requireWindowsNativeAuthority} from ${JSON.stringify(loaderUrl)};
const {inspection}=requireWindowsNativeAuthority();
process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVersion:inspection.abiVersion}));\n`,
);
const renamedHostResult = execFileSync(renamedHost, [renamedHostFixture], {
encoding: "utf8",
timeout: 20_000,
windowsHide: true,
});
assert.deepEqual(JSON.parse(renamedHostResult), {
executable: "rea-renamed-node.exe",
abiVersion: 1,
});
report.controls.renamedHostNodeApi = true;
const sourceDirectory = join(workspace, "source"); const sourceDirectory = join(workspace, "source");
await mkdir(sourceDirectory); await mkdir(sourceDirectory);
const source = join(sourceDirectory, "target.bin"); const source = join(sourceDirectory, "target.bin");
@@ -110,6 +139,47 @@ try {
]) ])
assert.throws(() => native.call("open", [rejected])); assert.throws(() => native.call("open", [rejected]));
report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true; report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true;
const driveRoot = parse(workspace).root;
const runtimeParent = ghidraSessionRoot({
base: join(driveRoot, ".Tmp", "scratch"),
});
assert.equal(runtimeParent, driveRoot);
const driveRootRuntime = native.call("runtime_create", [
runtimeParent,
"rea-drive-root-",
]);
try {
assert.equal(driveRootRuntime.path.startsWith(`${driveRoot}\\`), false);
assert.equal(dirname(driveRootRuntime.path), driveRoot);
const snapshot = await native.call("runtime_snapshot", [
driveRootRuntime.handle,
source,
"probe.bin",
]);
const sourceBytes = await readFile(source);
assert.equal(
snapshot.sha256,
createHash("sha256").update(sourceBytes).digest("hex"),
);
assert.equal(snapshot.source.requestedPath, source);
const readback = native.call("runtime_open", [
driveRootRuntime.handle,
"probe.bin",
]);
try {
assert.equal(readback.size, sourceBytes.byteLength);
assert.deepEqual(
native.call("read", [readback.handle, 0, 65536]),
sourceBytes.subarray(0, 65536),
);
} finally {
native.call("close", [readback.handle]);
}
} finally {
native.call("runtime_close", [driveRootRuntime.handle]);
}
assert.equal(await exists(driveRootRuntime.path), false);
report.controls.driveRootRuntimeParent = true;
const snapshotOwner = WindowsPrivateRuntime.create( const snapshotOwner = WindowsPrivateRuntime.create(
workspace.replace(/\\/u, "/"), workspace.replace(/\\/u, "/"),
"single-flight-", "single-flight-",
+2 -2
View File
@@ -8,13 +8,13 @@
"url": "https://github.com/morluto/rea", "url": "https://github.com/morluto/rea",
"source": "github" "source": "github"
}, },
"version": "6.3.0", "version": "6.4.0",
"packages": [ "packages": [
{ {
"registryType": "npm", "registryType": "npm",
"registryBaseUrl": "https://registry.npmjs.org", "registryBaseUrl": "https://registry.npmjs.org",
"identifier": "rea-agents", "identifier": "rea-agents",
"version": "6.3.0", "version": "6.4.0",
"runtimeHint": "npx", "runtimeHint": "npx",
"transport": { "transport": {
"type": "stdio" "type": "stdio"
+21 -20
View File
@@ -1,17 +1,21 @@
import { execFile } from "node:child_process";
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises"; import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { promisify } from "node:util";
import { parseBinary } from "plist";
import { z } from "zod"; import { z } from "zod";
import { preflightPropertyListDecode } from "../artifacts/apple/InterfaceBuilderDecodeBudget.js";
import { BinaryTargetError } from "../domain/configurationErrors.js"; import { BinaryTargetError } from "../domain/configurationErrors.js";
import { isPathWithinRoot } from "../domain/localPath.js"; import { isPathWithinRoot } from "../domain/localPath.js";
import { parseXmlPropertyList } from "../domain/propertyListKeys.js"; import {
omitPrototypeKeys,
parseXmlPropertyList,
} from "../domain/propertyListKeys.js";
import { decodeXmlPlistText } from "../domain/propertyListXmlText.js"; import { decodeXmlPlistText } from "../domain/propertyListXmlText.js";
import { err, ok, type Result } from "../domain/result.js"; import { err, ok, type Result } from "../domain/result.js";
const execFileAsync = promisify(execFile); /** Decoded representation bound for one binary Info.plist. */
const BINARY_PLIST_DECODE_BYTES = 16 * 1024 * 1024;
/** Where one app bundle layout keeps its Info.plist and program file. */ /** Where one app bundle layout keeps its Info.plist and program file. */
interface BundleLayout { interface BundleLayout {
@@ -193,9 +197,8 @@ const resolveLayoutExecutable = async (
try { try {
name = name =
plist.subarray(0, 6).toString("ascii") === "bplist" plist.subarray(0, 6).toString("ascii") === "bplist"
? await (fileSystem.decodeBinaryPlist ?? readBinaryPlistExecutable)( ? await (fileSystem.decodeBinaryPlist?.(plistPath) ??
plistPath, parseBinaryPlistExecutable(plist))
)
: parseXmlPlistExecutable(decodeXmlPlistText(plist)); : parseXmlPlistExecutable(decodeXmlPlistText(plist));
} catch (cause: unknown) { } catch (cause: unknown) {
return err( return err(
@@ -248,6 +251,10 @@ const resolveLayoutExecutable = async (
const parseXmlPlistExecutable = (plist: string): string => { const parseXmlPlistExecutable = (plist: string): string => {
// An unrelated `__proto__` entry must not make the bundle unreadable. // An unrelated `__proto__` entry must not make the bundle unreadable.
const { value } = parseXmlPropertyList(plist); const { value } = parseXmlPropertyList(plist);
return executableName(value);
};
const executableName = (value: unknown): string => {
const executable = executableEntrySchema.safeParse(value); const executable = executableEntrySchema.safeParse(value);
if (!executable.success) throw new Error("CFBundleExecutable is missing"); if (!executable.success) throw new Error("CFBundleExecutable is missing");
return executable.data.CFBundleExecutable; return executable.data.CFBundleExecutable;
@@ -255,19 +262,13 @@ const parseXmlPlistExecutable = (plist: string): string => {
const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() }); const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() });
const readBinaryPlistExecutable = async (plistPath: string): Promise<string> => /** Decode a binary plist in-process, so no host plist tool is required. */
// `-n` strips only the newline plutil appends; it requires macOS 12+. const parseBinaryPlistExecutable = (plist: Buffer): string => {
( preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
await execFileAsync("/usr/bin/plutil", [ // `plist` assigns a `__proto__` key as the prototype; copying drops it, as
"-extract", // the XML path does.
"CFBundleExecutable", return executableName(omitPrototypeKeys(parseBinary(plist)).value);
"raw", };
"-n",
"-o",
"-",
plistPath,
])
).stdout;
const isSafeExecutableName = (name: string): boolean => const isSafeExecutableName = (name: string): boolean =>
name.length > 0 && name.length > 0 &&
+2 -2
View File
@@ -138,7 +138,7 @@ export const runManagedProviderExecution = async (
signal?: AbortSignal, signal?: AbortSignal,
): Promise<Result<AnalysisExecution, AnalysisError>> => ): Promise<Result<AnalysisExecution, AnalysisError>> =>
withProcessCancellation(signal, async (operationSignal) => { withProcessCancellation(signal, async (operationSignal) => {
const session = dependencies.createManagedBinarySession(); const session = await dependencies.createManagedBinarySession();
return withSessionCleanup( return withSessionCleanup(
session, session,
async () => { async () => {
@@ -194,7 +194,7 @@ const runAnalysis = async (
const { logger, signal, snapshotPath } = options; const { logger, signal, snapshotPath } = options;
const config = dependencies.readConfiguration(); const config = dependencies.readConfiguration();
if (!config.ok) return cliError(config.error); if (!config.ok) return cliError(config.error);
const session = dependencies.createBinarySession(config.value, logger); const session = await dependencies.createBinarySession(config.value, logger);
return withSessionCleanup( return withSessionCleanup(
session, session,
async () => { async () => {
@@ -10,6 +10,8 @@ export interface DirectAnalysisDependencies {
readonly createBinarySession: ( readonly createBinarySession: (
config: AppConfig, config: AppConfig,
logger: Logger, logger: Logger,
) => BinarySession; ) => BinarySession | Promise<BinarySession>;
readonly createManagedBinarySession: () => BinarySession; readonly createManagedBinarySession: () =>
| BinarySession
| Promise<BinarySession>;
} }
+1 -1
View File
@@ -15,7 +15,7 @@ export const runSessionStatus = async (
): Promise<JsonValue> => { ): Promise<JsonValue> => {
const config = dependencies.readConfiguration(); const config = dependencies.readConfiguration();
if (!config.ok) return { error: projectAnalysisError(config.error) }; if (!config.ok) return { error: projectAnalysisError(config.error) };
const session = dependencies.createBinarySession(config.value, logger); const session = await dependencies.createBinarySession(config.value, logger);
try { try {
return { return {
...jsonObjectSchema.parse(session.status()), ...jsonObjectSchema.parse(session.status()),
@@ -0,0 +1,35 @@
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
summarizeEvidenceBundle,
type EvidenceBundleSummary,
type EvidenceRecordFilters,
} from "../../domain/evidenceRecordSummary.js";
/** Select complete bundle delivery or discovery metadata without analysis payloads. */
export const inspectEvidenceBundle = (
bundle: EvidenceBundle,
options: {
readonly detail: "complete" | "summary";
readonly filters?: EvidenceRecordFilters | undefined;
},
): Result<EvidenceBundle | EvidenceBundleSummary, AnalysisInputError> => {
if (options.detail === "complete" && options.filters !== undefined)
return err(
new AnalysisInputError("get_evidence_bundle", undefined, [
{
path: ["filters"],
reason: "invalid_value",
message:
"Record filters require detail: summary; complete returns the canonical bundle.",
expected: "detail: summary",
},
]),
);
return ok(
options.detail === "summary"
? summarizeEvidenceBundle(bundle, options.filters)
: bundle,
);
};
+7
View File
@@ -43,6 +43,7 @@ import {
type WebScreenshotDiff, type WebScreenshotDiff,
} from "../domain/webScreenshot.js"; } from "../domain/webScreenshot.js";
import { AnalysisError } from "../domain/analysisErrorBase.js"; import { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import { BrowserObservationError } from "../domain/browserObservationError.js"; import { BrowserObservationError } from "../domain/browserObservationError.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js"; import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js";
@@ -314,6 +315,12 @@ export class CdpBrowserProvider implements BrowserObservationPort {
try { try {
return ok(webScreenshotDiffSchema.parse(comparePngScreenshots(input))); return ok(webScreenshotDiffSchema.parse(comparePngScreenshots(input)));
} catch (cause: unknown) { } catch (cause: unknown) {
if (cause instanceof TypeError)
return err(
new AnalysisInputError("compare_web_screenshots", { cause }, [
{ path: [], reason: "invalid_format", message: cause.message },
]),
);
return err(providerError(cause, "compare_web_screenshots")); return err(providerError(cause, "compare_web_screenshots"));
} }
} }
@@ -606,3 +606,402 @@ describe("SVG xlink:href context", () => {
expect(dangling.urls).toEqual([]); expect(dangling.urls).toEqual([]);
}); });
}); });
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
const XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml";
const xhtmlRoot = (
attributes: readonly (readonly [string, string])[] = [
["xmlns", XHTML_NAMESPACE],
["xmlns:svg", SVG_NAMESPACE],
["xmlns:xlink", XLINK_NAMESPACE],
],
): SvgSnapshotNode => ({
name: "html",
parent: 0,
attributes,
});
const approvedHref = (
nodeIndex: number,
url: string,
): {
readonly node_index: number;
readonly attribute: "href";
readonly url: string;
readonly destination_scope: "approved";
} => ({
node_index: nodeIndex,
attribute: "href",
url,
destination_scope: "approved",
});
describe("XML SVG roots and namespace scope restoration", () => {
it.each(["svg", "document"])(
"resolves namespaces under an unprefixed XML %s root",
(name) => {
const result = captureSvg(
svgDocument([
{
name,
parent: 0,
attributes: [
["xmlns", SVG_NAMESPACE],
["xmlns:p", XLINK_NAMESPACE],
],
},
{ name: "a", parent: 1, attributes: [["p:href", "/linked"]] },
]),
);
expect(result.urls).toEqual([approvedHref(2, `${origin}/linked`)]);
},
);
it.each([2, 3, 99])(
"rejects self-parent, cyclic and dangling XML ancestry (%s)",
(parent) => {
const result = captureSvg(
svgDocument([
xhtmlRoot(),
{
name: "svg:a",
parent,
attributes: [
["xmlns:svg", SVG_NAMESPACE],
["xmlns:p", XLINK_NAMESPACE],
["p:href", "/invalid"],
],
},
{ name: "svg:g", parent: 2 },
]),
);
expect(result.urls).toEqual([]);
},
);
it("restores namespace declarations and undeclarations before visiting siblings", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot(),
{
name: "svg:a",
parent: 1,
attributes: [
["xmlns:xlink", ""],
["xlink:href", "/undeclared"],
],
},
{
name: "svg:a",
parent: 1,
attributes: [["xlink:href", "/inherited"]],
},
{
name: "svg:a",
parent: 1,
attributes: [
["xmlns:svg", "other"],
["xlink:href", "/rebound"],
],
},
{ name: "svg:a", parent: 1, attributes: [["xlink:href", "/restored"]] },
]),
);
expect(result.urls).toEqual([
approvedHref(3, `${origin}/inherited`),
approvedHref(5, `${origin}/restored`),
]);
});
});
describe("XHTML and XML SVG namespace prefixes", () => {
it("reports prefixed svg:a, svg:image, and an svg:a with no svg parent", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot(),
{ name: "svg:svg", parent: 1 },
{
name: "svg:a",
parent: 2,
attributes: [
["xlink:href", "/xl-pa1"],
["xlink:href", "/later"],
],
},
{
name: "svg:image",
parent: 2,
attributes: [["xlink:href", "guide"]],
},
{
name: "svg:a",
parent: 1,
attributes: [["xlink:href", "/xl-orphan"]],
},
]),
);
expect(result.nodes[3]?.node_name).toBe("svg:a");
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href"]);
expect(result.urls).toEqual([
approvedHref(3, `${origin}/xl-pa1`),
approvedHref(4, `${origin}/screens/assets/guide`),
approvedHref(5, `${origin}/xl-orphan`),
]);
});
it("reports svg:a under svg:foreignObject and skips XHTML anchors there", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot([
["xmlns", XHTML_NAMESPACE],
["xmlns:svg", SVG_NAMESPACE],
["xmlns:xlink", XLINK_NAMESPACE],
["xmlns:h", XHTML_NAMESPACE],
]),
{ name: "svg:svg", parent: 1 },
{ name: "svg:foreignObject", parent: 2 },
{
name: "a",
parent: 3,
attributes: [["xlink:href", "/xl-html-fo"]],
},
{
name: "h:a",
parent: 3,
attributes: [["xlink:href", "/xl-h"]],
},
{
name: "svg:a",
parent: 3,
attributes: [["xlink:href", "/xl-fo-svg"]],
},
]),
);
expect(urlFor(result, 4)).toBeUndefined();
expect(urlFor(result, 5)).toBeUndefined();
expect(result.urls).toEqual([approvedHref(6, `${origin}/xl-fo-svg`)]);
});
it("reads XLink by namespace, including a prefix other than xlink", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot([["xmlns", XHTML_NAMESPACE]]),
{
name: "svg",
parent: 1,
attributes: [
["xmlns", SVG_NAMESPACE],
["xmlns:xlink", "http://example.com/not-xlink"],
["xmlns:x", XLINK_NAMESPACE],
],
},
{
name: "a",
parent: 2,
attributes: [
["xlink:href", "/not-xlink"],
["x:href", "/xl-custom"],
["x:href", "/later"],
],
},
]),
);
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href", "x:href"]);
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-custom`)]);
});
it("keeps plain href ahead of a namespaced XLink href, including an empty href", () => {
const baseUrl = `${origin}/screens/assets/`;
const anchor = (
parent: number,
attributes: readonly (readonly [string, string])[],
): SvgSnapshotNode => ({
name: "svg:a",
parent,
attributes,
});
const result = captureSvg(
svgDocument(
[
xhtmlRoot(),
{ name: "svg:svg", parent: 1 },
anchor(2, [
["xlink:href", "/xlink-loses"],
["href", "/href-wins"],
]),
anchor(2, [
["href", "/href-wins"],
["xlink:href", "/xlink-loses"],
]),
anchor(2, [
["href", ""],
["xlink:href", "/xlink-should-lose"],
]),
],
baseUrl,
),
);
expect(result.urls).toEqual([
approvedHref(3, `${origin}/href-wins`),
approvedHref(4, `${origin}/href-wins`),
approvedHref(5, baseUrl),
]);
});
});
describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => {
it("classifies a prefixed SVG xlink:href outside the allowed origins as outside_policy", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot(),
{ name: "svg:svg", parent: 1 },
{
name: "svg:a",
parent: 2,
attributes: [["xlink:href", "https://cdn.example.test/out"]],
},
]),
);
expect(result.urls).toEqual([
{
node_index: 3,
attribute: "href",
url: null,
destination_scope: "outside_policy",
},
]);
});
it("does not treat bogus:svg in another namespace as SVG", () => {
const result = captureSvg(
svgDocument([
xhtmlRoot([
["xmlns", XHTML_NAMESPACE],
["xmlns:svg", SVG_NAMESPACE],
["xmlns:xlink", XLINK_NAMESPACE],
["xmlns:bogus", "http://example.com/not-svg"],
]),
{
name: "bogus:svg",
parent: 1,
attributes: [["xlink:href", "/xl-bogus"]],
},
{
name: "bogus:a",
parent: 2,
attributes: [["xlink:href", "/xl-bogus-a"]],
},
{
name: "svg:a",
parent: 1,
attributes: [["xlink:href", "/xl-kept"]],
},
]),
);
expect(result.nodes[2]?.node_name).toBe("bogus:svg");
expect(result.urls).toEqual([approvedHref(4, `${origin}/xl-kept`)]);
});
it("leaves HTML documents on literal svg and xlink:href names", () => {
const result = captureSvg(
svgDocument([
{ name: "HTML", parent: 0 },
{
name: "svg",
parent: 1,
attributes: [
["xmlns", SVG_NAMESPACE],
["xmlns:x", XLINK_NAMESPACE],
],
},
{
name: "a",
parent: 2,
attributes: [["xlink:href", "/xl-literal"]],
},
{
name: "a",
parent: 2,
attributes: [["x:href", "/xl-custom-html"]],
},
{
name: "svg:a",
parent: 1,
attributes: [
["xmlns:svg", SVG_NAMESPACE],
["xmlns:xlink", XLINK_NAMESPACE],
["xlink:href", "/xl-prefixed-html"],
],
},
{
name: "A",
parent: 1,
attributes: [["xlink:href", "/xl-html-anchor"]],
},
]),
);
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-literal`)]);
});
it("applies an element's own xmlns declarations to its name and href", () => {
const result = captureSvg(
svgDocument([
{ name: "html", parent: 0, attributes: [["xmlns", XHTML_NAMESPACE]] },
{
name: "svg:a",
parent: 1,
attributes: [
["xmlns:svg", SVG_NAMESPACE],
["xmlns:xlink", XLINK_NAMESPACE],
["xlink:href", "/xl-self"],
],
},
]),
);
expect(result.urls).toEqual([approvedHref(2, `${origin}/xl-self`)]);
});
it("resolves a prefix declared on a later ancestor", () => {
const result = captureSvg(
svgDocument([
{
name: "svg:a",
parent: 2,
attributes: [["xlink:href", "/forward"]],
},
{ name: "svg:svg", parent: 3 },
xhtmlRoot(),
]),
);
expect(result.urls).toEqual([approvedHref(1, `${origin}/forward`)]);
});
it("resolves a deep chain with distinct namespace declarations without quadratic expansion", () => {
const depth = 12_000;
const elements: SvgSnapshotNode[] = [
xhtmlRoot(),
{ name: "svg:svg", parent: 1 },
];
let parent = 2;
for (let index = 0; index < depth; index += 1) {
elements.push({
name: "svg:g",
parent,
attributes: [[`xmlns:p${index}`, `urn:prefix:${index}`]],
});
parent = elements.length;
}
elements.push({
name: "svg:a",
parent,
attributes: [["xlink:href", "/deep"]],
});
const snapshot = svgDocument(elements);
const result = captureSvg(snapshot);
expect(result.urls).toEqual([
approvedHref(elements.length, `${origin}/deep`),
]);
}, 30_000);
});
+40 -60
View File
@@ -11,6 +11,7 @@ import {
} from "./CdpCaptureValues.js"; } from "./CdpCaptureValues.js";
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js"; import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js"; import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
import { createDocumentSvgAncestry, domSvgLink } from "./CdpCaptureDomSvg.js";
export type CapturedResource = Omit< export type CapturedResource = Omit<
WebPageInspection["resources"][number], WebPageInspection["resources"][number],
@@ -171,24 +172,20 @@ export const captureDom = (
? documentNodes.attributes ? documentNodes.attributes
: []; : [];
const baseIndex = nodes.length; const baseIndex = nodes.length;
const svgContexts = new Map<number, boolean>(); const ancestry = createDocumentSvgAncestry({
nodeTypes,
nodeNames,
parents,
strings,
attributes,
});
total += nodeTypes.length; total += nodeTypes.length;
for (let index = 0; index < nodeTypes.length; index += 1) { for (let index = 0; index < nodeTypes.length; index += 1) {
const attributeIndexes = numberArray(attributes[index]); const attributeIndexes = numberArray(attributes[index]);
const parent = Math.trunc(parents[index] ?? -1); const parent = Math.trunc(parents[index] ?? -1);
const nodeIndex = nodes.length; const nodeIndex = nodes.length;
const nodeName = indexedString(strings, nodeNames[index]); const nodeName = indexedString(strings, nodeNames[index]);
const rawAttributeNames = attributeIndexes const attributeNames = uniqueAttributeNames(attributeIndexes, strings);
.filter((_value, attributeIndex) => attributeIndex % 2 === 0)
.map((value) => indexedString(strings, value));
const seenAttributeNames = new Set<string>();
const attributeNames: string[] = [];
for (const name of rawAttributeNames) {
if (!seenAttributeNames.has(name)) {
seenAttributeNames.add(name);
attributeNames.push(name);
}
}
nodes.push({ nodes.push({
index: nodeIndex, index: nodeIndex,
parent_index: parent < 0 ? -1 : baseIndex + parent, parent_index: parent < 0 ? -1 : baseIndex + parent,
@@ -206,13 +203,7 @@ export const captureDom = (
nodeIndex, nodeIndex,
nodeName, nodeName,
allowedOrigins, allowedOrigins,
svgElement: elementInSvgContext( ...domSvgLink(ancestry, index),
index,
nodeNames,
parents,
strings,
svgContexts,
),
}); });
for (const url of metadata.urls) { for (const url of metadata.urls) {
urls.push(url); urls.push(url);
@@ -367,6 +358,21 @@ const indexedString = (strings: readonly string[], index: unknown): string => {
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? ""); return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
}; };
const uniqueAttributeNames = (
attributeIndexes: readonly number[],
strings: readonly string[],
): string[] => {
const names: string[] = [];
const seen = new Set<string>();
for (let index = 0; index < attributeIndexes.length; index += 2) {
const name = indexedString(strings, attributeIndexes[index]);
if (seen.has(name)) continue;
seen.add(name);
names.push(name);
}
return names;
};
const numberArray = (value: unknown): readonly number[] => const numberArray = (value: unknown): readonly number[] =>
Array.isArray(value) Array.isArray(value)
? value.flatMap((item) => { ? value.flatMap((item) => {
@@ -384,41 +390,22 @@ interface DomMetadataOptions {
readonly nodeName: string; readonly nodeName: string;
readonly allowedOrigins: ReadonlySet<string>; readonly allowedOrigins: ReadonlySet<string>;
readonly svgElement: boolean; readonly svgElement: boolean;
readonly xmlDocument: boolean;
readonly xlinkHref: string | undefined;
} }
// DOMSnapshot carries no namespace URI. Chrome's HTML parser records SVG // Chrome reads an SVG XLink href only when href itself is absent, and an
// local names (`svg`, `foreignObject`). Descendants of foreignObject are HTML // empty href still wins. HTML keeps the literal attribute name `xlink:href`.
// until a nested `svg`. // XML uses any prefix bound to the XLink namespace. Both are stored as href.
const elementInSvgContext = ( const hrefAttributeValue = (
index: number, pairs: ReadonlyMap<string, string>,
nodeNames: readonly number[], attribute: string,
parents: readonly number[], options: DomMetadataOptions,
strings: readonly string[], ): string | undefined => {
contexts: Map<number, boolean>, const declared = pairs.get(attribute);
): boolean => { if (declared !== undefined) return declared;
let current = index; if (attribute !== "href" || !options.svgElement) return undefined;
const seen = new Set<number>(); return options.xmlDocument ? options.xlinkHref : pairs.get("xlink:href");
let inSvg = false;
while (current >= 0 && current < nodeNames.length && !seen.has(current)) {
const cached = contexts.get(current);
if (cached !== undefined) {
inSvg = cached;
break;
}
seen.add(current);
if (indexedString(strings, nodeNames[current]) === "svg") {
inSvg = true;
break;
}
const parent = Math.trunc(parents[current] ?? -1);
if (parent < 0 || parent === current) break;
if (indexedString(strings, nodeNames[parent]) === "foreignObject") break;
current = parent;
}
// Cache the entire examined chain, including forward references and cycles,
// so shared ancestors are examined once per document.
for (const node of seen) contexts.set(node, inSvg);
return inSvg;
}; };
const domMetadata = ( const domMetadata = (
@@ -435,7 +422,6 @@ const domMetadata = (
nodeIndex, nodeIndex,
nodeName, nodeName,
allowedOrigins, allowedOrigins,
svgElement,
} = options; } = options;
const pairs = new Map<string, string>(); const pairs = new Map<string, string>();
for (let index = 0; index + 1 < attributes.length; index += 2) { for (let index = 0; index + 1 < attributes.length; index += 2) {
@@ -447,13 +433,7 @@ const domMetadata = (
} }
const urls: WebPageInspection["metadata"]["dom_urls"] = []; const urls: WebPageInspection["metadata"]["dom_urls"] = [];
for (const attribute of domUrlAttributes) { for (const attribute of domUrlAttributes) {
// Chrome reads SVG xlink:href only when href itself is absent, and an const value = hrefAttributeValue(pairs, attribute, options);
// empty href still wins. Report the fallback as href.
const value =
pairs.get(attribute) ??
(attribute === "href" && svgElement
? pairs.get("xlink:href")
: undefined);
if (value === undefined) continue; if (value === undefined) continue;
const tagName = nodeName.toLowerCase(); const tagName = nodeName.toLowerCase();
// Chrome strips HTML whitespace for FORM.action, while submit controls // Chrome strips HTML whitespace for FORM.action, while submit controls
+275
View File
@@ -0,0 +1,275 @@
import { numberValue } from "./CdpCaptureValues.js";
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
interface DomSnapshotNodes {
readonly nodeTypes: readonly number[];
readonly nodeNames: readonly number[];
readonly parents: readonly number[];
readonly strings: readonly string[];
readonly attributes: readonly unknown[];
}
interface XmlSvgLink {
readonly svgElement: boolean;
readonly xlinkHref: string | undefined;
}
interface DocumentSvgAncestry {
readonly xml: boolean;
readonly nodes: DomSnapshotNodes;
readonly svgContexts: Map<number, boolean>;
readonly xmlLinks: Map<number, XmlSvgLink>;
}
interface DomSvgLink {
readonly svgElement: boolean;
readonly xmlDocument: boolean;
readonly xlinkHref: string | undefined;
}
const UNRESOLVED: XmlSvgLink = {
svgElement: false,
xlinkHref: undefined,
};
const stringAt = (strings: readonly string[], index: unknown): string => {
const integer = numberValue(index);
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
};
const numberArray = (value: unknown): readonly number[] =>
Array.isArray(value)
? value.flatMap((item) => {
const number = numberValue(item);
return number === undefined ? [] : [number];
})
: [];
// DOMSnapshot has no namespace URI. HTML documents use uppercase `HTML` and
// the parser's local names. XML documents preserve prefixed names and xmlns
// declarations, including those on standalone SVG roots.
const xmlDocument = (nodes: DomSnapshotNodes): boolean => {
const documentIndex = nodes.nodeTypes.findIndex(
(nodeType) => Math.trunc(nodeType) === 9,
);
if (documentIndex < 0) return false;
for (let index = 0; index < nodes.nodeTypes.length; index += 1) {
if (Math.trunc(nodes.nodeTypes[index] ?? 0) !== 1) continue;
if (Math.trunc(nodes.parents[index] ?? -1) !== documentIndex) continue;
const name = stringAt(nodes.strings, nodes.nodeNames[index]);
if (name === "HTML") return false;
if (name === "html" || name.includes(":")) return true;
// Explicit namespaces also identify standalone SVG and other XML roots.
// Namespace-free synthetic snapshots retain the legacy HTML behavior.
const attributes = numberArray(nodes.attributes[index]);
return attributes.some(
(value, offset) =>
offset % 2 === 0 &&
xmlnsPrefix(stringAt(nodes.strings, value)) !== undefined,
);
}
return false;
};
export const createDocumentSvgAncestry = (
nodes: DomSnapshotNodes,
): DocumentSvgAncestry => {
const xml = xmlDocument(nodes);
return {
xml,
nodes,
svgContexts: new Map(),
xmlLinks: xml ? resolveXmlElements(nodes) : new Map(),
};
};
export const domSvgLink = (
ancestry: DocumentSvgAncestry,
index: number,
): DomSvgLink => {
if (!ancestry.xml) {
return {
svgElement: elementInSvgContext(
index,
ancestry.nodes,
ancestry.svgContexts,
),
xmlDocument: false,
xlinkHref: undefined,
};
}
const resolved = ancestry.xmlLinks.get(index) ?? UNRESOLVED;
return {
svgElement: resolved.svgElement,
xmlDocument: true,
xlinkHref: resolved.xlinkHref,
};
};
// Chrome's HTML parser records SVG local names (`svg`, `foreignObject`).
// Descendants of foreignObject are HTML until a nested `svg`.
const elementInSvgContext = (
index: number,
nodes: DomSnapshotNodes,
contexts: Map<number, boolean>,
): boolean => {
let current = index;
const seen = new Set<number>();
let inSvg = false;
while (
current >= 0 &&
current < nodes.nodeNames.length &&
!seen.has(current)
) {
const cached = contexts.get(current);
if (cached !== undefined) {
inSvg = cached;
break;
}
seen.add(current);
if (stringAt(nodes.strings, nodes.nodeNames[current]) === "svg") {
inSvg = true;
break;
}
const parent = Math.trunc(nodes.parents[current] ?? -1);
if (parent < 0 || parent === current) break;
if (stringAt(nodes.strings, nodes.nodeNames[parent]) === "foreignObject")
break;
current = parent;
}
// Cache the entire examined chain, including forward references and cycles,
// so shared ancestors are examined once per document.
for (const node of seen) contexts.set(node, inSvg);
return inSvg;
};
interface NamespaceRestore {
readonly prefix: string;
readonly previous: string | undefined;
}
type XmlWalkStep =
| { readonly node: number }
| { readonly restore: readonly NamespaceRestore[] };
// Walk parent-before-child with one mutable namespace environment. Restore
// declarations on exit so siblings retain their own scope. Only declarations
// on the active path are retained, rather than cumulative maps at every node.
const resolveXmlElements = (
nodes: DomSnapshotNodes,
): Map<number, XmlSvgLink> => {
const children = new Map<number, number[]>();
const pending: XmlWalkStep[] = [];
for (let node = 0; node < nodes.nodeNames.length; node += 1) {
const parent = Math.trunc(nodes.parents[node] ?? -1);
if (parent < 0) pending.push({ node });
else if (parent < nodes.nodeNames.length && parent !== node) {
const siblings = children.get(parent);
if (siblings === undefined) children.set(parent, [node]);
else siblings.push(node);
}
}
const bindings = new Map<string, string>();
const resolved = new Map<number, XmlSvgLink>();
while (pending.length > 0) {
const step = pending.pop();
if (step === undefined) break;
if ("restore" in step) {
for (const { prefix, previous } of step.restore) {
if (previous === undefined) bindings.delete(prefix);
else bindings.set(prefix, previous);
}
continue;
}
const { node } = step;
const attributes = numberArray(nodes.attributes[node]);
const restore = applyDeclarations(bindings, attributes, nodes.strings);
const svgElement =
Math.trunc(nodes.nodeTypes[node] ?? 0) === 1 &&
elementNamespace(
stringAt(nodes.strings, nodes.nodeNames[node]),
bindings,
) === SVG_NAMESPACE;
resolved.set(node, {
svgElement,
xlinkHref: svgElement
? xlinkHrefValue(attributes, nodes.strings, bindings)
: undefined,
});
pending.push({ restore });
for (const child of children.get(node) ?? []) pending.push({ node: child });
}
// Cycles (including self-parents) and dangling chains cannot reach a root,
// so remain unresolved instead of deriving URLs from an invalid ancestry.
return resolved;
};
const applyDeclarations = (
bindings: Map<string, string>,
attributes: readonly number[],
strings: readonly string[],
): readonly NamespaceRestore[] => {
const restore: NamespaceRestore[] = [];
const seen = new Set<string>();
for (let index = 0; index + 1 < attributes.length; index += 2) {
const prefix = xmlnsPrefix(stringAt(strings, attributes[index]));
if (prefix === undefined || seen.has(prefix)) continue;
seen.add(prefix);
restore.push({ prefix, previous: bindings.get(prefix) });
const value = stringAt(strings, attributes[index + 1]);
if (value.length === 0) bindings.delete(prefix);
else bindings.set(prefix, value);
}
return restore;
};
const xmlnsPrefix = (name: string): string | undefined => {
if (name === "xmlns") return "";
if (!name.startsWith("xmlns:")) return undefined;
const prefix = name.slice("xmlns:".length);
if (prefix.length === 0 || prefix === "xmlns" || prefix.includes(":"))
return undefined;
return prefix;
};
const elementNamespace = (
nodeName: string,
bindings: ReadonlyMap<string, string>,
): string => {
const colon = nodeName.indexOf(":");
if (colon < 0) return bindings.get("") ?? "";
const prefix = nodeName.slice(0, colon);
const local = nodeName.slice(colon + 1);
if (
prefix.length === 0 ||
local.length === 0 ||
local.includes(":") ||
prefix === "xmlns"
)
return "";
return bindings.get(prefix) ?? "";
};
// First prefixed href bound to the XLink namespace. Duplicate names keep the
// earlier value. Plain href is applied by the caller and wins even when empty.
const xlinkHrefValue = (
attributes: readonly number[],
strings: readonly string[],
bindings: ReadonlyMap<string, string>,
): string | undefined => {
const seen = new Set<string>();
for (let index = 0; index + 1 < attributes.length; index += 2) {
const name = stringAt(strings, attributes[index]);
if (seen.has(name)) continue;
seen.add(name);
const colon = name.indexOf(":");
if (colon <= 0) continue;
const prefix = name.slice(0, colon);
const local = name.slice(colon + 1);
if (local !== "href" || bindings.get(prefix) !== XLINK_NAMESPACE) continue;
return stringAt(strings, attributes[index + 1]);
}
return undefined;
};
+268 -1
View File
@@ -3,6 +3,10 @@ import { crc32, deflateSync } from "node:zlib";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { comparePngScreenshots } from "./PngVisualDiff.js"; import { comparePngScreenshots } from "./PngVisualDiff.js";
import {
AnalysisResourceConstraintError,
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import { import {
compareWebScreenshotsInputSchema, compareWebScreenshotsInputSchema,
createWebScreenshotArtifact, createWebScreenshotArtifact,
@@ -117,7 +121,7 @@ describe("PNG visual diff", () => {
}); });
it("rejects malformed PNG dimensions after validating image data", () => { it("rejects malformed PNG dimensions after validating image data", () => {
const image = artifact(32_000_001, 1, [0, 0, 0, 255]); const image = artifact(2, 1, [0, 0, 0, 255]);
expect(() => expect(() =>
comparePngScreenshots( comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({ compareWebScreenshotsInputSchema.parse({
@@ -149,6 +153,256 @@ describe("PNG visual diff", () => {
}); });
}); });
describe("PNG comparison memory budget", () => {
it("classifies legal huge dimensions as a memory constraint before inflate", () => {
const image = createWebScreenshotArtifact(headerOnlyPng(0x7fffffff, 1));
const input = compareWebScreenshotsInputSchema.parse({
before: image,
after: image,
});
try {
comparePngScreenshots(input);
throw new Error("expected PNG working-memory rejection");
} catch (cause: unknown) {
expect(cause).toBeInstanceOf(AnalysisResourceConstraintError);
expect(cause).toMatchObject({
resource: "memory",
reportedLimits: {
maximum_working_memory_bytes: 256 * 1024 * 1024,
estimated_working_memory_bytes: "34359738499",
before_dimensions: { width: 0x7fffffff, height: 1 },
after_dimensions: { width: 0x7fffffff, height: 1 },
},
});
}
});
});
describe("PNG chunk integrity", () => {
it.each(["IHDR", "IDAT", "IEND"])(
"rejects a corrupt %s chunk checksum",
(type) => {
const bytes = png(1, 1, [0, 0, 0, 255]);
let offset = 8;
while (offset < bytes.length) {
const length = bytes.readUInt32BE(offset);
if (bytes.subarray(offset + 4, offset + 8).toString("ascii") === type) {
bytes[offset + 8 + length] = (bytes[offset + 8 + length] ?? 0) ^ 1;
break;
}
offset += length + 12;
}
const image = createWebScreenshotArtifact(bytes);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: image,
after: image,
}),
),
).toThrow("Invalid PNG chunk checksum");
},
);
it("reports a CRC-valid but corrupt compressed stream as invalid PNG data", () => {
const header = Buffer.from(png(1, 1, [0, 0, 0, 255]).subarray(16, 29));
const image = createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
chunk("IHDR", header),
chunk("IDAT", Buffer.from("bad")),
chunk("IEND", Buffer.alloc(0)),
]),
);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({ before: image, after: image }),
),
).toThrow("Invalid PNG image data");
});
it("rejects high-bit bytes in a CRC-valid PNG chunk name", () => {
const bytes = png(1, 1, [0, 0, 0, 255]);
const headerChunk = Buffer.from(bytes.subarray(8, 33));
headerChunk[4] = 0xc9;
headerChunk.writeUInt32BE(crc32(headerChunk.subarray(4, 21)), 21);
const malformed = createWebScreenshotArtifact(
Buffer.concat([bytes.subarray(0, 8), headerChunk, bytes.subarray(33)]),
);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: malformed,
after: malformed,
}),
),
).toThrow("Invalid PNG chunk type");
});
it("rejects duplicate image headers and unknown critical chunks while preserving ancillary chunks", () => {
const bytes = png(1, 1, [0, 0, 0, 255]);
const duplicate = Buffer.concat([
bytes.subarray(0, 33),
bytes.subarray(8, 33),
bytes.subarray(33),
]);
const unknown = Buffer.concat([
bytes.subarray(0, 33),
chunk("ABCD", Buffer.alloc(0)),
bytes.subarray(33),
]);
for (const malformed of [duplicate, unknown]) {
const image = createWebScreenshotArtifact(malformed);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: image,
after: image,
}),
),
).toThrow(/Invalid PNG header order|Unsupported critical PNG chunk/u);
}
const ancillary = createWebScreenshotArtifact(
Buffer.concat([
bytes.subarray(0, 33),
chunk("tEXt", Buffer.from("Comment\0original capture")),
bytes.subarray(33),
]),
);
expect(
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: ancillary,
after: createWebScreenshotArtifact(bytes),
}),
).status,
).toBe("identical");
});
it("rejects bytes after IEND and reports valid unsupported PNG encodings separately", () => {
const bytes = png(1, 1, [0, 0, 0, 255]);
const trailing = createWebScreenshotArtifact(
Buffer.concat([bytes, Buffer.from([0])]),
);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: trailing,
after: trailing,
}),
),
).toThrow("Invalid PNG end chunk");
const grayscaleHeader = Buffer.from(bytes.subarray(16, 29));
grayscaleHeader[9] = 0;
const grayscale = createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
chunk("IHDR", grayscaleHeader),
chunk("IDAT", deflateSync(Buffer.from([0, 30]))),
chunk("IEND", Buffer.alloc(0)),
]),
);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: grayscale,
after: grayscale,
}),
),
).toThrow(AnalysisUnsupportedTargetError);
});
});
describe("PNG chunk ordering", () => {
it("rejects malformed palettes, interrupted data and misplaced transparency while retaining valid chunks", () => {
const header = Buffer.from(png(1, 1, [10, 20, 30, 255]).subarray(16, 29));
header[9] = 2;
const ihdr = chunk("IHDR", header);
const compressed = deflateSync(Buffer.from([0, 10, 20, 30]));
const idat = chunk("IDAT", compressed);
const iend = chunk("IEND", Buffer.alloc(0));
const palette = chunk("PLTE", Buffer.from([10, 20, 30]));
const transparency = chunk("tRNS", Buffer.from([0, 10, 0, 20, 0, 30]));
const ancillary = chunk("tEXt", Buffer.from("Comment\0retained"));
const image = (...chunks: readonly Buffer[]) =>
createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
...chunks,
]),
);
const original = image(ihdr, idat, iend);
const cases = [
[ihdr, chunk("PLTE", Buffer.alloc(1)), idat, iend],
[ihdr, palette, palette, idat, iend],
[ihdr, idat, palette, iend],
[ihdr, transparency, palette, idat, iend],
[ihdr, transparency, transparency, idat, iend],
[ihdr, idat, transparency, iend],
[
ihdr,
chunk("IDAT", compressed.subarray(0, 6)),
ancillary,
chunk("IDAT", compressed.subarray(6)),
iend,
],
[ihdr, idat, chunk("IEND", Buffer.from([1]))],
];
for (const chunks of cases) {
const malformed = image(...chunks);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: malformed,
after: original,
}),
),
).toThrow(
/Invalid PNG palette|Invalid PNG transparency order|Nonconsecutive PNG image data|Invalid PNG end chunk/u,
);
}
const indexedHeader = Buffer.from(header);
indexedHeader[8] = 1;
indexedHeader[9] = 3;
const emptyTransparency = image(
chunk("IHDR", indexedHeader),
chunk("PLTE", Buffer.from([10, 20, 30])),
chunk("tRNS", Buffer.alloc(0)),
chunk("IDAT", deflateSync(Buffer.from([0, 0]))),
iend,
);
expect(() =>
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: emptyTransparency,
after: original,
}),
),
).toThrow("Unsupported PNG transparency");
for (const valid of [
image(ihdr, palette, idat, iend),
image(ihdr, ancillary, idat, iend),
image(
ihdr,
chunk("IDAT", compressed.subarray(0, 6)),
chunk("IDAT", compressed.subarray(6)),
iend,
),
]) {
expect(
comparePngScreenshots(
compareWebScreenshotsInputSchema.parse({
before: valid,
after: original,
}),
).status,
).toBe("identical");
}
});
});
const artifact = (width: number, height: number, pixels: readonly number[]) => const artifact = (width: number, height: number, pixels: readonly number[]) =>
createWebScreenshotArtifact(png(width, height, pixels)); createWebScreenshotArtifact(png(width, height, pixels));
@@ -173,6 +427,19 @@ const png = (
]); ]);
}; };
const headerOnlyPng = (width: number, height: number): Buffer => {
const header = Buffer.alloc(13);
header.writeUInt32BE(width, 0);
header.writeUInt32BE(height, 4);
header.set([8, 6, 0, 0, 0], 8);
return Buffer.concat([
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
chunk("IHDR", header),
chunk("IDAT", deflateSync(Buffer.from([0]))),
chunk("IEND", Buffer.alloc(0)),
]);
};
const chunk = (type: string, data: Buffer): Buffer => { const chunk = (type: string, data: Buffer): Buffer => {
const result = Buffer.alloc(12 + data.byteLength); const result = Buffer.alloc(12 + data.byteLength);
result.writeUInt32BE(data.byteLength, 0); result.writeUInt32BE(data.byteLength, 0);
+267 -31
View File
@@ -1,10 +1,14 @@
import { inflateSync } from "node:zlib"; import { constants as zlibConstants, crc32, inflateSync } from "node:zlib";
import type { import type {
CompareWebScreenshotsInput, CompareWebScreenshotsInput,
WebScreenshotArtifact, WebScreenshotArtifact,
WebScreenshotDiff, WebScreenshotDiff,
} from "../domain/webScreenshot.js"; } from "../domain/webScreenshot.js";
import {
AnalysisResourceConstraintError,
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import { decodeCanonicalBase64 } from "../domain/webScreenshot.js"; import { decodeCanonicalBase64 } from "../domain/webScreenshot.js";
interface DecodedPng { interface DecodedPng {
@@ -13,12 +17,32 @@ interface DecodedPng {
readonly rgba: Buffer; readonly rgba: Buffer;
} }
interface ParsedPng {
readonly bytes: Buffer;
readonly width: number;
readonly height: number;
readonly channels: number;
readonly compressed: readonly Buffer[];
readonly transparentColor: readonly number[] | undefined;
readonly expectedRawBytes: bigint;
readonly unfilteredBytes: bigint;
readonly rgbaBytes: bigint;
}
/** Conservative allowance for the simultaneously live PNG comparison buffers. */
const PNG_COMPARISON_WORKING_MEMORY_BYTES = 256 * 1024 * 1024;
const PNG_INFLATE_CHUNK_BYTES = zlibConstants.Z_DEFAULT_CHUNK;
/** Compare two validated screenshots using deterministic RGBA metrics. */ /** Compare two validated screenshots using deterministic RGBA metrics. */
export const comparePngScreenshots = ( export const comparePngScreenshots = (
input: CompareWebScreenshotsInput, input: CompareWebScreenshotsInput,
): WebScreenshotDiff => { ): WebScreenshotDiff => {
const before = decodePng(input.before); admitEncodedInputPair(input.before, input.after);
const after = decodePng(input.after); const beforePlan = parsePng(input.before, "before");
const afterPlan = parsePng(input.after, "after");
admitPngPair(beforePlan, afterPlan);
const before = decodePng(beforePlan);
const after = decodePng(afterPlan);
if (before.width !== after.width || before.height !== after.height) if (before.width !== after.width || before.height !== after.height)
return { return {
status: "dimension_mismatch", status: "dimension_mismatch",
@@ -73,7 +97,10 @@ export const comparePngScreenshots = (
}; };
}; };
const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => { const parsePng = (
artifact: WebScreenshotArtifact,
field: "before" | "after",
): ParsedPng => {
const bytes = decodeCanonicalBase64(artifact.data_base64); const bytes = decodeCanonicalBase64(artifact.data_base64);
if (bytes === undefined || !bytes.subarray(0, 8).equals(PNG_SIGNATURE)) if (bytes === undefined || !bytes.subarray(0, 8).equals(PNG_SIGNATURE))
throw new TypeError("Invalid PNG signature"); throw new TypeError("Invalid PNG signature");
@@ -81,68 +108,277 @@ const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => {
let header: ReturnType<typeof parseHeader> | undefined; let header: ReturnType<typeof parseHeader> | undefined;
const compressed: Buffer[] = []; const compressed: Buffer[] = [];
let sawEnd = false; let sawEnd = false;
let sawPalette = false;
let imageDataEnded = false;
let paletteEntries = 0;
let transparentColor: readonly number[] | undefined; let transparentColor: readonly number[] | undefined;
let sawTransparency = false;
let unsupportedReason: string | undefined;
while (offset < bytes.length) { while (offset < bytes.length) {
if (offset + 12 > bytes.length) throw new TypeError("Truncated PNG chunk"); if (offset + 12 > bytes.length) throw new TypeError("Truncated PNG chunk");
const length = bytes.readUInt32BE(offset); const length = bytes.readUInt32BE(offset);
const type = bytes.subarray(offset + 4, offset + 8).toString("ascii"); if (length > 0x7fffffff) throw new TypeError("Invalid PNG chunk length");
const type = bytes.subarray(offset + 4, offset + 8).toString("latin1");
const dataStart = offset + 8; const dataStart = offset + 8;
const dataEnd = dataStart + length; const dataEnd = dataStart + length;
if (dataEnd + 4 > bytes.length) throw new TypeError("Truncated PNG data"); if (dataEnd + 4 > bytes.length) throw new TypeError("Truncated PNG data");
const data = bytes.subarray(dataStart, dataEnd); const data = bytes.subarray(dataStart, dataEnd);
if (type === "IHDR") header = parseHeader(data); if (
crc32(bytes.subarray(offset + 4, dataEnd)) !== bytes.readUInt32BE(dataEnd)
)
throw new TypeError("Invalid PNG chunk checksum");
if (!/^[A-Za-z]{4}$/u.test(type) || type[2] !== type[2]?.toUpperCase())
throw new TypeError("Invalid PNG chunk type");
if (type !== "IDAT" && compressed.length > 0) imageDataEnded = true;
if (type === "IHDR") {
if (offset !== 8 || header !== undefined)
throw new TypeError("Invalid PNG header order");
header = parseHeader(data);
} else if (header === undefined)
throw new TypeError("Invalid PNG header order");
else if (type === "tRNS") { else if (type === "tRNS") {
if (header?.channels !== 3 || data.length !== 6) if (sawTransparency || compressed.length > 0)
throw new TypeError("Invalid PNG transparency order");
sawTransparency = true;
if (header.colorType === 2 && data.length === 6) {
transparentColor = [
data.readUInt16BE(0),
data.readUInt16BE(2),
data.readUInt16BE(4),
];
} else if (
(header.colorType === 0 && data.length === 2) ||
(header.colorType === 3 &&
sawPalette &&
data.length > 0 &&
data.length <= paletteEntries)
) {
unsupportedReason ??=
"PNG transparency for this color type is not supported.";
} else {
throw new TypeError("Unsupported PNG transparency"); throw new TypeError("Unsupported PNG transparency");
transparentColor = [ }
data.readUInt16BE(0), } else if (type === "PLTE") {
data.readUInt16BE(2), if (
data.readUInt16BE(4), sawPalette ||
]; sawTransparency ||
} else if (type === "IDAT") compressed.push(data); compressed.length > 0 ||
else if (type === "IEND") { length === 0 ||
length > 768 ||
length % 3 !== 0 ||
(header.colorType === 3 && length / 3 > 2 ** header.bitDepth) ||
(header.colorType !== 2 &&
header.colorType !== 3 &&
header.colorType !== 6)
)
throw new TypeError("Invalid PNG palette");
sawPalette = true;
paletteEntries = length / 3;
} else if (type === "IDAT") {
if (imageDataEnded) throw new TypeError("Nonconsecutive PNG image data");
compressed.push(data);
} else if (type === "IEND") {
if (length !== 0 || dataEnd + 4 !== bytes.length)
throw new TypeError("Invalid PNG end chunk");
sawEnd = true; sawEnd = true;
break; break;
} else if (/^[A-Z]/u.test(type)) {
unsupportedReason ??= `Unsupported critical PNG chunk ${type}.`;
} }
offset = dataEnd + 4; offset = dataEnd + 4;
} }
if (header === undefined || !sawEnd || compressed.length === 0) if (header === undefined || !sawEnd || compressed.length === 0)
throw new TypeError("Incomplete PNG"); throw new TypeError("Incomplete PNG");
const rowBytes = header.width * header.channels; if (header.colorType === 3 && !sawPalette)
const expected = (rowBytes + 1) * header.height; throw new TypeError("Missing PNG palette");
const raw = inflateSync(Buffer.concat(compressed), { if (unsupportedReason !== undefined)
maxOutputLength: expected + 1, throw new AnalysisUnsupportedTargetError(
}); "compare_web_screenshots",
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size"); field,
unsupportedReason,
);
if (!header.supported)
throw new AnalysisUnsupportedTargetError(
"compare_web_screenshots",
field,
"PNG encoding is not supported for pixel comparison.",
);
const pixels = BigInt(header.width) * BigInt(header.height);
const expectedRawBytes =
(BigInt(header.width) * BigInt(header.channels) + 1n) *
BigInt(header.height);
return { return {
bytes,
width: header.width, width: header.width,
height: header.height, height: header.height,
channels: header.channels,
compressed,
transparentColor,
expectedRawBytes,
unfilteredBytes: pixels * BigInt(header.channels),
rgbaBytes: pixels * 4n,
};
};
const decodePng = (plan: ParsedPng): DecodedPng => {
const expected = Number(plan.expectedRawBytes);
const raw = inflatePng(Buffer.concat(plan.compressed), expected + 1);
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size");
return {
width: plan.width,
height: plan.height,
rgba: expandRgba( rgba: expandRgba(
unfilter(raw, header.width, header.height, header.channels), unfilter(raw, plan.width, plan.height, plan.channels),
header.channels, plan.channels,
transparentColor, plan.transparentColor,
), ),
}; };
}; };
const admitEncodedInputPair = (
before: WebScreenshotArtifact,
after: WebScreenshotArtifact,
): void => {
const sourceBytes = BigInt(before.bytes) + BigInt(after.bytes);
if (sourceBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
throw pngWorkingMemoryError(sourceBytes, null, null);
};
const admitPngPair = (before: ParsedPng, after: ParsedPng): void => {
// Input buffers remain live for both plans. For each decode, count the
// concatenated IDAT, rounded zlib output chunks, any concatenated output and
// unfiltered rows, plus RGB-to-RGBA output. The first completed RGBA image
// remains live while the second is decoded.
const inputBytes = BigInt(before.bytes.byteLength + after.bytes.byteLength);
const estimate = (current: ParsedPng, retained: ParsedPng | undefined) => {
const compressedBytes = current.compressed.reduce(
(total, chunk) => total + BigInt(chunk.byteLength),
0n,
);
const rgbaExpansion = current.channels === 3 ? current.rgbaBytes : 0n;
return (
inputBytes +
(retained?.rgbaBytes ?? 0n) +
compressedBytes +
inflateBufferAllowance(current.expectedRawBytes) +
current.unfilteredBytes +
rgbaExpansion
);
};
const estimateBytes = [
estimate(before, undefined),
estimate(after, before),
].reduce((maximum, value) => (value > maximum ? value : maximum), 0n);
if (estimateBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
throw pngWorkingMemoryError(estimateBytes, before, after);
};
const inflateBufferAllowance = (expectedRawBytes: bigint): bigint => {
const chunkBytes = BigInt(PNG_INFLATE_CHUNK_BYTES);
const maximumOutputBytes = expectedRawBytes + 1n;
const roundUpToChunk = (bytes: bigint): bigint =>
((bytes + chunkBytes - 1n) / chunkBytes) * chunkBytes;
// Node checks maxOutputLength after writing a chunk, so a rejected stream can
// retain one chunk's worth of output past the limit before it throws.
const rejectedOutputChunks = roundUpToChunk(
maximumOutputBytes + chunkBytes - 1n,
);
// Sync zlib retains each output chunk while Buffer.concat creates the final
// result. A full final chunk also allocates the next scratch chunk first.
const returnedOutputChunks = roundUpToChunk(maximumOutputBytes);
const scratchChunk =
expectedRawBytes % chunkBytes === 0n ||
maximumOutputBytes % chunkBytes === 0n
? chunkBytes
: 0n;
const concatenatedOutput =
maximumOutputBytes > chunkBytes ? maximumOutputBytes : 0n;
const returnedOutput =
returnedOutputChunks + scratchChunk + concatenatedOutput;
return rejectedOutputChunks > returnedOutput
? rejectedOutputChunks
: returnedOutput;
};
const pngWorkingMemoryError = (
estimateBytes: bigint,
before: Pick<ParsedPng, "width" | "height"> | null,
after: Pick<ParsedPng, "width" | "height"> | null,
) =>
new AnalysisResourceConstraintError(
"compare_web_screenshots",
"memory",
"Screenshot comparison exceeds REA's 256 MiB estimated working-memory budget.",
{
boundary: "png-screenshot-comparison",
estimated_working_memory_bytes: estimateBytes.toString(),
maximum_working_memory_bytes: PNG_COMPARISON_WORKING_MEMORY_BYTES,
before_dimensions:
before === null ? null : { width: before.width, height: before.height },
after_dimensions:
after === null ? null : { width: after.width, height: after.height },
},
{
remediationAction:
"Compare smaller screenshots or reduce their viewport dimensions before capture.",
},
);
const inflatePng = (compressed: Buffer, maxOutputLength: number): Buffer => {
try {
return inflateSync(compressed, {
maxOutputLength,
chunkSize: PNG_INFLATE_CHUNK_BYTES,
});
} catch (cause: unknown) {
if (
cause instanceof Error &&
"code" in cause &&
(cause.code === "Z_DATA_ERROR" ||
cause.code === "Z_BUF_ERROR" ||
cause.code === "ERR_BUFFER_TOO_LARGE")
)
throw new TypeError("Invalid PNG image data", { cause });
throw cause;
}
};
const parseHeader = (data: Buffer) => { const parseHeader = (data: Buffer) => {
if (data.byteLength !== 13) throw new TypeError("Invalid PNG header"); if (data.byteLength !== 13) throw new TypeError("Invalid PNG header");
const width = data.readUInt32BE(0); const width = data.readUInt32BE(0);
const height = data.readUInt32BE(4); const height = data.readUInt32BE(4);
const bitDepth = data[8]; const bitDepth = data[8] ?? 0;
const colorType = data[9]; const colorType = data[9] ?? -1;
const validDepth =
(colorType === 0 && [1, 2, 4, 8, 16].includes(bitDepth)) ||
(colorType === 2 && [8, 16].includes(bitDepth)) ||
(colorType === 3 && [1, 2, 4, 8].includes(bitDepth)) ||
(colorType === 4 && [8, 16].includes(bitDepth)) ||
(colorType === 6 && [8, 16].includes(bitDepth));
if ( if (
width === 0 || width === 0 ||
height === 0 || height === 0 ||
!Number.isSafeInteger(width * height) || width > 0x7fffffff ||
bitDepth !== 8 || height > 0x7fffffff ||
(colorType !== 2 && colorType !== 6) || !validDepth ||
data[10] !== 0 || data[10] !== 0 ||
data[11] !== 0 || data[11] !== 0 ||
data[12] !== 0 (data[12] !== 0 && data[12] !== 1)
) )
throw new TypeError("Unsupported PNG"); throw new TypeError("Invalid PNG header");
return { width, height, channels: colorType === 6 ? 4 : 3 }; return {
width,
height,
colorType,
bitDepth,
interlace: data[12],
channels: colorType === 6 ? 4 : 3,
supported:
bitDepth === 8 && (colorType === 2 || colorType === 6) && data[12] === 0,
};
}; };
const unfilter = ( const unfilter = (
+53
View File
@@ -1,6 +1,12 @@
import { z } from "incur"; import { z } from "incur";
import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js"; import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js";
import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js";
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
import {
getEvidenceBundleInputSchema,
importEvidenceBundleInputSchema,
} from "../contracts/sessionToolSchemas.js";
import { CLI_COMMANDS } from "../cliCommandNames.js"; import { CLI_COMMANDS } from "../cliCommandNames.js";
import { parseCliJsonInput } from "../cliJsonInput.js"; import { parseCliJsonInput } from "../cliJsonInput.js";
import { logCliCommand } from "../cliLogging.js"; import { logCliCommand } from "../cliLogging.js";
@@ -16,6 +22,53 @@ export const registerAnalysisViewCommands = (
cli: CliInstance, cli: CliInstance,
logger: Logger, logger: Logger,
): void => { ): void => {
cli.command(CLI_COMMANDS.inspectEvidenceBundle, {
description:
"Inspect complete or selected discovery metadata from a local Evidence bundle",
args: z.object({
inputJson: z
.string()
.describe(
"JSON or JSON file with absolute path, detail and optional record filters",
),
}),
run: ({ args }) =>
logCliCommand(logger, CLI_COMMANDS.inspectEvidenceBundle, async () => {
const input = await parseCliJsonInput(
args.inputJson,
CLI_COMMANDS.inspectEvidenceBundle,
);
if (!input.ok) return input.error;
const schema = getEvidenceBundleInputSchema.safeExtend({
path: importEvidenceBundleInputSchema.shape.path,
});
const parsed = schema.safeParse(input.value);
if (!parsed.success)
return {
error: "Evidence bundle inspection failed",
...projectAnalysisError(
analysisInputErrorFromIssues(
CLI_COMMANDS.inspectEvidenceBundle,
parsed.error.issues,
input.value,
),
),
};
const loaded = await readEvidenceBundle(parsed.data.path);
if (!loaded.ok)
return {
error: "Evidence bundle inspection failed",
...projectAnalysisError(loaded.error),
};
const result = inspectEvidenceBundle(loaded.value, parsed.data);
return result.ok
? result.value
: {
error: "Evidence bundle inspection failed",
...projectAnalysisError(result.error),
};
}),
});
cli.command(CLI_COMMANDS.inspectAnalysisView, { cli.command(CLI_COMMANDS.inspectAnalysisView, {
description: description:
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON", "Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
+1
View File
@@ -43,6 +43,7 @@ export const CLI_COMMANDS = Object.freeze({
inspectBinaryLayout: "inspect-binary-layout", inspectBinaryLayout: "inspect-binary-layout",
inspectPeResources: "inspect-pe-resources", inspectPeResources: "inspect-pe-resources",
inspectAnalysisView: "inspect-analysis-view", inspectAnalysisView: "inspect-analysis-view",
inspectEvidenceBundle: "inspect-evidence-bundle",
inspectFirmwareRegions: "inspect-firmware-regions", inspectFirmwareRegions: "inspect-firmware-regions",
extractFirmware: "extract-firmware", extractFirmware: "extract-firmware",
inspectAndroidPackage: "inspect-android-package", inspectAndroidPackage: "inspect-android-package",
+12 -5
View File
@@ -7,7 +7,6 @@ import {
} from "../application/DirectAnalysis.js"; } from "../application/DirectAnalysis.js";
import { runSessionStatus as executeSessionStatus } from "../application/DirectAnalysisStatus.js"; import { runSessionStatus as executeSessionStatus } from "../application/DirectAnalysisStatus.js";
import type { DirectAnalysisDependencies } from "../application/DirectAnalysisDependencies.js"; import type { DirectAnalysisDependencies } from "../application/DirectAnalysisDependencies.js";
import { createBinarySession, createManagedBinarySession } from "./binary.js";
/** Bind one-shot analysis and status commands to one caller-selected environment. */ /** Bind one-shot analysis and status commands to one caller-selected environment. */
export const createDirectAnalysis = ( export const createDirectAnalysis = (
@@ -16,9 +15,14 @@ export const createDirectAnalysis = (
const environment = snapshotEnvironment(selectedEnvironment); const environment = snapshotEnvironment(selectedEnvironment);
const dependencies: DirectAnalysisDependencies = { const dependencies: DirectAnalysisDependencies = {
readConfiguration: () => parseConfig(environment), readConfiguration: () => parseConfig(environment),
createBinarySession: (config, logger) => createBinarySession: async (config, logger) => {
createBinarySession(config, logger, environment), const { createBinarySession } = await import("./binary.js");
createManagedBinarySession, return createBinarySession(config, logger, environment);
},
createManagedBinarySession: async () => {
const { createManagedBinarySession } = await import("./binary.js");
return createManagedBinarySession();
},
}; };
return { return {
runDirectAnalysis: executeDirectAnalysis.bind(undefined, dependencies), runDirectAnalysis: executeDirectAnalysis.bind(undefined, dependencies),
@@ -34,6 +38,9 @@ export type DirectAnalysis = ReturnType<typeof createDirectAnalysis>;
export const runManagedProviderExecution = executeManagedProvider.bind( export const runManagedProviderExecution = executeManagedProvider.bind(
undefined, undefined,
{ {
createManagedBinarySession, createManagedBinarySession: async () => {
const { createManagedBinarySession } = await import("./binary.js");
return createManagedBinarySession();
},
}, },
); );
+1 -1
View File
@@ -159,7 +159,7 @@ export const SESSION_TOOL_CONTRACTS = [
), ),
session( session(
"get_evidence_bundle", "get_evidence_bundle",
"Return every Evidence record and residual unknown currently retained by this session as one inline bundle for direct inspection or follow-up workflows.", "Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply only to summary delivery.",
getEvidenceBundleInputSchema, getEvidenceBundleInputSchema,
), ),
session( session(
+6 -2
View File
@@ -6,9 +6,13 @@ import { localPathStringSchema } from "../domain/localPath.js";
import { processTraceSpecificationSchema } from "../domain/process/processTraceSpecification.js"; import { processTraceSpecificationSchema } from "../domain/process/processTraceSpecification.js";
import { evidenceSchema } from "../domain/evidence.js"; import { evidenceSchema } from "../domain/evidence.js";
import { prefixedDigestSchema } from "./../domain/digests.js"; import { prefixedDigestSchema } from "./../domain/digests.js";
import { evidenceRecordFiltersSchema } from "../domain/evidenceRecordSummary.js";
/** Return the current canonical Evidence bundle inline. */ /** Complete retained Evidence, or selected discovery metadata without payloads. */
export const getEvidenceBundleInputSchema = z.strictObject({}); export const getEvidenceBundleInputSchema = z.strictObject({
detail: z.enum(["complete", "summary"]).default("complete"),
filters: evidenceRecordFiltersSchema.optional(),
});
/** Optional document selection for volatile navigation context. */ /** Optional document selection for volatile navigation context. */
export const navigationContextInputSchema = z.strictObject({ export const navigationContextInputSchema = z.strictObject({
+4
View File
@@ -53,6 +53,10 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
trace_native_ui_action: { action: "buildTapped:" }, trace_native_ui_action: { action: "buildTapped:" },
open_binary: { path: "/tmp/fixture" }, open_binary: { path: "/tmp/fixture" },
export_evidence_bundle: { path: "/tmp/evidence.json" }, export_evidence_bundle: { path: "/tmp/evidence.json" },
get_evidence_bundle: {
detail: "summary",
filters: { operation: "analyze_function", procedure_address: "0x1000" },
},
inspect_address_context: { address: "0x1000" }, inspect_address_context: { address: "0x1000" },
import_evidence_bundle: { path: "/tmp/evidence.json" }, import_evidence_bundle: { path: "/tmp/evidence.json" },
capture_process_scenario: { capture_process_scenario: {
+4 -1
View File
@@ -1,4 +1,5 @@
import { z } from "zod"; import { z } from "zod";
import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js";
import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js"; import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js";
import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js"; import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js";
import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js"; import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js";
@@ -470,7 +471,9 @@ export const sessionOutputSchemas = {
unknowns: z.number().int().min(0), unknowns: z.number().int().min(0),
}), }),
), ),
get_evidence_bundle: lifecycleResultOf(evidenceBundleSchema), get_evidence_bundle: lifecycleResultOf(
z.union([evidenceBundleSchema, evidenceBundleSummarySchema]),
),
get_navigation_context: lifecycleResultOf( get_navigation_context: lifecycleResultOf(
z.object({ z.object({
document: z.string(), document: z.string(),
+1 -1
View File
@@ -47,7 +47,7 @@ export const analysisErrorRemediationAction = (
return ( return (
error.remediationAction ?? error.remediationAction ??
(error.resource === "transport" (error.resource === "transport"
? "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable." ? "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."
: error.resource === "cpu" : error.resource === "cpu"
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits." ? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
: error.resource === "file-size" : error.resource === "file-size"
@@ -21,6 +21,12 @@ describe("cooperative canonical JSON digest", () => {
), ),
{ payload: `${"x".repeat(8191)}𐀀${"\n\ud800".repeat(20_000)}` }, { payload: `${"x".repeat(8191)}𐀀${"\n\ud800".repeat(20_000)}` },
{ [`${"x".repeat(8191)}𐀀${"\n".repeat(20_000)}`]: "value" }, { [`${"x".repeat(8191)}𐀀${"\n".repeat(20_000)}`]: "value" },
Object.fromEntries(
Array.from({ length: 512 }, (_, index) => [
`escaped\n\"\\𐀀${"x".repeat(32)}${index}`,
{ repeated: index },
]),
),
]; ];
it.each(values.map((value) => ({ value })))( it.each(values.map((value) => ({ value })))(
"preserves complete canonical meaning (%#)", "preserves complete canonical meaning (%#)",
@@ -55,6 +61,18 @@ describe("cooperative canonical JSON digest", () => {
expect(steps.next().done).toBe(true); expect(steps.next().done).toBe(true);
}); });
it("continues object hashing when a pending property is omitted", () => {
const value: { [key: string]: JsonValue } = {
a: "x".repeat(32_768),
b: 1,
c: { retained: true },
};
const steps = canonicalJsonDigestSteps(value);
expect(steps.next().done).toBe(false);
delete value.b;
expect(complete(steps)).toBe(digestCanonicalValue(value));
});
it("rejects sparse arrays, nonfinite numbers and cycles", () => { it("rejects sparse arrays, nonfinite numbers and cycles", () => {
const sparse: JsonValue[] = []; const sparse: JsonValue[] = [];
sparse.length = 2; sparse.length = 2;
+34 -13
View File
@@ -26,6 +26,9 @@ export function* canonicalJsonDigestSteps(
const hash = createHash("sha256"); const hash = createHash("sha256");
const ancestors = new Set<object>(); const ancestors = new Set<object>();
const stack: DigestFrame[] = [{ kind: "value", value }]; const stack: DigestFrame[] = [{ kind: "value", value }];
const encodedKeys = new Map<string, string>();
// Keep reusable key encodings within one existing digest-buffer quantum.
let remainingKeyCodeUnits = 8192;
let buffered = ""; let buffered = "";
let flushed = false; let flushed = false;
const emit = (part: string): void => { const emit = (part: string): void => {
@@ -69,22 +72,22 @@ export function* canonicalJsonDigestSteps(
stack.push(frame); stack.push(frame);
continue; continue;
} }
let item: JsonValue;
if (frame.kind === "array") { if (frame.kind === "array") {
if (frame.index === frame.length) { if (frame.index === frame.length) {
emit("]"); emit("]");
ancestors.delete(frame.value); ancestors.delete(frame.value);
continue; continue;
} }
const item = frame.value[frame.index]; const element = frame.value[frame.index];
if (item === undefined) if (element === undefined)
throw new TypeError( throw new TypeError(
"Canonical JSON digest requires dense JSON arrays", "Canonical JSON digest requires dense JSON arrays",
); );
if (frame.index++ > 0) emit(","); if (frame.index++ > 0) emit(",");
stack.push(frame, { kind: "value", value: item }); stack.push(frame);
continue; item = element;
} } else if (frame.kind === "object") {
if (frame.kind === "object") {
if (frame.index === frame.keys.length) { if (frame.index === frame.keys.length) {
emit("}"); emit("}");
ancestors.delete(frame.value); ancestors.delete(frame.value);
@@ -93,25 +96,42 @@ export function* canonicalJsonDigestSteps(
const key = frame.keys[frame.index++]; const key = frame.keys[frame.index++];
if (key === undefined) if (key === undefined)
throw new TypeError("Canonical JSON object changed during hashing"); throw new TypeError("Canonical JSON object changed during hashing");
const item = frame.value[key]; const property = frame.value[key];
if (item === undefined) continue; if (property === undefined) {
stack.push(frame);
continue;
}
if (frame.written++ > 0) emit(","); if (frame.written++ > 0) emit(",");
if (key.length > 8192) { if (key.length > 8192) {
emit('"'); emit('"');
stack.push( stack.push(
frame, frame,
{ kind: "value", value: item }, { kind: "value", value: property },
{ kind: "separator" }, { kind: "separator" },
{ kind: "string", value: key, index: 0 }, { kind: "string", value: key, index: 0 },
); );
continue; continue;
} }
emit(JSON.stringify(key)); let encodedKey = encodedKeys.get(key);
if (encodedKey === undefined) {
encodedKey = JSON.stringify(key);
if (encodedKey.length <= remainingKeyCodeUnits) {
encodedKeys.set(key, encodedKey);
remainingKeyCodeUnits -= encodedKey.length;
}
}
emit(encodedKey);
emit(":"); emit(":");
stack.push(frame, { kind: "value", value: item }); stack.push(frame);
continue; item = property;
} else {
item = frame.value;
}
// Resume containers directly; scalar members need no temporary value frame.
if (flushed) {
flushed = false;
yield;
} }
const item = frame.value;
if (typeof item === "string" && item.length > 8192) { if (typeof item === "string" && item.length > 8192) {
emit('"'); emit('"');
stack.push({ kind: "string", value: item, index: 0 }); stack.push({ kind: "string", value: item, index: 0 });
@@ -151,6 +171,7 @@ export function* canonicalJsonDigestSteps(
} finally { } finally {
stack.length = 0; stack.length = 0;
ancestors.clear(); ancestors.clear();
encodedKeys.clear();
buffered = ""; buffered = "";
} }
} }
+55
View File
@@ -0,0 +1,55 @@
import { expect, it } from "vitest";
import { createEvidence } from "./evidence.js";
import { createEvidenceBundle } from "./evidenceBundle.js";
import { summarizeEvidenceBundle } from "./evidenceRecordSummary.js";
it("preserves native identity independently of missing or incompatible value-flow metadata", () => {
const records = [
undefined,
{ available: true, truncated: true },
{ unexpected: "legacy" },
{ available: false, reason: "Decompiler unavailable" },
].map((flow) =>
createEvidence(
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
{ id: "provider", name: "Provider", version: "1" },
{
operation: "analyze_function",
parameters: {},
result: {
procedure: { address: "space:0x10", name: "entry" },
...(flow === undefined ? {} : { native_value_flow: flow }),
},
},
),
);
const summary = summarizeEvidenceBundle(createEvidenceBundle(records), {
procedure_address: "space:0x10",
});
expect(summary.matching_records).toBe(4);
const native = summary.records.map((r) => r.native_dossier);
expect(
native.every((r) => r.available && r.procedure_address === "space:0x10"),
).toBe(true);
expect(native).toEqual(
expect.arrayContaining([
expect.objectContaining({ value_flow: { status: "not-recorded" } }),
expect.objectContaining({
value_flow: {
status: "observed",
truncated: true,
omitted_operations_lower_bound: null,
known_omitted_inputs: null,
known_omitted_edges: null,
},
}),
expect.objectContaining({
value_flow: { status: "unknown", reason: expect.any(String) },
}),
expect.objectContaining({
value_flow: { status: "unavailable", reason: "Decompiler unavailable" },
}),
]),
);
});
+166
View File
@@ -0,0 +1,166 @@
import { z } from "zod";
import { digestSchema, prefixedDigestSchema } from "./digests.js";
import { evidenceSchema, type Evidence } from "./evidence.js";
import type { EvidenceBundle } from "./evidenceBundle.js";
/** Exact retained-record selectors; selection never resolves a provider address. */
export const evidenceRecordFiltersSchema = z.strictObject({
evidence_id: prefixedDigestSchema("ev").optional(),
operation: z.string().min(1).optional(),
target_sha256: digestSchema.optional(),
analysis_profile_digest: digestSchema.optional(),
procedure_address: z.string().min(1).optional(),
});
export type EvidenceRecordFilters = z.infer<typeof evidenceRecordFiltersSchema>;
const countSchema = z.number().int().nonnegative();
const nativeMetadataSchema = z.object({
procedure: z.object({ address: z.string(), name: z.string().nullish() }),
native_value_flow: z.unknown().optional(),
});
const valueFlowMetadataSchema = z.union([
z.object({
available: z.literal(true),
truncated: z.boolean(),
omitted_operations_lower_bound: countSchema.optional(),
known_omitted_inputs: countSchema.optional(),
known_omitted_edges: countSchema.optional(),
}),
z.object({ available: z.literal(false), reason: z.string() }),
]);
const nativeSummarySchema = z.discriminatedUnion("available", [
z.strictObject({
available: z.literal(true),
procedure_address: z.string(),
procedure_name: z.string().nullable(),
value_flow: z.discriminatedUnion("status", [
z.strictObject({
status: z.literal("observed"),
truncated: z.boolean(),
omitted_operations_lower_bound: countSchema.nullable(),
known_omitted_inputs: countSchema.nullable(),
known_omitted_edges: countSchema.nullable(),
}),
z.strictObject({ status: z.literal("unavailable"), reason: z.string() }),
z.strictObject({ status: z.literal("not-recorded") }),
z.strictObject({ status: z.literal("unknown"), reason: z.string() }),
]),
}),
z.strictObject({
available: z.literal(false),
reason: z.enum(["different-operation", "not-recorded-or-incompatible"]),
}),
]);
/** Metadata for complete retained records, independent of analysis completeness. */
export const evidenceBundleSummarySchema = z.strictObject({
kind: z.literal("evidence-bundle-summary"),
filters: z.record(z.string(), z.string()),
total_retained_records: countSchema,
matching_records: countSchema,
unknown_revisions: countSchema,
records: z.array(
z.strictObject({
evidence_id: evidenceSchema.shape.evidence_id,
subject: evidenceSchema.shape.subject,
provider: evidenceSchema.shape.provider,
operation: evidenceSchema.shape.operation,
confidence: evidenceSchema.shape.confidence,
authority: evidenceSchema.shape.authority,
analysis_profile_digest: digestSchema.nullable(),
retention: z.literal("complete-record"),
limitation_count: countSchema,
native_dossier: nativeSummarySchema,
}),
),
});
export type EvidenceBundleSummary = z.infer<typeof evidenceBundleSummarySchema>;
const nativeSummary = (
evidence: Evidence,
): z.infer<typeof nativeSummarySchema> => {
if (evidence.operation !== "analyze_function")
return { available: false, reason: "different-operation" };
// Read only producer metadata. Parsing the full dossier would allocate every
// large collection merely to discover its already authenticated parent ID.
const parsed = nativeMetadataSchema.safeParse(evidence.normalized_result);
if (!parsed.success)
return { available: false, reason: "not-recorded-or-incompatible" };
const flow = valueFlowMetadataSchema.safeParse(parsed.data.native_value_flow);
return {
available: true,
procedure_address: parsed.data.procedure.address,
procedure_name: parsed.data.procedure.name ?? null,
value_flow:
parsed.data.native_value_flow == null
? { status: "not-recorded" }
: !flow.success
? {
status: "unknown",
reason: "Value-flow metadata has an incompatible representation.",
}
: flow.data.available
? {
status: "observed",
truncated: flow.data.truncated,
omitted_operations_lower_bound:
flow.data.omitted_operations_lower_bound ?? null,
known_omitted_inputs: flow.data.known_omitted_inputs ?? null,
known_omitted_edges: flow.data.known_omitted_edges ?? null,
}
: { status: "unavailable", reason: flow.data.reason },
};
};
/** Discover retained Evidence without reading or copying its complete payloads. */
export const summarizeEvidenceBundle = (
bundle: EvidenceBundle,
filters: EvidenceRecordFilters = {},
): EvidenceBundleSummary => {
const selectedFilters: Record<string, string> = {};
for (const [field, value] of Object.entries(filters))
if (value !== undefined) selectedFilters[field] = value;
const records: EvidenceBundleSummary["records"] = [];
for (const evidence of bundle.records) {
if (
(filters.evidence_id !== undefined &&
filters.evidence_id !== evidence.evidence_id) ||
(filters.operation !== undefined &&
filters.operation !== evidence.operation) ||
(filters.target_sha256 !== undefined &&
filters.target_sha256 !== evidence.subject?.digest.sha256) ||
(filters.analysis_profile_digest !== undefined &&
filters.analysis_profile_digest !== evidence.analysis_profile?.digest)
)
continue;
const native = nativeSummary(evidence);
if (
filters.procedure_address !== undefined &&
(!native.available ||
filters.procedure_address !== native.procedure_address)
)
continue;
records.push({
evidence_id: evidence.evidence_id,
subject: evidence.subject,
provider: evidence.provider,
operation: evidence.operation,
confidence: evidence.confidence,
authority: evidence.authority,
analysis_profile_digest: evidence.analysis_profile?.digest ?? null,
retention: "complete-record",
limitation_count: evidence.limitations.length,
native_dossier: native,
});
}
return {
kind: "evidence-bundle-summary",
filters: selectedFilters,
total_retained_records: bundle.records.length,
matching_records: records.length,
unknown_revisions: bundle.unknowns.length,
records,
};
};
+56
View File
@@ -72,6 +72,62 @@ describe("read-only NativeAOT PE parser", () => {
}); });
}); });
it("does not discover an RTR directory in raw padding beyond VirtualSize", () => {
const bytes = nativeAotPeFixture();
const sectionTable = 0x98 + 0xf0;
const dataSection = sectionTable + 40;
bytes.writeUInt32LE(0x20, dataSection + 8);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
);
expect(parsed.summary).toMatchObject({
status: "not_applicable",
reason:
"No supported ReadyToRun directory signature was found in initialized non-executable PE sections.",
method_tables: 0,
});
});
it("uses raw size as the virtual extent when VirtualSize is zero", () => {
const bytes = nativeAotPeFixture();
const sectionTable = 0x98 + 0xf0;
bytes.writeUInt32LE(0, sectionTable + 40 + 8);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
);
expect(parsed.summary).toMatchObject({
status: "complete",
method_tables: 3,
derived_overlay: { address: "0x140002380", size_bytes: 1 },
});
});
it("accepts executable-section targets in the zero-filled virtual tail", () => {
const bytes = nativeAotPeFixture();
const sectionTable = 0x98 + 0xf0;
bytes.writeUInt32LE(0x300, sectionTable + 8);
const slotOffset = 0x600 + (0x2280 - 0x2000) + 24;
// RVA 0x1200 is executable-section zero fill: virtual size 0x300, raw size 0x200.
bytes.writeBigUInt64LE(0x140001200n, slotOffset);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
"0x140000000",
);
expect(parsed.summary).toMatchObject({
status: "complete",
method_tables: 3,
});
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
virtual_slots: [{ slot: 0, target_address: "0x140001200" }],
});
});
it("does not claim an image mapping when Ghidra rebased the image", () => { it("does not claim an image mapping when Ghidra rebased the image", () => {
const bytes = nativeAotPeFixture(); const bytes = nativeAotPeFixture();
expect( expect(
+21 -24
View File
@@ -13,7 +13,7 @@ export const NATIVE_AOT_MAX_WORK_UNITS = 64 * 1024 * 1024;
export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024; export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024;
/** Version of the built-in read-only PE metadata interpretation in profiles. */ /** Version of the built-in read-only PE metadata interpretation in profiles. */
export const NATIVE_AOT_PROFILE_CONTRACT_REVISION = export const NATIVE_AOT_PROFILE_CONTRACT_REVISION =
"rtr-9.1-x64-pe-read-only-v1"; "rtr-9.1-x64-pe-read-only-v3";
export type NativeAotParserLimits = { export type NativeAotParserLimits = {
readonly sourceBytes: number; readonly sourceBytes: number;
@@ -81,6 +81,12 @@ type Section = {
readonly rawSize: number; readonly rawSize: number;
readonly characteristics: number; readonly characteristics: number;
}; };
// Compatible PE loaders use raw size when VirtualSize is zero. Otherwise,
// RVA containment follows the declared VirtualSize, independent of file size.
const virtualExtent = (section: Section): number =>
section.virtualSize === 0 ? section.rawSize : section.virtualSize;
const fileBackedVirtualExtent = (section: Section): number =>
Math.min(virtualExtent(section), section.rawSize);
type RtrSection = { type RtrSection = {
readonly type: number; readonly type: number;
readonly flags: number; readonly flags: number;
@@ -417,10 +423,7 @@ class PeImage {
}); });
for (const section of this.sections) { for (const section of this.sections) {
this.#range(section.rawOffset, section.rawSize); this.#range(section.rawOffset, section.rawSize);
if ( if (section.rva + virtualExtent(section) > this.sizeOfImage)
section.rva + Math.max(section.virtualSize, section.rawSize) >
this.sizeOfImage
)
throw new Error("PE section exceeds the declared image extent"); throw new Error("PE section exceeds the declared image extent");
} }
} }
@@ -456,7 +459,7 @@ class PeImage {
for (const section of this.sections) { for (const section of this.sections) {
if (this.#budget !== undefined && !this.#budget.consumeWork(1)) if (this.#budget !== undefined && !this.#budget.consumeWork(1))
return null; return null;
const extent = Math.max(section.virtualSize, section.rawSize); const extent = virtualExtent(section);
if (relative < section.rva || relative - section.rva > extent - size) if (relative < section.rva || relative - section.rva > extent - size)
continue; continue;
const delta = relative - section.rva; const delta = relative - section.rva;
@@ -466,6 +469,7 @@ class PeImage {
} }
return null; return null;
} }
// This checks executable section membership, not file-backed instruction bytes.
isExecutable(address: bigint): boolean { isExecutable(address: bigint): boolean {
if (address < this.imageBase) return false; if (address < this.imageBase) return false;
const rva = address - this.imageBase; const rva = address - this.imageBase;
@@ -475,7 +479,7 @@ class PeImage {
if ( if (
(section.characteristics & 0x20000000) !== 0 && (section.characteristics & 0x20000000) !== 0 &&
rva >= BigInt(section.rva) && rva >= BigInt(section.rva) &&
rva < BigInt(section.rva + section.rawSize) rva < BigInt(section.rva + virtualExtent(section))
) )
return true; return true;
} }
@@ -492,10 +496,7 @@ class PeImage {
if (this.#budget !== undefined && !this.#budget.consumeWork(1)) return 0; if (this.#budget !== undefined && !this.#budget.consumeWork(1)) return 0;
if ( if (
rva >= BigInt(candidate.rva) && rva >= BigInt(candidate.rva) &&
rva < rva < BigInt(candidate.rva + virtualExtent(candidate))
BigInt(
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
)
) { ) {
section = candidate; section = candidate;
break; break;
@@ -503,9 +504,7 @@ class PeImage {
} }
return section === undefined return section === undefined
? 0 ? 0
: section.rva + : section.rva + virtualExtent(section) - Number(rva);
Math.max(section.virtualSize, section.rawSize) -
Number(rva);
} }
#range(offset: number, size: number): void { #range(offset: number, size: number): void {
if ( if (
@@ -602,7 +601,7 @@ const findDirectory = (
) )
continue; continue;
const begin = section.rawOffset; const begin = section.rawOffset;
const end = begin + section.rawSize; const end = begin + fileBackedVirtualExtent(section);
const first = (begin + 7) & ~7; const first = (begin + 7) & ~7;
const candidateBytes = const candidateBytes =
first <= end - 16 ? (Math.floor((end - 16 - first) / 8) + 1) * 8 : 0; first <= end - 16 ? (Math.floor((end - 16 - first) / 8) + 1) * 8 : 0;
@@ -970,10 +969,11 @@ const discoverTables = (
(section.characteristics & 0x20000000) !== 0 (section.characteristics & 0x20000000) !== 0
) )
continue; continue;
if (section.rawSize > 0) { const extent = fileBackedVirtualExtent(section);
if (extent > 0) {
ranges.push({ ranges.push({
start: image.addressForRva(section.rva), start: image.addressForRva(section.rva),
end: image.addressForRva(section.rva + section.rawSize), end: image.addressForRva(section.rva + extent),
}); });
} }
} }
@@ -984,9 +984,7 @@ const discoverTables = (
(candidate.characteristics & 0x20000000) === 0 && (candidate.characteristics & 0x20000000) === 0 &&
hydrated.base >= image.addressForRva(candidate.rva) && hydrated.base >= image.addressForRva(candidate.rva) &&
hydrated.base + BigInt(hydrated.sizeBytes) <= hydrated.base + BigInt(hydrated.sizeBytes) <=
image.addressForRva( image.addressForRva(candidate.rva + virtualExtent(candidate)),
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
),
); );
if (section !== undefined) if (section !== undefined)
ranges.push({ ranges.push({
@@ -1592,11 +1590,10 @@ export const parseNativeAotPe = (
? "raw-image-pointer-scan" ? "raw-image-pointer-scan"
: "raw-image-and-rehydrated-pointer-scan", : "raw-image-and-rehydrated-pointer-scan",
}, },
diagnostics: [ diagnostics:
...(hydrationError === null hydrationError === null
? [] ? []
: [`Read-only hydration stopped: ${hydrationError}`]), : [`Read-only hydration stopped: ${hydrationError}`],
],
limitations: reportLimitations, limitations: reportLimitations,
}; };
if (Buffer.byteLength(JSON.stringify(summary)) > maximumSummaryBytes) if (Buffer.byteLength(JSON.stringify(summary)) > maximumSummaryBytes)
+1 -1
View File
@@ -1,7 +1,7 @@
/** Generated from package.json and package-lock.json; do not edit. */ /** Generated from package.json and package-lock.json; do not edit. */
export const PACKAGE_METADATA = { export const PACKAGE_METADATA = {
name: "rea-agents", name: "rea-agents",
version: "6.3.0", // x-release-please-version version: "6.4.0", // x-release-please-version
serverSdkVersion: "2.3.1", serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1", clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1", coreSdkVersion: "2.3.1",
+1 -1
View File
@@ -226,7 +226,7 @@ describe("Ghidra platform support", () => {
executable_role: target.executableRole, executable_role: target.executableRole,
managed: target.managed, managed: target.managed,
native_aot_metadata: { native_aot_metadata: {
contract_revision: "rtr-9.1-x64-pe-read-only-v1", contract_revision: "rtr-9.1-x64-pe-read-only-v3",
source_bytes: 128 * 1024 * 1024, source_bytes: 128 * 1024 * 1024,
work_units: 64 * 1024 * 1024, work_units: 64 * 1024 * 1024,
working_memory_bytes: 64 * 1024 * 1024, working_memory_bytes: 64 * 1024 * 1024,
+9
View File
@@ -44,6 +44,15 @@ describe("ghidraSessionRoot", () => {
).toBe(posixBase("Users", "operator")); ).toBe(posixBase("Users", "operator"));
}); });
it("can select a filesystem root when the inherited temp starts with a dot-prefixed element", () => {
expect(
ghidraSessionRoot({
base: posixBase(".Tmp", "scratch"),
platform: "win32",
}),
).toBe(resolve(sep));
});
it.skipIf(!POSIX)("prefers the platform temp directory by default", () => { it.skipIf(!POSIX)("prefers the platform temp directory by default", () => {
expect( expect(
ghidraSessionRoot({ base: posixBase("home", "operator", ".cache") }), ghidraSessionRoot({ base: posixBase("home", "operator", ".cache") }),
+3 -2
View File
@@ -28,8 +28,9 @@ interface Section {
readonly size: number; readonly size: number;
} }
// The image loader maps VirtualSize bytes, using SizeOfRawData only when // RVA ownership follows the declared virtual extent. Raw size describes
// VirtualSize is zero. Raw bytes past that extent are file-alignment padding. // initialized file bytes and can include file-alignment padding; loaders may
// map page-rounded tails, but those bytes are outside the declared section.
const virtualExtent = (section: Section): number => const virtualExtent = (section: Section): number =>
section.virtualSize === 0 ? section.size : section.virtualSize; section.virtualSize === 0 ? section.size : section.virtualSize;
+60 -15
View File
@@ -339,11 +339,61 @@ interface OwnedCleanupPlan {
readonly unverified: readonly OwnershipSweepUnverifiedProcess[]; readonly unverified: readonly OwnershipSweepUnverifiedProcess[];
} }
const createOwnedCleanupPlan = async ( // A process-table read can race exit and retain the old state with an altered
// command. Only a fresh table proving that PID is no longer live can settle
// that mismatch; surviving group members still require normal token checks.
const settleLauncherExit = async (
ownership: OwnedProcessGroup, ownership: OwnedProcessGroup,
processes: readonly ProcessTableEntry[], processes: readonly ProcessTableEntry[],
host: ProcessOwnershipHost, host: ProcessOwnershipHost,
): Promise<
| readonly ProcessTableEntry[]
| Extract<ProcessCleanupResult, { readonly cleaned: false }>
> => {
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
if (launcher === undefined) return processes;
const reason = launcherIdentityFailure(launcher, ownership);
if (reason === null) return processes;
const refreshed = await readLiveProcessTable(host);
if (
refreshed.available &&
!refreshed.processes.some(({ pid }) => pid === launcher.pid)
) {
// Refreshing must not discard a surviving detached descendant's lineage,
// including a child that changed groups while the launcher exited.
const descendants = descendantsOf([launcher.pid], processes);
const descendantPids = new Set(descendants.map(({ pid }) => pid));
const descendantGroupIds = new Set(
descendants.map(({ processGroupId }) => processGroupId),
);
const refreshedDescendantPids = new Set(
descendantsOf(
descendants.map(({ pid }) => pid),
refreshed.processes,
).map(({ pid }) => pid),
);
if (
refreshed.processes.some(
(process) =>
process.processGroupId !== ownership.processGroupId &&
(refreshedDescendantPids.has(process.pid) ||
descendantPids.has(process.pid) ||
descendantGroupIds.has(process.processGroupId)),
)
)
return { cleaned: false, reason };
return refreshed.processes;
}
return { cleaned: false, reason };
};
const createOwnedCleanupPlan = async (
ownership: OwnedProcessGroup,
initialProcesses: readonly ProcessTableEntry[],
host: ProcessOwnershipHost,
): Promise<OwnedCleanupPlan | ProcessCleanupResult> => { ): Promise<OwnedCleanupPlan | ProcessCleanupResult> => {
const processes = await settleLauncherExit(ownership, initialProcesses, host);
if ("cleaned" in processes) return processes;
const sampledProcessGroupIds = new Set( const sampledProcessGroupIds = new Set(
(ownership.sampledProcessGroupIds ?? []).filter( (ownership.sampledProcessGroupIds ?? []).filter(
(processGroupId) => (processGroupId) =>
@@ -397,10 +447,7 @@ const createOwnedCleanupPlan = async (
}; };
let descendants: readonly ProcessTableEntry[] = []; let descendants: readonly ProcessTableEntry[] = [];
if (launcher !== undefined) { if (launcher !== undefined) {
const identityFailure = launcherIdentityFailure(launcher, ownership); descendants = descendantsOf([launcher.pid], processes);
if (identityFailure !== null)
return { cleaned: false, reason: identityFailure };
descendants = descendantsOf(launcher.pid, processes);
} }
const descendantPids = new Set(descendants.map(({ pid }) => pid)); const descendantPids = new Set(descendants.map(({ pid }) => pid));
const processGroupIds = new Set<number>([ownership.processGroupId]); const processGroupIds = new Set<number>([ownership.processGroupId]);
@@ -496,7 +543,13 @@ const revalidateOwnedProcessGroup = async (
): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => { ): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => {
let members: readonly ProcessTableEntry[]; let members: readonly ProcessTableEntry[];
try { try {
members = liveProcesses(await host.listProcesses()).filter( const processes = await settleLauncherExit(
ownership,
liveProcesses(await host.listProcesses()),
host,
);
if ("cleaned" in processes) return processes;
members = processes.filter(
({ processGroupId: observedGroupId }) => ({ processGroupId: observedGroupId }) =>
observedGroupId === processGroupId, observedGroupId === processGroupId,
); );
@@ -513,14 +566,6 @@ const revalidateOwnedProcessGroup = async (
expectedIdentities, expectedIdentities,
); );
if (failures.length > 0) return cleanupValidationFailure(failures); if (failures.length > 0) return cleanupValidationFailure(failures);
if (processGroupId === ownership.processGroupId) {
const launcher = members.find(({ pid }) => pid === ownership.leaderPid);
if (launcher !== undefined) {
const identityFailure = launcherIdentityFailure(launcher, ownership);
if (identityFailure !== null)
return { cleaned: false, reason: identityFailure };
}
}
if ( if (
processGroupId !== ownership.processGroupId && processGroupId !== ownership.processGroupId &&
members.length > 0 && members.length > 0 &&
@@ -850,7 +895,7 @@ const scanTokenOwnedProcesses = async (
const relatedGroupIds = new Set<number>(); const relatedGroupIds = new Set<number>();
if (relation !== undefined) { if (relation !== undefined) {
relatedPids.add(relation.leaderPid); relatedPids.add(relation.leaderPid);
for (const descendant of descendantsOf(relation.leaderPid, processes)) for (const descendant of descendantsOf([relation.leaderPid], processes))
relatedPids.add(descendant.pid); relatedPids.add(descendant.pid);
relatedGroupIds.add(relation.processGroupId); relatedGroupIds.add(relation.processGroupId);
for (const processGroupId of relation.sampledProcessGroupIds ?? []) for (const processGroupId of relation.sampledProcessGroupIds ?? [])
+187 -22
View File
@@ -3,6 +3,7 @@ import {
cleanupOwnedProcessGroup, cleanupOwnedProcessGroup,
cleanupWindowsProcessTree, cleanupWindowsProcessTree,
type ProcessOwnershipHost, type ProcessOwnershipHost,
type ProcessTableEntry,
type WindowsProcessTreeHost, type WindowsProcessTreeHost,
} from "./ProcessOwnership.js"; } from "./ProcessOwnership.js";
import { import {
@@ -11,18 +12,20 @@ import {
} from "./ProcessOwnershipObservation.js"; } from "./ProcessOwnershipObservation.js";
import { host, ownership } from "./ProcessOwnership.fixture.js"; import { host, ownership } from "./ProcessOwnership.fixture.js";
const launcher: ProcessTableEntry = {
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
};
describe("owned process-group cleanup validation: ownership and lineage", () => { describe("owned process-group cleanup validation: ownership and lineage", () => {
it("fails closed when a descendant in another process group lacks the token", async () => { it("fails closed when a descendant in another process group lacks the token", async () => {
const adapter: ProcessOwnershipHost = { const adapter: ProcessOwnershipHost = {
listProcesses: () => listProcesses: () =>
Promise.resolve([ Promise.resolve([
{ launcher,
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{ {
pid: 101, pid: 101,
parentPid: 100, parentPid: 100,
@@ -211,16 +214,9 @@ describe("owned process-group cleanup liveness rechecks", () => {
); );
it("accepts a member that exits during ownership revalidation", async () => { it("accepts a member that exits during ownership revalidation", async () => {
const liveLauncher = {
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
};
const listProcesses = vi const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>() .fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce([liveLauncher]) .mockResolvedValueOnce([launcher])
.mockResolvedValue([]); .mockResolvedValue([]);
const signalGroup = vi.fn(); const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = { const adapter: ProcessOwnershipHost = {
@@ -239,6 +235,181 @@ describe("owned process-group cleanup liveness rechecks", () => {
}); });
}); });
describe("owned process-group cleanup validation: refreshed descendant lineage", () => {
it.each(["planning", "signaling"])(
"retains cleanup uncertainty for newly detached descendants during %s",
async (phase) => {
const child = { ...launcher, pid: 101, parentPid: 100 };
const stale = { ...launcher, command: "[MainThread]" };
const signalGroup = vi.fn();
let reads = 0;
const adapter: ProcessOwnershipHost = {
listProcesses: () => {
reads += 1;
if (phase === "signaling" && reads === 1)
return Promise.resolve([launcher, child]);
if (reads === (phase === "planning" ? 1 : 2))
return Promise.resolve([stale, child]);
return Promise.resolve([
{ ...child, parentPid: 1 },
{ ...child, pid: 102, parentPid: 101 },
{ ...child, pid: 103, parentPid: 102, processGroupId: 103 },
]);
},
environment: () =>
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
signalGroup,
};
await expect(
cleanupOwnedProcessGroup(
{ ...ownership, expectedCommand: "fixture" },
adapter,
),
).resolves.toMatchObject({ cleaned: false });
expect(signalGroup).not.toHaveBeenCalled();
},
);
});
describe("owned process-group cleanup validation: launcher exit races", () => {
it.each([101, 999])(
"retains cleanup uncertainty for a detached descendant in group %s after launcher exit",
async (processGroupId) => {
const child = {
...launcher,
pid: 101,
parentPid: 100,
processGroupId: 101,
};
const signalGroup = vi.fn();
const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce([
{ ...launcher, command: "[MainThread]" },
child,
])
.mockResolvedValue([{ ...child, parentPid: 1, processGroupId }]);
await expect(
cleanupOwnedProcessGroup(
{
...ownership,
expectedCommand: "fixture",
},
{
listProcesses,
environment: () =>
Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
signalGroup,
},
),
).resolves.toMatchObject({ cleaned: false });
expect(signalGroup).not.toHaveBeenCalled();
},
);
it.each(["planning", "signaling"])(
"settles a stale launcher command during %s without skipping surviving group ownership",
async (phase) => {
for (const remaining of ["absent", "zombie", "owned", "foreign"]) {
const stale = { ...launcher, command: "[MainThread]" };
const child = { ...launcher, pid: 101, command: "child" };
let reads = 0;
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => {
reads += 1;
if (phase === "signaling" && reads === 1)
return Promise.resolve([launcher]);
if (reads === (phase === "planning" ? 1 : 2))
return Promise.resolve([stale]);
return Promise.resolve(
remaining === "absent"
? []
: remaining === "zombie"
? [{ ...stale, state: "Z" }]
: [child],
);
},
environment: () =>
Promise.resolve({
REA_PROCESS_RUN_ID:
remaining === "foreign" && reads > 1
? "other-run"
: "run-token",
}),
signalGroup,
};
const result = await cleanupOwnedProcessGroup(
{ ...ownership, expectedCommand: "fixture" },
adapter,
);
if (remaining === "foreign")
expect(result).toMatchObject({
cleaned: false,
reason: "process tree contains an unowned or PID-reused process",
});
else
expect(result).toEqual({
cleaned: true,
signaled: remaining === "owned",
});
if (remaining === "owned")
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
else expect(signalGroup).not.toHaveBeenCalled();
}
},
);
it("fails closed when a mismatched launcher's exit cannot be inspected", async () => {
const { adapter, signalGroup } = host({
100: { REA_PROCESS_RUN_ID: "run-token" },
});
const listProcesses = vi
.fn<ProcessOwnershipHost["listProcesses"]>()
.mockResolvedValueOnce(await adapter.listProcesses())
.mockRejectedValue(new Error("process table unavailable"));
await expect(
cleanupOwnedProcessGroup(
{ ...ownership, expectedCommand: "/owned/hopper" },
{ ...adapter, listProcesses },
),
).resolves.toMatchObject({
cleaned: false,
reason: expect.stringContaining("command identity did not match"),
});
expect(signalGroup).not.toHaveBeenCalled();
});
it.each(["planning", "signaling"])(
"rejects a surviving launcher PID in another group during %s",
async (phase) => {
let reads = 0;
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => {
reads += 1;
return Promise.resolve([
phase === "signaling" && reads === 1
? launcher
: reads === (phase === "planning" ? 1 : 2)
? { ...launcher, command: "[MainThread]" }
: { ...launcher, processGroupId: 999 },
]);
},
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
signalGroup,
};
await expect(
cleanupOwnedProcessGroup(
{ ...ownership, expectedCommand: "fixture" },
adapter,
),
).resolves.toMatchObject({
cleaned: false,
reason: expect.stringContaining("command identity did not match"),
});
expect(signalGroup).not.toHaveBeenCalled();
},
);
});
describe("owned process-group cleanup validation: exited members", () => { describe("owned process-group cleanup validation: exited members", () => {
it("preserves cancellation during process-table observation instead of reporting unverifiable ownership", async () => { it("preserves cancellation during process-table observation instead of reporting unverifiable ownership", async () => {
const controller = new AbortController(); const controller = new AbortController();
@@ -293,13 +464,7 @@ describe("owned process-group cleanup validation: exited members", () => {
const adapter: ProcessOwnershipHost = { const adapter: ProcessOwnershipHost = {
listProcesses: () => listProcesses: () =>
Promise.resolve([ Promise.resolve([
{ launcher,
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{ {
pid: 101, pid: 101,
parentPid: 100, parentPid: 100,
+1 -1
View File
@@ -116,7 +116,7 @@ export const observeOwnedProcessLineage = async (
const identityFailure = launcherIdentityFailure(launcher, ownership); const identityFailure = launcherIdentityFailure(launcher, ownership);
if (identityFailure !== null) if (identityFailure !== null)
return unavailableLineage(ownership, identityFailure); return unavailableLineage(ownership, identityFailure);
const descendants = descendantsOf(launcher.pid, processes); const descendants = descendantsOf([launcher.pid], processes);
const verifiedDescendants: ProcessTableEntry[] = []; const verifiedDescendants: ProcessTableEntry[] = [];
for await (const { process: member, observation } of readProcessRunTokens( for await (const { process: member, observation } of readProcessRunTokens(
host, host,
+4 -4
View File
@@ -6,9 +6,9 @@ export const liveProcesses = (
): readonly ProcessTableEntry[] => ): readonly ProcessTableEntry[] =>
members.filter(({ state }) => !state.startsWith("Z")); members.filter(({ state }) => !state.startsWith("Z"));
/** Return every live-table descendant of a launcher in breadth-first order. */ /** Return every live-table descendant of the selected roots in breadth-first order. */
export const descendantsOf = ( export const descendantsOf = (
launcherPid: number, rootPids: readonly number[],
processes: readonly ProcessTableEntry[], processes: readonly ProcessTableEntry[],
): ProcessTableEntry[] => { ): ProcessTableEntry[] => {
const childrenByParent = new Map<number, ProcessTableEntry[]>(); const childrenByParent = new Map<number, ProcessTableEntry[]>();
@@ -18,8 +18,8 @@ export const descendantsOf = (
childrenByParent.set(process.parentPid, siblings); childrenByParent.set(process.parentPid, siblings);
} }
const descendants: ProcessTableEntry[] = []; const descendants: ProcessTableEntry[] = [];
const pending = [...(childrenByParent.get(launcherPid) ?? [])]; const pending = rootPids.flatMap((pid) => childrenByParent.get(pid) ?? []);
const visited = new Set<number>([launcherPid]); const visited = new Set<number>(rootPids);
for (const process of pending) { for (const process of pending) {
if (visited.has(process.pid)) continue; if (visited.has(process.pid)) continue;
visited.add(process.pid); visited.add(process.pid);
+3 -2
View File
@@ -8,6 +8,7 @@ import {
import { toolContract } from "../contracts/toolContracts.js"; import { toolContract } from "../contracts/toolContracts.js";
import type { EvidenceBundle } from "../domain/evidenceBundle.js"; import type { EvidenceBundle } from "../domain/evidenceBundle.js";
import { ok } from "../domain/result.js"; import { ok } from "../domain/result.js";
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
interface EvidenceToolRegistration { interface EvidenceToolRegistration {
@@ -72,9 +73,9 @@ const registerSnapshotEvidenceTool = ({
server.registerTool( server.registerTool(
snapshotContract.name, snapshotContract.name,
toolRegistrationOptions(snapshotContract), toolRegistrationOptions(snapshotContract),
() => (input) =>
server.delivery.toCallToolResult( server.delivery.toCallToolResult(
ok(bundleForSerialization(session)), inspectEvidenceBundle(bundleForSerialization(session), input),
snapshotContract, snapshotContract,
), ),
); );
@@ -6,7 +6,7 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client"; import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { describe, expect, it } from "vitest"; import { afterAll, describe, expect, it } from "vitest";
import { z } from "zod"; import { z } from "zod";
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js"; import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
@@ -18,6 +18,27 @@ const requireFixture = createRequire(import.meta.url);
const comparisonEnvelope = z.object({ const comparisonEnvelope = z.object({
normalized_result: javaScriptExportShapeComparisonResultSchema, normalized_result: javaScriptExportShapeComparisonResultSchema,
}); });
let mcpComparisonConnection:
| {
readonly client: Client;
readonly transport: StdioClientTransport;
readonly ready: Promise<void>;
}
| undefined;
afterAll(async () => {
if (mcpComparisonConnection === undefined) return;
try {
await mcpComparisonConnection.ready;
} catch {
// Keep the failed connection owned until both resources are closed below.
}
try {
await mcpComparisonConnection.client.close();
} finally {
await mcpComparisonConnection.transport.close();
}
});
describe("exported function binding resolution through the CLI", () => { describe("exported function binding resolution through the CLI", () => {
it.each([ it.each([
@@ -289,6 +310,113 @@ describe("CommonJS assignment-chain receiver identity through the CLI", () => {
}); });
}); });
describe("CommonJS export identity and scope regressions", () => {
it("does not export a named write through the original exports alias after replacement", async () => {
const fixture = await analyzeVersions(
(count) =>
`const VERSION = ${String(count)};\nexports.parse = (module.exports = () => ({ count: VERSION }));`,
"parse",
"app.cjs",
);
for (const path of [fixture.leftPath, fixture.rightPath]) {
const value = requireFixture(path);
expect(typeof value).toBe("function");
expect(Reflect.ownKeys(value)).toEqual(["length", "name"]);
}
const cli = await compareThroughCli(fixture);
expect(cli).toMatchObject({
left: { status: "missing" },
right: { status: "missing" },
summary: { added: 0, removed: 0, changed: 0 },
});
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
});
it("keeps a dead var shadow of the alias-chain RHS unavailable", async () => {
const fixture = await analyzeVersions(
(count) =>
`function parse() { return { count: ${String(count)} }; }\nfunction configure() { if (false) { var parse = () => ({ count: 7 }); } module.exports = exports = parse; }\nconfigure();`,
"default",
"app.cjs",
);
expect(requireFixture(fixture.leftPath)).toBeUndefined();
expect(requireFixture(fixture.rightPath)).toBeUndefined();
const cli = await compareThroughCli(fixture);
expectUncertainComparison(cli);
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
});
it.each(["+=", "||=", "??="] as const)(
"does not treat module.exports %s as a definite function export",
async (operator) => {
const fixture = await analyzeVersions(
(count) =>
`module.exports ${operator} (() => ({ kind: "result", count: ${String(count)} }));`,
"default",
"app.cjs",
);
const left = requireFixture(fixture.leftPath);
const right = requireFixture(fixture.rightPath);
if (operator === "+=") {
expect(typeof left).toBe("string");
expect(typeof right).toBe("string");
} else {
expect(left).toEqual({});
expect(right).toEqual({});
}
const cli = await compareThroughCli(fixture);
expectUncertainComparison(cli);
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
},
);
it("keeps a parameter-default mutation unknown in a CommonJS return shape", async () => {
const fixture = await analyzeVersions(
(count) =>
`const source = { value: 0 }; module.exports = exports = function current(arg = null, set = (arg = source)) { var arg; arg.value = ${String(count)}; return { kind: "result", count: source.value }; };`,
"default",
"app.cjs",
);
expect(requireFixture(fixture.leftPath)()).toEqual({
kind: "result",
count: 1,
});
expect(requireFixture(fixture.rightPath)()).toEqual({
kind: "result",
count: 2,
});
for (const evidence of [fixture.input.left, fixture.input.right]) {
const analysis = z
.object({
normalized_result: javascriptApplicationAnalysisResultSchema,
})
.parse(evidence).normalized_result;
const shapes = analysis.graph.nodes
.flatMap(({ observations }) => observations)
.find(
({ properties }) =>
properties.semantic_role === "export-return-shapes" &&
properties.exported_name === "default",
)?.properties.static_return_shapes;
expect(shapes).toEqual([
expect.objectContaining({
fields: expect.arrayContaining([
expect.objectContaining({ path: "/count", state: "unknown" }),
]),
}),
]);
}
const cli = await compareThroughCli(fixture);
expect(cli).toMatchObject({
left: { status: "selected" },
right: { status: "selected" },
summary: { added: 0, removed: 0, changed: 0 },
changes: [],
});
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
});
});
describe("stable exported function bindings through the CLI", () => { describe("stable exported function bindings through the CLI", () => {
it.each([ it.each([
{ {
@@ -798,34 +926,37 @@ const compareThroughCli = async (
}; };
const compareThroughMcp = async (input: Record<string, unknown>) => { const compareThroughMcp = async (input: Record<string, unknown>) => {
const transport = new StdioClientTransport({ const client = await getMcpComparisonClient();
command: process.execPath, const response = await client.callTool({
args: [resolve("scripts/rea.mjs"), "mcp"], name: "compare_javascript_export_shapes",
cwd: process.cwd(), arguments: input,
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
stderr: "pipe",
}); });
const client = new Client({ expect(response.isError).not.toBe(true);
name: "exported-function-bindings", await client.ping();
version: "1", return comparisonEnvelope.parse(response.structuredContent).normalized_result;
}); };
try {
await client.connect(transport); const getMcpComparisonClient = async (): Promise<Client> => {
const response = await client.callTool({ if (mcpComparisonConnection === undefined) {
name: "compare_javascript_export_shapes", const transport = new StdioClientTransport({
arguments: input, command: process.execPath,
args: [resolve("scripts/rea.mjs"), "mcp"],
cwd: process.cwd(),
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
stderr: "pipe",
}); });
expect(response.isError).not.toBe(true); const client = new Client({
await client.ping(); name: "exported-function-bindings",
return comparisonEnvelope.parse(response.structuredContent) version: "1",
.normalized_result; });
} finally { mcpComparisonConnection = {
try { client,
await client.close(); transport,
} finally { ready: client.connect(transport),
await transport.close(); };
}
} }
await mcpComparisonConnection.ready;
return mcpComparisonConnection.client;
}; };
const runtimeExport = async ( const runtimeExport = async (
@@ -39,6 +39,57 @@ const expectPeInputFailureAcrossAdapters = async (input: {
expect(parseMcpToolError(response)).toEqual({ error: result.json }); expect(parseMcpToolError(response)).toEqual({ error: result.json });
}; };
const expectPePaddingFailureAcrossAdapters = async (input: {
readonly cli: TestCli;
readonly client: Client;
readonly toolName: string;
readonly path: string;
}) => {
const { cli, client, toolName, path } = input;
const fixture = peResourceFixture();
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
fixture.bytes.writeUInt32LE(
0x2000 + virtualSize,
fixture.dataOffsets[0] ?? 0,
);
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
await writeFile(path, fixture.bytes);
const cliResult = await cli.run({
arguments: ["inspect-pe-resources", path, "--json"],
});
expect(cliResult.exitCode).not.toBe(0);
const mcpResult = await client.callTool({
name: toolName,
arguments: { path },
});
expect(mcpResult.isError).toBe(true);
expect(peResourcesSchema.safeParse(mcpResult.structuredContent).success).toBe(
false,
);
};
const expectPeInputFailuresAcrossAdapters = async (input: {
readonly cli: TestCli;
readonly client: Client;
readonly toolName: string;
readonly root: string;
}) => {
const { cli, client, toolName, root } = input;
const malformedPath = join(root, "malformed.exe");
await writeFile(malformedPath, Buffer.from("not a PE image"));
for (const [path, reason] of [
[join(root, "missing.exe"), "invalid_value"],
[malformedPath, "invalid_format"],
] as const)
await expectPeInputFailureAcrossAdapters({
cli,
client,
toolName,
path,
reason,
});
};
cliTest( cliTest(
"inspects PE32/PE32+ resources through compiled CLI and real stdio MCP without an active target", "inspects PE32/PE32+ resources through compiled CLI and real stdio MCP without an active target",
async ({ cli }) => { async ({ cli }) => {
@@ -121,20 +172,18 @@ cliTest(
).toBe(evidence.evidence_id); ).toBe(evidence.evidence_id);
expect(await readFile(path)).toEqual(fixture.bytes); expect(await readFile(path)).toEqual(fixture.bytes);
} }
const malformedPath = join(root, "malformed.exe"); await expectPePaddingFailureAcrossAdapters({
await writeFile(malformedPath, Buffer.from("not a PE image")); cli,
for (const [path, reason] of [ client,
[join(root, "missing.exe"), "invalid_value"], toolName: contract.name,
[malformedPath, "invalid_format"], path: join(root, "raw-padding.exe"),
] as const) { });
await expectPeInputFailureAcrossAdapters({ await expectPeInputFailuresAcrossAdapters({
cli, cli,
client, client,
toolName: contract.name, toolName: contract.name,
path, root,
reason, });
});
}
const absent = peResourceFixture(); const absent = peResourceFixture();
absent.bytes.fill(0, absent.directoryAt, absent.directoryAt + 8); absent.bytes.fill(0, absent.directoryAt, absent.directoryAt + 8);
const path = join(root, "absent.exe"); const path = join(root, "absent.exe");
+158 -2
View File
@@ -30,8 +30,9 @@ const chromeExecutable = [
); );
// Issue 1586's page, plus the HTML, image, use, and foreign-origin cases the // Issue 1586's page, plus the HTML, image, use, and foreign-origin cases the
// real snapshot must distinguish. The test is skipped when no Chrome-family // real snapshot must distinguish. The XHTML page is the prefixed-svg gap
// executable is installed. // reported on PR 1602. The tests are skipped when no Chrome-family executable
// is installed.
const pageHtml = `<!doctype html><title>x</title> const pageHtml = `<!doctype html><title>x</title>
<svg width="10" height="10"> <svg width="10" height="10">
<a id="s2" xlink:href="/xl-s2h"><text>s2</text></a> <a id="s2" xlink:href="/xl-s2h"><text>s2</text></a>
@@ -159,6 +160,161 @@ it.skipIf(chromeExecutable === undefined)(
}, },
); );
const xhtmlPage = `<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:svg="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<head><title>x</title></head>
<body>
<svg:svg width="10" height="10">
<svg:a id="pa1" xlink:href="/xl-pa1"><svg:text>p</svg:text></svg:a>
<svg:image id="img" xlink:href="/xl-image"/>
<svg:foreignObject width="10" height="10">
<a id="fo-a" xlink:href="/xl-html-fo">no</a>
<h:a xmlns:h="http://www.w3.org/1999/xhtml" id="fo-h" xlink:href="/xl-h">no</h:a>
<svg:a id="fo-svg" xlink:href="/xl-fo-svg"><svg:text>y</svg:text></svg:a>
</svg:foreignObject>
</svg:svg>
<svg:a id="orphan" xlink:href="/xl-orphan"><svg:text>o</svg:text></svg:a>
<svg xmlns="http://www.w3.org/2000/svg" xmlns:x="http://www.w3.org/1999/xlink" width="10" height="10">
<a id="custom" x:href="/xl-custom"><text>c</text></a>
<a id="both" href="/href-wins" x:href="/xlink-loses"><text>b</text></a>
<a id="empty" href="" x:href="/xlink-empty"><text>e</text></a>
</svg>
<svg:a id="out" xlink:href="https://cdn.example.test/out"><svg:text>out</svg:text></svg:a>
<bogus:svg xmlns:bogus="http://example.com/not-svg" id="bogus" xlink:href="/xl-bogus"/>
<a id="xhtml-a" xlink:href="/xl-xhtml-a">no</a>
</body>
</html>
`;
for (const { format, page, contentType, nodeNames, link, destinations } of [
{
format: "XHTML",
page: xhtmlPage,
contentType: "application/xhtml+xml",
nodeNames: ["html", "svg:a", "bogus:svg"],
link: { name: "svg:a", attribute: "xlink:href" },
destinations: [
"/xl-pa1",
"/xl-image",
"/xl-fo-svg",
"/xl-orphan",
"/xl-custom",
"/href-wins",
"",
null,
],
},
{
format: "SVG",
page: `<svg xmlns="http://www.w3.org/2000/svg" xmlns:p="http://www.w3.org/1999/xlink"><a p:href="/linked"><text>link</text></a></svg>`,
contentType: "image/svg+xml",
nodeNames: ["svg", "a"],
link: { name: "a", attribute: "p:href" },
destinations: ["/linked"],
},
]) {
it.skipIf(chromeExecutable === undefined)(
`reports namespaced SVG links from a standalone ${format} document through headless Chrome`,
async ({ processes }) => {
const executable = chromeExecutable;
if (executable === undefined) return;
const profile = await createTestTempDirectory("rea-svg-xhtml-");
const server = createServer((_request, response) => {
response.writeHead(200, {
"content-type": `${contentType}; charset=utf-8`,
});
response.end(page);
});
await new Promise<void>((resolve) => {
server.listen(0, "127.0.0.1", resolve);
});
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("XHTML fixture server did not bind a TCP port");
const origin = `http://127.0.0.1:${String(address.port)}`;
const pageUrl = `${origin}/page.xhtml`;
let stderr = "";
const child = processes.spawn(executable, [
"--headless=new",
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
? ["--no-sandbox"]
: []),
"--remote-debugging-address=127.0.0.1",
"--remote-debugging-port=0",
`--user-data-dir=${profile}`,
"--no-first-run",
"--no-default-browser-check",
"--disable-background-networking",
"--disable-component-update",
"--disable-dev-shm-usage",
"--disable-sync",
"--metrics-recording-only",
pageUrl,
]);
child.stdout.on("data", () => undefined);
child.stderr.on("data", (chunk: Buffer) => {
if (stderr.length < 8_192) stderr += chunk.toString("utf8");
});
try {
const port = await waitForBrowserDevtoolsPort({
child,
executable,
activePortPath: join(profile, "DevToolsActivePort"),
stderr: () => stderr,
timeoutMs: 20_000,
});
const endpoint = `http://127.0.0.1:${String(port)}`;
const provider = new CdpBrowserProvider();
const targetId = await pageTarget(provider, endpoint, origin, pageUrl);
const result = await provider.inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: endpoint,
allowed_origins: [origin],
target_id: targetId,
observation_ms: 0,
}),
);
if (!result.ok) throw result.error;
const urls = result.value.metadata.dom_urls;
for (const name of nodeNames) {
expect(
result.value.dom.nodes.some((node) => node.node_name === name),
).toBe(true);
}
expect(
result.value.dom.nodes.some(
(node) =>
node.node_name === link.name &&
node.attribute_names.includes(link.attribute),
),
).toBe(true);
expect(urls).toEqual(
destinations.map((destination) =>
expect.objectContaining({
attribute: "href",
url:
destination === null
? null
: destination === ""
? pageUrl
: `${origin}${destination}`,
destination_scope:
destination === null ? "outside_policy" : "approved",
}),
),
);
} finally {
child.kill("SIGTERM");
if (!(await waitForExit(child, 2_000))) child.kill("SIGKILL");
await waitForExit(child, 2_000);
await new Promise<void>((resolve) => {
server.close(() => resolve());
});
}
},
);
}
const pageTarget = async ( const pageTarget = async (
provider: CdpBrowserProvider, provider: CdpBrowserProvider,
endpoint: string, endpoint: string,
@@ -9,6 +9,53 @@ import { cliTest } from "../../support/cli/cliFixture.js";
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js"; import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
import { createEvidence } from "../../../src/domain/evidence.js"; import { createEvidence } from "../../../src/domain/evidence.js";
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js"; import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
cliTest(
"discovers native records from a complete portable bundle using the shared workflow",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-evidence-discovery-cli-");
const parent = createEvidence(
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
{
operation: "analyze_function",
parameters: {},
result: ghidraFunctionDossier(),
},
);
const bundle = createEvidenceBundle([parent]);
const path = join(root, "evidence.json");
await writeFile(path, JSON.stringify(bundle));
const options = {
detail: "summary" as const,
filters: { evidence_id: parent.evidence_id },
};
const result = await cli.run({
arguments: [
"inspect-evidence-bundle",
JSON.stringify({ path, ...options }),
"--json",
],
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
});
expect(result.exitCode).toBe(0);
const expected = inspectEvidenceBundle(bundle, options);
if (!expected.ok) throw expected.error;
expect(result.json).toEqual(expected.value);
const complete = await cli.run({
arguments: [
"inspect-evidence-bundle",
JSON.stringify({ path }),
"--json",
],
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
});
expect(complete.exitCode).toBe(0);
expect(complete.json).toEqual(bundle);
},
);
cliTest( cliTest(
"projects one section from a JSON file of inline layout Evidence", "projects one section from a JSON file of inline layout Evidence",
async ({ cli }) => { async ({ cli }) => {
@@ -97,6 +97,14 @@ describe("browser comparison JSON file inputs", () => {
INTEGRATION_TEST_TIMEOUT_MS, INTEGRATION_TEST_TIMEOUT_MS,
); );
it(
"reports malformed PNG chunks as invalid input and valid unsupported PNGs as unsupported targets",
async () => {
await assertPngInputErrorProjections();
},
INTEGRATION_TEST_TIMEOUT_MS,
);
it( it(
"identifies a missing capture file instead of reporting malformed JSON", "identifies a missing capture file instead of reporting malformed JSON",
async () => { async () => {
@@ -123,3 +131,105 @@ describe("browser comparison JSON file inputs", () => {
INTEGRATION_TEST_TIMEOUT_MS, INTEGRATION_TEST_TIMEOUT_MS,
); );
}); });
const assertPngInputErrorProjections = async (): Promise<void> => {
const damagedBytes = noisePng(1, 1);
damagedBytes[29] = (damagedBytes[29] ?? 0) ^ 1;
const damaged = JSON.stringify(createWebScreenshotArtifact(damagedBytes));
expect(
await runCli(["compare-web-screenshots", damaged, damaged, "--json"]),
).toMatchObject({
code: "invalid_request",
details: {
operation: "compare_web_screenshots",
issues: [
{ path: [], reason: "invalid_format", message: expect.any(String) },
],
},
});
const header = Buffer.alloc(13);
header.writeUInt32BE(1, 0);
header.writeUInt32BE(1, 4);
header.set([8, 2, 0, 0, 0], 8);
const malformedDeflate = JSON.stringify(
createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", header),
pngChunk("IDAT", Buffer.from("bad")),
pngChunk("IEND", Buffer.alloc(0)),
]),
),
);
expect(
await runCli([
"compare-web-screenshots",
malformedDeflate,
malformedDeflate,
"--json",
]),
).toMatchObject({
code: "invalid_request",
details: {
operation: "compare_web_screenshots",
issues: [{ path: [], reason: "invalid_format" }],
},
});
const grayscaleHeader = Buffer.alloc(13);
grayscaleHeader.writeUInt32BE(1, 0);
grayscaleHeader.writeUInt32BE(1, 4);
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
const grayscale = JSON.stringify(
createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", grayscaleHeader),
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
pngChunk("IEND", Buffer.alloc(0)),
]),
),
);
expect(
await runCli(["compare-web-screenshots", grayscale, grayscale, "--json"]),
).toMatchObject({
code: "unsupported_target",
details: { operation: "compare_web_screenshots" },
});
const oversizedHeader = Buffer.alloc(13);
oversizedHeader.writeUInt32BE(0x7fffffff, 0);
oversizedHeader.writeUInt32BE(1, 4);
oversizedHeader.set([8, 6, 0, 0, 0], 8);
const hugeDimensions = JSON.stringify(
createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", oversizedHeader),
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
pngChunk("IEND", Buffer.alloc(0)),
]),
),
);
expect(
await runCli([
"compare-web-screenshots",
hugeDimensions,
hugeDimensions,
"--json",
]),
).toMatchObject({
code: "resource_constraint",
details: {
operation: "compare_web_screenshots",
resource: "memory",
reported_limits: {
maximum_working_memory_bytes: 256 * 1024 * 1024,
estimated_working_memory_bytes: "34359738499",
before_dimensions: { width: 0x7fffffff, height: 1 },
after_dimensions: { width: 0x7fffffff, height: 1 },
},
},
});
};
+58 -8
View File
@@ -1,5 +1,5 @@
import { execFile } from "node:child_process"; import { execFile } from "node:child_process";
import { mkdir } from "node:fs/promises"; import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
@@ -9,20 +9,34 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const execute = promisify(execFile); const execute = promisify(execFile);
const MARKER = "REA_TEST_RESOLVED "; const MARKER = "REA_TEST_RESOLVED ";
// Dependencies that only one command uses; startup must not resolve them. // Dependencies and composition modules used by one command family; startup
const DEFERRED_PACKAGES = ["playwright-core", "isomorphic-git"] as const; // must not resolve them just to register the complete command catalog.
const DEFERRED_IMPORTS = [
{
name: "playwright-core",
path: "/node_modules/playwright-core/",
},
{
name: "isomorphic-git",
path: "/node_modules/isomorphic-git/",
},
{
name: "binary-composition",
path: "/dist/composition/binary.js",
},
] as const;
// Report each resolved deferred package on stderr without changing resolution. // Report each resolved deferred package on stderr without changing resolution.
const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(` const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(`
import { registerHooks } from "node:module"; import { registerHooks } from "node:module";
const packages = ${JSON.stringify(DEFERRED_PACKAGES)}; const imports = ${JSON.stringify(DEFERRED_IMPORTS)};
registerHooks({ registerHooks({
resolve(specifier, context, next) { resolve(specifier, context, next) {
const resolved = next(specifier, context); const resolved = next(specifier, context);
const name = packages.find((item) => const name = imports.find((item) =>
resolved.url.includes("/node_modules/" + item + "/"), resolved.url.includes(item.path),
); );
if (name !== undefined) if (name !== undefined)
process.stderr.write(${JSON.stringify(MARKER)} + name + "\\n"); process.stderr.write(${JSON.stringify(MARKER)} + name.name + "\\n");
return resolved; return resolved;
}, },
}); });
@@ -68,9 +82,45 @@ describe("CLI startup imports", () => {
).resolves.toContain("isomorphic-git"); ).resolves.toContain("isomorphic-git");
}); });
it("does not load command-specific packages to register commands", async () => { it("does not load command-specific code to register the command catalog", async () => {
await expect( await expect(
resolvedPackages(["scripts/rea.mjs", "--help"]), resolvedPackages(["scripts/rea.mjs", "--help"]),
).resolves.toEqual(new Set()); ).resolves.toEqual(new Set());
}); });
it("does not load native provider composition for JavaScript application analysis", async () => {
const source = await createTestTempDirectory("rea-cli-js-startup-");
await mkdir(join(source, "src"));
await writeFile(
join(source, "package.json"),
JSON.stringify({
name: "startup-fixture",
version: "1.0.0",
main: "src/index.js",
}),
);
await writeFile(
join(source, "src/index.js"),
"export const answer = 42;\n",
);
await expect(
resolvedPackages([
"scripts/rea.mjs",
"analyze-javascript-application",
source,
"--json",
]),
).resolves.toEqual(new Set());
});
it("loads native provider composition when a provider operation is called", async () => {
await expect(
resolvedPackages(
evaluate(
'const { createDirectAnalysis } = await import("./dist/composition/directAnalysis.js"); await createDirectAnalysis({}).runSessionStatus();',
),
),
).resolves.toContain("binary-composition");
});
}); });
@@ -80,7 +80,11 @@ describe("compiled reference-source import preflight failures", () => {
const directory = await createTestTempDirectory( const directory = await createTestTempDirectory(
"rea-reference-cli-status-", "rea-reference-cli-status-",
); );
const root = join(directory, "input"); // Exercise JSON escaping on POSIX too; Windows paths already need it.
const root = join(
directory,
process.platform === "win32" ? "input" : "input\\escaped",
);
if (scenario.rootKind === "regular-file") if (scenario.rootKind === "regular-file")
await writeFile(root, "inert input\n"); await writeFile(root, "inert input\n");
const result = await cli.run({ const result = await cli.run({
@@ -10,6 +10,7 @@ import {
import { join } from "node:path"; import { join } from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
import { buildBinary } from "plist";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js"; import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
@@ -691,6 +692,71 @@ describe("iOS-style app bundle targets", () => {
); );
}); });
describe("binary Info.plist app bundle targets", () => {
it.each([
["Mac.app", ["Contents", "MacOS"], ["Contents", "Info.plist"]],
["Phone.app", [], ["Info.plist"]],
])(
"resolves %s from a binary Info.plist on every host",
async (bundle, programs, plist) => {
const directory = await createTestTempDirectory("rea-app-bplist-");
// Windows file names cannot hold a tab or end in a space.
const names =
process.platform === "win32"
? ["Ordinary"]
: ["Ordinary", " App ", "Tab\t"];
for (const name of names) {
const app = join(directory, name, bundle);
const executable = join(app, ...programs, name);
await mkdir(join(app, ...programs), { recursive: true });
await mkdir(join(app, ...plist.slice(0, -1)), { recursive: true });
await writeFile(
join(app, ...plist),
buildBinary({ CFBundleExecutable: name, CFBundleVersion: "1" }),
);
await writeFile(executable, thinMach(0xfeedfacf, 0x0100000c));
const result = await parseBinaryTarget(app, {
cwd: directory,
hostArchitecture: "arm64",
});
expect(result.ok && result.value).toMatchObject({
path: await realpath(executable),
format: "mach-o",
});
}
},
);
it.each([
[
"XML",
"<plist><dict><key>__proto__</key><dict><key>CFBundleExecutable</key><string>Forged</string></dict></dict></plist>",
],
[
"binary",
buildBinary(JSON.parse('{"__proto__":{"CFBundleExecutable":"Forged"}}')),
],
])(
"ignores CFBundleExecutable nested under __proto__ in %s Info.plist files",
async (_encoding, contents) => {
const directory = await createTestTempDirectory("rea-app-proto-only-");
const app = join(directory, "Proto.app");
await mkdir(join(app, "Contents", "MacOS"), { recursive: true });
await writeFile(join(app, "Contents", "Info.plist"), contents);
await writeFile(
join(app, "Contents", "MacOS", "Forged"),
thinMach(0xfeedfacf, 0x0100000c),
);
expect(await resolveAppBundleExecutable(app)).toMatchObject({
ok: false,
error: {
reason: expect.stringContaining("CFBundleExecutable is missing"),
},
});
},
);
});
describe("DOS binary target I/O", () => { describe("DOS binary target I/O", () => {
it("validates a complete DOS file beyond the initial metadata probe", async () => { it("validates a complete DOS file beyond the initial metadata probe", async () => {
const directory = await createTestTempDirectory("rea-dos-target-"); const directory = await createTestTempDirectory("rea-dos-target-");
+37 -5
View File
@@ -83,6 +83,7 @@ async function fixture() {
author("platformbot", "Bot"), author("platformbot", "Bot"),
author("claude"), author("claude"),
author("cursoragent"), author("cursoragent"),
author("morluto"),
]), ]),
associatedPullRequests: page([]), associatedPullRequests: page([]),
}, },
@@ -186,7 +187,7 @@ it("inventories side branches and paginated credits, then checks and renders syn
); );
expect(f.record).toContain("Thanks @dana."); expect(f.record).toContain("Thanks @dana.");
expect(f.record).not.toMatch( expect(f.record).not.toMatch(
/@(?:merger|automation|platformbot|claude|cursoragent)/u, /@(?:merger|automation|platformbot|claude|cursoragent|morluto)/u,
); );
const body = join(f.directory, "body.md"); const body = join(f.directory, "body.md");
await writeFile( await writeFile(
@@ -211,6 +212,37 @@ it("inventories side branches and paginated credits, then checks and renders syn
await expect( await expect(
f.invoke(["render", "--manifest", f.manifest, "--output", output]), f.invoke(["render", "--manifest", f.manifest, "--output", output]),
).rejects.toMatchObject({ stderr: expect.stringContaining("EEXIST") }); ).rejects.toMatchObject({ stderr: expect.stringContaining("EEXIST") });
const { stdout: summary } = await f.invoke([
"record",
"--manifest",
f.manifest,
"--summary",
]);
expect(summary).toContain("3 commits and 1 merged PR]");
expect(summary).toContain(
"/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json",
);
expect(summary).not.toContain("Thanks");
const conciseNotes = f.notes
.replace(f.record, summary)
.replace(
"### Changes\n",
"### Changes\n\n- Capability details (#101).\n- Related migration (#51).\n",
);
await writeFile(join(f.directory, "CHANGELOG.md"), conciseNotes);
await f.invoke(["check", "--manifest", f.manifest]);
await writeFile(
join(f.directory, "CHANGELOG.md"),
conciseNotes.replace(
"6.4.0.contributions.json",
"6.3.0.contributions.json",
),
);
await expect(
f.invoke(["check", "--manifest", f.manifest]),
).rejects.toMatchObject({
stderr: expect.stringContaining("Contribution record differs"),
});
await expect( await expect(
f.invoke([ f.invoke([
"inventory", "inventory",
@@ -238,6 +270,10 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
throw new Error("Fixture is missing its direct contribution"); throw new Error("Fixture is missing its direct contribution");
for (const [replacement, error] of [ for (const [replacement, error] of [
[f.bullet.replace(", @bob", ""), "Missing Thanks @bob"], [f.bullet.replace(", @bob", ""), "Missing Thanks @bob"],
[
f.bullet.replace(", @bob", "") + "\n- Thanks @bob for other work.",
"Missing Thanks @bob",
],
[f.bullet + " (#999)", "Unknown release reference #999"], [f.bullet + " (#999)", "Unknown release reference #999"],
[f.bullet + "a".repeat(65_536), "inline PR body limit"], [f.bullet + "a".repeat(65_536), "inline PR body limit"],
[f.bullet + "汉".repeat(43_000), "GitHub's body budget"], [f.bullet + "汉".repeat(43_000), "GitHub's body budget"],
@@ -245,10 +281,6 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
f.bullet + `\n- Fixed direct behavior (${directLink}).`, f.bullet + `\n- Fixed direct behavior (${directLink}).`,
"Missing Thanks @dana", "Missing Thanks @dana",
], ],
[
f.bullet + "\n- Fixed the reported problem (#51).",
"Missing Thanks @reporter",
],
[ [
f.bullet + " Thanks @automation[bot].", f.bullet + " Thanks @automation[bot].",
"Unverified human credit @automation[bot]", "Unverified human credit @automation[bot]",
+111
View File
@@ -7,6 +7,8 @@ import { z } from "zod";
import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js"; import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js";
import { toolContract } from "../../../src/contracts/toolContracts.js"; import { toolContract } from "../../../src/contracts/toolContracts.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
import { jsonObjectSchema } from "../../../src/domain/jsonValue.js";
import { functionDossierSchema } from "../../../src/domain/hopperValues.js"; import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js"; import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
import { ok } from "../../../src/domain/result.js"; import { ok } from "../../../src/domain/result.js";
@@ -46,6 +48,115 @@ const connect = async (binaryLayout?: BinaryLayoutService) => {
return { client, session }; return { client, session };
}; };
it("discovers an oversized native dossier by exact metadata and reads its retained view", async () => {
const { client, session } = await connect();
const provider = { id: "ghidra", name: "Ghidra", version: "12.1.4" };
const profile = createAnalysisProfile(provider, { language: "fixture" });
const parent = createEvidence(
{ path: "/fixtures/large.exe", format: "pe", sha256: "a".repeat(64) },
provider,
{
operation: "analyze_function",
parameters: { procedure: "0x401000" },
analysisProfile: profile,
result: {
...jsonObjectSchema.parse(ghidraFunctionDossier()),
pseudocode: "x".repeat(6 * 1024 * 1024),
},
rawResult: null,
},
);
const other = createEvidence(
{ path: "/fixtures/other.exe", format: "pe", sha256: "b".repeat(64) },
provider,
{
operation: "analyze_function",
parameters: {},
result: ghidraFunctionDossier(),
},
);
for (const record of [parent, other]) {
const retained = session.recordEvidence(record);
if (!retained.ok) throw retained.error;
}
const broad = await client.callTool({
name: "get_evidence_bundle",
arguments: {},
});
const broadError = parseMcpToolError(broad).error;
expect(broadError.code).toBe("resource_constraint");
expect(broadError.remediation?.action).toContain(
"get_evidence_bundle and detail: summary",
);
const response = await client.callTool({
name: "get_evidence_bundle",
arguments: {
detail: "summary",
filters: {
operation: "analyze_function",
target_sha256: "a".repeat(64),
analysis_profile_digest: profile.digest,
procedure_address: "0x401000",
},
},
});
expect(response.isError).not.toBe(true);
const result = toolContract("get_evidence_bundle").outputSchema.parse(
response.structuredContent,
).result;
expect(result).toMatchObject({
kind: "evidence-bundle-summary",
total_retained_records: 2,
matching_records: 1,
records: [
{
evidence_id: parent.evidence_id,
analysis_profile_digest: profile.digest,
retention: "complete-record",
native_dossier: { available: true, procedure_address: "0x401000" },
},
],
});
expect(JSON.stringify(response)).not.toContain("pseudocode");
expect(
JSON.parse(response.content.find((c) => c.type === "text")?.text ?? "null"),
).toEqual(response.structuredContent);
const page = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
view: { kind: "native", facet: "pseudocode", offset: 0, limit: 32 },
},
});
expect(page.isError).not.toBe(true);
expect(
toolContract("inspect_analysis_view").outputSchema.parse(
page.structuredContent,
).normalized_result,
).toMatchObject({
item: { text: "x".repeat(32) },
coverage: { total: 6 * 1024 * 1024, exhausted: false },
});
const empty = await client.callTool({
name: "get_evidence_bundle",
arguments: {
detail: "summary",
filters: { evidence_id: `ev_${"e".repeat(64)}` },
},
});
expect(
toolContract("get_evidence_bundle").outputSchema.parse(
empty.structuredContent,
).result,
).toMatchObject({ matching_records: 0, records: [] });
await client.ping();
const invalid = await client.callTool({
name: "get_evidence_bundle",
arguments: { filters: { operation: "analyze_function" } },
});
expect(parseMcpToolError(invalid).error.code).toBe("invalid_request");
});
it("advertises exact schemas and projects one layout section from retained Evidence", async () => { it("advertises exact schemas and projects one layout section from retained Evidence", async () => {
const layout = analysisViewLayoutFixture(); const layout = analysisViewLayoutFixture();
const service = new BinaryLayoutService({ const service = new BinaryLayoutService({
+103
View File
@@ -1,4 +1,5 @@
import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { crc32, deflateSync } from "node:zlib";
import { parseEvidence } from "../../../src/domain/evidence.js"; import { parseEvidence } from "../../../src/domain/evidence.js";
import { ok as resultOk } from "../../../src/domain/result.js"; import { ok as resultOk } from "../../../src/domain/result.js";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
@@ -7,6 +8,7 @@ import { afterEach, expect, it } from "vitest";
import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js"; import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js";
import { createWebScreenshotArtifact } from "../../../src/domain/webScreenshot.js";
import { JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE } from "../../../src/contracts/javascript/javascriptRuntimeReconciliationExample.js"; import { JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE } from "../../../src/contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js"; import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
import { createServer } from "../../../src/server/createServer.js"; import { createServer } from "../../../src/server/createServer.js";
@@ -262,6 +264,8 @@ const verifySessionAndComparisonTools = async (
expect(visual.structuredContent).toMatchObject({ expect(visual.structuredContent).toMatchObject({
normalized_result: { status: "identical", changed_pixels: 0 }, normalized_result: { status: "identical", changed_pixels: 0 },
}); });
await assertScreenshotPngFailures(connected.client);
expect(inspected.structuredContent).toMatchObject({ expect(inspected.structuredContent).toMatchObject({
normalized_result: expect.any(Object), normalized_result: expect.any(Object),
operation: "inspect_web_page", operation: "inspect_web_page",
@@ -270,6 +274,68 @@ const verifySessionAndComparisonTools = async (
}); });
}; };
const pngChunk = (type: string, data: Buffer): Buffer => {
const chunk = Buffer.alloc(12 + data.length);
chunk.writeUInt32BE(data.length, 0);
chunk.write(type, 4, 4, "ascii");
data.copy(chunk, 8);
chunk.writeUInt32BE(
crc32(chunk.subarray(4, 8 + data.length)),
8 + data.length,
);
return chunk;
};
const assertScreenshotPngFailures = async (client: Client): Promise<void> => {
const header = Buffer.alloc(13);
header.writeUInt32BE(1, 0);
header.writeUInt32BE(1, 4);
header.set([8, 2, 0, 0, 0], 8);
const malformedPng = Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", header),
pngChunk("IDAT", Buffer.from("bad")),
pngChunk("IEND", Buffer.alloc(0)),
]);
const malformedArtifact = createWebScreenshotArtifact(malformedPng);
const malformed = await client.callTool({
name: "compare_web_screenshots",
arguments: {
before: malformedArtifact,
after: malformedArtifact,
},
});
expect(parseMcpToolError(malformed)).toMatchObject({
error: {
code: "invalid_request",
category: "invalid_input",
details: {
operation: "compare_web_screenshots",
issues: [{ path: [], reason: "invalid_format" }],
},
},
});
const grayscaleHeader = Buffer.alloc(13);
grayscaleHeader.writeUInt32BE(1, 0);
grayscaleHeader.writeUInt32BE(1, 4);
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
const unsupportedPng = Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", grayscaleHeader),
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
pngChunk("IEND", Buffer.alloc(0)),
]);
const unsupported = createWebScreenshotArtifact(unsupportedPng);
const rejectedFormat = await client.callTool({
name: "compare_web_screenshots",
arguments: { before: unsupported, after: unsupported },
});
expect(parseMcpToolError(rejectedFormat)).toMatchObject({
error: { code: "unsupported_target", category: "unsupported_target" },
});
};
it("does not attach to a target outside the request's allowed origin scope", async () => { it("does not attach to a target outside the request's allowed origin scope", async () => {
const browser = await startFakeCdpBrowser(); const browser = await startFakeCdpBrowser();
browsers.push(browser); browsers.push(browser);
@@ -295,6 +361,43 @@ it("does not attach to a target outside the request's allowed origin scope", asy
expect(browser.commands).toHaveLength(0); expect(browser.commands).toHaveLength(0);
}); });
it("projects legal oversized PNG dimensions as a memory constraint", async () => {
const header = Buffer.alloc(13);
header.writeUInt32BE(0x7fffffff, 0);
header.writeUInt32BE(1, 4);
header.set([8, 6, 0, 0, 0], 8);
const artifact = createWebScreenshotArtifact(
Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
pngChunk("IHDR", header),
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
pngChunk("IEND", Buffer.alloc(0)),
]),
);
const connected = await connectBrowser();
const result = await connected.client.callTool({
name: "compare_web_screenshots",
arguments: { before: artifact, after: artifact },
});
expect(parseMcpToolError(result)).toMatchObject({
error: {
code: "resource_constraint",
category: "resource_constraint",
details: {
operation: "compare_web_screenshots",
resource: "memory",
reported_limits: {
maximum_working_memory_bytes: 256 * 1024 * 1024,
estimated_working_memory_bytes: "34359738499",
before_dimensions: { width: 0x7fffffff, height: 1 },
after_dimensions: { width: 0x7fffffff, height: 1 },
},
},
},
});
});
const connectBrowser = async () => { const connectBrowser = async () => {
const session = createTestBinarySession(() => ({ const session = createTestBinarySession(() => ({
execute: () => Promise.resolve(observed(null)), execute: () => Promise.resolve(observed(null)),
+1 -1
View File
@@ -118,7 +118,7 @@ const responseBodyResult = (options: FakeOptions) =>
}; };
const screenshotResult = () => ({ const screenshotResult = () => ({
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PzWvWQAAAABJRU5ErkJggg==", data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
}); });
const storageUsageResult = () => ({ const storageUsageResult = () => ({
+283 -4
View File
@@ -142,8 +142,161 @@
terminal. terminal.
</p> </p>
<p class="requirements"> <p class="requirements">
You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+, and npm. You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+. npm comes with
Node.js.
</p> </p>
<details class="evidence-details" id="install-node">
<summary>Don't have Node.js, or not sure?</summary>
<p>Open a terminal and check:</p>
<div class="code-block">
<div class="code-header">
<span>Terminal</span
><button
class="copy-button"
type="button"
data-copy="node-check"
aria-label="Copy Node.js version check"
>
Copy
</button>
</div>
<pre><code id="node-check">node -v
npm -v</code></pre>
</div>
<p>
If <code>node -v</code> prints v22.19+, v24.11+ or v26+, you're
ready. If it prints an older version or "not recognized" / "command
not found", install Node.js 24 LTS:
</p>
<p><strong>Windows (PowerShell)</strong></p>
<div class="code-block">
<div class="code-header">
<span>Windows PowerShell</span
><button
class="copy-button"
type="button"
data-copy="node-windows"
aria-label="Copy Node.js installation command for Windows"
>
Copy
</button>
</div>
<pre><code id="node-windows">winget install OpenJS.NodeJS.LTS</code></pre>
</div>
<p>
Type Y if asked to accept the terms. Close PowerShell, open it
again, then run <code>node -v</code>.
</p>
<p>
If PowerShell later says
<em>"running scripts is disabled on this system"</em> when you run
<code>npx</code>, run this once and try again:
</p>
<div class="code-block">
<div class="code-header">
<span>Windows PowerShell</span
><button
class="copy-button"
type="button"
data-copy="node-windows-policy"
aria-label="Copy PowerShell execution policy command"
>
Copy
</button>
</div>
<pre><code id="node-windows-policy">Set-ExecutionPolicy -Scope CurrentUser RemoteSigned</code></pre>
</div>
<p><strong>macOS</strong></p>
<p>If you use Homebrew:</p>
<div class="code-block">
<div class="code-header">
<span>macOS · Homebrew</span
><button
class="copy-button"
type="button"
data-copy="node-macos-brew"
aria-label="Copy Homebrew Node.js installation command"
>
Copy
</button>
</div>
<pre><code id="node-macos-brew">brew install node</code></pre>
</div>
<p>Otherwise:</p>
<div class="code-block">
<div class="code-header">
<span>macOS</span
><button
class="copy-button"
type="button"
data-copy="node-macos-fnm"
aria-label="Copy fnm installation command for macOS"
>
Copy
</button>
</div>
<pre><code id="node-macos-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
</div>
<p>Close Terminal, open it again, then:</p>
<div class="code-block">
<div class="code-header">
<span>macOS</span
><button
class="copy-button"
type="button"
data-copy="node-macos-install"
aria-label="Copy Node.js 24 commands for macOS"
>
Copy
</button>
</div>
<pre><code id="node-macos-install">fnm install 24
fnm default 24</code></pre>
</div>
<p><strong>Linux</strong></p>
<div class="code-block">
<div class="code-header">
<span>Linux</span
><button
class="copy-button"
type="button"
data-copy="node-linux-fnm"
aria-label="Copy fnm installation command for Linux"
>
Copy
</button>
</div>
<pre><code id="node-linux-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
</div>
<p>
The installer needs <code>curl</code> and <code>unzip</code> (on
Debian/Ubuntu: <code>sudo apt install curl unzip</code>). Close the
terminal, open it again, then:
</p>
<div class="code-block">
<div class="code-header">
<span>Linux</span
><button
class="copy-button"
type="button"
data-copy="node-linux-install"
aria-label="Copy Node.js 24 commands for Linux"
>
Copy
</button>
</div>
<pre><code id="node-linux-install">fnm install 24
fnm default 24</code></pre>
</div>
<p>
Your distribution's own <code>nodejs</code> package is often older
than 22.19, so check <code>node -v</code> if you use it instead.
</p>
<p>
Prefer a regular installer? Download the LTS version from
<a href="https://nodejs.org/en/download">nodejs.org</a>.
</p>
</details>
</div> </div>
<div class="article-layout"> <div class="article-layout">
<nav class="contents" aria-label="On this page"> <nav class="contents" aria-label="On this page">
@@ -164,6 +317,44 @@
Build, and other supported MCP clients. Grok Bot's connector is Build, and other supported MCP clients. Grok Bot's connector is
added from its chat. added from its chat.
</p> </p>
<details class="evidence-details">
<summary>New to coding agents?</summary>
<p>
A coding agent is an AI assistant that works on your computer
instead of in a browser tab. It can read files and run commands,
and it asks before making changes. REA gives it
reverse-engineering tools.
</p>
<p>
If you don't have one yet, pick one, follow its install page and
sign in:
</p>
<ul>
<li>
<strong
><a href="https://code.claude.com/docs/en/setup"
>Claude Code</a
></strong
>
by Anthropic. Needs a paid Claude plan.
</li>
<li>
<strong
><a href="https://github.com/openai/codex">Codex</a></strong
>
by OpenAI. Sign in with your ChatGPT account.
</li>
<li>
<strong
><a href="https://opencode.ai/docs">OpenCode</a></strong
>, open source. Connect the model provider of your choice.
</li>
</ul>
<p>
Then open it and copy the prompt below into it. The agent runs
REA's setup and shows you the plan before changing anything.
</p>
</details>
<div class="agent-terminal"> <div class="agent-terminal">
<div class="terminal-header"> <div class="terminal-header">
<span class="terminal-title" <span class="terminal-title"
@@ -191,6 +382,28 @@
</div> </div>
<details class="evidence-details"> <details class="evidence-details">
<summary>Or run setup yourself</summary> <summary>Or run setup yourself</summary>
<details class="evidence-details">
<summary>How do I open a terminal?</summary>
<ul>
<li>
<strong>Windows:</strong> open the Start menu, type
<em>PowerShell</em> and press Enter.
</li>
<li>
<strong>macOS:</strong> press <kbd>⌘</kbd> <kbd>Space</kbd>,
type <em>Terminal</em> and press Enter.
</li>
<li>
<strong>Linux:</strong> press <kbd>Ctrl</kbd> <kbd>Alt</kbd>
<kbd>T</kbd> on most desktops.
</li>
</ul>
<p>
Paste a command with <kbd>Ctrl</kbd> <kbd>V</kbd> (macOS:
<kbd>⌘</kbd> <kbd>V</kbd>; Linux terminals: <kbd>Ctrl</kbd>
<kbd>Shift</kbd> <kbd>V</kbd>), then press Enter.
</p>
</details>
<div class="code-block"> <div class="code-block">
<div class="code-header"> <div class="code-header">
<span>Terminal</span <span>Terminal</span
@@ -211,10 +424,24 @@
connection and the matching investigation skill, keeping a backup connection and the matching investigation skill, keeping a backup
of existing configuration. It asks before applying the changes. of existing configuration. It asks before applying the changes.
</p> </p>
<p>Restart your client when setup finishes.</p>
<details class="evidence-details">
<summary>How do I restart my client?</summary>
<p>
<strong>Desktop apps</strong> (Claude Desktop, Cursor, Windsurf,
VS Code): quit the app completely, then open it again. Closing
the window isn't enough. On macOS press <kbd>⌘</kbd>
<kbd>Q</kbd>; on Windows, also quit from the tray icon if there
is one.
</p>
<p>
<strong>Terminal agents</strong> (Claude Code, Codex, Gemini
CLI): exit the session and start it again.
</p>
</details>
<p> <p>
Restart your client when setup finishes. Give your agent a target Give your agent a target path and a specific question. For a
path and a specific question. For a JavaScript or Electron app, JavaScript or Electron app, you might start with:
you might start with:
</p> </p>
<blockquote class="prompt"> <blockquote class="prompt">
<p> <p>
@@ -224,6 +451,27 @@
</p> </p>
</blockquote> </blockquote>
<p>Replace the path with your local target.</p> <p>Replace the path with your local target.</p>
<details class="evidence-details">
<summary>How do I get a folder's full path?</summary>
<ul>
<li>
<strong>Windows:</strong> in File Explorer, right-click the
folder and choose <em>Copy as path</em>. On Windows 10, hold
Shift while right-clicking.
</li>
<li>
<strong>macOS:</strong> in Finder, right-click the folder,
hold Option and choose <em>Copy "…" as Pathname</em>.
</li>
<li>
<strong>Any system:</strong> drag the folder into a terminal
window and its path appears.
</li>
</ul>
<p>
In a terminal, put quotes around a path that contains spaces.
</p>
</details>
</section> </section>
<section class="article-section" id="terminal"> <section class="article-section" id="terminal">
@@ -252,6 +500,37 @@
<code>.asar</code> file. On Windows, a path such as <code>.asar</code> file. On Windows, a path such as
<code>"D:/apps/example"</code> works here. <code>"D:/apps/example"</code> works here.
</p> </p>
<details class="evidence-details">
<summary>Where do I find an app's .asar file?</summary>
<p>
Electron apps usually keep it in a
<code>resources</code> folder:
</p>
<ul>
<li>
<strong>Windows:</strong>
<code
>C:\Users\&lt;you&gt;\AppData\Local\Programs\&lt;App&gt;\resources\app.asar</code
>
</li>
<li>
<strong>macOS:</strong>
<code
>/Applications/&lt;App&gt;.app/Contents/Resources/app.asar</code
>
(right-click the app and choose
<em>Show Package Contents</em>)
</li>
<li>
<strong>Linux:</strong> often
<code>/opt/&lt;App&gt;/resources/app.asar</code>
</li>
</ul>
<p>
Some apps install elsewhere. Search the app's folder for
<code>app.asar</code>.
</p>
</details>
<p> <p>
The result identifies the app, shows connections between modules The result identifies the app, shows connections between modules
and gives their source locations. Connections REA cannot resolve and gives their source locations. Connections REA cannot resolve
+8 -8
View File
@@ -143,6 +143,14 @@
</p> </p>
</div> </div>
<div class="guide-index"> <div class="guide-index">
<section class="guide-entry">
<p class="eyebrow">First investigation</p>
<div>
<h2>Set up REA</h2>
<p>Connect your coding agent or use REA from the terminal.</p>
<a class="text-link" href="../get-started/">Get started with REA</a>
</div>
</section>
<section class="guide-entry"> <section class="guide-entry">
<p class="eyebrow">Native binaries</p> <p class="eyebrow">Native binaries</p>
<div> <div>
@@ -199,14 +207,6 @@
<a class="text-link" href="browser/">Browser observation guide</a> <a class="text-link" href="browser/">Browser observation guide</a>
</div> </div>
</section> </section>
<section class="guide-entry">
<p class="eyebrow">First investigation</p>
<div>
<h2>Set up REA</h2>
<p>Connect your coding agent or use REA from the terminal.</p>
<a class="text-link" href="../get-started/">Get started with REA</a>
</div>
</section>
</div> </div>
</main> </main>
<!-- website:footer:start --> <!-- website:footer:start -->
+80 -2
View File
@@ -168,6 +168,33 @@
returns its instructions and callers, which give the agent a way returns its instructions and callers, which give the agent a way
to investigate the missing arithmetic. to investigate the missing arithmetic.
</p> </p>
<details class="evidence-details">
<summary>New to these terms?</summary>
<ul>
<li>
<strong>Instructions:</strong> the steps the processor runs,
shown in readable form such as <code>MOV</code> or
<code>ADD</code>.
</li>
<li>
<strong>Decompiler result / pseudocode:</strong> an analysis
tool's attempt to turn instructions back into C-like code. It
can miss parts, as in this example.
</li>
<li>
<strong>Caller:</strong> a function that calls this one. It
shows which values are passed in.
</li>
<li>
<strong>Address:</strong> a location in the program, written
in hexadecimal, such as <code>0x00406400</code>.
</li>
<li>
<strong>Constants:</strong> fixed numbers stored in the
program, read as raw bytes and decoded.
</li>
</ul>
</details>
<figure class="workflow-figure"> <figure class="workflow-figure">
<p class="figure-title"> <p class="figure-title">
Three pieces of evidence, one calculation Three pieces of evidence, one calculation
@@ -211,9 +238,40 @@
<h2>Ask about the behavior</h2> <h2>Ask about the behavior</h2>
<p> <p>
After After
<a href="../../get-started/#agent">connecting REA to your agent</a <a href="../../get-started/#agent"
>connecting REA to your agent</a
>
and
<a href="../../get-started/#native"
>setting up a native analysis tool</a
>, provide the local executable and a question: >, provide the local executable and a question:
</p> </p>
<details class="evidence-details">
<summary>Which analysis tool should I use?</summary>
<p>REA needs one of these to read native programs:</p>
<ul>
<li>
<strong>Hopper</strong> (macOS or Linux): REA's setup detects
it or offers to install it. On macOS, open Hopper once and
choose demo mode or activate a license before your first
analysis.
</li>
<li>
<strong>Ghidra</strong> (macOS or Linux; experimental on
Windows): free, but needs an existing Ghidra 12.1.x
installation and a 64-bit JDK 21 or newer.
</li>
<li>
<strong>IDA</strong> (Windows): for an existing IDA setup with
an MCP registration.
</li>
</ul>
<p>
See
<a href="../../get-started/#native">Set up native analysis</a>
for the steps.
</p>
</details>
<div class="agent-terminal"> <div class="agent-terminal">
<div class="terminal-header"> <div class="terminal-header">
<span class="terminal-title" <span class="terminal-title"
@@ -239,8 +297,28 @@
</div> </div>
<p class="caption"> <p class="caption">
Example prompt. Give your agent the path to your local DX-Ball Example prompt. Give your agent the path to your local DX-Ball
1.07 executable. 1.07 executable. REA doesn't include DX-Ball, so use your own
copy, or ask the same kind of question about another small Windows
program you have.
</p> </p>
<details class="evidence-details">
<summary>How do I get a file's full path?</summary>
<ul>
<li>
<strong>Windows:</strong> in File Explorer, right-click the
file and choose <em>Copy as path</em>. On Windows 10, hold
Shift while right-clicking.
</li>
<li>
<strong>macOS:</strong> in Finder, right-click the file, hold
Option and choose <em>Copy "…" as Pathname</em>.
</li>
<li>
<strong>Any system:</strong> drag the file into a terminal
window and its path appears.
</li>
</ul>
</details>
</section> </section>
<section class="article-section" id="follow"> <section class="article-section" id="follow">
<h2>Read the REA results</h2> <h2>Read the REA results</h2>